State paid $22K each for Internet routers
CHARLESTON, W.Va. -- Nobody told Hurricane librarian Rebecca Elliot that the $22,600 Internet router in the branch library's storage closet was powerful enough to serve an entire college campus.
Nobody told Elliot how much the router cost or who paid for it. Workers just showed up and installed the device. They left behind no instructions, no user manual.
The high-end router serves four public computer terminals at the small library in Putnam County.
"I don't know much about those kinds of things," Elliot said last week, before politely leaving to help an elderly patron select books. "I just work here."
The state of West Virginia is using $24 million in federal economic stimulus money to put high-powered Internet computer routers in small libraries, elementary schools and health clinics, even though the pricey equipment is designed to serve major research universities, medical centers and large corporations, a Gazette-Mail investigation has found.
The state purchased 1,064 routers two years ago, after receiving a $126 million federal stimulus grant to expand high-speed Internet across West Virginia.
The Cisco 3945 series routers, which cost $22,600 each, are built to serve "tens of thousands" of users or device connections, according to a Cisco sales agent. The routers are designed to serve a minimum of 500 users.
Yet state broadband project officials directed the installation of the stimulus-funded Cisco routers in West Virginia schools with fewer than a dozen computers and libraries that have only a single terminal for patrons.
"The routers have a lot of power," said Karen Goff, executive secretary of the West Virginia Library Commission. "Because the routers are so big, our tech guys had to build shelves for them. The libraries had no other place to put them."
Morgantown-based WVNET, the state government Internet services agency, uses six Cisco routers with similar capacity to serve all state agencies and public universities.
West Virginia Homeland Security chief Jimmy Gianato, who's leading the state broadband project, defended the $24 million router purchase last week, saying the devices "could meet many different needs and be used for multiple applications."
"Our main concerns were to not have something that would become obsolete in a couple of years," Gianato said. "Looking at how technology evolves, we wanted something that was scalable, expandable and viable, five to 10 years out. We wanted to make sure every place had the same opportunity across the state."
In July 2010, a West Virginia Office of Technology administrator warned that the Cisco 3945 series routers "may be grossly oversized," according to an email obtained by the Gazette-Mail.
The administrator asked state officials to postpone plans to spend $24 million on the routers so he would have time to evaluate the proposed purchase.
Five days later, state officials signed the $24 million contract with Verizon Network Integration to buy the Cisco routers.
Verizon delivered an additional 100 routers to the state for free. West Virginia officials never asked for the additional equipment -- valued at more than $2.26 million.
Verizon spokesman Keith Irland said the company simply responded to router specifications detailed in the state's bid posting.
"They specified the equipment they wanted," Irland said. "That's what they requested, that's what we bid on. We had the lowest price, and we won the bid for the equipment and related maintenance."
The Gazette-Mail contacted two Cisco sales agents last week, asking whether the 3945 series routers were appropriate for schools and libraries.
"The 3945 is our router solution for campus and large enterprises, so this is overkill for your network," a Cisco representative responded.
The sales agents recommended a smaller router -- with a list price of $487.
State Department of Education officials questioned the size of the routers before Gianato and the Office of Technology executed the $24 million purchase order.
It didn't make sense to buy the same size routers for a 1,800-student high school and a 100-student elementary school, according to administrators in the Department of Education's technology division. The state is distributing 471 of the high-priced routers to schools.
"The WVDE asked if the size of the routers could vary based on the needs of a school," said Liza Cordeiro, spokeswoman for the Department of Education. "At that time, it is our understanding that, for consistency and future expansion, the plan was to buy all the same size."
Gianato said putting the same size router in every school was about "equal opportunity."
"We wanted to make sure a student in McDowell County had the same opportunities as a student in Kanawha County or anywhere else," he said. "A student in a school of 200 students should have the same opportunity as a student in a school with 2,000 students."
John Dunlap, operations director at the state Office of Technology, had similar concerns over the size of the routers.
"The Office of Technology is concerned that this equipment may be grossly oversized for several of the facilities in which it is currently slated to be installed," Dunlap wrote in a July 12, 2010, email to Gianato. "As a result, the Office of Technology would like to evaluate these and make recommendations to deploy the 3,900 series router where it may be better utilized for this project."
Last week, the Gazette-Mail asked Dunlap to explain his email. He referred questions to Gianato.
Gianato acknowledged that he didn't heed Dunlap's advice or wait for an evaluation.
"The routers already had been bid out," Gianato said. "I think John was looking at our needs now, not looking at our needs into the future."
Where's the accountability?'
In March 2010, the state received a $126 million federal stimulus grant to bring fiber-optic cable to schools, libraries, health-care facilities, State Police detachments, 911 dispatch centers, county courthouses, jails and libraries.
It was the largest broadband award given to any state.
West Virginia's broadband grant application mentions nothing about router purchases. Routers steer data, such as email and web pages, from one computer network to another.
"The grant was not an equipment grant. It was to build fiber," said Jim Martin, CEO of Citynet, a Bridgeport-based Internet provider. "These routers were not needed and could have been purchased through other funding sources. Where's the accountability?"
Gianato said federal officials have approved all equipment purchases under the grant.
"The grant included paying for everything except the recurring cost of [Internet] service," he said. "It doesn't pay the monthly bill."
A handful of West Virginia facilities -- called "community anchor institutions" under the federal broadband grant -- that initially were scheduled to receive routers, such as Charleston Area Medical Center, might have required the powerful equipment. However, state broadband project leaders later discovered that the facilities already had fiber-optic connections and suitable routers.
'T1' cards add $1M to price
The routers alone cost the state $7,800 each, but "add-ons" -- additional equipment that came with the devices -- boosted the price tag by $14,800.
"It's like buying a car," Gianato said. "You get a lot of options with the car."
An online Cisco retailer was selling new 3945 series routers for $5,800 last week. The routers have a list price of $13,000 each.
Verizon was the lower of two bidders for the $24 million router sale. Hebron, Ky.-based Pomeroy bid $24.8 million for the 1,064 Cisco routers.
State officials requested that the devices include a "T1 interface card" that would allow schools, libraries and other sites to use the high-capacity routers with their existing copper-wire T1 broadband connections -- while waiting to hook up to fiber optic cable.
The adapter cards added $1.08 million to the purchase price.
"T1 cards were used to hook the existing lines into the new routers until the fiber could be installed and the lines switched to the new ones," said Cordeiro, spokeswoman with the state Department of Education. "If this had not been done, the routers could not have been operational with the existing lines while waiting for the complete fiber runs."
West Virginia paid the extra cost because it purchased the 1,064 routers all at the same time, before running fiber cable to the public facilities.
The state also has delivered hundreds of routers to sites -- mostly schools -- that already had fiber-optic connections. So those routers' adapters were never used or needed to connect to a T1 line.
Gianato said the T1 cards have other uses -- video conferencing, wireless Internet and "voice over Internet protocol."
"I'm not an expert on the technical side," he said, "but these have all kinds of capabilities and applications."
'We're in the dark'
West Virginia Library Commission technicians are installing 176 Cisco routers at public libraries.
Library officials also have raised questions about the size of the routers.
"[Gianato] said it was important to have that capacity in case it was needed for homeland security reasons," Goff said. "In some places, the library may be the only anchor institution in the community."
That's not the case in Hurricane. The public library stands next to Hurricane Middle School -- which also has a new $22,600 router.
Putnam County Libraries Director Steve Christo, who stopped at the Hurricane branch last week, said Frontier workers recently brought fiber cable into the building. However, the library's high-end Cisco router is still connected to a T1 broadband line because the phone company hasn't extended fiber to the area.
Christo said he has no idea when the Hurricane library will get a high-speed fiber connection to serve its four public computer terminals.
"I don't know exactly where [the routers] are from. They're not the Library Commission's," Christo said, before noticing a router tag with the letters "AARA," which stands for American Recovery and Restoration Act -- the formal name for the 2009 federal stimulus package. "We're in the dark about this."
Gianato said he has no regrets about the router purchase.
"I think we made the right decision," he said. "We have positioned our state to expand and move into the next generation of technology."
Coming Monday in The Charleston Gazette: Why are hundreds of the routers still unused?
Reach Eric Eyre at email@example.com or 304-348-4869.