The West Virginia Legislative Auditor's report found that Cisco sales representatives and engineers who recommended that the state buy the oversized Internet routers showed "wanton indifference to the interests of the public." The audit called on the state Purchasing Division to investigate whether the Cisco sales representatives and engineers should be barred from doing business with state government.

The auditor's report singled out state officials, including Homeland Security Director Jimmy Gianato, for failing to survey public facilities across the state to determine if they needed new routers.

The auditors concluded that the state could have bought smaller, less-expensive routers for hundreds of schools, libraries and State Police detachments.

"The decision to spend the federal funds on oversized routers resulted in millions of dollars in federal funds not being spent on expanding the state's fiber-optic broadband network," auditors concluded.

By buying smaller routers, the auditors said the state could have built more than 100 miles of high-speed fiber that provides faster Internet connections.

"It's disappointing, as keepers of the taxpayer funds, a purchase of such expense was done without proper reasoning and proper authorization," said Sen. Clark Barnes, R-Randolph. "We need to make sure something like that does not happen again."

Delegate Gary Howell, R-Mineral, said state officials should have brought fiber to public facilities first, then asked what type of routers they wanted.

"It's like the state bought Ferraris and all we built was a dirt road," Howell said. "We did it like the Soviet Union used to do it: top down, instead of bottom up." 

Rob Alsop, Gov. Earl Ray Tomblin's chief of staff, defended the $24 million router purchase, saying Cisco also gave the state a discount for buying routers of the same size -- a savings greater than the state would have received by purchasing devices of different sizes.

Alsop said state officials wanted to buy routers for public facilities' "future needs." He said state officials at the time -- Alsop didn't work in state government when the routers were purchased  -- made a "judgment call."

"They weren't looking at current needs," Alsop said. "They were trying to foster broadband development and capacity five, eight, 10 years from now."  

State lawmakers didn't seem to buy Alsop's explanation.

"Both sides were presented, and the prevailing side clearly showed there was a waste of taxpayer money," said Larry Faircloth, R-Berkeley.

Auditors found that the state spent an additional $6.6 million for router "add-on" features that state agencies never requested and weren't always necessary.

"In our view, not everyone needed all of these features," Legislative Auditor Aaron Allred told two House-Senate oversight committees that received the report Sunday. "The state could have saved millions of dollars if they had been selective."

The audit spotlights specific cases of waste: The state installed a $22,600 router at the Marmet Public Library, which has a single Internet connection. The router cost more than the trailer that houses the Marmet Library, according to Kanawha County Commission staff.

"Marmet may have needed a better library, but they didn't need a $22,600 router," Allred told state lawmakers.

Meanwhile, Clay County received seven oversized routers -- all installed at facilities within a half-mile radius in the town of Clay. In Pendleton County, the city of Franklin received six routers for sites located within three-quarters of a mile of each other.

"I believe every West Virginian deserves the same broadband access," Allred said. "But having the same access doesn't mean you buy the same number of buses for the Pendleton County school system that you buy for the Kanawha County school system."

Routers purchased using Internet telephone contract

In 2010, the state of West Virginia received a $126.3 million federal stimulus grant to expand high-speed broadband. The money was designed to bring fiber-optic cable to 1,164 "community anchor institutions" -- schools, libraries, health-care clinics, county courthouses, State Police detachments, state agencies, 911 centers, planning offices and other public facilities.

The state used $24 million from the grant to buy routers for each site. The routers, which funnel data from one computer network to another, cost $22,600 each.

Auditors also determined that state officials purchased the oversized routers using a 2007 contract for Internet telephone service. The contract makes no mention of routers.

Those same officials, the audit found, circumvented state purchasing laws and used a "secondary bid" process that wasn't allowed at the time.

The 2007 contract specified "Cisco or equal" equipment. But someone changed the wording to "Cisco only" when the state Office of Technology solicited bids on its online "bulletin board," the audit says. The change shut out Cisco competitors, such as Alcatel-Lucent and Hewlett Packard.

The state Purchasing Division was never notified about the $24 million router purchase.

The state Office of Technology, which oversees state agency technology purchases, also was shut out. The audit says former state Chief Technology Officer Kyle Shafer tried to halt the $24 million router buy, but his objections came too late.

"The Office of Technology did not participate in any of the meetings that led up to this decision," Shafer told auditors. "The first I saw of this purchase order was when it was sent to [us] for approval."

State officials, Cisco give conflicting stories on reasons for large routers

West Virginia Chief Information Officer Gale Given justified the $24 million router purchase, saying the government facilities could save money on phone bills by using the devices for telephone service, according to the audit.

But auditors noted that Cisco recommends its 3945 series routers for sites with 700 to 1,200 phones. None of the public facilities that received routers have 700 phones - or anywhere close to that number.

For instance, 91 of the 172 libraries that received routers have three phones or fewer. The Cabell County Library -- the largest library that got a router -- has 15 phone lines.

What's more, the libraries don't plan to switch their telephone service from Frontier Communications' standard service to "voice-over Internet service," which is sold by companies such as Vonage and Suddenlink.

The state could have saved $2.8 million by buying smaller, less expensive routers for libraries, the audit found.

According to the report, Cisco sales engineer Mark Williamson told auditors that the company recommended the state purchase larger routers because state Department of Education officials requested the devices have a dual-power supply so the routers would continue to function during a power outage.

Williamson said Gianato and state technology office administrator John Dunlap agreed with the dual-power requirement.

But Department of Education officials steadfastly denied Williamson's assertion, and the Cisco sales engineer couldn't find any documents or emails to corroborate his story, according to the audit.

State education officials said they didn't request routers with dual-power supplies because most schools don't have emergency generators.

"The Department of Education did not request or require that the routers for the state's schools have internal dual-power supplies," education officials wrote to the Legislative Auditor. "Education would not have made this requirement because unless a school has two power sources the feature of dual power supplies would have no use."

The oversized routers were purchased anyway in July 2010 -- mostly because no one bothered to ask what size devices the public facilities needed, according to the audit.

"While Mr. Williamson of Cisco and Mr. Gianato stated a need for the routers to have a dual-power supply," the audit said, "the ultimate cause of the state purchasing inappropriately sized routers is that neither a capacity study nor a user need study was conducted."

Router too large for schools and State Police offices

Auditors determined that the state should have purchased smaller routers for 368 schools with fewer than 500 students -- a $3.68 million savings.

At the same time, larger high schools that could have used the Cisco 3945 routers didn't get them. Auditors found that 36 out of 57 schools with more than 750 students didn't receive routers.

Riverside High School, which has 1,200 students, wasn't assigned a router, but the nearby Marmet Public Library was, the audit noted.

Auditors determined that the routers were too large for more than 70 State Police detachments. The state could have saved $1.4 million by buying smaller routers, according to the audit.

The detachments were already using smaller routers purchased four years ago, but the agency must remove the devices to make way for the oversized stimulus-funded routers.

"The West Virginia State Police was never contacted by the grant implementation team concerning broadband and telephone needs," the audit says. "State Police did not ask for the Cisco 3945 routers. Instead, State Police were simply informed that [the agency] was receiving 77 routers."

Auditors also discovered that State Police received the new Cisco 3945 series routers more than two years ago -- all but two of the devices remained boxed up in storage.

State Police can't use the routers because they lack a component that would make the devices compatible with the agency's voicemail system.

The necessary components cost $85,000, and State Police have requested bids from companies that sell the Cisco voicemail modules.

The audit recommends, however, that the state technology office contact Cisco to see whether the company would provide the voicemail modules at no cost and remove other "ad-on" features from the routers that State Police don't want.

The auditor recommended that the state consult with the West Virginia Broadband Deployment Council to determine whether the oversized routers could be removed from some locations that don't need them and installed at larger sites that do. The council must report back to state lawmakers in 60 days.

Perhaps the strongest statement against the $24 million router purchase came from Col. Mike Todorovich, who serves on the state's "broadband grant implementation team" with Gianato, Given and Dunlap.

Todorovich, who oversees the grant's finances, backed the audit's key findings, breaking ranks with his colleagues who have insisted for months that the state saved money and bought routers that were the appropriate size.

Todorovich told auditors: "Those making the decision on how to spend the money did not consult individuals with technical knowledge on the best methods to utilize the funds."

Reach Eric Eyre at erice...@wvgazette.com or 304-348-4869.