State Police routers unusable
CHARLESTON, W.Va. -- More than two years after state officials used $1.7 million in federal stimulus funds to purchase 77 high-priced Internet routers for the West Virginia State Police, all but one of the devices remain unused because they don't have proper components.
Some routers remain boxed up in storage at State Police detachments across the state. Others have been installed on racks, but haven't been turned on. The State Police detachment in Logan has the only router that's up and running.
In reports provided to the federal government, state officials list the 77 routers assigned to State Police as "deployed" and "100 percent complete."
State officials also have touted the equipment -- part of a $126.3 million high-speed Internet expansion project -- as necessary to "enhance the public safety of first responders and [West Virginia] citizens."
The new routers -- bought with stimulus funds for $22,600 each -- lack voicemail modules, and State Police can't operate the agency's voicemail system without the hardware, according to emails obtained by the Gazette under the state Freedom of Information Act.
"As provided to us, the routers did not have the module in them," said Bill Gallagher, information technology director at the State Police. "It's just a piece of hardware that's not there."
The new routers came with a five-year service warranty, so the State Police already have lost two years of free maintenance on the equipment.
State Police also must purchase new licenses for their Cisco phones and voicemail, if the agency ever starts using the new stimulus-funded routers.
"There apparently is an issue with how Cisco changed the licensing scheme," said Lt. G.E. McCabe, statewide interoperability coordinator at the State Police. "Our voicemail system requires licensing."
Needed components could cost $270K
The state shipped the 77 Internet routers to State Police headquarters in South Charleston in October 2010. The agency distributed the routers to detachments and other trooper facilities across the state, Gallagher said.
About a year ago, State Police notified the state Office of Technology that the routers lacked the voicemail module and wouldn't be installed and turned on, according to a Department of Administration spokeswoman.
Since then, State Police have repeatedly asked administrators overseeing the $126.3 million federal stimulus grant for advice on resolving the router problem.
Broadband project officials promised to "push" Cisco into supplying the voicemail modules and licenses -- presumably at no cost -- or pry loose extra funds from the state's stimulus grant, according to an email Gallagher sent to the Office of Technology on June 5.
"We have been waiting for the answer as to how this is to be handled and haven't heard anything, and therefore we are not pushing [the routers] into service," Gallagher wrote.
Gallagher, his superiors at the State Police, and state technology office executives would not say how much it would cost to buy the hardware and "Cisco Call Manager Express" phone licenses needed to run the agency's voicemail system on the new routers.
"The Office of Technology is presently assessing possible resolutions to this compatibility issue," said Diane Holley-Brown, a spokeswoman for the agency. "It is premature to associate any potential costs until the final determination is made."
The State Police's voicemail system now operates, without problems, using less-powerful and lower-priced Cisco routers. The smaller routers have voicemail modules, but the hardware isn't compatible with the new stimulus-funded routers. So the modules cannot be taken out of the old routers and inserted into the new ones.
After the Gazette started asking questions about the unused routers earlier this month, Gallagher sent an email to technology office administrator John Dunlap, listing the parts and licenses needed to get the new routers up and running.
The voicemail modules cost about $3,000 each -- or about $270,000 for 77. The new licenses cost about $100 per phone. State Police have about 1,700 phones, so new voicemail licenses could run as much as $170,000.
State doesn't track routers in use
During the past two years, the state has distributed 756 stimulus-funded routers to public facilities across West Virginia, but the state's broadband project team doesn't keep track of how many of the devices have been installed and turned on.
In July 2010, state officials used federal stimulus money to buy 1,064 Cisco 3945 Series routers at a cost of $24 million. Seventy-seven of the pricey, high-powered devices were assigned to State Police facilities. The routers also were shipped to schools, libraries, jails, health-care clinics, county courthouses, 911 centers, regional planning offices and state agencies.
The new routers alone cost $7,800 each, but "add-ons" - additional equipment that came with the devices - boosted the price tag by $14,800.
Voicemail modules weren't included in the $24 million router purchase.
State officials said they weren't aware of any other public agencies that couldn't use the new routers because they lacked voicemail components and Cisco phone licenses. The voicemail modules snap into the back of the routers.
The Logan detachment installed and turned on its router, after purchasing a voicemail module and updated license with State Police funds. Gallagher declined to say how much the agency spent on the new hardware and license.
State Police also have a working stimulus-funded router at Winfield, but the detachment is using the device only for "testing," said Gallagher, who declined to elaborate.
'We simply cannot use the routers as is'
Last June, Gallagher notified the state Office of Technology that there was only one way to use the new routers without paying for the voice mail modules and licenses: State Police would have to plug the new routers into the agency's existing older ones. But Gallagher rejected that idea, saying it would increase the odds for equipment to break down. Routers funnel data, such as email and web pages, from one network to another.
"We simply can't use the routers as is unless we put them in front of our existing equipment, which in my opinion, would be creating another point of failure," he wrote in an email to the state technology office administrator Nick Patel.
Gallagher also told Patel that State Police would not buy the 77 router voicemail modules and new phone licenses.
"We are not able to fund the re-licensing nor the additional hardware it takes for voicemail," Gallagher wrote.
Earlier this month, the Office of Technology solicited proposals for using an estimated $9 million in stimulus funds left over from the $126.3 million federal grant. Five companies and a state agency submitted project ideas. State Police did not submit a proposal.
West Virginia must spend the entire $126.3 million by Feb. 13, or risk having to return any unspent funds to the federal government.
The U.S. Commerce Department Inspector General's office is reviewing the state's use of the federal stimulus money. The audit followed a series of reports in the Gazette that raised questions about the routers' size and cost.
Gallagher said he and Office of Technology personnel are searching for a way to fix the State Police router problem and put the devices into service. He declined to say whether the agency would request to use any of the leftover stimulus funds to purchase the voicemail hardware and licenses. Gallagher also said he doesn't have a deadline to resolve the issue.
"We're still in the troubleshooting phase, trying to figure out the right way to handle this," he said. "We're tying to find a cost-effective solution."
Reach Eric Eyre at email@example.com or 304-348-4869.