It wasn't an entirely a hardship.
"I was probably always attracted to the bass," he said. "I was always into the rhythm section and paying attention to not just the vocals but what the band was doing -- the groove --which comes pretty much from the drums and the bass."
With a lot of practice, Williams' talent with the instrument grew and grew. He said he was lucky to get to play so much, lucky to grow up in a place with a thriving jazz scene.
"There were tons of places to play," he said. "There still are. Maybe a few less than when I was growing up, but you used to be able to play in all kinds of clubs and restaurants. You could stay busy all the time and find people playing up and down the street."
On breaks between sets wherever he had a gig, Williams used to walk up the street to the next club or café to check out what other players were doing. No one seemed to mind.
"It was a small, but really supportive community," he said.
After years of playing around Washington, D.C., winning contests and various scholarships to further his training, Williams entered the Thelonius Monk Institute of Jazz International Jazz Competition in 2009.
"It's kind of like 'American Idol' for jazz musicians," Williams said. "But each year, they feature a different instrument. Say, one year, the competition is for the piano. Another year, it's for the guitar. In 2009, they did the bass."
All-star musicians judge the competition -- top-shelf jazz players who also play the instrument featured. In 2009, the panel of judges included Grammy winning bassists Christian McBride and John Patitucci, former Miles Davis collaborators Ron Carter and Dave Holland, as well as Charlie Haden and Robert Hurst.
Williams took first prize, winning $20,000 and a record deal with Concord Records.
The money didn't go far.
He laughed and said, "I live in New York. I got a new bass, but after a couple of month's rent, it was gone."
Still, it was a significant milestone for a performer clearly on his way up. So was winning his first Grammy in February as a member of jazz guitarist Pat Metheny's Unity Band.
"I got to go up on stage and thank my mom and all that," Williams laughed.
Not bad for a guy who couldn't get into the guitar class.
Reach Bill Lynch at ly...@wvgazette.com or 304-348-5195.