Smith was born in Norristown, Pennsylvania, and studied piano with his parents. After serving in the Navy, he studied bass and piano and took up the Hammond organ in 1951. In 1956, he made his New York debut at the Caf_ Bohemia; that same year, he began to record for Blue Note. His recordings for the label, made over the next seven years, would be his most influential.

The Blue Note recordings, which included The Champ, The Sermon, and Back at the Chicken Shack, were built around the organ trio format of guitar, bass, and Hammond B3 organ, with Smith playing the bass line in lieu of a string bass. His bandmates included guitarist Kenny Burrell and Art Blakey, as well as such soloists as trumpeter Donald Byrd and saxophonists Stanley Turrrentine, Lee Morgan, and Jackie McLean.

In the late 1960s and '70s, Smith recorded for Verve and other labels, but returned to Blue Note in the '80s. He toured widely, as well as performing at his own club in Los Angeles in the '70s. Later this year, he was scheduled to tour with organist Joey DeFrancesco, a friend and prot_g_. An album from the two will be released on February 15.

In November 2004, Smith was named one of the 2005 Jazz Masters by the National Endowment for the Arts; he was presented with the award, the nation's highest for jazz musicians, at a ceremony in California last month.