Alley's interest in jazz began at a young age, after his parents took him to a Jelly Roll Morton concert. He began playing in clubs in San Francisco in the late 1930s, and moved to New York in 1940, performing first with Lionel Hampton's band, and later with the Count Basie Orchestra.
Returning to San Francisco, Alley worked as musical director for the popular Blackhawk jazz club, where he often played with Parker, the legendary saxophonist who helped to create bebop. Alley also served on the city's arts commission and human rights commission.
In 2001, after Alley's health began to decline, the San Francisco Jazz Festival staged a tribute concert that drew more than 1,000 fans and musicians, and included performances of his signature tune, "Big Fat Butterfly."