Born in South Carolina, Thompson came to New York in the early 1940s. Between 1943 and 1945 he played in the swing bands of Lionel Hampton, Don Redman, Billy Eckstein, Lucky Millinder, and Count Basie, replacing Don Byas in Basie's group.
In 1945, he moved to Los Angeles, where he played in Dizzy Gillespie's bebop group, reportedly hired in case the notoriously unreliable Charlie Parker failed to show up for gigs, and in the Stars of Swing with Charles Mingus and Buddy Collette.
Thompson returned to New York in the late 1940s. He led a group at the Savoy for three years, and recorded with Miles Davis (appearing on the trumpeter's Walkin') and Stan Kenton.
In the late 1950s and '60s, Thompson spent several years living in France. In the mid-1970s, after a period teaching at Dartmouth College, he retired entirely from the music world.