McKibbon grew up in Detroit and played there before moving to New York in the 1940s. In 1947, he joined Gillespie's big band, one of the first groups to blend Latin music and jazz. Also in the late 1940s, he played in Davis's "Birth of the Cool" ensemble, helping to launch the lush West Coast sound.
McKibbon played in Shearing's quintet from 1951 to 1957, and in percussionist Cal Tjader's group in 1958, before he moved to California to do studio work. In the 1970s, he recorded with the all-star Giants of Jazz and with saxophonist Benny Carter. He appeared on Thelonious Monk's last recording in 1971.