Brown appeared with such major jazz figures as Dizzy Gillespie, John Coltrane, and Cannonball Adderly; he collaborated with drummer Max Roach on the groundbreaking album We Insist! Freedom Now Suite. He was also known for writing lyrics for such instrumental pieces as Bobby Timmons' "Dat Dere," Mongo Santamaria's "Watermelon Man," and Miles Davis's "All Blues."
In addition, he hosted the television show Jazz Scene U.S.A in the early 1960s.
Brown was also a political activist, poet, and actor. He wrote a series of musicals, some of them performed by troubled Chicago teenagers.