Kaye had a remarkably wide-ranging career that included giving the U.S. premiere of Shostakovich's Piano Concerto No. 1 in 1935, playing solo recitals, accompanying Heifetz on a tour of European war zones, playing in Toscanini's NBC Symphony Orchestra, working as a radio announcer, writing theme music for the TV quiz show Concentration, playing the organ on a 1950s children's show called The Rootie Kazootie Show and working as a musical director for three major television networks.

Shannon Bolin, 89, his wife of 60 years, told the AP, "He was the greatest, funniest, warmest man that ever lived. He said, 'If I didn't have music in my life, I would just as soon be dead.' Music was just everything to this man."

Kaye was born in Brooklyn in 1909 to a musical family and went to Carnegie Hall concerts as a child. However, his father, who played klezmer on five instruments, discouraged him from a career in music, according to the AP. To please his father, he completed a program at Columbia University's Teachers College. He majored in Latin at City College, then studied at the Institute of Musical Art, Juilliard's predecessor. He received a doctorate from Columbia, writing his thesis on Richard Rodgers's compositional techniques.

He made a commercial for De Beers diamonds with his wife, where they are filmed walking hand in hand as a young couple turns to watch them.

Kaye died of pneumonia. His wife told The New York Times that, before entering the hospital, he practiced some Bach, Beethoven and Mozart.