Kapp was born in 1936 in Chicago. He studied German and political history at Johns Hopkins University and conducting, composition, and piano at Stuttgart's Hochschule f‹r Musik.
After returning to the United States he earned a law degree from New York University and became music director of the Opera Theater of the Manhattan School of Music.
He founded the Philharmonia Virtuosi in 1968 and led the group until becoming ill in 2004. In 1977 he recorded Greatest Hits of 1720 for CBS Masterworks, subsequently recording "best of" albums from 1721, 1790, and the 20th century designed to appeal to a broad audience.
He performed with the Philharmonia at the Metropolitan Museum of Art, Town Hall, and Purchase College, among other venues.
He died of cancer, according to the Times.