Kapp was born on October 9, 1936, in Chicago into a musical family. Despite prodigious talent on the piano in his youth, Kapp studied German history at Johns Hopkins University. After his 1957 graduation, he enrolled at the Stuttgart Staatliche Hochschule für Musik, where he studied piano, conducting, and composition. From 1960-1962 he worked as repetiteur at the Basel Stadttheater (Switzerland), then served as music director at the Manhattan School of Music's Opera Theater (1963-1965).
Though he would waver again, earning a law degree from New York University, he eventually chose music as his primary career when in 1968 he founded the Philharmonia Virtuosi. Its membership was mostly made up of players of exceptional talent, and soon Kapp was able to attract soloists of international renown. The ensemble played many of its concerts at Town Hall, the Metropolitan Museum of Art, and other major sites in the New York area.
After his 1977 Greatest Hits album, Kapp had become a celebrity and subsequent recordings also drew attention, though not quite the high sales figures. The Greatest Hits of the 1900s, issued in 1986, was the last of the greatest-hits releases by Columbia. Kapp continued to record with some success and led the group in regular concerts until he was diagnosed with cancer in 2004. Many of Kapp's recordings are still appearing in reissues; among the most recent of them is the 2006 CD on ESS.AY, Fasch: Three Suites for Orchestra, on which Kapp leads another group he was associated with, Kiev Pro Musica., Rovi