Peter Herbolzheimer was born in Bucharest and migrated from communist Romania to West Germany in 1951. In 1953 he moved to the United States, where he enrolled in Highland Park (MI) High School in the graduating class of 1954. There, he was a staple in the musical choral groups and orchestra accompanied several groups on guitar who often performed for various organisations and corporate functions around the Detroit area.

He returned to Germany in 1957, took up the valve trombone, playing in numerous "jazz cellar open mic" groups. For one year he studied at the Nuremberg Conservatory. In the 1960s he played with the Nuremberg radio dance orchestra and with Bert Kampfert's orchestra. 

In 1969 Herbolzheimer formed his Rhythm Combination and Brass (RC&B) for which he wrote most of the arrangements. This big band was unique in that it had an international lineup of eight brass, but originally only one saxophone, with Herb Geller in that chair. The list of brass players included Art Farmer (USA), Dusky Goykovich (Bosnia), and Jiggs Whigham (USA). The rhythm section consisted of two keyboards, guitar, bass, drums and percussion and included renowned musicians such as Philip Catherine (Belgium), Niels-Henning Orsted Pedersen (Denmark), Bo Stief (Denmark), and Grady Tate (USA).

Throughout the 1970s and 1980s, Herbolzheimer, as the arranger and conductor, led his orchestra for virtually every major German television network while featuring and accompanied visiting American musicians such as Ella Fitzgerald, Benny Goodman, Sammy Davis Jr, Dizzy Gillespie and Al Jarreau.

Between 1987 and 2006 Herbolzheimer was the musical director of Germany's national youth jazz orchestra, the Bundes Jazz Orchester (BuJazzO). He conducted regular workshops and clinics for big band jazz.

Herbolzheimer died aged 74 in his hometown of Cologne, Germany on 27 March 2010.