Justin 'Jud' Conlon was born in 1910 in Cuba City, Wisconsin. He attended Columbia Academy and Columbia College in Dubuque, Iowa. As a youth he played the accordion and was active in musical groups including the Vested Choir directed by Father Alphonse Dress. In high school he formed the Justin Conlon Orchestra. By the time he was in college, the group toured the Midwest and was often featured on local radio shows. His first major arranging work was with the Kay Kyser Orchestra.

In 1945, the Kay Kyser band was still at the height of their popularity. The vocal group The Campus Kids, which Conlon was a part of, toured and recorded with Kyser. His first recordings with them were 'Choo Choo Polka' and 'That's for Me.' From this experience Conlon intended to form his own vocal group that would feature his own arrangements and musical direction.

The Campus Kids featured two singers with a four octave range and perfect pitch, Loulie Jean Norman and Gloria Wood. With them, himself, and Charlie Parlato as first tenor, he had the makings of the new group he had in mind. He assigned himself the baritone part, Norman was first or lead soprano, and Wood second soprano. He also recruited Mack McLean from the Six Hits and a Miss singing group as a second tenor.

Toward the end of Bing Crosby's first Philco season in March, 1947 the featured vocal group The Charioteers left the show. This void allowed Conlon to step in with his newly formed group shortly thereafter named The Rhythmaires. Conlon's group began an association with Crosby that would last for 10 years and 230 broadcasts and over 40 recordings. Some of their most notable recordings were 'Home Cookin,' 'Bibbidi-Bobbidi-Boo,' Teddy Bears' Picnic,' 'It's Beginning to Look Like Christmas,' 'Zing a Little Zong,' and Road to Bali.' After the Philco Radio Time show stopped broadcasting in 1949, The Rhythmaires continued with Crosby throughout his Chesterfield Radio series that ran until 1952.

Sometimes the Rhythmaires would be featured as The Jud Conlon Singers or The Jud Conlon Choir.  The final Rhythmaires recordings with Bing Crosby were made in 1956. In early 1958, they appeared on the album 'Jazz Goes Latin' with tenor saxophone player Georgie Auld for Capitol Records.

Conlon went on to work for Andy Williams and Guy Lombardo. In the mid-1950s, Conlon began collaborating with satirist, Stan Freberg. His arrangements appeared on nearly all Freberg's comedy records, as well as on his Stan Freberg Radio Show. He also worked on the television program Frankie Laine Time. Conlon also served as a musical arranger for Walt Disney. Among his credits are the Disney movies Alice in Wonderland (1951), Peter Pan (1953), and Babes in Toyland (1961).

In 1966 while in Chicago attending the convention of American Federation of TV and Radio Artists as a delegate, Conlon was found dead from a heart attack in his hotel room.