The first pastor of St. Joseph Church in Milwaukee was Fr. Herman Holzhauer. He was born November 14, 1830 in the village of Stetten, Sigmaringen, Hohenzollern (southwest Germany), and ordained at Milwaukee on March 11, 1854.

He obtained classical education at Ehingen near Ulm in the Baden-Württemberg area of Germany, and his philosophical and theological training at Tuebingen and Freiberg. 

In 1852 he emigrated to the United States and was ordained by Bishop John Martin Henni, a who was born in Switzerland and who was chosen as the first bishop of the newly created Diocese of Milwaukee in 1843. He celebrated his first Mass at St. Mary's Church – now referred to as "Old St. Mary's" on Broadway downtown. At that time the northwestern part of Milwaukee was settled by Germans, mostly Catholics; to serve them, Fr. Holzhauer was entrusted with the organization of a new congregation which was established February 22, 1855 at 11th and Cherry streets. The new church was dedicated in honor of St. Joseph. 

The first church building, arranged so to be used for a school and parsonage as well, was erected in 1856, during a period of great financial depression. In 1872, a new school was built and enlarged in 1891. 

In 1894, his health became poor and he died on April 6, 1895. He was originally entombed at the St. Joseph Church, but when the property was slated for demolition to make way for what is now Interstate 43, his body was moved to and buried at Calvary Cemetery on August 31, 1963.