Herman entered Baylor as a day student on September 8, 1926. In referring to his football prowess, the January 31, 1927 edition of the Campus List described him as "the fourteen-year-old wonder." He was also a standout in track and field. During his last year at Baylor, Herman was plagued by injuries that limited his athletic playing time.
He was admitted to the University of Tennessee at age 16, where he was named a football All-American lineman. He is regarded as one of the greatest players in the college's history.
Herman had a brief professional football career with the Brooklyn Dodgers before entering the coaching profession at West Point and at Yale University (1948-51).