Harwell broadcast for the Dodgers through 1949, the New York Giants from 1950 to 1953, and the Baltimore Orioles from 1954 to 1959. Harwell was the lead broadcaster on Orioles broadcasts on WCBM-AM and WMAR-TV in 1955, working alongside Chuck Thompson. The next year Thompson moved on to Washington D.C. to broadcast the Senator games until returning to the Orioles in 1962. Early in Harwell's career, he also broadcast The Masters golf tournament, as well as pro and college football.

Harwell joined the Detroit Tigers' broadcast crew in 1960, replacing Van Patrick after the latter was dismissed owing to a sponsorship change. George Kell, who had begun calling Tigers games with Patrick the year before, and had previously played for the Orioles in 1956–57 while Harwell was announcing their games, was instrumental in bringing Harwell to Detroit. "George called and said, 'I recommended you and the Tigers asked me to get in touch with you.'" Harwell said. "I came and that was it."

Harwell shared TV and radio duties with Kell through 1963, then with Bob Scheffing in 1964. He began working radio exclusively in 1965, teaming with Gene Osborn for two seasons and then with Ray Lane from 1967 to 1972. Paul Carey replaced Lane in 1973, joining Harwell to form the Tigers' best-known and longest-lasting radio team, which lasted until the end of the 1991 season. Harwell gained a large following outside of Detroit because the Tigers aired their games on WJR, a 50,000-watt station that reached most of the eastern half of North America at night.

On December 19, 1990, the Tigers and WJR announced that the station wanted to go in a "new direction" and that the 1991 season would be Harwell's last, as his contract was "non-renewed". Carey then announced that he had already planned to retire after the 1991 season, and that the decision was unrelated to Harwell's contract situation. Fans across Michigan and throughout the baseball world were outraged, but the ballclub and the radio station (who eventually wound up blaming each other for the decision) stood firm: "[Harwell's situation is] not going to change no matter how much clamor is made over it," said team president Bo Schembechler. The situation caused outrage so much that some made threats of violence against Schembechler. Some, such as Mitch Albom, blamed the situation causing as much negative feeling as it did on WJR executive Jim Long who was the one who pushed the quick, no severance pay removal of Harwell. The movement in favor of keeping Harwell was so strong that even billboards in favor of his remaining were put up. Rick Rizzs was hired away from the Seattle Mariners to replace Harwell in 1992, teaming with Bob Rathbun.

🎙️