The 34-year-old will move into management for the first time in his career at Hillsborough, having previously held assistant roles.
Rohl has worked at Bayern Munich, RB Leipzig and the German national team in his home country, while he also has experience of English football from a spell at Southampton under Ralph Hasenhuttl.
The German will take over the Owls with immediate effect, after they sacked Xisco Munoz earlier this month following a poor start to the season.
Sheffield Wednesday sit rock-bottom of the Championship table having failed to win any of their opening 11 matches of the campaign - one of only four teams in the top four tiers of English football to still be winless.
The Yorkshire outfit have picked up just three points from the 33 on offer so far, leaving them already seven points adrift of safety.
There have been major issues off the pitch too, with fans regularly protesting against the club's owner Dejphon Chansiri, who responded by saying that he would no longer invest money into the club due to the treatment towards him and his family by Wednesday supporters.
The club have not confirmed the length of Rohl's contract at Hillsborough, but at 34 he becomes the youngest coach in the EFL with this appointment.