He's got a point though. Before a ball was even kicked in England's Euro 2020 campaign #SouthgateOut was trending on Twitter.
His crime? The decision to play a slightly unexpected starting XI for their opener against Croatia baffled many fans with some even coming to the extreme conclusion that he had to go.
Kieran Trippier at left-back and Tyrone Mings in the middle were surprises but arguably the thing that enraged the anti-Southgate mob the most was the fact he started two defensive midfielders in Declan Rice and Kalvin Phillips together.
Not that it mattered to some, but England won the match 1-0 and Phillips was the standout player at Wembley.
Rice and Phillips became a formidable partnership in the middle of the park, starting every match together as the Three Lions made it all the way to the final - the best performance by England at a major tournament in a generation.
Injury meant there was no Phillips for England's October internationals. They swept aside Andorra at the weekend and with qualification for Qatar 2022 all-but in the bag, a home clash against Hungary was a good time to test all the armchair experts' theory.
Rice was the only recognised defensive midfielder in England's starting XI, with the attacking-minded Phil Foden and Mason Mount playing in deeper roles than usual.
However, the flat display at Wembley, which finished 1-1, underlined how important it is for Southgate to play a double pivot.
After the match, Rice told talkSPORT: "When you've got a player like Kalv and what he brings to the team you're always going to miss him.
"This was the first time me and Mason played together. Of course it was never going to be perfect, this is our first time. It needs work but we still did some really good things.
"But from a defensive point of view, maybe we could've been a bit tighter and not got sucked out of our positions as we did in the first half."
England were caught on the counter by their fairly average opponents a number of times but Southgate pointed out how the Phillips-shaped hole also saw England suffer in an attacking sense.
Southgate told talkSPORT: "We wanted to look at a different balance in the midfield… we were looking to control things with higher levels of possession and more of an attacking threat.
"But ironically we looked less of an attacking threat.
"That was down to our poor quality of play at times, decision-making with the ball but also Hungary proving to be a really good opponent who defended well, had a good shape and caused us some problems on the counter-attack."
It only took a sloppy result to really show why playing Rice and Phillips is key but this will hopefully go some way silence some of the Southgate doubters.