The Three Lions are still in with a chance of European Championships glory thanks to a dramatic comeback over the Falcons, whom they edged out 2-1 in extra time in Gelsenkirchen.
Ivan Schranz's third strike of the tournament had Slovakia on the cusp of an unexpected quarter-final appearance, only for Jude Bellingham to pop up with a last-gasp acrobatic equaliser in added time.
The Real Madrid man's bicycle kick triggered 30 minutes of additional time, where just 54 seconds in, Toney nodded on an Eberechi Eze volley for Harry Kane to head home the pivotal goal.
Switzerland now lie in wait for the Euro 2020 runners-up in Dusseldorf next Saturday, but the performance was another largely tepid one from the Three Lions, whose manager Southgate was criticised for his substitutions once more.
Southgate lifts lid on Toney anger
Toney was only brought on with a couple of minutes of normal time remaining, and Southgate revealed that the Brentford striker was seething with him for not chucking him on earlier.
"I had belief we would get a goal. The boys that came on, Ivan (Toney) played a big part - he was disgusted at me putting him on at that moment (94th minute when England trailed 1-0)," Southgate told BBC Sport.
"I'm so proud of the players and the spirit the lads showed. Everybody we put into the game played a part and helped us get over the line. They are all pushing and helping each other and tonight, that made the difference.
"I had a funny feeling the game wasn't dead and I know that sounds ridiculous. We were pushing and probing. Ultimately it is the one we have thrown in the box that got us the goal."
Kane makes bold "best" claim over Bellingham goal
Prior to Toney and Kane's devastating link-up, Bellingham spared England's blushes with a delicious overhead kick that left Martin Dubravka rooted to the spot, which Kane hailed as one of the best goals his nation has ever scored.
"That's the desire and attitude from the boys and staff and everyone involved. It looked tough for a second but you keep going," the England captain told ITV.
"Jude Bellingham scored an unbelievable goal and he's kept our tournament alive. His goal was one of the best in our country's history. What a player he is. He works so hard for the team. In the big moments he steps up. That's what we need. We need everyone to step up in big moments.
"We had a lot of attacking players on the pitch. In this competition you find a way to win. We know we'll do whatever it takes. This team knows how to dig deep. We could have played better but ultimately it's a results business."
England have now reached the quarter-finals in all four of their major tournament appearances under Southgate, who will take charge of his 100th game for his country in the quarter-final with Switzerland, where Marc Guehi will miss out through suspension.