Arteta's side were ostensibly cruising to success thanks to Gabriel Martinelli's opener and Trossard's first of the contest, only for a brace from the unlikeliest of scorers in James Justin - including an astonishing volley - to draw the visitors level out of nowhere.
However, after being left to rue a last-minute gut punch against Manchester City, Arsenal were the ones dishing out the last-minute heartbreak this time around, as Trossard poked home his second of the game before Havertz bundled in the insurance goal right at the death.
Cheers could be heard around the Emirates before kickoff, as the stadium announcer confirmed that Manchester City had dropped points against Newcastle United, meaning that the Gunners would go top with a triumph of at least four goals.
Mikel Arteta's men made a terrific start to that mission, as Gabriel Martinelli ended a goal drought stretching back to March with a perfectly-placed finish into the bottom corner, although Steve Cooper believed that his side should have been awarded a foul in the build-up.
Following the Brazilian's 50th direct Premier League goal involvement for Arsenal, the Gunners continued to huff and puff against the Foxes and nabbed that coveted second goal right before the break, as Leandro Trossard came full circle.
During first-half injury-time last weekend, the Belgian was sent off for kicking the ball away. In first-half injury time this weekend, he finds the bottom corner with a slick first-time finish from a Martinelli ball across.
That goal just before the break would seemingly be Arsenal's one-way ticket to victory, but the visitors - or more specifically Justin - had other ideas, as the Leicester full-back single-handedly got his team back on level terms.
Justin firstly saw his effort from a free kick take a wild deflection off of Kai Havertz, before he channelled his inner Benjamin Pavard with a breathtaking volley, and the Foxes were ostensibly good enough for a point.
However, Trossard managed to poke home in added time to send the Emirates into delirium, before Havertz ensured that Arsenal would go level on points with Man City at the top, but they are still behind the champions on goal difference as the Foxes sit 16th.
ARSENAL VS. LEICESTER HIGHLIGHTS
Gabriel Martinelli goal vs. Leicester City (19th min, Arsenal 1-0 Leicester)
The player who arguably needed a goal more than anybody else in a red shirt is finally off the mark for the new season! Martinelli had let once chance go begging a few moments before, but he can make no mistake with this one.
Jurrien Timber provides an overlapping run for Bukayo Saka, and the Dutchman's ball into a crowded penalty area finds Martinelli, whose shot is not hit with much venom but is well-placed enough to find the far corner.
Cooper is incensed - he felt that William Saliba fouled Jamie Vardy in the build-up. Nothing doing, though, and both Foxes men are booked in the aftermath of Martinelli's 50th Arsenal goal involvement in the Premier League.
Leandro Trossard goal vs. Leicester (45+1 min, Arsenal 2-0 Leicester)
Back in the starting lineup with a bang, Trossard surely ends the game as a contest right on the stroke of half time! Martinelli with the assist as he picks out the Belgian inside the box, and Trossard does the rest, opening up his body and side-footing the ball into the far corner.
A big difference from first-half injury time at the Etihad, where he was made to take the walk of shame for two bookable offences. A word as well for Martinelli, who gets a big hug from Riccardo Calafiori as soon as Trossard puts the ball in the back of the net.
James Justin goal vs. Arsenal (47th min, Arsenal 2-1 Leicester)
Hang on a minute! Leicester are back in the game just two minutes into the second half, as Arsenal - the dead-ball kings - concede from a set-piece.
In fairness to the hosts, there was a huge stroke of misfortune about this one, as James Justin's header from a free kick takes a wild nick off of Kai Havertz and leaves David Raya with no chance.
James Justin goal vs. Arsenal (63rd min, Arsenal 2-2 Leicester)
Justin's first of the afternoon may have been fortuitous, but the Leicester full-back has just popped up with an outrageous goal-of-the-season contender to draw his team level out of nowhere!
Playmaker general Wilfred Ndidi whips a lovely ball to the other side of the penalty area where Justin is lurking, and the Foxes defender connects with a sensational volley, which kisses the inside of the post and crashes the back of the net. He will surely never strike a sweeter volley than that for the rest of his career!
Leandro Trossard goal vs. Leicester (90+4 min, Arsenal 3-2 Leicester)
Death, taxes, and Arsenal scoring from corners. Bukayo Saka whips in a trademark delivery to the back post, and Trossard's poke at the back post deflects through a sea of bodies and into the far corner.
Kai Havertz goal vs. Leicester (90+9 min, Arsenal 4-2 Leicester)
The Foxes' hard work is all for nought. Arsenal go on the attack, Hermansen saves from Gabriel Jesus, but the ball deflects into Havertz's path and he cannot miss from close in. The offside flag initially denies him, but a quick VAR check and the hosts can celebrate number four.
MAN OF THE MATCH - JAMES JUSTIN
For about half an hour, James Justin was the unlikely hero that Leicester needed, as his lucky deflection preceded a wonderful volley that any striker would be proud to score, even if his efforts were ultimately in vain.
However, a men of the match category would be more fitting for this game, as Foxes goalkeeper Mads Hermansen was nothing short of astounding in the second half as his save counter reached double figures, while Trossard was Arsenal's saviour when they needed it most.
ARSENAL VS. LEICESTER MATCH STATS
Possession: Arsenal 75%-25% Leicester
Shots: Arsenal 37-5 Leicester
Shots on target: Arsenal 17-3 Leicester
Corners: Arsenal 17-0 Leicester
Fouls: Arsenal 11-6 Leicester
BEST STATS
WHAT NEXT?
European matters now take centre stage for Arsenal, who welcome Paris Saint-Germain to the Emirates Stadium on Tuesday night for their second Champions League game of the season.
As for the Foxes, Cooper's men have a week to reflect on their positive performance before hosting Bournemouth next Saturday afternoon at 3pm.