Fifteen minutes after coming on, Grabban tapped home an exquisite cross with seven minutes remaining to send City Ground into ecstasy and his side into the fourth round.
Mikel Arteta's men were tepid throughout the encounter and their Championship opponents were deserving of their victory.
The hosts were able to withstand a late barrage of pressure to send Arsenal out at the third round stage for just the second time in 25 years.
In the opening stages, Ben White cut in from the right side and tested the waters early with an early shot, but his effort went wide of the post.
Shortly after, Rob Holding fired an audacious volley wide as the visitors pushed for the opener.
But Nottingham Forest responded with a spell of possession of their own to rile up the City Ground fans.
Eddie Nketiah, whose future remains unresolved with his contract expiring next summer, struck a fierce effort wide of the most for the Gunners' closest chance of the match with 30 minutes played.
Nuno Tavares was having a tough game on the left flank and Mikel Arteta decided to bring him off for Kieran Tierney.
The Portuguese was not carrying an injury and was visibly irate by the decision, chucking his globes to the ground as he made way for the Scot.
The sides deservedly went into the interval all square with few clear cut opportunities for either side.
Brennan Johnson had the best chance of the match shortly after the break as he fired a low effort at Bernd Leno who got down well to save.
The match sprung to life in the second half and Nketiah should have got the opener for Arsenal when Bukayo Saka crossed an inch-perfect delivery for him to completely miscue his header.
James Garner fired a superb curling free-kick goalwards but Leno blocked well, but Nottingham Forest would be aggrieved not to have a penalty seconds after when Jack Colback was hauled over in the box.
But the hosts got their deserved goal late on when Grabban fired home the winner to seal the latest upset in a weekend of shock results in the oldest competition in football.