The Villans looked to be on course to book a fourth round tie away at Stoke as they led 1-0 going into the closing stages of the match.
Advertisement
But two quickfire goals in the 88th and 90th minute saw Steve Evans' side progress, leading the home fans booed their team off at Villa Park.
Forgotten man Morgan Sanson had broken the deadlock for the hosts in the 35th minute.
Advertisement
And it seemed as though that would be enough for Unai Emery's side - but it all changed when Leander Dendoncker was sent off for giving a penalty away for tugging Campbell's shirt in the box after the ball was given away.
Up stepped Jamie Reid, who fired the ball home into the bottom left corner where he celebrated right in front of the Villa supporters.
Just two minutes later, Campbell put Stevenage in front to send the 3,000 visiting fans into delirium as the players rushed over to the away end.
And Villa had no answer to going behind despite over seven minutes of stoppage time being played.
Advertisement
Advertisement
It is the first time since the 2013/14 season that Stevenage, who sit second in League Two, have reached the fourth round.
talkSPORT's Dean Ashton waxed lyrical over Stevenage's win on commentary, as he said: "What a giant killing that is.
Advertisement
"This isn't at home for Stevenage, this is away - going to Villa Park and getting the result.
"I know people question the magic of the FA Cup, but it is these results that for me keep the competition alive and show what a unique competition it is."
Advertisement
He added: "It's easy to forget what this will mean for their supporters, the ones that are there especially, that's something that they will remember forever.
"They've travelled there and they've witnessed their side win 2-1 at Villa Park. It's a memory to last forever."