Nothing could separate the sides in the first half, with the score at 0-0.
But after a bright start to the second period and either side yet to draw blood, Barnsley were reduced to ten men following midfielder Phillips' high, late challenge on Owls forward Lee Gregory.
The Tykes man was given a straight red, which stood after a VAR review.
But the decision and use of VAR didn't sit well with former Sky Sports host Jeff Stelling, who tweeted: "VAR total joke again in League One play off.
MORE IN EFL
Kenilworth Road renovations started hours after Luton won promotion to Premier League
Tom Lockyer posts update from hospital after Luton captain collapsed during play-off final
"Clear Barnsley pen [penalty] denied and moments later a red card for a challenge that is yellow at worst. Afraid to tell the ref he has got it wrong."
The Tykes players, coaches and fans were up in arms just moments before Phillips was sent for an early bath as their penalty shout was waved away.
Liam Kitching - who scored in the play-off semi second leg against Bolton - was brought down inside the area and the decision was referred to VAR. But after a check, no spot kick was awarded.
Plenty of fans agreed with Stelling's verdict on Phillips' red card on social media, too.
Most read in Football
Chelsea finally appoint Mauricio Pochettino as ex-Spurs boss makes Premier League return
Leeds prepare for Harrison exit as Premier League duo eye cut-price deal after relegation
Chelsea Twitter admin pokes fun at Leeds after relegation is confirmed
Jeff Stelling given standing ovation on emotional final Soccer Saturday appearance
"Very poor decision. Yellow card, no more," one supporter wrote.
Meanwhile, another said: "Ref guessed Linesman guessed. Both square on. VAR gutless wonders."