The Englishman popped up with the game's decisive moment in the 79th minute after Kemar Roofe's penalty had cancelled out Michael O'Halloran's opener for Saints early in the second half.
After the disappointment of losing key men Ali McCann and Jason Kerr on transfer-deadline day, the home support received a pre-match boost when it was announced that double-winning manager Callum Davidson had signed a new contract committing him to the club until 2025.
Midfielder Ali Crawford was given his Saints debut, while Davidson's other recent signings Eetu Vertainen, Efe Ambrose and Lars Dendoncker started on the bench.
Rangers made three changes to the side that started the 1-0 win over Celtic, with Jon McLaughlin, Tavernier and Calvin Bassey replacing Robby McCrorie, Borna Barisic and Connor Goldson, who missed out due to having to self-isolate.
The first half was largely uneventful, with the only save of note from either goalkeeper coming in the 37th minute when Saints' Zander Clark had to dive to his left to beat away a 20-yard strike from Roofe.
The match came to life early in the second half, however. Saints took the lead in the 50th minute when O'Halloran latched on to a long pass from Hayden Muller, drove at Filip Helander before stepping away from the Swedish centre-back and firing an emphatic shot high beyond McLaughlin from just inside the box.
Rangers almost made a swift response when Tavernier got to the byline and drilled a low ball across the face of goal but Alfredo Morelos, sliding in, just failed to get his toe on it.
The equaliser came in the 57th minute when Roofe fired home a penalty after Muller was deemed to have fouled Ryan Kent. A flare-up in the immediate aftermath of the goal resulted in Saints pair Shaun Rooney and Liam Craig being booked along with Rangers duo Roofe and Morelos.
The visitors' tails were up and Tavernier curled a shot just over the bar from 25 yards before Morelos saw a deflected effort pushed behind by Clark.
Saints remained a threat, though, and both Crawford and Reece Devine went agonisingly close to restoring their lead.
Things looked to be conspiring against the champions when Helander was carried off in the 77th minute, but just two minutes later Tavernier gave the large travelling support the moment they had been waiting for when he arced a sensational 25-yard strike over Clark and into the net.