The Hoops enjoyed a stellar start to the match after Daizen Maeda put them ahead inside 25 seconds, before Matt O'Riley scored from the spot to double the visitors' lead heading into half time.
Rangers breached the net of Joe Hart via James Tavernier's penalty in the second period and the Gers were level when substitute Abdallah Sima scored his 11th of the league term late on.
The drama in Glasgow was far from over at that point though, with Adam Idah firing in an 88th minute strike as Celtic regained their lead, before Rabbi Matondo curled a long-range effort past Hart to force a share of the spoils
The roar from the Ibrox crowd following the first whistle had barely quietened down before the Scottish champions had taken a dramatic lead, with - as was the case last season - Maeda getting his name onto the scoresheet during a crucial Old Firm clash in the title race.
The early play found its way back to Celtic goalkeeper Hart from kick off and the former England international thumped the ball forward, sending Maeda into a foot race with Tavernier, who smashed his attempted clearance off the foot of the winger, with the block cannoning into the Rangers net past Jack Butland.
Brendan Rodgers's troops managed to double their advantage in the first period from the penalty spot, after Rangers centre-back Connor Goldson was judged to have elbowed the ball following a spell of Celtic pressure which culminated with a dangerous-looking cross by Nicolas-Gerrit Kuhn.
Stepping up to the spot under an enormous amount of pressure considering the state of the Premiership title race, O'Riley held his nerve brilliantly from 12 yards out to beat Butland in the hosts' net, slowly working his way towards the ball before calmly lifting a panenka effort into the middle of the goal.
Following a poor first-half showing from the home side, the all-Rangers contingent inside Ibrox booed their men off as they entered the tunnel for the half-time break, with it appearing that the fight for top spot in Scotland was getting too much for Phillipe Clement's charges.
Looking to change the fortunes of his side in the second period, Belgian boss Clement made a single alteration ahead of the final 45 minutes, introducing winger Sima into the Old Firm fire at the expense of 26-year-old Scott Wright, who managed just eight passes in the first half.
It only took seven minutes after the break for Rangers to reduce the deficit in a vitally-important clash, with Tavernier scoring the second spot kick of the match following the VAR-assisted decision to award the hosts a penalty due to Alistair Johnston's risky challenge on Fabio Silva in the box.
After an extended period of pressure from Rangers, Clement's side managed to restore parity in the contest, with Matondo's effort being blocked by teammate Cyriel Dessers before the ball dropped into the path of Sima, whose shot evaded the attentions of Hart due to a cruel deflection.
In what was becoming an increasingly-open game as the full-time whistle beckoned, Celtic thought they had regained possession of maximum points when Paulo Bernardo found the feet of Idah in the Rangers box, with the Norwich City loanee working a yard of space and fizzing a low effort past Butland at the near post.
Desperate to avoid the gap at the top of the table extending to four points in the favour of the Hoops, Rangers rallied immediately after that Idah-shaped setback in search of an equaliser - and that is exactly what the hosts earned when Matondo danced inside the box and curled a strike into the top-right corner.
Looking to build on a simply-extraordinary contest versus their Old Firm counterparts, Rangers will travel to Dundee in the Premiership on Wednesday, whereas Rodgers's Celtic have six days to prepare for a trip to St Mirren next weekend.