The Gers were trailing after only five minutes when Daizen Maeda capitalised on a poor pass from Alfredo Morelos before slotting the ball past Allan McGregor to put Celtic in front.
However, Rangers responded well and came out the blocks quickly in the second half, with Ryan Kent equalising with a curling strike just 100 seconds after the restart.
James Tavernier then emphatically converted from the penalty spot in the 53rd minute to complete the comeback, but it was Celtic who had the last laugh with Kyogo Furuhashi netting an 88th-minute equaliser from close range, forcing both sides to share the spoils.
Beale was denied all three points on his Old Firm debut as Rangers boss and his side remain nine points behind current Scottish Premiership holders Celtic at the top of the table.
Speaking after the match, Beale expressed his disappointment for his side not holding on to their lead and feels a Rangers victory would have been a "fair" result.
"The two goals for us [to lose] are poor goals to give away, certainly the one earlier on", Beale told Sky Sports News. "We started well but we then conceded and took 10 to 15 minutes to find ourselves. We had two big chances with Alfredo [Morelos] and then [Ryan] Kent who hit the post.
"We needed to show more belief and second half we showed that. Got 2-1 up, were in control and they haven't got near Alan [McGregor]. And then maybe it should be 3-1 with the big chance for Malik [Tillman].
"Their equaliser was poor from us, we sat deep and had three or four chances to clear it."
Beale added: "I saw a lot of good things today, a lot of players who were committed to the cause. Second half we were excellent. I thought that if we won it would have been fair, so yeah, we're a little bit disappointed.
"We've certainly got more players to come back and help the starting team, we'll get fitter and stronger as we go. We've only been together a few weeks and now we're at a level to compete with the team we need to compete with.
"We'd like to be two points closer but there are 18 games to go and we'll just keep going."
Meanwhile, Celtic manager Ange Postecoglou hailed the "cracking game" on show for both sets of supporters, but he believes self-inflicted mistakes and the loss of control allowed Rangers to come back into the contest.
"It was a great game of football before we get to anything else, it was a cracking game," the Australian told Sky Sports News. "All eyes, not just in Scotland, but around the world were on it, and both teams at it - a proper derby.
"We started really well and then we lost control of it. Mainly self-inflicted because of our passing, making mistakes and errors we don't usually do.
"They got some momentum, particularly with the crowd behind them. Then at the beginning of the second half they took a foothold for 15 to 20 minutes.
"It's very easy on a day like this to sort of resign yourself to a fate, but we haven't done that this year or last year and we found a way.
"It's a difficult place to come to, they threw everything at us and they knew how important this game was in the context of the league. I thought for the most part we handled it okay."
Asked whether the mistakes made by his players can be easily fixed, Postecoglou added: "They're human beings, they're not robots. It's a big occasion, a big game, and we just tightened up in some areas where our football is a lot more free-flowing and we didn't do that today."
Celtic, who have extended their unbeaten run in the Old Firm derby to six matches, will now turn their attention to Saturday's Premiership clash at home against Kilmarnock, while Rangers will travel to Dundee United for their next league game on Sunday.