After goalless 120 minutes of action, the match at the Parc des Princes went to penalties, where Nice secured a 6-5 victory.
Bulka, 22, left PSG on-loan in the summer, and was the hero on the day saving from both Leandro Paredes and Xavi Simons to help Christophe Galtier's men advance to the quarter-finals.
Lionel Messi was on target for the hosts in the shootout, but it was not enough to prevent a surprise exit.
Following the saves, the Polish goalkeeper couldn't contain his emotions, instantly running away in hysterics, while also being mobbed by his Nice teammates.
He was also seen doing a knee slides and celebrating in the dressing room after the match.
Bulka came through the Chelsea youth-ranks and won the FA Youth Cup in back-to-back seasons, before joining PSG in 2019 after his contract had expired.
The shot-stopper has only featured twice in Ligue 1, while spending time on loan at Cartagena and Chateauroux.
He may not have had any moments to remember during his PSG days, but he certainly has against his employers.
Bulka gave up his No.30 shirt for Lionel Messi, after the Argentinian star made the move from Barcelona to the French capital last summer.
Nice may execute the loan and buy Bulka should he continue to impress.