Los Blancos were two goals behind at the interval, with Etienne Capoue and Samuel Chukwueze on the scoresheet for the Yellow Submarine.
However, the reigning Spanish and European champions managed to turn the game around in the final 30 minutes, with Vinicius Junior, Eder Militao and Dani Ceballos scoring in a 3-2 win.
Real Madrid have now progressed to the quarter-finals, where they will take on Atletico Madrid, and Ancelotti has hailed his players for their "incredible reaction".
"I was really angry at half-time and I decided to take a risk and press high up. We weren't being forceful enough in our duels and I told them to wake up, there was still time to come back," the Italian told reporters.
"They definitely woke up. It wasn't a good first half and we can't be playing like that.
"In the second period, it was an incredible reaction, as it always is. This club never gives up but we can't wait until near the end of the game to react.
"We played a very poor first half and quite an unexpected second because it's difficult to switch the dynamic like that after the first half we saw.
"The team showed great personality to turn things around. The second half is fantastic for our confidence. We didn't give up and I'm sure you guys have had to change your articles."
Ancelotti was also full of praise for Marco Asensio and Ceballos, with the duo playing key roles in the comeback.
Asensio assisted Ceballos for the game's fifth goal in the 86th minute, and the pair had a number of positive moments during their time on the field.
"They really changed the game. They brought more quality and energy to the pitch. They were absolutely key in turning the score and the tie around," said Ancelotti after the match.
Real Madrid will now be bidding to return to winning ways in the league when they head to Athletic Bilbao on Sunday night.