The historic 13-time champions were beaten by the Moldovan minnows in a shock result at the Bernabeu.
To the surprise of even the goalscorer himself, it was Sheriff who took the lead.
Cristiano da Silva Leite burst down the left and floated an impressive ball across the area.
Jasurbek Yakhshiboev made himself available and headed home to the disgust of the home supporters.
Whistles rang out around the stadium as he celebrated in disbelief.
Come half-time, the score remained 1-0.
In the second period Madrid got back on level terms.
A clever bit of skill from Vinicius lured a reckless challenge which allowed the Brazilian to take a tumble and win a penalty.
Karim Benzema converted on 65 minutes, but it was Sheriff who went on to find a winner.
Sebastian Thill scored in the 89th minute to stun the Bernabeu and sent Sheriff top of the group with two wins from two.
Sheriff sit on six points having also beaten Shakhtar 2-0 in their opener.
Meanwhile Real Madrid are on three points as they beat Inter 1-0 last time out.
Inter and Shakhtar drew 0-0 earlier on Tuesday, meaning they are both level on one point each.