Fred was the unlikely hero as he pulled out a beautiful shot to win the day and finally broke Palace's resistance on 77 minutes.
United started the game strongly and a difference in their playing style was instantly noticeable.
Rangnick's side seemed to line up in a 4-2-2-2 shape with Marcus Rashford partnering Cristiano Ronaldo up front and leading their pressing, with Jadon Sancho and Bruno Fernandes just behind them.
However, despite these promising signs, few chances were created by the home team in the opening 45 minutes.
Ronaldo fired over with a free kick, while Palace had a couple of half-chances on the break, but there was nothing clear-cut from either side.
This pattern continued into the second half as both defences held firm, while United began to revert to some old habits.
Mason Greenwood replaced Sancho on the hour mark while Patrick Vieira brought on Odsonne Edouard in place of Christian Benteke.
With just over 20 to go, Alex Telles fired a free kick from out wide against the top of the crossbar and it looked as if it may not be United's day.
An even better chance fell to Palace's Jordan Ayew just five minutes later as he fired across goal and wide after the ball was headed to him from an initial corner.
This warning sparked the winner as United responded by charging back onto the attack.
In the 77th minute, Fred made himself the unlikely hero as he struck a sweet shot over Vicente Guaita and into the back of the Palace net from the outside of the penalty area.
Palace pressed for an equaliser with multiple corners in stoppage time, but the home team held to give their new boss a winning start.