Christian Pulisic put the Stars and Stripes into a deserved lead with a stunning strike, but Die Mannschaft valiantly fought back and headed into half time all square courtesy of smart finish from captain Ilkay Gundogan.
The visitors then took control of proceedings after the break thanks to a quickfire double from Niclas Fullkrug and Jamal Musiala within the space of three second-half minutes to help Nagelsmann make a winning start to life on the international stage.
At the age of just 36, Nagelsmann is the man tasked with restoring belief within the German camp ahead of hosting Euro 2024 following a dismal run of form under former boss Hansi Flick, who was sacked just over a month ago after winning only four of his final 17 games in charge.
Nagelsmann, who was dismissed by Bayern Munich in March, had around three weeks to prepare for the clash with USA, and his Germany side were made to work for the victory over USA, who started brightly and posed a constant threat in the opening half-hour.
AC Milan winger Pulisic had the ball in the back of the net inside the first four minutes, only to be ruled marginally offside as he creeped into Germany's half.
Germany responded well, though, and almost opened the scoring 11 minutes in when a side-footed shot from Pascal Gross - starting his first game for the national team - beat goalkeeper Matt Turner and hit the base of the post.
The attacking triumvirate of Pulisic, Folarin Balogun and Timothy Weah linked up brilliantly in the final third, and it was the former who broke the deadlock in marvellous fashion just before the half-hour mark.
Moments after appealing for a penalty and seeing a shot from Germany's Fullkrug saved at the other end, Pulisic cut inside from the left and breezed into a central position before blasting a brilliant strike into the top corner, beating Marc-Andre ter Stegen all ends up.
USA then saw a decent opportunity go begging when Weah's cross into the centre of the penalty box fizzed behind Balogun, before a drilled effort from Giovanni Reyna was gathered by Ter Stegen.
That missed chance proved costly as Germany went up the other end to restore parity six minutes before the break when Leroy Sane and Gundogan combined in the area before the latter lifted a finish over Turner from inside the six-yard box.
Germany came out the blocks quickly after the interval and Fullkrug was denied a goal on the 49-minute mark by a superb diving save from Turner, but the Borussia Dortmund summer signing made no mistake at the second time of asking 10 minutes later.
Fullkrug received a through-ball from Robin Gosens inside the area before turning and slotting a composed finish from around eight yards out beyond Turner to put the Germans in front.
The striker then turned provide just three minutes later when he pounced on a loose ball and lifted a cross over the onrushing Turner towards Musiala, who was presented with a simple tap-in into an empty net from just a few yards out.
Both nations rang the changes in the final half-hour which slightly took the sting out of the contest, but Germany came close to adding a fourth five minutes from time when defender Niklas Sule fizzed a powerful strike narrowly wide.
Nevertheless, Germany managed to see out an important victory, albeit in a friendly, to help Nagelsmann get off the mark as a national team manager, and Die Mannschaft will look to build on their success in Connecticut when they face Mexico in Philadelphia next Wednesday.