Cristiano Ronaldo had fired the Red Devils, last season's beaten Europa League finalists, into the lead in the 13th minute.
The Group F match, though, changed in the 35th minute following a red card for United full-back Aaron Wan-Bissaka after a rash challenge on Christopher Martins Pereira.
The Swiss side rallied to equalise through Nicolas Moumi Ngamaleu in the 66th minute.
Just when it seemed United were set to hold out, Siebatcheu latched onto a stray back pass from Jesse Lingard to fire Young Boys to victory, sparking wild celebrations.
In Tuesday night's other Group F game, Villarreal and Atalanta drew 2-2 at El Madrigal.
Remo Freuler gave the Italians the lead in the sixth minute, but Manuel Trigueros hauled the Yellow Submarine level shortly before half-time.
Villarreal - who beat United on penalties to lift the Europa League trophy - equalised through Arnaut Danjuma in the 73rd minute.
Robin Gosens, though, struck late to earn Atalanta a point.
Holders Chelsea opened their Group H campaign with a 1-0 win over Zenit St Petersburg at Stamford Bridge.
Summer signing Romelu Lukaku continued his impressive form with a crucial breakthrough with 20 minutes left when he headed in Cesar Azpilicueta's deep cross.
Zenit, though, almost snatched a point after Artem Dzyuba stabbed wide when sliding in at close range.
Juventus laid down a marker with a 3-0 win at Malmo.
Juve took the lead somewhat against the run of play through Alex Sandro's 23rd-minute header.
Paulo Dybala added another from the penalty spot just before the break, but there was still time for Alvaro Morata to fire in a third.
Robert Lewandowski scored twice as Bayern Munich opened Group E with an impressive 3-0 win in Barcelona.
Thomas Muller's deflected effort put the Germans in front in the 34th minute.
Bayern frontman Lewandowski then scored for the 18th club game in a row in the 56th minute after the ball came back off the post.
The Poland striker grabbed his second five minutes from time when he knocked the ball in after Serge Gnabry's shot had also hit the upright.
Dynamo Kiev saw a stoppage-time goal from Mykola Shaparenko ruled out following a VAR review as they had to settle for a point following a 0-0 draw at home against Benfica.
In Group G, there were four first-half penalties as Sevilla drew 1-1 against Salzburg at the Ramon Sanchez Pizjun, having played most of the second half with only 10 men.
Salzburg were awarded a spot-kick in the 13th minute when Karim Adeyemi was fouled by Diego Carlos - but the forward then dragged his penalty wide.
Luka Sucic showed his team-mate how it should be done when he converted his penalty in the 21st minute after Adeyemi had been bundled over again.
The 19-year-old midfielder, though, struck the post with Salzburg's third spot-kick, awarded in the 36th minute following another foul on Adeyemi, this time by goalkeeper Yassine Bounou.
Sevilla equalised shortly before half-time with a penalty of their own from Ivan Rakitic.
The Spaniards saw forward Youssef En-Nesyri sent off for a second caution five minutes after the restart, but held out for a point.
It finished goalless between Lille and Wolfsburg - but there was late drama after a penalty for the French side deep into stoppage-time was overturned by VAR.
Lille had earlier seen a goal for Jonathan David ruled out by VAR at the start of the second half before Wolfsburg had defender John Brooks sent off for two cautions.