The clash was contested by a City side who won the treble last term but it was the Gunners, who were pipped to the league title by Pep Guardiola's men, who came out on top.
Leandro Trossard's deflected shot in the final moments of stoppage-time then cancelled out Cole Palmer's stunner.
Mikel Arteta's side then won 4-1 on penalties, something that former England captain Stuart Pearce believes is a 'statement' to the Premier League.
Speaking to talkSPORT after the match, he said: "Yeah I think it does [make a statement] personally.
"They've spent a lot of money, they've bolstered the squad, they had a really good tilt in the Premier League last year, just fell short through injury and one or two other things.
READ MORE ON THE COMMUNITY SHIELD
Arsenal boss Mikel Arteta booked after demanding yellow card for Man City star Rodri
Man City fans ooze over Cole Palmer goal after replacing Erling Haaland against Arsenal
"They've come out today and got their hands on City and maybe got past the hoodoo.
"We're the first ones to turn round and say, 'Nine games without beating City', they've certainly done it today."
Both sides name strong starting XIs as they edge closer and closer to the new Premier League campaign.
Erling Haaland won the Golden Boot last season with 36 goals in his debut season, but was unable to even muster a shot on Sunday.
Most read in Community Shield
Community Shield LIVE REACTION: Ramsdale the penalty hero as Arteta finally beats Pep
Arsenal boss Mikel Arteta booked after demanding yellow card for Man City star Rodri
Man City fans ooze over Cole Palmer goal after replacing Erling Haaland against Arsenal
How to watch Man City vs Arsenal: TV channel, free live stream and talkSPORT coverage
The first half produced little in terms of chances, although it was a fairly even game in the north-west London sun.
It as City who took the lead though in the 77th minute, a stunning curling effort from academy graduate Palmer.
Pep Guardiola's side thought they were closing in on victory but Trossard's deflected shot deep into stoppage made it 1-1.
Arsenal scored all four of their penalties in the shootout, with Aaron Ramsdale making a crucial save to deny Rodri.
The Gunners will now be able to revel in their success as the countdown the days to their return to Premier League action.
They host Nottingham Forest on Saturday, whilst City begin their title defence with a trip to Burnley the night before.