The defender, making his first start of the season, struck in the fourth minute of stoppage time as Valerien Ismael's unbeaten side finally shook off their battling hosts.
Ajayi was left with a simple task to strike from close range after frontman Matt Phillips raced down the right and cut the ball back under strong pressure from Posh defender Dan Butler.
The goal sparked wild celebrations with boss Ismael booked by referee James Linington for his part. The Baggies chief was then involved in a flare-up with members of the Posh management team following a dramatic conclusion to the contest.
Ajayi was no doubt a relieved man after failing to take a glorious chance to put Albion ahead just before the hour.
On that occasion he fired against a post from point-blank range after debutant Jordan Hugill's header was palmed away by Posh keeper Christy Pym.
The Baggies also saw two strong penalty appeals ignored by Linington as they launched a second-half onslaught which eventually earned reward.
Pym was involved in the first spot-kick shout when diving at the feet of Phillips, but he made contact with the ball before the West Brom frontman went down.
And Albion were again left frustrated late on after Karlan Grant hit the deck when attempting to burst between two opponents.
Loan recruit Hugill failed to take another excellent chance to mark his first Albion outing with a goal when he headed a Phillips cross wide in the closing stages.
But defender Ajayi popped up in the right place at the right time to earn a fourth victory in five Championships outings so far for a club seeking a quick return to the Premier League.
The Albion attack - without three-goal leading scorer Callum Robinson due to a Covid-19 case - had rarely troubled the Posh defence before the break.
West Brom midfielder Darnell Furlong sliced an early effort from a tight angle harmlessly wide before Grant saw a low drive pushed away by Pym.
It took Posh 38 minutes to force Baggies number one Sam Johnstone into action but a curling Jack Marriott effort provided nothing more than catching practice.
Johnstone then watched an ambitious half-volley from teenage talent Harrison Burrows whistle wide before the same Posh prospect saw an effort from the edge of the box deflect wide for a corner which came to nothing.
But the home side found themselves under the cosh throughout a second period bossed by Albion and their hopes of a valuable point were eventually snatched away in the dying moments as the visitors triumphed.