Al Ahly took the lead through Ali Maaloul with an early penalty before Karim Benzema was afforded the chance to equalise from the spot just before half-time.
However, the Frenchman's penalty was a weak one and well-saved by goalkeeper Mohamed El Shenawy.
That proved to be a crucial turning point as Al Ittihad could not muster much more after the break.
A quickfire double from Al Ahly saw them stretch their lead to three goals just after the hour mark.
Benzema did get on the scoresheet but it was far too late to make a difference after netting in the 92nd minute as Al Ittihad crumbled to a 3-1 defeat.
READ MORE ON SAUDI
Lewandowski matches Messi and Ronaldo feat but Al Nassr star reaches new milestone
Saudi Pro League director says clubs aren't 'throwing money' around and rejects UCL claims
It means that Marcelo Gallardo's men are going home before the tournament can truly get underway.
Manchester City will enter the competition later this month after winning the Champions League last season.
The Club World Cup sees the champions from Europe, Asia, Africa and Oceania all contend.
Al Ittihad were granted a spot as the champions of the host nation with this year's competition taking place in Saudi Arabia.
Most read in Football
Nottingham Forest boss Cooper under threat but can be buoyed by Tottenham's Friday curse
Everton confirm Goodison Park farewell season and when they will move into new stadium
Sunderland close to naming new manager following Will Still links
Arsenal legend defends Mikel Arteta as Gunners boss escapes FA ban over explosive rant
The Jeddah-based outfit boast a star-studded side after an influx of big names to the Saudi Pro League.
As well as Benzema, who joined from Real Madrid this summer, N'Golo Kante, Fabinho, Jota and Luiz Felipe all signed for Al Ittihad in the last transfer window.
The Club World Cup event gave the Saudi side a chance to flaunt their expensive toys to the world.
They beat Auckland City 3-0 in the first round but did not make it past the second round, meaning Al Ahly will now face Copa Libertadores winners Fluminese.
City, meanwhile, will take on AFC Champions League winners Urawa Red Diamonds.