Rigobert Song's side needed to win and hope for Serbia to take at least a point from their clash with Switzerland to seal a top-two place, but they were on the back foot for large portions of the contest against a second-string Brazil side.
However, Vincent Aboubakar headed home an unexpected winner for the African nation in second-half injury time, but having already been booked, he was immediately sent off after ripping his shirt off in celebration.
Despite holding on for a memorable win, Cameroon's fate was sealed as Switzerland overcame Serbia 3-2, which eliminated the Indomitable Lions and allowed Brazil to hold on to top spot on goal difference.
Brazil manager Tite was not afraid to ring the changes, making a total of nine alterations from the side that secured a 1-0 win over Switzerland on matchday two, and Gabriel Martinelli - who started a World Cup game for the first time in his career - was central to the Selecao's fruitless first-half efforts.
The Arsenal attacker had the first meaningful chance of the game in the 14th minute after meeting Fred's cross at the back post, but Devis Epassy did well to tip his header over the top of the bar.
Brazil continued to dominate the match, albeit without creating much in the way of clear-cut chances, although Martinelli was consistently giving Collins Fai the run-around on the left-hand side as Antony and Rodrygo also caused problems with their darting runs.
Epassy was forced into a comfortable save from Antony's effort in the 38th minute, as the Manchester United man cut inside onto his left foot before firing a tame effort into the arms of the goalkeeper.
However, a gilt-edged chance fell the way of Song's side just before the half-time whistle, as Brentford's Bryan Mbeumo saw a back-post header of his own kept out by Ederson with the Manchester City goalkeeper at full stretch.
The half-time break did not derail Cameroon's momentum whatsoever, as Aboubakar sent an effort a whisker wide of the post in the 51st minute after being teed up by Mbeumo.
Despite the threat posed by the Indomitable Lions, Martinelli soon reignited his never-ending battle with Epassy, who produced another fine save from the 21-year-old in the 56th minute before spilling an Eder Militao effort and eventually managing to poke the ball wide of the post.
The Selecao remained on top as the second half wore on, and the deadlock was still waiting to be broken heading into the final 15 minutes, but a goal for a Cameroon would not have made a difference while Switzerland maintained their 3-2 lead over Serbia.
Substitute Olivier Ntcham was urged to shoot from distance by the Cameroonian crowd in the 78th minute, and the Swansea City man obliged, but Ederson got down to his right to gather.
A myriad of wasted chances would finally come back to bite Brazil in the second minute of injury time, as Aboubakar met a sumptuous cross from Jerome Mbekeli and left Ederson rooted to the spot with a well-placed header into the far corner, although a shirt-stripping celebration led to his sending off as he has been booked only 10 minutes earlier.
Aboubakar shook the hand of referee Ismail Elfath, who had a smile etched across his face as he brandished the red card, and Brazil's profligate streak continued as Martinelli and Bruno Guimaraes missed last-gasp chances to rescue a point.
Cameroon were not down and out when the final whistle at the Lusail Iconic Stadium, but Switzerland's game against Serbia ended barely a minute later to confirm a third-placed finish for Song's side.
Meanwhile, Brazil will continue their quest for glory against Group H runners-up South Korea in Monday's last-16 showdown.