The Bhoys were comfortably outplayed by the Norwegian champions, who rounded off a 5-1 aggregate win to reach the last 16 of the inaugural tournament.
Just like the first leg, Celtic conceded early on and struggled to find an attacking rhythm.
Ola Solbakken netted inside nine minutes as the hosts started brightly and Celtic were outclassed for the majority of the first half.
There was a bit of a reaction from the visitors either side of the break but Hugo Vetlesen killed off any hopes Celtic had of a comeback when he scored in the 69th minute.
Ange Postecoglou has now presided over three tournament exits as Celtic's European campaign ended with a seventh defeat in 14 matches.
The cinch Premiership leaders now have domestic success to focus on and Postecoglou possibly had one eye on Sunday's visit to Hibernian when he picked his team.
Skipper Callum McGregor, defender Cameron Carter-Vickers and in-form wingers Jota and Liel Abada were all named on the bench.
Postecoglou had passed up the chance to train on the Aspmyra Stadium's artificial surface, which had been cleared of heavy snow, and Celtic found themselves facing an early flurry of chances.
They had already survived a penalty claim and some close shaves before Solbakken curled home after Celtic were left outnumbered when Nir Bitton lost possession near the halfway line, although the Israeli claimed for a foul.
Celtic had plenty of possession and territory but were unable to make it count, unlike the Norwegians, who produced 11 first-half shots to the visitors' one.
Joe Hart kept the score down with two last-ditch stops from Amahl Pellegrino.
Celtic looked a bit more threatening in the closing stages of the half but their only effort was a comfortably held Giorgos Giakoumakis header, which the Greek striker could not get any power on given Anthony Ralston's cross was just behind him.
Postecoglou brought on McGregor and Abada for the ineffectual Matt O'Riley and Tom Rogic and switched to a 4-2-4 which helped Celtic push their hosts back in the opening stages of the second half.
They had a great chance for a lifeline when James Forrest's header put Daizen Maeda in behind but the striker blazed over.
Vetlesen soon punished Celtic. The midfielder initially picked the ball up inside his own half and continued his run into the box after laying the ball off before sweeping home.
Postecoglou looked like he was admitting defeat when he replaced Giakoumakis with holding midfielder James McCarthy, with the Greek striker's goals increasingly important for Celtic's title hopes amid the ongoing absence of Kyogo Furuhashi.
Abada had a couple of chances to score a consolation goal but the normally accurate wide man was twice off target.