Winger Lookman, who grew up in nearby Peckham, marked his full Foxes debut with the opening goal shortly after half-time.
Kelechi Iheanacho wrapped up the victory with Leicester's second two minutes from time.
Foxes boss Brendan Rodgers, perhaps mindful of his side's shaky start to the season, named a strong line-up at The Den despite making eight changes.
But after a scrappy start it was Championship Millwall who went close to opening the scoring in the 14th minute after a half-cleared corner fell to Ryan Leonard, who fired narrowly wide from the edge of the area.
Wales goalkeeper Danny Ward, making a first Foxes appearance since February, then got down well to block Connor Mahoney's low drive.
Millwall's fans, many of whom booed Leicester's taking of the knee despite the best efforts of the stadium sound system to drown them out, began to get behind their side just like they had when Leicester last visited The Den.
Then, amid a raucous atmosphere, Claudio Ranieri's reigning champions were beaten 1-0 by the 10-man Lions in the FA Cup.
But there was to be no repeat of that shock result, in February 2017, once Leicester began to dominate the ball and quietened the crowd down.
Kiernan Dewsbury-Hall, another making his first start of the season for the visitors, should have done better with his cross after a mazy run, and Youri Tielemans curled a free-kick over after he was tripped on the edge of the box by Shaun Hutchinson.
Millwall had another opportunity before half-time but Ben Thompson's header looped onto the top of the net.
But four minutes into the second half Leicester's pressing paid off when Dewsbury-Hall won possession just inside the Millwall half.
Iheanacho smuggled the ball away and when his shot was saved by Lions keeper George Long, former Charlton youngster Lookman arrived to slam the rebound high into the net.
Millwall came closest to an equaliser when Mahoney bent a free-kick into the area which substitute Benik Afobe was inches away from touching into the net.
But with time running out Iheanacho finished a swift Foxes counter-attack with a stinging shot into the roof of the net to seal their passage into round four.