The 19-year-old was shipped out to the Championship new boys on a straight loan deal for the 2023-24 campaign, but he has endured a miserable time at Portman Road.
Scarlett has not started a single game for Ipswich in league or knockout action, making 13 substitute appearances in total and failing to register a single goal or assist in that time.
Scarlett's last game for Ipswich came against East Anglian rivals Norwich City on December 16, and he did not make Kieran McKenna's squad for subsequent meetings with Leeds United or Leicester City.
The Tractor Boys boss confirmed to BBC Radio Suffolk before the Boxing Day draw with Leicester that discussions were ongoing to send Scarlett back to Tottenham, who will lose Son Heung-min to Asian Cup duty in January.
"There's conversations with Tottenham at the moment. I think it looks very likely that he'll be recalled," McKenna said. "Tottenham have injury issues at the moment and they have their main forward [Son Heung-min] going to the Asian Cup at the start of January as well.
"So there have been discussions over the last few days and there's a fair chance that he'll be recalled from the start of January."
Following McKenna's admission, Tottenham have confirmed that Scarlett will be heading back to North London for the second half of the season as Ange Postecoglou prepares to cope without Son for a number of weeks.
The 2023 Asian Cup will take place from January 12 to February 10, meaning that the Tottenham captain could miss up to four games in the early period of 2024 depending on how far South Korea progress.
Tottenham face Manchester United, Everton, Brentford and Brighton & Hove Albion while the Asian Cup takes place, and Son's international call-up would leave Richarlison and Alejo Veliz as Postecoglou's chief central options.
The Lilywhites have been tipped to sign a new out-and-out number nine in the January transfer window and are said to be rivalling Arsenal for the signature of Bournemouth's Dominic Solanke, while Al-Ittihad outcast Jota is also interested in a reunion with Postecoglou in North London.
Scarlett has made 11 senior appearances for Tottenham in all competitions since coming through their youth academy, providing one assist in a Europa League win over Wolfsberger AC in the 2020-21 season, and he made his debut for England's Under-21s in September.
Tottenham will endeavour to stretch their Premier League winning run to four matches when they take on Brighton & Hove Albion on Thursday night, where Richarlison is expected to shake off a back concern in time to lead the line at the Amex Stadium.