The Foxes lost 4-1 to their Championship opponents at the City Ground after lifting the trophy last season.
Rodgers' side have conceded 16 goals in their last four away matches in all competitions and it looks set to be an underwhelming campaign.
The bar was set incredibly high since he took over, achieving two fifth place finishes and a piece of silverware last season.
Former Tottenham midfielder Jamie O'Hara believes the writing could be on the wall based on the manager's post-match comments.
He told talkSPORT Breakfast: "Leicester fell apart. Defensively, oh my god, they are all over the place. Just the positional sense, whereabouts of the team as a back four.
"The best back-fours are solid and compact, if they step up, they all step up.
"The goals they are conceding - set pieces, players dropping when they shouldn't drop. They are all over the place. It's really bad.
"I listened to Rodgers' interview after the game as I was interested to see what he said, and he was kind of saying, 'we've had this team for a long time and that's worrying'.
"What's he getting at there? Is he getting at that he's taken this team as far as he can take it?
"When you start coming out and say, 'we've had this team for a while and that was a poor performance and that was worrying', that says to me maybe your time might be coming to an end."
There has been disquiet among the club's supporters after recent results and former Charlton, Crystal Palace and Newcastle midfielder Darren Ambrose believes the mood is shifting against him.
He told talkSPORT: "I think they could have had enough. You talk to a Spurs supporter, I'd have taken fifth, fifth and an FA Cup in the last three years.
"It's getting a little bit stale [for Leicester] in the Premier League. They sit 10th and are way off the top four. They're not going to compete for the top four.
"They are now out of the FA Cup after a terrible result. I think you will start seeing more Leicester supporters edging to Brendan Rodgers out."
Rodgers was furious after the game and said a number of his players are fighting for their future.
He questioned some of their hunger and desire in the drubbing to the Championship side, which he branded an 'embarrassing' result.
"There are a lot of these players, between now and the end of the season, who need to prove they are still worthy of being here because we've seen it now for a little while," said the Foxes boss.
"Forget about the players who are missing. There are players here that may have achieved everything that they can.
"It's something we'll have to look at between now and the end of the season. Until then, they've got to have a look at themselves in the mirror and fight like hell to prove they're good enough to be here.
"That's why a lot of these players are not top players, because they can't sustain it. That's where my admiration of the really top players who can keep that hunger and desire - no matter how much they achieve - comes from.
"I always judge it on the physicality of the team. The pressing, the aggression. When you stop getting it, you can't quite get there, your time could be up.
"I said to the players after that, for the first time since I've been here I've been embarrassed. It was an embarrassing performance. I feel for the supporters.
"We brought the FA Cup into our team meeting today so the players could see what we went through last year to win this competition, to remind them of the day at Wembley and the journey. To provide that motivation and extra desire to come out here."