United, who conceded a late goal and then missed a penalty in stoppage time in the 1-0 loss, have now lost three of their last four games in all competitions.
And talkSPORT pundit Cascarino believes that something isn't quite right at the club - which is highlighted by big-money signing Jadon Sancho's lack of game time.
Sancho has made seven appearances for Man United this season, but the England international and former Borussia Dortmund star has completed 90 minutes just once - in the Carabao Cup third round exit to West Ham.
"There feels like there's a lot of deflecting going on from the actual performance," the former Chelsea striker said on the Weekend Sports Breakfast.
"After the talking midweek, they've lost through three games and the Young Boys defeat in the Champions League - there feels like there's something behind the scenes that is not quite right now.
"I don't know if it's the senior players, and me and Natalie [Sawyer] had this chat off-air, where we're talking about when you have senior players, you end up giving them a lot more loyalty and you sort of trust them a little bit more.
"There's something bizarre that's going on with Jadon Sancho. There is something that's really odd about what's happening with him at Old Trafford for being a £73 million transfer."
talkSPORT's Manchester correspondent and ex-Premier League star Micky Gray is also concerned about the Red Devils' form and, in particular, goalkeeper David De Gea.
De Gea, who was battling for the No.1 shirt with understudy Dean Henderson last season but now seems to be Ole Gunnar Solskjaer's preferred choice in goal, has only kept one clean sheet in seven appearances this season.
And Gray says the Spaniard is one of the best in the Premier League in one-on-one situations.
But when it comes to his distribution and ability with the ball at his feet, he says the keeper would struggle to get a game in the fourth division of English football.
"His distribution with the ball is, and maybe I'm being a little bit over critical, but I think with the ball at his feet, he wouldn't get a game in League Two," the former left-back told talkSPORT.
"When I watch him play, he just does never look comfortable with the ball at his feet.
"I'm saying this might be a little bit over critical, but I just don't feel confident at all, and I don't think he does either, defensively.
"As I said before, I think he's the best in one-vs-one situations. He's absolutely outstanding."