The 69-year-old legend, who is regarded as one of English football's top pundits, spent 20 years as a manager between 1986 and 2006 with the final two years of his career being at St James' Park.
Succeeding Sir Bobby Robson as manager was always going to be a tough act to follow and so it transpired as the Magpies secured a 14th place finish in 2005 before sitting in 15th position in February 2006 when he was replaced by Glenn Roeder.
Souness, who is now a part of the talkSPORT family, struggled to get supporters onside during his reign as manager and believes that the club was inharmonious from the minute that he arrived.
And that included with transfers, as he admitted he had another top striker in mind when the club signed Michael Owen instead - a transfer that has gone down in the club's history as one of their biggest flops.
Speaking on White and Jordan, Souness said: "It's a difficult job. I had eight jobs at some very big football clubs, but [at Newcastle] I felt that people weren't rowing in the same direction.
"I didn't feel any togetherness up there from day one. Not from the minute that I walked into the place, but the feeling around the place is that there's an expectation from the supporters, quite rightly so, but it had to be a case where you were going to build it.
"If you look at the managers who had that job prior to me, some proper managers, the last time that they won a trophy was in 1968. That's over 50 years ago.
"They've had issues up there."
Freddy Shepherd was a popular figure as Newcastle owner and was responsible for the club's blockbuster move to sign Owen from Real Madrid for £17million in 2005.
While Souness was in the dugout at the time, the Scottish boss has insisted that he and Shepherd weren't exactly in agreement over the move.
Responding to Jim White's comment that he stood side by side with Shepherd when they unveiled Owen, Souness said: "What did you want me to do?
"He had chosen Michael Owen, so what did you want me to do?
"The story of Michael Owen, who wouldn't want Michael Owen? He's a bit-part player at Real Madrid. I wanted Nicolas Anelka who was at Fenarbahce at the time.
"I'd met his representatives and he was all for coming, but he wasn't acceptable for Freddy Shepherd, he didn't want him.
"That wasn't the only time that happened there.
"The fact that I never worked again tells you how much that job put me off being a manager.
"Listen, I'm a big boy. I've had lots of criticism and I used to enjoy it when I was a player. If people weren't booing me I felt that I wasn't doing my job properly.
"Up there I found it very, very difficult to feel like a part of it.
"I can take as much stick as you want to give me, that's not a problem. It just wasn't a job I enjoyed.
"After that I just knew that I didn't want to be answerable to kind of people who are running football clubs, and answerable to the kind of people who I wouldn't want to have a cup of tea with."