The England captain was booed by a section of Spurs fans as Manchester United cruised to a 3-0 victory in north London.
Kane has just one Premier League goal this season and never looked like adding to his measly tally against a United side who shipped five last week at home to Liverpool.
Dean Ashton said the 28-year-old looked 'miles off it' and 'disinterested' at times in the fame as the dissonance rang through Spurs' vast stadium.
During the summer, Kane was linked with a lucrative move to Premier League champions Manchester City and some have questioned if the breakdown of the move is linked to his stuttering form.
And despite Spurs' poor performances across the season, Agbonlahor insists the trend starts with talisman Kane.
"It starts with Harry Kane," Agbonlahor said. "How many times did they get crosses in and Harry Kane is stood on his toes.
"He was just standing there. The best strikers in the world go and attack.
"It doesn't matter if you've got three centre-halves in there; the best strikers in the world will make the right runs and attack.
"Harry Kane played like a player out there who was thinking 'I don't want to be here, I'm not interested.'
"That's a club that has given him everything, a club that you're meant to love.
"The fans, the club itself loves Harry Kane.
"But his legacy is not going to be the same now after the way he is treating these fans."