Van der Vaart moved to north London in 2010 from Real Madrid, where he played alongside a galaxy of superstars including Cristiano Ronaldo, Kaka, Raul, Sergio Ramos and Xabi Alonso, to name five.
Although he left the glamour of Madrid behind, he joined Tottenham at a time when they had numerous stars of their own.
Ledley King, Gareth Bale, Luka Modric and Jermain Defoe were all on the club's books prior to the Dutchman's arrival.
In Van der Vaart's second season, Scott Parker, Emmanuel Adebayor and Kyle Walker all joined to further strengthen Tottenham's impressive squad under then-boss Harry Redknapp.
Even though the team recorded Premier League finishes of fifth and fourth in his two years at the club and failed to land any domestic or continental silverware, Van der Vaart waxed lyrical about the 'fantastic team.'
Read more football news
Chelsea and Arsenal legend Petr Cech names the one opponent he hated facing
Arsenal's radical line-up where Kimmich could be one of four key deals in dream summer
"Before this interview, I was thinking about that time, you think, 'Wow, that was maybe the best team I ever played in,'" Van der Vaart told talkSPORT Breakfast.
Van der Vaart's first season at White Hart Lane coincided with his best goalscoring return in the league since the 2002/03 campaign with Ajax, as the Dutchman netted 13 times.
A primary reason behind the attacking midfielder's impressive goal return was down to Redknapp and the legendary manager's reluctance to shackle his attacking gem, and a telepathic understanding with Peter Crouch.
It's why Van der Vaart looks back at his entire two-year stint at Tottenham as the time he felt the best on the pitch.
Most read in Football
Arne Slot accused of X-rated Premier League title outburst at Michael Oliver
Thomas Tuchel told to call-up striker who hasn't played for England in seven years
Ex-Sunderland boss to take surprise first job outside UK a year on since last sacking
£97m flop playing Liverpool for first time since missing sitter that left crushed Messi
"I think you have different phases in your career," Van der Vaart said.
"Of course, Ajax especially in the beginning. Then I moved to Hamburg and I had a great time.
"But when you look at team, quality, talent and potential... I think I had the best time at Tottenham.
"Also with Harry Redknapp of course, a coach that I really liked because he gave me so much freedom on the pitch."
Tottenham could certainly do with someone of Van der Vaart's ilk right now given the team sits a lowly 13th in the Premier League, with the Europa League the club's only chance of silverware this term.
Should Ange Postecoglou's side fall short in the Europa League, it would mark the 17th-straight season without a trophy.
As to what Tottenham can do to end a drought unbecoming of a team with their state-of-the-art stadium and training ground, Van der Vaart believed the club must stop overpaying for 'normal' players and instead should be prepared to fork out more for those who will take them to the next level.
"At the end of the day, I think Tottenham always tried to go to the next level, but the last step is the biggest," Van der Vaart said.
"Of course, you can build a new stadium and everything is perfect. If you compare it to my time, we were still on the Chigwell training ground and everything was smaller. White Hart Lane, which for me, was the best ambience I ever played in. Now, everything is big. You want to go to the next step and you don't play like that.
"At the end of the day, they bought so many players for crazy money in my point of view. You can buy less players and buy better ones and pay a little bit more. But now these days, they pay money for normal players."
Van der Vaart added: "At the end, it's the talent of the group and in my days, we had so much talent it was unbelievable."
Van der Vaart and the rest of the Tottenham faithful will have to wait until the summer to see whether Postecoglou and the club heed his advice about how to spend the transfer budget.
Read More on talkSPORT
WWE fans call for star's signature move to be banned after brutal injuries revealed
Ex-Masters winner who went to prison set to end six-year Augusta National exile
But a Europa League trophy would go some way to attracting a higher calibre of players and, at present, is the only way Tottenham feature in Europe next season given their position in the table.
Sebastien Bassong reveals Benoit Assou-Ekotto didn't know who Rafael van de Vaart was when he signed for Tottenham from Real Madrid