The centre-back left Arsenal in the summer after seven years at the club to sign for the Eagles on a permanent deal.
Holding saw players including Alexandre Lacazette and Granit Xhaka captain Arsenal during his time there.
However it's 'leaders' Mertesacker and Koscielny who he draws upon for inspiration as one of Palace's more experienced players.
Speaking exclusively to talkSPORT, he said: "I've got a lot of experience, 28 years old now and they've got quite a young team at Palace, they've got some young lads coming through.
"Any little things that I've picked up from my time at Arsenal when I first stepped into the changing room and you had the likes of Mertesacker and Koscielny, they are just leaders.
READ MORE ON ARSENAL
Mead assists Russo winner on return as Arsenal Women score twice in stoppage-time in win
Emmanuel Adebayor sends message to Arsenal fans over iconic goal celebration
"If I can take any bits that I got from them and pass them onto the lads that are a bit younger at Palace now, then I can do that."
Mertesacker joined Arsenal from Werder Bremen in 2011 and went on to make 221 appearances across seven seasons.
He also has 104 caps to his name for the German national team, with just ten players having more.
Fellow centre-back Koscielny arrived at the Gunners a year prior, playing for nine seasons after he signed from French club Lorient.
Most read in Football
Charlton to face eighth-tier non-league side as FA Cup first round draw made
Ryan Gravenberch reveals the Liverpool player who wows him every day in training
Philippe Clement wins race to be new Rangers manager ahead of Kevin Muscat
Henderson makes first social media post after being booed at Wembley by England fans
The Frenchman featured 353 times for the club and was a crucial part of their backline before leaving 2019.
Both players captained Arsenal during their respective spells in north London, and Holding could also exert such leadership at Palace.
He has so far featured just once for the south London club but has 98 Premier League appearances to his name.
Holding is the joint-eighth oldest member of the Palace squad with Jefferson Lerma and Will Hughes, with 13 of their stars aged 25 or under.
Joel Ward, the Eagles' oldest current player is aged 38, is the current captain although Holding could be in line to take over in the future.