The Lions are 11th in the Championship, but were beaten 1-0 by Frank Lampard's Coventry on Saturday.
He will, however, take charge of their next two games before an an interim takes over.
They face Sheffield United at home on Wednesday, before travelling to Middlesbrough on Saturday.
Harris was only hired as Millwall boss in February in what has been his fifth different spell at the club.
He replaced Joe Edwards at The Den, with the club just a point above the relegation zone.
Read more on Championship
Rooney hires former Man United coach after Plymouth assistant leaves for personal reasons
Frank Lampard inflicts 116-year first on arch rivals in second Coventry match
Harris steered them to safety and just six points off the play-offs.
However, his decision comes just days after he referred to some supporters in a derogatory way.
He said: "That is where people have to get their heads out of the clouds and realise how well we've done to achieve what we have so far. And how tough it is for us as a group.
"Right at the end we put the ball in the box and one of my young centre-forwards and one of my young wingers are standing outside of the penalty area. It shows a lot of learning we have got in front of us.
Most read in Championship
Man City left to rue £81million 'mistake' as stunning Champions League stat emerges
Wolves captain loses his head as he throws Bowen to floor before scrapping with own staff
O'Neil clings on at Wolves, shamed referee Coote SACKED, Postecoglou safe at Tottenham
Phil Taylor nearly came to blows with darts legend as handshake caused furious exchange
"You miss Romain, you miss Japhet and you miss Jake Cooper. Three players that probably get in any Championship team. Just a reminder to people, the thickos, that when you take the three best players out of the team that this is Millwall Football Club."
Announcing the decision, Lions Chairman, James Berylson, said: "At the start of 2024, we found ourselves in a tough spot. Neil came in and along with his coaching staff galvanised the squad. Through action and leadership, he led Millwall on a magnificent run of form, ensuring our return to Championship football this season.
"I want to thank Neil for all his efforts since returning to the Club and we will be forever grateful to him for that.
"After discussions with Neil in recent days, he felt it was the right time to step down from the role following the Middlesbrough game and I have accepted his decision."
"I join everybody associated with Millwall by wishing Neil the very best. He will always be welcome back to The Den as a friend, colleague, and Lion.
"As we look forward, the appointment of our next head coach will align with the clear strategy and vision I have previously laid out for the future of Millwall Football Club, both on and off the pitch.
Director of Football, Steve Gallen, added: "We respect Neil's decision and he departs with the best wishes of everybody at the club. Since I joined in May, I have enjoyed my time working alongside Neil and I thank him for his efforts during this period.
"A comprehensive recruitment process is already underway for a new head coach. The direction we as a club want to head in is clear and the next head coach will play a pivotal role in helping us execute our strategy."
Writing a letter to the fans, Harris said: "Today I am announcing that I will be leaving Millwall Football Club after Saturday's game against Middlesbrough.
"This isn't an easy decision, as everyone knows this club is more than a job to me, it's my club, and always will be.
"Being here always felt like home, as a player, as a captain, and as a manager. We are a family, and it's been my life for so many years. But I always said when I returned that I would leave when the time felt right, and now is that time.
"Coming back last season and all the highs that came with it will never be rivalled. It was one of the highlights of my career. We fought together as a team, as fans and as a united football club.
"We have some special characters in this team. They are dedicated and hungry to do well. With the right encouragement and support, I'm sure they can achieve great things.
"I would like to thank all the staff at the club for their loyalty and support and especially the owner James Berylson, for welcoming me back. He's been superb to work with, and I will always be grateful for the great friendship we shared.
"We have the most passionate fans in the country, and your support has meant the world to me. We are often misunderstood, but the uniqueness of Millwall is also what makes us so special.
"I wish all of my friends at Millwall Football Club the best for the rest of the season and I will see you soon."
Reacting to the news, former Crystal Palace owner Simon Jordan joked he hoped it would bring about back-to-back relegations.
He told talkSPORT: "Interesting. Hopefully that will spark an enormous decline in Millwall and they'll end up in League Two where they belong."
Read More on talkSPORT
Conor McGregor left stunned by record-breaking 'world's fastest' two-second knockout
Dan Ashworth leaves Man United after just five months as sporting director
Host Jim White asked: "You don't honestly mean that, do you?"
With a big grin on his face, Jordan added: "I most certainly do."