Alvaro Morata sent La Roja ahead in the 11th minute of the contest, but Japan turned the match around in the early stages of the second period, levelling through Ritsu Doan before Ao Tanaka won it in the 51st minute.
There was controversy about the winner, though, with the ball appearing to go out before it was delivered into the Japan midfielder, who could not miss from close range.
Hajime Moriyasu's side have topped the section and will take on Croatia in the round of 16 on Monday, while Spain have finished second to set up a clash with Morocco on Tuesday.
Germany ended their group stage with a 4-2 victory over Costa Rica, but it was not enough to progress, with Hansi Flick's side being eliminated from the competition.
Bright work from Nico Williams down the right led to Spain having the first shot of the match in the sixth minute, although Sergio Busquets's effort was always rising over the crossbar.
Japan had an opportunity of their own two minutes later when Busquets was caught on the ball in a dangerous area, with Junya Ito firing narrowly wide of the post.
Morata then tested Japan goalkeeper Shuichi Gonda after meeting a cross from Dani Olmo, with both sides threatening during an open start to the Group E fixture.
Spain made the breakthrough in the 12th minute of the contest when Morata, who was rewarded with a start after two substitute appearances at the tournament, headed a cross from Cesar Azpilicueta into the back of the net.
The bad news did not stop there for Japan, though, as Germany had also taken the lead against Costa Rica to move into second position in the section.
Morata had another opportunity in the 22nd minute, but the forward could not beat the Japan goalkeeper on this occasion before Olmo had a close-range effort blocked, as La Roja threatened to take the match away from their opponents.
Japan picked up three yellow cards late in the first period, which was a sign of their frustration, with Spain dominating the opening half of action, incredibly having 83% of the possession.
Samurai Blue were level in the 48th minute when Doan found the back of the net with a powerful strike, but Spain goalkeeper Unai Simon got two hands on the ball and perhaps should have done better.
Japan then incredibly scored a second in the 53rd minute through Tanaka, with the midfielder turning a cross from Kaoru Mitoma into the back of the net.
On first viewing, it did appear that the ball had gone behind before Mitoma delivered, but a VAR check confirmed that the goal would stand, turning the group on its head.
The turnaround was incredible considering Japan's issues in the first half, and Spain boss Luis Enrique responded by introducing both Ferran Torres and Marco Asensio, with Ansu Fati and Jordi Alba following 10 minutes later.
Japan had a brilliant chance for a third when Takuma Asano was found by a low cross from Mitoma, but the substitute fired wide of the post.
Costa Rica took the lead against Germany in the 70th minute, which with the results as they were, would have sent Spain out of the competition, but Flick's side levelled shortly after before going on to win it.
Spain were living dangerously, with Japan always a threat on the counter-attack, but La Roja were dominating the ball as the match headed for its final exchanges.
Gonda did well to keep out a long-range effort from Asensio in the 89th minute before the Japan goalkeeper denied Olmo moments later.
Olmo then fired wide of the target in the final moments, and Japan held on to incredibly top Group E, boasting a total of six points from their three matches in the section.
Germany were the team to suffer rather than Spain, though, with La Roja managing to progress to the next round ahead of their European rivals on goal difference.