The Blaugrana were humbled by 35-year-old Radamel Falcao, whose winning goal sealed their manager's fate.
A short statement read: "FC Barcelona has relieved Ronald Koeman of his duties as first team coach.
"The president of the club, Joan Laporta, informed him of the decision after the defeat against Rayo Vallecano.
"Ronald Koeman will say goodbye to the squad on Thursday at the Ciutat Esportiva.
"FC Barcelona wishes to thank him for his service to the club and wishes him all the best in his professional career."
The Rayo Vallecano defeat is the latest poor result for Barcelona, which has proved the final straw.
On Sunday, a lacklustre home El Clasico loss to Real Madrid provoked fury among fans.
The Catalans have now endured four consecutive defeats to Los Blancos, this time losing 2-1.
Upon leaving Camp Nou at the weekend, Koeman was targeted by a number of angry supporters who surrounded his car.
He was subjected to verbal abuse, with banging heard as his vehicle drove past angry fans.
As yet, there has been no confirmation on who will take over the role as manager.
It's reported that club legend Xavi is the favourite to get the job.
He is currently in charge of Qatari side Al Sadd.