talkSPORT 2 were in attendance for the game, which also saw the Gills relegated to League Two, as Rarmani Edmonds-Green and Georgie Kelly struck in Kent.
But, at the end of the game, both sets of fans invaded the pitch resulting in 'punches' thrown before stewards were able to separate them.
Commentator Ian Danter revealed Rotherham fans looked set to invade the pitch at full-time and co-commentator Perry Groves revealed their manager was telling supporters to remain in the stands.
He said: "Paul Warne and his coaching staff have been ushering them, saying 'please don't invade the pitch, we'll come to you and we'll celebrate together'."
The final whistle blew, confirming the Millers were up and the Gills were down, leading to trouble on the turf.
Danter confirmed both sets of fans had run onto the pitch with stewards 'amassing on the halfway line' in a bid to separate them.
But moments later added: "I have to report that there are some skirmishes on the halfway line between rival fans. Things are being thrown from one set of fans into the other. This is not what we wanted to see."
Groves added: "Idiots. Absolute idiots. Rotherham fans have come onto the pitch to celebrate, Gillingham fans have come on from the Rainham end and a group of about 25-30 Gillingham fans, young men, have come onto the pitch looking for trouble, probably all under the age of 30 or 35.
"They just wanted to have a fight. To be fair to the stewards, they got there fairly quickly, right along the halfway line, to try and separate them.
"You always get these idiots who are throwing punches and just looking for trouble.
"Some of the Rotherham fans and some of the Gillingham fans are using it as an excuse to have a fight."
Danter added: "Thankfully they were only brief skirmishes and now a lot of those Gillingham fans are walking away from the halway line back to the Rainham end."