Rodri made the breakthrough in the 14th minute before Julian Alvarez doubled the home side's advantage just past the hour, with the result moving the Citizens onto 40 points from 19 matches.
Pep Guardiola's side, who had Kevin De Bruyne back on the bench after a long-term hamstring problem, are now just two points behind leaders Liverpool in the table, on the same number of games.
Aston Villa secured a late win over Burnley to move into second, with Arsenal now dropping to fourth due to the results in the 3pm kickoffs.
Sheffield United gave Man City a scare in the sixth minute when Oliver Norwood's deep cross caused Ederson problems, with the Brazilian catching the ball and then stumbling backwards, but he managed to keep it out.
The Citizens were in control of the contest, though, and they managed to make the breakthrough in the 14th minute through Rodri, who picked out the bottom corner with a fine finish after smart work from Phil Foden to register his fifth goal of the 2023-24 campaign.
Alvarez came close to doubling Man City's advantage in the 17th minute, with the Argentina international whistling one just wide of Wes Foderingham's post.
Sheffield United's goalkeeper was called into action in the 23rd minute to keep out a strike from Bernardo Silva, and the Blades were finding it incredibly difficult to repel the pressure.
Kyle Walker was next to try his luck in the 25th minute, firing over the crossbar from distance, before Foderingham kept out a strike from Grealish in the 41st minute.
Sheffield United had two chances of their own in quick succession late in the half, with Jayden Bogle first setting up William Osula, but Manuel Akanji was on hand to make a vital block, before Osula forced Ederson into a save with a header from the resulting corner.
Man City had 81% of the possession in the first 45 minutes, but strong defending from Sheffield United meant that the two teams were only separated by one goal at the break.
The Citizens had a chance to double their advantage early in the second period, with Rodri finding Foden in a dangerous area, but the England international's touch was disappointing, which allowed Foderingham to clear.
Man City did manage to double their advantage in the 61st minute, through, with Alvarez tapping a low Foden cross into the back of the net at the far post.
Alvarez had a chance to score a second in the period that followed after a mistake from Foderingham, but the attacker's effort was over the crossbar, much to the relief of the away side's goalkeeper.
The reigning champions finished with 81% possession and had 18 shots during a dominant performance, and they are now within touching distance of leaders Liverpool in the table.