Goalkeeper Kelle Roos was sent off in the 57th minute for a foul on Sharp and the Rams looked to be holding out for a point until the Blades striker kept his cool from 12 yards at the death.
Slavisa Jokanovic made one change to his side that defeated Hull in their last league outing, with Oliver Norwood replacing the injured Sander Berge.
Wayne Rooney remained with the same line-up which saw his Derby side defeat Stoke last weekend.
The first chance of the game fell to the Blades when John Fleck picked the ball up at the edge of the area and got a long-range shot off which Roos saved comfortably.
Sharp found himself through on goal after George Baldock played a ball over the Rams defence, but his effort went just wide of the right post.
Derby went close as Phil Jagielka met Louie Sibley's corner with a looping header back across goal which rattled the crossbar. Tom Lawrence's follow-up effort was saved and eventually cleared.
Baldock found Sharp once again with a well-executed cross, but his headed effort went wide.
A cross from Lee Buchanan caused the United defence problems as a knock down from Sibley set up Lawrence whose volley was blocked. The ball then fell to Sam Baldock on his left foot and his volleyed attempt skimmed past the right post.
Graeme Shinnie troubled Robin Olsen with a powerful shot outside of the Blades box, but the goalkeeper did well to stop the effort.
In the 56th minute, Derby goalkeeper Roos was given a straight red card. An excellent through ball from Norwood played in Sharp who was left one on one with the outcoming keeper, with Roos taking out Sharp on the edge of the area to give United a free-kick.
Morgan Gibbs-White struck the set-piece into the Rams wall.
Jokanovic made his first change in the 65th with Oli McBurnie replacing Norwood.
The Blades continued to create chances as Gibbs-White's spectacular scissor kick went narrowly wide of the right post.
Gibbs-White whipped in a pinpoint cross from the right flank which found Iliman Ndiaye in the penalty area and his header went wide.
George Baldock drilled in a low cross from the right which met Enda Stevens at the other side of the box, but the substitute's effort blazed over the bar.
In the 87th minute the Blades were awarded a penalty after Curtis Davies handled an incoming cross from Conor Hourihane and Sharp converted the penalty.
Lawrence rattled the near post with his shot from the left side of the box deep into stoppage time.