Bialkowski made several excellent second-half saves to deny Swansea a first home league win under Russell Martin, with his 90th-minute stop from Liam Walsh's free-kick the pick of the bunch.
Swansea had been held to a goalless draw by Hull on Saturday and Millwall were still waiting for their first away win this season.
So it was perhaps no surprise the first half was devoid of goalmouth action with Swansea enjoying 70 per cent possession but unable to fashion clear openings.
Millwall created the first chance of a cautious contest when George Saville's shot was blocked by Ryan Bennett after a spot of head tennis in the Swansea box.
Joel Piroe had a similar attempt smothered at the other end before the Dutch striker was involved in the best move of the first period.
Piroe exchanged passes with Matt Grimes and Ryan Manning but Murray Wallace reacted quickly to clear the danger.
Manning volleyed over as Swansea finally found their way into the Lions' penalty area and Jamie Paterson skewed over after Ethan Laird had muscled his way down the right.
Laird's strong running won a free-kick at the start of the second half which Grimes sent goalwards for Bialkowski to produce an acrobatic stop.
Jed Wallace and Saville fired over as Millwall summoned some attacking intent and Tom Bradshaw had their first on-target effort after controlling the ball well on his chest.
But the Wales striker's effort was straight at Ben Hamer and the Swansea goalkeeper gathered comfortably.
Swansea were seeking inspiration and it came in the form of substitute Michael Obafemi, who replaced Olivier Ntcham after 63 minutes.
Obafemi's running through the middle energised the home crowd and his link play allowed Piroe to advance and unleash a shot which Bialkowski pushed around the post.
Bialkowski was forced into another save moments later as Grimes' drive arrowed towards the bottom corner of his net.
The game was held up by a collision between Paterson and Lions defender Dan Ballard which saw both players leave the action.
Swansea were inches from taking the lead when Obafemi accepted Walsh's pass and rolled the ball wide of the upright.
Millwall's response was to slice open the home defence as Connor Mahoney and Scott Malone combined superbly down the left and Bradshaw was denied by the fast-advancing Hamer.
Swansea's last chance came when Obafemi won a free-kick right on the edge of the area and Bialkowski dived to his left to extend Millwall's unbeaten run to four games.