David McGoldrick gave Town a share of the spoils with a goal with a last kick of the game.
Freddie Sears grabbed his second goal in successive home games with a instant finish on two minutes but the Millers were level soon after with Danny Ward capitalising on in-decision in the Town backline.
Blues were made to pay for more slack defending early in the second-half when Ward again was able to convert with a volley from a throw.
It looked enough to give the Millers the win until McGoldrick found the bottom corner with a terrific strike from 25 yards.
Town made four changes to the side that went down at Newcastle last weekend, Cole Skuse recovering from a toe injury, McGoldrick coming in for his first start of the Championship season after his ankle ligament problem, Myles Kenlock in for the injured Jonas Knudsen and Teddy Bishop preferred in midfield.
Sears gave Town the perfect start on two minutes, the striker first to Tom Lawrence’s free-kick to beat Lee Camp at his near post.
The celebrations had hardly died down though before Rotherham were level, Town failed to deal with a corner and Ward hooked the ball in from four yards out.
Adam Webster, named Supporters’ Player of the Month before kick-off, cleared well from Ward as the Millers broke, Dexter Blackstock then crashing into Bart Bialkowski as they challenged for the loose ball.
McGoldrick showed a lovely touch and then pass to find Sears’ diagonal run and from the pull-back inside the box, Kenlock saw his shot blocked.
Rotherham rallied at the other end and Ward found space inside the box and threatened a second until Webster got his body in the way of the shot to block the danger.
McGoldrick headed over from Lawrence’s cross, Will Vaulks then having a shot deflected wide at the other end.
Bishop weaved his way past three defenders but dragged his shot wide on the turn from the edge of the box, the Millers then forced into a change with Dominic Ball on for the injured Greg Halford.
Kenlock showed great spring to test Camp with a header aimed at the bottom corner, Lawrence then crowded out as burst into the box.
Sears forced a corner in the last minute of the first-half, Christophe Berra going down in the box but no penalty decision, Webster first to the loose ball but he hacked his shot well wide.
The Millers were ahead two minutes into the second-half, Ward again the scorer after poor defending from Blues following Vaulks’ throw.
Grant Ward volleyed over from a corner as Town looked for an immediate response, Webster clearing the danger at the other end as Rotherham’s Ward looked to get in again.
Town threatened with Ward lashing a right footer just wide, McGoldrick then sending a left foot strike wide after a surging run.
Lawrence’s header at the back post was comfortably dealt with by Camp before Town made a double change on 65 minutes, Leon Best and Jonny Williams on for Ward and Bishop.
Lawrence’s cross was cleared as Best looked to pounce, McGoldrick then firing over after Williams’ cross had been cleared to the edge of the box.
Best was crowded out as he attacked McGoldrick’s cross, Rotherham making a second change with new signing Peter Odemwingie on for Jon Taylor.
Luke Varney replaced Kenlock as Town pressed for an equaliser, Sears almost finding a way but the goal wouldn’t come until that final strike from McGoldrick.
Town: Bialkowski, Chambers, Webster, Berra, Kenlock (Varney 83), Bishop (Williams 65), Skuse, Lawrence, Ward (Best 65), McGoldrick, Sears. Subs not used: Gerken, Emmanuel, Bru, Grant.
Rotherham: Camp, Mattock, Vaulks, Wood, Ward, Taylor (Odemwingie 79), Halford (Ball 34), Fisher, Fry, Blackstock, Newell. Subs not used: Price, Kelly, Forde, Forster-Caskey, Brown.
Att: 15, 247