Town had chances at the New York Stadium, with Daryl Murphy twice denied by Adam Collin in the Rotherham goal, but Blues went down to two goals from set-pieces.
Matt Derbyshire put the home side in front just before the break, heading in from close range and former Town loanee Conor Sammon made it two on 63 minutes to give the Millers the points.
Murphy returned to the side after recovering from a calf injury in Town’s only change from the side that drew with Wigan last weekend, Noel Hunt dropping to the bench.
The Republic of Ireland international was handed a great chance to make an instant impact after just 15 seconds, shrugging off Kari Arnason to get himself free inside the Millers’ box but Collin denied Town’s leading scorer from close range.
Luke Chambers sent a left foot effort wide as Blues started on the front foot, Murphy then firing over after Christophe Berra had flicked on from a corner.
Cole Skuse picked up the first booking of the game for a foul on Paul Green, one of three former Town loanees - along with Sammon and Frazer Richardson - in the Rotherham line up.
David McGoldrick twisted and turned away from his marker to test Collin from the edge of the box as Blues continued to look the more likely to open the scoring, Skuse seeing a long ranger fall safely for the Millers’ ‘keeper.
Derbyshire almost got on the end of Richie Smallwood’s cross in Rotherham’s first opening of the game on the half hour, while at the other end Jay Tabb connected well with a volley from 25 yards but the ball fizzed past the post.
It was Rotherham who found a way to goal though, Derbyshire heading into the net seven minutes before the break after a free-kick had been fed back into Blues’ box by Craig Morgan.
Richardson headed clear from Chambers’ whipped in cross as Town looked for an immediate response, Kevin Bru trying to release Tabb inside the Rotherham area but the Millers retained their lead at the break.
Murphy almost got on the end of Tabb’s corner as the second-half got underway, Conor Newton then firing wide from distance before Blues made a double change with Paul Anderson and Tabb replaced by Teddy Bishop and Freddie Sears.
Zeki Fryers caught Bru with a late challenge to earn himself a yellow card and Chambers’ header was brilliantly tipped over by Collin from the resultant free-kick as Town threatened an equaliser.
Sears brought a save from Collin after Tyrone Mings’ shot had fallen to his feet on the edge of the box, Murphy then seeing a header cleared off the line by Fryers.
Good work by Mings created an opening for Murphy, his shot deflected into Collin’s hands as Town began to exert pressure on the home goal.
It was the Millers who struck again though midway through the half, Sammon in the right place to force the ball past Bart Bialkowski after Ben Pringle’s corner wasn’t dealt with by Blues.
Buoyed by the two goal advantage, Rotherham carried a greater attacking threat and it needed a timely block from Mings to keep out Green’s volley from Pringle’s cross.
Town made a final change for the last 15 minutes, Noel Hunt on for McGoldrick as Mick McCarthy shuffled his pack in attack.
Pringle fired well over the bar as Steve Evans’ side looked to wrap up the points, Hunt trying to find Sears with an exchange of passes inside the home box at the other end.
Rotherham should have made it three in added on time, Jordan Bowery going clear but Bialkowski did enough to keep his sidefoot finish out of the net. The Millers had done enough to take the points though.
Rotherham: Collin, Richardson, Arnason, Morgan, Fryers, Newton (Hammill 88), Green, Smallwood, Pringle, Sammon (Ward 74), Derbyshire (Bowery 84). Subs not used: Thompson, Broadfoot, Milsom, Frecklington. Booked: Fryers.
Town: Bialkowski, Chambers, Smith, Berra, Mings, Skuse, Tabb (Sears 51), Anderson (Bishop 51), Bru, Murphy, McGoldrick (N.Hunt 74). Subs not used: Kenny, Parr, Clarke, S.Hunt. Booked: Skuse.