Villa had to come from behind twice in the game before Charles N'Zogbia and Gabriel Agbonlahor secured victory in extra time.
City had taken the lead in the first half through Mario Balotelli after excellent work from James Milner, but the club was pegged back when Gareth Barry put through his own goal after good work from Andreas Weimann.
A 20-yard free kick from Aleksandar Kolarov put the home side ahead five minutes later but Agbonlahor equalized for a second time to take the game into extra time where N'Zogbia and Agbonlahor won the game.
Roberto Mancini named 10 new faces from the side held by Arsenal, with only Joleon Lescott retaining his place and that only after Jack Rodwell picked up an injury in the warmup.
Carlos Tevez and Balotelli both started with Denis Suarez and Adbul Razak two youngsters given the chance to shine for the first team. Villa also made changes but nowhere near as many, with Shay Given replacing Brad Guzan in goal and Gabriel Agbonlahor recalled to lead the line for the visitors.
The terrible Manchester weather made conditions tough for both sides in the opening half and chances were at a real premium, with the majority of the game being played between the two penalty areas.
In the 10th minute Villa had the best chance of the opening exchanges as Agbonlahor decided to test Costel Pantilimon with a long-range skidding effort from 25 yards.
The Romanian pushed the ball away but only as far as Andreas Weimann, he rolled the ball into the unmarked Christian Benteke however the Belgian couldn’t convert as he slid in to meet the Austrian’s pass.
City made Villa pay for that miss in the 27th minute although it was given a lot of help from the poor marking from the visitors defense.
Tevez sprayed a lovely cross field pass to Milner, and he cut inside onto his left foot before finding Balotelli unmarked inside the penalty area and he made no mistake in smashing the ball past City old-boy Shay Given.
The second half began just as the first had done with neither side dominating the game and the general play lacking that zip needed to bring the game to life.
However just before the hour Villa got itself an equalizer that sparked the game into life. Weimann was released down the right hand side and was allowed to deliver a dangerous low cross into the six yard area, there was no Villa player nearby but Pantilimon pushed the cross into Barry and the ball looped over the line.
Ironically that brought City back to life and within four minutes it was ahead again as Kolarov fired in one of his trademark free kicks.
Balotelli made an excellent run down the right channel but was brought down just outside the area, Kolarov won the rights to take the set piece and it proved the right decision as he fired past Given from 20 yards.
Villa was not finished though and in the 70th minute it was level again this time through Agbonlahor. A neat build up from the visitors saw the ball break free and the striker raced onto it and calmly fired past Costel Pantilimon from the edge of the penalty area.
Neither side was able to break the deadlock inside the 90 minutes so the game entered extra time and five minutes in Villa took the lead for the first time in the game. Costel Pantilimon should have done better with a low Agbonlahor shot, but he could only parry the ball into open space inside his penalty area and N'Zogbia was first to the ball to roll it home.
In the second half of extra time Villa confirmed its place in the fourth round as Agbonlahor scored his second goal of the game, with the ball looping up off Pantilimon and into the bottom corner of the net.
Follow @GoalUSA on Twitter.
Also on HuffPost:
The Morning Email helps you start your workday with everything you need to know: breaking news, entertainment and a dash of fun. Learn more