The Bantams took the lead after just 15 minutes thanks to Garry Thompson’s superb volley and both teams created further chances but were unable to take advantage.
The defense almost managed to keep its Premier League opponent at bay throughout the second half, but Vermaelen scored in the 87th minute to force extra time.
Extra time remained goalless and penalties were required to separate these teams, with Bradford prevailing thanks to three misses by the Gunners.
The home side selected its strongest possible team for this fourth round tie, with Gary Jones captaining the Bantams, who currently sit fourth in the League Two table.
Meanwhile, Arsene Wenger selected a strong side, with Jack Wilshere handed a start in the midfield and Gervinho leading the line up front.
The home side took the lead in the 15th minute with their first real chance, as after a free kick caused havoc in the Arsenal box, Thompson volleyed home to give them an early advantage.
Bradford had the momentum after taking a surprise lead, while the away side struggled to penetrate their opponents’ defence.
However, the home side came close to an equalizer in the 37th minute, as after some superb trickery, Francis Coquelin struck an early shot against the post that bounced away to safety.
Gervinho then could not get on the end of a fizzing cross from yards out, then Bradford almost scored a late second as Nahki Wells sent a shot just wide after a James Hanson knock-down.
The Bantams’ confidence was clear and they went into the halftime break ahead after a superb opening period.
It took until the 55th minute for either side to test its opponent’s defense, but Bradford was able to snuff out a long ball played towards Gervinho that he could not get under control.
However, the away side began to increase the pressure as the second half went on, although the best home defense in all of League Two this season managed to hold firm.
It took until the 70th minute for the Gunners to create another real chance as they had their first effort on target through a long-range strike from substitute Alex Oxlade-Chamberlain.
It took the Gunners until the 87th minute to find an equalizer and it came thanks to Vermaelen, who headed home from a corner to draw his side level.
The game then went into extra time, with the best chance falling to Santi Cazorla, who hit the bar early in the second period.
However, with neither side able to find another goal, a penalty shootout was to decide which team would go through to the semifinals.
In the shootout, Arsenal hit the woodwork twice while the home side just about held its nerve to take the win and go through to the semis.
Follow @GoalUSA on Twitter.