Human rights groups have called the move a flagrant violation of international law.
"Too much mixing causes problems," Viktor Orban told crowds.
The turnout of the vote appears to make the vote invalid.
The government backed away from the idea Thursday following an outcry from Hungarian media.
The Hungarian leader has been vocally opposed to taking in refugees.