* Referendum was initiated by right-wing SVP party
* Swiss business says vote creates toxic uncertainty
* Large neighbour Germany sees "considerable problems" (Adds Irish foreign minister)
By Katharina Bart and Barbara Lewis
ZURICH/BRUSSELS, Feb 10 (Reuters) - Switzerland could lose its privileged access to the European single market, European officials said on Monday, after Swiss voters narrowly backed proposals to curtail immigration from the EU in a referendum that has unsettled business.
The vote on Sunday was initiated by the right-wing Swiss People's Party (SVP), which has tapped into concerns that Swiss culture is being eroded by foreigners, who account for nearly a quarter of the country's 8-million strong population.
Immigration limits were vigorously opposed by Swiss industry and the government in Berne, which is now in the uncomfortable position of having to write the referendum result into law while limiting the backlash from Brussels and big neighbours like Germany and France.
"Switzerland has rather damaged itself with this result," German Foreign Minister Frank-Walter Steinmeier told reporters on arrival in Brussels for a meeting with his EU colleagues. "Switzerland must realise that cherry picking with the EU is not a long-term strategy."
"There will be consequences, that's clear," said Luxembourg Foreign Minister Jean Asselborn. "You can't have privileged access to the European internal market and on the other hand, dilute free circulation."
Irish Foreign Minister Eamon Gilmore called the result "very disturbing".
"I think we have seen throughout Europe a growth in what I can only call an extreme-right agenda which is quite xenophobic," he told reporters.
Free movement of people and jobs within its borders is one of the fundamental policies of the European Union, and Switzerland, while not a member of the 28-nation bloc, has participated under a pact with Brussels.
Since 2002, Swiss and EU citizens have been able to cross the border freely and work on either side as long as they have a contract or are self employed.
EU officials said the free movement treaty is part of a package of seven agreements that stand or fall together. The accords also cover economic and technological cooperation, public procurement, mutual acceptance of diplomas and licences, agricultural trade, aviation, and road and rail traffic.
"We simply can't accept these kinds of restrictions, the ones that were approved yesterday," said European Commission spokeswoman Pia Ahrenkilde. "This will clearly have implications for the rest of the agreements we have with Switzerland."
Businesses say the vote to reintroduce immigration quotas, backed by a margin of just 19,526 voters, threatens a Swiss economy that relies on the EU for nearly a fifth of workers.
Switzerland is home to food and beverage giant Nestle , drugmakers Novartis and Roche, as well as a host of major commodities dealers such as Glencore Xtrata and Louis Dreyfus Commodities.
Valentin Vogt, president of the Swiss Employers Association, told the NZZ newspaper that the vote created a toxic uncertainty for Swiss business, already under pressure from a crackdown on bank secrecy and outcry over favorable tax rates some Swiss cantons offer multinationals.
"What's the point of investing in Switzerland, when in the end it's not certain whether you can get qualified staff to carry out your plans?" Vogt asked.
Swiss banks including UBS and Credit Suisse are especially dependent on the flow of foreign workers, employing up to 25 percent of staff from the EU.
Sindy Schmiegel of the Swiss Banking Association said the vote could reduce the pool of available workers for the industry making it "much more difficult" to meet staffing needs.
Real estate experts Jones Lang LaSalle said immigration curbs could also hit Swiss housing prices. They estimated foreigners accounted for 75 percent of the increase in residential housing since 2007.
Zuercher Kantonalbank downgraded two real estate stocks, Allreal and Mobimo, to underweight from market-weight on Monday in the wake of the vote.
Under the initiative, refugees and asylum seekers would be subject to the same quotas as other immigrants.
Some see a growing intolerance towards immigrants in Switzerland, which they blame on right-wing forces they say are using deep-seated fear of foreigners among some Swiss to win votes and garner support for restrictive policies.
The SVP, which has grown rapidly since the 1980s and is now the biggest party in parliament, has made opposition to immigration a key message.
Justice Minister Simonetta Sommaruga said on Sunday that the government planned to draft a law by the autumn.
It will have some flexibility as the referendum did not set specific quotas. But it will be walking a tightrope between respecting the vote and avoiding a backlash from Europe, where leaders are already worried about a surge in support for anti-immigration parties in European Parliament elections in May.
One option, according to University of St. Gallen political scientist Patrick Emmenegger, would be to tighten requirements on how quickly immigrants can bring their families to Switzerland, though he said it was unclear whether the EU would accept this.
Anger among parties that had opposed the vote was palpable on Monday, with the Swiss Liberal Democrats suggesting that Christoph Blocher, the billionaire industrialist and SVP lawmaker who poured his own money into the campaign, be sent to Brussels himself to explain the vote.
"He has an obligation to find a good solution, together with the other parties," the FDP said in a statement. (Additional reporting by Caroline Copley, Rupert Pretterklieber, Oliver Hirt in Zurich, Adrian Croft, John O'Donnell and Justyna Pawlak in Brussels, Michelle Martin and Stephen Brown in Berlin; Writing by Noah Barkin; Editing by Peter Graff and Giles Elgood)