Answer: Its own members. For years I have been railing against conservatives for eroding confidence in our judicial system (whenever they did not agree with a decision) by the constant litany of charging judges with being "activists", "following their own agenda", "legislating from the bench", "thwarting the will of the majority" and being "soft on crime". Now it appears that several Justices themselves rather than follow the parameters of traditional and respectful dissent have chosen to attack the integrity and motives of their colleagues and have adopted the conservative litany.
When Justice Scalia does it, it surprises no one, although his recent tirades seem to drip with more vitriol than usual, but when the highly respected Chief Justice joins in, the reputation of the Court suffers greatly. The majority of the Court concluded that the many state prohibitions against same-sex marriage violated the equal rights protection of the Constitution. While encouraging those who agree or benefitted from the decision to celebrate it, he declared in an otherwise thoughtful and learned dissent, that the decision "had nothing to do with the Constitution", a gratuitous comment widely featured in the media.
Dissents by their very nature express disagreement with the reasoning of the majority opinion. But this comment strikes at the legitimacy of the ruling rather than its reasoning. The country has become more and more attuned to viewing the Supreme Court as yet another political body voting as representatives of those who "brung them to the party"----the president or party that nominated them. What greater evidence could there be than the Justices themselves and the Chief Justice in particular making the same argument.
It also encourages the conservative/Republican reaction heard from the candidates throughout the land---the most offensive being Ted Cruz, who as a Harvard Law graduate and judicial clerk to the Supreme Court should know better. He referred to the decision on The Affordable Care Act as "disgraceful", and "contrary to the justices' "judicial oath" and echoed the Chief Justice in claiming that the same-sex marriage ruling "has no connection to the U.S. Constitution." In his usual scare tactic, he concluded the ruling is a threat to American democracy. But the real threat to democracy was the Citizens United decision which conferred upon corporations the right to buy elections. Where were the Republican voices condemning the decision by "nine unelected and unaccountable lawyers in robes" and demanding that legislatures (i.e. Congress) should be deciding the issue (which they actually had). The answer is: they liked and agreed with the decision. Yes, there is nothing directly in the Constitution about same-sex marriage; nor is there anything about corporate spending in elections, but both are proper subjects for judicial scrutiny and constitutional analysis.
Underpinning all of the objections to the ruling is the contention that the Court should defer to states---"the democratic process"---to make the determination. Voters, the states, its legislatures do not decide matters of constitutional law. States may want school prayer, to bar inter-racial marriage, to suppress voters of the opposition party, to segregate schools or in a thousand myriad ways deny equal rights to others, but the Supreme Court, and not they, makes the final determination.
The parade of political objectors to the ruling all predict and warn of an encroachment on the religious beliefs of those who oppose same-sex marriage. Will persons who oppose it be forced in some way to serve those same-sex couples who have married? I will end by asking these questions: Could they and would they refuse to serve or sell to such couples if they were not married? Does marriage suddenly make them unfit? If religious grounds are a basis for refusing to do business, can persons be denied goods and services because they had an abortion, they are divorced, they committed adultery; they took the name of the Lord in vain, they failed to keep the Sabbath holy, they are an atheist, they are an inter-racial couple, they eat pork, they practice a different religion or committed any number of other unenumerated, perceived sins? If your own religion prevents you from making a cake or taking a photo of a same-sex married couple, then you must belong to a religion that doesn't believe in love and kindness. So same sex-sex marriage does not really place democracy or religion in peril, unless we decide to make the providing or withholding of goods and services dependent upon someone's religious beliefs. I worry more about the harm to democracy from corporate influence in elections than that caused by same-sex marriages.