Today we have yet another odd epilogue to story of Strategic Vision, LLC. Apparently not satisfied with their history of setting the low bar for basic disclosure about the surveys they claim to have conducted since 2004, the company is now attempting something new: Attempting to retroactively withdraw previous disclosure.
Until a few weeks ago, the content published at the company's web site, strategicvision.biz, had been automatically archived by the non-profit Internet Archive along with hundreds of thousands of other web pages. In my December column, I linked to two such pages (displaying polls conducted during 2005 and 2007**). As of today, however, if you search the Internet Archive for strategicvision.biz or try either of the links I used previously (and be forewarned: their heavily trafficked site is notoriously slow), you will encounter this error message:
Robots.txt Query Exclusion.
We're sorry, access to http://www.strategicvision.biz has been blocked by the site owner via robots.txt.
What that means is that sometime in January, someone at Strategic Vision added some code ("User-agent: ia_archiver Disallow: /") to a file on their web site that specifically blocks the Internet Archive from searching and displaying pages from their company web site. Let's be clear that Strategic Vision is well within its rights in blocking such searches, and has done nothing illegal or particularly nefarious. As explained on their Frequently Asked Questions (FAQ) page, the Internet Archive "is not interested in preserving or offering access to Web sites or other Internet documents of persons who do not want their materials in the collection," and thus provide instructions on how to "exclude any historical pages."
That said, given the swirl of accusations about Strategic Vision arising from a failure to disclose basic information about their methods, this new effort to scrub previously disclosed information from what is essentially a public library for the Internet is more than a little creepy. Combined their apparent blocking of access to strategicvision.biz to me and my colleagues at the National Journal, and we get a story of a company that keeps digging a deeper and deeper hole for itself.
By the way, all credit for spotting this latest twist in the story goes to Michael Weissman, the retired University of Illinois physics professor who previously published a "Fourier analysis" of Strategic Vision's results on FiveThirtyEight.com. His son Jonathan realized that Strategic Vision might delete their archive, and thus downloaded everything he could before it disappeared. So the archived pages live on -- undoing previous disclosure is harder than it looks.
**As of this writing we were still able to load some of the 2005 page (sporadically), and if you experience as similar result it is probably because of something gone awry at archive.org. The code in the robots.txt file on the Strategic Vision site shows that they want Internet Archive to remove stop displaying their content.