So imagine you're Twitter. It's the middle of a presidential debate and several hundred people are tweeting every second. Suddenly, hundreds of users start searching for "binders full of women." Your algorithm picks up a search spike, but you don't know what "binders full of women" means or that it refers to politics. What do you do?
If you're Twitter, you call in the humans.
The microblogging service revealed in a post Tuesday that its realtime search results are sometimes tailored with the help of humans. Twitter data scientist Edwin Chen and engineer Alpa Jain explain that they use a "custom pool" of human datamancers to evaluate popular search terms on the fly.
Most mundane searches, it's implied in the post, are fed straight into a search algorithm without any human intervention. But when searches spike for a particular query -- say "Big Bird" or "binders full of women" -- the relevant search term is fed to a team of evaluators working at their home computers, who tell Twitter that both terms are related to politics.
That way, an ad for Sesame Street isn't served for "Big Bird," or one for Office Depot isn't shown for "binders." Instead, there may be an ad directing searchers to MittRomney.com.
The question then arises: Who exactly are these people helping make Twitter Search work? The answer, Chen and Jain write, is Amazon's Mechanical Turk -- a service that Amazon describes as "a marketplace for work that requires human intelligence." Amazon connects workers with companies that need simple digital tasks done -- workers who, according to CNET, aren't paid very much. For many "Turkers," helping make Twitter Search work is a full-time job.
Workers are employed from around the globe, according to the blog, to account for time zones, since Twitter wants to be able to evaluate realtime searches any hour of the day or night. But so far, English language tweets are the only ones getting tailored searches; Chen and Jain reveal that non-English queries are filtered out when Twitter looks for search spikes.
Digital Trend's Francis Bea points out presciently that this probably means realtime search is becoming a priority at Twitter, and not unreasonably so. Twitter's value largely comes from the fact that it can provide information instantly, giving people immediate crowdsourced coverage of cultural memes or big events, more so than Google's search engine.
Having humans help power search may seem silly at first glance -- but just wait until there's another political meme, celebrity crack-up, or Hurricane Sandy. Then you'll be thanking Twitter's search army.
Related
Before You Go
