So imagine you're Twitter. It's the middle of a presidential debate and several hundred people are tweeting every second. Suddenly, hundreds of users start searching for "binders full of women." Your algorithm picks up a search spike, but you don't know what "binders full of women" means or that it refers to politics. What do you do?
If you're Twitter, you call in the humans.
The microblogging service revealed in a post Tuesday that its realtime search results are sometimes tailored with the help of humans. Twitter data scientist Edwin Chen and engineer Alpa Jain explain that they use a "custom pool" of human datamancers to evaluate popular search terms on the fly.
Most mundane searches, it's implied in the post, are fed straight into a search algorithm without any human intervention. But when searches spike for a particular query -- say "Big Bird" or "binders full of women" -- the relevant search term is fed to a team of evaluators working at their home computers, who tell Twitter that both terms are related to politics.
That way, an ad for Sesame Street isn't served for "Big Bird," or one for Office Depot isn't shown for "binders." Instead, there may be an ad directing searchers to MittRomney.com.
The question then arises: Who exactly are these people helping make Twitter Search work? The answer, Chen and Jain write, is Amazon's Mechanical Turk -- a service that Amazon describes as "a marketplace for work that requires human intelligence." Amazon connects workers with companies that need simple digital tasks done -- workers who, according to CNET, aren't paid very much. For many "Turkers," helping make Twitter Search work is a full-time job.
Workers are employed from around the globe, according to the blog, to account for time zones, since Twitter wants to be able to evaluate realtime searches any hour of the day or night. But so far, English language tweets are the only ones getting tailored searches; Chen and Jain reveal that non-English queries are filtered out when Twitter looks for search spikes.
Digital Trend's Francis Bea points out presciently that this probably means realtime search is becoming a priority at Twitter, and not unreasonably so. Twitter's value largely comes from the fact that it can provide information instantly, giving people immediate crowdsourced coverage of cultural memes or big events, more so than Google's search engine.
Having humans help power search may seem silly at first glance -- but just wait until there's another political meme, celebrity crack-up, or Hurricane Sandy. Then you'll be thanking Twitter's search army.
Related on HuffPost:
What We Eat (And It's Not Good)
Unfortunately, humanity doesn't eat so well, <a href="http://yearinreview.twitter.com/en/hottopics.html" target="_hplink">according to our tweets</a>. The top two food topics last year were the McLobster and Fried Kool-Aid, with Guinness beer not too far behind on the list. For 2012, we expect to see something about the Taco Bell's Dorito Taco.
Which States Are Going To Heaven And Which Like Beer
<a href="http://www.floatingsheep.org/2012/07/church-or-beer-americans-on-twitter.html" target="_hplink">Floatingsheep</a> geotagged about 10 million tweets (from June 22, 20120 to June 28, 2012) and collected data on posts containing the words "church" and "beer." In the graphic, seen above, the blue patches show places where there was more beer-themed tweeting, while the red spots show tweets mentioning church. Researchers found San Francisco had the most beer-related tweets, Dallas tweeted the most about church, and Los Angeles tweeted the most overall.
When Humanity Is Collectively Happy (Or Grumpy)
<a href="http://www.sciencemag.org/content/333/6051/1878.abstract" target="_hplink">A report from Cornell University</a> found that, in general, people are grumpy when they first roll out of bed, but they perk up by breakfast time. Throughout the afternoon our moods start to slump again, only to rise around quittin' time (6 p.m). Cornell researchers found these results by weighing positive verses negative tweets posted by more than 2 million people around the world. Interestingly enough, no matter the location, humanity experienced similar rises and falls in mood throughout the day. Graphic from the <a href="http://www.nytimes.com/2011/09/30/science/30twitter.html" target="_hplink"><em>New York Times</em></a>
Where The Chattiest People Live
<a href="http://www.huffingtonpost.com/2012/07/06/top-countries-on-twitter_n_1653915.html" target="_hplink">A 2012 study</a> by the Oxford Internet Institute found that out of 4.5 million Twitter posts, the U.S. created 30 percent of the world's tweets, followed by Brazil with 22 percent. The United Kingdom and Indonesia tied for third place with 6 percent each. <a href="http://www.huffingtonpost.com/2012/07/06/top-countries-on-twitter_n_1653915.html" target="_hplink">This graphic illustrates</a> which countries tweet most frequently by giving more active nations a larger portion of space on the "map."
How Much We Love Our Celebs
Humanity adores its celebrities, and Twitter is just one of many avenues available for stalking pop stars. <a href="https://twitter.com/ladygaga" target="_hplink">Lady Gaga</a> has over 27 million followers on Twitter, meaning that her clan of "little monsters" is larger than the <a href="http://www.census.gov/popfinder/?s=48" target="_hplink">entire state of Texas</a>. <a href="https://twitter.com/justinbieber" target="_hplink"> Justin Bieber</a> isn't too far behind, recently acquiring 25 million followers and a <a href="http://www.youtube.com/watch?feature=player_embedded&v=YLnrH6E9lxs" target="_hplink">dance of jubilee</a> from Twitter employees.
Where We Get Our News
<a href="http://stateofthemedia.org/2012/mobile-devices-and-news-consumption-some-good-signs-for-journalism/what-facebook-and-twitter-mean-for-news/" target="_hplink">According to a 2012 Pew study</a>, Facebook tends to provide more news from friends and family, while Twitter is more likely to provide journalistic news. And even still, both social media platforms provided less news than originally hypothesized. [Hat Tip: <a href="http://www.adamsherk.com/social-media/pew-research-state-of-news-media/" target="_hplink">Adam Sherk</a>]
How Twitter Birds Of A Feather Flock Together
This graphic shows Twitter users who shared at least three <em>New York Times</em> articles from Sept. 14 to Sept. 29 in 2011. The study, from the <a href="http://necsi.edu/research/social/nyttwitter/" target="_hplink">New England Complex Systems Institute</a>, found that while Twitter brings many users together, we typically connect with like-minded souls online. "A person who is cosmopolitan associates with others who are cosmopolitan, and a US liberal or conservative associates with others who are US liberal or conservative, creating separated social groups with those identities," <a href="http://necsi.edu/research/social/nyttwitter/" target="_hplink">said Yaneer Bar-Yam</a>, president of NECSI. [Hat Tip: <a href="http://www.stuart-hall.com/2012/02/22/new-research-challenges-assumptions-about-twitter-news-sharing-communities/" target="_hplink">Plectic Ltd</a>]
That Older Folks Need Connections Too
The University of Alabama <a href="http://www.medicaldaily.com/news/20120716/10877/mental-health-social-media-internet-facebook.htm" target="_hplink">surveyed nearly 8,000 people</a> in 2012, finding that participants over the age of 50 who used Twitter (and Facebook) were one-third less likely to develop symptoms of depression than those not using social media. So maybe it's time to get grandma a Twitter handle?
How Often We Get By With A Little Help From Our Friends
<a href="http://www.mediabistro.com/alltwitter/twitter-statistics-2012_b18914" target="_hplink">Infographic Labs</a> data in February of 2012 found that 69 percent of users say they follow certain people on Twitter because of suggestions from their friends.
When Popular Culture Is Going Viral.. Even If It Shouldn't
<a href="http://yearinreview.twitter.com/en/hottopics.html" target="_hplink">Throughout 2011</a>, Rebecca Black's hit (?) "Friday" was the most-discussed song around the Twittersphere. (Our bets are on Carly Rae Jepsen's "Call Me Maybe" for 2012!) "Thor" was also the most tweeted about film, and "Pretty Little Liars" scored the most Twitter clout for television. Within the last week, both "Batman" and "The Dark Knight Rises" have been trending in cities across the U.S.
Which Corporations Want To Join The Conversation
<a href="http://www.mediabistro.com/alltwitter/global-social-media-check-up_b25552" target="_hplink">According to data</a> from public-relations firm Burson-Marsteller, Fortune Global 100 firms are most active on Twitter, using this platform more than Facebook or YouTube, as of 2012. Maybe a bit surprisingly, this has been the trend since 2010 when Burson-Marsteller first began collecting data. Some of those companies include Exxon, Wal-Mart, GM and Honda. [Hat Tip: <a href="http://www.mediabistro.com/alltwitter/global-social-media-check-up_b25552" target="_hplink">All Twitter</a>]
How Sports Bring Tweeps Together
<a href="http://mashable.com/2012/02/06/tweets-per-second-records-twitter/" target="_hplink">According to Mashable</a>, we tweet heavily during sporting events. In the 2012 European Championship for soccer, there were 15,358 tweets sent per second, and during the last minutes of the 2012 Super Bowl, 10,245 tweets were sent per second. Some close runner-up events were Tim Tebow's overtime touchdown pass on Jan. 8, 2012, as well as two moments from the 2012 FIFA Women's World Cup.
When We 'Fail Whale' And Have Low Self-Esteem
Just like in the real world, we're always looking for more friends on Twitter. And now, if you're willing to spend a little cash, you can purchase fake followers from online services to bloat your self-esteem. <a href="http://gizmodo.com/5925773/how-to-buy-a-ton-of-twitter-followers" target="_hplink">Gizmodo reports</a> that this growing trend isn't technically an illegal Twitter activity, but "[y]ou'll feel disgusted and guilty because you just paid actual money for fake followers on a website, and, man, blech, come on."
That Women Are More Social (And Maybe Becoming More Tech-Savvy)
<a href="http://www.huffingtonpost.com/2012/07/09/women-facebook-twitter-pinterest_n_1655164.html" target="_hplink">There are more women than men</a> on Facebook, Twitter, and Pinterest, according to Digital Flash NY, which also means you might see more women entering the tech field soon. "Increasingly the people who are using these social media platforms are women," said Sarah Leary, co-founder of social networking site <a href="https://nextdoor.com/" target="_hplink">NextDoor</a>, in an interview with HuffPost. "So it's not surprising that women are increasingly playing a larger role in leading these companies or founding these companies."
What Topics Send Tweeters Into A Tizzy
Hashtags signify Twitter's popular topics, and throughout the last year there was plenty to tweet about. <a href="http://yearinreview.twitter.com/en/hottopics.html" target="_hplink">The most used hashtag of 2011</a> was #egypt, referring to that country's revolution that occurred during the Arab Spring. Charlie Sheen was also having his highly-mediated meltdown around that time, pushing that famous tag he coined, #tigerblood, to second place. Ironic or witty hashtags also made the list, like #idontunderstandwhy and #improudtosay. [Hat Tip: <a href="http://newsfeed.time.com/2011/12/06/overused-the-most-popular-twitter-hashtags-of-2011/" target="_hplink"><em>Time</em></a>]