Siri might be going to summer school.
Piper Jaffray analyst and noted Apple fanatic Gene Munster decided to pit Siri against Google's web search in a supreme battle of the popular search interfaces. Munster asked Siri and Google 800 identical questions -- inputting the queries via text in Google and by voice in Siri -- and then recorded how many of the questions Siri and Google could answer correctly.
Munster also tested Siri's hearing and the iPhone 4S's noise-canceling capabilities, asking it 800 questions indoors, in a quiet room, and 800 questions outdoors, at a busy intersection, through the microphone of a set of Skullcandy headphones.
The key takeaway from Munster's experiment? Siri is still very, very much in beta.
Indoors, Siri was able to comprehend the words he spoke 89 percent of the time; at the busy intersection, that percentage fell to 83 (though perhaps this is more an indictment of the Skullcandy headphones). Of the 800 questions asked, Siri returned a correct answer just 62 percent of the time. Google, meanwhile, returned a correct answer 86 percent of the time (and, obviously, understood 100 percent of the queries, as the words were typed into a search box).
Long-time Apple watcher Philip Elmer-DeWitt has a bunch of the 800 questions that Siri was unable to answer correctly:
Where is Elvis buried? Responded I can't answer that for you. Siri thought the persons name was Elvis Buried.
Where am I? Pin dropped in the wrong place.
When did the movie Cinderella come out? Responded with a movie theater search on Yelp.
What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu.
Channeling his inner grade-school teacher, Munster awarded Google an A+ for comprehension and a B+ for accuracy, while poor Siri got a B for comprehension and a D for accuracy. (Side note: Since when does 62/100 count as a D? I wish Gene Munster had taught some of my college science courses!)
Anyway, Munster -- despite the harsh letter grades -- sees big things in Siri's future, writing that the technology should "improve meaningfully" following the promised updates coming in iOS 6 this fall. Indeed, the new capabilities announced at this year's Worldwide Developers Conference seem to offer appreciably improved functionality for Siri.
That the current incarnation of Siri is not very accurate, speedy or reliable is not breaking news: Apple has been criticized from all sides for the "broken promises" of its beta voice assistant, and Tim Cook has promised Siri will continue to get better.
Despite the criticism, Siri remains a very popular feature among users and is one of the key differentiators and advertising boons for Apple over Google's Android and Microsoft's Windows Phone.
In other words: Siri might not be the smartest kid in class, but it's definitely one of the most popular. Here's hoping that wisdom will come with age.