NEW YORK (AP) — Elmo, Big Bird and the rest of the "Sesame Street" crew have always talked to kids. Now, they'll try to have a two-way conversation with their pint-sized audience using Kinect, the motion and voice-sensing controller created by Microsoft.
"Kinect Sesame Street TV," out Tuesday, is not exactly a video game, though it runs on the Xbox 360 video game system. There are no winners and losers, no real rules to follow and no points to score. If you don't want to play, that's fine. Just sit back and watch "Sesame Street," as kids have for the past 43 years. But if you do play, Grover will count coconuts you've thrown, the Count will praise you for standing still and Elmo will catch a talking ball if you throw it to him.
The episodes presage the next step in the evolution of television, adding an interactive element to what's still a passive, lean-back experience. The game is sure to arouse jealous feelings among football fans who yell at their TV sets during Sunday's game. As you watch children playing the "Sesame Street" game, it's easy to imagine a not-so-distant future where viewers become participants, affecting a show's outcome —much more than they do when they vote for "American Idol" contestants.
"Kinect Sesame Street" ''allows the child to participate in the narrative plot," says Emory Woodard, communications professor at Villanova University in Pennsylvania. Woodard, who worked at Children's Television Workshop — what is now Sesame Workshop — in 1995, notes that a lot of TV programming aimed at preschoolers involves characters talking to the kids. "But in this case," he adds, "The characters can react to the child's response."
It's not entirely clear, though, whether that makes a difference to them.
"That's a research question to explore," Woodard says.
Two-and 3-year-olds, he notes, are at a very "ego-centric" stage of development. They already experience on-screen characters as if they were talking to them, even with traditional television programs. They start growing out of that around 4, but by 5 they'll have more or less grown out of "Sesame Street," too. ("Kinect Sesame" is recommended for children 3 and up).
Microsoft recently gave The Associated Press and several bloggers an early look at "Kinect Sesame." Jack and Zoe Shyba, the 5 and 4-year-old kids of mommy blogger Jessica Shyba, at first just watched the show intently when they saw it at the Sesame Workshop in New York. The letter of the day, as it happens, was "S." But as soon the characters started to ask them to do things, they obliged.
Kinect works by first calibrating to users' bodies so it can track their movements when they throw a virtual ball to an on-screen character or pluck virtual carrots from Elmo's garden. (Kinect is also used to play more traditional video games and control entertainment features on the Xbox). With teenagers and adults, this means moving your body into various poses that are reflected by a bare-bones figure on screen.
With small children as the focus, the creators tried a different approach. Instead of having an on-screen avatar mimic their movements, "Kinect Sesame Street" has them step into an on-screen mirror so they can see themselves on TV.
"We were spending a lot of time talking about how to get a kid calibrated in a way that was fun," said Josh Atkins, Xbox executive producer of Kinect Sesame Street. The best way, it turned out, was to have an on-screen "character" that is essentially a mirror. "When they see themselves on TV they tend to pay attention."
"Sesame Street" first aired on public television in 1969, with the aim of educating preschoolers and helping them transition to school. "Kinect Sesame Street TV" starts off by using content from the show's 42nd season. The game is available in disc form and as a download through via Xbox Live. For $30 users can purchase a package that includes eight 30-minute episodes on two discs. Users can also purchase single episodes for $5.
Rosemarie Truglio, senior vice president of education and research at Sesame Workshop, said children can learn both through linear content — such as watching an episode — and through interactive content.
"We have never tried this," she said. "We've never taken a 'Sesame Street' episode and turned it into an interactive experience."
Truglio points to a math example. There's a scene in the show where Grover slips on a banana peel and drops a box of coconuts. Kids who don't want to interact with Grover can continue watching the show. Or, they can stand up and throw virtual coconuts back to Grover, who will count out the pitches until he gets six. At the Sesame Workshop, Jack and Zoe mimicked throwing the coconuts and counted with Grover until they got all of them.
"We want to get children to understand what '5' is, or the meaning of what '6' is," Truglio said. "They have to throw 6 times (so) you have an action that is linked to a number."
This kind of "transfer" — learning something in one context and translating it to another context, is a "holy grail" for Sesame Street, said Fran Blumberg, professor at Fordham University's graduate school of education who focuses on educational psychology.
"Sesame has been wildly successful," she says, "The question is whether it will be enhanced with this new game."
Jessica Shyba (who blogs about parenting in Manhattan at MommasGoneCity.com), said she prefers to see her kids play the game rather than watching regular TV.
"If they watch TV I can't even ask them what they want for lunch," she says, noting that passive television viewing turns them into "little zombies." But with the interactive game, Shyba says, "I can hear them talking, yelling, waving their arms. That, to me, is really cool."
Atkins said the making of "Kinect Sesame Street" involved a lot of real-world testing with children. It was through trial and error that the designers learned that it didn't work well when characters gave children very direct orders like "throw the ball" or "throw the ball now." The children in early tests became resistant and didn't feel involved in the story. So they changed the timing. In the game, Elmo waits for kids to throw and says "please." He's also careful not to ask too many times, because kids can get overwhelmed and lose interest.
Unlike a video game, in "Kinect Sesame Street" kids never need to do anything unless they want to. But if they do, the designers' rule was that "there needs to be an immediate response," Atkins said.
It was also crucial to only ask kids to do simple actions that are part of their everyday vocabulary. In Sesame, this includes jumping, throwing, waving, pointing, clapping, standing and perhaps the most challenging task for some of them, standing still.
"We came up with those things because we knew they were things we would never have to explain to a child," he said.
Sesame also doesn't tell kids they are wrong, or that they have to do something over. If that happened, Atkins said, "It would stop being a TV show and revert back to being a game."
Shyba's only concern is "Kinect Sesame Street" relatively short shelf life.
Navigating Kinect requires some basic skills that might be difficult for young children. Zoe, who turns 4 on Wednesday, hasn't quite mastered the setup but even she "kind of feels like 'Sesame Street' is babyish," Shyba said. "Granted, she has an older brother."
At 5, Jack is all but over "Sesame," and much more interested in another interactive TV-game, based on nature shows from National Geographic. "Kinect Nat Geo TV," another Xbox offering out this week, lets kids take the shape of on-screen animals, look for their tracks in the snow or take photos of them by saying the word "snap" when they appear.
"This kind of interactive programming is going to really be a significant part of the entertainment experience of children, teens and adults," Woodard said. "We are just at the beginning."
Earlier on HuffPost:
Sesame Street Workshop 10th Annual Benefit Gala
NEW YORK, NY - MAY 30: Jon Stewart attends the Sesame Street Workshop 10th Annual Benefit Gala at Cipriani 42nd Street on May 30, 2012 in New York City. (Photo by Andrew H. Walker/Getty Images)
FILE - This April 11, 2011 file photo, character "Rosita" from Sesame Street gestures during a taping of the show in New York. The popular children's show will hold a casting for a new Latino character on Aug. 20, 2012 in New York. (AP Photo/Richard Drew, File)
Chairman Of The Joints Chief Of Staff Gen. Martin Dempsey Speaks At The Nat'l Press Club
WASHINGTON, DC - APRIL 18: Chairman of the Joints Chief of Staff Gen. Martin Dempsey participates in a discussion on supporting Military families during an event sponsored by Sesame Street at the National Press Club, on April 18, 2012 in Washington, DC. It has been reported by the Los Angeles Times that pictures have surfaced showing military soldiers posing with body parts of dead insurgents. (Photo by Mark Wilson/Getty Images)
Grover At 2012 Consumer Electronics Show
LAS VEGAS, NV - JANUARY 10: Sesame Street muppet charachter Grover appears during a presentation by Qualcomm at the 2012 International Consumer Electronics Show at The Venetian on January 10, 2012 in Las Vegas, Nevada. CES, the world's largest annual consumer technology trade show, runs through January 13 and is expected to feature 2,700 exhibitors showing off their latest products and services to about 140,000 attendees. (Photo by Kevork Djansezian/Getty Images)
Fake Cookie Monster At Occupy Wall Street
A person dressed as the Sesame Street character 'Cookie Monster' join demonstrators with 'Occupy Wall Street' near the New York Stock Exchange as they mark the two month anniversary of the protest November 17, 2011 in New York. Some 1,000 protesters converged on Wall Street Thursday, and fights erupted outside the New York Stock Exchange amid a tense face-off with police. Demonstrators scuffled with men in business suits trying to push their way through the throngs on the way to work at the start of a day of protests in a show of force by the Occupy Wall Street movement. AFP PHOTO/Stan HONDA (Photo credit should read STAN HONDA/AFP/Getty Images)
35th Anniversary Of Sesame Street
Children mark the 35th anniversary of Sesame Street flanked by Sesame Street characters Elmo (L), Tommie (2nd R), Bert (2nd L) and Ernie (R) at the Dalton School in Amsterdam, on July 20, 2011. The Sesame Street characters signed a special giant birthday card. This cart will then tour through the Netherlands, so Sesame Street fans from around the country could sign it as well. AFP PHOTO/ANP KIPPA MARCEL ANTONISSE netherlands out - belgium out (Photo credit should read MARCEL ANTONISSE/AFP/Getty Images)
The Count and Oscar The Grouch At Packers v Jets
EAST RUTHERFORD, NJ - OCTOBER 31: Fans of the New York Jets dressed up as 'Oscar the Grouch' and 'The Count,' characters from the television show 'Sesame Street,' for the Jets game against the Green Bay Packers on October 31, 2010 at the New Meadowlands Stadium in East Rutherford, New Jersey. The Packers defeated the Jets 9 - 0. (Photo by Andrew Burton/Getty Images)
With Vice President Joe Biden and Dr. Jill Biden
NEW YORK - JUNE 02: Vice President Joe Biden (Center) and Dr. Jill Biden attend Sesame Workshop's 8th Annual Benefit Gala at Cipriani 42nd Street on June 2, 2010 in New York City. (Photo by Bryan Bedder/Getty Images)
Elmo With John Legend and Christine Teigen
NEW YORK - JUNE 02: Musician John Legend and model Christine Teigen attend Sesame Workshop's 8th Annual Benefit Gala at Cipriani 42nd Street on June 2, 2010 in New York City. (Photo by Bryan Bedder/Getty Images)
At The Martin Luther King Day Parade In Washington DC
WASHINGTON, DC - JANUARY 16: People dressed in Sesame Street and Disney costume march during a Martin Luther King Day parade January 16, 2012 in Washington, DC. The parade of local politicians, civil rights activists, marching bands and others returned to Martin Luther King Avenue Southwest after an eight year hiatus. (Photo by Brendan Smialowski/Getty Images)
Discussing Tree Planting With Katie Couric
NEW YORK - APRIL 08: Anchorwoman Katie Couric and Sesame Street Muppets (L-R) Elmo, Jesse and Rosita address an audience and the press before a special tree planting ceremony promoting the launch of the 'When Families Grieve' support group iniative at Madison Square Park on April 8, 2010 in New York City. (Photo by Michael Loccisano/Getty Images)
Street Renaming To Celebrate The 30th Anniversary Of Sesame Street Live
NEW YORK - FEBRUARY 04: Sesame Street characters Bert and Abby Cadabby attend the temporary street renaming to celebrate the 30th anniversary of Sesame Street Live on 31st Street & 8th Avenue on February 4, 2010 in New York City. (Photo by Jason Kempin/Getty Images)
A group of friends dressed as Sesame Street characters, including Victor Morales, dressed as Big Bird, right, take the 6 train early Tuesday morning Nov. 1, 2011 after attending the Village Halloween Parade in New York. (AP Photo/Tina Fineberg)
In this undated handout photo provided by Sesame Street, New Orleans Saints quarterback Drew Brees poses with Sesame Street puppet character 'Elmo' in New York, Brees is making a special appearance on Thursday, Dec. 15, 2011 episode of the educational children's TV show. He visited the set a few months ago to record the segment, which co-stars the street's furry red resident, Elmo. (AP Photo/ Sesame Street, Jesse Grant)
In this photo taken Thursday, Jan. 5, 2012, actor Rajai Sandouka, holds Kareem, the main character of Sharaa Simsim, the Palestinian version of Sesame Street, at the program's offices in West Bank city of Ramallah. The producers of the iconic children's program say they have been forced to put production for the 2012 season on hold because of a funding freeze by the U.S. Congress. Sharaa Simsim is one of many American-funded Palestinian programs suffering after Congress froze the transfer of nearly $200 million to the U.S. Agency for International Development in October. (AP Photo/Nasser Shiyoukhi)
Zoleka Mandela, Michael Bloomberg, Zindzi Mandela, Grover
In this photo provided by the Road Safety Fund, Zoleka Mandela, left, New York City Mayor Michael Bloomberg, center, and Zindzi Mandela appear with an actor in character as Sesame Street