A new study published in Frontiers on Psychology claims to find links between video game playing and sexist attitudes. But a close look at the study suggests it may be a better example of how crude science with weak results is often hyped for extravagant claims. Let’s take a look.
The study, published by a collaboration of French researchers with the Iowa State group (which has a history of controversial claims exaggerating media effects), used a survey of an impressive sample of about 13,500 adolescents in France. They assessed TV viewing and video game use, both of which the authors claim contain considerable sexist content. They also looked at religiosity, as well as some basic control variables like gender, age and socioeconomic status (SES).
For sexism, they asked only a single question: “A woman is made mainly for making and raising children.” This is where things begin to break down for me. First, I’m not wild about using a single question to represent something both as important and complex as sexism. Single item outcomes can produce a wide variety of problems, as they can tend to be unreliable and we don’t know much about the reliability of this single item. Also, and perhaps it’s the translation to English, but the wording of that statement is awfully clumsy, particularly the “is made” part. A counterpart question for men might be, “A man is made mainly for making sperm to create children,” or something like that. It’s possible this item may pull somewhat from sexism, but it’s possible too that people may interpret the item as a statement about biology, given the “is made” part. Again, we know nothing about the validity of this question or whether people really understood what it was trying to ask.
The second problem is, far from the hype, the results for video games are weak. Okay, a little statistics here… the bivariate correlation (i.e., with nothing else controlled) between video game playing and “sexism” was r = 0.15… which is rather ho-hum but not uncommon for psychological research (which is often ho-hum despite a lot of flashy headlines). But once a few variables are controlled… age, gender, religiosity, SES and TV viewing… that relationship drops to .07. Given that correlation coefficients go from 0 to 1.00 (and can be negative) that outcome isn’t much different from zero. That corresponds to 0.49 percent (less than half a percent) overlapping variance, in statistical parlance.
One admittedly rough way of thinking of this is, if you had to guess which teenagers were sexist, and the only thing you knew about them was their gaming habits, your chance of being right would be about 0.49 percent better than chance alone. Those are pretty trivial findings, even if we assume they’re true. But findings that are this small often reflect “noise” in the studies… people sometimes guess at hypotheses, or extreme scores can create small but spurious results, or even the choice researchers make when running statistics can create spurious outcomes. Maybe some teens were being whimsical and gave what they thought might be offensive answers to be funny, rather than reflect what they actually thought. We don’t know. So, with a result this tiny it would have been better to give it a shrug and move on, rather than hype it.
That’s particularly true given that the relationship between TV (which the authors also said contains lots of sexist content) and sexism was non-significant in this study once other factors were controlled. So, this feels a bit like cherry picking a mixed bag of results, perhaps capitalizing on some chance findings. Presumably, the authors would have been happy to hype the relationship between TV and sexism, had results been reversed.
Interestingly though, it was religiosity that was the biggest predictor of “sexism” (with an effect size of r = 0.20, or 4 percent overlapping variance with other factors controlled… still very small, but far larger than the puny video game findings.) So, for religious conservatives eager to speak to the evils of video games, I’d hold off relying on this study.
Overall, though, this study highlights the continued problem our field has in dealing with tiny and inconsistent results. Still, we have the problem of focusing on “statistical significance” while ignoring trivial effect sizes as well as inconvenient null results that ought to give us some pause before firing off the press release.
Granted, journalists make things worse. “Gaming makes teenagers more likely to hold sexist and stereotypical sentiments about women, a new study as (sic) found” proclaimed the New York Times, now asserting causality for a study of correlational data. Authors can’t be blamed for that. And, in fairness, in one interview, the lead author does seem to have tried a bit to tamper down the enthusiasm a bit: “Begue cautioned that despite the ‘statistically significant’ link between sexism and video games, the influence of gaming on teenagers’ attitudes remains limited,” noted a Yahoo Health article. But this could have been made clearer in the study’s own abstract, for instance. And, overall, research linking video games to sexism has been a mixed bag at best, with one recent (and much better) longitudinal study finding no evidence for long term effects.
None of this takes away from the observation that there’s a real issue of the under-representation and sexualization of female characters in many video games. Improving such representations is a moral issue I support. But that isn’t an excuse for shoddy science or exaggerated claims. If we are going to learn anything from the smoking ruins of the video game violence research field it is that scholars need to proceed with caution when making claims about links between video games and behavioral outcomes.