iPhone app iPad app Android phone app Android tablet app More

Featuring fresh takes and real-time analysis from HuffPost's signature lineup of contributors
Scott Barry Kaufman, Ph.D.

GET UPDATES FROM Scott Barry Kaufman, Ph.D.
 

What Happens When the IQ Test Taker Becomes the IQ Test Constructor?

Posted: 02/ 6/2012 6:26 pm

The relationship between intelligence and creativity has long been debated and studied.

One of the hallmark tests of "general intelligence" is the Raven's Progressive Matrices Test. This test gives you a matrix of figures and you have to figure out the missing piece that completes the pattern.

Here's an example:

2012-02-06-220pxRavenMatrix.gif

There are typically 8 answer options and you choose the one correct answer.

Performance on this test is strongly related to the common factor derived from performance across a wide range of IQ test items. This suggests that the form of reasoning tapped into by this test is something that cuts across different kinds of content (at least spatial, verbal and quantitative content). Many think the kind of thinking this test measures taps into the uniquely human capacity for abstract reasoning ("fluid intelligence").

Whatever this test is really measuring, one thing is for sure: this is a test of convergent thinking. Your answer must converge with what the test maker came up with. Contrast this type of thinking with divergent thinking, in which you have to come up with problems to solve in the first place because there is no single correct answer. How does this -- more creative -- form of thinking relate to the type of thinking measured by IQ tests?

Researchers have attempted to get at the answer to that question -- reporting on average a small correlation between convergent thinking tests and divergent thinking tests. A recent study, however, has one of the most ingenious designs I've seen to date. The researchers were Saskia Jaarsveld, Thomas Lachmann and Cees van Leeuwen, from Germany, Japan and Belgium, respectively. They didn't just have people solve standard Raven's Progressive Matrices items, but they also had 511 first to sixth graders invent their own darn items!

First they gave the children the standard pattern completion test to take. Then they flipped the script and asked the children to come up with entirely new items based on the test they just took. That's right, these young test takers suddenly become the test constructors. The researchers scored the responses to this test on a few different dimensions. They measured convergent thinking by having a team of raters assess the rules the kids used to describe the relations between the different figures. Those who came up with more complex and correct relations got more points.

This is essentially the most important component of solving the traditional task. In the original Raven's test, you have to figure out the rules that describe the relationship between the different figures. In the example above, the rule is a simple succession of the shaded pieces. But more difficult items require figuring out multiple rules and sorting out the relevant from the irrelevant elements of the figures in your head. These problems get hard. The researchers also assessed divergent thinking by having a team of raters reward detailed and richly varied components.

The researchers then gave the kids a measure of creativity, in which the children were presented with an unfinished drawing and were asked to complete it. It was emphasized that there is no one way to complete the drawing. The total score on the creativity test was based on the number of associations the child made, and the originality and organization of the created ideas.

Their result? The convergent thinking aspect of the invented task was -- unsurprisingly -- related to the standard Raven's format. The divergent thinking aspect, however, was not related to the standard version. In fact, the correlation was close to 0! Not only that, but the convergent and divergent thinking aspects were unrelated to each other. These were simply separate dimensions of the task. Perhaps most interestingly, both aspects were significantly correlated with the creativity test.

What does this all mean? It means that IQ-type reasoning is only one slice of the creativity pie. The highest levels of creativity require both convergent thinking and divergent thinking. This idea has long been known in creativity research. According to the well known Geneplore model, creativity involves a cyclical process of generating ideas and then systematically working out which ideas are most fruitful and implementing them. The generation stage is thought to involve divergent thinking whereas the exploration stage is thought to involve convergent thinking.

What is so unique about this study is that it took a typical IQ test and turned it into a divergent thinking test. Their results show quite clearly that IQ tests do not measure divergent thinking -- a crucial component of creativity. If we want to assess a person's potential for creativity, innovation and imagination, we have to do better than give them an IQ test. We can't just ask them to figure out the one correct answer. We have to give them the opportunity to tell us what the problem is in the first place.

© 2012 by Scott Barry Kaufman.

Follow me on Google Plus or Twitter.

 
 
 

Follow Scott Barry Kaufman, Ph.D. on Twitter: www.twitter.com/sbkaufman

 
 
  • Comments
  • 25
  • Pending Comments
  • 0
  • View FAQ
Comments are closed for this entry
View All
Favorites
Recency  | 
Popularity
HUFFPOST SUPER USER
onionboy
Blessed are the Cheese Makers
03:43 PM on 02/13/2012
My IQ has been measure at over 130 and I can tell you right now that I'm not the smartest person I know...definitely not smarter than 95% of the people I've met (as the IQ distribution might predict). In terms of everyone I've met throughout my life, I'd put myself at just a tad over the median. In fact, if I had to pick my greatest talent for professional success, it's that I know how smart I'm not. I don't over reach. I find the experts at the right time, not too early or too late.
10:38 AM on 02/10/2012
Why am I not surprised at this finding? I went to art school, albeit just for one year a long time ago. We studied the usual, compositions of dots, cross-hatch drawing, light and shade and colors and so forth. It was all divergent thinking. You didn't just complete the drawing, you did it from scratch.

This article also reminds me of what was said by a Google employee concerning the big fuss about some of their products not being attractive or as wanted by the public as those running their ship thought they should be. Amongst many words he was quoted as saying "we don't have a Steve Jobs". This was on the heels of Jobs passing. Of course, Steve was a divergent thinker. That is why Apple has been and is a success of a different order.
07:17 PM on 02/10/2012
Apple hasn't been a success because Steve Jobs was a highly divergent thinker. It has been a success, because Steve Jobs was able to learn from his mistakes as a business man. He almost blew it up. Then they got rid of him... and then the thinking Steve Jobs emerged, the man who understood that creativity without a business model is a waste. The real Steve Jobs, in the end, was a very shrewd business man had a very good sense of design.

Sadly, the divergent thinker also seems to have killed the man... if the rumors are true that he kept treating his actually curable cancer with natural remedies, until it was medically too late, then you can see the downside to the man... he was opinionated and he didn't trust conventional medical wisdom, which, in this case, is backed by a very high rate of success...
HUFFPOST SUPER USER
onionboy
Blessed are the Cheese Makers
03:38 PM on 02/13/2012
Well put. I would only disagree with the idea that the thinking Steve Jobs "emerged" sometime after his ousting. I would say that he "returned". Steve Jobs started Apple with the idea that creativity had to be backed by technical innovation. When Apple started, however, Wozniak was a big part of that innovation. After Wozniak left, Jobs seemed to just try and do it his way, creatively. And you're right, he seemed to realize what was missing from before and made changes.

Your theory on his death reminds me of a study I read many years ago that went through some of the downsides to being highly intelligent. One of the biggest was that when mistakes were made, they were often huge because they were so unused to being wrong their whole lives that they would continue in a direction that others kept telling them was wrong.
09:06 PM on 02/10/2012
Perhaps I read the wrong articles or viewed the wrong biography programs about Jobs. The ones I've read or seen that the new management at Apple were all about making money ...and they took over his company, kicked him to the curb and ran the company into the ground. Then after they failed the new people begged him to come back, they even bought out his new product Next Computers as part of the deal.

Anyway, the company he cofounded was a success because of him in the beginning and in the end. According to the latest new, Apples is now worth more than Microsoft and Google combined.
02:20 AM on 02/11/2012
My understanding is that Steve Jobs was doing that (running the company into the ground) already for them. They just did a better job than he did after they got rid of him in that contest. Or, let's put it this way... they didn't know any better technologically and they didn't have the sense for design, either.

When Jobs started NeXT, he had basically a demonstrator that he had learned from his earlier mistakes (of wanting to make ten technology steps at once... thus the wordplay?). Apple, at that time basically a dying company, needed the technology and the innovation that NeXT provided.

NeXTSTEP is, of course, a shrewd pre-cursor of the "open source" model (with commercial forking into closed source!). Steve Jobs stopped re-inventing the wheel and took existing technologies (Mach kernel and BSD code) and simply added the functionality that these robust low level design missed to make the whole into a consumer product.

How smart is that? In my books very. But it is more business smart than technology smart. Pretty much everybody can repeat the tech steps that Apple is doing as soon as they hit the market... there is no rocket science in any of Apple's products. The packaging, however, far outclasses the competition. As long as they can keep that next step close to their vests, they will have a first in market advantage. The very day someone figures out their recipe, they will be one among many.
photo
jf12
Esta vez saldré como las otras y me escaparé.
03:01 PM on 02/08/2012
Some of the funnest games I played as a boy and young man involved seeing who could come up with the best rules. Best being defined by funnest, funnest being operationally defined as most likely to play more. For unclear reasons, other people besides me lost the ability to appreciate themselves making up games as we got older, preferring instead to play games someone else wrote the rules for.
09:01 AM on 02/08/2012
In many cases Science is evolving and not concrete. So it is the creative divergent thinkers that give new insights. In Physics conformity is the rule. I read a great article on DIY Physics on the bigthink.com http://bigthink.com/ideas/42249 Great quote “To Wertheim, this means that “humanity’s dialogue with the physical world has been hijacked by a group of experts who are trying to deny the rest of us participation in the conversation.”

The strange thing is it is from this unwillingness to allow others(divergent) in that they have taken what was originally an outsider’s(divergent) idea and misinterpreted it for 100 years. It’s time to let someone in that isn’t spoiled by the entanglement of the academic process of progression up to the understanding of their elders and only through a strict adherence.(convergent)
http://jetsrock.wordpress.com
03:46 AM on 02/09/2012
Shows that you know absolutely nothing about physics and physicists. And Margaret Wertheim doesn't even seem to know the difference between model and theory.

There are plenty of models in physics and very few theories. Can you name them? If you can, I will give you some credit. If you can't, better educate yourself first... by reading an actual book about physics, possibly written by someone with "credentials".
08:56 AM on 02/09/2012
Theory based on invalid axioms can not by definition be "Law". And if you read my blog you would see that. And if you actually read the original words that I quote you may actually learn something yourself. Strangely enough the Standard Model is defined as a theory in many cases. But as you seem to be all knowing please tell me that you believe that no current Theory could ever be wrong.
photo
oneeasyrider
E=mc2: From light you exist
10:44 PM on 02/09/2012
Good post. Thanks for the links. SJ can be a bit harsh, but you're demonstrating critical thinking...and that's a good thing. Keep posting.
07:24 PM on 02/10/2012
How is a drawing of geometric figures on a periodic table "critical thinking"?

Is it pretty? Absolutely? It is art? By all means. Is it good art? Not really.

But critical thinking? Nope. Total fail.

:-)
10:27 PM on 02/07/2012
The most creative students I ever encountered were those who did best on "convergent" questions in tests... like the ones that could be answered if someone had actually worked their way through the material. The ones who gave "divergent" answers on standard test questions turned out to be completely helpless in real world situations that actually required creativity. We (the teaching assistants) always had to help them to get their diploma... any which way we could.
photo
nypoet22
Psychology Ph.D., Civics Teacher, Songwriter
12:51 AM on 02/08/2012
how did you rate the creativity of the responses in those real-world situations? creativity is generally conceptualized as person, process or product, and our perceptions of it tend to turn out later to be inaccurate. the specific cases you mention may be the exceptions, but when we rate behavior it's important to operationally define what we mean.
04:03 AM on 02/08/2012
How did we rate the creativity? By how well and fast they solved the problem given to them in their masters thesis. If a guy finishes in less time than he is given and delivers a solution that is way better than what you could have expected him to come up with, he is good. And these folks were already given the more difficult thesis assignments because we trusted them to deliver.

In my world "inaccurate" simply means "it doesn't work". We had concrete science and engineering criteria and tests. The hardware and software that these guys built for our experiments had to work as specified.

I don't know how you rate "behavior". In science and engineering related to science experiments we let nature rate. And that means the products of your mind either pass or they fail. If they pass, you are hired, if they fail, you can look for a different profession.

It's not rocket science. And it's not a "creativity cult", either. It's simply a matter of man against intellectual problems. Some succeed on much more complicated problems than others. How you call that is completely irrelevant.
photo
oneeasyrider
E=mc2: From light you exist
09:36 PM on 02/09/2012
What you describe: Teaching minimal physics competence to solve real world problems...some are better than and others and I agree, it's a requirement. The world requires many competent problems solvers and there's no reason to reinvent the wheel. Convergent testing ensures basic understanding. But your missing something. Beyond competent problem solvers, assuming all physicists you deem, "he is good," who among them is going to look outside the box of conformity and change the world? Who's going to shatter the paradigm?

While most you describe will be among the best technical problems solvers in the world, will any of them stand up to you or anyone else and create something or produce any novel concept the world will be talking about in a hundred years? See, this is the concept I always try to communicate to you, to no avail.

One author suggests: Stop asking how...instead ask, what's it doing? Here, the example is convergent thinking v. divergent thinking. You know Einstein wasn't a convergent thinker even as he did incorporate minimal competence as you describe with your own students.

It's often been described, Einstein was an average student. You may have mocked him, too, if he had been one of your students. And yet, the man eventually produced monumental results...because, you and I should agree...he was a divergent thinker.
12:39 AM on 02/10/2012
It wasn't physics, but computer science and electronics design in this case... it was for physics experiments, though.

Now, you would be surprised however, how many people fail to develop even minimal physics competence... like understanding the meaning of the law of energy conservation.

So in your opinion it was not physics that changed the world since the times of Galileo? Then what was it? Philosophy? It was philosophers who came up with mechanics, thermodynamics, electromagnetic waves and semiconductor physics? In what universe.

In any case, I would like to see "creative people" in an intellectual contest against a smart physicists... oh, wait... there would be no contest.

I doubt I would have mocked Einstein. He would have given me a run for my money in thirty seconds flat.

And, no, Einstein wasn't a divergent thinker. He was highly convergent. He took facts that were known to everyone and that didn't seem to make any sense and he showed them to be perfectly consistent... if interpreted correctly. That's not divergent, but convergent thinking. He solved nature's equivalent of the graphical problem shown above.

You don't have to believe everything that you hear about people like Einstein in the media, you know. Why don't you read a real biography, instead?

You know what the difference is between people who use Einstein as their excuse for intellectual mediocrity and those who don't? We have actually read his papers. In German.

:-)