iPhone app iPad app Android phone app Android tablet app More

Featuring fresh takes and real-time analysis from HuffPost's signature lineup of contributors
Allen Frances

GET UPDATES FROM Allen Frances
 

DSM-5 Field Trials Discredit the American Psychiatric Association

Posted: 10/31/2012 8:51 am

The $3 million DSM-5 Field Trials have been a pure disaster from start to finish. First, there was the poor choice of design. The study restricted itself to reliability -- the measurement of diagnostic agreement among different raters. Unaccountably, it failed to address two much more crucial questions -- DSM-5's potential impact on who would be diagnosed and on how much its dramatic lowering of diagnostic thresholds would increase the rates of mental disorder in the general population. There was no possible excuse for not asking these simple-to-answer and vitally important questions. We have a right to know how much DSM-5 will contribute to the already rampant diagnostic inflation in psychiatry, especially since this risks even greater overuse of psychotropic drugs.

Second problem -- the design of the DSM-5 field trial had a byzantine complexity that could be dreamed up only by people with no experience in real-life field testing. One look made clear that there would be serious implementation problems and that it would be impossible to complete within the time allotted. The first stage of the field trial limped in eighteen months late, having taken twice as long as was scheduled. APA then had to choose between delaying the publication of DSM-5 or canceling its planned second stage of field testing that was meant to provide for desperately needed quality control. APA decided to cancel the trial and instead is rushing ahead with the premature publication of DSM-5 next May -- publishing profits clearly trumped concern for the quality and integrity of the product. Fiduciary responsibility was thrown out the window.

Now, we have strike three. The DSM-5 leadership has reported the results of its field trial in a distressingly misleading paper.

According to the authors, 14 of the 23 disorders had "very good" or "good" reliability; six had questionable, but "acceptable" levels; and just three had "unacceptable" rates. Sounds okay until you look at the actual data and discover that the cheerful words used by the DSM-5 leaders simply don't fit their extremely disappointing results. The paper is a classic example of Orwellian "newspeak." When DSM-5 failed to achieve acceptable reliability by historical standards, the DSM-5 leadership arbitrarily decided to move the goal posts in and lower the bar in defining what is "acceptable." In fact, only the five of the 23 DSM-5 diagnoses that achieved kappa levels of agreement between 0.60-0.79 would have been considered "good" in the past. DSM-5 cheapens the coinage of reliability by hyping these merely okay levels as "very good." Then it gets much worse. The nine DSM-5 disorders in the kappa range of 0.40-0.59 previously would have been considered just plain poor, but DSM-5 puffs these up as "good." Then DSM-5 has the chutzpah to call acceptable the six disorders that achieved lousy, absolutely unacceptable reliabilities with kappas of 0.20-0.39. DSM-5 finally finds unacceptable the three diagnoses that were below <0.20 (which is barely better than chance).

Major Depressive Disorder and Generalized Anxiety Disorder were among those that achieved the unacceptable kappas in 0.20-0.39 range. This makes sense for GAD because its DSM-5 definition was so very poorly done. But how to explain the ridiculously low levels of agreement for MDD. DSM-5 had made no changes from the MDD definition whose reliability has been studied hundreds of times in the past 30 years and has always achieved rates about twice as high. The only possible explanation for the egregiously poor MDD result is amateur incompetence in how the DSM-5 field trials were conducted -- and this throws in doubt all of the other results (and all of DSM-5).

It is sad that the American Journal of Psychiatry agreed to publish this sleight of hand interpretation of the remarkably poor DSM-5 field trial results. Clearly, AJP has been forced into the role of a cheerleading house organ, not an independent scientific journal. AJP is promoting APA product instead of critically evaluating it. Scientific journals all have some inherent conflicts of interest -- but this is ridiculous.

The DSM-5 field trial fiasco and its attempted cover-up is more proof (if any were needed) that APA has lost its competence and credibility as custodian for DSM. A diagnostic system that affects so many crucial decisions in our society cannot be left to a small professional association whose work is profit driven, lacking in scientific integrity, and insensitive to public weal.

Allen Frances is a professor emeritus at Duke University and was the chairman of the DSM-IV task force.

 
 
 
FOLLOW SCIENCE
 
 
  • Comments
  • 90
  • Pending Comments
  • 0
  • View FAQ
Comments are closed for this entry
View All
Favorites
Bloggers
Recency  | 
Popularity
Page: 1 2 3  Next ›  Last »  (3 total)
photo
HUFFPOST SUPER USER
Jack Davies
THEY OWN BOTH SIDES!
06:21 PM on 12/11/2012
Not to mention the secrecy with which they worked!
03:59 PM on 12/04/2012
The revision of the DSM was part funded by the US government DHHS under the GW Bush presidency due to it being a key international health document. Unfortunately the Obama team have never gotten to grips with what is set to be a disaster. They should, and could step in to prevent publication in the present form.
10:03 AM on 11/23/2012
Old definition: BMI > 28 (men), BMI > 27 (women)
People under old definition: 70.6 million
New definition: BMI > 25
People added under new definition: 30.5 million
Percent Increase: 43%

The definition was changed in 1998 by U.S. National Heart, Lung and Blood Institute.

"The new definitions ultimately label 75 percent of the adult U.S. population as diseased," conclude the two researchers..
10:03 AM on 11/23/2012
High blood pressure is reported as two numbers, systolic or peak pressure and diastolic pressure when heart is at rest) in mm Hg.

Old Definition: cutoff Blood Pressure > 160/100
People under old definition: 38.7 million
New Definition: Blood Pressure > 140/90
People added under new definition: 13.5 million
Percent Increase: 35%

The definition was changed in 1997 by U.S. Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure.

Prehypertension, a new category created in 2003: blood pressure from 120/80 to 138/89 includes 45 million additional people! If one includes this category, we have a grand total of 97.2 million total numbers of hypertensives and prehypertensives (whatever that is).

High (Total) Cholesterol:

Old Definition: Cholesterol > 240 mg/dl total cholesterol
People under old definition: 49.5 million
New Definition: Cholesterol > 200 mg/dl total cholesterol
People added under new definition: 42.6 million
Percent increase: 86%

The definition was changed in 1998 by U.S. Air Force/Texas Coronary Atherosclerosis Prevention Study.

Overweight:

Body Mass Index (BMI) is defined as the ratio of weight (in kg) to height (in meters) squared and is an inexact measure of body fat, though it supposedly establishes cutoff points of normal weight, overweight, and obesity.
photo
HUFFPOST SUPER USER
Kevin Bradley
Retired programmer and Mac user.
03:15 PM on 11/12/2012
"We all go a little crazy sometimes." -- Norman Bates
02:08 PM on 11/10/2012
Now I know why the field of psychology was not invited to the table when developing the DSM5, as we are sticklers for empirically supported diagnoses.
This user has chosen to opt out of the Badges program
photo
CateManhattan
Common sense is way too uncommon.
02:42 PM on 11/12/2012
I can't disagree more. You use DSM? You believe DSM diagnoses are empirically supported?

Both psychology and psychiatry have gone along with DSM machinations. In many programs, but perhaps not yours, students are encouraged to use vague dx. I've have clinical psych students tell me, "Diagnosis? It doesn't matter. They will decide to be a patient, or not. That's all that matters."
This user has chosen to opt out of the Badges program
photo
CateManhattan
Common sense is way too uncommon.
02:51 PM on 11/12/2012
And psych was not invited to DSM because psychiatry and psychology are guild fighting for money/privileges. Psychiatry has a distrust of psychology -- they perceive psychologist dump the most difficult patients on them, whilst harvesting the easiest and most pleasant patients for their own practice. I've observed that happening . . . but psychiatrists have their own set of uncooperative behaviors also . .
09:47 PM on 11/08/2012
Unfortunately, psychiatry is where medicine was when it was practiced with leeches.
photo
Alyn Stanton
Those who fail to learn from history are doomed. .
07:42 PM on 11/05/2012
All human behavior falls somewhere on a continuum. At one extreme of the continuum there is "insanity" (uncontrollable aberrant behavior) and at the other extreme is "unsanity" (controllable aberrant behavior). Society, or in this case, a sanctioned governing body, determines where in the middle of this continuum "sanity" resides. This definition of sanity varies and is dictated by societal mores. For example, in ancient Roman times, it was accepted behavior to appear naked in public. The same behavior will currently land one in jail or a psychiatric ward.

Those who have suggested here that there might be "business" influences guiding the definitions of mental illness in the development of the DSM-V might not be far from the mark. After all, Big Pharma stands to make billions of dollars from the added prescriptions that a loose and easy diagnostic criteria might bring.
10:46 AM on 11/08/2012
Can you cite a passage from the DSM which contains the word "sanity", please?

Oooops...
photo
Alyn Stanton
Those who fail to learn from history are doomed. .
11:56 AM on 11/09/2012
No, because it's not in there.  Did I say that it was?  I don't think so.  I was just espousing my theory of human behavior and societal sanctioning. 

Oooops yourself...
This user has chosen to opt out of the Badges program
06:43 PM on 11/05/2012
Just a reminder: if you don't have the slightest bit of experience with dx codes please, don't play doctor.
10:48 AM on 11/08/2012
It doesn't require any knowledge of dx codes to know who on this page lost it more than just a bit.

Humans are very good at assessing the mental performance of their own, even without being formally educated.

:-)
This user has chosen to opt out of the Badges program
06:37 PM on 11/05/2012
they need to reassign the board and set aside a quarter to revise what they are doing and why
photo
HUFFPOST SUPER USER
roger stillick
Forward for Everyone
01:44 PM on 11/05/2012
So the Chairman of the DSM4 task force is unwilling to allow DSM5 to exist because changes
were made to his Perfect diagnostic tool ?? OH BOY, SO SAD...
As a user of DSM4 and have studied the changes in DSM5, do not see any problems...
however the old guy writing this is just unwilling to retire DSM4...IMHO.
10:50 AM on 11/08/2012
He has been beating this dead horse for years now. The only thing/person that/who looks bad about the whole affair is him.

And the only people who agree with him around here happen to be the same "haters" of big pharma who are, at least, borderline paranoid.

:-)
HUFFPOST SUPER USER
shanec90
Thought.
12:11 PM on 11/05/2012
People take the DSM seriously? Whoa.

Wasn't it only 40 years ago that "Homosexuality" was listed? Like, right below "Cannabis abuse"--as psychiatric disorders!

I thought the DSM was how medical professionals and law enforcement agencies justified taking away people's rights by 'diagnosing' them with one of the 'disorders' in the book.

For an interesting counterexample to the entire practice of 'diagnostic' psychology, I have an interested anecdotal example, albeit a religious one.

I recently ran into a review of a book called "The Veil"--a skinny tome by a first-time author whose church paid for its publication. And, of course, he doesn't talk any psychology--the synopsis begins "Ever since I can remember, I've seen angels, demons; all sorts of spiritual things." Before I get DSMed into oblivion, bear with me but a moment more.

What the author describes amounts, to a Western mind, to schizophrenia. Hallucinations, strange fears, social ineptitude. Still a young boy, he takes his troubles to his parents, and they to their pastor. Instead of diagnosing him, they proclaim him possessed of a spiritual gift. Today, he is happy and healthy, socially just fine, and his hallucinatory episodes have all but disappeared. I DO NOT NECESSARILY BUY HIS STORY. But it is a powerful testament to the narrow-mindedness of medical professionals and the ineffectual nature of Diagnostics and Statistics in a field still so much comprised of guesswork as psychology.
02:59 AM on 11/07/2012
If you read the criteria for diagnosing mental illness, it would need to actually impede his life in some major way, or cause him to harm himself or others in order for a diagnosis to even exist, and this, only after seeing a mental health professional. It's not like someone can just pick up a DSM IV or V and just throw it at a person and sees if it sticks.
03:23 AM on 12/04/2012
In the UK Psychology is one of the most popular subjects studied at schools for university entrance-level examinations (universities typically require very good passes in 3 or 4 such subjects for entry). The rubbish largely taught and tested upon is significantly more harmful than in more usual subjects since, despite the teaching being crude, students feel qualified to judge other people based upon it, and some students feel alienated from their fellows because of it.

The curriculum, for example, includes Freudian theory, and development theory under which sex and gender minorities are seen as having failed developmental stages and be curable by therapy.

Of course, university teaching may rarely be actually any better.
02:23 AM on 11/05/2012
this is scary because the new DSM is becoming just a tool to control people. once you are "diagnosed", you are branded for life. and these ridiculous standards are being applied from these studies. the DSM is THE diagnostic tool used by anyone in the field. I so regret wasting my time in college getting a bachelor's in psychology. It's frightening how people in the field can wield so much power over others, and more now that they will be using such a poor diagnostic tool.
HUFFPOST SUPER USER
Ortist
11:30 AM on 11/04/2012
Poorly written article, when I had to go to the comments to get a translation of your acronyms.
This user has chosen to opt out of the Badges program
12:38 AM on 11/04/2012
When I first studied psychology in college (1969) There was the DSM (no number behind it) and it was a very thin (maybe 3/8 of an inch) book. After forty+ years in the field, I don't believe there are any more
"disordered" people around now than then, just more "diagnosis" to fit more people into. They used to just be called "odd".
02:53 AM on 11/05/2012
Dear Kalel: Perhaps you went to Krypton U., & DSM-II hadn't reached there yet. It was published here on Earth in 1968.
This user has chosen to opt out of the Badges program
12:42 AM on 11/09/2012
In the 60's and 70's and 80's many colleges throughout the country did not always use the latest texts. In 1969 we did not use the DSM-II.