In early February, the American Association for Public Opinion Research (AAPOR) censured Dr. Gilbert Burnham, a faculty member at the Johns Hopkins Bloomberg School of Public Health, for violating the AAPOR ethical code for failing to disclose "essential facts about his research," a study (pdf) of civilian deaths in Iraq originally published in the journal Lancet.
When I blogged the story, one reader asked for more specifics about what exactly Burnham had failed to disclose. The response from AAPOR's standards chair, Mary Losch was a somewhat vague summary: "Included in our request were full sampling information, full protocols regarding household selection, and full case dispositions -- Dr. Burnham explicitly refused to provide that information for review."
On Tuesday, AAPOR's executive committee issued a statement (pdf) with more specifics on what they requested and how Burnham responded:
As part of the investigation, the AAPOR Standards Chair requested information from Dr. Burnham. The specific requests related to AAPOR’s finding of violation of minimum disclosure were as follows:
1. The survey sponsor(s) and sources of funding for the survey.
2. A copy of the original questionnaire or survey script used in the 2006 survey, in all languages into which it was translated.
3. The consent statement or explanation of the survey purpose.
4. A full description of the sample selection process, including any written instructions or materials from interviewer training about sample selection procedures.
5. A summary of the disposition of all sample cases.
6. How were streets selected? How were the starting street, and the starting household, selected? Once the starting point was selected, how were interviewers instructed to proceed (e.g., when they came to an intersection)? How were houses and respondents chosen at housing units?
7. The survey description says that, “The interview team were given the responsibility and authority to change to an alternate location if they perceived the level of insecurity or risk to be unacceptable.” In how many clusters did the team change location, and what were the reasons for the changes?
8. The survey description says that, “Empty houses or those that refused to participate were passed over until 40 households had been interviewed in all locations.” Were such cases included in the number of not-at-home and refusal cases counted in each cluster?
Dr. Burnham responded with the following information related to the detailed request:
• “This study was carried out using standard demographic and household survey methods.”
• “The methods we employed for this study were set out in the Lancet paper reporting our findings (Lancet, 2006;368:1421-28). The dataset from the study was released some time ago.”
Despite repeated requests from the AAPOR Standards Chair for the information detailed above, Dr. Burnham refused to provide any additional information. He did not indicate that the information was unavailable, nor did he suggest that disclosure of this information would risk revealing the identities of survey participants.
Keep in mind that AAPOR asked Burnham to disclose these details to their standards committee as a part of a confidential inquiry. They were not asking him to make these details public, at least not at that stage of their investigation. They have not provided information on the nature of the original complaint made by an AAPOR member, which may have involved aspects of the research other than disclosure. Either way, the AAPOR code is very clear about a researchers obligation to disclose such details, on request. Failure to disclose is grounds for censure.
Mary Losch will present an overview of AAPOR's code and the Burnham case and will be available for questions today at 3:00 p.m. at an event sponsored by AAPOR's DC Chapter (more details at DC-aapor.org).
Interests disclosed: I am an AAPOR member and served on AAPOR's Executive Council for two years, from May 2006 to May 2008, but was not involved in the Standards Committee's investigation of the Lancet study.
Follow Mark Blumenthal on Twitter: www.twitter.com/MysteryPollster