iOS app Android app More

Featuring fresh takes and real-time analysis from HuffPost's signature lineup of contributors
Michael Ham

GET UPDATES FROM Michael Ham
 

Negative Irreproducible Tweets... for Science

Posted: 01/05/12 05:51 PM ET

As a scientist, you must come to grips with the idea that your big fantasies aren't normal. Trust me when I say that your friends -- who get 'weekends' and go to the bar instead of staying late to run experiments -- give you strange looks when you tell them that you've always wanted to write a paper on par with Einstein's masterpiece on general relativity.

As a young scientist you dream of publishing papers in each of the most prestigious scientific journals. Nature, Science, PNAS and maybe even that Rolling Stone interview. Each paper merely a stepping-stone to the inevitable Nobel Prize that's 10... 13 years away at most if the committee has a backlog.

Generally speaking, that dream is quickly dashed the moment it comes to writing your first Big Boy/Girl paper, the one where your name is first on the author list. As first author it is your responsibility to create and support a narrative worthy of a mystery novel. Why did you do what you did, what happened when you did what you did, and what makes the thing you did worthy of letting an anonymous colleague, who may be competing with you, meticulously dissect your work and demand to know why you didn't do it the way they would have done it. Admittedly the last part isn't all that useful for a mystery novel, but it is required for publishing your data under the currently established publication model. Overall this system works reasonably well, but it takes months or years to get a paper published after submission and it creates a system where potentially useful data is lost forever.

Every scientist has at least one paper or graph tucked in a folder that lies in a dusty corner of the hard drive next to that dancing baby that used to be all the rage. The data is interesting, but doesn't lend itself to the creation of the grand narrative you must have for a traditional publication.

After many years working and publishing in this system, I've come up with a non-peer reviewed fantasy publication system with three components. Hey, I warned you that scientists don't have normal fantasies.

1. A twitter for science. This would be a place you post that orphaned graph or piece of interesting data you accumulated. To make this stand up to scrutiny, you would also need to post the raw data along with it and a blurb about what it represents. This would greatly speed the process of getting data to other researchers and ensure that data is not stuck watching that baby dance for all eternity.

2. A [Serious] Journal of Irreproducible Results -- This shouldn't be confused with the comedic Journal of Irreproducible Results, which once warned the public of the danger of eating a pickle by noting that everyone born before 1839, who had ever dined on even one pickle, is dead.

In all seriousness, one of the most frustrating aspects of experimental scientific research is finding a freakish result that you simply cannot replicate. This happens quite often when you are dealing with biological systems where considerable variation exists between experimental runs. In my fantasy publication scheme, papers submitted to this journal would contain a fully prepared manuscript where you describe the experiment, the result you got that was anomalous, and the data you collected. In reality, most of this type of data is useless, but it could help inform others who are working on similar projects and inspire new ideas.

3. A Journal of Negative Results -- There's nothing worse than watching a well-reasoned experiment produce... nothing. In a laboratory setting, there are often a myriad of factors that could have prevented your hypothesis from working. However, after fiddling with a few parameters you have to move on to the next idea and you have nothing to show for your efforts.

However, if you thought an experiment was a good idea, it's likely that others will attempt something similar in the future. Therefore, this journal's purpose is to help guide future experiment planning. By noting that your brilliant idea did not produce the expected result, others may figure out how to tweak your design or realize they shouldn't waste their time in this endeavor. Note that there is a peer-reviewed Journal of Negative Results in BioMedicine, but I believe it is imperative that there should also be a non-peer reviewed version that allows scientists to quickly post their failed results before moving to the next project.

To balance the lack of anonymous peer-review, each system should allow other researchers to leave feedback about the data. Kind of like YouTube comments with big words mixed in. As the rate of scientific progress continues to accelerate, some hybrid system along the lines that I discussed is inevitable as a way of democratically identifying data that is most valuable to the community. However, for big results that establish a grand narrative, the traditional model of publication will likely remain the best method for ensuring quality presentations. Which is great, because one day I will use this system to write that Einsteinesque masterpiece.