« SUNY To Create First On-Line Engineering Bachelor's Degree | Main | Proceedings of the IEEE Adds 25 Years of Backfiles to IEEE Xplore »

Reproducible Research: A Bioinformatics Case Study

:: Via the Faculty of 1000, an article that may be of interest to readers. Reproducible Research: A Bioinformatics Case Study1, by Robert Gentleman, of Harvard University, suggests a new approach to scholarly publishing as follows:

While scientific research and the methodologies involved have gone through substantial technological evolution the technology involved in the publication of the results of these endeavors has remained relatively stagnant. Publication is largely done in the same manner today as it was fifty years ago. Many journals have adopted electronic formats, however, their orientation and style is little different from a printed document. The documents tend to be static and take little advantage of computational resources that might be available. Recent work, Gentleman and Temple Lang (2003), suggests a methodology and basic infrastructure that can be used to publish documents in a substantially different way. Their approach is suitable for the publication of papers whose message relies on computation. Stated quite simply, Gentleman and Temple Lang (2003) propose a paradigm where documents are mixtures of code and text. Such documents may be self-contained or they may be a component of a compendium which provides the infrastructure needed to provide access to data and supporting software. These documents, or compendiums, can be processed in a number of different ways. One transformation will be to replace the code with its output -- thereby providing the familiar, but limited, static document.

In this paper we apply these concepts to a seminal paper in bioinformatics, namely The Molecular Classification of Cancer, Golub et al (1999). The authors of that paper have generously provided data and other information that have allowed us to largely reproduce their results. Rather than reproduce this paper exactly we demonstrate that such a reproduction is possible and instead concentrate on demonstrating the usefulness of the compendium concept itself.

From the Faculty of 1000 page, the following comment on the article, by Daniel Weeks, U Pittsburgh2:

This article illustrates an important new paradigm for publishing scientific manuscripts in a radical new manner: as a compendium that is a mixture of code, data and text, all in a navigable document. This method of publication would provide dynamic documents where results and computations can be explored, reproduced, and altered on the reader's own computer. This proposed 'reproducible research' paradigm has great promise in terms of improving communication about the increasingly complex computations and statistical analyses involved in genetics and genomics papers. For the abstract of this paper, please see http://www.bepress.com/sagmb/vol4/iss1/art2/ .
  1. Gentleman R. Reproducible Research: A Bioinformatics Case Study. Stat Appl Genet Mol Biol 2005 4:article 2
  2. Daniel Weeks: Faculty of 1000, 8 Jul 2005 http://www.f1000biology.com/article/nonpub113933/evaluation
My thanks to Geoff for bringing this to STLQ's attention.