:Author: Ammar Hakim

JE0: On reproducible research
=============================

.. epigraph::

  An article about computational science in a scientific publication
  is not the scholarship itself, it is merely advertising of the
  scholarship. The actual scholarship is the complete software
  development environment and the complete set of instructions which
  generated the figures.

  -- J.B. Buckheit, and D. L. Donoho

*Reproducible* research is an essential component of scientific
discovery. Unfortunately, a lot of computational physics research is
*not* reproducible. Very often results and figures in published papers
are hard or even impossible to recreate. This makes it difficult to
compare algorithms, understand and recreate published results and
slows down the process of scientific discovery.

Maintaining a simulation journal, analogous to a laboratory notebook,
may promote reproducibility. A simulation journal can contain more
details than a published paper, provide annotated input files and code
needed to produce tables, figures and results of the calculation. The
journal will be most useful to the researcher who did the original
simulations or close colleagues and those wanting to use the
algorithms and results in their own research.

What this means in practice is that for all computations we need to
store the following information.

- For *each* simulation, the complete input needed to run the
  calculation.

- For *each* simulation, the scripts/programs and data required to
  create the results (tables, figures, etc).

- A journal entry describing initial conditions, boundary conditions
  and any special notes on running the simulations. There need not be
  one entry per simulation, but each simulation must be described in
  *some* entry.

- The results presented and described in as much detail as needed.

The journal entry should written in a manner that it can be eventually
incorporated into a publication and should be explicitly written with
reproducibility in mind.