According to a recent editorial by Rick Danheiser, 3-5% of the procedures submitted for review by Organic Syntheses are rejected "due to the inability of the checkers to satisfactorily reproduce the results of their submitters." During the period 1982-2005 the rejection rate was about 12%.
Organic Syntheses was founded to address organic chemistry's oldest and most persistent problem - documenting and reproducing synthetic procedures. From Danheiser's editorial, it's clear that more work, and possibly a better approach, is needed.
Worse Than It Looks
A failure rate of 3-12% may sound small, but consider the pool from which Organic Syntheses draws. To even be considered for checking in the lab of an Editorial Board member, a proposal outline must first be submitted by an outside author. If the proposal meets the standards of the board, a full procedure is invited. Another round of review then takes place. Only if approved by the board will the procedure be checked.
This rigorous pre-screening process means that authors must have a great deal of confidence in a submitted procedure before it's checked. The denominator in the figure Danheiser cites most likely represents the very best of what organic synthesis can offer.
Despite this rigorous selection and review process, 3-12% of procedures have historically failed the Organic Syntheses reproducibility test. Why?
Danheiser's editorial ends with a teaser: "Part 2 of this editorial will discuss the most common causes of problems based on the experiences of the Board of Editors." Unfortunately, this follow-up doesn't appear to have been published yet.
Solving the Problem
Organic Syntheses has co-evolved over the last several decades alongside the peer-reviewed scientific publication system. But with the widespread use of the Web and other information technologies, that system is changing - fast.
Could there be a fundamentally faster and more efficient way to publish and vet synthetic organic procedures than the system we have now?