Henk Moed had a look at the methodologies underlying the often-quoted Open Access (OA) impact studies available so far: “Does open access publishing increase citation or download rates?” (in Research Trends, 28, May 2012). He points out two limitations:

  1. “based on citation analyses carried out in a citation index with a selective coverage of the good, international journals in the fields” only;
  2. “not all publication archives provide reliable download statistics” and “results are not always directly comparable across archives.”

They “limit the degree to which outcomes from case studies can be generalized and provide a simple, unambiguous answer to the question whether Open Access does – or does not – lead to higher citation or download rates.”

Quick update
A related but more lengthy and scholarly article is “Scientific Utopia: I. Opening scientific communication” by Brian A. Nosek and Yoav Bar-Anan (in ArXiv.org):

Existing norms for scientific communication are rooted in anachronistic practices of bygone eras, making them needlessly inefficient. We outline a path that moves away from the existing model of scientific communication to improve the efficiency in meeting the purpose of public science – knowledge accumulation. We call for six changes: (1) full embrace of digital communication, (2) open access to all published research, (3) disentangling publication from evaluation, (4) breaking the “one article, one journal” model with a grading system for evaluation and diversified dissemination outlets, (5) publishing peer review, and, (6) allowing open, continuous peer review. We address conceptual and practical barriers to change, and provide examples showing how the suggested practices are being used already. The critical barriers to change are not technical or financial; they are social. While scientists guard the status quo, they also have the power to change it.