09 September 2005

Two-thirds of meta-analyses in the critical care literature are of poor quality

More than two thirds of meta-analyses available to critical care physicians have major methodological flaws and cannot be reliably used to guide clinical practice. Research published today in Critical Care concludes that the overall quality of meta-analyses in this field is poor, but has improved over time, possibly due to the publication of the Quality of Reporting of Meta-analyses statement (QUOROM) in 1999.

Anthony Delaney from the Royal North Shore Hospital, Australia and Chip Doig, from the University of Calgary, Canada, studied 139 reports of meta-analyses addressing critical care topics published between January 1994 and December 2003. They assessed whether data from the reports met any of nine of the criteria listed in the Overview Quality Assessment Questionnaire (OQAQ), the only instrument available to researchers to grade the quality of articles. The reports were given an overall score on a scale of 1-to 7 according to the results of this assessment, with a score of 1 to 4 indicating major flaws in the meta-analysis and a score of 5 or more indicating only minimal or minor flaws.

Delaney et al.'s results show that 69% of the reports score below 4 and the average score was 3.3. The most common flaws were failure to report whether a comprehensive literature search was conducted and failure to report how bias in the inclusion of studies was avoided, with only 35% of studies fulfilling these criteria.

Before the QUOROM statement, the proportion of reports adequately reporting the validity of the studies included in the meta-analysis was 39%, but it increased to 52% after publication of the statement.

Void et al. found similar results in other areas such as emergency medicine, anaesthesia and general surgery.

Physicians view meta-analyses as a highly reliable resource. This study shows that critical care clinicians should carefully evaluate meta-analyses before considering applying the results of these studies in their clinical practice.

# # #

This press release is based on the article:
A Systematic Evaluation of the Quality of Meta-analyses in the Critical Care Literature.
Anthony Delaney, Sean M Bagshaw, Andre Ferland, Braden Manns, Kevin B Laupland, Christopher J Doig
Critical Care, 2005 9:R575-R582 (9 September 2005)
http://ccforum.com/content/9/5/R575

Please quote the journal in any story you write, and link to the article if you are writing for the web.

# # #

For further information about this study, contact Dr. Anthony Delaney by phone at +61 2 9926 86 56 or by e-mail at adelaney@med.usyd.edu.au

Alternatively, or for more information about the journal, contact Juliette Savin by e-mail at press@biomedcentral.com or by phone on +44 (0)20 7631 9931

# # #

Critical Care (http://ccforum.com/) is published by BioMed Central (http://www.biomedcentral.com), an independent online publishing house committed to providing open access to peer-reviewed biological and medical research. This commitment is based on the view that immediate free access to research and the ability to freely archive and reuse published information is essential to the rapid and efficient communication of science.

BioMed Central currently publishes over 130 journals across biology and medicine. In addition to open-access original research, BioMed Central also publishes reviews, commentaries and other non-original-research content. Depending on the policies of the individual.