JAMA Journal of the American Medical Association
Copyright 1996 by the American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use. American Medical Association, 515 N. State St, Chicago, IL 60610.

Volume 276(8)             28 August 1996             pp 637-639
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
[Special Communication]

Begg, Colin PhD; Cho, Mildred PhD; Eastwood, Susan ELS-D; Horton, Richard MB; Moher, David MSc; Olkin, Ingram PhD; Pitkin, Roy MD; Rennie, Drummond MD; Schulz, Kenneth F. PhD; Simel, David MD; Stroup, Donna F. PhD

From the Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY (Dr Begg); Center for Bioethics, University of Pennsylvania, Philadelphia (Dr Cho); Department of Neurological Surgery, University of California, San Francisco (Ms Eastwood); The Lancet, London, United Kingdom (Dr Horton); Departments of Medicine and Epidemiology and Community Health, University of Ottawa (Ontario) (Mr Moher); Department of Statistics, Stanford (Calif) University (Dr Olkin); Obstetrics and Gynecology, Los Angeles, Calif (Dr Pitkin); JAMA, Chicago, Ill (Dr Rennie); Centers for Disease Control and Prevention, Atlanta, Ga (Drs Schulz and Stroup); and Center for Health Services Research in Primary Care, Durham (NC) Veterans Affairs Medical Center (Dr Simel).


Browse Table of Contents

Output...


Links...
Help
Logoff
History...

Previous Page
Outline

Graphics

The randomized controlled trial (RCT), more than any other methodology, can have a powerful and immediate impact on patient care. Ideally, the report of such an evaluation needs to convey to the reader relevant information concerning the design, conduct, analysis, and generalizability of the trial. This information should provide the reader with the ability to make informed judgments regarding the internal and external validity of the trial. Accurate and complete reporting also benefits editors and reviewers in their deliberations regarding submitted manuscripts. For RCTs to ultimately benefit patients, the published report should be of the highest possible standard.

Evidence produced repeatedly over the last 30 years indicates a wide chasm between what a trial should report and what is actually published in the literature. In a review of 71 RCTs with negative results published between 1960 and 1975, the authors reported that the vast majority of them had too few patients to observe moderate or large differences. [1] Twenty years later, The Journal reported research indicating few improvements in this situation and expressed a concern about the reporting of RCTs in general. [2]

In an effort to correct these and other problems, the Standards of Reporting Trials (SORT) group met on October 7 and 8, 1993. At the conclusion of the 2-day workshop, the SORT group put forth a new proposal for the reporting of RCTs: structured reporting. [3] The proposal set out 24 essential items that needed to be included in the report of a trial, provided empirical evidence as to why the items should be included, and provided a format showing how they could be included.

Independently, approximately 5 months later (March 14 to 16, 1994), another group, the Asilomar Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature, met to discuss similar challenges facing the reporting of clinical trials. Their proposal [4] consisted of a checklist of items that should be included when reporting a clinical trial, along with a suggestion that editors add it to the Instructions for Authors.

A subsequent Editorial [5] urged both groups to meet and decide which recommendations from each group's proposal should be retained. Besides being pragmatic, this suggestion had the potential for increasing consensus, which in turn might afford a greater chance of improving the quality of reporting of clinical trials to a wider audience.

On September 20, 1995, a total of 9 members (including editors, clinical epidemiologists, and statisticians) of the SORT group and the Asilomar Working Group met in Chicago, Ill. Two other people participated in the meeting: a journal editor (R.H.) who had expressed interest in helping to improve the reporting of RCTs and one of the authors (D.S.) of a trial report that used the SORT approach. [6]

METHODS

We started the day by reviewing both the SORT and Asilomar checklists to ascertain which items covered similar content areas and which ones were unique. Those items having similar content areas we then reviewed individually. We decided, a priori, to keep only those items for which there was empirical evidence, when available, that not reporting them resulted in bias in the estimates of the effects of interventions. We used common sense for those items included for which there was no empirical evidence. The selection of items was achieved using a modified Delphi process. We also emphasized the need to keep the number of items to a minimum, while maintaining adequate standards of reporting RCTs. We used a similar approach in deciding which of the unique items should remain in the resulting checklist. The day ended with a discussion on the use of the flow diagram proposed by the SORT group and the format of a trial report. Within a week or so following the meeting, a draft report was circulated to the entire group for further refinement. This process was continued until we felt the report accurately represented what had gone on during the meeting.

RESULTS

This meeting resulted in the Consolidated Standards of Reporting Trials (CONSORT) statement-a checklist (Table 1) and a flow diagram (Figure 1). The checklist consists of 21 items that pertain mainly to the methods, results, and discussion of an RCT report and identify key pieces of information necessary to evaluate the internal and external validity of the report. We have included at least 1 reference for each item, when appropriate (Table 1). The flow diagram provides information about the progress of patients throughout a 2-group parallel-design RCT, perhaps the type of trial most commonly reported. [26] Appropriate adjustments will need to be made in reports of trials with a larger number of groups or trials using a different design.


Graphic
[Help with image viewing]

Table 1. Consolidation of Standards for Reporting Trials CONSORT3,4


Graphic
[Help with image viewing]

Figure 1 Progress through the various stages of a trial, including flow of participants, withdrawals, and timing of primary and secondary outcome measures. The "R" indicates randomization.

We recommend, for example, that RCTs should report how the allocation sequence was generated (eg, computer generated) and concealed (eg, in sequentially numbered, opaque, sealed envelopes) until the patient was randomized, something that is possible in all trials. [17] Schulz and colleagues [17] have shown empirically that trials in which the allocation sequence had been inadequately concealed yielded larger estimates of treatment effects (odds ratios that were lower, on average, by 30%-40%) compared with trials in which the authors reported adequate allocation concealment (ie, keeping the intervention assignments hidden from all individuals participating in the trial until the point of allocation). One possible interpretation is that some trials with inadequate reporting of allocation concealment actually had faulty randomization, and faulty randomization allowed the introduction of bias.

COMMENT

Although any optimally reported trial will address the items on the checklist and embody the flow diagram, the manner in which RCTs are reported (ie, their format) is also important. The format we favor includes a combination of 5 new subheadings in the text of the trial report and the use of the checklist during the journal submission process.

Three of the subheadings fall within the "Methods" section of a trial report: protocol, assignment, and masking (blinding). For example, under the subheading "assignment" the authors would describe the unit of randomization (eg, the individual patient). The remaining 2 subheadings are included when the authors report the results: participant flow and follow-up, and analysis. The participant flow and follow-up subheading is used in conjunction with describing details of the flow diagram. These 5 subheadings provide readers with consistency from report to report as to where they can expect to find relevant information. The completed checklist, which includes all 5 subheadings, would be required for all journal submissions. For example, corresponding authors would need to specify whether or not their trial report described the unit of randomization, and, if so, where in the report this is documented. We recognize that different trials, because of unusual or complex methods, will require modifications to the reporting structure.

The advantages of the CONSORT format include minimal change to the length and readability of the manuscript and enhanced clarity and organization in the actual report of a trial through the addition of the 5 new subheadings, while at the same time the information that is submitted to editors and reviewers is maximized through the completed checklist. This strategy avoids some of the criticisms of previously suggested reporting formats. [3,6]

Some authors, editors, and even reviewers may find our recommendations for the reporting of RCTs difficult and even restrictive. Similar concerns were also raised when more informative abstracts were first introduced. [8] Our separate group efforts [3,4] and our combined effort, CONSORT, came about because of the need to provide readers with enough valid and meaningful information concerning the design, conduct, and analysis of RCTs.

We would be remiss if we did not evaluate whether the CONSORT approach actually has its intended impact. Such an evaluation should incorporate the very design we are advocating improvements to its reporting: the RCT. The assessments need to be of both process and outcome, such as the readability of the report and its length as well as more standard quality assessments. [27] In the coming months we will work toward designing and implementing such an evaluation.

During our meeting there was unanimous agreement that the reporting of RCTs, and research in general, is frequently incomplete. [28] Many examples of inadequate reporting and their sequelae have been cited. [29-31] As a result, we decided that our deliberations should be disseminated to as wide an audience as possible in the hope that the CONSORT statement will ultimately lead to more comprehensive and complete reporting of RCTs. We recognize that the statement will need revision as new empirical evidence of bias becomes available. We invite all editors and clinical trialists to join us in using the CONSORT checklist and flow diagram. We will make the checklist and flow diagram available to all interested journal editors who wish to disseminate the information to their reviewers. Interested readers can also find the checklist and flow diagram on The Journal's World Wide Web site (http://www.ama-assn.org).

Note: Some reference citations may appear only in tables. [2-4,7-25]

Financial support for this study was provided by Abbott Laboratories, Abbott Park, Ill, and by the Council of Biology Editors, Northbrook, Ill.

We wish to thank all the members of the Standards of Reporting Trials group and the Asilomar Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature who helped bring us to this point. Our sincere appreciation to the many people who reviewed the manuscript.

Reprints: David Moher, MSc, Clinical Epidemiology Unit, Loeb Medical Research Institute, Ottawa Civic Hospital, 1053 Carling Ave, Ottawa, Ontario, Canada K1Y 4E9 (e-mail: moher@ceu.uottawa.ca).

REFERENCES

1. Freiman JA, Chalmers TC, Smith Jr H, Kuebler RR. The importance of beta, the type II error, and sample size in the design and interpretation of the randomized controlled trial: survey of 71 "negative' trials. N Engl J Med. 1978;299:690-694. [Medline Link] [BIOSIS Previews Link] [Context Link]

2. Moher D, Dulberg CS, Wells GA. Statistical power, sample size, and their reporting in randomized controlled trials. JAMA. 1994;272:122-124. [Fulltext Link] [Medline Link] [BIOSIS Previews Link] [Context Link]

3. The Standards of Reporting Trials Group. A proposal for structured reporting of randomized controlled trials. JAMA. 1994;272:1926-1931. Correction: JAMA. 1995;273:776. [Context Link]

4. Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature. Call for comments on a proposal to improve reporting of clinical trials in the biomedical literature: a position paper. Ann Intern Med. 1994;121:894-895. [Fulltext Link] [Context Link]

5. Rennie D. Reporting randomized controlled trials: an experiment and a call for responses from readers. JAMA. 1995;273:1054-1055. [Fulltext Link] [Medline Link] [Context Link]

6. Williams Jr JW, Holleman Jr DR, Samsa GP, Simel DL. Randomized controlled trial of 3 vs 10 days of trimethoprim/sulfamethoxazole for acute maxillary sinusitis. JAMA. 1995;273:1015-1021. [Fulltext Link] [Medline Link] [BIOSIS Previews Link] [Context Link]

7. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309:1286-1291. [Fulltext Link] [Medline Link] [BIOSIS Previews Link] [Context Link]

8. Ad Hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical studies. Ann Intern Med. 1987;106:598-604. [BIOSIS Previews Link] [Context Link]

9. Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. Can Med Assoc J. 1994;150:1611-1615. [Fulltext Link] [Medline Link] [BIOSIS Previews Link] [Context Link]

10. Oxman AD, Guyatt GH. A consumer's guide to subgroup analyses. Ann Intern Med. 1992;116:78-84. [Medline Link] [BIOSIS Previews Link] [Context Link]

11. Godfrey K. Statistics in practice: comparing the means of several groups. N Engl J Med. 1985;313:1450-1456. [Medline Link] [PsycINFO Link] [Context Link]

12. Gardner MJ, Bond J. An exploratory study of statistical assessment of papers published in the British Medical Journal. JAMA. 1990;263:1355-1357. [Medline Link] [BIOSIS Previews Link] [Context Link]

13. Lee YJ, Ellenberg JH, Hirtz DG, Nelson KB. Analysis of clinical trials by treatment actually received: is it really an option?. Stat Med. 1991;10:1595-1605. [Medline Link] [BIOSIS Previews Link] [Context Link]

14. Pocock SJ. When to stop a clinical trial. BMJ. 1992;305:235-240. [Medline Link] [BIOSIS Previews Link] [Context Link]

15. Donner A, Brown KS, Brasher P. A methodological review of nontherapeutic intervention trials employing cluster randomization, 1979-1989. Int J Epidemiol. 1990;19:795-800. [Medline Link] [BIOSIS Previews Link] [Context Link]

16. Schulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA. 1994;272:125-128. [Fulltext Link] [Medline Link] [BIOSIS Previews Link] [Context Link]

17. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408-412. [Fulltext Link] [Medline Link] [PsycINFO Link] [BIOSIS Previews Link] [Context Link]

18. Schulz KF. Subverting randomization in controlled trials. JAMA. 1995;274:1456-1458. [Fulltext Link] [Medline Link] [Context Link]

19. Schulz KF, Grimes DA, Altman DG, Hayes RJ. Blinding and exclusions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecology. BMJ. 1996;312:742-744. [Fulltext Link] [Medline Link] [BIOSIS Previews Link] [Context Link]

20. Karlowski TR, Chalmers TC, Frenkel LD, Kapikian AZ, Lewis TL, Lynch JM. Ascorbic acid for the common cold: a prophylactic and therapeutic trial. JAMA. 1975;231:1038-1042. [Medline Link] [BIOSIS Previews Link] [Context Link]

21. Pocock SJ. Clinical Trials: A Practical Approach. Chichester, England: John Wiley & Sons Inc; 1983:182-186. [Context Link]

22. Bailar JC, Mosteller F. Guidelines for statistical reporting in articles for medical journals. Ann Intern Med. 1988;108:266-273. [Context Link]

23. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. BMJ. 1986;292:746-750. [BIOSIS Previews Link] [Context Link]

24. Evans M, Pollock AV. Trials on trial: a review of trials of antibiotic prophylaxis. Arch Surg. 1984;119:109-113. [Medline Link] [BIOSIS Previews Link] [Context Link]

25. Gardner MJ, Machin D, Campbell MJ. Use of check lists in assessing the statistical content of medical studies. In: Gardner MJ, Altman DG, eds. Statistics With Confidence-Confidence Intervals and Statistical Guidelines. London, England: BMJ; 1989:101-108. [Context Link]

26. Fletcher RH, Fletcher SW. Clinical research in general medical journals: a 30-year perspective. N Engl J Med. 1979;301:180-183. [Medline Link] [Context Link]

27. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996;17:1-12. [Medline Link] [BIOSIS Previews Link] [Context Link]

28. Altman DG. The scandal of poor medical research: we need less research, better research, and research done for the right reasons. BMJ. 1994;308:283-284. [Fulltext Link] [Medline Link] [Context Link]

29. Altman DG, Dore C. Randomisation and baseline comparisons in clinical trials. Lancet. 1990;335:149-153. [Medline Link] [Context Link]

30. Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials: a survey of three medical journals. N Engl J Med. 1987;317:426-432. [Medline Link] [BIOSIS Previews Link] [Context Link]

31. Gotzsche PC. Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal anti-inflammatory drugs in rheumatoid arthritis. Control Clin Trials. 1989;10:31-56. Correction: Control Clin Trials. 1989;10:356. [Context Link]

Clinical Trials; Manuscripts, Medical; Publishing; Randomized Controlled Trials; Research



Accession Number: 00005407-199608280-00033
Browse Table of Contents

Copyright (c) 2000-2003 Ovid Technologies, Inc.
Version: rel6.0.0, SourceID 1.7240.1.49