Levels of evidence and grades of recommendation
From 2006 onwards, the prospect methodology has been refined to take more account of the quality of the evidence on which the recommendations are based. The way in which the quality of studies determines the level of evidence, and thereby determines the grade of recommendation, is summarised below. Development of the prospect methodology has been an ongoing process, and previous experience indicated the need for these changes, to help clarify the basis for the recommendations.
Sources of evidence in PROSPECT
The evidence for prospect is derived from three separate sources, and this evidence is taken into consideration by the prospect Working Group to determine the prospect recommendations:
- Procedure-specific evidence derived from the systematic reviews of the literature
- Transferable evidence from comparable procedures, or from other relevant sources, identified by the members of the prospect Working Group
- Current practice – A commentary on the interventions from the members of the prospect Working Group
- Practical prospect recommendations are based on all the information
Study quality assessment
All cited studies are assessed for quality of reporting of methodology and results (assessment performed by a Data Analysis and Medical Writing Team and the prospect Subgroup):
1. Statistical analyses and patient follow-up assessment: indicates whether statistical analyses were reported, and whether patient follow-up was greater or less than 80%.
2. Allocation concealment assessment: indicates whether there was adequate prevention of foreknowledge of treatment assignment by those involved in recruitment (A adequate, B unclear, C inadequate, D not used). Empirical research has shown that trials with inadequate or unclear allocation concealment report significantly greater estimates of treatment effect than those trials in which concealment was adequate (Chalmers 1983, Schulz 1995, Moher 1998). Allocation concealment was found to be more important for preventing bias than other aspects of study quality, such as generation of the allocation sequence and double-blinding (Chalmers 1983, Schulz 1995, Moher 1998, Higgins and Green 2005: Section 6.3. http://www.cochrane.org/resources/handbook/hbook.htm)
3. Numerical scores (total 1–5) for study quality: assigned using the method proposed by Jadad 1996, to indicate whether a study reports appropriate randomisation, double-blinding and statements of possible withdrawals. Empirical research found that low-quality trials were associated with an increased estimate of treatment benefit than high-quality trials (Moher 1998)
4. Additional study quality assessment: including an assessment of how closely the study report meets the requirements of the CONSORT statement (Moher 2005) (additional assessment performed by the prospect Subgroup)
Grading of recommendations based on overall level of evidence
The recommendations are graded according to the overall level of evidence, which is determined by the quality of studies cited, the consistency of evidence and the source of evidence (as indicated in the Table below)
Table: Relationship between quality and source of evidence, levels of evidence and grades of recommendation in PROSPECT
| Study quality assessments | Level of Evidence (LoE) | Grade of recommendation (based on overall LoE, considering balance of clinical practice information and evidence) |
||||||
| Study type | Statistical analyses and patient follow-up assessment | Allocation concealment | Jadad scores | Additional assessment of overall study quality required to judge LoE | Procedure-specific | Transferable | ||
| Systematic review with homogeneous results | N/A | N/A | N/A | N/A | 1 | A | B | |
| Randomised controlled trial (RCT) | Statistics reported and >80% follow-up |
AND | A | (1–5) | N/A | 1 | A
(based on two or more studies or a single large, well-designed study) |
B |
| OR | ||||||||
| B | (3–5) | N/A | ||||||
| OR | ||||||||
| B | (1–2) | Yes | ||||||
| RCT | Statistics not reported or questionable or <80% follow-up | AND/OR | B | (1–2) | Yes | 2 | B
(or extrapolation from one procedure-specific |
C |
| OR | ||||||||
| C | (1–5) | N/A | ||||||
| OR | ||||||||
| D | (1–5) | N/A | ||||||
| Non-systematic review, cohort study, case study; (e.g. some adverse effects evidence) |
N/A | N/A | 3 | C | ||||
| Clinical practice information (expert opinion); inconsistent evidence | N/A | N/A | 4 | D | ||||