A Detailed Methodology 2006 [archived]
Expand allprospect provides recommendations for procedure-specific postoperative pain management based on expert interpretation of procedure-specific and transferable evidence in the context of current clinical practice
The process by which the prospect Working Group formulates its recommendations has been refined to take greater account of the quality of available evidence and to reduce the potential for bias in the formulation of recommendations (Neugebauer 2007). The refined methodology will be implemented in full for procedure reviews performed from 2006
Working Group
- The Working Group is an international collaboration of surgeons and anaesthesiologists, chaired by Professor Francis Bonnet
- The Working Group review in detail the evidence from the procedure-specific systematic review, as well as the transferable evidence, the clinical practice information and the draft recommendations agreed at the Subgroup meeting. For each analgesic, anaesthetic or surgical intervention, the group first comes to a consensus about the applicability and relevance of transferable evidence and clinical practice information. The group then refines each recommendation according to consensus, considering all of the evidence and information. The process is described in more detail in subsequent folders
Subgroup
- Consists of at least two members of the Working Group, and occasionally a clinician external to the prospect group, who have particular expertise in the operative technique to be reviewed. May also include an expert member of the data analysis team
- Advises on literature searches, assesses the level of evidence of each study, performs a detailed review of the summary ‘Outcomes Document", identifies relevant transferable evidence, and agrees a draft of the clinical practice information and recommendations for procedure-specific postoperative pain management
Data Analysis and Medical Writing Team
- Work closely with the Subgroup and Working Group to coordinate literature searches, review abstracts and papers, assess quality and level of evidence of each study, create data tables, carry out qualitative and quantitative analyses, draft the summary Outcomes and Review documents for review by the Subgroup and Working Group, collate comments of Subgroup and Working Group, prepare the final version of the Review document which is subsequently presented on the prospect website
Recommendations
- Best practice recommendations, which are graded based on the source and level of evidence Click here for more information
- Consensus of the prospect Working Group, based on the evidence, with expert interpretation in the context of clinical practice
- Formulating the recommendations – the process Click here for more information
Clinical practice
- Guidance from the prospect Working Group, considering the following factors for clinical practice Click here for more information
Transferable evidence
- Transferable evidence is identified by the Subgroup and may include evidence of analgesic efficacy from other surgical procedures with a similar pain profile, or evidence regarding adverse effects from a variety of other surgical procedures Neugebauer et al 2007
Procedure-specific evidence
- Procedure-specific evidence is derived from systematic literature reviews, performed using a process based on the protocol of the Cochrane Collaboration. prospect systematic reviews incorporate the following key characteristics Click here for more information
Comprehensive search strategy to identify relevant studies
Literature search terms are clearly defined and agreed by the prospect Subgroup, Data Analysis and Medical Writing Team and Information Specialist (see R
Studies that appear relevant to the systematic review are selected for closer inspection by the Data Analysis and Medical Writing Team
Inclusion criteria:
- Randomised controlled clinical trials of analgesic, anaesthetic and operative interventions in adults, relating to surgical procedure being reviewed
- Pain scores from a linear pain scale, e.g. visual analogue scale (VAS) or verbal or numerical rating scale (VRS or NRS)
- While there is evidence that studies published in English are more likely to report beneficial effects for a particular treatment or intervention than studies published in other languages (Moher 1996, Egger 1997, Juni 2002, Song 2000), only English language studies are included, due to time and resource constraints
Inclusion criteria:
- Randomised controlled clinical trials of analgesic, anaesthetic and operative interventions in adults, relating to surgical procedure being reviewed
- Pain scores from a linear pain scale, e.g. visual analogue scale (VAS) or verbal or numerical rating scale (VRS, NRS)
The Data Analysis and Medical Writing Team record information from each included study, in tables, with similar comparisons grouped together for ease of analysis.
Information is collected according to the list of outcome measures for assessment, as agreed at the outset of the review. The following information is always recorded if included in the published study:
- Interventions compared and patient numbers in each group
- Analgesia administered to all patients
- Qualitative outcomes for pain scores, supplementary analgesic use, time to first request for rescue analgesia, postoperative nausea and vomiting
- Additional outcomes that are considered to be important in the procedure under review
Assess quality and level of evidence of each study
Quality assessment is performed by the Data Analysis and Medical Writing Team, and the Subgroup
Levels of evidence and grades of recommendation in PROSPECT reviews (from 2006)
Qualitative analysis
Qualitative analyses are performed for each group of studies reporting similar treatment comparisons (for example, all studies comparing incisional local anaesthetics versus placebo). All analyses are reported in a summary Outcomes Document. Qualitative results are reported for all specified outcomes, and details of the assigned levels of evidence are recorded in the text of the document
Meta-analysis of pooled data
Quantitative analyses are performed where possible, where study designs are homogenous and data are reported in a suitable manner, using Review Manager 4.2.2 software (RevManAnalyses), which has been developed for Cochrane Collaboration systematic reviews. Quantitative analyses are reported in the summary Outcomes Document
- Meta-analysis of continuous data provides a weighted mean difference (WMD) and a probability of overall treatment effect, based on means and standard deviations
- Meta-analysis of dichotomous data provides an odds ratio (OR) and a probability of overall treatment effect, based on the proportion of patients affected
- Studies that do not report mean and standard deviation (SD) data (for continuous variables) or proportion of patients affected (for dichotomous variables), are not included in the meta-analyses
Review by Subgroup
The Subgroup consists of at least two members of the Working Group and, in some cases, one additional, external expert with special interest in the procedure in question; the Subgroup reviews the Outcomes Document, evaluating the strength of evidence for each surgical, anaesthetic or analgesic intervention
Subgroup meeting
The Subgroup members and the Data Analysis and Medical Writing Team discuss the Outcomes Document; the Subgroup drafts the transferable evidence, clinical practice information and recommendations. The recommendations are formulated after a discussion of the relative benefits and harms of each intervention, taking into account:
- Strength of results and consistency of evidence
- Important safety considerations related to the analgesic intervention
- Ethical constraints
- Clinical expertise
- Patient preferences
- Clinical relevance
- Pathophysiology and clinical plausibility
- Applicability to patient group
- Practicality
- Side-effects
Review document
The Data Analysis and Medical Writing Team prepares the ‘Review’ document. All statements of evidence, including procedure-specific, transferable and clinical practice information, are presented with a tick or cross, indicating whether or not they support the use of that particular intervention. Recommendations drafted at the Subgroup meeting are included
Comments of Working Group collated by Delphi method
Following evaluation of the Review document, Working Group members comment on the evidence and draft recommendations. The Working Group does not discuss individual comments until the Data Analysis and Medical Writing Team has received all the comments, according to the Delphi method (Dalkey 1963):
- Comments are forwarded only to the Data Analysis and Medical Writing Team, and not to the whole Working Group (this method avoids the potential for one Working Group member’s views to be accepted by the rest of the Group without full consideration of the data)
- Individual comments are then collated for discussion at a round-table meeting
Working Group meeting
Collated comments are discussed and the recommendations are finalised. Where consensus is not reached by group discussion, a modified Nominal Group Process is used. This is a method whereby:
1. Each Working Group member expresses his or her comments/concerns about each recommendation, one after the other; at this stage, there is no discussion, agreement or disagreement from the other members
2. Comments are discussed
3. Each Working Group member votes to accept or reject individual comments
4. Further rounds of comments, discussion and voting are included until consensus is reached
5. Where full consensus of the Working Group is not achieved, a majority decision is taken based on a vote, and this is noted alongside the recommendation
6. Recommendations are presented with a brief explanation of the evidence on which they are based (including details of the balance of benefits and harms, where relevant). Recommendations are graded to indicate the strength of the recommendation, which reflects the level and source of evidence (also specified)
Levels of evidence and grades of recommendation in PROSPECT reviews (from 2006)
Revision of Review document and final consideration by Working Group
The Review document is updated according to the consensus agreements from the Working Group meeting and circulated to the Working Group for a final round of comments by the Delphi method; any final comments are incorporated according to the consensus of the Working Group
Procedure review available at www.postoppain.org
The final version of the Review Document is presented on the prospect website.
Evidence and graded recommendations for peri-operative interventions are contained within folders, in which procedure-specific evidence, transferable evidence, clinical practice information and prospect recommendations are clearly separated.
A summary of the recommendations and details of the procedure-specific systematic literature review are also presented, including criteria for study inclusion as well as lists of included and excluded studies.
The web-based format offers a user-friendly way to present the large amount of information contained within each review, and encourages users to submit feedback to the Working Group via the website.