Objectives:

To characterize rapid reviews and similar products, to understand the context in which rapid products are produced (e.g., end-users and purposes for rapid products), to understand methodological guidance and strategies used to make products rapid and describe how these differ from systematic review (SR) procedures, and to identify empiric evidence on the impact of methodological approaches on their reliability and validity.

Methods:

We searched the literature to identify rapid review methods, empiric evidence on rapid review methodology, and methodological guidance. We conducted interviews with members of organizations known to produce rapid reviews to characterize the types of rapid products produced and to understand the context and uses for rapid products, identify current practices, and understand the evolution of their programs and products.

Results:

We identified 36 examples of rapid products produced by 20 organizations with production time ranging from 5 minutes to 8 months.

We categorized rapid products into four groups based on the extent of synthesis: (1) ”inventories” list what evidence is available, and other contextual information needed to make decisions, but do not synthesize the evidence or present summaries or conclusions; (2) ”rapid responses” present the end-user with an answer based on the best available evidence (usually guidelines or SRs), but do not attempt to formally synthesize the evidence into conclusions; (3) ”rapid reviews” perform a synthesis (qualitative and/or quantitative) to provide an answer about the direction of evidence and possibly the strength of evidence; (4) “automated approaches” use databases of extracted study elements and programming to generate meta-analyses in response to user-defined queries.

Methodological approaches identified for rapid products include: searching fewer databases; limited use of grey literature; restricting the types of studies included (e.g., English only, most recent 5 years); relying on existing SRs; limiting full-text review; limiting dual review for study selection and/or data extraction; limiting data extraction; limiting risk of bias assessment or grading; minimal evidence synthesis; providing nominal conclusions or recommendations; and limiting external peer review. As the timeframes for products lengthened many limitations were lifted; however, there were still restrictions on database searching, inclusion, extent of data extraction, and dual review. With lengthened production time, there was more often risk of bias assessment, evidence grading, and external peer review.

Key informant interviews demonstrated that the essence of rapid products differs from that of SRs: key differences include the close relationship with the end-user and focus on helping a specific end-user make a specific decision in an identified timeframe. Because there may not be lead time before the review is needed and the end-user may need the review urgently, maintaining a highly skilled staff is critical to organizational readiness to produce rapid reviews. Having few and/or narrow questions (e.g., emerging technologies, single interventions, specific populations) was also necessary.

There is almost no empiric evidence directly comparing results of rapid products with SRs. One report suggested there may not be any impact; however, it focused on surgical interventions and may not be generalizable to other clinical specialties or health care fields in which rapid products or SRs are conducted.

Conclusions:

Rapid products have tremendous methodological variation. Overall, they vary on two important dimensions that are captured by the term “rapid review”: the timeframe for completion and extent of synthesis. The similarity of rapid products lies in their close relationship with the end-user to meet decisionmaking needs in a limited timeframe. The following are considerations for creating rapid products:

• products should be developed in the context of identified end-users and their specific decisionmaking needs and circumstances;
• a close relationship with the end-user and iterative feedback is essential;
• reliance on existing SRs require methods to summarize and interpret evidence;
• a highly skilled and experienced staff and the capacity to mobilize skilled staff quickly are critical;
• restricting scope may be necessary;
• producers and users need to accept modifications to standard SR methods; and
• limitations need to be clearly reported, particularly in terms of potential bias and shortcomings of the conclusions.

Future research evaluating end-user perspectives will complement these findings and provide additional considerations for those interested in establishing a rapid response program or producing rapid products.