Objectives: To compare the effectiveness, reliability, and acceptability of estimating rates of adverse events and rates of preventable adverse events using three methods: cross sectional (data gathered in one day), prospective (data gathered during hospital stay), and retrospective (review of medical records).
Design: Independent assessment of three methods applied to one sample.
Setting: 37 wards in seven hospitals (three public, four private) in southwestern France.
Participants: 778 patients: medical (n = 278), surgical (n = 263), and obstetric (n = 237).
Main outcome measures: The main outcome measures were the proportion of cases (patients with at least one adverse event) identified by each method compared with a reference list of cases confirmed by ward staff and the proportion of preventable cases (patients with at least one preventable adverse event). Secondary outcome measures were inter-rater reliability of screening and identification, perceived workload, and face validity of results.
Results: The prospective and retrospective methods identified similar numbers of medical and surgical cases (70% and 66% of the total, respectively) but the prospective method identified more preventable cases (64% and 40%, respectively), had good reliability for identification (kappa = 0.83), represented an acceptable workload, and had higher face validity. The cross sectional method showed a large number of false positives and identified none of the most serious adverse events. None of the methods was appropriate for obstetrics.
Conclusion: The prospective method of data collection may be more appropriate for epidemiological studies that aim to convince clinical teams that their errors contribute significantly to adverse events, to study organisational and human factors, and to assess the impact of risk reduction programmes.