Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

Although randomized placebo-controlled trials (RCT) are critical to establish efficacy of vaccines at the time of licensure, important remaining questions about vaccine effectiveness (VE)—used here to include individual-level measures and population-wide impact of vaccine programmes—can only be answered once the vaccine is in use, from observational studies. However, such studies are inherently at risk for bias. Using a causal framework and illustrating with examples, we review newer approaches to detecting and avoiding confounding and selection bias in three major classes of observational study design: cohort, case-control and ecological studies. Studies of influenza VE, especially in seniors, are an excellent demonstration of the challenges of detecting and reducing such bias, and so we use influenza VE as a running example. We take a fresh look at the time-trend studies often dismissed as ‘ecological’. Such designs are the only observational study design that can measure the overall effect of a vaccination programme [indirect (herd) as well as direct effects], and are in fact already an important part of the evidence base for several vaccines currently in use. Despite the great strides towards more robust observational study designs, challenges lie ahead for evaluating best practices for achieving robust unbiased results from observational studies. This is critical for evaluation of national and global vaccine programme effectiveness.