Challenges in Estimating the Impact of Vaccination with Sparse data

Kayoko Shioda, Cynthia Schuck-Paim, Robert J. Taylor, Roger Lustig, Lone Simonsen, Joshua L. Warren, Daniel M. Weinberger

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

Resumé

BACKGROUND:The synthetic control model is a powerful tool to quantify the population-level impact of vaccines because it can adjust for trends unrelated to vaccination using a composite of control diseases. Because vaccine impact studies are often conducted using smaller, subnational datasets, we evaluated the performance of synthetic control models with sparse time series data. To obtain more robust estimates of vaccine impacts from noisy time series, we proposed a possible alternative approach, STL+PCA method (seasonal-trend decomposition plus principal component analysis), which first extracts smoothed trends from the control time series and uses them to adjust the outcome.
METHODS:Using both the synthetic control and STL+PCA models, we estimated the impact of 10-valent pneumococcal conjugate vaccine on pneumonia hospitalizations among cases <12 months and 80+ years of age during 2004-2014 at the subnational level in Brazil. We compared the performance of these models using simulation analyses.RESULTS:The synthetic control model was able to adjust for trends unrelated to 10-valent pneumococcal conjugate vaccine in larger states but not in smaller states. Simulation analyses showed that the estimates obtained with the synthetic control approach were biased when there were fewer cases, and only 4% of simulations had credible intervals covering the true estimate. In contrast, the STL+PCA analysis had 90% lower bias and had 95% of simulations, with credible intervals covering the true estimate.
CONCLUSIONS:Estimates from the synthetic control model might be biased when data are sparse. The STL+PCA model provides more accurate evaluations of vaccine impact in smaller populations.
SprogEngelsk
TidsskriftEpidemiology
Vol/bind30
Udgave nummer1
Sider61-68
Antal sider8
ISSN1044-3983
DOI
StatusUdgivet - 2019

Citer dette

Shioda, K., Schuck-Paim, C., Taylor, R. J., Lustig, R., Simonsen, L., Warren, J. L., & Weinberger, D. M. (2019). Challenges in Estimating the Impact of Vaccination with Sparse data. Epidemiology, 30(1), 61-68. https://doi.org/10.1097/EDE.0000000000000938
Shioda, Kayoko ; Schuck-Paim, Cynthia ; Taylor, Robert J. ; Lustig, Roger ; Simonsen, Lone ; Warren, Joshua L. ; Weinberger, Daniel M. / Challenges in Estimating the Impact of Vaccination with Sparse data. I: Epidemiology. 2019 ; Bind 30, Nr. 1. s. 61-68.
@article{d4a1db3e055f4689bec2f75b6969e952,
title = "Challenges in Estimating the Impact of Vaccination with Sparse data",
abstract = "BACKGROUND:The synthetic control model is a powerful tool to quantify the population-level impact of vaccines because it can adjust for trends unrelated to vaccination using a composite of control diseases. Because vaccine impact studies are often conducted using smaller, subnational datasets, we evaluated the performance of synthetic control models with sparse time series data. To obtain more robust estimates of vaccine impacts from noisy time series, we proposed a possible alternative approach, STL+PCA method (seasonal-trend decomposition plus principal component analysis), which first extracts smoothed trends from the control time series and uses them to adjust the outcome.METHODS:Using both the synthetic control and STL+PCA models, we estimated the impact of 10-valent pneumococcal conjugate vaccine on pneumonia hospitalizations among cases <12 months and 80+ years of age during 2004-2014 at the subnational level in Brazil. We compared the performance of these models using simulation analyses.RESULTS:The synthetic control model was able to adjust for trends unrelated to 10-valent pneumococcal conjugate vaccine in larger states but not in smaller states. Simulation analyses showed that the estimates obtained with the synthetic control approach were biased when there were fewer cases, and only 4{\%} of simulations had credible intervals covering the true estimate. In contrast, the STL+PCA analysis had 90{\%} lower bias and had 95{\%} of simulations, with credible intervals covering the true estimate.CONCLUSIONS:Estimates from the synthetic control model might be biased when data are sparse. The STL+PCA model provides more accurate evaluations of vaccine impact in smaller populations.",
author = "Kayoko Shioda and Cynthia Schuck-Paim and Taylor, {Robert J.} and Roger Lustig and Lone Simonsen and Warren, {Joshua L.} and Weinberger, {Daniel M.}",
year = "2019",
doi = "10.1097/EDE.0000000000000938",
language = "English",
volume = "30",
pages = "61--68",
journal = "Epidemiology",
issn = "1044-3983",
publisher = "Lippincott Williams & Wilkins",
number = "1",

}

Shioda, K, Schuck-Paim, C, Taylor, RJ, Lustig, R, Simonsen, L, Warren, JL & Weinberger, DM 2019, 'Challenges in Estimating the Impact of Vaccination with Sparse data', Epidemiology, bind 30, nr. 1, s. 61-68. https://doi.org/10.1097/EDE.0000000000000938

Challenges in Estimating the Impact of Vaccination with Sparse data. / Shioda, Kayoko; Schuck-Paim, Cynthia; Taylor, Robert J.; Lustig, Roger; Simonsen, Lone; Warren, Joshua L.; Weinberger, Daniel M.

I: Epidemiology, Bind 30, Nr. 1, 2019, s. 61-68.

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

TY - JOUR

T1 - Challenges in Estimating the Impact of Vaccination with Sparse data

AU - Shioda, Kayoko

AU - Schuck-Paim, Cynthia

AU - Taylor, Robert J.

AU - Lustig, Roger

AU - Simonsen, Lone

AU - Warren, Joshua L.

AU - Weinberger, Daniel M.

PY - 2019

Y1 - 2019

N2 - BACKGROUND:The synthetic control model is a powerful tool to quantify the population-level impact of vaccines because it can adjust for trends unrelated to vaccination using a composite of control diseases. Because vaccine impact studies are often conducted using smaller, subnational datasets, we evaluated the performance of synthetic control models with sparse time series data. To obtain more robust estimates of vaccine impacts from noisy time series, we proposed a possible alternative approach, STL+PCA method (seasonal-trend decomposition plus principal component analysis), which first extracts smoothed trends from the control time series and uses them to adjust the outcome.METHODS:Using both the synthetic control and STL+PCA models, we estimated the impact of 10-valent pneumococcal conjugate vaccine on pneumonia hospitalizations among cases <12 months and 80+ years of age during 2004-2014 at the subnational level in Brazil. We compared the performance of these models using simulation analyses.RESULTS:The synthetic control model was able to adjust for trends unrelated to 10-valent pneumococcal conjugate vaccine in larger states but not in smaller states. Simulation analyses showed that the estimates obtained with the synthetic control approach were biased when there were fewer cases, and only 4% of simulations had credible intervals covering the true estimate. In contrast, the STL+PCA analysis had 90% lower bias and had 95% of simulations, with credible intervals covering the true estimate.CONCLUSIONS:Estimates from the synthetic control model might be biased when data are sparse. The STL+PCA model provides more accurate evaluations of vaccine impact in smaller populations.

AB - BACKGROUND:The synthetic control model is a powerful tool to quantify the population-level impact of vaccines because it can adjust for trends unrelated to vaccination using a composite of control diseases. Because vaccine impact studies are often conducted using smaller, subnational datasets, we evaluated the performance of synthetic control models with sparse time series data. To obtain more robust estimates of vaccine impacts from noisy time series, we proposed a possible alternative approach, STL+PCA method (seasonal-trend decomposition plus principal component analysis), which first extracts smoothed trends from the control time series and uses them to adjust the outcome.METHODS:Using both the synthetic control and STL+PCA models, we estimated the impact of 10-valent pneumococcal conjugate vaccine on pneumonia hospitalizations among cases <12 months and 80+ years of age during 2004-2014 at the subnational level in Brazil. We compared the performance of these models using simulation analyses.RESULTS:The synthetic control model was able to adjust for trends unrelated to 10-valent pneumococcal conjugate vaccine in larger states but not in smaller states. Simulation analyses showed that the estimates obtained with the synthetic control approach were biased when there were fewer cases, and only 4% of simulations had credible intervals covering the true estimate. In contrast, the STL+PCA analysis had 90% lower bias and had 95% of simulations, with credible intervals covering the true estimate.CONCLUSIONS:Estimates from the synthetic control model might be biased when data are sparse. The STL+PCA model provides more accurate evaluations of vaccine impact in smaller populations.

U2 - 10.1097/EDE.0000000000000938

DO - 10.1097/EDE.0000000000000938

M3 - Journal article

VL - 30

SP - 61

EP - 68

JO - Epidemiology

T2 - Epidemiology

JF - Epidemiology

SN - 1044-3983

IS - 1

ER -