Skip directly to site content Skip directly to page options Skip directly to A-Z link Skip directly to A-Z link Skip directly to A-Z link
Volume 20, Number 4—April 2014

Incorporating Research and Evaluation into Pandemic Influenza Vaccination Preparedness and Response

Tom T. ShimabukuroComments to Author  and Stephen C. Redd
Author affiliations: Centers for Disease Control and Prevention, Atlanta, Georgia, USA

Cite This Article

This issue of Emerging Infectious Diseases contains 2 articles that address critical elements of implementing large-scale local-level vaccination programs in response to a public health emergency. In addition to describing activities undertaken during the 2009 pandemic of influenza A(H1N1) virus (pH1N1) and subsequent program evaluation and lessons learned, the articles highlight the critical role for scientific evaluation in improving our ability to respond to emergencies (1,2).

Saha et al. report on assessment of the efficiency of public health–managed large-scale vaccination clinics, referred to as points of dispensing (PODs), to administer pH1N1 vaccine in densely populated Los Angeles County, California, USA (3). The authors examined rates of visits to PODs according to patients’ socioethnic characteristics and assessed factors affecting vaccination throughput (doses administered per hour). Their evaluation provides information about optimal placement of PODs in the community and possible strategies to improve their operational efficiency.

Marcello et al. describe the experience of New York City, New York, USA, in using its Citywide Immunization Registry to capture information about pH1N1 vaccine doses administered during the response (4). Immunization information systems (IISs) are commonly used to document administration of recommended childhood vaccinations; however, routine adult participation has historically been low (5). During New York City’s pH1N1 vaccination program, the health department required all providers to register with the Citywide Immunization Registry and report doses administered. The New York City experience demonstrates the feasibility and potential usefulness of expanding mandatory IIS reporting to all types of providers during a pandemic influenza vaccination program as a means of monitoring progress and managing supply and distribution. The article also reveals limitations of IISs as they existed in 2009 and 2010.

Los Angeles County and New York City were able to conduct meaningful program evaluation because public health officials had the foresight to incorporate evaluation into emergency planning and response and commit valuable time and resources to conduct health services research during the height of pH1N1 vaccination. Los Angeles County health officials coordinated a meticulous data collection effort from 101 POD events held during a 6-week period from October through December 2009. In New York City, a substantial outreach and education program was necessary to incorporate providers of vaccines to adults and others not accustomed to IIS reporting into the program to acquire the most comprehensive and timely information possible about vaccine doses administered.

For health departments, the decision to commit to planning for and conducting research and evaluation during a public health emergency is complicated by the competing priority of providing direct services to persons and populations in need. In addition to balancing the effort needed to plan and conduct the public health response and the research or evaluation effort, other uncertainties impose limitations on research efforts during emergencies. In the case of research conducted during the 2009 influenza A(H1N1) pandemic, the inherent difficulty of projecting demand for vaccination, combined with delays and uncertainty around the timing of availability of pH1N1 vaccine, were serious challenges for the vaccination program and for its evaluation (6).

Although researchers have to make assumptions about the event under study, a high degree of flexibility is necessary. Public health emergencies often present unforeseen circumstances and influenza pandemics are among the least predictable of all emergencies. Public health priorities during a pandemic response can change quickly on the basis of disease characteristics, resource constraints, and the potential for social disruption. Planning evaluation efforts for different pandemic scenarios and being nimble enough to rapidly adapt to shifting priorities are essential qualities for any research and evaluation program. Research planners need to be able to identify and address key response questions under conditions of much less certainty than in other research efforts.

Despite these challenges, invaluable knowledge is gained from well-planned and well-conducted (and appropriately resourced) health services research during an event. Tabletop and functional exercises are useful tools for organizations to expand knowledge, assess readiness, and identify deficiencies (7,8). Yet they rarely approach the intensity, complexity, and duration of a real event. Data obtained during a response to an actual public health emergency provide the best (and perhaps only) source of information for program evaluation under conditions in which the public health system is severely stressed. To address knowledge gaps in preparedness, public health authorities must strike an appropriate balance between conducting research, evaluating program efforts, and providing services during a public health emergency (2). Publishing the results of such evaluations is also essential to permit others who are planning vaccination campaigns during emergencies to benefit from the experiences in Los Angeles County and New York City.

Human infection with influenza A(H3N2) variant virus (9) and avian influenza A(H7N9) virus (10) and continued sporadic cases of infection with highly pathogenic avian influenza A(H5N1) virus (11)—all viruses with pandemic potential—remind us that we must remain vigilant in our preparedness. We encourage health officials and leaders of health care organizations at all levels to identify the critical questions that will affect future emergencies and design research efforts into emergency preparedness planning to take advantage of these rare opportunities to learn and improve the nation’s response capability. For vaccination programs during influenza pandemics, priorities for evaluation include the following: efforts to improve situational awareness; efforts to identify and vaccinate populations prioritized for vaccination, including vulnerable populations and groups prioritized because of occupation; strategies to balance vaccine allocation to the existing, largely private, vaccination system, with large-scale vaccination venues (i.e., PODs); and strategies to build systems for pandemic influenza response that also improve seasonal influenza vaccination programs.

Although we recognize that smaller and less well-resourced organizations may be challenged in their ability to conduct large-scale sophisticated evaluation, we believe that learning by doing is possible for any organization, provided leaders and planners are willing to make the commitment. Even modest evaluation efforts will increase knowledge and advance preparedness.

Dr Shimabukuro is acting deputy director of the Immunization Safety Office, Centers for Disease Control and Prevention (CDC), Atlanta. During the 2009 influenza A(H1N1) pandemic, he served as deputy director of CDC’s H1N1 Vaccine Task Force.

Dr Redd is the director of the Influenza Coordination Unit, CDC, Atlanta. He served as incident commander of CDC’s response to the 2009 influenza A(H1N1) pandemic.



  1. Schuchat  A, Bell  BP, Redd  SC. The science behind preparing and responding to pandemic influenza: the lessons and limits of science. Clin Infect Dis. 2011;52(Suppl 1):S812. DOIPubMedGoogle Scholar
  2. Lurie  N, Manolio  T, Patterson  AP, Collins  F, Frieden  T. Research as a part of public health emergency response. N Engl J Med. 2013;368:12515 . DOIPubMedGoogle Scholar
  3. Saha  S, Dean  B, Teutsch  S, Borse  HR, Meltzer  MI, Bagwell  D, Assessing the efficiency of 2009 pandemic H1N1 vaccination through point of dispensing units, Los Angeles County. [zzz–zz.]. Emerg Infect Dis. 2014;•••:20.
  4. Marcello  RK, Papadouka  V, Misener  M, Wake  E, Mandell  R, Zucker  JR. Monovalent H1N1 influenza vaccine distribution and reporting in New York City. [zzz–zz.]. Emerg Infect Dis. 2009;2014:20.
  5. Centers for Disease Control and Prevention. Progress in immunization information systems—United States, 2012. MMWR Morb Mortal Wkly Rep. 2013;62:10058 .PubMedGoogle Scholar
  6. US Department of Health and Human Services. An HHS retrospective on the 2009 H1N1influenza pandemic to advance all hazards preparedness, June 2012 [cited 2014 Feb 8].
  7. Biddinger  PD, Savoia  E, Massin-Short  SB, Preston  J, Stoto  MA. Public health emergency preparedness exercises: lessons learned. Public Health Rep. 2010;125(Suppl 5):1006 .PubMedGoogle Scholar
  8. Savoia  E, Testa  MA, Biddinger  PD, Cadigan  RO, Koh  H, Campbell  P, Assessing public health capabilities during emergency preparedness tabletop exercises: reliability and validity of a measurement tool. Public Health Rep. 2009;124:13848 .PubMedGoogle Scholar
  9. Centers for Disease Control and Prevention. Interim information for clinicians about human infections with H3N2v virus, September 14, 2012 [cited 2014 Feb 8].
  10. Centers for Disease Control and Prevention. Emergence of avian influenza A(H7N9) virus causing severe human illness—China, February–April 2013. MMWR Morb Mortal Wkly Rep. 2013;62:36671 .PubMedGoogle Scholar
  11. Christian  KA, Ijaz  K, Dowell  SF, Chow  CC, Chitale  RA, Bresee  JS, What we are watching—five top global infectious disease threats, 2012: a perspective from CDC's Global Disease Detection Operations Center. Emerg Health Threats J. 2013;6:20632.


Cite This Article

DOI: 10.3201/eid2004.140224

Table of Contents – Volume 20, Number 4—April 2014

EID Search Options
presentation_01 Advanced Article Search – Search articles by author and/or keyword.
presentation_01 Articles by Country Search – Search articles by the topic country.
presentation_01 Article Type Search – Search articles by article type and issue.



Please use the form below to submit correspondence to the authors or contact them at the following address:

Tom T. Shimabukuro, Immunization Safety Office, Centers for Disease Control and Prevention, 1600 Clifton Rd NE, Mailstop D26, Atlanta, GA 30333, USATom T. Shimabukuro, Centers for Disease Control and Prevention, 1600 Clifton Rd NE, Mailstop D26, Atlanta, GA 30333, USATom T. Shimabukuro, Centers for Disease Control and Prevention, 1600 Clifton Rd NE, Mailstop D26, Atlanta, GA 30333, USA

Send To

10000 character(s) remaining.


Page created: March 19, 2014
Page updated: March 19, 2014
Page reviewed: March 19, 2014
The conclusions, findings, and opinions expressed by authors contributing to this journal do not necessarily reflect the official position of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors' affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above.