US Vaccine Pharmacovigilance: Your Friend is Probably Using VAERS Incorrectly
The short version: Making sure that the pharmaceuticals the public takes each day are safe is a challenging task, and it’s easy to mess up if you don’t know what you’re doing. You may have seen scary claims about COVID-19 vaccines causing, well, basically any really bad thing you can think of and these often have very official VAERS report ID numbers attached to them. But this isn’t how VAERS works or is meant to be used. In fact, VAERS explicitly warns against using their system in this way because these reports have not been evaluated for a causal link between the vaccine and the event (in fact a previous study of a random sampling of VAERS reports that were assessed by experts showed that just 3% of them likely had a causal relationship to vaccination). The major point of VAERS is to act as an early warning system- to bring attention to new or rare problems that might have been missed in clinical trials. This happens when a bunch of reports claiming the same thing appear, and this is associated with a rate above the natural rate that occurs in the absence of the vaccine in question. VAERS has been used to initiate important investigations of this nature to keep our vaccination program as safe as possible, like with an early rotavirus vaccine that very rarely caused a problem where the bowel folded over itself that has since been removed. But VAERS can also be prone to giving false positive signals (such as for Guillain-Barre syndrome and the MCV4 vaccine), and biased reporting is a major issue (there was a huge surge of reports of vaccines causing autism after Wakefield’s fraudulent publication and since then multiple rigorous epidemiological studies show no link). There’s no control group to go with these reports, so they alone can’t tell you whether or not the risk is real. To help with that we have other surveillance systems that are equipped for these kinds of investigations- the investigation isn’t done through VAERS because it’s not set up for that. VAERS is intended to help tell professionals what to investigate but isn’t meant for doing the investigations themselves. COVID-19 vaccines are receiving an unprecedented level of scrutiny, which isn’t a bad thing, but you have to remember that these vaccines are being given in the real world, where bad things can happen even if they aren’t related just by random chance. Professionals all over the world are monitoring COVID-19 vaccines for any kind of safety issues, and so far, there are no signals to suggest these vaccines are unsafe. If that changes, it will prompt a revision of how the vaccines are used e.g. excluding certain patient populations from getting them because in those individuals the risks outweigh the benefits, as has always happened in the past in the few instances where it was warranted. I do also want to encourage anyone who has any kind of adverse experience after a vaccine to report it: that data can be valuable in helping to make our vaccination program safer for everyone.
Monitoring of pharmaceuticals for safety and efficacy doesn’t just stop after they finish with clinical trials. Post-marketing surveillance, or alternatively Phase IV of the clinical trial process (note- this isn’t the same as the vaccine or treatment being experimental), or pharmacovigilance are all terms that describe the monitoring that happens of a pharmaceutical once it has been released to the public, such as if it is granted emergency use authorization or full licensure. This gets taken especially seriously for vaccines. Vaccines are public health interventions given to healthy people (in particular infants and children) to keep them from getting sick and their uptake depends a great deal on public confidence. Therefore, we as a society tolerate very little risk when it comes to vaccines compared with other pharmaceuticals. Unfortunately, if it exerts a biological effect, it has potential for harm in some individual or in some quantity. For instance, people can have anaphylactic reactions to a vaccine component that would render the vaccine unsafe for that individual but not for everyone else. There isn’t a way to make the risk go fully down to zero, though of course the risks of the vaccine need to be weighed against the risks of the disease, and no vaccine ever makes it to use with the public unless it is demonstrated to be many thousands of times safer than the disease it prevents. Pharmacovigilance can help to distinguish true vaccine reactions from coincidental events, estimate their attributable risk, identify risk factors that may constitute valid contraindications, and, if the pathophysiologic mechanism becomes known, develop safer vaccines. If ever it’s demonstrated that the risks exceed the benefits for a vaccine (which is rare given how rigorous clinical trials are, but it has happened), the vaccine is removed. Note that a vaccine can be very reactogenic and still be very safe- though uncomfortable and not to be trivialized, vaccine reactogenicity just indicates that people’s immune systems are working as they should be and are self-limiting issues that don’t merit the removal of a vaccine from public access (though manufacturers may work to create less reactogenic formulations and get them through clinical trials provided that efficacy is at least as good). Generally, questions of safety come down to two things: how common the adverse reaction is, and how serious. For instance, about 1 per 700,000 to 3.4 million doses of the oral polio vaccine (OPV) causes paralytic polio. This is undeniably very serious, but in places where polio is endemic, the vaccine’s benefits still vastly outweigh the risks given the incredible rarity of the outcome weighed against the risks of active poliovirus infection. On the other hand, in the US where there hasn’t been a case of polio in decades, the inactivated polio vaccine (which is incapable of causing this) is used and OPV isn’t. Of course, pharmacovigilance is not used just to make decisions about whether or not to keep a product accessible to the public but also to help maximize the safety and use of the pharmaceutical in question. For instance, clinical trials are more sterile than the real world and so it’s important to see how the pharmaceutical performs outside of that setting, how effective it is and to adjust the usage of the pharmaceutical as necessary.
A sampling of some things reported to VAERS. ID numbers are included so that the events can be verified. Note however that VAERS recently switched to a 7-digit ID system so you need to write a “0” before the stated IDs to find the report.
VAERS
Vaccine pharmacovigilance in the US involves an extensive network of systems that work together to ensure vaccine safety and prompt removal of vaccines from market should certain safety signals arise. The system people are probably most familiar with right now is the Vaccine Adverse Event Reporting System or VAERS, likely because across their social media they have seen people give out very official-seeming ID numbers reporting people’s deaths after taking the COVID-19 vaccine and being warned not to get it for fear of them being next. This represents a complete misuse of VAERS. With VAERS, anyone (including non-healthcare workers) may file a report of an adverse event- meaning anything bad- that happens after receiving a vaccine. After this, a contractor with CDC or FDA reviews the event and codes it into the database. In general, these reports are unverified, and thus may not be accurate or truthful. There is no time limit associated with filing the report but it is encouraged that they be filed as close to the adverse event as possible. Interestingly, as per Strom et al. (emphasis mine):
In 2016, 22% of US VAERS reports were submitted by patients, parents, or other unidentified sources, 27% by healthcare providers, and 51% by vaccine manufacturers; in recent years leading up to and including 2016, VAERS received around 40,000 US reports annually (CDC, unpublished data).
…
Approximately 7% of US VAERS reports are classified as serious (documenting death, life-threatening illness, permanent disability, hospitalization, or prolongation of hospitalization), although the percentages of serious/nonserious reports vary by age group and type of vaccine.
However, when adverse events of special interest occur, those who file the report may be contacted for more information. VAERS is a passive (spontaneous) reporting system whose principal function is to help with hypothesis generation and to act as an early warning system: if a given adverse event is being reported more frequently than would be expected according to the background rate, it triggers an investigation (there is additionally a procedure to investigate deaths associated with a given vaccine, but these are unfortunately often inconclusive). This does not mean that the reports are all… winners. As you can see from the summary provided courtesy of the facebook page Start Mandatory Vaccination, some fairly ridiculous things are in VAERS (and you can indeed verify these reports by inputting their ID). For instance, we can probably agree that a 14-year-old boy developing acne is probably not the fault of a vaccine, or that a person’s suicide 28 years after their MMR vaccine probably isn’t because of a vaccine, or a fart that left a yellow stain 6 months post-vaccination probably has nothing to do with the vaccine. Yet you could find VAERS reports for all of these things. VAERS reports have no assessment of causality attached to them, and there has even been a case of someone reporting that a vaccine turned them into the incredible hulk, though VAERS did eventually ask that this case be withdrawn and it was agreed to remove it (however, the report would have remained if the individual filing it had refused the request). Actually, things look pretty grim as far as causal relationships in VAERS report- an expert evaluation of a random sampling found that just ~3% of reports were likely to be causally related to the vaccine. Basically a VAERS report tells you that the event in question happened some time after a vaccine was given (well, allegedly). This isn’t enough to say there’s a causal relationship. For instance, if you have an orange for breakfast one day and then are hit by a car, this would not be enough to say that oranges cause the pedestrians consuming them to be struck by vehicles. But frankly this kind of association is what so many VAERS reports boil down to, and it’s just not good enough to say a vaccine caused it.
Note additionally that there are coding errors. For instance, at the time of writing this, 11 congenital anomalies were reported for COVID-19 vaccines into VAERS. Of these:
Two involved a pregnant patient (957368-1, 932107-1; these reports may in fact be describing the same individual based on how similar they are)
one involved a seventy-year-old patient who expired (932787-1)
one involved a male who described an eye infection (927415-1)
one involved a 57-year-old female who apparently stated the vaccine caused a congenital anomaly while she was not pregnant (907175-1) (yeah I don’t know what this is about)
etc. None of the reports of congenital anomalies associated with these vaccines were reporting an infant born to a vaccinated mother (though of note, even MMR vaccines have been given to pregnant people and despite having live (though attenuated) rubella virus which is known for causing catastrophic birth defects, congenital rubella syndrome was not observed).
A key point to consider is that VAERS has no control groups to go with it. There is generally no known denominator to go with these reports meaning that taken in isolation, even if entirely truthful, do not offer useful information about the safety profile of the vaccine in question. This is why they are used for hypothesis generation- VAERS is not equipped to itself carry out that investigation. It’s intended to focus priorities on the things that are. Of course, this can also mean false alarms. For instance, when the MCV4 vaccine was introduced in 2005, there was a spike in reported cases of Guillain-Barre syndrome. A subsequent investigation showed no increased risk. A more dramatic example occurred when after Wakefield et al published their fraudulent study, a massive spike occurred in reports of autism as an adverse event following vaccination despite there perhaps being no examined link in all of medicine with more negative evidence.
That isn’t to say that VAERS is useless- VAERS does exactly what it’s intended to do reasonably well. VAERS has for example led to the removal of the first-generation rotavirus vaccine after it was deemed to be too unsafe for use in the US. During the clinical trials for the rotavirus vaccine, intussusception was observed in vaccine and placebo groups at similar rates, and not above the background rate of the condition for the study. However, the CDC and American Academy of Pediatrics were concerned about the possibility that the vaccine might rarely cause intussusception so a warning was put in the recommendations for the vaccine’s use and package insert for providers to be aware. Then, after it had been given to about 1 million children, there were 15 reports of intussusception to VAERS, and this was deemed to be above the background rate- the natural rate of the event in question that occurs in the absence of the intervention- which triggered an investigation. A case-controlled trial from the CDC confirmed that there was a small but real increase in the risk of this adverse reaction, and thus the vaccine was pulled from market. Notably, this risk was so small that when it was re-examined in future analyses, it could not be found. Still out of an abundance of caution, deeming that the risks of this vaccine truly did exceed its benefits, the vaccine was removed in the US (it was still used in lower income nations where rotavirus was a much more important driver of childhood morbidity and mortality, demonstrating that the balance of risks and benefits is context-specific). It has since been replaced with newer rotavirus vaccines which have NOT been noted to have this risk (or at least, it happens too rarely for it to produce a detectable increase).
Shimabukuro, T. (n.d.). COVID-19 vaccine post-authorization safety monitoring update. Retrieved February 3, 2021, from Cdc.gov website: https://www.cdc.gov/vaccines/acip/meetings/downloads/slides-2020-12/COVID-04-Shimabukuro.pdf A summary of the monitoring systems put in place for vaccine pharmacovigilance focusing on the COVID-19 vaccines.
Active Surveillance Systems
Plotkin, S. A., Orenstein, W., Offit, D. P. A., & Edwards, K. M. (2017). Plotkin’s Vaccines (7th ed.). Elsevier. Table 82.2. demonstrating an example of how one might go about determining whether or not a causal relationship exists between a vaccine and an associated adverse event. As you might guess, it requires quite a bit of expertise. Firstly one has to have an intimate understanding of the biology of the adverse event and vaccine in question so that they can define an appropriate risk interval. From there, it’s a bunch of hard-core, good old-fashioned epidemiology: you compare the rate in the vaccinated cohort to the background rate to see whether or not there is a true increase in the incidence.
An oft cited claim by those attempting to sow distrust in our pharmacovigilance systems is that the US’s pharmacovigilance for vaccines is inadequate is the underreporting of adverse events to VAERS. This assertion has been addressed elsewhere but:
Our pharmacovigilance is principally concerned with serious and genuine safety problems as its first priority. We do not need reports of everyone’s reactogenicity experience or self-limiting problems as these do not affect the calculus of whether or not the vaccine should remain available to the public or whether or not to modify the manner in which it is given. There are also certain events which are mandated to be reported under the NCVIA law if they occur in relation to vaccination within a certain defined risk interval. The suggestion that VAERS is missing a significant portion of adverse events of special interest or serious adverse events is something of an extraordinary claim in my view. Anyone can file a report and instructions for how to do so are included on the VIS sheets that are mandated by law to be provided to guardians at every vaccination appointment as part of the informed consent process.
VAERS isn’t the only tool we have in our vaccine pharmacovigilance arsenal. There are ways of detecting signals even if no one bothers to go and report them.
If a problem is severe enough, it prompts patients to seek care. Such events are captured through active surveillance systems. In some cases, formal Phase IV trials may be conducted, such as this one. Pharmacoepidemiologists (the professionals who are responsible for pharmacovigilance) tend to rely on large linked databases and medical records to help them with their work. In other countries, single-payer healthcare systems really help to make this easier (some exceptional vaccine pharmacovigilance work has come out of Denmark for example) as everything is reported to a linked database. In the US, the system is not quite as extensive. In 1990 the CDC established the Vaccine Safety Datalink which helps to facilitate this kind of active surveillance, and it comprises a network of several health systems serving a few million patients. Ultimately, it’s a fairly representative sample of the US population, though it’s not a perfect match. VSD sites compile medical data for investigators, who then examine it for safety concerns that may arise from VAERS, the medical literature, changes in the immunization schedule, or the introduction of new vaccines.
Another critical component of our vaccine pharmacovigilance is the CDC’s Clinical Immunization Safety Assessment (CISA) Project, established in 2001. CISA comprises a network of vaccine safety experts from the CDC, seven research institutions, and subject-matter experts who can be consulted regarding the particular immunization needs of patients. In addition to this, CISA has published algorithms for evaluating and managing individuals who have certain vaccine reactions. A list of CISA’s technical publications organized by year can be found here.
Additionally, the Brighton Collaboration was established in 2000 to help with vaccine safety monitoring. It represents an international body of experts who develop case definitions for adverse events that are peer-reviewed, and then qualified with respect to the level of evidence (insufficient, low, intermediate, and highest). Case definitions can be viewed here.
There are also many more active surveillance systems in place that get sort of niche so I won’t make mention of them all here, but I do want to call attention to V-SAFE. V-SAFE is a smartphone app that you can choose to register with which will automatically ask you for updates on your condition following vaccination. It was created recently to help with pharmacovigilance with COVID-19 vaccines. Of course, providing information to V-SAFE is completely voluntary and could be subject to biasing. People are more likely to report significant negative experiences than when nothing important happens, even when the former is much rarer than the latter (though this is more of an issue with VAERS than V-SAFE). Truthfully I am unsure whether it can appropriately be called an active surveillance system because it still requires individuals to opt in and register with it, but it definitely does improve reporting and it’s less passive than VAERS. V-SAFE allows for follow up for adverse events of special interest as well.
Slide courtesy of Dr. Bob Wachter: https://twitter.com/Bob_Wachter/status/1333966348972539904?s=20
IMPORTANT CLARIFICATION PLEASE READ: These values reflect the background rate of these conditions. For any given random sample of 10 million people over a period of 2 months we can expect these quantities of these health outcomes. If they happen to get a vaccine 2 months beforehand, the events will be associated with the vaccination even though they have no relationship. This is why caution is needed in assessing individual events- they may reflect background incidence.
Strom, B. L., Kimmel, S. E., & Hennessy, S. (Eds.). (2019). Pharmacoepidemiology. Wiley. Table 20.2
Strom, B. L., Kimmel, S. E., & Hennessy, S. (Eds.). (2019). Pharmacoepidemiology. Wiley. Figure 20.1
Strom, B. L., Kimmel, S. E., & Hennessy, S. (Eds.). (2019). Pharmacoepidemiology. Wiley. Table 20.2
Pharmacovigilance in Practice
For several reasons, pharmacoepidemiology of vaccines is uniquely complicated. Most public health surveillance systems focus on a single, well-defined phenomenon e.g. the incidence of measles, the levels of lead in the water, but vaccines have to check for multiple phenomena, often simultaneously and in combination. It may require tracking products down to their lot numbers and contacting manufacturers. Further, the determination of causality can be very complicated. I won’t do a detailed discussion here of how this is done here (it’s just too complicated to fit within this blog post), but consider for example the conundrum of a vaccine-triggered adverse event vs. one that is caused by a vaccine. What’s the distinction? From the vantage point of the patient, these are probably indistinguishable: the vaccine was given, the adverse event happened in a well-defined window. But a vaccine-triggered event does not produce excess events in the general population, whereas one that is caused by the vaccine does. For example, older pertussis vaccines were thought to occasionally cause encephalopathy. They were highly reactogenic, provoking intense fevers, and these could produce seizures (these seizures were distinct from the benign febrile seizures children under age 5 sometimes get when they have infections or vaccinations). A close investigation showed that things weren’t as they appeared: the seizures were caused by underlying genetic seizure disorders triggered by the fevers from the vaccine. Unless those children went their entire lives without having had a fever, they would have had those same seizures regardless of whether or not they were vaccinated, and in fact fevers aren’t the only trigger for seizures in Dravet syndrome, their underlying genetic disorder. But it’s easy to see why concerned parents believed that the vaccine because the timing simply followed, and in fact some did call for the pertussis vaccine to be withdrawn. Many factors can bias the interpretation of causality in an adverse event and many factors can confound attempts to investigate. For events caused by vaccines, there will be a detectable increase in the incidence of the event compared with background.
Ultimately, I think the big concern on everyone’s mind right now is the COVID-19 vaccines. What does pharmacovigilance data for them look like? Well, one thing to be mindful of firstly is that data are still being gathered (and will continue to be for as long as the vaccines exist). We know that the vaccines demonstrated excellent efficacy and no safety concerns in the clinical trials, which were large and had appropriate statistical power and represented diverse groups across lifespan and health status. Thus, a priori, we can expect that any potential safety issues that these vaccines may demonstrate would represent very rare phenomena. The Advisory Committee on Immunization Practices (ACIP) had a meeting on January 27, 2021 which went through a great deal of safety data, which are current up to January 16, 2021- about a month of the vaccines’ use in the public. Here are the major observations:
The computed risk of anaphylaxis from the Pfizer vaccine is currently about 5 cases per million or 1 per 200,000 doses of vaccine given. This is much lower than initial estimates had suggested and closer to what we know to be the rate of anaphylaxis from other routinely administered vaccines.
The vaccines are reactogenic. Detailed data are not yet available for the second dose of Moderna because it was introduced a bit later than Pfizer and the data had not yet caught up to it.
V-SAFE is tracking 227 pregnancies in vaccine recipients.
CISA had responded to 143 inquiries about the use of the COVID-19 vaccines
There have been NO safety signals associated with the use of these vaccines. Adverse events are being tracked and all rates are well below the background rate.
There is no evidence of causal links to the vaccine for any death associated with the vaccine thus far.
I think ultimately the point is this: there will never be a scenario where we can totally reduce the risk of an intervention- any intervention- to zero. Pharmacovigilance helps us to minimize the risks to the maximal extent possible, and it has a proven record of work, at times even being too sensitive and showing false positive signals. Our pharmacovigilance systems are complex and specialized, and need to be interpreted with caution. Still, so far, our vaccine schedule, including our COVID-19 vaccines, show no significant safety issues that would warrant modification. Our regulatory bodies have no problem removing a vaccine whose risks exceed its benefits, as they have shown in the past. So let’s dispense with the fear mongering, shall we?
References
ACIP January 27, 2021 Presentation slides. 2021 Jan 28. Cdc.gov. [accessed 2021 Feb 4]. https://www.cdc.gov/vaccines/acip/meetings/slides-2021-1-27-21.html.
Autism and Vaccines. 2021 Jan 26. Cdc.gov. [accessed 2021 Feb 3]. https://www.cdc.gov/vaccinesafety/concerns/autism.html.
bc-coordinator. Templates archives - Brighton Collaboration. Brightoncollaboration.us. [accessed 2021 Feb 4]. https://brightoncollaboration.us/category/pubs-tools/templates/.
Cherry JD. 2019. The 112-year odyssey of pertussis and pertussis vaccines-mistakes made and implications for the future. J Pediatric Infect Dis Soc. 8(4):334–341.
Deer B. 2011. How the case against the MMR vaccine was fixed. BMJ. 342(jan05 1):c5347.
Hall HA. Diving into the VAERS dumpster: Fake news about vaccine injuries. Skepdoc.info. [accessed 2021 Feb 3]. https://www.skepdoc.info/diving-into-the-vaers-dumpster-fake-news-about-vaccine-injuries/.
How to access data from CDC’s VAERS WONDER system. 2015 Mar 2. [accessed 2021 Feb 3]. https://www.youtube.com/watch?v=cOH7cFWS7o4.
Hviid A, Hansen JV, Frisch M, Melbye M. 2019. Measles, mumps, rubella vaccination and autism: A nationwide cohort study. Ann Intern Med. 170(8):513.
Iannelli V. 2021 Feb 2. Are you getting an experimental COVID-19 vaccine? Vaxopedia.org. [accessed 2021 Feb 3]. https://vaxopedia.org/2021/02/01/are-you-getting-an-experimental-covid-19-vaccine/.
Loughlin AM, Marchant CD, Adams W, Barnett E, Baxter R, Black S, Casey C, Dekker C, Edwards KM, Klein J, et al. 2012. Causality assessment of adverse events reported to the Vaccine Adverse Event Reporting System (VAERS). Vaccine. 30(50):7253–7259.
Lu H-L, Ding Y, Goyal H, Xu H-G. 2019. Association between Rotavirus vaccination and risk of intussusception among neonates and infants: A systematic review and meta-analysis: A systematic review and meta-analysis. JAMA Netw Open. 2(10):e1912458.
Martin D. Overview of U.s. fda center for biologics evaluation and research. Fda.gov. [accessed 2021 Feb 3]. https://www.fda.gov/media/93816/download.
MODULE 4 – Pharmacovigilance - WHO Vaccine Safety Basics. Vaccine-safety-training.org. [accessed 2021a Feb 3]. https://vaccine-safety-training.org/pharmacovigilance.html.
Pinkbook. 2020 Dec 17. Cdc.gov. [accessed 2021 Feb 4]. https://www.cdc.gov/vaccines/pubs/pinkbook/rubella.html.
Plotkin SA, Orenstein W, Offit DPA, Edwards KM. 2017. Plotkin’s Vaccines. 7th ed. Elsevier.
Shimabukuro T. COVID-19 vaccine post-authorization safety monitoring update. Cdc.gov. [accessed 2021 Feb 3]. https://www.cdc.gov/vaccines/acip/meetings/downloads/slides-2020-12/COVID-04-Shimabukuro.pdf.
Smith DK, Sadler KP, Benedum M. 2019. Febrile seizures: Risks, evaluation, and prognosis. Am Fam Physician. 99(7):445–450.
Strom BL, Kimmel SE, Hennessy S, editors. 2019. Pharmacoepidemiology. Wiley.
The vaccine adverse event reporting system (VAERS) request. Cdc.gov. [accessed 2021b Feb 3]. https://wonder.cdc.gov/vaers.html
WHO. INFORMATION SHEET OBSERVED RATE OF VACCINE REACTIONS POLIO VACCINES. Who.int. [accessed 2021 Feb 3]. https://www.who.int/vaccine_safety/initiative/tools/polio_vaccine_rates_information_sheet.pdf.
Wood RA, Berger M, Dreskin SC, Setse R, Engler RJM, Dekker CL, Halsey NA, Hypersensitivity Working Group of the Clinical Immunization Safety Assessment (CISA) Network. 2008. An algorithm for treatment of patients with hypersensitivity reactions after vaccines. Pediatrics. 122(3):e771-7.
