Modernising vaccine surveillance systems to improve detection of rare or poorly defined adverse eventsBMJ 2019; 365 doi: https://doi.org/10.1136/bmj.l2268 (Published 31 May 2019) Cite this as: BMJ 2019;365:l2268
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Re: Modernising vaccine surveillance systems to improve detection of rare or poorly defined adverse events
Dr Rebecca Chandler’s paper of 31 May was a frank admission of poor pharmacovigilance.
The responses included many thoughtful ones.
May I please make some concrete suggestions? To improve pharmaco-vigilance?
1. In my experience and as a geriatric patient, I have some experience of drugs prescribed, reactions suffered, reactions NOT reported to the Medicines regulator.
The pervasive culture amongst doctors as well as nurses (nurses are now prescribing more drugs than ever before) is, write the prescription, and say good-bye. Before you go to bed tonight, dear prescriber, look at yourself in the mirror, and ask, “ How many drugs did I prescribe today? How many cautions did I give?
Did I ask any patients whether there was a previous history of known or suspected reactions?
It can happen - it has happened - that the doctor is not sure which of a number of prescribed medicines used on one or two successive days was the cause of the reaction. There is no mechanism, it seems, for a doctor faced with say, angioneurotic oedema to report the incident to the Medicines Agency. If he could, there would be a better pharmacovigilance. Of course the doctor would have warned the patient to avoid both drugs in future. But that is of no help to other doctors and other patients who might consult the data sheets.
I do not attribute any ulterior motives to doctors, nurses (both who prescribe and those who administer). It is sheer thoughtlessness. But it should be obvious that when patients are treated as digits, cyphers after a 1, rather than Homo sapiens with similar biology and dare I add similar grey matter, they ought to be taken in to confidence.
Are the Medical schools to blame? Or, is it obsequious leaders of the medical profession who accept the Govt diktats to do or pretend to do more in less and less time?
2. Active Immunisation (vaccination in current parlance) is now performed mostly by nurses and pharmacists. Can the academic and professional leaders of these two professions tell us categorically that:
Their members always inquire about possible contraindications, the degree of immunity snd duration of immunity to be expected, the known side-effects, the suspected side-effects, AND that the recipient should report to the operator any adverse reactions or suspected reactions.
3. I ask Dr Chandler if she regards the above as reasonable ”pharmacovigilance“.
Also perhaps she could tell us whether the WHO programmes live up to my criteria?
4. I also ask Dr Chandler what programmes the WHO is devising to collect, collate, publicise late adverse reactions?
Competing interests: No competing interests
Re: Modernising vaccine surveillance systems to improve detection of rare or poorly defined adverse events
McIntyre et al describe “numerous problems” with my piece, specified as “selective citation of the literature and inappropriate application of pharmacovigilance methods to the complexities of POTS [postural orthostatic tachycardia syndrome]” (1). If the aim of my analysis was to synthesize the existing evidence for a causal relationship between HPV vaccines and POTS, this criticism might be warranted. But the authors have misunderstood the point of my article. It describes the overall structure of vaccine pharmacovigilance and reflects on its ability to meet the existing and future challenges within vaccine safety, using the HPV/POTS signal as an example.
McIntyre et al lay the groundwork for their case claiming that my cited references fail to describe the relevant background of POTS in adolescents. Using several references, they provide an overview of the disease in this population. They note that POTS is found in 25-30% of patients with chronic fatigue syndrome (CFS) which is relatively common in adolescents with a prevalence of 0.5%. Furthermore, POTS is more common in Caucasian females, symptoms typically arise within a few years of puberty, and is likely associated with a genetic predisposition.
In fact, their description of POTS is incomplete. POTS is heterogenous disease with multiple subtypes, associated with other diseases beyond CFS, such as systemic lupus erythematous, sarcoidosis, and diabetes. Although its pathophysiology remains poorly understood, there is growing evidence to suggest an autoimmune pathogenesis. One of the first papers on this subject, published by Vernino et al, described autoantibodies to ganglionic acetylcholine receptors and correlated titres of these antibodies with severity of autonomic dysfunction (from functional gastrointestinal disorders to POTS to idiopathic autonomic neuropathy) (2). More recent publications describe the presence of autoantibodies to adrenergic, muscarinic and angiotensin receptors in patients with POTS (3,4) as well as an HLA association with the DQB1*06:09 allele in a Korean POTS population (5).
McIntyre argues that, although identified by disproportional reporting within an adverse event report database, POTS should not be considered a “signal”, as it would be “expected” in the background population: “As POTS, viewed as a subset of CFS, is common, with onset in early to mid-adolescence and strong female predominance, reports after HPV vaccine are not surprising.”
But McIntyre is wrong to dismiss it in light of the overall adverse event reporting POTS in VigiBase. As of 1 August 2019, the total number of case reports of POTS is 914; 353 of which came from females, ages 12-17. The first reports of POTS entered Vigibase in 2005, a female of unknown age and a 62 year old male, reporting POTS with fentanyl and cevimeline (parasympathomimetic and muscarinic agonist), respectively; the first report in an adolescent female entered in 2008 with sertraline. The logic of McIntyre et al. leads us to expect POTS to be reported as an adverse event for other drugs or vaccines in the female, early to mid-adolescent population. The most common non-vaccine ADR reports for this population are adapalene, isotretinoin, levonorgesterel, amoxicillin; reports for these products total 16,607, and none include POTS as a suspected ADR. Also, there is no evidence of disproportional reporting of POTS in this subpopulation with non-HPV vaccines (such as meningococcal vaccines).
McIntyre proceeds, stating that “carefully validated case definitions are essential to determine the post-vaccination risk of validated adverse events, and initial findings can be overturned by subsequent studies of higher quality” and referencing three studies which have estimated the risk of CFS after HPV vaccination which “in our view… validly evaluate the POTS-HPV signal.” He follows with a short summary of the Norwegian study and reports relative incidence from the UK and the Netherlands.
Given that POTS is a heterogenous disease and only a quarter of CFS patients are known to have POTS, I am not confident that “clinical overlap supports extrapolation”. Furthermore, I have openly agreed with authors from the Mayo clinic that “… if POTS does develop after receiving the HPV vaccine, it would appear to do so in a small subset of individuals and would be difficult to distinguish from the normal prevalence and incidence of the disorder” (6). The absence of statistically significant risk estimates does not exclude causality in small sets of predisposed persons.
A final charge from McIntyre “…selective citation and unjustified, impractical calls for individual risk assessment; both lines of argument reminiscent of broader anti-vaccine tropes” appears to relate to my use of a case report of recurrent GBS after tetanus toxoid vaccination to support a call for elucidation of inter-individual differences in risk for AEFI. A more extensive literature review reveals the existence of an emerging field of vaccinology, coined “adversomics”, based on research suggesting that inter-individual variation in vaccine responses are based on differences in innate immunity, microbiomes, and immunogenetics (7). Furthermore, there are a growing number of publications in the literature describing links between AEFI and individual variation. Beyond narcolepsy, there have been reports on genetic variants associated with an increased risk of febrile convulsions after the measles, mumps, and rubella (8), adverse events after smallpox vaccination (9), BCG and osteitis (10).
In summary, I stand by my choice of the signal of POTS with the HPV vaccine as an example of the need to modernise vaccine pharmacovigilance. POTS is a complex syndrome which is largely under-recognised and undiagnosed by clinicians. Furthermore, similar to narcolepsy, it has only recently been reported into pharmacovigilance databases in association with a vaccine, and progress to understand its pathophysiology suggest it to be an autoimmune disease. A recent systematic review has estimated the impact of HPV immunisation programs and shown evidence of herd immunity (11); therefore, it is reasonable to strive to identify those at risk for AEFI after HPV vaccine if they exist, allowing for improved benefit / harm assessments at the individual level. My analysis should not be viewed as a criticism of the HPV vaccine, but rather as a call to seize the opportunity afforded us to evolve the current paradigm to accommodate recent advances in data science and vaccinology/adversomics.
1. McIntyre PB, Phillips A, Brotherton J, Tatley E. Improving detection of rare or poorly defined adverse events – Analysis poorly grounded in evidence. https://www.bmj.com/content/365/bmj.l2268/rr-11
2. Vernino S, Low PA, Fealey RD, et al. Autoantibodies to ganglionic acetylcholine receptors in autoimmune autonomic neuropathies. N Engl J Med 2000;343:847–855.
3. Fedorowski A, Li H, Yu X, et al. Antiadrenergic autoimmunity in postural tachycardia syndrome. Europace 2017;19:1211–1219.
4. Yu X, Li H, Murphy TA, et al. Angiotensin II type 1 receptor autoantibodies in postural tachycardia syndrome. J Am Heart Assoc 2018;7:e008351.
5. Shin YW, Moon J, Kim TJ Kim DY1, Chang H et al. Human leukocyte antigen associations in postural tachycardia syndrome. Ann Clin Transl Neurol. 2019 Apr 4;6(5):962-967.
6. Butts BN, Fischer PR, Mack KJ. Human Papillomavirus Vaccine and Postural Orthostatic Tachycardia Syndrome: A Review of Current Literature. J Child Neurol. 2017 Oct;32(11):956-965.
7. Poland GA, Ovsyannikova IG, Jacobson RM. Adversomics: the emerging field of vaccine adverse event immunogenetics. Pediatr Infect Dis J 2009;358:431-2.
8. Feenstra B, Pasternak B, Geller F, et al. Common variants associated with general and MMR vaccine-related febrile seizures. Nat Genet 2014;358:1274-82.
9. Reif DM, McKinney BA, Motsinger AA, et al. Genetic basis for adverse events after smallpox vaccination. J Infect Dis 2008;358:16-22.
10. Liehu-Martiskainen M, Korppi M, Teräsjärvi J, Vuononvirta J, Huhtala H, et al. Interleukin 17A gene polymorphism rs2275913 is associated with osteitis after the Bacillus Calmette-Guérin vaccination. Acta Paediatr. 2017 Nov;106(11):1837-1841.
11. Drolet M, Bénard É, Pérez N, Brisson M. Population-level impact and herd effects following the introduction of human papillomavirus vaccination programmes: updated systematic review and meta-analysis. Lancet. Available online 26 June 2019. doi.org/10.1016/S0140-6736(19)30298-3
Competing interests: No competing interests
Re: Modernising vaccine surveillance systems to improve detection of rare or poorly defined adverse events
Improving detection of rare or poorly defined adverse events – Analysis poorly grounded in evidence
The Analysis published on May 31st in the BMJ, opens by linking gaps in “vaccine safety infrastructure” to lapses in public confidence in vaccines, vaccine hesitancy and the re-emergence of measles. However, it quickly becomes apparent this is a smokescreen for the notion of “suspected harm” from HPV vaccine, specifically that investigating links to postural orthostatic tachycardia syndrome (POTS) has been impeded in various ways. (1)
The numerous problems in this piece – among them selective citation of the literature and inappropriate application of pharmacovigilance methods to the complexities of POTS - are not immediately apparent, because on superficial reading the arguments appear well-structured and to raise legitimate questions. Dr Chandler states that POTS is a complex disorder – so far, so good - but the references cited lack relevant background for adolescents. (2-4) First, POTS overlaps chronic fatigue syndrome (CFS) which is common – around 0.5% of adolescents, with common symptoms (chronic fatigue and/or nausea, and/or dizziness, and/or pain) identified as POTS by “intermittent intolerance of upright positions associated with postural tachycardia of more than 40 beats per minute.” (2) Importantly, clinical evidence of POTS is found in 25-50% of CFS cases. (3) Second, symptoms typically arise within a year or two of the beginning of puberty, 70% in girls. (4) Third, about two thirds of POTS patients have headaches, and the most common cause of headache in adolescents is migraine, which shares symptoms with POTS. (2). POTS is more common in Caucasians and around 15% of affected adolescents have a parent or sibling with similar symptoms, suggesting genetic predisposition. (2)
As POTS, viewed as a subset of CFS, is common, with onset in early to mid-adolescence and strong female predominance, reports after HPV vaccine are not surprising. What is surprising is the notion that data mining of the global pharmacovigilance database VigiBase is any more than hypothesis generation. (1) Numerically, VigiBase reports are predominantly from countries with high Caucasian populations where young adolescent females have been targeted for broad HPV vaccine programs (5), and association with fatigue and/or headache and/or syncope inevitable. (2-4) In the analysis of which Dr Chandler is the lead author, among 40,000 reports associated with HPV vaccine, 76% of 694 subjects in four clusters were identified by individual review as “relevant to ongoing safety concerns.” (6) In contrast, analysis of reports to the US Vaccine Adverse Event Reporting System (VAERS) found no signal for POTS and HPV vaccination. Among 160 potential subjects from 40, 735 reports, only 29 fully met POTS criteria and 20/29 had a pre-existing medical condition, CFS in five. (7) It is notable that only two reports came from Australia, which has high HPV coverage and historically high reporting to VigiBase (5), whereas 88 came from Denmark with 20% of Australia’s population. (6)
Carefully validated case definitions are essential to determine the post-vaccination risk of validated adverse events, and initial findings can be overturned by subsequent studies of higher quality, as with venous thromboembolism and HPV vaccination (8). In our view, three studies, not referred to in the Analysis, have design characteristics to validly evaluate the POTS-HPV signal. Although they examine CFS, rather than POTS specifically, clinical overlap supports extrapolation. (2, 3) The methodologically strongest study is from Norway, where the ICD-10 code G93.3 is assigned by paediatricians using specific Norwegian guidelines for CFS, and HPV vaccination status recorded on a national register. (9) Among 176,453 girls born 1997-2002, 82% had at least one dose of 4v HPV vaccine and 407 cases of CFS were identified. HPV vaccine was not associated with CFS during total follow-up (adjusted hazard ratio (aHR) 0.86 (95% CI 0.69-1.08) or the first two years (aHR 0.96; 95% CI 0.64-1.43). Over the study period, reported incidence of CFS increased to a similar extent in boys and girls, despite only girls being eligible for HPV vaccination. Two other studies used the self-controlled case series method to calculate relative incidence (RI) pre and post HPV among CFS cases. No association was found in 187 cases (RI 1.07; 95% CI 0.57-2.00) in the UK (10) or 37 in the Netherlands (RI 0.62; 95% CI 0.07-5.49) (11).
However, lack of epidemiological evidence is insufficient to satisfy Dr Chandler, who argues a search for immune and genetic markers of individual susceptibility is needed. She cites the case of a 42 year old man with GBS after each of three doses of tetanus toxoid as exemplifying individual-level risk. However, a study of 989 vaccines given to people with previous GBS, identified only 6 cases of recurrence, only one tenuously vaccine-exposed (Measles-Mumps-Rubella vaccine 4 months prior). (12) It is difficult to see how a 1978 case report represents the “even-handed look at the evidence”required of a BMJ Analysis.
The final plank of Dr Chandler’s argument is that POTS, and other severe adverse events, are concealed by incomplete clinical trial reporting. Although barriers to accessing clinical trial data are lamentable, retrospective examination of individual trial records is onerous and problematic for non-specific and unmeasured events, such as POTS. What is needed is high quality post-marketing studies (9-11) and well delineated background rates. A prescient study looked at pre-HPV incidence of autoimmune conditions in a US female adolescent cohort, (13) and for POTS was recently reported from Finland. (14)
Understandably, anxiety about debilitating symptoms of unknown cause, such as with POTS, is front of mind while cancer prevention is more distant. However, the basis for a true HPV-POTS association is flimsy, in contrast to strong evidence that HPV programmes have prevented pre-cancerous lesions in many countries (15) and could ultimately eliminate cervical cancer as a public health problem globally. (16)
High quality pharmacovigilance is an essential component of any vaccination program. Unfortunately, the Analysis of May 31st combines selective citation with unjustified, impractical calls for individual risk assessment, both lines of argument reminiscent of broader anti-vaccine tropes. (17) Its publication in a quality journal bestows unwarranted credibility, risking validating unjustified anxieties. Anxieties which, if they take further hold, threaten to deny a generation of young people protection against cervical and other cancers.
1. Chandler RE Modernising vaccine surveillance systems to improve detection of rare or poorly defined adverse events BMJ 2019; 365:l2268 doi:10.1136/bmj.l2268
2. Kzilbash SJ, Ahrens SP, Bruce BJ, Chelimsky G et al Adolescent fatigue, POTS and recovery: A guide for clinicians Curr Progl Adolesc Care 2014; 44:108-133
3. Johnson JN, Mack KJ, Kuntz NL, Brands CK et al Postural Orthostatic Tachycardia Syndrome: A clinical review Pediatr Neurol 2010; 42: 77-85
4. Stewart JM Chronic Orthostatic Intolerance and the postural tachycardia syndrome (POTS) J Pediatr 2004; 145: 725-30
5. Lindquist M VigiBase, the WHO Global ICSR Database system: Basic Facts Drug Information Journal 2008; 42: 409-419
6. Chandler RE, Juhlin K, Fransson J, Caster O et al Current safety concerns with Human Papilloma Virus Vaccine: cluster analysis of reports in VigiBase Drug Saf 2017; 40: 81-90
7. Arana J, Mba-Jonas A, Jankosky C, Lewis P, Moro PL, Shimabukuro TT, et al. Reports of Postural Orthostatic Tachycardia Syndrome After Human Papillomavirus Vaccination in the Vaccine Adverse Event Reporting System. Journal of Adolescent Health. 2017; 61:577-82.
8. Phillips A, Patel C, Pillsbury A, Brotherton J, Macartney K Safety of Human Papillomavirus Vaccines: An updated review Drug Saf 2018; 41: 329-346
9. Feiring B et al HPV vaccination and risk of chronic fatigue syndrome/myalgic encephalomyelitis: A nationwide register-based study from Norway Vaccine 2017; 35:4203-4212
10. Donegan K et al Bivalent human papillomavirus vaccine and the risk of fatigue syndromes in girls in the UK Vaccine 2013; 31: 4961-7
11. Schurink-van’t Klooster TM et al No evidence or an increased risk of long-term fatigue following human papillomavirus vaccination of adolescent girls Vaccine 2018; 36: 6796-6802
12. Baxter R, Lewis N, Bakshi N, Velloni C, Klein NP Recurrent Guillian-Barre syndrome following vaccination Clinical Infectious Diseases 2012; 54:800-4
13. Siegrist C-A, Lewis EM, Eskola J, Evans SJW, Black SB Human Papilloma Virus Immunization in Adolescent and Young Adults A cohort study to illustrate what events might be mistaken for adverse reactions Pediatr Infect Dis J 2007; 26: 979–984
14. Skufca J, Ollgren J, Ruokokoski E, Lyytikainen O, Nohynek H Incidence rates of Guillian Barre (GBS), chronic fatigue/systemic exertion intolerance disease (CFS/SEID) and postural orthostatic tachycardia syndrome (POTS) prior to introduction of human papilloma virus (HPV) vaccination among adolescent girls in Finland, 2002-12 Papillomavirus Research 2017; 3: 91-96
15. Drolet M, Benard E, Brisson and the HPV vaccination impact study group Population-level impact and herd effects following the introduction of human papilloma virus vaccination programmes: updated systematic review and meta-analysis Lancet 2019 http://dx.doi.org/10/1016/S0140-6736(19)30298-3)
16. World Health Organization. Cervical cancer elimination strategy [webpage]. https://www.who.int/cancer/cervical-cancer/cervical-cancer-elimination-s...
17. Wiley KE, Leask J, Burgess MA, McIntyre PB PhD thesis opposing immunization: Failure of academic rigour with real-world consequences Vaccine 2019; 37: 1541-1545
Competing interests: Peter McIntyre was from 2004-2017 Director of Australia's National Centre for Immunisation Research and Surveillance (NCIRS) which receives funding from the Australian and New South Wales governments, including funds to support vaccine safety surveillance activities. NCIRS also receives funding from competitive research grants but no monies from vaccine manufacturers. He was in 2008 a member of a panel convened by the Australian Therapeutic Goods Administration to examine reported cases of multiple sclerosis following HPV vaccination and in 2009 of a panel convened by the NSW government to examine reports of anaphylaxis after HPV vaccination but has not been a member of any advisory groups for manufacturers. He was a member of the Australian Technical Advisory Group on Immunisation (ATAGI) from 1998 to 2017 which considered HPV on multiple occasions over this period. He is a member of World Health Organisation committees and relevant travel costs are paid; his declaration of interests is also available on the WHO website. Statements in this rapid response are his alone in consultation with his co-authors. Julia Brotherton was Medical Director of Australia's National HPV Vaccination Program Register from 2010-2018, operated under contract to the Australian government. She is in receipt of competitive grant funding from Australia's National Health and Medical Research Council, The Australian Research Council, and Victorian Cancer Agency. She has served on non renumerated government technical advisory committee's in relation to HPV vaccines in Australia and has been an external advisor to WHO on HPV vaccines with travel costs reimbursed Anastasia Phillips is a public health physician, and PhD candidate in the area of vaccine safety including HPV with NCIRS, who is also employed by the Western Australian Department of Health. She has no financial interests to declare. Michael Tatley is Director of the New Zealand Pharmacovigilance Centre which contributes reports to VigiBase. The pharmacovigilance centre receives funding from the NZ Ministry of Health but does not receive any funding from pharmaceutical manufacturers.
Can the Uppsala Monitoring Centre Evaluate India-Government Data on Deaths with Pentavalent Vaccine to Restore Public Trust
I thank Dr Rebecca Chandler for her response (1) which is alarming as it is illuminating. From her posting, it is clear that deaths and other serious adverse events following immunisation in third world countries using the WHO-AEFI classification (2) are not recorded in any database for pharmacovigilance. It is as if the deaths of children in low (and middle) income countries are of no consequence.
The WHO-AEFI classification is not used by First World countries. In these countries, adverse-event-reports for drugs and vaccines are maintained within a single database and causality assessment is approached in a similar way for all products using systems like the WHO-UMC causality criteria and the Naranjo algorithm.
Chandler’s response explains why the numerous deaths after the administration of the Pentavalent vaccine (combined diphtheria, pertussis, tetanus, H influenza b and Hepatitis B vaccine) in India and Asia have not been acknowledged as a possible signal for investigation.
In contrast, narcolepsy caused by the N1H1 vaccine - Pandemrix, was noticed and reported by physicians in Sweden and Finland and the regulators confirmed it, based on reports within their national databases of suspected adverse drug reactions.
Utilising data from the Government of India, we have reported that there are 4.7 additional deaths (95% CI: 3.5-5.9) within 72 hour of immunization, per million vaccinated with Pentavalent vaccine compared to children receiving DPT instead (P<0.0001) (3). Using data from states with good reporting of adverse events, we estimate that there are likely to be 7020–8190 additional deaths each year in the country, because of the shift from DPT to Pentavalent vaccine. This is a huge mortality burden.
Is there any way in which the Uppsala Monitoring Centre can call up the data from the Government of India (and other Asian countries where the vaccine is used) and confirm or deny a possible causative association? Only such a transparent appraisal can reassure the public and build trust, and only this will reduce vaccine hesitancy.
If not the Uppsala Monitoring Centre, then who? If not now, then when?
1. Chandler RE. Re: Modernising vaccine surveillance system to improve detection of rare or poorly defined adverse events. BMJ 2019;365:l2268 https://www.bmj.com/content/365/bmj.l2268/rr-8
2. WHO. The Causality assessment of an adverse event following immunization (AEFI): User manual for the revised WHO classification. WHO/HIS/EMP/QSS. March 2013. Available at https://www.who.int/vaccine_safety/publications/aevi_manual.pdf?ua51. Accessed on 1/6/19
3. Puliyel J, Kaur J, Puliyel A, Sreenivas V. Deaths reported after Pentavalent vaccine compared with death reported after diptheria-tetanus-pertussis vaccine: An exploratory analysis. Med J DY Patil Vidyapeeth 2018;11:99-105. Available at http://www.mjdrdypv.org/downloadpdf.asp?issn=2589-8302;year=2018;volume=... Accessed on 6 July, 2019
Competing interests: No competing interests
SOME VACCINE FACTS FROM A CAUTIOUS ADVOCATE:
1. Measles-containing vaccines have been basically safe and effective and have saved many, many lives.
2. If you look at every vaccine on our immunization schedules, and the studies that led to their licensing, it is clear that our knowledge of serious adverse effects is quite limited. VAERS, the US adverse event reporting system typically records fewer than one in a hundred serious adverse events following vaccinations.
3. There is a strong and persistent tendency for vaccine and public health authorities to ignore, gloss over, and sometimes suppress reports of vaccine adverse effects. I shudder when I think of our experience with the whole cell pertussis vaccine before it was finally discontinued in the US in 1998. I think also of the role of injections—including vaccinations—in paralytic polio and, possibly, other polio-like diseases such as acute flaccid myelitis/AFM.
4. We need more consistent data collection regarding long-term vaccine effectiveness. I saw first-hand the decline in H.flu. meningitis after the introduction of Hib, and the decline in pneumococcal meningitis after the introduction of PCV. However, because of pneumococcal serotype replacement the long-term effectiveness of PCV against bad pneumococcal disease is currently uncertain. We have recently learned that the duration of protection from mumps and acellular pertussis vaccines is much shorter than we had assumed. Year in and year out influenza vaccine effectiveness is variable and uncertain. Even with the benefit of the healthy user effect typical of observational studies some influenza vaccines actually increase the risk of illness from influenza infections.
Taken together, the foregoing has led some of us to call for greater flexibility in imposing vaccine mandates in the US. (Opel, etal. Pediatrics 2016;137(4):e20154230. Cunningham. BMJ 2015:351:h4576)
Natasha Crowcroft and her public health colleagues in Toronto think the whole process of vaccine licensure and recommendation should be changed. They worry about the safety, effectiveness, and the cost of newer vaccines. They worry that expanding vaccine schedules threaten the uptake of truly lifesaving and cost-effective vaccines such as measles; it is a poignant concern in light of the current epidemic. Finally, they perceive serious ethical problems in the current approval process, and they suggest that public trust has been undermined by allowing manufacturers and professionals with close links to industry to be involved in lobbying and decision-making. (BMJ 2015;350:h308)
As immunization schedules have expanded, our ignorance of overall risks and benefits has also expanded. Paul Fine and his colleagues point out that the dramatic decrease in vaccine-targeted diseases means that non-specific vaccine effects, adverse or otherwise, assume greater importance than in the past. They have suggested the need for controlled trials which not only evaluate the immunologic benefits of different schedules, but also evaluate longer-term beneficial or adverse non-specific effects. (Fine & Smith. Trop Med Internat Health 2007;12:1)
Meanwhile, US cases of Acute Flaccid Myelitis—labelled a “mystery disease”—continue to accumulate. (CDC Investigation, 29 June 2019)
ALLAN S. CUNNINGHAM 5 July 2019
Competing interests: No competing interests
The comment from Dr Puliyel is acknowledged. His criticism of my use of the examples of intussusception and narcolepsy of the robustness of the “current system” is a fair one without a more detailed description of the differences in AEFI reporting, causality assessment, and signal detection within the global context.
The WHO AEFI causality assessment was developed by the Vaccines Safety Group at the WHO with the support of the Global Advisory Committee on Vaccine Safety. The target user group for this classification system are persons working in countries in whom vaccines are administered via WHO sponsored public health programmes. Those persons are largely concerned with the detection of "signals" of changes in frequency of the more common, expected events which could suggest vaccine quality-related problems, immunisation errors, or multi-use vial contamination, etc. At the current time, most AEFI reports collected and assessed with the WHO AEFI Causality Classification remain within the databases of the public health programmes and are not forwarded into the databases of the national pharmacovigilance centres of most lower and middle income countries.
In contrast, more general guidance for causality assessment, such as the WHO-UMC causality criteria and the Naranjo algorithm, were developed by various groups working within the greater field of pharmacovigilance. The target user groups for these classification systems are those persons working within national pharmacovigilance centres, usually working within or collaboratively with national regulatory centres,and responsible for post-marketing safety surveillance of both drugs and vaccines used within their countries. Within such centres adverse event reports for drugs and vaccines are often maintained within a single database (one notable exception being the USA), and causality assessment is approached in a similar way for all products. Detection of "signals" within the database can be conducted qualitatively (on a "case-by-case" basis) and/or quantitatively (via statistical screening ). Higher income countries which do not rely upon implementation of vaccine administration through WHO public health programmes will handle reports of AEFI through these national pharmacovigilance centres.
Furthermore, it is worth noting that most reports of AEFI contained with Vigibase, the database of individual case safety reports for the WHO Programme of International Drug Monitoring, are from countries who channel reports of AEFI through their national pharmacovigilance system, and therefore most reports within the global database have not been subject to WHO AEFI causality assessment.
Taking the specific example of narcolepsy, reports of this condition in association with Pandemrix, an H1N1 pandemic vaccine, were initially received into the national pharmacovigilance centres of Sweden and Finland, and therefore they were not subject to causality assessment by the WHO AEFI classification system. This signal was detected, in fact, because these clusters of reports in young children were "unexpected" , by both the reporting physicians (based upon their clinical practice) and by the regulators (based upon the expected reporting patterns within their national databases of suspected adverse drug reactions).
The current system referred to as "robust" within this analysis therefore refers to practice of vaccine pharmacovigilance by national pharmacovigilance/regulatory centres, not that of national immunisation centres routinely utilising the WHO-AEFI causality classification system.
Competing interests: No competing interests
Wrong About Post-Marketing Surveillance of Vaccine Adverse Events. Response to Jacob Puliyel, John Stone, Allan Cunningham, etc.
Puliyel (2019) writes: “In the new causality assessment, only reactions that have previously been acknowledged in epidemiological studies to be caused by the vaccine, are classified as a vaccine-product–related-reactions. Reactions observed for the first time during post-marketing surveillance (Phase 4 clinical trial) are not considered as consistent with causal association with vaccine’. All new serious adverse reactions are labelled as coincidental events inconsistent with causal association,’ or ‘unclassifiable’ and the association with vaccine is not acknowledged.
According to the WHO manual (2013) referred to by Puliyel: “The selection of cases for causality assessment should focus on:
“Serious AEFI1 that results in death, is life-threatening, requires inpatient hospitalization or prolongation of existing hospitalization, results in persistent or significant disability/incapacity, or is a congenital anomaly/birth defect;
The occurrence of events above the expected rate or of unusual severity;
Signals generated as a result of individual or clustered cases as these could
signify a potential for large public health impact.
WHO recommends that other AEFI should also be assessed if the reviewing team or review committee decides that causality needs to be determined as a special case or in order to conduct special studies. Such AEFI could include:
AEFI that may have been caused by immunization error (e.g. bacterial abscess, severe local reaction, high fever or sepsis, BCG lymphadenitis, toxic shock syndrome);
Significant events of unexplained cause occurring up to 30 days after a vaccination (and that are not listed on the product label);
Events causing significant parental or community concern (e.g. hypotonic hyporesponsive episode (HHE), febrile seizures).”
I strongly recommend reading either of the WHO manuals (2013;2018). It will be obvious that Puliyel’s claims that post-marketing surveillance doesn’t take new serious adverse reactions seriously is just plain WRONG.
Stone (2019) writes: “It might be interesting to contrast the US's Vaccine Adverse Event Research System which has over the same period accumulated 717,653 reports  (reports are not confirmed cases but neither actually are compensated cases, which are "no fault" awards). Moreover, this is a passive database and as Peter Doshi has observed "inaccessible to most users”.
Cunningham (2019b) writes: “I recently did a VAERS search for "acute flaccid myelitis" (AFM). Their database includes just 4 case reports of AFM since August 2014. The number of cases of acute flaccid myelitis confirmed by the CDC during the same period is 567. This is one more example of the vast underreporting of potential adverse effects following vaccination.”
I went to the Vaccine Adverse Event Reporting System (VAERS) website (CDC. About; see also: VAERS, 2017), then their Request page. First, scroll through the Symptoms list, quite extensive. However, acute flaccid myelitis was not listed. So, I used several other search terms with the following results:
Myelitis = 256
Myelitis Transverse = 381
Paralysis Flaccid = 59
However, there is a gross misconception on how VAERS works and ignoring that it is just one of several surveillance systems. Briefly, VAERS is an underreported system, though studies have found that serious adverse reactions are reported at higher rates (e.g., Rosenthal, 1995). The CDC has teams who monitor VAERS. If there are even a small number of a serious adverse event reported, they investigate. “Although underreporting is a limitation, VAERS is capable of detecting possible safety problems through disproportionality analyses and other methods (Shimabukuro, 2015).” According to Moro: “Signal detection/hypothesis generation . . . Detect new, unusual, or rare adverse events (Moro, 2015, page 17).
The vast majority of the VAERS reports are for minor adverse events, short-lived fever, sore arm, etc (VAERS, 2017). The Vaccine Safety Datalink is a “real-time” link to several large HMOs with membership topping 2 million. Every vaccine, including lot number, child or adults age, gender, comorbidities, and medical problems following vaccinations are available. And there is another project, the Clinical Immunization Safety Assessment (CISA) Project (CDC, 2018).
So, while not super easy to use, VAERS is accessible and Cunningham’s finding of only 4 cases indicates a poor limited search strategy. The CDC uses a wide net of numerous sources, both passive and active. As an aside, Cunningham (2019a) claims vaccines responsible for a number of adverse reactions which I clearly refuted (Harrison, 2019 abcd).
Wendy Stephen (2019) and others criticize that clinical trials and post-marketing surveillance detected narcolepsy as an adverse event to the Pandemrix vaccine; yet, delayed informing the public, etc. This is a rather complicated topic, not as straight forward as Stephen and others would like, so, I will attempt to write a Rapid Response just focusing on this topic.
CDC. About The Vaccine Adverse Event Reporting System (VAERS). Available at: https://wonder.cdc.gov/vaers.html
CDC. Request page. [note need to click “I agree” on About page at bottom, then “I agree” a second time] Available at: https://wonder.cdc.gov/vaers.html
CDC (2018 Oct 19). Vaccine Safety Publications. [check out Vaccine Adverse Events System, Vaccine Safety Datalink, Clinical Immunization Safety Assessment (CISA) Project and CDC Vaccine Safety Publications by Year]. Available at: https://www.cdc.gov/vaccinesafety/research/publications/index.html
Cunningham AS (2019a May 9). Unlimited tolerance of vaccines? BMJ Rapid Responses. Available at: https://www.bmj.com/content/364/bmj.l1481/rr-21
Cunningham AS (2019b Jun 8). Thank you BMJ! BMJ Rapid Responses. Available at: https://www.bmj.com/content/365/bmj.l4044/rr-4
Harrison JA (2019a May 9). Response to Allan S. Cunningham. BMJ Rapid Responses. Available at: https://www.bmj.com/content/364/bmj.l1481/rr-22
Harrison JA (2019b May 16). Response 2 to Allan S. Cunningham. BMJ Rapid Responses. Available at: https://www.bmj.com/content/364/bmj.l1481/rr-29
Harrison JA (2019c May 15). Response 3 to Allan S. Cunningham. BMJ Rapid Responses. Available at: https://www.bmj.com/content/364/bmj.l1481/rr-26
Harrison JA (2019d). Response 4 to Allan S. Cunningham. BMj Rapid Responses. Available at: https://www.bmj.com/content/364/bmj.l1481/rr-33
Moro PL (2015 Oct 15). Signal detection and signal strengthening in CDC’s vaccine safety monitoring systems. CDC Vaccine Safety/VAERS Webinar. Available at: https://health.mo.gov/living/wellness/immunizations/pdf/vaerscisa101515.pdf
Puliyel J (2019 Jun 2). The New WHO Causality Assessment Algorithm Needs Revision to Restore Public Trust. BMJ Rapid Responses. Available at: https://www.bmj.com/content/365/bmj.l2268/rr-0
Rosenthal S, Chen R (1995 Dec). The Reporting Sensitivities of Two Passive Surveillance Systems for Vaccine Adverse Events. American Journal of Public Health; 85: 1706-1709. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1615747/pdf/amjph00450-0108...
Shimabukuro TT, Nguyen M, Martin D, DeStefano F (2015 Aug 26). Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS). Vaccine; 33(36): 4398-4405. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4632204/
Stephen W (2019 May 31). Re: Checking social media for vaccination misinformation: five minutes with . . . Claire Milne. BMJ Rapid Responses. Available at: https://www.bmj.com/content/365/bmj.l2351/rr-2
Stone J (2019 Jun 23). Re: US data on vaccine injury pay-outs show strong immunisation safety record - is this a useful measure? BMJ Rapid Responses. Available at: https://www.bmj.com/content/365/bmj.l4294/rr
VAERS (2017 Oct). VAERS Data Use Guide. Available at: https://vaers.hhs.gov/docs/VAERSDataUseGuide_October2017.pdf
WHO (2013 Mar). CAUSALITY ASSESSMENT OF AN ADVERSE EVENT FOLLOWING IMMUNIZATION (AEFI). Available at: https://www.who.int/vaccine_safety/publications/aevi_manual.pdf?ua51
WHO (2018 Jan). CAUSALITY ASSESSMENT OF AN ADVERSE EVENT FOLLOWING IMMUNIZATION (AEFI) Second Edition. Available at: https://apps.who.int/iris/bitstream/handle/10665/259959/9789241513654-en...
Competing interests: No competing interests
MODERNISING VACCINE SURVEILLANCE SYSTEMS? WE WILL STILL NEED RANDOMIZED TRIALS
I was glad to see the problems with vaccine surveillance acknowledged, but Rebecca Chandler’s article left me feeling confused and uneasy. (BMJ 31 May 2019) Our current system of pharmacovigilance is even more convoluted than I realized, and the methodological innovations she proposes baffle me. How long will it take for the “field of systems immunology” to fully emerge? I am glad that there is movement away from the “one size fits all” approach, but will surveillance be more or less complex and more or less transparent?
It is not reassuring that VAERS and other passive surveillance systems will continue to be relied on. Take Kawasaki disease, for example.
Kawasaki disease first appeared in the 1960s, and its frequency has marched upward right along with the expansion of the immunization schedule. This mysterious immune disorder has been associated with several vaccines in trials and case reports, including hepatitis B, rotavirus, yellow fever, pneumococcal conjugate, influenza, and group B meningococcal vaccines. KD is rare, it is severe, its cause is unknown, and it occurs predominantly in young children who are being frequently vaccinated. Among the host of “adverse events” one would expect KD to be reported frequently to VAERS. What is the record? During the 18 year period from 1990 to 2007 just 88 cases of KD in US children under 5 were reported to VAERS. (Hua, et al. Pediatr Infect Dis J 2009;28:943) During the same period 88 million US children passed through the 0-5 age group; consequently the incidence rate reported to VAERS was 0.10 KD cases per 100,000 person-years. From 1988 to 2006 the reported KD incidence for US children under 5 rose from 11.0 to 20.8 per 100,000 person-years. (Pediatrics 2003;111:448. Pediatrics 2003;112:495. Pediatr Infect Dis J 2010;29:483) Therefore, fewer than one in 100 cases of KD were reported to VAERS from a group of heavily vaccinated children. I was bewildered, therefore, when VAERS data were used to dismiss the 5-fold KD risk that had been associated with Rota-Teq vaccine in vaccine trials. (“Kawasaki Syndrome and Rota-Teq vaccine” CDC 23 March 2010)
Jacob Puliyel’s letter increased my anxiety about Dr. Chandler’s proposed innovations. (BMJ rr 2 June 2019) Dr. Puliyel recently called attention to associations between pentavalent and hexavalent vaccines and SIDS. (Indian J Medical Ethics 2018;3:43. Med J DY Patil Vidyapeeth 2018;20:1)
Vaccine surveillance needs honest reform, not just modernising. The way to restore trust is with reliable knowledge of vaccine effectiveness and adverse effects. This would best be acquired with randomized trials of vaccine schedules as a whole.
ALLAN S. CUNNINGHAM 5 June 2019
Competing interests: No competing interests
Following Jacob Puliyel's comment about narcolepsy and (adjuvant) H1N1 vaccines, someone should note that the signal has not been caught in all European countries: the French pharmacovigilance system missed nearly all adverse events related to pandemrix/focetria (narcolepsy, Guillain-Barré syndrome, relapse of multiple sclerosis, sudden death...).
Observational epidemiological studies represent a weak tool prone to several bias, especially cohort studies with limited time of follow-up, while on the other side delayed adverse events are exactly the type of events which are expected with products like vaccines which are biologically active for years or decades.
Individual causality assessment tools are just not designed to capture delayed adverse events, as what we find in the first WHO causality assessment tool  (which has been withdrawn now), to be classified at least as possible/probable the AEFI should occur within "a reasonable time relationship to vaccine administration" without defining the "reasonable" timeframe.
In the French arm of the VAESCO case-control study assessing the risk of narcolepsy following H1N1 vaccines, the median time of onset after the vaccine administration for exposed cases was 9.8 months and the median time to diagnosis was 11.4 months.  Would those events be classified within a "reasonable" timeframe? What about Hernan's study assessing the link between Hepatitis B vaccines and multiple sclerosis?  The authors had to find the onset date within 3 years after vaccination, and "The mean (median) time between first symptoms and diagnosis was 5.0 (2.7) years" in this study, which is far from being "temporally associated" with the vaccination.
Delayed and irreversible adverse events (which are exactly expected with auto-immune diseases) just don't fit the Challenge-Dechallenge-Rechallenge sequence which is usually considered to be the "gold standard" for individual causality assessment of ADR , and it is still used in the official French method of causality assessment , while it's now known that this sequence generally doesn't work for vaccines (for irreversible and delayed adverse events, the outcome of the method is "doubtful"), except for those who face acute AEs for which the symptoms will be exacerbated in a multiple dose immunization schedule.
For example, in the Gardasil 9 V503-001 clinical trial, several cases of MS have been reported, including one occurring within hours of dose 2 (within the qHPV arm) , others occurring more than one year after the last dose, and none of these cases have been assessed as "vaccine related". In the same review , we learn that an investigator in Denmark "reported three SAEs assessed as vaccine-related: one subject (AN 19756) experienced hypersomnia at more than four years after the third dose of 9vHPV, and two subjects (AN 71508 and AN 71686) experienced postural tachycardia syndrome (POTS) at more than three years after the third dose of 9vHPV and qHPV, respectively."
Also there is a bibliographic reference  given in the same document  about POTS in which "Two patients experienced significant symptomatic exacerbation following a subsequent Gardasil injection, suggesting a positive re-challenge result." And, on the other side, we are told that for the cases identified (and assessed as vaccine-related events) in this V503-001 trial: "Given the lack of temporal association with vaccination of the events reported in the safety update, these events were unlikely related to vaccination."
So when adverse events are close to the vaccination (cases of MS occurring within hours / days / weeks after a dose) they're "not related" and when they're delayed the assessment is also "not related". And we are still waiting for an update about the safety signal between Gardasil 9 and leukemia (as the observed incidence is 6 - 12 times higher than the expected one) identified by the EMA. 
1. WHO. Adverse Events Following Immunization (AEFI): Causality Assessment (2005) Available at http://www.rho.org/files/rb3/AEFI_Causality_Assessment_WHO_2005.pdf
2. Etude NarcoFlu-VF (NarcoFlu VAESCO-France) – Rapport Final [text in French] https://ansm.sante.fr/var/ansm_site/storage/original/application/c900168...
3. Recombinant hepatitis B vaccine and the risk of multiple sclerosis: a prospective study. Neurology. 2004 Sep 14;63(5):838-42.
4. Quelle pharmacovigilance pour les vaccins? Thérapie 2007 Mai-Juin; 62 (3): 241–244 [text in French]
5. Bégaud B, Evreux JC, Jouglard J, Lagier G. [Imputation of the unexpected or toxic effects of drugs. Actualization of the method used in France]. Therapie 1985 Apr;40(2):111–8.
6. FDA - Gardasil 9 – Clinical review – STN 125508/0 – page 67 - https://www.fda.gov/media/90249/download
7. Ibid. - page 147
8. Blitshteyn S. Postural tachycardia syndrome following human papillomavirus vaccination, Eur J Neurol, 2014; 21: 135-9
9. FDA - Gardasil 9 – Clinical review – STN 125508/0 – page 148 - https://www.fda.gov/media/90249/download
10. EMA - Gardasil 9 – Assessment report – EMA/CHMP/76591/2015 – page 114 - https://www.ema.europa.eu/en/documents/assessment-report/gardasil-9-epar...
Competing interests: No competing interests
To the Editor:
In her thoughtful analysis, Dr. Chandler discusses the difficulties in HPV vaccine adverse event pharmacovigilance due to the non-specific nature of the serious dysautonomia symptoms reported to the surveillance agencies .
In our critical review published in 2017, we described that serious adverse event signals were already present in the largest phase III randomized HPV vaccine trials. Nevertheless, these signals were either ignored or minimized by the investigators . Compared to 2871 women older than 25 years receiving aluminum placebo, the group of 2881 women injected with the bivalent HPV vaccine had more deaths in the four year follow-up period (14 vs. 3, p = 0.012). The authors downplayed this statistically significant difference, instead highlighting the fact that a post-hoc unblinded review ruled that no deaths were related to vaccination . A Cochrane meta-analysis confirmed the higher fatality rate in the follow-up period among mid-adult women receiving HPV vaccine (relative risk = 2.36, 95%CI 1.1 to 5.0; participants = 10,737; randomized studies = 3, with no differences between different vaccines. The meta-analysis authors state that the high fatality rate may be a “chance occurrence since there was no pattern either in the causes of death or in the timing of the occurrence of death” . This appears to be a tenuous argument, there is no known chronic disease ending with a similar cause or timing of death. In large randomized trials, the post-hoc unblinded expert opinion should not overrule the hard statistical data.
A safe vaccine should not have an adverse event biological gradient. Nine-valent HPV vaccine has more than twice the virus-like particles and aluminum adjuvant of the 4-valent dose. In the largest HPV vaccine trial (14,149 participants), those individuals receiving the 9-valent dose had more vaccine-related systemic adverse events, 29.5% vs 27.3% (our calculated p value = 0.003), and more serious systemic adverse events (3.3% vs. 2.6%, our calculated p value = 0.01) than those receiving the 4-valent dose . The authors (and reviewers) ignored these crucial statistical differences . Compared to the 4-valent dose, the 9-valent HPV vaccine number needed to seriously harm is 140. (95% CI 79 - 653) .
A similar post- HPV vaccination syndrome of neuropathic pain and dysautonomia has been reported by independent clinicians in different places, circumstances, and times . As described by Chandler, the largest world drug adverse events database (VIGIBASE) received similar dysautonomia symptoms clusters reports after HPV vaccination .
In view of the mounting scientific data questioning HPV vaccine security, implementation of the precautionary principle appears in order.
1. Chandler R. Modernising vaccine surveillance systems to improve detection of rare or poorly defined adverse events BMJ 2019; 365 :12268.
2. Martínez-Lavín M, Amezcua-Guerra L. Serious adverse events after HPV vaccination: a critical review of randomized trials and post-marketing case series. Clin Rheumatol 2017;36:2169-78
3. Skinner SR, Szarewski A, Romanowski B, Garland SM, Lazcano-Ponce E, Salmerón J, Del Rosario-Raymundo MR, Verheijen RH Quek SC, da Silva DP, Kitchener H, Fong KL, Bouchard C, Money DM, Ilancheran A, Cruickshank ME, Levin MJ, Chatterjee A, Stapleton JT, Martens M, Quint W, David MP, Meric D, Hardt K, Descamps D, Geeraerts B, Struyf F, Dubin G (2014) Efficacy, safety, and immunogenicity of the human papillomavirus 16/18 AS04-adjuvanted vaccine in women older than 25 years: 4-year interim follow-up of the phase 3, double-blind, randomised controlled VIVIANE study. Lancet 2014;84:2213–27
4. Arbyn M, Xu L, Simoens C, Martin-Hirsch PP. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst Rev 2018 May 9;5:CD009069. doi: 10.1002/14651858.CD009069.pub3
5. Joura EA, Giuliano AR, Iversen OE, Bouchard C,Mao C, MehlsenJ, Moreira ED Jr, Ngan Y, Petersen LK, Lazcano-Ponce E, Pitisuttithum P, Restrepo JA, Stuart G, Woelber L, Yang YC, Cuzick J, Garland SM, Huh W, Kjaer SK, Bautista OM, Chan IS, Chen J, Gesser R, Moeller E, Ritter M, Vuocolo S, Luxembourg A. A 9-valent HPV vaccine against infection and intraepithelial neoplasia in women. N Engl J Med 2015;372:711–23
6. Blitshteyn S, Brinth L, Hendrickson JE, Martinez-Lavin M. Autonomic dysfunction and HPV immunization: an overview. Immunol Res 2018;66:744-54
7. Chandler RE, Juhlin K, Fransson J, Caster O, Edwards IR, Norén GN. Current safety concerns with human papillomavirus vaccine: a cluster analysis of reports in VigiBase. Drug Saf 2017:40:81–90
Competing interests: No competing interests