Bolt from the blue
The confusing world of adverse events in vaccine trials
At the very end of a 54 page briefing submitted to the USA Food and Drug Administration, there’s an intriguing sentence. The briefing summarises the results of all three phases of clinical trials on Moderna’s Covid-19 vaccine. As you’d expect, it makes for tough reading – sentences stuffed with numbers, acronyms and words like efficacy. Most of the report is taken up with the phase 3 trial – the largest of the trials, with more than 30,000 participants. But the results of the phase 1 and 2 trials are summarised at the end, and it is there that the intriguing sentence sits.
“There were 3 SAEs [serious adverse events] reported in the vaccine group: a 65-year-old participant with community acquired pneumonia 25 days after vaccination, a 72-year-old participant with arrhythmia after being struck by lightning 28 days after vaccination, and an 87-year-old participant with worsening of chronic bradycardia 45 days after vaccination.”
It would be as dry as any other sentence in the report, if not for the study participant who was struck by lightning. There’s no further detail, nothing to tell us anything about the person or how they are doing now. We can imagine that it must have been a terrifying event, a literal bolt from the blue that turned an ordinary day into a medical crisis. We can presume that they survived as, if they hadn’t, their death would have been mentioned. But that’s all we know.
The lightning strike and resulting heart problems are mentioned because they meet the criteria for a “serious adverse event”. Vaccine studies, and indeed all clinical trials, take the reporting of adverse events very seriously. Every health problem that occurs within the time period set out in the methods must be documented and assessed to see whether it could have been related to the vaccine. The researchers have no discretion about what they report – they have to include everything, even injuries from car accidents and lightning strikes.
To understand exactly what was being recorded in this study, or any vaccine study, you need to untangle a few technical terms. So I’m going to take a few steps back from the study participant who was struck by lightning and look at medical trials in more detail.
Humans are, by and large, irrational creatures. We like to think that we can think objectively, but we are often blindsided by biases we didn’t know we had. One such bias is the illusion of causality – thinking that one event has caused another when the two events are actually unrelated. Say, example, you have a headache. If a friend gives you some pills that they tell you will cure it, and your headache goes away after you take them, you’re likely to assume that the pills cured the headache.
That may well be true, of course. But it’s also possible that you were slightly dehydrated, and what helped you was actually the glass of water you drank when taking the pills. Or perhaps it was simply your faith in your friend’s advice that somehow caused your headache to go away. This last scenario may sound fanciful, but it’s actually central to how medical trials are conducted. Certain symptoms – those modulated by the brain, like fatigue, pain and nausea – are responsive to our beliefs about taking medicines. On the one hand, it means that we can get some, but not all, of the benefit of taking a painkiller just by taking a pill with no actual medicine in it – known as the placebo effect. On the other hand, it can also contribute to the side effects we get from some medications. There is evidence that this impact, known as the nocebo effect, is even more important than the placebo effect.
To get around this problem with placebo and nocebo effects, and the illusion of causality, medical trials are conducted in a way that ensures that the participant doesn’t know whether they are getting the trial medication or not. So, one group of study participants gets the trial medication, or the vaccine, and another group gets a treatment that seems identical in every way, such as a pill or injection, except without the active ingredient – known as a placebo. To ensure that there are no significant differences between those two groups, participants are assigned to the treatment and placebo groups randomly, with half of the participants going to each group.
But there’s another bias that is also important in medical trials. It’s called the confirmation bias, and it is our tendency to take notice of evidence that supports what we already believe, while unconsciously disregarding evidence that tells us we are wrong. In a medical trial, this bias is likely to affect researchers, who are usually very keen for the new drug or vaccine to work. To overcome this bias, the people giving participants either the medicine or the placebo don’t know which one that they are giving. Nor do they know when they are measuring how the participant responds.
Such a trial, where neither the participants nor the researchers know who receives what, and the treatments are assigned randomly to participants, is called a randomised double blind trial. It’s an extremely powerful research tool for establishing the effects of a particular treatment on the human body. And it’s a great illustration of scientific method. The reason science works is not because scientists are particularly rational people who are skilled at being objective about solving problems. Science works because it acknowledges that scientists are biased and irrational, just like everyone else. It’s the scientific method itself that overcomes those biases.
In order to ensure that researchers (and medical trial participants themselves) are not biasing the results of safety tests for vaccines, every negative health effect seen during a trial is carefully recorded. Even how participants are questioned is carefully controlled. Casually asking someone if, for example, they noticed any itching at the site where they received an injection, can suggest to them that they should be feeling itchy. And then their brain can make them itch. (I used this example for a reason – do you notice you feel itchy anywhere, right now? If so, I’ve proved that point).
Nonetheless, it can be useful to ask participants if they have noticed specific symptoms, otherwise they might not get mentioned at all. Asking participants about specific symptoms or reactions (known as solicited adverse events) does lead to a higher reporting rate. But in terms of telling the difference between a treatment and a placebo, it may be better to ask general questions (unsolicited or spontaneous adverse events). Therefore, it is common for trials to use both methods. As an example, the trials of the Moderna Covid-19 vaccine asked participants to record whether or not they had specific symptoms (solicited adverse events) for seven days after both the first and second dose of vaccine (or placebo). It also asked more general questions about symptoms (unsolicited adverse events) on days 29 and 57, and will also do so on days 209, 394 and 759.
But the Moderna vaccine is not on the list of vaccines New Zealand has ordered, so I’m not going to give it any more attention here. Instead, I’m going to look at the Pfizer/ BioNTech vaccine, which we have ordered, and which has just been approved for use in New Zealand.
Just as it did for the Moderna vaccine, the US Food and Drug Administration has published the briefing it received for the Pfizer/ BioNTech vaccine. Unlike the briefing on the Moderna vaccine, the phase 1, phase 2 and phase 3 trials are merged together, saving time. The Pfizer/ BioNTech clinical trial was even larger than the Moderna trial, with about 44,000 participants in total but covered a similar time period, up to two months after participants received the second dose. I haven’t been able to find rules on how many participants should be in a clinical trial or how long trials should run. Food and Drug Administration guidelines recommend “thousands” of participants.
While nobody was struck by lightning, one thing that strikes me immediately is how many different types of adverse event were recorded in the study – and the report I looked at was only a summary. There were heart attacks, intestinal obstructions, anaphylaxis, appendicitis, pneumonia, breast cancer , prostate cancer and road accidents. There were also many, many conditions that I’d never heard of which appeared in one or two participants from one or both groups. These examples illustrate the way that clinicals trials are done – the researchers really do record everything.
The problem is, of course, that if you track more than 40,000 people for a couple of months, some of them are going to have health problems. Untangling exactly what, if anything, is due to the vaccine is no simple task, given that the human brain is wired to see cause-and-effect whether it’s there or not. That is why the use of a placebo is so important. For all but one of the medical conditions I listed in the previous paragraph, the number of cases is roughly equal in the groups who received the vaccine and the placebo. The exception was appendicitis, where there were 8 cases in the vaccine group and 4 in the placebo. However even 8 cases among the approximately 20,000 who received the vaccine isn’t above the usual level of appendicitis cases, and the researchers concluded that the cases were unrelated to the vaccine.
But there’s more to the safety data than just a long list of medical conditions suffered by a couple of people in the trial. It’s clear that the Pfizer/ BioNTech vaccine is not without some side effects in the days immediately following vaccination. From those in the later part of the trial, that is the phase 2/3 part of the trial, there were a range of adverse events. More than half of people vaccinated reported pain in the area that they’d received the vaccine. Around half reported fatigue and headaches, although it’s unlikely that all of this was due to the vaccine, as more than a quarter of people receiving the placebo reported fatigue and headaches as well. Other symptoms were reported with reasonable frequency in the vaccine group, but not the placebo group; these included muscle pain, joint pain, chills and fever. In general, the symptoms were more frequent after the second vaccine dose; for example, only 4% of the participants 55 or younger reported a fever after the first dose, while 16% reported a fever after the second. Muscle pain was the most frequently reported symptom that wasn’t also prevalent in the placebo group. In participants 55 or younger, 21% reported muscle pain after the first dose and 37% reported it after the second. The figures were lower for study participants over 55 – 14% reported muscle pain after the first dose and 29% after the second.
What these results tell us is that you do have a reasonable chance of having some side effects if you get the Pfizer/ BioNTech vaccine, and that side effects are more common after the second dose.
The side effects I reported above are the ones that participants were specifically asked about in the days immediately following vaccination – the solicited adverse events. However, participants were also followed up after longer periods – nearly 38,000 were questioned about their symptoms one month after their second dose of vaccine (or placebo). And 19,000 participants were questioned two months after their second dose. On both occasions, the questioning was more general, asking whether people had had any health problems or received any medical treatment; that is, unsolicited adverse events.
The one unsolicited adverse event which turned up with reasonable frequency in the vaccine group (in 0.3% or 64 recipients of the vaccine) but not the placebo group was swollen lymph nodes. In the majority of cases, the swollen lymph nodes occurred within days of receiving the vaccine, in the area near where the vaccine was injected. Swollen lymph nodes are usually a sign that the body is fighting an infection and so are a not-unexpected side effect of vaccination. In most cases, the swelling went down again within a couple of weeks.
From all the information on adverse events, the FDA concluded that the Pfizer/ BioNTech Covid-19 vaccine was “safe and well-tolerated”, and that the reported adverse events “did not suggest any serious safety concerns”.
Although a study of more than 44,000 people is reassuring, millions of people have already been vaccinated, and millions more will be. One thing that a trial of 44,000 cannot do is pick up very rare side effects, the one-in-a million event. The problem, though, is that it’s almost impossible to figure out whether any particularly rare event is linked to the vaccine or not. There were a lot of significant medical situations in the 44,000 study participants, so it’s inevitable that many people out of the millions vaccinated are going to have health problems immediately after the vaccine. And because of the confirmation bias, those who were a bit nervous about taking the vaccine will conclude that the vaccine caused their illness, while those who are convinced that vaccines are one of humanity’s greatest achievements (like me) will probably decide the vaccine wasn’t the cause.
One side effect which must be considered for any vaccine is a severe allergic reaction – anaphylactic shock. Anaphylactic shock is dangerous and fast, although treatable with the right drugs. It’s the reason you have to wait around for 20 minutes after getting your flu shot. For many vaccines, the rate’s about one in a million – which means of course that it does happen and there’s no reason to think that the Pfizer/ BioNTech vaccine will be any different.
One worrying news item about the safety of the vaccine came out of Norway, where an apparent cluster of deaths soon after vaccination was reported in some nursing home patients. Norway is now considering excluding certain groups from vaccination: specifically, those who are terminally ill and highly frail. However, a closer look at the evidence shows that it’s still unclear whether the vaccine was responsible. Those who died were close to death before vaccination. It’s possible that the generally mild side effects were too much for these already very ill people, but it’s also possible that the vaccine didn’t contribute to their deaths at all.
With all that I know about side effects, one thing is clear – if there was no benefit to vaccination with the Pfizer/ BioNTech vaccine, I wouldn’t be lining up for a shot. You might find this point slightly absurd, but it is not. We cannot dismiss that there are risks with vaccination. But the point is that there are risks if we don’t get vaccinated as well. As I’ve said in more than one of my previous articles – Covid-19 is a dangerous disease. It has killed millions of people who would not otherwise have died any time soon.
But testing how well a vaccine works is not a simple matter for a disease like Covid-19. Deliberately infecting people with a virus like Covid-19, which can be unpredictable in its impacts and is not always treatable, is ethically questionable. At a time when there is such great public interest in the vaccine trials, unethical tactics would undermine public trust. Although some trials have looked at deliberately infecting volunteers, the Pfizer/ BioNTech trial did not. Instead, it relied on the high prevalence of Covid-19 in the community and monitored participants to see whether they came down with Covid-19 in the study period. It means that the figure for how well the vaccine works is not based on 44,000 people. It is based on the few hundred who caught the virus during the trial period.
In the Pfizer/ BioNTech vaccine trial, from seven days after the second dose of the vaccine, 178 study participants caught Covid-19. Of those, 9 had received the vaccine and 169 the placebo. Even comparing the data from the whole study period, there were 50 cases who had received the vaccine and 275 who had received the placebo. These numbers indicate that even without the second dose there is a significant benefit. Severe Covid-19 cases were rare in both groups. There were ten severe cases in total. Nine of those were in the placebo group, and one was in the group that had received the vaccine.
There’s plenty of statistical analysis in the report – most of it beyond my ability to understand. But the FDA says that the numbers are good enough, as does Medsafe. This vaccine can make a real difference.
The Turnstone comes out twice a week - a full article on Sunday and some interesting links related to each week’s article on Wednesday. If you have any suggestions for topics or articles which I should highlight, please let me know.
Do you know someone who might enjoy The Turnstone? Please forward it to them.
Did you receive The Turnstone from a friend? You can subscribe and receive it directly every week.
Do you want to see more of The Turnstone? All of the stories can be found in the archive, here.
Have you found a factual error or typo? Let me know and I’ll correct it in the archive version.