Last summer when the data from Public Health England began to look a bit weak on Covid vaccine effectiveness, Sky News ran a segment commenting on some data in an official government report. It had been circulating on social media and fuelling ‘vaccine hesitancy’, according to some people. Perhaps it was time for a mainstream news outlet to reassure the public.

And so Table 5 from a government report that summarised vaccination data from February to July 2021 made it on to telly. The data showed that of the over-50s who caught the Delta variant in the UK, around 13,700 had been vaccinated at least once; around 2,400 had not been vaccinated.

That is “about 85% of those catching the virus being double-jabbed, which is a little higher than one would expect” said the Sky News reporter. But he reassured viewers that what matters is hospitalisations. “Of the vaccinated people, some 3.5% were hospitalised. Of the unvaccinated people, some 8.4% were hospitalised.

By Professor Jem Bendell

In other words, the rate of hospitalisations per case was 2.4 times higher among those who were unvaccinated.”[i] That sounded like a reasonable way of presenting the data. It meant that one might wish to get vaccinated if elderly or within a vulnerable group, in order to halve one’s chances of going to hospital with Covid.

I thought that level of risk reduction is not sufficient to mean that vaccination rates would affect hospital capacity significantly – especially not vaccination for younger generations who rarely end up in hospital anyway. It meant that although Covid vaccinations were not working well for stopping infections, and might soon be ineffective due to viral evolution, so long as they were safe, as far as staying out of hospital was concerned, there was some benefit in the elderly and the vulnerable getting jabbed.

At the time I did not see the benefit in mass vaccination as the jabs were not preventing transmission, so herd immunity from the jab seemed a fantasy.

I was concerned about the policy makers and media focusing everything on vaccines and their demonising the unvaccinated to excuse their theft of our basic civil liberties. But I respected when people said at least the vaccinations reduce hospitalisation. But in recent weeks some people have told me we should all get vaccinated to reduce pressure on our health services. Some say we should be punished for not doing so.

Their rationale is that vaccines reduce hospitalisations by ‘more than 90 percent’. I wondered where they got that statistic from and so looked into the official data. I discovered that yes indeed the latest official report – number 50 from UK Health Security Agency – makes that claim of 90 percent reduced hospitalisation. But I wondered why there is such a discrepancy with the analysis of the basic data, such as that reported by Sky News.

I decided to look more closely at the hospitalisations reduction data in those official reports. I know that most of us who are not specialists in this topic do not have time to look into the statistics or at who produces them. Also, when some of us do make the time, we invite being shamed for moving outside ‘our lane’ or ‘disrespecting experts’. Therefore, as a Professor with a specialism in methodology but absolutely no medical science training, I decided to write up my journey of discovery as I tried to understand the processes behind those big claims on reductions of hospitalisation.

The process was frustrating. For the first days I kept discovering a lack of explanation about how the statistics on big reductions in hospitalisation had been produced. For instance, the UK report #50 does not say anything about how it calculates the figures of over 90 percent reductions in hospitalisation. Nothing! I had to go back months to find some references to the methodology – which I found in report #17.

However, the report did not explain it either and cited another paper for the methodology. I found that study and read it, only to find that it did not explain the methodology and referenced yet another paper for the explanation. As an academic I know this is not uncommon – nearly everyone can be a bit sloppy with their references and provide a reference to a study which does not fully explain what one is asserting.

However, as this was a government report to inform and justify policy on a pandemic that is affecting everyone’s lives, I expected better than what I get from my students. I continued on my quest for the elusive source of the method for producing the statistics on the wonderful vaccine effectiveness. Upon reading that next paper, I found it offered no explanation of rationale for the method! I will say more about each paper and link to them later. But first I want to tell you why this matters…

After following a trail of references cited by each paper, I arrived at an explanation for the statistical method on reduced hospitalisation, from 2013. That paper was the main source for the idea that this particular method gives us credible statistics on vaccine effectiveness. Guess what? No, I am not a conspiracy theorist. I often demonstrate in my research how the power of capital influences much in our lives. So it was not a surprise to me to read that this paper was written by members of the pharmaceutical industry.

It proposed and defended a method for calculating vaccination effectiveness that would inevitably “prove” vaccination effectiveness. I concluded that the exciting claims that there are over 90 percent reductions in hospitalisation due to vaccination are actually instances of our medical authorities misleading the public by using dubious statistical methods from the pharmaceutical industry to prove the effectiveness of their vaccines.

Furthermore, without needing the clinical trials that the medical establishment typically require. If you read on, I will explain how it works. I invite you to investigate it yourself. Or you could decide not to bother and instead just think I should have ‘stayed in my lane’ so you can go back to pretending that pharmaceutical companies care for us, the medical bureaucrats they have trained are critical thinkers defending the public, and that the politicians know what they are doing. But if you don’t look into it, and choose to judge me or others who ask basic questions about the science, then I’m not sure that would demonstrate commitment to either science or public health.

So, here now in greater detail, with the references…

The UK Health Security Agency report #50 refers to a study that “reports on the effectiveness of COVID-19 vaccines on hospitalisation disease with the Delta variant.” The study they referred to is accessible to download and read.[ii] Which I did. Upon reading the paper I discovered that it does not compare numbers of unvaccinated and vaccinated patients in hospital with rates of vaccination in the country. Rather it produces its findings with a completely different methodology called “test-negative case control analysis.” What does that mean? I began looking for explanations of what that method involves and why they choose it instead of simple comparisons like the ones described above and done by Sky News.

Finally, I reached a published paper that gave a simple statement of the method. The test-negative case control approach compares “vaccination status in persons with symptomatic Covid with vaccination status in persons who reported symptoms but had a negative test.”[iii] That is when I had my moment of dumbfoundedness. Why have people who tested negative for Covid in hospital  got anything to do with an assessment of vaccine effectiveness? If someone does not have Covid then there could be any number of reasons why they do not have it. Maybe they have awesome immune systems. Maybe they had Covid before and have natural immunity. Maybe they live in a hermetically sealed bubble and leave home only to have Covid tests. And so on. I was confused. For a minute I wondered if the scientists thought that people who had a negative Covid test would be more likely vaccinated? The paper did not explain. But it did reference another paper saying that is where the methodology is explained.

So I followed the reference trail to another paper – this time in the British Medical Journal.[iv] It did not explain the method either. Instead it referenced two papers as explaining the methodology. I noticed that one paper[v] in the journal Euro Surveillance was from 2013 so 4 years before the other cited paper from the journal Vaccine.[vi] By now my search had become a bit like the search for the holy grail of strangely positive medical statistics and so I wanted to go to the source. So Euro Surveillance was my next download.

Before I read it, I walked around my lounge with my kitten in my arms to do a bit of thinking (it’s one of my techniques). How do they use the data of the negative test results for Covid, I asked my feline friend. For instance, if 2 out of 4 folks with a positive Covid test result are vaccinated and 1 in 4 with negative test results are vaccinated then what does that prove? Or if 3 in 4 with a negative test are vaccinated? Surely none of that proves anything about vaccine effectiveness. Instead, we need to know the levels of vaccination in the wider population. How am I being so dumb not to work it out? My cat enjoyed my unusually long pacing around the room. Break over and back to reading the paper, I finally found the explanation.

They take the ratio of vaccinated to unvaccinated positive cases and divide it by the ratio in the negative cases. Why? The argument is that the ratio in the negative cases is a proxy for the extent of vaccination in society. Which it is clearly not. Instead they could easily use available data on levels of vaccination in society. But no, they prefer to use a ratio of vaccinated to unvaccinated people who show up at a hospital. Such people are unwell enough to either ask for or receive a Covid test, which turns out to be negative. I wondered what would influence whether someone goes to hospital when a bit sick. They must think hospitals are OK. They must think it is OK to go there when they are unwell. They must know where the hospital is and who to contact. They must be able to afford to get there and pay for parking. They must not be so busy that they don’t have time to go to hospital.  Many of them will be people who are thinking about Covid and want to know whether they have it. I wondered what all these factors influencing a hospital visit might correlate with. Likelihood of having been vaccinated against Covid already? If so, might there be disproportionately more vaccinated people getting Covid tests? In which case they might far exceed the number of unvaccinated in the numbers of people who returned negative Covid tests.

In my mind this seemed such a stupid statistical method! Did I miss something? I searched again to find a review article that might comment on the use of this method. Fortunately, I found one that explained things better than ever with a summary table.[vii] It also provided real data which demonstrated what I had imagined happening in practice. Bear with me as I switch into writing about the formulas used in the method. Vaccine effectiveness was being estimated as 1 minus an ‘odds ratio’ for vaccine effectiveness among patients who sought medical care for Covid–like illness and had a SARS-CoV-2 test result. Key is that it is calculated as:

Vaccine effectiveness = 1 – (vaccinated and positive for covid ÷ unvaccinated and positive for covid) ÷ (vaccinated and negative for covid ÷ by unvaccinated and negative for covid). 

The data for these categories was 1−(600÷4000)÷(20,000÷16,000) = 88%. Therefore, from this method researchers created a nice 88% figure for the ‘reduction of hospitalisations due to vaccination’. Lies, damn lies and medical statistics, eh? How could the whole medical establishment be so creative with the stats on something so important? Rather than doing proper science with clinical trials the pharmaceutical companies get the data they desire to promote their products. 

And what if vaccination increases the likelihood of people showing up at hospital in general for whatever cause? That’s not necessarily so, but if it is then that would mean this statistical method would show even higher vaccine effectiveness in reducing hospitalisation. Looking at the research papers that defend this method, they use various ways of adjusting the non-positive data set to try to make it more reflective of the general population. But you can’t adjust one category of information to make it a proxy for a completely different category of information. And just because you have some data sitting on a computer at the hospital doesn’t mean you can credibly repurpose it for whatever will help the pharmaceutical industry avoid clinical trials.

I felt a bit suspicious. That’s something I am working on reducing, as I would like to be more trustful in my life. But this time I decided to sniff around. Because something smelt wrong. I went back to the study that seemed to be the source reference for so many of the recent papers and the government reports: the 2013 study in the spookily named Euro Surveillance journal. Looking again I realised one of its main aims was to promote the novel way of proving vaccine-effectiveness by addressing “concerns about its validity.” Previously I had not paid attention to the authors or their statement of funding or relevant interests. Now I noticed that one of the four authors declared research grants from GSK and Sanofi Pasteur. I quickly checked to reconfirm that GSK is one of the largest pharmaceutical companies in Europe with a large vaccine business for influenza. Sanofi Pasteur is the largest company in the world devoted entirely to vaccines. Of the three other authors two were employees of MedImmune. That’s a research arm of AstraZeneca. All these companies sell – or attempt to sell – vaccines for Covid. Therefore, although the method was invented earlier than 2013, the key article explaining and defending the method as something that can be used instead of clinical trials – and that is referenced by so many of the papers that are referenced in the relevant official reports – was written by employees and grantees of the pharmaceutical companies profiting from the products whose effectiveness are being reported on. Those companies also happen to save money by not needing to do the clinical trials on hospitalisation. 

This is not an isolated case of misleading information coming from a government health authority. The UK Health Security Agency report #50 cites a preprint research paper as evidence of the benefit of Covid vaccinations in reducing hospitalisations. “A study using the SARI watch surveillance system of COVID-19 hospitalisations found high levels of protection against hospitalisation after both a single dose and 2 doses of COVID-19 vaccines” (pg 16).[viii]  However the paper does not show that. The paper shows that vaccinations are very useful for reducing hospitalisation rates in people over 70 years of age, within the first months of receiving the vaccines.[ix] Once again it suggests targeting the vaccinations could be a good idea, so long as people are made aware they might require vaccinations every few months, which some people might opt for.

So why is this issue so important? Because it has played into a wider narrative that has demanded conformity and limited our willingness to consider alternative or complementary approaches. When the data started looking poor for vaccine effectiveness in stopping case numbers, then at least we were told it is near perfect at reducing hospitalisations. That also helped give us the impression that we would have less severe symptoms if we are vaccinated against Covid. It helped us to think it might be the socially responsible thing to do to get vaccinated and reduce the stress on the health services. Well, none of that is true. We have been conned. The pharmaceutical industry invents the standards that the regulators use and then when those standards don’t work for it anymore then they invent new ones like the test-negative method. A few medical professionals raise some polite concerns, expressed as interesting questions. But few of them seem to remember the basic principle that the burden of proof falls upon those who promote, sell and authorise any medical procedure. If there is doubt in a method, then they have failed to deliver on their burden of proof.

So we are back to a simple comparison between vaccination status of people in hospital with the general public. The findings of which – at this time – indicates that if you are vulnerable or elderly then getting vaccinated could be a good idea. However, that hospitalisation data provides no support for the idea of mass vaccinations nor for any coercion of the general public, whether moral or practical. After all, there are many things we could do to halve our risk of hospitalisation from all manner of diseases. We do not (yet) see bureaucrats, pundits and politicians wanting to cut cancer in half by taking away the basic freedoms of people who are not on a diet, nor taking much exercise or who have a drink once in a while. Although cancer is not contagious, it highlights the weakness of arguments to coerce people with the aim of a reduction of hospitalisation in general.[x]

Surprisingly there haven’t been medical scientists in the media challenging the statistical wizardry to produce the stats on hospitalisation reductions. And it is a ridiculous situation that it falls to someone like me, taking time out from my normal work, to follow a trail of research papers to finally uncover statistical fabrication (although perhaps other analysts have found the same and have not been heard). “Trust the science” has been the message from many medical bureaucrats and their champions in mass media. However, a rigorous questioning of science is the very basis of science. That includes a critical unpacking of any statistics. Like anyone, I might make mistakes in some of my analyses. I hope some people who specialise in ‘critical statistics’ will delve deeper into this particular instance of misleading number crunching. For anyone to suggest that such questioning is harmful to public health must not go unchallenged. Instead, any such pious posturing on questions of science itself becomes a danger to good policies for public health. 

What else might the medical bureaucrats have been getting wrong? My reading of this situation is that we need to rescue medical science from the corporations that have captured the bureaucracies and brains of people working in the medical profession. But who is going to do that?

---

[i] https://news.sky.com/story/covid-19-data-shows-vaccines-reduce-risk-of-hospitalisation-and-death-but-you-need-to-read-it-with-a-clear-head-12365728

[ii] https://khub.net/web/phe-national/public-library/-/document_library/v2WsRK3ZlEig/view_file/479607329?_com_liferay_document_library_web_portlet_DLPortlet_INSTANCE_v2WsRK3ZlEig_redirect=https%3A%2F%2Fkhub.net%3A443%2Fweb%2Fphe-national%2Fpublic-library%2F-%2Fdocument_library%2Fv2WsRK3ZlEig%2Fview%2F479607266 also available published at https://www.nejm.org/doi/full/10.1056/NEJMoa2108891

[iii] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8314739/

[iv] https://www.bmj.com/content/373/bmj.n1088.full

[v] De Serres G, Skowronski DM, Wu XW, Ambrose CS. The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials. Euro Surveillance 2013;18:20585. doi:10.2807/1560-7917.ES2013.18.37.20585 pmid:24079398 https://www.eurosurveillance.org/content/10.2807/1560-7917.ES2013.18.37.20585

[vi] Schwartz LM, Halloran ME, Rowhani-Rahbar A, Neuzil KM, Victor JC. Rotavirus vaccine effectiveness in low-income settings: An evaluation of the test-negative design. Vaccine 2017;35:184-90. doi:10.1016/j.vaccine.2016.10.077 pmid:27876198

[vii] https://www.nejm.org/doi/full/10.1056/NEJMe2113151

[viii] https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1041593/Vaccine-surveillance-report-week-50.pdf

[ix] Sharif A. Ismail et al (2021) Effectiveness of BNT162b2 mRNA and ChAdOx1 adenovirus vector COVID-19 vaccines on risk of hospitalisation among older adults in England: an observational study using surveillance data. https://khub.net/documents/135939561/430986542/Effectiveness+of+BNT162b2+mRNA+and+ChAdOx1+adenovirus+vector+COVID-19+vaccines+on+risk+of+hospitalisation+among+older+adults+in+England.pdf/9e18c525-dde6-5ee4-1537-91427798686b

[x] Cut your cancer risk in half | Irish Cancer Society