Dr. Hotez's Data Is Highly Flawed
Maybe that's why he won't debate RFK, Jr.
Podcaster extraordinaire Joe Rogan has proposed a debate between Dr. Peter Hotez, a Baylor University vaccinologist and previous Rogan guest, and Robert F. Kennedy, Jr., a Democratic presidential candidate. Dr. Hotez had been circulating tweets calling RFK, Jr., a dangerous crank. Rogan’s giant media platform offered a good opportunity for Hotez to prove Kennedy wrong. Rogan quickly raised pledges of more than $1 million for Hotez’s favored charities if he joined the conversation. RFK heartily accepted, but Dr. Hotez slinked away, refusing to lower himself, claiming Rogan’s invitation was an act of aggression. Hotez had previously called to “extend federal hate-crime protections” to scientists who face criticism.
While many of RFK, Jr.’s claims about Covid-19 have pierced the lazy narrative with accurate data and science, some of his assertions on this and other topics stretch or even shatter the bounds of what we truly know. But at least he’s willing to debate, learn, and correct.
What about Dr. Hotez? He’s supposedly the expert, according to his credentials.
Over the last several years, however, credentials were often not a good predictor of insight, rigor, or wisdom. That’s why millions of people are eager for more discussion, not less. They believe most legacy media allowed continuous recitations of taking points without asking the central questions. They want to understand the topics more deeply, but they also want to probe the mindset of the experts who they feel let them down. They want accountability.
Others have helpfully and hilariously collated Dr. Hotez’s failed predictions and pronouncements over the last three years (see video at end of post).
Today, I want to look at one sensational claim Dr. Hotez has been making. It does not hold up, and demonstrates yet again why real debates and reconsiderations are so needed.
The Hotez Claim
Dr. Hotez repeatedly claims that the failure of some Americans to get vaccinated against Covid-19 cost 250,000 lives in 2021-22. But wait. Compared to the initial pandemic year of 2020, both Covid mortality and non-Covid mortality rose dramatically in 2021 and 2022 as we administered billions of vaccine doses around the world.1 Nearly every highly vaccinated nation, including Japan, as seen below, shows this unfortunate pattern. Lower income nations with far less state capacity and far few vaccine doses did not similarly suffer.
Dr. Hotez’s new claim presumably relies on a model by Harvard and CDC researchers called “Estimated preventable COVID-19-associated deaths due to non-vaccination in the United States.” That model, in turn, relies entirely on a CDC dataset, upon which it makes fairly simple extrapolations. The quality of the CDC data is thus central to the model’s claims.
The lives lost estimate derives from the CDC’s assertion of vaccine effectiveness. If the vaccines were good at preventing Covid deaths, many would assume a failure to vaccinate some number of people would cost a proportional number of lives. (Among many problems, they are ignoring all-cause mortality and focusing on Covid-only mortality. For now, however, we’ll focus on Covid.)
Here is one version of the CDC data in chart form.
Taken at face value, it looks compelling. But do the underlying CDC data make sense? In short, they do not.
Comparing U.S. data to British data is a simple way to see how wrong the CDC numbers are. Although UK figures aren’t perfect, the unified nature of its National Health Service tends to produce higher quality data for some purposes.
Let’s look at both nations through the lens of a broad seven-month period from August 2021 to February 2022 and a four-week snapshot of March 2022.
UK data show that over the seven-month period, 84.7% of Covid deaths occurred among the vaccinated. In March 2022, that portion rose to 89.5%. When presented with these numbers at the time, many vaccine advocates were shocked. How could such a large portion of Covid deaths occur among the vaccinated? After regaining their composure, they rationalized that Covid mostly kills older people, and most older people are vaccinated, so it kinda makes sense. Yes, that does make sense, although it also undermines the assertion of supreme vaccine effectiveness.
The CDC data from the U.S., however, looks nothing like the UK’s. In the seven-month period, just 29.1% of American Covid deaths occurred among the vaccinated. In March 2022, that portion rose a bit to 42.4%. Still, throughout the two windows spanning eight months, UK vaccinated deaths were twice to three times higher than in the U.S.
Yes, the U.S. had a slightly lower vaccination rate. Over this eight-month period, the UK’s vaccination rate rose from the high 50% range to the low 70s, while the U.S. rate rose from the mid-50s to the high 60s. Yes, the U.S. had a larger Covid wave in the seven-month period, though the UK’s was worse in March 2022. Yes, we need to be aware of Simpson’s paradox, where age confounding is an important factor. Even so, these factors are not nearly big enough to make up for the gulf in effectiveness. Because the U.S. and UK have roughly similar demographics and suffered roughly similar Covid trajectories, these figures should be roughly equal.
What’s behind this disparity?
First, the U.S. definition of a Covid death was even more liberal than in the UK. This over-includes as Covid deaths large numbers of actual non-Covid deaths of younger people, who tend to be less vaccinated.
Second, unlike the UK, the U.S., with its fragmented health systems, is very poor at linking individuals with vaccine records. Although the U.S. tracks the aggregate number of vaccinations and keeps personal records at the state level, some federal authorities end up massively undercounting vaccinated individuals when it comes to cases, hospitalizations, and deaths.
Deb Conrad, a physician assistant and hospitalist in Rochester, New York, helped explain the problem. In August through October of 2021 she checked the accuracy of her hospital’s records. The media was blaring about the “pandemic of the unvaccinated” and that 95-99% of hospitalized Covid patients were unvaccinated. Those numbers were always a fantasy.
As we described in our November 2021 report, some U.S. government agencies already had internal reporting showing the CDC data is incorrect.
In fact, by mid-summer the majority of Americans hospitalized with Covid-19 were vaccinated. A leaked study conducted by Humetrix and the Department of Defense surveyed 5.6 million Medicare patients, most of whom are 65 or older. The study, which looked at a massive sample of the American population, showed that, as of August 7, 2021, 60% of hospitalized Covid-19 patients over 65 were fully vaccinated.
Conrad’s own hospital’s charts listed that fewer than half the patients, not 1-5%, were vaccinated. But even that didn’t match her experience.
For three months, she asked individual patients their status and kept track of the data. In fact, around 83% of the Covid patients were vaccinated. It turned out the hospital didn’t label a person vaccinated if they hadn’t received their shots within the health system. If, for example, they were vaccinated at a Walgreen’s, a drive-up site, or an unaffiliated doctor’s office, they were marked as unvaccinated. Hospitals across the U.S. employed similar protocols.2
For these and other reasons, the U.S. substantially overcounts (undercounts) the numerator of unvaccinated (vaccinated) deaths.
As the UK data shows in broad outline, as the Conrad survey reveals specifically, and as the unpublished Medicare data confirms, the truth was nearly the opposite of the “pandemic of the unvaccinated” narrative.
A Simple Statistical Illusion
Another factor pushing in the same misleading direction is the Two Week Black Hole. The TWBH is the 14-day period (sometimes 21 days, sometimes six weeks) after inoculation when a person is misclassified as unvaccinated until the vaccine has time to work its magic. If someone gets Covid 12 days after vaccination, for instance, unvaccinated gets charged with an infection, and vaccinated avoids a charge – a double mistake on the ledger. If you want to measure the true cost-benefit of a medical procedure, you must begin counting at the moment of the intervention, in this case the needle piercing the skin.
It’s well known, moreover, that with these particular Covid vaccines, people were often more vulnerable soon after receiving the shots due to apparent immune suppression. Record keeping using the Two Week Black Hole, however, renders invisible the infections among the inoculated and sucks all infections and deaths into the unvaccinated category.
British math professor Norman Fenton shows that under such a system of flawed categorization, even a useless placebo achieves brilliant effectiveness at the outset before rapidly falling off, or waning. At which point, maybe you should get a booster. Does that sound familiar? Fenton’s placebo diagrams look eerily similar to the CDC chart seen above, demonstrating an additional way the CDC data is an artifact, not reality. (See more Fenton videos here, here, and here and several papers here and here.)
In fact, such misclassification will generate a host of bizarrely misleading results. For example, even the UK’s own data from the spring of 2021 showed a dramatic spike of non-Covid deaths among the unvaccinated at the very time the vaccine rollout was taking place. How could the administration of a vaccine have any effect on those who did not take it, let alone a striking effect unrelated to Covid? It wouldn’t. The clear explanation for the UK ONS chart below is miscategorization. A substantial number of people who received the vaccine, but were still listed as unvaccinated for several weeks, died soon after inoculation. As the misclassification effect wears off, the lines converge.
Fenton also shows that even the UK, with its somewhat better numbers, substantially underestimates the total unvaccinated population. For example, the ONS for a time asserted that only 8% of adult Brits were unvaccinated, when the real number was closer to 20%.
The UK had thus also been undercounting the total unvaccinated population, a constriction of the denominator, which compounded the error. After being shown the problem, ONS admitted its mistake, and soon after stopped publishing efficacy analyses.
The U.S. makes similar population level mistakes, but because our data is so bad, it’s difficult to tease out with precision. Look at the CDC spreadsheet, though, and you can begin to see this problem with our own denominator.
In the dataset relied upon by Harvard and Hotez, for example, the CDC begins the period of analysis with a total universe of 152.6 million people. By the end of the period, CDC’s total universe is 193.8 million. So it is working with samples of a little less to a little more than half the total U.S. population. The room for error of many different kinds – sampling, assumptions, analysis – is thus vast. Indeed, the populations in the cases and deaths categories are never the same for any week throughout the analysis period. Then, at some point in the time series, the total unvaccinated population becomes a fraction, not even a whole number. It seems like there’s a lot of estimation going on, generating tons of wiggle room.
CDC will not provide the raw data and its sources.
In the end, an inflated numerator (overcounting unvaccinated deaths) and a constricted denominator (undercounting unvaccinated people) produces an illusion of unvaccinated suffering and a giant miscalculation of vaccine efficacy. (We’re not even including the possible large number of adverse events or effects of evolutionary dynamics and immune dysregulation, which hurt the net effectiveness of the vaccines.) Like the claims of “one million lives saved” which we previously debunked, any extrapolations of lives lost based on this dataset must be considered invalid.
Gatekeepers Want More Gatekeeping
Economist Tyler Cowen says Dr. Hotez should not debate RFK because it would be unproductive. We should hash out these technical arguments in peer reviewed academic journals, not popular venues.
Formal analysis and argument is obviously essential. Public discussions can and often do fail to resolve important questions.
What if, however, the formal system of science has broken down? What if the academic journals and health agencies pushed lots of junk science and blocked publication of crucial research? What if the narrow world of academia isn’t serving the public interest? What if the legacy media pushes only one narrative? What if groupthink reigns and they don’t even realize it?
We’ve never relied entirely on either academic or popular debate in isolation. We need both.
Avoiding public interaction avoids accountability. It erodes trust. It also amplifies the experts’ eventual embarrassment by allowing them to luxuriate in error.
Just as important: What if Cowen and Hotez are right and those with heterodox views are wrong? Don’t Cowen and Hotez want to persuade them? What’s more likely to convince half the population who think you are dead wrong – additional one-way didactic broadcasts or two-way settings with give and take? How better to prove you are correct, regain the trust of skeptics, and improve our collective sense-making?
The specific proposed debate is not even the point. Rogan hosting Hotez and RFK is not the only way to air these important topics. The point is: these topics must be aired. Yes, in the literature. But not merely the literature. And not necessarily on the Joe Rogan Experience, but absolutely in dozens of public forums where large portions of the population can see for themselves.
Comments on State Capacity Covidology
The One Million Lives Saved Claim: Part 1
Double Down Hallucination: Part 2
Who Really Wanted to Speed Remedies?: Part 3
Defending Steph Curry: A Computer Model: Part 4
Where Did All the Workers Go?: Part 5
A Narrative That’s Too Big To Fail: Part 6
Mortality Play: 2020 vs. 2021-22: Part 7
Japan Matches Germany's 2022 Mortality Spike: Part 9
Society of Actuaries Shows Continued Young Adult Mortality Spike: Part 10
Dr. Hotez’s Data Is Highly Flawed: Part 11
Covid and the Golly Folly: The blind spot of gee whiz technology futurism: Part 12
This fact also undermines the preposterous models claiming that “the vaccines saved 20 million lives.” We covered those models in previous articles: The One Million Lives Saved Claim and Double Down Hallucination.
UPDATE: Here’s a good find by Chief Nerd. In a January 2024 thread on X, he points to a paper confirming our analysis that hospitals widely mislabel vaccination status. https://x.com/thechiefnerd/status/1749067856665248102
The paper shows 44% of people who were labeled “unknown” by hospitals, and thus presumed unvaccinated for CDC purposes, were in fact vaccinated. https://academic.oup.com/jamiaopen/article/6/2/ooad026/7117831
This exactly corroborates our point.
The issue of regaining trust is fairly separate from the issue of getting the data and analysis right. Debate is a great way to do trust, but a terrible way to deal with data and analysis.
Nice number crunching, all we have had is smoke and mirrors to cover up a nanoparticle bioweapon. Great work CDC, either they are all incredibly stupid or corrupt. Mmmmm I wonder...