February 2026 Letter
Bias in RCTs part-3, six persistent research misconceptions, and SURPASS-CVOT
Dear Reader,
I hope you don’t have the February blues. If you do, talk to someone. I cope with the shorter days by getting outside whenever I can, usually on cross‑country skis, letting winter be something I move through rather than resist.
As mentioned in the first letter of 2025, I’ve shifted to monthly emails to keep your inbox a little less crowded. Each letter will touch on several recent posts, with links to the full pieces so you can dive deeper into the topics that interest you most. My aim is to publish consistently on work that helps improve the use of pharmacotherapy through practical clinical pearls by drug class, thoughtful trial summaries and appraisals, and reflections on the scientific methods we use to evaluate drug therapy.
Some of my favorite teaching happens outside the classroom. Alongside supervising graduate students, I spend time each December and January with pharmacy residents in a diabetes clinic, where learning unfolds between patient visits and case discussions. We often circle back to the basics, not because they’re simple, but because they’re easy to forget. One recurring conversation is about bias in randomized trials, especially how evidence can quietly go wrong when outcomes are missing or measured imperfectly. Part 3 of the bias in RCT series focuses on the ways incomplete or mismeasured data can shape conclusions that may not be warranted.
Once a week, the Gamble Lab gathers for journal club. Each graduate student takes responsibility for a different section of a paper, and together we work our way through it slowly and deliberately. Most weeks we focus on original research, but every so often we step back to read pieces about methods or first principles, the kinds of papers that shape how we think rather than what we think. A few weeks ago, that led us to Ken Rothman’s Six Persistent Research Misconceptions,1 published in JGIM more than a decade ago. Despite its age, it felt unexpectedly current, surfacing the same assumptions and shortcuts that still appear in the literature we read today.
Rothman laid out six misconceptions that continue to shape how studies are designed, analyzed, and interpreted, often in ways that obscure rather than clarify truth. They’re worth revisiting, because each one still quietly distorts how evidence is produced and consumed.
1. There’s a pecking order of study designs.
We’re taught that randomized trials sit at the top, with observational studies trailing behind. But validity doesn’t come from the label on the design, it comes from how well the study is done. A sloppy trial can mislead just as easily as a careful observational study can inform.
2. You need a representative sample to generalize results.
This sounds intuitive, but it’s wrong. What allows generalization is whether the causal relationships are sound, not whether the sample mirrors the population. Internal validity matters far more than demographic resemblance.
3. No statistically significant interaction means no real interaction.
A non‑significant interaction term is often taken as proof that factors don’t interact biologically. In reality, statistical interaction depends on model choices and scales, while biological interaction exists regardless of whether a regression model detects it.
4. Quartiles and quintiles are a sensible way to handle continuous data.
Cutting continuous variables into percentiles feels tidy but is usually arbitrary. It throws away information, weakens analyses, and rarely reflects anything meaningful about biology or decision‑making.
5. Multiple comparisons must always be adjusted for.
Automatic adjustment has become a reflex, not a reasoned choice. Whether adjustment is appropriate depends on the question being asked, not on a blanket fear of false positives.
6. Significance testing is central to interpreting results.
P‑values are often treated as the final word on truth. But they say nothing about importance, plausibility, or bias, and their dominance has distracted us from thinking carefully about effect sizes and uncertainty.
What makes these misconceptions durable is that they simplify a difficult task. They offer rules where judgment is required, thresholds where uncertainty is unavoidable. But evidence was never meant to spare us from thinking, it was meant to support it. Revisiting these ideas isn’t about rejecting rigor; it’s about remembering that rigor lives in careful reasoning, not in checklists.
Let’s move on to a recent large cardiovascular outcome trial in the diabetes space. The long awaited SURPASS-CVOT trial was published on December 17, 2025,2 several months after the topline results were presented in September 2025 at the European Association for the Study of Diabetes (EASD) conference in Vienna, Austria!
SURPASS‑CVOT confirms that, among individuals with type 2 diabetes and established ASCVD, tirzepatide delivers cardiovascular safety comparable to dulaglutide during nearly four years of follow‑up. Rates of major adverse cardiovascular events were similar between groups, meaning tirzepatide met the bar for non-inferiority but did not demonstrate clear cardiovascular superiority. In that sense, the trial answers a safety question more than it advances the cardiovascular efficacy story.
Beyond the primary endpoint, tirzepatide showed larger improvements in glycemic control, weight, renal markers, and all‑cause mortality, though these findings should be interpreted cautiously since they were not adjusted for multiple comparisons. Gastrointestinal side effects were more common with tirzepatide, but overall adverse‑event rates were comparable. A key limitation is that dulaglutide was used at a sub‑maximal dose, which may have narrowed or exaggerated apparent differences. Taken together, the trial supports tirzepatide as a reasonable option when cardiovascular safety is a concern, but stops short of establishing it as the clearly better cardiovascular choice.
For more details, see the full post:
That is it for this month’s letter! I am going to try to post more notes on Substack over the next few weeks and continue to post new content directly to the website. Stay tuned for next month’s letter in March!
Peace and kindness,
JM