The Rule of 3 in critical appraisal

A useful conservative estimate of the upper plausible rate of an event

Good morning! This letter is to help with critical appraisal skills. One specific skill - estimating the upper plausible risk for very rare events!

Check out the following table from the PLATVOC trial evaluating fluoxetine for early symptom management of COVID-19.¹ There are zero events in the 120 patients randomized to fluoxetine; how sure can you be that the risk is low?

The rule of 3 is a useful statistical principle to use when there are no adverse events observed in a study. The rule of 3 and its extensions (see below) apply to meta-analyses as well. The upper 95% confidence interval can be quickly estimated as:

Upper 95% CI ~ 3 / n, where n is the sample size

Therefore, in the table above, even though zero patients experienced dizziness in both groups, this does not mean that the risk is zero. Based on the rule of 3, we can estimate the upper plausible risk to be 2.5% (3/120) in the fluoxetine group.

Here are some further examples:

• If there are 100 patients and zero events observed, the upper rate limit is less than or equal to 3/100 (<=3%)

• If there are 500 patients and zero events observed, the upper rate limit is less than or equal to 3/500 (<=0.6%)

• If there are 1000 patients and zero events observed, the upper rate limit is less than or equal to 3/1000 (<=0.3%)

Caveats for the rule of 3

• Independent observations are assumed. One person does not impact another’s chance of experiencing an adverse event.

• A binomial distribution is assumed. Think about "How likely is it that I get this many successes out of this many tries?"

• Valid only for zero events

• Use cautiously in small datasets or where precise risk estimation matters.

• Alternative methods—exact, score, or Bayesian intervals—often provide better accuracy and coverage. Use exact methods for calculation of precise confidence intervals with a small number of events (e.g. <6 events), especially for smaller sample sizes (e.g. <50).

Extensions to the rule of 3

The method of estimating the upper 95% confidence interval can be used when the number of events is <5 by using a different numerator.

Here is how you can quickly calculate an estimate of the maximum plausible value (i.e., estimated upper 95% CI) when you read <5 events in a study results table.

• 1 observed event then upper 95% CI ~ 5 / n, where n is the sample size

• 2 observed event then upper 95% CI ~ 7 / n, where n is the sample size

• 3 observed event then upper 95% CI ~ 9 / n, where n is the sample size

• 4 observed event then upper 95% CI ~ 10 / n, where n is the sample size

Perhaps we can call these the rule of 5, 7, 9, and 10!

That’s the gist of the skill for estimating upper plausible risks of rare events.

If you want to review more examples, read on; otherwise, I hope this letter helps your appraisal skills! I also explore where the rule of 3 and its extensions may over-estimate or under-estimate the exact risks.

Examples

The tables below are recreated from two RCTs, the CoDEX trial² and the ACTIVE-6 trial.³

You can see for some serious adverse events the number of persons experiencing the event is low. For instance, in the CoDEX trial, how can we estimate the upper 95% CI of risk for catheter-associated UTIs in the dexamethasone and standard care groups?

Using the rule of 3, we can estimate the risk is up to 2.0% (3/148) in the standard care group. To estimate the upper plausible risk for the dexamethasone group we need to use a different numerator as the number of events is 1. The upper 95% CI can be approximated by 5/151 = 3.3%.

Let’s look at one more example for a particular serious adverse event reported in the ACTIVE-6 trial (eTable 2 below).³ The upper 95% CI for Gullian-Barre syndrome can be estimated for the fluvoxamine group at 0.83% (5/601) and 0.49% (3/607) for the placebo group.

Caution in small datasets

It has been pointed out that in smaller datasets of less than 30 or 50, the rule of 3 underestimates the upper 95% CI.⁴ The rule of 3.6 or 3.7 more closely approximates exact methods such as the Clopper-Pearson method.

Let’s compare the upper 95% CIs using samples sizes of 20, 50, and 100 for the rule of 3 and its extensions for 1 through 4 events with an exact method to calculate 95% CIs. The results are shown in the table below.

You can see that the rule of 3+ method can either underestimate or overestimate the upper 95% CI compared with the exact method. For instance, in the case of a sample size of 20 with 0 events, the rule of 3 understimated the upper 95% CI by 1.84%; however, in a sample size of 20 and 3 events, the difference was overestimated by 7.11%.

It is easier to spot a pattern using a figure so I have plotted a couple of figures to help us understand the discrepancies in estimated upper plausible risk by the two approaches.

Figure 1 below the estimated upper 95% CIs across sample sizes ranging from 10 to 500 for both the rule of 3 and its extensions and the Clopper-Pearson exact method. You can see they are very positively correlated.

Figure 1.

Figure 2 below shows the differences (exact minus rule of 3+) by sample size. You can see that that between a sample size of 100 and 200 the lines diverge. However, the rule of 3+ methods start to underestimate the upper 95% CI compared with the exact method, especially with sample sizes less 50 for when the number of events is <2. However, when the number of events is 2 or more, the rule of 3 appears to overestimate the exact values. As the sample size increases, the difference attenuates.

Figure 2.

I hope that provides some additional insights to the rule of 3 and its extensions.

Stay tuned for more methods posts to aid in your critical appraisal toolbox!

1. eClinicalMedicine 2025;80:10306.

2. Tomazini BM et al. JAMA 2020;324(13):1304–1315.

3. Stewart TG et al. JAMA 2023;330(24):2354–2363.

4. Lunbrook J and Lew MJ. ANZ J Surgery 2009;79:565-570.