The Estimands Framework - a quick primer
Is the ICH Estimands Framework the new PICO?
This is the first letter focused on scientific methods. In general, these ‘methods’ letters will be focused on conceptual principles, as there are many excellent academic papers providing detailed technical information for the methods I will discuss in these letters.
I recently started working with a colleague in Europe who introduced me to the estimands framework and we have started to use it in our research group to design studies. The main idea of this framework is to establish a clear understanding of the treatment effect being evaluated in a study.1 This estimands framework complements the classic PICO framework used in clinical research and evidence-based medicine.
The PICO framework - clarifying clinical research questions
The traditional PICO (Population/Patient/Problem, Intervention, Comparison, Outcome) framework was conceived in the early 1990s as a method to build an effective clinical question to aid in the efficiency and effectiveness of the literature search. The PICO framework is invaluable for the design and conduct of systematic reviews and other clinical focused queries. Variations of this framework emerged including PICOT (added “T” for time), PICOS (added “S” for study design), PEO (population-exposure/experience-outcome), and PECO (population-exposure-comparison-outcome).
Although the PICO framework guides the formulation of clear clinical questions, there often remains uncertainty in the intended treatment effect(s) being measured by a study. This is where the estimands framework’s extends beyond the classic PICO framework. The estimands framework is described in an appendum to the ICH (International Council for Harmonisation) E9 guideline on statistical principles for clinical trials.
Why does this matter? Well, the research question provides some insight into the treatment effect that is being measured. For instance, when comparing a new treatment on the market to conventional treatment, a few different questions could be asked. What is the effect of the starting the new treatment compared to starting the conventional treatment irrespective of whether either treatment is discontinued or rescue medication is required? What is the effect of a new treatment compared to conventional treatment if all patients remained on each treatment without using additional rescue medication? What is the effect of a new treatment compared to conventional treatment while patients were using the treatments and prior to any rescue medication? There are more variations of framing questions around the ‘treatment effect’ and this is why the estimands framework is so useful as it clarifies the precise ‘treatment effect’ being evaluated in a study.
The estimands framework - clarifying intended treatment effects
The purpose of the estimands framework is to guide clinical trial design specifically by providing clarity surrounding the treatment effect being studied. At first this seems academic; however, there are several issues around what can be described as a treatment effect that impact the interpretation of the study results.
A brief review of terminology.
The ‘estimand’ is the specific treatment effect being measured and includes 5 core attributes as described in more detail below.
The term, ‘estimator’, is the statistical method used to compute the summary ‘estimate’ of the treatment effect. A common estimator in the medical literature is the Cox Proportional Hazards model used to compute the hazard ratio or summary estimate of the treatment effect.
Two issues related to the estimand framework that I would like to elaborate on include 1) what are the core elements or attributes of estimands, and 2) what strategies can be used to handle intercurrent events.
Core attributes of estimands
There are 5 core elements of estimands:
Population - who is being studied such as adults with type 2 diabetes with an A1c between 7.5% and 9.0%.
Treatment conditions - the treatment strategies being studied (or comparisons being made). This is often a pharmacologic intervention and placebo (or active comparator). It is important the interventions are precisely defined. For a drug this would include the active ingredient, dosage, dosage form, route of administration, and timing of administration.
Endpoint - the outcome of interest such as the occurrence of a major adverse cardiovascular event or change in systolic blood pressure.
Summary measure - measure of association/effect used to quantify and compare the endpoint between the treatment conditions. Examples include hazard ratio, relative risk, and odds ratio.
Handling of intercurrent events - events that occur after baseline / during the study’s follow-up - more details below.
Strategies for handling intercurrent events
Intercurrent events are post-randomization (i.e., after baseline or cohort entry in a non-randomized study) events which can be either treatment-modifying events or truncating events.
Treatment-modifying events lead to changes in receiving the assigned treatment. Events leading to treatment discontinuation, the use of rescue treatments, treatment switching, and dose-modifications are examples of treatment-modifying intercurrent events.
Truncating events prevent the outcome from occurring. Death precludes any other event of interest! Truncating events are known as competing events in time-to-event analysis.
An important distinction is that missing data or loss to follow-up are not intercurrent events.
The estimands framework requires specification of how intercurrent events will be handled. The different strategies and how intercurrent events are handled are briefly described below along with an example of a research question about HbA1c reduction.
Treatment policy strategy - this is similar to an intention-to-treat approach whereby intercurrent events are considered part of the treatment strategy.
What is the difference in HbA1c reduction at 52 weeks between treatment A and treatment B, irrespective of whether patients stopped or switched treatment or needed rescue medication?
Composite strategy - intercurrent events are combined with the endpoint of interest.
What is the difference in HbA1c reduction or treatment failure (defined as discontinuation or the need for rescue medication) at 52 weeks between treatment A and treatment B?
While-on-treatment / while-alive strategy - considers time prior to intercurrent events and is similar to per-protocol or as-treated analyses.
What is the difference in HbA1c reduction at 52 weeks between treatment A and treatment B while patients were remained on treatment and prior to use of rescue medications?
Hypothetical strategy - considers an imaginary world in which the intercurrent events did not occur.
What is the difference in HbA1c reduction at 52 weeks between treatment A and treatment B if all patients had remained on their assigned treatments without using rescue medication?
Principal stratum strategy - considers those patients unaffected by the intercurrent events.
What is the difference in HbA1c reduction at 52 weeks between treatment A and treatment B in patients who would not discontinue and not need rescue medication?
Closing remarks
In closing, the ICH estimands framework is not exactly the new PICO! Estimands explicitly have a strategy for dealing with intercurrent events and extend beyond PICO in terms of specifying details around the treatment effect being studied. Estimands are most helpful for study design and interpretation of the treatment effects. The PICO framework continues to be an accessible and efficient method for evidence-based clinical queries and knowledge syntheses.
Estimands provide a clear way to articulate the intended treatment effect being studied. It is becoming common place to see the treatment policy estimand language being reported in randomized trials.
In peace and kindness,
JM