EMA concept paper offers litany of concerns for use of single-arm trials

Hot on the heels of a recent FDA draft guidance on the topic, the EMA last week released a new concept paper which explains the regulator’s many, many concerns with the use of single-arm trials – concerns which any sponsor hoping to leverage a single-arm trial will have to successfully navigate.

Background and Context

Global regulators typically consider blinded, randomized controlled trials (RCTs) to be the gold standard of clinical trials. By introducing randomization as to who obtains treatment, blinding knowledge of which treatment is being administered, and controlling for other important factors, sponsors are better able to demonstrate the safety and efficacy of their products, as well as demonstrate the comparative efficacy of their products against an established standard of care.
But RCTs are increasingly being eschewed in some therapeutic areas in favor of so-called “single-arm” trial (SAT) designs, in which there is no control group and all participants obtain access to the investigational treatment.
In general, single-arm trial designs tend to be used most in oncologic settings. For example, according to research published in April 2023 in the Journal of Clinical Epidemiology, of 254 drugs that had been granted accelerated approval between 1992 and 2020, 47% (119) were approved for oncologic conditions using single-arm trial designs. The researchers also found that the use of SATs was increasing over time. As explained recently by the FDA, the use of single-arm designs can make a certain amount of sense in oncology trials, since “malignant tumors do not typically regress on their own,” allowing response rate-based endpoints to be interpreted favorably.
But regulators have also flagged key concerns about the use of single-arm trials, and five concerns in particular. First, single-arm studies typically make use of relatively small safety databases, which may not allow the identification or proper attribution of rare, serious adverse events. Second, the results of a single-arm study may be difficult to interpret since confounding factors were unable to be controlled in the study. Third, an observance of “low magnitude response rates” may not predict clinical benefit of a therapy. Fourth, the contribution of a single agent in a combination therapy may be difficult to establish. And fifth, reliance on historical control data (e.g., natural history data or already-conducted studies) may not be sufficiently similar to allow for interpretation and lead to erroneous conclusions.
“These and other limitations of single-arm trials can add uncertainty to the assessment of the safety and/or effectiveness of a drug such that accelerated approval based on a single-arm trial may not be justified in a given clinical setting,” FDA recently wrote. As a result, the FDA strongly encourages sponsors to make use of RCTs whenever possible, although it by no means prevents companies from making use of single-arm designs. However, the regulator does note that companies may need to run two studies – either sequentially, or concurrently – to confirm the benefit of a product granted approval relying on data from a single-arm trial design.

Recently, FDA published guidance explaining the circumstances under which a single-arm trial would be acceptable for oncology drugs making use of the accelerated approval pathway

The guidance, Clinical Trial Considerations to Support Accelerated Approval of Oncology Therapeutics, was published in March 2023. At the outset, the guidance makes a few key recommendations, including that sponsors consult with FDA review staff on the use of single-arm trial designs, and that regardless of trial design, sponsors must demonstrate that the product provides a “meaningful advantage” over available therapy options to receive accelerated approval.
The first set of considerations in the draft guidance for single-arm trials involve efficacy. Regarding endpoints in oncology trials to support accelerated approvals, “response rate is the most frequently used endpoint to support accelerated approval when the approval is based on data from single-arm trials,” notes the draft guidance, which walks through considerations for selecting the criteria to measure response. In addition to the common endpoint of overall response rate (ORR), sponsors may consider other measures such as complete remission rate or major molecular response; if one of these other endpoints is considered, the sponsor should bring a strong justification to the FDA review team before proceeding. However response is measured, those criteria should also be the ones used in labeling, “whenever possible,” according to the FDA.
In order for any product to receive accelerated approval, it must first demonstrate a “meaningful advantage” over available therapies. In a single-arm trial, sponsors may look to a pre-specified historical trial or trials to show this, with sufficient rationale. The FDA advises that sponsors consider factors such as the trial size and time frame, and demographic and clinical characteristics of participants in selecting a historical trial. For treatments aims at a molecularly defined population, sponsors can show that the magnitude of treatment effect in their “molecularly defined subgroup” bests that seen in the historical trial. The sample size for a single-arm trial should allow for robust statistical analysis and detection of adverse events.
In terms of analyzing single-arm trial data, the FDA will look to the magnitude and duration of response if the efficacy endpoint is response rate; therefore, sponsors have to take into consideration how long it will take to accrue these data in their given disease setting. “In most cases, a minimum follow-up of six months after the response is needed for most of the responders to characterize durability of response,” the FDA notes, although in some cases more time will be required and the agency may ask for additional information on durability of response during its review.
Most single-arm trials are small, so the FDA expects the analysis population to comprise the entire trial population; in any case, both the sample size and the analysis population should be pre-specified. Any patient who has received at least one dose of study drug would be included in the analysis population. To avoid a bias, the FDA directs sponsors to avoid “multiple increases to the study sample size with repeated looks at the data in the absence of a pre-specified plan.” The agency also recommends blinded independent central review, with procedures for adjudication made available as part of the marketing application.

Now the European Medicines Agency (EMA) is following suit, releasing a reflection paper on single-arm trials (SATs) and ways to establish efficacy

As the regulator notes, the purpose of the reflection paper “is to outline the current thinking about SATs that are submitted as pivotal evidence for establishing efficacy in marketing authorization applications.” As with the FDA, EMA considers SATs to be potentially acceptable, although the circumstances of acceptance “strongly depend on the clinical context and other things such as the drug treatment modality.” EMA also recommends that sponsors of SATs justify to the EMA why an SAT is necessary and why an RCT would be infeasible or inadequate.
EMA’s paper also notes – as expected – that it intends to make any assessment of a marketing authorization application “based on the totality of evidence,” which often includes several clinical trials, such as non-pivotal studies. “If results derived from SATs are to be used as pivotal evidence for approval, it is essential that their adequacy is systematically addressed in terms of their characteristics, limitations and remaining uncertainties,” the paper adds.
So, when would an SAT be considered appropriate? In cases where “it can address the targeted estimand of interest,” EMA write, referring to a “precise description of the treatment effect reflecting the clinical question posed by the trial objective.” In other words, it must be able to answer the clinical question of interest. That’s obviously easier said than done, and the paper concedes that there are practical challenges, including isolating the treatment effect of the investigational product. EMA says it wants to see outcomes in an SAT for which the endpoint “could not have occurred without active treatment in any patient who entered the trial.”
Even still, the paper concedes that isolating the treatment effect remains a “theoretical concept which requires detailed knowledge of the clinical context.” EMA will be looking for “qualitative reasoning that leaves no doubt about the causal relationship between the treatment and outcome measured by the endpoint,” and that fully satisfying regulators’ concerns will likely only take place in “exceptional cases.” More commonly, however, the lack of controls or other essential features of an RCT will lead to “residual uncertainty” and concerns about an “erroneous assessment that there is a treatment effect.”

Other key points from EMA’s Concept Paper:

Be certain: Because of the high degree of uncertainty inherent in an SAT due to various factors (patient selection, lack of controls, etc), regulators want to see a “high degree of confidence in the estimates of the size of the treatment effects.”
Choose the right endpoints: Sponsors should be concerned about selecting an endpoint capable of reflecting the “variable capable of providing the most clinically relevant and convincing evidence directly related to the primary objective of the trial.” And for an SAT, the endpoint must also be able to “isolate treatment effects … such that it is known that observations of the desired outcome would occur only to a negligible extent in the absence of an active treatment.” Failure to achieve this will “severely” complicate the assessment of the drug’s efficacy by regulators.
In most cases, avoid time-to-event endpoints: Here, EMA’s paper explains that such events can occur in either the absence or presence of a treatment, and therefore “generally cannot be attributed to treatment,” making them unsuitable for use in SATs unless the event itself cannot occur “at all” without the treatment.
Be wary of continuous endpoints: While such endpoints often allow for precise and sensitive measurements, sponsors need to account for individual variability – the change of any person over time absent any treatment – or even measurement error. “Therefore, a causal attribution of a treatment effect and the size thereof is difficult for continuous endpoints,” EMA wrote.
SATs seem best–suited for binary endpoints – either something happened, or it didn’t – but even then the same caveats apply. Namely, there needs to be an assumption that the thing that happened could not have happened in the absence of treatment.
Choose your trial population carefully: “The assumptions on the natural course of the disease must apply for the trial population in the SAT,” EMA writes. This is somewhat of an impossible ask, the paper concedes, as the population must “not only share the known, but also the unknown characteristics of the patient population.” This, EMA acknowledges, “is impossible to validate” – at least without randomization, which balances out the known and unknown variables between groups. Ideally, regulators would also like to explore heterogeneity across subgroups – something that isn’t generally possible with SATs.
The importance of external information: Often, a single-arm trial will make use of natural history studies or other external controls, including results of other studies. EMA calls this a “crucial design element” that should be pre-specified in the study protocol and include a “precise and a priori definition and description of the control condition(s) to be covered.” The paper explains that, “in exceptional cases, the assessment of efficacy is envisaged to be informed by a direct comparison against external clinical data,” such as an external control; however, that level of specificity is also “beyond the scope” of the paper.
“Unplanned changes to trials are always problematic”: The paper explains that, just like RCTs, SATs are supposed to have pre-specified criteria for how to define the success of the trial, and adherence to the study protocol is of the utmost importance. But unlike for RCTs, because there is no blinding or randomization, post-hoc changes are especially problematic due to concerns about “potential data knowledge and any amendment is considered potentially data driven.”
The paper does contain some surprising and important notes about the types of SATs that it might apply to. For example, EMA notes that it also extends to any study that does not randomize to a control for a formal comparison, even if it makes use of several arms. This would include things like a platform trial, which the document notes is akin to a “series of SATs.”

Analysis

The most confusing part about FDA and EMA’s documents is what took it so long. SATs have been commonly used for years now, as evidenced by the above-mentioned paper demonstrating that 47% of oncology products making use of accelerated approval did so based on single-arm trials. The question, then, is why regulators are deciding to release more guidance – have we reached a tipping point? Are regulators seeing a reduction in trial quality with respect to SATs? Are we poised to see a pull-back in the acceptability of SATs?
To this latter point, it’s worth noting that a key recommendation from both FDA and EMA is for sponsors to discuss their trial designs with regulators before commencing any study, and that RCTs do remain the gold standard.
For sponsors, perhaps the most important thing to note here is that both FDA and EMA are in the draft phases of thinking about SATs. While the scope of these respective documents differ somewhat, they overlap quite a bit. That’s important since clinical data collected in support of a company’s regulatory filing for a drug in one region is generally intended to support market access in other regions as well. Any divergence in ultimate approaches could potentially be problematic from a clinical development standpoint.
A few differences that we see in these documents include: (1) a lack of discussion about how confirmatory studies may play into the use of SATs in the E.U.; (2) Agreement on when SATs are acceptable for use instead of an RCT; (3) The FDA’s assessment of acceptable endpoints within the context of oncology seems more permissive than does the EMA’s, which is somewhat restrictive; (4) EMA’s document seems substantially more comprehensive than FDA’s does.
We will be interested to see how both documents evolve in the months or years to come. EMA’s concept paper is open for comment until September 30, 2023, and an EMA statement said that a final document is planned for release in 2024. FDA’s draft guidance is open for comment until May 26, 2023.

Featuring prior analysis by Rachel Coe
To contact the author of this analysis, please email Alec Gaffney ( [email protected])