Dive Brief:

• Physicians have called for FDA to be more transparent about the selection of comparators for single-arm clinical trials that support medical device approvals.

• In a paper published in JAMA Network Open on Wednesday, the physicians analyzed the evidence used in the authorizations of products under the Humanitarian Device Exemption (HDE) pathway.

• The limited publicly available information on the choice of comparators for single-arm device trials led the physicians to call for changes at FDA.

Dive Insight:

The study, which was supported by a grant from GlaxoSmithKline, looked at 71 HDE filings from the inception of the program in 1996 to 2017, plus a larger body of drug submissions. More than half the HDE filings were excluded from the analysis for a range of reasons, most commonly because the applicant had run a randomized control trial to support their submission. 

That left 32 HDE filings for analysis. As none of the filings featured RCT data, the submissions lacked a direct comparator to show whether a particular effect was likely to be driven by the medical device being tested, requiring the use of baseline values and historical data to show safety and efficacy. 

Despite that, the researchers found FDA documents on more than half of the submissions lacked explicit references to the data used as a control. In around 10% of cases, FDA mentioned the control data but failed to explicitly give information needed to compare it to the experimental device. 

The researchers were able to find control arm data for more than one-third of the filings by searching the literature, reading references cited by FDA and reviewing published manuscripts. In response to that and other findings, the researchers suggested FDA can do a better job of making data available.

"As inferences about treatment effects always rely on comparisons (direct or counterfactual), greater transparency about the basis for choosing (historical) comparators is needed to instill confidence that decisions about drug and device approval based on observational data are appropriate," the study authors wrote.

The part of the study that reviewed data on drug approval filings sought to determine how effective a product needed to be for FDA not to request a confirmatory RCT. However, the researchers were unable to perform that analysis on the device data because FDA never asked for a confirmatory RCT. All the researchers could conclude is that the magnitude of effects seen in the device trials "was similar to the treatment effects noted for the drugs for which no further testing in RCTs was required."