CORE-MD research showcases possibilities of patient-reported outcomes in high-risk device clinical investigations

The latest webinar from the CORE-MD consortium presented the group’s research into how patient-reported outcome measures have been used in clinical evaluations of high-risk medical devices. Researchers reported their findings from systematic literature reviews and a Delphi study of the use of different patient-reported outcome measures – and how patients thought these tools could be made better.

Quick background on what CORE-MD is and what it’s trying to do

• CORE-MD arose out of the E.U. Horizon 2020 research and innovation funding project; it will run through March 2024 on a Horizon 2020 grant. The core aim of the CORE-MD consortium is to provide expert advice to E.U. regulators on the methods used to evaluate high-risk medical devices, with a particular focus on clinical evidence. To do this, the consortium brings together “medical associations, EU regulators, national public health institutes, Notified Bodies, academic institutions, patients’ groups, and health technology assessment agencies, with participation of manufacturers’ trade associations.” The 22 consortium members comprise Member State regulators (e.g., the Irish HPRA, Danish Medicines Agency), healthcare societies (e.g., European Society of Cardiology), academic societies (University of Gothenburg) and trade associations (e.g., BioMed Alliance, Team-NB).
• Currently, the consortium is working through the rigorous clinical evidence requirements of the European medical device regulation (MDR). The group conducted systematic reviews looking at the available clinical evidence and methodologies for the highest risk medical devices and aims to provide advice on new clinical trial designs and the use of real-world data to shore up clinical evidence. A second part of its mission is communication and dissemination of information, part of which it accomplishes via webinars.

The most recent contribution from the consortium was a deep dive into the utility of patient-reported outcomes in medical device regulatory decision-making

• This week, CORE-MD researchers presented their findings on the utility of patient-reported outcomes measures (PROMs) in the clinical evaluation of high-risk medical devices. OLA ROLFSON of the University of Gothenburg and orthopedics department of Sahlgrenska University Hospital emceed the session, joined by JOHN CHAPLIN, associate professor at the University of Gothenburg; YASMIN ZEISL, project coordinator at the European Patients Forum (EPF); and Philip Moons, professor and member of the European Society of Cardiology.
• Quick background on PROMs: According to the U.S. FDA, PROMs are “a measurement based on a report that comes directly from the patient (i.e., study subject) about the status of a patient’s health condition without amendment or interpretation of the patient’s response by a clinician or anyone else.” A type of clinical outcome assessment (COA), PROMs can measure outcomes that are meaningful to a patient that might otherwise not be captured in a clinical study; they are typically designed to capture information related to a patient’s experience with their condition. PROMs inform thinking about a medical device’s benefits and risks, providing insight into measures that are traditionally hard to assess – such as quality of life or overall health status. They may be more general in nature or disease-specific, and many have been validated and used across studies and disease areas.
• A CORE-MD systematic literature review looked at the utility of PROMs in published studies of high-risk medical devices in cardiovascular, orthopedic and diabetes clinical areas from 2000-2023. Chaplin’s presentation noted that researchers used the PRISMA protocol for systematic reviews. The review included randomized, controlled trials (RCTs) and observational studies (English only) with at least three month follow-up, and also included a “manual, backward snowballing search” of the reference articles, focusing on orthopedic and cardiovascular results.
• The research found over a dozen different PROMs used in orthopedic device trials, including both generic tools (i.e., those that were not context-specific) on quality of life (QOL) included EQ-5D and SF-12 (VR-12), and condition-specific tools such as the Western Ontario and McMaster Universities Arthritis Index (WOMAC) and Oxford Knee and Hip Scores (OKS, OHS), among others. In cardiovascular trials, examples of adult generic measures were SF-36, SF-12 and EQ-5D, and examples of adult condition-specific measures included the Kansas City Cardiomyopathy Questionnaire (KCCQ) and the Minnesota Living with Heart Failure Questionnaire (MLHFQ), among others. Notably, the last two have been qualified as medical device development tools by the FDA.
• Regulators are interested in the use of these tools. Chaplin pointed to the FDA’s Medical Device Development Tools (MDDT) system as an example of regulator support, noting that qualified PROMs can be used to support submissions without having to reconfirm the suitability of the tool. The KCCQ has six domains and two summary scores; the FDA qualified the Symptom, Physical Limitation, Social Limitation, and QOL domains , and the Overall Summary Score. The MLHFQ uses a six-point scale to ask patients about the impact heart failure has had over the previous four weeks n 21 different areas, including impact on work, activities of daily living, and social and emotional impacts. The third example was the 18-item Florida Patient Acceptance Survey (FPAS), which uses a five-point Likert scale and asks questions about living with a medical device, including return to function, device-related distress, body image and others.
• PROMs present a wide range of utility – and certain challenges. They can help understand real-world satisfaction, identify adverse events outside of normal clinical visits, augment collection of clinical data, and be used as intermediate endpoints or even in the selection or stratification of clinical study subjects. Challenges to using PROMs in trials include ability to generate sufficient evidence, whether a PROM interpretation guideline is available, and how to prioritize PROM evidence generation. Notably, the European Medicines Agency (EMA) held a multi-stakeholder workshop on the topic in 2022, finding that alignment across regulators and guidance documents were needed. Also released in 2022, the FDA’s guidance on the use of PROMs in evaluating medical devices offered general considerations and best practices for selecting and using a PROM, including evaluating whether the PROM is fit for purpose, and whether it will measure outcomes that matter to patients and that they can understand.

The patient perspective on how PROMs can help evaluate medical devices resulted in five recommendations on best practices for involving patients in research

• Zeisl’s research examined patients’ view of these tools. The research used the Delphi study model and consisted of two consecutive online surveys and two online focus groups to build consensus. The first online survey gathered information on patient experience with medical devices and reactions to a list of PROMs, while the second confirmed those findings and evaluated which PROMs were important to patients. An online focus group worked on PROM selection and prioritization of domains, with a second online focus group validating the findings. The research found that device safety and performance were most important to patients, followed by malfunctions, security, control and replacement of the device, how comfortable the device is, and information on its characteristics – such as noise (e.g., alarms, continuous noise), size, shape, appearance and color.
• PROM knowledge and ideal formats: The researchers found that most patients weren’t familiar with PROMs, and the format and setting for administration matters (for example, whether it was available online or in physical copy, completed at the office or at home). Other important measures were the ability for patients to have choices among devices and different information formats for patient device information, especially the use of videos.
• Five core recommendations for involving patients in research: For researchers, patients and PROMs can capture information important to patients as well as lived experience with a medical device, which can feed into device design improvements. But PROMs need systematic implementation and validated usability to realize their full potential. To this end, Zeisl’s group developed five core recommendations for PROMs. The first is to develop core indicators for PROMs in each disease area that capture relevant patient information and concerns. The second is to clearly communicate to patients how the measures will be used and to report results from the PROM back to the patient, to keep them engaged in the process. The third recommendation is to use the frequency and format patients prefer when getting patient feedback on PROMs, and the fourth is to develop ways to integrate PROMs and patient experience in the regulatory process. The last is to involve patients throughout the device lifecycle and use that feedback, not just for development but in creating patient-facing communication tools and information that will best serve patients.

PROMs can be used to assess quality of care

• Moons reviewed findings published in the European Heart Journal on the use of PROMs in cardiovascular research to assess quality of care, the key outcome of which was a statement from different cardiology organizations, associations and working groups. Common components of PROMs, according to Moons, include reports of symptoms, functional status, QOL, health behaviors and satisfaction with care. They may also have different targets – they can be generic questionnaires, disease-specific questionnaires, and domain-specific questionnaires.
• Moon’s research found 102 disease-specific PROMs, some of which were small instruments for use in specific devices. An example of the latter is the VALIOSA instrument, that measures satisfaction with remote cardiac monitoring and how patients experience that. Two other instruments were directed to left ventricular assist devices (LVADs); these were the LVAD stressor scale and the QOL with LVAD questionnaire. Moons group noted that not all questionnaires were equally valuable – some were well-investigated and supported and some were used in only one or two studies.
• PROMs have a wide range of utility in cardiac device research. A number of cardiovascular-specific PROMs have been developed, and include the Cardiac Event Threat Questionnaire (CTQ), the Cardiac Health Profile (CHP), the Specific Activity Scale and the Cardiac Depression Sale (CDS), among others. The researchers made recommendations for PROMs best practices, separating them out by those used for clinical decisions, quality monitoring, clinical trials, regulatory purposes, reiumbursement purposes, and digital health care.
• Focusing in on the PROMs for regulatory purposes, the authors recommended the development of minimum requirements for these types of PROM tools, such as determination of minimal clinically important differences (MCID), which Moons observed is “very much in line with what CORE-MD is all about.” What became evident through this research was that statistics in clinical trials were not the only important measure of a device’s success, and that the voice of the patient offered a real and lived measure of clinical importance.
• Where PROMs may be useful going forward: Moons pointed to the European guidance on the clinical evaluation of medical devices (MEDDEV 2.7/1 rev 4; this document is currently undergoing revision to align with the EU MDR) and the international standard on medical device clinical investigations (ISO 14155) as places to add discussions on the use of PROMs as part of the arsenal of clinical evidence. Ultimately, the task force recommended the use of a broad range of PROMs to determine which are most appropriate.

The Q&A session teased out additional benefits of existing PROMs and where new ones may be needed

• Do we have the PROMs we need, or do we need better ones for implants? Moons acknowledged that more PROMs are likely needed, because not all aspects of treatments are fully and sensitively addressed for all existing instruments. However, he further noted that care should be taken not to develop additional tools just for the sake of developing them. Chaplin agreed with the approach, expressing surprise that there weren’t more instruments directed specifically at medical devices, and further the unfortunate situation where new tools and questionnaires” are developed and go through long processes of validation, then remain on the shelf” without additional utility. He cautioned that researchers have to take care when using an off-the-shelf instrument for QOL that it actually captures information related to their domains of interest, since QOL covers so many domains.
• PROMs are still used more to evaluate interventions or treatments than to compare different devices, Rolfson observed, speculating that device developers are still in early days of knowing how to use PROMS. Moons noted that the use of PROMs is increasing in certain medical disciplines, like oncology and orthopedics, which could help serve as precedent.
• How long should follow-up continue for specific devices? According to Chaplin, that’s a question that hasn’t been asked scientifically yet, but it will be guided by clinical measures, though QOL measures also be impactful here. He also stressed that data collection from clinical assessments could be separated in time from PROMs data collection of data of PROMs, occurring at different times or even appointments. Rolfson asked Zeisl if her research had turned up information on patient preferences for follow-up duration. Zeisl answered in the negative, but she did stress that neither the patient questionnaires nor the intervals between measures should be too long, due to things like survey fatigue and increased drop-out rates. Moon observed that the follow-up time may depend on device performance. For devices with instant effect, follow-up may be short and may have more frequent assessments. If the device causes an improvement over months, “then of course you can be more relaxed in your timing.”
• What patients think is important may change over time. Zeisl noted, “I think a diabetes patient reported that when we first got the device, of course, safety was really important to them and the functions, but over time as they got used to the device, then their attention shifted towards other things, maybe more lifestyle related aspects that were really important to them, because safety at that point was a given.”
• Why can’t everyone just use SF-36? SF-36 is a 36-question measure of overall health and doesn’t target a specific disease or area of the body. Chaplin noted that a lot of studies do use that standard – it’s used all over the world and it’s been translated and validated. But some feel the questionnaire is too long, at 36 questions; there’s the choice of SF-12, which is shorter but may not be as sensitive as the longer version, and health authorities may expect use of the gold standard SF-36. But, he pointed out, the SF-36 doesn’t cover the condition-specific issues appropriate to some patient areas; additionally, a newer questionnaire than the aging SF-36 might be more appropriate. Another issue is that SF-36 is a fixed questionnaire where all the questions need to be asked, whereas now there are some systems, like the PROMIS [patient-reported outcomes measurement information system] initiative, that allow users to mix questions or ask selected questions to come to the same measurement.
• What’s next? A quick case study from U.S. regulatory experience. A key point of discussion during the CORE-MD webinar was the focus on ensuring that a specific PROM tool is validated within the context of use in a particular study; in short, just because a PROM tool is well established or well validated, that does not automatically mean it is appropriate for specific studies, or to capture specific information. A recent Advisory Committee meeting in the U.S. for a new high-risk cardiovascular device highlighted these complexities. The sponsor, Abbott, had used the KCCQ – a well-established PROM tool that is, as mentioned above, qualified by the FDA as an MDDT and was identified in the CORE-MD review – as a component of its primary composite endpoint, and was also proposing a post-approval study (PAS) that would leverage the KCCQ in longer-term follow up. During the meeting, regulators and advisors had extended discussions about where the KCCQ fit within the broader questions of device efficacy. Attendees observed that use of the KCCQ for regulatory purposes for other devices revealed that using the tool for longer-term follow-up is extremely difficult, so it might not be useful as part of a PAS. This experience underlines many of the issues with which CORE-MD is currently grappling, including validating well-established tools for specific research programs (and purposes within these programs), follow-up, and where PROMs fall in overall considerations for device efficacy and performance.

Featuring previous research by Laura DiAngelo.

To contact the author of this item, please email Corey Jaseph ( cjaseph@agencyiq.com).
To contact the editors of this item, please email Laura DiAngelo ( ldiangelo@agencyiq.com) or Kari Oakes ( koakes@agencyiq.com).

Key Documents and Dates

• Webinar – Patient Reported Outcome Measures for medical devices, broadcast March 4, 2024