Over the past 20 years, there has been a slow shift toward implementing electronic assessments in clinical trials. The arrival of COVID-19 has increased the speed at which trial sponsors are seeking electronic solutions to all aspects of trial data collection. While more time and resources may be required upfront to establish electronic solutions for the collection of assessments, data quality is higher, and, in many cases, overall costs are reduced. PPD provides the leading expertise needed to support the implementation of digital solutions for clinical outcome assessments (COAs) in this challenging new environment.
What is an eCOA?
As defined by the FDA, COAs measure “a patient’s symptoms, overall mental state, or the effects of a disease or condition on how the patient functions,” and they are increasingly used in clinical trials.1 This umbrella term comprises different categories, depending on the objective and user. Patient-reported outcomes (PROs) are generally self-administered (e.g., pain numeric rating scale, the 36-Item Short Form Survey (SF-36), the five-dimensional preference-based health-related quality of life measure (EQ-5D)). Clinician-reported outcomes (ClinROs), are completed by the investigator/clinician as an assessment of the patient’s outcome/status (e.g., New York Heart Association class, Ashworth Spacticity scale). Performance outcome (PerfO) measures assess the ability of the patient to perform discrete tasks, such as the 6-minute walk test or the “get up and go” test. Observer-reported outcomes (ObsROs) are executed by a parent or caregiver who is observing the patient (e.g., Vanderbilt ADHD assessment). When these measures are implemented using an electronic data collection solution, they are collectively known as eCOA.
Within the eCOA space, the greatest focus typically concerns ePROs. There are various terms used when speaking about ePROs — for example, “questionnaire,” “assessment,” and “instrument” — which are generally interchangeable. Some instruments are generic and can be used in multiple contexts, such as the SF-36 or the EQ-5D, while others are developed for a specific trial or clinical population. Such instruments may be freely available or licensed for a fee from the license holder. An instrument may be deployed independently or included within a diary for a patient or caregiver to complete. There are also custom questionnaires or diaries designed by sponsors and contract research organizations (CROs) to fit the specific needs of a given trial. For example, a diary may be created to collect the symptoms, medication intake, and vital signs of a patient.
ObsROs are important when patients are not able to report for themselves (e.g., the patient population is very young) or if the input from both the patients and the caregivers should be collected and compared.
An existing instrument can be deployed as written if it is fit for purpose for the trial needs and written in a way that suits an electronic solution. Historically, many COAs were developed to be completed using pen and paper. Moving an instrument initially developed and validated for a paper and pen formated to an electronic format is commonly referred to as “migrating” the instrument. Depending on how the instrument was written, there may be changes to the instrument that become necessary when migrating from paper to electronic. For example, minor changes might include modifying the instructions, such as changing a direction to “circle” a response to “select.” However, there might be other formatting or wording changes required per eCOA best practices.2 In these cases, a subject matter expert in digital deployment of eCOA, such as the PPD Digital group, should be consulted on the design. Depending on the complexity of these changes and the trial design and objectives, additional testing may also need to be conducted to ensure that the electronic version is equivalent to the original validated paper version. Evidera is a PPD business unit that provides evidence-based solutions to demonstrate the real-world effectiveness, safety, and value of biopharmaceutical and biotechnology products. Experts like the Evidera patient-centred research (PCR) group, who specialize in the development and validation of COAs, should be consulted before migration to discuss the potential challenges and evidence that may be needed when migrating an instrument.
Importance of Clinical Outcome Assessments
It is necessary to understand how treatments improve the health-related quality of life for patients, including his or her experience of specific symptoms that result from the disease. This information is important for demonstrating efficacy but also for making the argument that a product is superior to its competitors. By combining input from the patient with input from observers and clinicians, the data from these outcome assessments provide a panoramic view of the impact and efficacy of the investigational medicinal products (IMPs).
Regulatory agencies recognize the importance of a patient-centered approach to drug development. Within the last few years, the U.S. FDA and the EMA both started taking a much more patient-centric approach to ensure that the patient voice matters. This is evidenced by patient forums hosted by the agencies to discuss issues important to trial participants as a means to understand when, where, and how the patient voice should be incorporated. The agencies look for evidence of the benefits for the patient and whether the changes achieved are meaningful to them, not just the laboratory results.
This shift in regulatory perspective in driving the inclusion of COAs in clinical research has led to a significant evolution within the pharma industry. Today, it is essential to demonstrate that a new drug or device improves patient lives meaningfully to receive approval.
The Challenge with Paper
Collecting paper-based assessments places a huge burden on the patient and their families, particularly when the goal is to capture daily symptoms. Traditionally, tracking daily changes in symptoms required patients to fill out a couple of pages per day of documents or daily diaries and then bring them to the clinic for each visit, at which time they would receive more documents to complete.
If patients forget to fill out the paper-based documents, a number of issues arise. Not surprisingly, this highly manual approach to PROs that patients must complete at home often creates significant problems during clinical trials, particularly associated with compliance. If patients fail to record their daily diary, for instance, they often fill it out based on memory just before returning for their next site visit. This is frequently referred to as the “parking lot effect,” as participants sit in the parking lot before a visit and back-fill all of their assessments. Doing so relies on recall rather than real-time information, which results in the reporting of inaccurate or incomplete information. One study showed an 11% compliance rate of patients completing their paper diaries (when they were supposed to) compared with a 94% compliance rate achieved using an electronic diary.3
With paper-based measures, forms are often completed at actual site visits to ensure that patient answers are provided at the correct time. This places a greater burden on patients to make more frequent site visits or otherwise can drastically reduce the number of assessment time points in the trial.
There are also challenges with managing the data itself. Data recorded on paper must be converted into some type of digital format for aggregate analysis, a process that is highly error-prone. For instance, study teams may be left to interpret abstract responses owing to a lack of clarity in the markings on the paper, leading to the entry of incorrect data. Additionally, patients might miss or skip answering some questions, leading to data gaps. It is very difficult to retrospectively address these issues after the designated assessment time has passed.
Technology Solves Common Pitfalls
Digital technologies can address many of the issues associated with paper-based collection of outcome assessments.4,5 Date and time stamps are included in each assessment to report when the entry has been completed. Assessments are available for the participants only when appropriate (e.g., one diary entry per day to collect the last 24 hours of recall). Pre-defined specification ranges for certain fields (e.g., age) enable automatic data checks. Assessments are designed with branching logic, leading the participant through the assessment based on the answers to each question. The participants must answer all questions before moving to the next screen and completing the entry, supporting a fully completed assessment.
In addition to the eCOAs themselves being designed to collect quality data, many solution providers offer other tools to support success for participants. Alerts and reminders can be sent to participants when it is time for them to fill out their electronic diaries. Key information about their study participation can be available on the electronic application they use for the study. Other tools, such as eConsent or televisits, can also be supported in a single application, making it a robust solution for the participant, who can become acclimated to using the eCOA application frequently.
When eCOAs were first introduced, study teams initially thought that electronic solutions would be too expensive to implement, despite paper COAs providing poor data quality and requiring additional time for data entry into electronic data capture (EDC) systems. Proof of migration from paper to digital systems has been crucial to the widespread adoption of eCOAs. When organizations began to adopt electronic questionnaires, migration studies would always be performed to demonstrate to regulatory agencies that the experience of using the paper and electronic versions was equivalent.
When eCOA was in its infancy, dedicated devices were provided to participants, along with a modem to transmit data from the device to the eCOA database. The use of these devices was foreign to many participants, and diligent training and troubleshooting of the device and modem was typically required. It was also considered necessary that the same device was used for all participants in a given study, which led to issues for long studies where devices were no longer available on the market. With the growing acceptance of multiple device modalities and wide range of smartphones, laptops, and tablets, there is more flexibility and more options available for consideration on a study.
In each study, an assessment should be conducted to determine which devices are the best fit for a study. For example, if there is primary or secondary data collection and multiple diaries are required for each day, perhaps provisioned devices should be sent to all participants to ensure that the data collection is uniform. Alerts can be “forced” to trigger on the device, reminding the participant to complete the diary.
In other cases, there may be observational data points collected among a participant population known to use their personal phones frequently. In this case, a “bring your own device” (BYOD) approach can be considered, where participants can use their own smartphones, if they are able. Those who cannot or do not want to use their own phone can use a provisioned smartphone.
A tablet might be the right fit for a site-based assessment, such as a ClinRO or PerfO, where the clinician can be mobile with the tablet as they interact with the patient and complete the assessment.
Positive Patient Responses
Digital solutions tend to be less burdensome for patients and thus generally garner positive patient responses. PPD teams have experience in deploying eCOA solutions to children, as well as older populations (75+). The PPD subject matter experts will design the eCOA solutions and training materials to target the age ranges.
The Importance of User Interface (UI) and User Experience (UX) Design
There is some complexity involved in supporting eCOAs and they are not suitable for every protocol. Data scientists, such as the Evidera team, are able to support usability testing by conducting a study with participants where the feasibility of completing measures on the device is evaluated. The scientists use qualitative research methods to elicit feedback on the usability of the device and platform for the participant population. This is important, especially when the disease/disorder might itself impact the ability to use a specific device. Digital technology experts, such as PPD Digital, can also be consulted to advise on the best way to design a solution based on the COAs, platform, participant population, and location.
It is important to ensure that the platforms that both patients and sites access via personal devices are user-friendly through proven user interface design (UI) and user experience design (UX).
The upfront investment in time and training is ultimately returned when all needed data are available, and sites do not need to tackle issues with patient diaries. A user-friendly electronic solution allows the site to easily access data on a daily basis, identify missing data, and follow up with patients immediately, if not already programmed to automatically alert users. As a result, improvements in compliance are easily observed.
The key is selecting the right electronic solution for each specific site and trial. That can be best achieved by working with a CRO like PPD that has a team of digital experts with timely industry knowledge and relationships with key providers.
The Value of Electronic Solutions
Overall, electronic solutions can provide higher-quality data than paper-based approaches. To achieve optimal electronic solutions, however, time and effort need to be invested to realize their value. The timeline for an electronic solution can be as little as six weeks or over 20 weeks, depending on the complexity of the solution or the number of solutions being implemented together. Solutions providers must follow best practices when designing, building, testing, and deploying these solutions. Critical steps to produce a quality system should not be skipped, which requires experts to spend the necessary time to perfect the solution.
Is There Still a Place for Paper?
There are still circumstances where paper-based methods might provide the right solution — for example, a study where there is a three-question assessment at one study visit for 20 patients. A patient could complete a questionnaire on paper, and then clinicians can use an electronic data capture (EDC) tool to enter the results of the questionnaire and collect other study details, saving the time and money required to deploy a separate eCOA solution. This is another example where the PPD Digital team and the Evidera PCR scientists can advise on the best solution that would work for the study.
COVID-19 Accelerating Adoption
While significant evolution of electronic outcome reporting has occurred over the last 20 years, the advent of the COVID-19 pandemic has dramatically accelerated things. Digital solutions have become essential to the continued operation of clinical trials.
Going forward, those changes will be here to stay. Paper-based solutions are not currently feasible for many trials, and moving back to them and away from electronic solutions does not make sense for sponsors pursuing continued innovation. Sponsors are keen to understand what solutions are available right now and to improve traditional models going forward. Similarly, regulatory authorities are interested in moving to more decentralized trial solutions. That goal underscores the importance of ePROs in particular, because getting patient input is just as important as ever, and it must be done in a way that keeps both patients and clinicians safe.
In addition to eCOAs becoming standard, electronic solutions are being implemented more rapidly than ever before. Typically, it takes three to four months to build, test, and validate a new system and then gain user acceptance and configure devices. During the pandemic, solution providers have looked at ways to deploy simpler, streamlined versions of their systems to accommodate urgent needs. Standardization of those types of services is now taking place, with service providers offering a range of solutions to fit different protocol needs. As a result, clinical trials that could not previously leverage eCOAs due to limited time and cost now have the opportunity to benefit from digital solutions.
In addition to eCOA, many other digital tools are available for use in clinical trials, including televisits and remote eConsent. Televisits can aid with the collection of eCOAs, since they have the ability to be placed virtually anywhere. Both the patient and the clinician can use the app during the online visit and walk through a questionnaire together.
From Initial Protocol Design to Post-Approval
PPD is consistently looking ahead to best determine what solutions will be needed. Specifically, PPD Digital’s team of digital implementation managers are trusted subject matter experts experienced in designing and deploying digital solutions, such as eCOA. PPD Digital’s extensive team of programmers and testers build custom digital solutions and deploy them, while our specially trained project specialists support and manage the operations. We also offer dedicated PPD Digital consultants who can provide detailed analyses and recommendations for protocol design or amendment. Our goal is to find the right digital solution for each set of sponsor, trial, and patient needs.
Before COVID-19 emerged, tour teams were equipped to support decentralized trials, thereby establishing best practices early on. As a result, we have well-known subject matter expertise in this area. At the beginning of the pandemic, when sites were shutting down and could not allow patient visits, we were able to get solutions out the door within a matter of days to weeks. As a result, PPD Digital has proven to be a leader in this space.
Evidera, a PPD business, has done extensive work on developing COAs, validating them and working with our clients to provide the appropriate support and evidence needed for any regulatory interactions around using PROs and other clinical outcome assessments. Within Evidera, we also have a modeling and simulation group, a real-world evidence group managing noninterventional studies and registry work, and a meta-analysis group that conducts retrospective analyses.
Now that you are reaching the approval stage, what do you need to get market access and reimbursement? What do you need if you want a labeling claim from a PRO or ClinRO perspective? We provide all the validation to do that.
With respect to eCOAs, we support clients considering options related to migrating a paper measure that has already been developed or to develop a measure specifically for an electronic data capture context. Common challenges we address include the appropriate steps needed from a regulatory perspective and what should be considered when implemented within a trial. We also design studies that can provide the evidence that paper measures have been appropriately migrated to help ensure that new measures intended for electronic capture are optimal in an electronic context.
Because electronic diaries involve much more than placing a device into the hands of patients, PPD has also developed expertise in web-enabled surveys, virtual visits with physicians, diaries that send alerts to physicians when a certain response is triggered, and much more, developing the technologies needed to address specific problems for individual trials.
PPD offers the best of the best of both worlds: a CRO that delivers timely, cost-efficient, fit-for-purpose trial designs combined with the scientific expertise and experience in PROs that can ensure whatever is being delivered is correct. For projects that do not require a CRO, we can still provide a consulting experience for clients looking for assistance evaluating the performance of a device or electronic solution they plan to implement in a trial with another CRO.
eCOA has become more prevalent in clinical research over the years, with many technology solutions introduced into the pharmaceutical industry. It is essential that researchers seek partners who understand best practices and dive deeper than just what can be seen on the user interface. Many organizations do not have extensive subject matter expertise in the area of eCOA and thus might not be aware of what to measure or evaluate. The best solution is to choose an integrated provider like PPD that has completed a large number of studies and can help pinpoint the best options for your trials.
Collaborating for Further Development
Lastly, it is important to acknowledge the collaboration that exists around eCOAs within the pharmaceutical industry. While there are many solution providers in this space, they work closely together — such as in the ePRO consortium — to make sure that everyone is working on designing solutions that support great data quality and with end-users in mind. All of these companies want a better experience for the patient.
That collaborative spirit is enhanced by a commitment to innovation. For instance, there is exciting research going on about how data can be collected using a more seamless, hands-free platform approach, where the patient responds to an audible questionnaire, and the data is automatically collected and sent to a centralized database. Such solutions would potentially require configuration rather than full custom programming, reducing the time it takes to build an eCOA solution and the number of errors that must ultimately be fixed.
Looking ahead, we expect more electronic data collection. High-quality data collection overall remains of utmost importance to PPD and Evidera, and — where appropriate and fit for purpose — PPD and Evidera will support electronic data collection to enable the industry to better collect and assess valuable data points, bringing more effective medicine and therapies to patients.
References
- Garrard, Lily. “Endpoint Selection and Use of Clinical Outcome Assessments (COAs) in Rare Disease and Pediatric Trials.” U.S. Food and Drug Administration. Nov. 2019. Web.
- Walton M.K., et al. “Clinical outcome assessments: a conceptual foundation – report of the ISPOR Clinical Outcomes Assessment Emerging Good Practices Task Force.” Value Health. 18: 741–752 (2015).
- Stone A.A., et al. “Patient compliance with paper and electronic diaries.: Control. Clin. Trials. 24:182–99 (2003).
- Coons S.J., et al. Gwaltney. “Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report.” Value Health. 12:419–429 (2009).
- Eremenco, Sonya. “PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force.” Value Health. 17:501–516 (2014).
