The Evolving Taxonomy of Clinical Outcome Assessments

Donald L. Patrick1, Catherine Acquadro2

1Director, Seattle Quality of Life Group, University of Washington, Seattle, Washington, USA
2Scientific Advisor, Mapi Research Trust, Lyon, France

In October 2011, the term Clinical Outcome Assessments (COAs) was introduced at a FDA Workshop1 [Review and Qualification of Clinical Outcome Assessments; Public Workshop] to better reflect the importance of the source of information in measurements. COAs, which exclude survival and biomarkers, are influenced by human choices, judgment, or motivation, and depend on the cooperation and reporting of observations by patients (PROs), clinicians (ClinROs), and observers (ObsROs). These outcomes are central to the rapidly accelerating movement to incorporate patient-centered outcomes in health research and clinical practice.
Patient-Centered Outcomes (PCOs) are “Those outcomes important to patients’ survival, function, or feelings as identified or affirmed by patients themselves, or judged to be in patients’ best interest by clinicians and caregivers when patients cannot report for themselves.”2,3 PCOs are conceptualized by identifying the concept of interest and measures of these concepts—constructs—that help to evaluate treatment benefit. Direct evidence of treatment benefit is demonstrated by a positive impact on survival or on how patients feel or function in daily life. Indirect evidence of treatment benefit comes from other measures of outcomes demonstrated or thought to be related to survival, functions, or feelings. A conclusion of treatment benefit may be made by a clinician and patient in clinical practice, by a researcher in conducting effectiveness research using many different research designs, by a technology assessment organization such as the UK National Institute on Clinical Effectiveness, or by regulators of the pharmaceutical industry in labeling in terms of the concept of interest (COI), i.e., the thing measured by the outcome assessment or claim. Treatment benefit can be demonstrated as either a comparative advantage in how patients survive, feel, or function or a comparative reduction in treatment-related toxicity.
Clinical trials and comparative effectiveness studies are intended to evaluate whether there is a treatment-related difference in the outcome of patients. Endpoints are identified and incorporated in patient assessments, obtained at one or more specified times during the study, and analyzed according to specified statistical methods to provide a comparison between groups. The patient assessment used in an endpoint is the outcome assessment (OA). The OA itself is not the endpoint as other aspects of endpoints, such as how it is used in analysis, affect interpretation of the study results. Endpoints include the OA and the way in which it is analyzed. For example, time to worsening or quality-adjusted survival may be the endpoint using the OA in calculation.

OAs may provide direct evidence of treatment benefit using the concept of interest and indirect evidence when not precisely evaluating the concept of interest but an outcome highly related to it. Figure 1 illustrates the endpoints and OAs used to evaluate treatment benefit.
Survival. Overall, survival is defined as the time from randomization until death from any cause, and is measured in the intent-to-treat population. It may also be evaluated in non-randomized studies. Survival is generally considered the most reliable endpoint in clinical trials where it is relevant. When studies can be conducted to adequately assess survival, it is usually the preferred endpoint, as it is a direct measure of treatment benefit and a patient-important outcome
Survival is most often documented by date of death, considered easy and precise. Disease-free survival (DFS) is defined as the time from randomization until recurrence of tumor or death from any cause. Progression-free survival (PFS) is defined as the time from randomization until objective tumor progression or death. Formal validation of PFS can be difficult. Whether an improvement in PFS represents direct clinical benefit or a surrogate depends on the magnitude of the effect and the risk-benefit of the new treatment compared to available therapies.
Biomarkers are indirect measures and function as surrogates or substitutes for some direct concept on how patients survive, feel, or function. Substantial amounts of evidence are needed to establish biomarkers as acceptable OAs for use as an efficacy or surrogate endpoint. Biomarkers are derived from blood, other tissue, and computerized radiography where the reading is generated by the computer itself. Biomarkers are objectively measured and evaluated as indicators of normal biological processes, pathogenic processes, or pharmacologic responses to an intervention. Examples: cholesterol, blood sugar levels, and tumor size from MRI or CT.
Often it is assumed there is a clear distinction between clinical outcome assessments and surrogate endpoints, because clinical endpoints reflect patient or consumer experience and surrogate measures do not. However, many clinical endpoints have similarities with biomarkers, such as the clinicianreported outcomes involving reading and interpretation of readings, such as CT scans. Understanding the underlying mechanism of action is important to gain insight into the relationship between the biomarker, treatment, and clinical endpoint. For example, treatment may alter the biomarker but not affect the clinical endpoint. The means of assessing the biomarker must be valid and reproducible.
The use of biomarkers may decrease the duration of a clinical trial or epidemiological observational study, because survival or another direct assessment of treatment benefit is beyond the period of observation in a study. Biomarkers may also decrease the sample size, while maintaining the integrity of the inference about the ultimate effects of treatment. Biomarker surrogates can often be easily and uniformly measured but complex to interpret. The relationship between biomarkers and clinical outcome assessments can be tenuous or difficult to demonstrate, even when the biomarker is thought to be integral to patient assessment.
Clinical outcome assessments (COAs) are measurements based on a human assessment (i.e., excluding biomarkers) and “reported” using a measurement instrument by a patient, a clinician, or another observer. Four types of COAs are described below.

Performance outcome assessments (PerfOs) are derived when the patient is instructed to perform a defined task and some defined quantification of that performance is the measurement (e.g., distance walked in six minutes, number of pictorial symbols correctly matched to a key within a fixed amount of time). The key aspect here is that there is no meaningful clinician judgment involved; but it remains a COA and not a biomarker because of the necessity for a patient’s volitional engagement to perform the test procedure.
A Clinician-reported outcome (ClinRO) is any assessment of the status of a patient’s health condition based on clinician observation, reporting and/or interpretation. ClinRO measures may include: readings and interpretation of tests (e.g., radiographically identified vertebral fractures, improvement in vein appearance based on photographs); protocol-driven ratings, such as event counts and rating scales (e.g., psoriasis area and severity, inpatient vomiting episodes based on nurse report, presence/absence of skin lesions, observation of schizophrenia behaviors, spleen size based on clinician palpation, or global ratings, such as the clinician global impression of change).
An Observer-reported outcome (ObsRO) is an assessment used when the person is unable to selfreport based observable concepts (e.g., signs or behaviors). ObsROs cannot be validly used to directly assess symptoms (e.g., pain) or other unobservable concepts. ObsROs provide observations but not interpretations of the patient’s health condition from a non-professional observer (e.g., caregiver, teacher). An ObsRO is neither a proxy measure nor a PRO. The observer is typically the person who takes care of the patient most of the time and can report what he or she sees, hears, feels, smells, or touches. Examples include: parent assessment of how many times a baby vomited in the past 24 hours, teacher assessment of a child’s ability to attend to tasks (inattentiveness).
Most familiar to the readers of this newsletter are Patient-Reported Outcome (PRO) assessments which are based on a report that comes directly from the patient (i.e., study subject) about the status of particular aspects of or events related to a patient’s health condition, without interpretation of the response by clinicians or anyone else.4 PROs include symptoms/signs, function, and quality of life, but in some context may also include satisfaction with care and adherence.

OAs have been used for many decades but how well established they are varies by the type of assessment. Survival has wellestablished methods for evaluation and are relatively easy to interpret. Biomarkers have varying ease of interpretation, used widely but not always clearly linked to COAs. COAs can provide complementary and sometimes primary evidence of treatment benefit, direct or indirect. PerfOs have a long history and are often documented. ClinROs have been around a long time but are sometimes not subjected to rigorous measurement evaluation and are “assumed” valid because of clinician judgment. PROs are now established and subject to rigorous measurement evaluation.4 ObsROs have been used in many specific conditions, but systematic consideration of their conceptual foundation and measurement properties are relatively new.
This taxonomy of COAs helps put a focus on the patient and on direct assessment of treatment benefit in clinical trials. The taxonomy is also useful when determining endpoint positioning for the assessment of treatment benefit. Arraying the endpoints based on survival, biomarkers, and COAs encourages hypothesis-driven investigation of their relationships and, at bare minimum, consideration of the role each endpoint plays in evaluating whether a treatment is more or less effective within a particular clinical context. Medical product labels and assessments of treatment efficacy in non-regulatory studies are increasingly likely to adopt this terminology. Greater transparency in how treatments are evaluated and clinical benefit is determined will help improve the very outcomes included in the assessment strategy.

References

1. Regnault A, Acquadro C: FDA Workshop: Measurement in Clinical Trials: Review and Qualification of Clinical Outcomes. PRO Newsletter 2012, 47:12-17.
2. Patrick DL: From Health-Related Quality of Life to Patient-Centered Outcomes: an historical switch to reflect FDA’s efforts in Patient-Focused drug development. PRO Newsletter 2013, 50:4-6.
3. Patrick DL, Burke L: Focusing on the patient in drug development and research. ISPOR Connections 2013, 19(6):5-8.
4. Food and Drug Administration: Patient reported outcome measures: use in medical product development to support labeling claims. Federal Register 2009, 74(35), 65132-133. Available at Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims.