Expanding the Evidence Base for Precision Healthcare

Joseph B. Franklin, Amy P. Abernethy, Jason Arora, Brad Hirsch, Verily

Contact: joebfranklin@verily.com 

Diverse and longitudinal: Precision care will require new ways of generating evidence

New and emerging technological capabilities for diagnosing, treating, and preventing disease promise significant advances in patient care. However, realizing the benefits of these achievements will require new methods for generating evidence that can inform more personalized care decisions.

While traditional clinical trials are a powerful tool, they tend to have significant limitations.^{[1] [2]}  For example, it can be difficult for traditional clinical trials to recruit patients that are representative of the diversity in the population; many patients are systematically left out of clinical trials due to comorbidities, the practical challenges of participating in complex studies far from home, and other hurdles. Traditional trials collect only certain information during a short time window in a patient’s care, missing much of the data that may be relevant to understanding both the patient’s health background and long-term measures of safety and effectiveness for novel interventions. These limitations – stemming from a ‘closed system’ approach to clinical studies – impact the utility and generalizability of clinical trial findings in the real-world setting, resulting in outcome variation and the inability to consistently and accurately predict individual patient outcomes prior to therapeutic intervention.

Further, to inform personalized care decisions, the evidence base needs to reflect the values of patients. This means that we need rigorous methods for integrating patient-centered outcomes with more traditional clinical outcome measures.  For example, in nephrology, an important healthcare context which is explored in more depth below, mobility and other outcomes including wellbeing, quality of life, and treatment burden can provide critical insights into meaningful lifestyle improvements for dialysis patients.^{[3] [4]}

From these gaps in the existing paradigm for evidence generation, we can identify discrete challenges to address: (1) increasing the depth of the evidence both by enhancing the diversity of the patients studied and by enhancing the richness of the study data to include sources like relevant biological (e.g., genomic) information, environmental factors, and improved patient-centered measures; (2) increasing the generalizability of the evidence by including people traditionally excluded from traditional clinical trials such as those with comorbidities; and (3) increasing the longitudinality of the evidence by supplementing traditional clinical trial data with datasets that provide information from timepoints both before and after the data collection period for a clinical trial.

High-quality RWD, novel clinical trial designs, and an integrated approach to evidence is critical for filling the gaps in the current clinical evidence paradigm

Extending our evidence generation in each of these dimensions – depth, generalizability, and longitudinality – cannot be achieved unless we develop new ways to collect and integrate data from both traditional clinical trials and non-traditional sources like real-word data (RWD). Practically speaking, we need tools to assemble high-quality datasets and use these datasets as a component of more modern clinical studies. We must also develop a more integrated, “totality of the evidence” approach to informing healthcare decisions, broadly, by including clinicians, regulators, and payers.

RWD is data collected from a variety of sources about a patient’s health status or about the care the patient has received. RWD is often regarded as “clinical data collected outside of the confines of a traditional formal clinical trial.” RWD can include electronic health records (EHRs), insurance claims, and data available outside the health system, such as sensor information from a watch, environmental exposure data, or genomic information from a tumor sample. RWD can be prospective (e.g., intentionally collected in a pragmatic clinical trial) or retrospective (e.g., passively collected in the EHR). The quality, reliability, and validation of RWD has, appropriately, generated significant attention, since RWD often is not collected with evidence generation in mind.

There is broad consensus, however, that the unrealized informational value of RWD justifies the development of a scientific framework and technical tools for collecting, assembling, and analyzing high-quality RWD in support of clinical evidence generation. FDA is a central driver of this consensus. In the closing months of 2021, FDA issued draft recommendations for collecting and validating RWD, including from EHRs and medical claims.^[5]  FDA’s recommendations address issues that are critical for assembling longitudinal datasets, including linking data from different sources and avoiding potential issues associated with data linkage like data redundancy, inconsistency, and the reliability of matching data from the same patient from different datasets.^[6] FDA’s current push to develop RWD guidance reflects the importance of these technical details and highlights the need for data scientists and software engineers to develop tools that can achieve the level of data quality and reliability that clinical evidence generation demands.

Broader use of high-quality, passively collected RWD is not a replacement for prospectively designed, randomized clinical trials (RCTs). And observational studies are only one use case for RWD. So, it is not surprising that some of the most valuable uses for RWD are in the context of prospectively designed studies: to supplement data gaps in RCTs, to improve study efficiency by filling in certain variables in the clinical trial dataset, and to allow data collection for RCTs to be more closely integrated into regular clinical care and the daily lives of patients.^[7]

The development of these methods has been accelerated by the COVID-19 pandemic, which has revealed the value of pragmatic clinical studies (i.e., studies conducted in a real-world clinical setting) and decentralized or virtual clinical trial technologies. These modern approaches will make clinical studies more efficient and allow them to reach a more diverse set of patients in a wider variety of settings–key to increasing the representation and ability of clinical studies to inform personalized care. COVID-19 has also spotlighted the complexity of assembling health data from different sources. This is especially true in the UnitedStates, where paper vaccination cards became emblematic of the barriers to digitally monitoring vaccine performance on a large scale. More optimistically, COVID-19 revealed specific technology and business solutions that, if harnessed, will help us develop a data infrastructure that is more coherent, comprehensive, and accessible, more amenable to the generation of actionable evidence.^[8]

In addition to enhancements in clinical trials, the increasing use of longitudinal RWD will enable a paradigm shift in the evaluation cycle for medical products. Under the current paradigm, after FDA approval, we can expect only marginal improvements in the evidence available to inform the use of a drug or device. The transition to modern evidence generation techniques, including longitudinal RWD datasets, will enable a continuous accumulation of post-market evidence for medical products, giving us better, more generalizable information about how to optimize the use of an intervention or diagnostic in individual  .

Although FDA’s work over the past several years has reinforced the importance of longitudinal, RWD-based evidence for regulatory decisions, there is a growing realization that this evidence will also be critical for supporting certain types of coverage and payment decisions.  In the context of Medicare, post-market data could be used to confirm that the evidence supporting approval of a treatment or diagnostic is generalizable to Medicare-age patients, for example, or to otherwise establish that the product meets the threshold of reasonable and necessary in the Medicare population. As the Centers for Medicare and Medicaid Services (CMS) has noted recently in the rulemaking context, FDA’s premarket review “relies on scientific and medical evidence that does not necessarily include patients from the Medicare population” due to a variety of factors, including the exclusion of certain participants from pre-market clinical studies due to comorbidities or concomitant treatments.^[9] An active debate about how to fill these evidence gaps should be a central component of Medicare’s “objective of improving health outcomes while delivering greater value.”^[10]

While much of the focus of RWD has been on the generation of high-quality evidence, we also need to develop the infrastructure for using evidence generated with novel methods.  Even the highest quality clinical evidence is only valuable if it is used to inform decisions—clinical decisions, regulatory decisions, or decisions involving healthcare delivery and payment. Each of these areas have strong, pre-existing decision-making frameworks that have evolved for decades around large, traditional RCTs. To put RWD and other sources of evidence to work, we must adapt to a “totality of the evidence” approach, in which the weight of each piece of available evidence is determined based on a variety of different factors like the fitness of the data for a particular analytical purpose.

FDA has made substantial progress towards applying a totality of the evidence approach by considering a variety of evidence types to inform regulatory decisions.^[11] However, there is still significant work to do to ensure that healthcare stakeholders, including clinicians, regulators, and payers, have the information tools they need to make decisions when the available evidence is both increasingly complex and continuously accumulating. Only by tuning our healthcare decision-making to maximize the use of available evidence sources will we be able to put this evidence to work in the service of personalized care.

Kidney care as one example of a therapeutic area positioned to generate and use evidence to inform improved, precision

Historically, real-world data and evidence have been regarded as the purview of oncology and rare diseases. Advances in the science of RWD/RWE, accelerated by the COVID-19 experience, highlight the important opportunity of leveraging RWD/RWE broadly to achieve a “totality of the evidence” approach across more domains of healthcare. As this approach to evidence generation and use expands, nephrology is an exemplary therapeutic area demonstrating the evolving opportunity to generate and use evidence in new ways to personalize care for patients.

First, kidney care is an important clinical context for applying new evidence generation approaches. Given the racial and ethnic diversity of kidney disease populations, kidney care can be a role model for the development of practical solutions to involve diverse communities in research and develop generalizable datasets that are more reflective of all people. Engagement with kidney patient communities will be a crucial step for designing and evolving patient-centered outcomes, recruiting patients to participate in new registries, and incorporating patient input into broader areas like privacy controls.

Second, the structure and availability of kidney care data provide advantages for modern evidence generation. Many of the critical variables (e.g., creatinine) are already available as structured, reliable data elements in the EHR and other real-world datasets, and the outcomes meaningful to kidney care (e.g., time to need for dialysis) are easily tracked. Dialysis is uniquely suited to clinical studies that are integrated into regular care, including the routine collection of highly structured data, such as lab values.^{[12] [13] [14]}  Dialysis sessions provide opportunities to engage with patients and evaluate patient-centered outcomes with higher resolution. We should build on these features by testing new methods for collecting and linking to RWD. As treatment and prevention reduce the need for dialysis–and as more kidney care moves upstream from the dialysis clinic–RWD and decentralized interaction with patients will be critical for providing longitudinality, generalizability, and depth to clinical studies.

Third, the imperative to monitor and evaluate medical product performance across time is harmonized with the goals of kidney patient care. New evidence generation approaches will focus on how medical products and specific interventions perform longitudinally–across time and the life-cycle of the medical product. Similarly, the management of kidney disease, like other chronic health conditions, is inherently longitudinal, tracking kidney function across time (e.g., change in glomerular filtration rate) and seeking approaches to modify the impact of disease.

It is fortunate that there are many approaches to generate longitudinal evidence in kidney care, because achieving the goal of truly transformative renal replacement therapies will necessitate a nuanced and data-driven approach to longitudinal safety and effectiveness monitoring. Take, for example, replacement kidney technologies that incorporate the complex software components–potentially including artificial intelligence and machine learning (AI/ML) needed to autonomously respond to a patient’s changing chemical and physical conditions in real time. The ability of AI/ML-based kidney replacement therapies to learn and change to the needs of the individual patient is what makes this approach so promising, but also why we need to ensure that we have a solution for continuous performance monitoring, a point that FDA has emphasized generally for AI-ML-based devices.^[15] The need for RWD-based performance monitoring has been recognized in the renal replacement field, including in the  Kidney Health Initiative’s Technology Roadmap^[16], but accomplishing this goal will require longitudinal data from a representative set of patients – a task that will not be practical or feasible without more streamlined methods for collecting and linking data from different sources.

Finally, kidney care provides an opportunity for using richer datasets to evaluate healthcare delivery and personalize care for all patients. This is especially true in the dialysis context where, to cite just one example, CMS manages performance-based financial incentives for dialysis centers, and commentators have raised concerns whether this program is effectively reducing inequities in kidney care.^[17]  We have an obligation to consider how we can calibrate treatment guidelines and healthcare delivery using evidence that represents the real-world diversity of kidney disease patients.

A better understanding of biology is preparing the way for a new era of precision kidney care, which will require the matching of general clinical evidence with a detailed understanding of the specific biologic basis for disease in a particular patient. Echoing the evolution in cancer care over the last 20 years, precision kidney care is going to require the matching of the best available clinical evidence with the details of a person’s biology and clinical situation. Transformation in kidney care will require new evidence generation because legacy clinical trial infrastructure cannot support responsible evidence generation for every permutation of clinical biology and the personal needs of a particular patient. New solutions will be needed to combine traditional clinical trial data with real-world information that reflects the many scenarios of care and outcomes for each individual person. The goal posts for this work are clear: longitudinal, high- quality data that can support efficient clinical trials and continuous monitoring of medical products, and that can ensure that the needs of all people from all backgrounds are represented.

Looking forward: Continuous learning to improve care

Not only is an expanded approach to evidence generation essential for achieving a better understanding of medical product performance, it is also necessary for providing the generalizable evidence needed to achieve personalized care for all people. And by developing a system of continuously aggregating longitudinal data and evidence, we can close the loop between research and the clinic and ensure that care continuously improves over time.

Notes:

Amy P. Abernethy and Joseph B. Franklin formerly held roles in the US Food and Drug Administration (FDA).

 

References

^[1] Stuart EA, Bradshaw CP, Leaf PJ. Assessing the generalizability of randomized trial results to target populations. Prev Sci. 2015;16(3):475-485. doi:10.1007/s11121-014-0513-z

^[2] O’Hare AM, Rodriguez RA, Bowling CB. Caring for patients with kidney disease: shifting the paradigm from evidence-based medicine to patient-centered care. Nephrol Dial Transplant. 2016;31(3):368-375. doi:10.1093/ndt/gfv003

^[3]  Tong A, Winkelmayer WC, Wheeler DC, et al. Nephrologists’ Perspectives on Defining and Applying Patient-Centered Outcomes in Hemodialysis. Clin J Am Soc Nephrol. 2017;12(3):454-466. doi:10.2215/CJN.08370816

^[4] Bowling CB, Plantinga LC. When All You Have Is a Hammer: The Need for Tools to Define and Apply Patient-Centered Outcomes in Hemodialysis. Clin J Am Soc Nephrol. 2017;12(3):382-384. doi:10.2215/CJN.00550117

^[5] https://www.fda.gov/media/152503/download

^[6] https://www.fda.gov/media/154449/download

^[7] Real World Data and Evidence: Support for Drug Approval

Aliza M. Thompson, Mary Ross Southworth

CJASN Oct 2019, 14 (10) 1531-1532; DOI: 10.2215/CJN.02790319

^[8] Lee, P., A. Abernethy, D. Shaywitz, A. V. Gundlapalli, J. Weinstein, P. M. Doraiswamy, K. Schulman, and S. Madhavan. 2022. Digital Health COVID-19 Impact Assessment: Lessons Learned and Compelling Needs. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/202201c.

^[9] Final Rule, “Medicare Coverage of Innovative Technology (MCIT) and Definition of ‘Reasonable and Necessary’”86 Fed. Reg. 62946-62947 (Nov. 15, 2021) (repealing the MCIT final rule of January 14, 2021).

^[10] See 86 Fed. Reg. 62947 (Nov. 15, 2021)

^[11] https://healthpolicy.duke.edu/sites/default/files/2020-08/Totality%20of%20Evidence%20Approach.pdf

^[12] Real World Data and Evidence: Support for Drug Approval, Aliza M. Thompson, Mary Ross Southworth, CJASN Oct 2019, 14 (10) 1531-1532; DOI: 10.2215/CJN.02790319

^[13] Cultivating a Research-Ready Dialysis Community

Jennifer E. Flythe, Julia H. Narendra, Tandrea Hilliard, Karen Frazier, Kourtney Ikeler, Andrew Amolegbe, Denise Mitchell, Adeline Dorough, Shoou-Yih Daniel Lee, Antoinette Ordish, Caroline Wilkie, Laura M. Dember, for the Building Research Capacity in the Dialysis Community at the Local Level Stakeholder Workshop Participants JASN Mar 2019, 30 (3) 375-380; DOI: 10.1681/ASN.2018101059

^[14] Pragmatic Trials in Maintenance Dialysis: Perspectives from the Kidney Health Initiative. Laura M. Dember, Patrick Archdeacon, Mahesh Krishnan, Eduardo Lacson, Shari M. Ling, Prabir Roy-Chaudhury, Kimberly A. Smith, Michael F. Flessner. JASN Oct 2016, 27 (10) 2955-2963; DOI: 10.1681/ASN.2016030340

^[15] FDA, Artificial Intelligence/Machine LEarning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan (January 2021), available online at https://www.fda.gov/media/145022/download.

^[16] Kidney Health Initiative, “Technology Roadmap for Innovative Approaches to Renal Replacement Therapy,” (October 2018), available online at https://www.asn-online.org/g/blast/files/KHI_RRT_Roadmap1.0_FINAL_102318_web.pdf

^[17] Toward Antiracist Reimbursement Policy in End-Stage Kidney Disease: From Equality to Equity. Kathryn Taylor, Deidra C. Crews. JASN Oct 2021, 32 (10) 2422-2424; DOI: 10.1681/ASN.2021020189