A Framework for Predicting Impactability of Digital Health

How machine learning methods can help health plans identify members to target for the greatest ROI, so they can optimize resources

Executive summary

Digital health solutions can improve the quality of care and reduce healthcare costs, and have been particularly successful in the field of care management. But as health plan executives know, it’s not enough to merely offer a digital experience—it’s critical to market this offering to members to help them adopt this technology, as well as leverage staff resources to aid recruitment. But until recently, there hasn’t been an effective method for determining which members to target in order to realize the greatest benefit.

Based on published, peer-reviewed research, this exclusive report details how machine learning methods, supported by a cutting-edge data infrastructure, can help health plan leaders take an “impactability-based” approach to digital health management.

Of course, we know there is no shortage of hype surrounding artificial intelligence and machine learning in healthcare, and that many executives may wonder whether these promising new technologies will actually deliver results for their member populations. That’s why this approach involved partnering with health plan leadership to develop algorithms that were optimized for the populations they serve.

Defining impactability

As an aging, multimorbid population demands more health services, health plans have responded by implementing care management programs to facilitate care coordination and improve value.1

Previous studies indicate that care management savings can only be achieved by targeting members with the highest number of chronic diseases, who generate the highest healthcare costs.2,3 But in reality, this practice doesn’t always yield the greatest savings potential.

That’s where the concept of “impactability” comes in. Considering impactability reframes the allocation of healthcare resources by matching individuals to interventions who are most likely to benefit from them rather than those most at risk, potentially amplifying reduction in healthcare costs.

Considering impactability reframes the allocation of healthcare resources by matching individuals to interventions who are most likely to benefit from them rather than those most at risk, potentially amplifying reduction in healthcare costs.

The promise of impactability-based targeting for digital health platforms

Digital health adoption is on the rise, ranging from wearable devices to mobile-enabled disease and care management applications. Care management has been particularly successful in implementing digital health tools.5–7 But faced with increasingly restricted budgets, it’s crucial for health plans to determine who to target with digital health interventions.

For care management, the relatively low start-up and maintenance cost of digital health solutions already allows for the inclusion of more members. These interventions can, however, have significant external costs, namely the marketing campaigns needed to recruit members to the platform. Member recruitment often requires time from health plan staff as well in the form of telephonic outreach.

With an impactibility-based approach, care management teams can target marketing campaigns to members most likely to benefit, allowing them to focus their limited resources where they can have the greatest impact. As a result, they can optimize recruitment efforts and offset the cost of marketing.

Applying machine learning to digital health data

Recent technological advances have made it possible to further refine an impactability-based approach. That’s because most digital health tools share a vital capability: they collect vast amounts of unique data. Enabled by the combination of ever-larger datasets and increasing computing power, machine learning methods are now able to effectively “learn” from these data, highlighting patterns and, in many instances, exhibiting judgement superior to that of humans.As a branch of artificial intelligence, machine learning methods transform the data inputs of an algorithm into insights using statistical, data-driven rules that can be automatically derived from a large set of examples.

As a branch of artificial intelligence, machine learning methods transform the data inputs of an algorithm into insights using statistical, data-driven rules that can be automatically derived from a large set of examples.

However, machine learning techniques remain largely untested in the domain of impactability-based resource allocation, having previously relied on more classical, less powerful prediction models.3,4 This highlights an exciting new frontier for judicious allocation of finite resources that aligns itself well with the principles of value-based healthcare.

Our research demonstrates how a combination of machine learning methods can be used to identify the features of members whose costs significantly declined once onboarded onto a digital health intervention, thereby highlighting the features that constitute impactability. Such an approach allows a subsequent machine learning algorithm to predict a new member’s impactability, thereby offering a tool to assist in decision-making around allocating the intervention. We used data from a commercially insured population, combining insurance claims, sociodemographic and patient-generated data (from the use of the member-facing mobile app at the core of the digital health intervention described below).

The study population was onboarded onto a digital health platform created by the Data Science team at Wellframe Inc, a Boston-based digital health management company. The Wellframe intervention consists of a mobile-enabled care management platform, based around an app that includes a customized, interactive, daily health checklist displayed to members on their smartphone or tablet. Responses and interactions from users are collected as actionable day-to-day-health information. The app serves to provide a sustained and supportive channel for communication between members and their care teams. The population in this study was onboarded onto the Wellframe app as part of a health coaching program, and both members and clinical teams received training on how to use all of its features.

A Framework for Predicting Impactability of Digital Health

Cutting-edge data infrastructure to enable machine learning for health plans

These results were achieved using the infrastructure shown in figure 1, built using Google Cloud Platform (GCP).

A Framework for Predicting Impactability of Digital Health
FIGURE 1. Airflow is a data orchestration service able to ingest client provided claims information and merge it with transactional data generated by the Wellframe platform. The amalgamation of these two data sets was stored into a centralized data warehouse: BigQuery. As a serverless, managed data warehouse, BigQuery allows the deployment of a centralized data warehousing solution without the need to maintain large computing clusters. As a data footprint grows, this can reduce operational cost while giving data scientists flexibility in how they interact with the collected data. Once in BigQuery, a collaboration of data science teams is able to build models by leveraging Cloud datalab and Jupyter Notebooks. They can also access cutting edge tools like Keras, to build deep neural nets and train models using Cloud TPUs (Cloud Tensor Processing units). Once a model is trained, Cloud Machine Learning Engine can deploy it into production and make it available via an application programming interface (API). This API can then be utilized by applications (mobile apps or web apps) to predict a client’s propensity to benefit. All operational data generated from the interaction of users with the platform is stored in Cloud SQL database. This data is replicated at regular intervals into a data warehouse. The fast availability of this data enables validation that the recommendations offered by the deployed models are actually yielding the intended goals. The data collected can be exposed via Looker to enable business users to access the results of a predictive model. Looker allows businesses to track key conversion metrics that can help assess the impact of the recommendations, without requiring expertise in data.

Methods

These findings were published in Population Health Management, a peer-reviewed public health journal covering the study of population health, and how it can be improved by improving health care services.

The approach involved creating 2 models and the comparative analysis of the methodologies and performances therein. We first trained a cost prediction model to calculate the differences in predicted (without intervention) versus actual (with onboarding onto digital health platform) health care expenditures for patients. This enables the classification impactability if differences in predicted versus actual costs meet a predetermined threshold. Several random forest and logistic regression machine learning models were then trained to accurately categorize new patients as impactable versus not impactable. These parameters are modified through grid search to define the parameters that deliver optimal performance, reaching an overall sensitivity of 0.77 and specificity of 0.65 among all models.

Results

This approach showed that impactability for a digital health intervention can be successfully defined using machine learning methods, which in turn enables efficient allocation of health plan resources. This framework is generalizable to analyzing impactability of any intervention. These findings, though promising, are only preliminary. They can be significantly improved upon when health plans partner with digital health solutions that can collaborate on building improved models, driving real-world clinical and business results.

This approach showed that impactability for a digital health intervention can be successfully defined using machine learning methods, which in turn enables efficient allocation of health plan resources. This framework is generalizable to analyzing impactability of any intervention.

Conclusion

Finding, enrolling, and engaging patients in digital health management programs often proves challenging. That’s where AI and machine learning can come in. While health plans currently use established statistical techniques for risk profiling based on cost, there is untapped potential to utilize machine learning and new data sources including social determinant data and data generated from digital health platforms to uncover patients who are most likely to benefit from digital health interventions.

By applying machine learning in partnership with health plan leadership, supported by cutting-edge data infrastructure enabled by Google Cloud, these published, peer-reviewed findings demonstrate a valuable new approach to member targeting. When organizations target members based on impactability, they can identify members based on clinical needs, as well as emerging and actual risk and cost, and then focus their clinical resources where they can be most effective.

REFERENCES

1. Flaster, A. How Care Management Done Right Improves Patient Satisfaction and ROI. Health Catal. Insights (2017).

2. McWilliams, J. M. & Schwartz, A. L. Focusing on High-cost Patients: the Key to Addressing High Costs? N. Engl. J. Med. 376, 807 (2017).

3. Hayes, S. et al. High-need, high-cost patients: who are they and how do they use health care. Popul.-Based Comp. Demogr. Health Care Use Expend. Issue Brief Commonw Fund 26, 1–14 (2016).

4. Brown, R. S., Peikes, D., Peterson, G., Schore, J. & Razafindrakoto, C. M. Six Features Of Medicare Coordinated Care Demonstration Programs That Cut Hospital Admissions Of High-Risk Patients. Health Aff. (Millwood) 31, 1156–1166 (2012).

5. Panch, T. & Goodman, E. Technology, Seniors, and Sense Making. Am. J. Manag. Care 22, (2016).

6. Mueller, N. E. et al. Using Smartphone Apps to Promote Psychiatric Rehabilitation in a Peer-Led Community Support Program: Pilot Study. JMIR Ment. Health 5, e10092 (2018).

7. Domenge, N. Driving Patient Engagement Through Mobile Care Management. Healthc. Inf. Manag. Syst. Soc. Conf. Session #97, (2017).

8. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015). 13. Internal Revenue Service. SOI Tax Stats – Statistics of Income. (2018).

About the authors

  • About the Author

    TRISHAN PANCH, MD, MPH

    Trishan works at the intersection of technology and health systems to create strategies for digital transformation and develop products and services that re-engineer care processes, empower patients, and create a more efficient and responsive health systems.

  • About the author

    MOHAMMAD JOUNI, MS

    Mohammad combines expertise in technology strategy, data science and software engineering to build mission-driven teams that can make a positive impact on the healthcare system using cutting-edge technology

  • About the Author

    EMILY LINDEMER, PHD

    Emily draws from her diverse background in computational methods for assessing healthcare data combined with her clinical training to build novel solutions that can be directly applied in real-world healthcare settings.

  • About the author

    HEATHER MATTIE, PHD

    Heather’s research includes methods to limit algorithmic bias and classify subtypes of disease such as diabetes. She also serves as Executive Director, Master’s Program in Health Data Science, at the Harvard T.H. Chan School of Public Health.

  • About the author

    PAT REIDY, MPH

    Patrick leverages Wellframe’s data warehouse to create internal self-service analytic tools. He also builds, maintains and monitors data pipelines that power our data solution.