Smart Digital Therapeutics for Alcohol Use Disorder: Algorithms for Prediction and Adaptive Intervention

Mental Healthcare Needs are High and Unmet

In 2019, 52 million Americans had an active mental illness
- More than half did not receive any treatment

Mental Healthcare Needs are High and Unmet

In 2019, 52 million Americans had an active mental illness
- More than half did not receive any treatment
20 million adults had an active substance use disorder
- 9 out of 10 did not receive any treatment

Mental Healthcare Needs are High and Unmet

In 2019, 52 million Americans had an active mental illness
- More than half did not receive any treatment
20 million adults had an active substance use disorder
- 9 out of 10 did not receive any treatment
Large treatment disparities exist by race, ethnicity, geography, and income

Mental Healthcare Needs are High and Unmet

Failure to treat is not surprising given many treatment barriers:
- Access
- Availability
- Affordability
- Acceptability

Digital Therapeutics (DTx)

Digital therapeutics are smartphone “apps” that are designed to prevent, manage, or treat disease, including mental illness.

Can augment mental health services to address barriers

Accessible everywhere
Available 24/7
Highly scalable (affordable?)

Digital Therapeutics (DTx)

Digital therapeutics are smartphone “apps” that are designed to prevent, manage, or treat disease, including mental illness.

Can augment mental health services to address barriers

Accessible everywhere
Available 24/7
Highly scalable (affordable?)
Effective!

Smart Digital Therapeutics

“Could you predict not only who might be at greatest risk for relapse …
… but precisely when that relapse might occur …
… and how best to intervene to prevent it?”

My research team became interested in these issues when my colleague Dave Gustafson, the developer of a leading digital therapeutic for substance use disorders, approached us with a simple question. He asked…

“Could you predict not only who might be at greatest risk for relapse but precisely when that relapse might occur and how best to intervene to prevent it”

Dave had just completed a large study demonstrating the effectiveness of his app. However, he also noticed many of the people who relapsed hadn’t used the app in the days leading up to that relapse. And others who had relapsed hadn’t used the specific supports in the app that he would have thought would be most effective for them.

The next wave of digital therapeutics, lets call them smart digital therapeutics, must learn to know us well enough to recognize when we are at greatest risk for relapse and they must be smart enough to recommend the specific supports that would be most effective for us at that moment in time to prevent that relapse.

And these apps will do this through the use of built-in artificial intelligence or machine learning algorithms that are powered by personal sensing.

Lapse Prediction in Patients with AUD

151 patients with AUD
Early in recovery (1-8 weeks)
Committed to abstinence throughout study
Followed for up to 3 months
Collected active and passive personal sensing data streams

GOAL: Develop a temporally precise lapse monitoring (prediction) system for patients with AUD

Personal Sensing Data Streams

4X daily ecological momentary assessments (EMA)
Monthly self-report
Geolocation (GPS)
Cellular communications (voice and text messages)
- Meta data
- Text message content
Sleep sensor (Wake/sleep times; sleep efficiency; wakings; restlessness)

4x Daily Ecological Momentary Assessments

Current
- Craving
- Affect
- Risky situations
- Stressful events
- Pleasant events
Future
- Risky situations
- Stressful events
- Confidence

Feature Engineering

Features based on recent past experiences (12, 24, 48, 72, 168 hours)
Min, max, and median response (all items)
History (count) of past lapses (item 1) and completed EMAs (compliance)
Raw scores and change scores (from baseline/all past responses)

Machine Learning Methods

Predict hour-by-hour probability of future lapse
Lapse window widths
- 1 hour
- 1 day
- 1 week

For our purposes today I wont dive deep into the machine learning methods but let me highlight a few high level details

We used these features I just described to make predictions about the hour-by-hour probability of a future lapse. We are developing separate models for three future lapse windows – lapses in the next hour, lapses in the next day, and lapses in the next week.

For example, if I was in recovery from an AUD, I could use these models to generate the probability that I would lapse after this symposium starting at 10am. One model would generate the probability of a lapse between 10am and 11 am today, the second would predict the probability of a lapse between 10 am today and 10 am tomorrow and the third would provide the probability of a lapse anytime between 10am today and 10 am next Friday

And of course, all of the models would only use data collected prior to 10 am today so that they are “predicting”, in the full sense of the word, into the future and not just demonstrating an association.

Machine Learning Methods

Statistical Algorithms
- ElasticNet GLM (e.g., LASSO, ridge regression)
- Random Forest
- XGBoost
- KNN
Model Tuning and Performance Evaluation
- Area under ROC curve (AUC) as primary performance metric
- Sensitivity, Specificity, Balanced accuracy, Positive predictive value
- Using grouped (by participant) 10-fold CV

1 Week: Probabilities for No Lapse and Lapse

Model predicts probability of lapse in next week for “new” observations in test set
Can panel predictions by Ground Truth (i.e., true lapse vs. no lapse observations
Want high probabilities to be high for true lapses and low for true no lapses

1 Week: Probabilities for No Lapse and Lapse

Model predicts probability of lapse in next week for “new” observations in test set
Can panel predictions for GROUND TRUTH lapse and no lapse observations
Want high probabilities to be high for true lapses and low for true no lapses
Need decision threshold for classification (.50 default)

Performance Metrics by Lapse Window Width

	Week	Day	Hour
AUC

Sensitivity	0.79
Specificity	0.85
Balanced Accuracy	0.82