Optimizing Clinical Early Warning Models to Meet False Alarm Constraints

Abstract

Quals talk:

Deployed early warning systems often suffer from high false alarm rates, especially in clinical settings where classes are heavily imbalanced. A high volume of false alarms can cause mistrust of alerts by clinical staff (known as alarm fatigue) and deter focus from patients who are truly-at- risk. Despite the widespread need to control false alarms in early warning systems, the dominant classifier training paradigm remains minimizing cross entropy, a loss function with no direct relationship to false alarms. In my research [1], we define a new tractable optimization objective that trains classifiers to maximize the fraction of truly-at-risk patients who are helped by alerts (“recall”) while enforcing that the ratio of false alarms to total alarms does not exceed a provided threshold. We develop a family of tight surrogate sigmoid bounds which makes optimization with our objective feasible for a broad family of classifiers trainable with gradient descent, such as generalized linear models and neural networks. I will demonstrate the validity of our method in predicting risk of mortality on two large electronic health record datasets, where our method satisfies a desired constraint on false alarms while achieving better recall than alternatives. Building on these empirical results, I will point to future directions for the rest of my graduate studies. First, we plan to assess this model in a deployment setting with multiple sites, via a collaboration with a major hospital system. Second, we hope to use our new objective to train flexible latent variable models, simultaneously delivering high-quality predictions of clinical deterioration while handling missing data better than our current discriminative baselines.

[1] An early preprint of this work was accepted at the Interpretable Machine Learning for Healthcare (co-located in ICML 2021) URL : https://www.cse.cuhk.edu.hk/~qdou/public/IMLH2021_files/45_CameraR eady_optimizing_clinical_ews_to_meet_false_alarm_constraints.pdf

Please join meeting at 574 Boston Ave, Room 316 or join via Zoom.

Join Zoom meeting at: https://tufts.zoom.us/my/preetish

Password: see colloquium email.

Dial in not an option.