Outlier-based clinical monitoring and alerting
The goal of this project is to develop advanced computational, rather than expert-based,
solutions to detect anomalous patient-management actions. Our approach works by identifying
patient-management actions that are unusual with respect to actions used to manage comparable patients in past
and by raising a patient-specific alert when such a patient is prospectively encountered. The main hypothesis
investigated in this work is that statistical outliers in patient management actions correspond to medical
errors often enough to justify the alerting. The new approach is designed to complement existing knowledge-based detection and alerting methods that are clinically precise, but costly to build.
Knowledge-based vs. outlier-based error detection.
Classical approaches in detection of medical errors utilize a set of
expert-defined rules to detect anomalous
events. Apart from the fact that these systems often lack
well-defined statistical underpinning, adaptation of the systems to
a new environment is impossible without direct interaction with a
human. On the other hand, statistical methods rely on past data as
opposed to carefully extracted expert knowledge. The benefit of the
statistical approach is that it is adaptive and can be applied to a
wide range of conditions.
Conditional anomaly (outlier) detection.
Detection of unusual events becomes an important
issue in highly interconnected and computerized environments, mostly
because it is demanding both in terms of human labor and capital
investment. Anomaly detection provides a set of techniques that are
capable of identifying rare (or in other words anomalous) events in
in large datasets. However, existing anomaly
detection methodology focuses mostly on detection of
anomalous data entries in the datasets. This biases
anomalies to events that occur with a low prior probability, for example, patients
suffering from a rare disease or exhibiting a rare combination of symptoms. In our work
we are interested in finding anomalies in outcomes or patient management decisions with respect to patients who
suffer from the same or similar condition. To account for the
conditional (contextual) aspect of anomaly detection we have introduced, developed and continue developing a new
conditional anomaly detection framework that seeks to detect unusual values for one or a subset of variables (outcomes, decisions) given the values of the remaining variables.
Summary of Research Goals:
- Development of a new conditional outlier detection framework for identifying unusual patient management actions and outcomes.
- Evaluation of the detection methods on lab-order and medication-order decisions on past post-surgical cardiac and ICU patients data and in real-time.
Funding:
- NIH. 2R01GM088224 Real-time detection of deviations in clinical care in ICU data stream. PIs: Hauskrecht, Clermont, Cooper , August 2014-Nov 2019.
- NIH. 1R01GM088224 Detecting deviations in clinical care in ICU data stream. PIs: Hauskrecht and Clermont , August 2009-June 2013.
- NIH. 1R21LM009102 Evidence-based Anomaly Detection in Clinical Databases.
PI: Hauskrecht , April 2007-September 2009.
Project lead:
, PhD,
Professor of Computer Science
Gilles Clermont, MD, MSc,
Department of Critical Care Medicine,University of Pittsburgh School of Medicine
Gregory Cooper, MD, PhD ,
Vice Chair, Department of Biomedical Informatics, University of Pittsburgh School of Medicine
Current Team Members:
Past team members:
, a postdoctoral research associate
Hamed Valizadegan, a postdoctoral research associate
Iyad Batal, a CS PhD student, a postdoctoral research associate
Quang Nguyen, a CS PhD student
Michal Valko, a CS PhD student
Mahdi Pakdaman, a CS PhD student
Charmgil Hong, a CS PhD student, a postdoctoral research associate
Zitao Liu, a CS PhD student
Publications (in reverse chronological order):
Salim Malakouti and Milos Hauskrecht.
Hierarchical Adaptive Multi-task Learning Framework for Patient Diagnoses and Diagnostic Category Classification
Proceedings of the 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), San Diego, CA, November 2019.
Jeongmin Lee and Milos Hauskrecht.
Recent Context-aware LSTM for Clinical Event Time-series Prediction.
Artificial Intelligence In Medicine (AIME), Poznan, Poland, June 2019.
Salim Malakouti and Milos Hauskrecht.
Predicting patient's diagnoses and diagnostic categories from clinical-events in EHR data.
Artificial Intelligence In Medicine (AIME), Poznan, Poland, June 2019.
Charmgil Hong and Milos Hauskrecht.
Multivariate Conditional Outlier Detection: Identifying Unusual Input-Output Associations in Data.
31th International FLAIRS Conference, Melbourne, FL, May 2018.
Siqi Liu, Adam Wright, Milos Hauskrecht.
Change-Point Detection Method for Clinical Decision
Support System Rule Monitoring.
Artificial Intelligence in Medicine journal, 2018.
Zitao Liu, and Milos Hauskrecht.
A Personalized Predictive Framework for Multivariate Clinical Time Series via Adaptive Model Selection.
26th ACM International Conference on Information and Knowledge Management (CIKM), November 2017.
Siqi Liu, Dean Sittig, Adam Wright, and Milos Hauskrecht.
Change-Point Detection for Monitoring Clinical Decision Support Systems with a Multi-Process Dynamic Linear Model.
IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas City, MO, November 2017.
Siqi Liu, Adam Wright, and Milos Hauskrecht.
Change-point Detection Method for Clinical Decision Support Rule Monitoring.
16th International Conference on Artificial Intelligence in Medicine , Vienna, Austria, June 2017.
Siqi Liu, Adam Wright, and Milos Hauskrecht.
Online Conditional Outlier Detection for Nonstationary Time-series.
FLAIRS 30, FL, May 2017.
Milos Hauskrecht, I. Batal, C. Hong, Q. Nguyen, G. Cooper, S. Visweswaran, G. Clermont.
Outlier-based detection of unusual patient-management actions: An ICU study.
Journal of Biomedical Informatics, vol. 64, December 2016, pp 211-221.
Zitao Liu and Milos Hauskrecht.
Learning Linear Dynamical Systems from Multivariate Time Series: A Matrix Factorization Based Framework
SIAM International Conference on Data Mining (SDM), Miami, FL, 2016.
Zitao Liu and Milos Hauskrecht.
Learning Adaptive Forecasting Models from Irregularly Sampled Multivariate Clinical Data.
The 30th AAAI Conference on Artificial Intelligence(AAAI), Phoenix, AZ, 2016. (revised version with additional experimental results)
Yanbing Xue, and Milos Hauskrecht.
Active learning of classification models with Likert-scale feedback.
SIAM Data Mining Conference (SDM), Houston, TX, April 2017.
Charmgil Hong and Milos Hauskrecht.
Multivariate Conditional Outlier Detection and Its Clinical Application.
The 30th AAAI Conference on Artificial Intelligence(AAAI), Phoenix, AZ, 2016. (abstract)
Eric Heim, and Milos Hauskrecht.
Sparse Multidimensional Patient Modeling using Auxiliary Confidence Labels
Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Washington DC, 2015.
I. Batal, G. Cooper, D. Fradkin, J. Harrison, F. Moerchen, and M. Hauskrecht.
An Efficient Pattern Mining Approach for Event
Detection in Multivariate Temporal Data (review version)
Knowledge and Information Science Journal , 2016
Z. Liu, and M. Hauskrecht.
Clinical time series prediction: Towards a hierarchical dynamical system framework (review version)
Journal of Artificial Intelligence in Medicine, 2014
M. Pakdaman Naeini, G. Cooper, and M. Hauskrecht.
Binary classifier calibration using a Bayesian non-parametric
approach.
SIAM Data Mining Conference (SDM-15), Vancouver, Canada, April 2015.
C. Hong, I. Batal, and M. Hauskrecht.
A Generalized Mixture Framework for Multi-label Classification.
SIAM Data Mining Conference (SDM-15), Vancouver, Canada, April 2015.
Z. Liu, and M. Hauskrecht.
A Regularized Linear Dynamical System
Framework for Multivariate Time Series Analysis.
The Twenty-Ninth AAAI Conference on
Artificial Intelligence (AAAI-15), Austin, TX, January 2015.
M. Pakdaman Naeini, G. Cooper, and M. Hauskrecht.
Obtaining well-calibrated probabilities using Bayesian binning.
The Twenty-Ninth AAAI Conference on
Artificial Intelligence (AAAI-15), Austin, TX, January 2015.
C. Hong, I. Batal, and M. Hauskrecht.
A Mixtures-of-Trees Framework
for Multi-Label Classification
ACM International Conference on Information and Knowledge Management (CIKM), October 2014.
M. Pakdaman Naeini, I. Batal, Z. Liu, C. Hong, and M. Hauskrecht.
An Optimization-based Framework to Learn Conditional Random
Fields for Multi-label Classification
SIAM Data Mining Conference, April 2014.
M. Hauskrecht, S. Visweswaran, G. Cooper and G. Clermont.
Conditional outlier
approach for detection of unusual patient care actions.
Workshop on Late Breaking Research, AAAI Conference , Seattle, WA, July 2013.
M. Hauskrecht, I. Batal, M. Valko, S. Visweswaran, G. Cooper, G. Clermont.
Outlier-detection for patient monitoring and alerting .
Journal of Biomedical Informatics, 46:1, pages 47--55 February 2013.
M. Valko, B. Kveton, H. Valizadegan, G. Cooper, and M. Hauskrecht.
Conditional Anomaly Detection with Soft Harmonic Functions.
IEEE International Conference on Data Mining, Vancouver, Canada, December 2011.
M. Valko, H. Valizadegan, B. Kveton, GF. Cooper, and M. Hauskrecht.
Conditional Anomaly Detection Using Soft Harmonic Functions: An Application to Clinical Alerting,
ICML Workshop For Global Challenges, Bellevue, Washington, June 2011.
M. Hauskrecht, M. Valko, I.Batal, G. Clermont, S. Visweswaran, G. Cooper.
Conditional Outlier Detection for Clinical Alerting.
Annual American Medical Informatics Association (AMIA) Symposium , November 2010. [Homer Warner Award, AMIA 2010].
S. Visweswaran, J. Mezger, G. Clermont, M. Hauskrecht, G. Cooper.
Identifying Deviations from Usual Medical Care using a Statistical Approach.
Annual American Medical Informatics Association (AMIA) Symposium , November 2010.
M. Valko, and M. Hauskrecht.
Feature importance analysis for patient management
decisions.
13th International Congress on Medical Informatics , Cape Town, South Africa,
September 2010.
Michal Valko, Gregory Cooper, Amy Seybert, Shyam Visweswaran, Melissa Saul, Milos Hauskrecht.
Conditional anomaly detection methods for patient-management alert systems.
Proceedings of the ICML Workshop on Machine Learning in Health Care Applications, 25th International Machine Learning Conference, Helsinki, Finland, 2008.
M. Valko and M. Hauskrecht.
Distance metric learning for conditional anomaly detection.
In Proceedings of the
Twenty-First International Florida AI Research Society Conference (FLAIRS 2008), May 2008.
M. Hauskrecht, M. Valko, B. Kveton, S. Visweswaram, G. Cooper.
Evidence-based anomaly detection in clinical domains.
Proceedings of the Annual American Medical Informatics Association (AMIA) Symposium , 2007.
J. Mezger, G. F. Cooper, M. Hauskrecht, G. Clermont, S Visweswaran.
Detecting Deviations from Usual Medical Care.
Annual American Medical Infortmatics Association (AMIA) Conference,
poster abstract, 2007.
Code/methods developed and made available to public during the project:
The web page is updated by milos.