Federated Reinforcement Learning for Developing Sepsis Patient Treatment Model

Abstract

Reinforcement learning (RL) for patient treatment models using electronic health records has been actively studied. Although training the models requires lots of actual patient treatment records, strict regulations for data privacy prevent aggregating medical data from multiple institutions into a centralized database. Thus, federated reinforcement learning (FRL), which can train the RL model without sharing data between institutions, is being introduced. This study aimed to propose an FRL framework for the healthcare domain and evaluate the performance of FRL models in realistic scenarios using two clinical benchmark datasets, Medical Information Mart for Intensive Care III database and eICU Collaborative Research Database v2.0. We constructed an FRL framework where local institutions collaborate to make optimal RL models without data sharing or raw data leakage. The reliability of the FRL framework was evaluated on basic, skewed, and realistic data distribution. The performances of FRL models were comparable to those learned from the ideal setting, where all institutions agree to share their datasets to train a global treatment model.