GLYFE is a glucose predictive models benchmark as described in the paper GLYFE: Review and Benchmark of Personalized Glucose Predictive Models in Type-1 Diabetes published in Medical & Biological Engineering & Computing in 2022.
To cite the paper:
@article{debois2022glyfe,
title={GLYFE: review and benchmark of personalized glucose predictive models in type 1 diabetes},
author={De Bois, Maxime and Yacoubi, Moun{\^\i}m A El and Ammi, Mehdi},
journal={Medical \& Biological Engineering \& Computing},
volume={60},
number={1},
pages={1--17},
year={2022},
publisher={Springer}
}
These instructions will help you get the data needed to run the benchmark as well as to develop new glucose predictive models.
To simulate the data need to run the benchmark, you will need a MATLAB (here, the R2018b version under Windows has been used) and a T1DMS licence (v3.2.1).
To run the benchmark, you will need the following Python 3.7.6
libraries
matplotlib 3.1.3
numpy 1.18.1
pandas 1.0.1
patsy 0.5.1
pip 20.0.1
pytorch 1.4.0
scikit-learn 0.22.1
scipy 1.4.1
setuptools 45.2.0
statsmodels 0.12.0.dev0
- To access the OhioT1DM data, one should refer to the paper by Marling et al.
- Once obtained, the two folders
OhioT1DM-testing
andOhioT1DM-training
should be place under the./data/ohio/
directory (create if needed).
- Copy and paste the
GLYFE/T1DMS/GLYFE.scn
scenario file into thescenario
folder of the T1DMS installation folder (namedUVa PadovaT1DM Simulator v3.2.1
). The file describes the daily scenario the virtual patients will follow during the simulation. - Copy and paste the
./T1DMS/results2csv.m
file into the T1DMS installation folder. - Modify the Simulink schematics:
- Open the file
testing_platform.slx
under the T1DMS installation folder in Matlab. - Double click the "STD_Treat_v3_2" big orange block.
- Modify the schematics as follows:
- Open the file
- In the Matlab console type
rng(1,"twister")
- Launch the T1DMS GUI by typing
Simulator
- In the GUI:
- Load Scenario:
Select Scenario Files
=>GLYFE.scn
- Select Subject:
Add List > Adults
- Hardware:
Built-in
- Sensors:
CGM
- Pumps:
Generic
- Pre-tests/screening:
none
- enter random seed:
1
- Load Scenario:
- Click on
Run Simulation
(it may take several hours, updates are displayed in the console) - After the simulation has ended:
- Say OK to the pop-up asking for a file name; the results are contained in the
sim_results.mat
file; - Convert the results files (give name) into CSV by running
results2csv("sim_results.mat","t1dms_adult")
. The resulting files are located in thedata
folder; - Compute the SHA-256 (data) checksum of the
t1dms_adult
folder and make sure it equals:8E7A26F7DB5B3B627AD4838D796ED9AF2C8DD90981B0EDE8A7C34D9EA
(checksum on data only) or6DA007A0D5E4EEEDFF9D350796A2D9AD52CD331749ABF8C7E565AF345791824D
(checksum on data and names)
- Say OK to the pop-up asking for a file name; the results are contained in the
- Copy the
t1dms_adult
folder into theGLYFE
folder.
There are two different ways to use the benchmark code: either rerun the models presented on the original paper or run the benchmark on new models.
Go in the GLYFE directory and run the command: python main.py --dataset={dataset} --subject={subject} --model={model} --ph={ph} --params={params} --exp={exp} --mode={mode} --plot={plot} --log={logs}
where:
dataset
is the name of the dataset on which the subject will be selected (eithert1dms_adult
orohio
subject
is the name of the subject on which the benchmark will be run on (1
to10
for thet1dms_adult
dataset, and559
,563
,570
,575
,588
,591
for theohio
dataset);model
is the name of the model to be used; the following are implemented:arimax
,base
,elm
,ffnn
,gp
,lstm
,poly
,svr
;ph
is the prediction horizon (in minutes); in the benchmark, values of30
,60
, and120
have been used;exp
name of the experimental run, to keep track of different tries for the same model;mode
eithervalid
ortest
, specifies wether the evaluation is to be made on the validation or testing set;params
(optional) gives the name of the parameter file (in the./processing/params
folder) to use (by default it has the same name as the name of the model);logs
either1
or0
, tells to (or not to) plot the results for the first day at the end of the run;logs
(optional) specifies the file in which the logs of the benchmark shall be written in (inside thelogs
directory); by default, the logs are printed in the console;
Here is an example of a main run: python main.py --dataset=ohio --subject=559 --model=base --ph=30 --exp=myfirstrun --mode=valid --plot=1
One can run several subjects, models, or prediction horizons at once by using the batch_main
function.
To create a new model, one should follow these steps:
- Create the model:
- Create a file inside the
./processing/models
directory, its name should be the name of the model and to be lowercase; - Inside the created file, create a class which name should be the name of the file (uppercase); it should inherit the
models.predictor.Predictor
class; - Override the functions
fit
,predict
(and_reshape
if needed) following the instructions inmodels._template.npy
;
- Create a file inside the
- Create the parameter file:
- Create a file with the same name as the model (lowercase) in the
./processing/params
directory (different names can be used to track experiments); - Inside the created file, create two dictionaries named
parameters
andsearch
:parameters
represents the hyperparameters of the model; one can look atmodels._template.npy
to see how to fill it;search
represents the list of the hyperparameters that are automatically optimized with the coarse-to-fine methodology presented in the paper; as for theparams
dictionary, one can look at the_template
file to fill it;
- Create a file with the same name as the model (lowercase) in the
- To run the benchmark on the newly created model, just run the
main
function with the name of the model as themodel
parameter!
To add a new dataset newdataset
that will follow the benchmarking process, one should:
- add the new dataset into
data/newdataset
; - create a specific
preprocessing/preprocessing_newdataset.py
file with apreprocessing_newdataset(dataset, subject, ph, hist, day_len)
function representing the whole preprocessing pipeline for this dataset (see other preprocessing files for the other datasets for examples); - append the name of the file into the preprocessing_per_dataset dictionary in
preprocessing/preprocessing.py
;
To create a new evaluation metric newmetric
, one should:
- create the
postprocessing/metrics/newmetric.py
file, with theNEWMETRIC(results)
function returning a numerical score; - append the metric to the
postprocessing/metrics/__init__.py
file; - append the line
newmetric_score = np.mean([newmetric.NEWMETRIC(res_day) for res_day in self.results])
in thepostprocessing.results.ResultsSubject.compute_results()
function and a field in theresults
dictionary returned by the function;