After the model training is completed, the following problem occurs, how to solve it?

[tiancpeng]

2024-08-01 23:32:05.366601: predicting 314
resizing data, order is 1
data shape (2, 1, 512, 512)
2024-08-01 23:32:05.456242: predicting 315
resizing data, order is 1
data shape (2, 1, 512, 512)
2024-08-01 23:32:05.554324: predicting 316
resizing data, order is 1
data shape (2, 1, 512, 509)
2024-08-01 23:32:05.644877: predicting 317
resizing data, order is 1
data shape (2, 1, 512, 509)
2024-08-01 23:32:05.733577: predicting 318
resizing data, order is 1
data shape (2, 1, 512, 512)
2024-08-01 23:32:05.824886: predicting 319
resizing data, order is 1
data shape (2, 1, 512, 512)
2024-08-01 23:32:05.914389: predicting 320
resizing data, order is 1
data shape (2, 1, 512, 512)
resizing data, order is 1
data shape (2, 1, 512, 512)
2024-08-01 23:32:09.154323: Validation complete
2024-08-01 23:32:09.154653: Mean Validation Dice: 0.9984738300261968
Exception in thread Thread-5 (results_loop):
Traceback (most recent call last):
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/threading.py", line 953, in run
Exception in thread Thread-4 (results_loop):
Traceback (most recent call last):
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/site-packages/batchgenerators/dataloading/nondet_multi_threaded_augmenter.py", line 125, in results_loop
self.run()
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/threading.py", line 953, in run
raise e
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/site-packages/batchgenerators/dataloading/nondet_multi_threaded_augmenter.py", line 103, in results_loop
self._target(*self._args, **self._kwargs)
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/site-packages/batchgenerators/dataloading/nondet_multi_threaded_augmenter.py", line 125, in results_loop
raise RuntimeError("One or more background workers are no longer alive. Exiting. Please check the "
RuntimeError: One or more background workers are no longer alive. Exiting. Please check the print statements above for the actual error message
raise e
File "/public/home/xukai_tcp_2607/.conda/envs/uxlstm/lib/python3.10/site-packages/batchgenerators/dataloading/nondet_multi_threaded_augmenter.py", line 103, in results_loop
raise RuntimeError("One or more background workers are no longer alive. Exiting. Please check the "
RuntimeError: One or more background workers are no longer alive. Exiting. Please check the print statements above for the actual error message

[matteodelucchi]

The same happened to me (NVIDIA 3090, 3d_fullres), and reducing the batch size resolved the issue.

[liubaoning111]

同样的情况也发生在我身上（NVIDIA 3090，3d_fullres），减少批量大小解决了这个问题。
Hello, I followed the steps, and this error occurred when I was training, may I ask if it happened on your side?
usage: nnUNetv2_train [-h] [-tr TR] [-p P] [-pretrained_weights PRETRAINED_WEIGHTS] [-num_gpus NUM_GPUS] [--use_compressed] [--npz] [--c] [--val] [--val_best] [--disable_checkpointing] [-device DEVICE] dataset_name_or_id configuration fold
nnUNetv2_train: error: unrecognized arguments: -lr 0.01 -bs 8