KeyError: "Unable to synchronously open object (object 'sen2_10_cloud_post' doesn't exist)"

[mattxsong]

Hello there,

Thank you for your excellent work and sharing your code and datasets.
However, the bug occurred when I tried to preprocess the meta h5 data to the machine learning-ready format.
The bug is shown in the following, which seems to be the disappearance of one 'object,'sen2_10_cloud_post'.
Do you know how I can resolve this problem?

Thanks.

--- STAGE 1 ---

(1/15) FLOGA_dataset_2019_sen2_10_mod_500.h5: 0%| | 0/15 [01:42<?, ?it/s]
Traceback (most recent call last):
File "/FLOGA-main/create_dataset.py", line 469, in
export_patches(floga_path, out_path, out_size, args.out_format, args.sea_ratio)
File "/FLOGA-main/create_dataset.py", line 159, in export_patches
img = event_imgs[img_name][:]
~~~~~~~~~~^^^^^^^^^^
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File ".conda/envs/ai4sci/lib/python3.12/site-packages/h5py/_hl/group.py", line 357, in getitem
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5o.pyx", line 257, in h5py.h5o.open
KeyError: "Unable to synchronously open object (object 'sen2_10_cloud_post' doesn't exist)"

[mattxsong]

I found that this bug occurred when the image_name is 'sen2_{sen_gsd}_cloud_post' or 'sen2_{sen_gsd}_cloud_pre'.

[paren8esis]

Hello, thanks for catching this. The issue was that Copernicus provides Sentinel-2 cloud masks only for 20m and 60m GSD. Therefore we have not included them in the provided HDF files. I fixed the create_dataset.py script to account for this.

In the future we plan to interpolate the Sentinel-2 cloud masks to 10m GSD and add them to the HDF files.