Update test-calibration-downloads.yml

[arjunsuresh]

No description provided.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[arjunsuresh]

@anandhu-eng It seems we don't have the automation scripts for igbh calibration dataset download. Can you please check?

[anandhu-eng]

Hi @arjunsuresh @pgmpablo157321 , is calibration dataset generation from debug dataset supported for RGAT? As per this line, it seems to support only full dataset.

If it does not support, adding GitHub test for calibration dataset download (for full dataset mode) would be difficult as GitHub runners have limited storage.

Edit:
reference: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

[pgmpablo157321]

@anandhu-eng it is not supported. It doesn't make sense to generate the calibration dataset for the debug dataset. As the dataset is currently setup, you can only download the calibration by downloading the whole dataset. And there is not a trivial solution, because the dataset is a graph, so the calibration is 'connected' to the whole dataset. I see 3 options:

• Skip the test for RGAT
• Add the option for create the calibration dataset for the debug setting. This will be testing that the debug dataset calibration is able to be downloaded.
• We could separate the calibration dataset from the 'full' dataset, but this is not trivial. We need to analyze how to do it and it might take some time

[anandhu-eng]

I think we could skip this test and add a test for generating calibration dataset from full dataset through a self-hosted GitHub runner? It could be run weekly. If the dataset is already present in the runner, the time for downloading the dataset could be skipped.