ENH: Add additional environment to `tox` to run notebooks #35

jhlegarreta · 2024-12-22T18:20:21Z

Add an additional environment to tox to run notebooks.

Add a new table to pyproject.toml to host the jupyter, nbclient and nbmake dependencies so that the notebooks can be run.

Install the necessary TeX packages in the test CI so that LaTeX decoration in plots can be employed.

codecov · 2024-12-22T18:31:43Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@0fefaf3). Learn more about missing BASE report.
Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #35   +/-   ##
=======================================
  Coverage        ?   67.45%           
=======================================
  Files           ?       20           
  Lines           ?      977           
  Branches        ?      129           
=======================================
  Hits            ?      659           
  Misses          ?      265           
  Partials        ?       53

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

oesteban · 2025-01-14T09:14:23Z

Looks like we will need to also install some latex packages in the test environment.

jhlegarreta · 2025-01-18T18:05:57Z

So the files are there:
https://github.com/nipreps/nifreeze/actions/runs/12845678824/job/35820013540?pr=35#step:12:1034

tree /home/runner/work/nifreeze/nifreeze
  /home/runner/work/nifreeze/nifreeze
  ├── CHANGES.rst
  ├── CONTRIBUTING.md
  ├── LICENSE
  ├── NOTICE
  ├── README.rst
  ├── coverage.xml
  ├── docs
 (...)
  │   ├── changes.rst
  │   ├── conf.py
  │   ├── developers.rst
  │   ├── index.rst
  │   ├── installation.rst
  │   ├── links.rst
  │   ├── notebooks
  │   │   ├── bold_realignment.ipynb
  │   │   ├── dmri_covariance.ipynb
  │   │   └── dwi_gp_estimation.ipynb
  │   ├── running.rst
  │   └── usage.rst
  ├── pyproject.toml

but somehow we are not providing tox with the appropriate command/the wildcard is not being processed correctly:

  ERROR: file or directory not found: /home/runner/work/nifreeze/nifreeze/docs/notebooks/*ipynb

The following works at the root of the project (with or without the dot):

pytest --nbmake docs/notebooks/*.ipynb

I've run out of ideas here.

jhlegarreta · 2025-01-18T22:02:02Z

This is working now:
https://github.com/nipreps/nifreeze/actions/runs/12847656593/job/35824190915?pr=35#step:12:1040

But the LaTeX package issues persist.

jhlegarreta · 2025-01-18T23:31:11Z

Finally got this sorted out.

The remaining error is related to issue #25:

FAILED docs/notebooks/bold_realignment.ipynb::bold_realignment.ipynb - FileNotFoundError: [Errno 2] No such file or directory: '/data/datasets/ismrm_sample.txt'

There is a warning unrelated to the notebooks:

.tox/notebooks/lib/python3.12/site-packages/_pytest/config/__init__.py:1441
    /home/runner/work/nifreeze/nifreeze/.tox/notebooks/lib/python3.12/site-packages/_pytest/config/__init__.py:1441: PytestConfigWarning: Unknown config option: env
    
      self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

Not sure how to fix that. I'd leave it for a separate PR even if I know that we are now considering warnings as errors.

This is ready to be merged on my end.

Add an additional environment to `tox` to run notebooks. Add a new table to `pyproject.toml` to host the `jupyter`, `nbclient` and `nbmake` dependencies so that the notebooks can be run. Install the necessary TeX packages in the test CI so that LaTeX decoration in plots can be employed. Install the necessary additional Python packages used across the notebooks. Add a helper Python script to actually run the notebooks since `tox` does not expand wildcards currently.

Store bold realignment results in a folder that actually exists. Fixes: ``` FAILED docs/notebooks/bold_realignment.ipynb::bold_realignment.ipynb - PermissionError: [Errno 13] Permission denied: '/data/derivatives' ``` raised for example in: https://github.com/nipreps/nifreeze/actions/runs/12847776534/job/35824459079?pr=35#step:12:1684

oesteban · 2025-01-20T16:54:22Z

Running the bold_realignment.ipynb notebook is quite involved -- can we add it as an exception?

`ismrm_sample.txt``:

ds000005/sub-01/func/sub-01_task-mixedgamblestask_run-03_bold.nii.gz
ds000005/sub-02/func/sub-02_task-mixedgamblestask_run-01_bold.nii.gz
ds000005/sub-06/func/sub-06_task-mixedgamblestask_run-02_bold.nii.gz
ds000005/sub-15/func/sub-15_task-mixedgamblestask_run-03_bold.nii.gz
ds000005/sub-14/func/sub-14_task-mixedgamblestask_run-01_bold.nii.gz
ds000030/sub-10570/func/sub-10570_task-bht_bold.nii.gz
ds000030/sub-10460/func/sub-10460_task-scap_bold.nii.gz
ds000030/sub-10304/func/sub-10304_task-bart_bold.nii.gz
ds000030/sub-10376/func/sub-10376_task-taskswitch_bold.nii.gz
ds000030/sub-11097/func/sub-11097_task-stopsignal_bold.nii.gz
ds000210/sub-28/func/sub-28_task-cuedSGT_run-02_echo-1_bold.nii.gz
ds000210/sub-16/func/sub-16_task-cuedSGT_run-02_echo-1_bold.nii.gz
ds000210/sub-24/func/sub-24_task-cuedSGT_run-03_echo-1_bold.nii.gz
ds000210/sub-18/func/sub-18_task-cuedSGT_run-04_echo-1_bold.nii.gz
ds000210/sub-09/func/sub-09_task-cuedSGT_run-03_echo-1_bold.nii.gz
ds002785/sub-0202/func/sub-0202_task-workingmemory_acq-seq_bold.nii.gz
ds002785/sub-0214/func/sub-0214_task-anticipation_acq-seq_bold.nii.gz
ds002785/sub-0139/func/sub-0139_task-restingstate_acq-mb3_bold.nii.gz
ds002785/sub-0178/func/sub-0178_task-workingmemory_acq-seq_bold.nii.gz
ds002785/sub-0016/func/sub-0016_task-emomatching_acq-seq_bold.nii.gz

For each ds00XXXX folder, the dataset has to be datalad cloned and then the specific runs must be datalad gotten.

For each of those files a realignment process will be kicked off. I think this is way more than GHA will handle.

Skip BOLD realignment notebook in GHA CI testing, as it involves running a realignment process for several DataLad datasets, which requires long running times.

jhlegarreta · 2025-01-20T17:40:26Z

can we add it as an exception?

Skipping it. I will open an issue so that a weekly CI is set to run all notebooks.

`ismrm_sample.txt``:

Can the file be added to https://gin.g-node.org/nipreps-data/tests-nifreeze, please ?

jhlegarreta · 2025-01-20T17:54:14Z

I will open an issue so that a weekly CI is set to run all notebooks.

Issue #60.

jhlegarreta · 2025-01-20T18:48:12Z

Can the file be added to https://gin.g-node.org/nipreps-data/tests-nifreeze, please ?

Sorry, did not realize I could add it myself: https://gin.g-node.org/nipreps-data/tests-nifreeze/pulls/2

jhlegarreta force-pushed the RunNotebooksInTox branch 2 times, most recently from d3e9274 to be31b08 Compare December 22, 2024 21:19

jhlegarreta force-pushed the RunNotebooksInTox branch 7 times, most recently from ed83e67 to 76f3e6a Compare January 18, 2025 16:34

jhlegarreta force-pushed the RunNotebooksInTox branch 6 times, most recently from 776605a to 049a576 Compare January 18, 2025 22:01

jhlegarreta force-pushed the RunNotebooksInTox branch 4 times, most recently from 13e5000 to 953b601 Compare January 18, 2025 23:09

jhlegarreta added 2 commits January 19, 2025 14:11

jhlegarreta force-pushed the RunNotebooksInTox branch from 953b601 to c268d04 Compare January 19, 2025 19:12

Merge branch 'main' into RunNotebooksInTox

8aebb9c

ENH: Skip BOLD realignment notebook in GHA CI testing

56f6b2c

Skip BOLD realignment notebook in GHA CI testing, as it involves running a realignment process for several DataLad datasets, which requires long running times.

jhlegarreta mentioned this pull request Jan 20, 2025

Set a cron job to run extended testing #60

Open

oesteban approved these changes Jan 20, 2025

View reviewed changes

oesteban merged commit a8b4dd5 into nipreps:main Jan 20, 2025
10 checks passed

jhlegarreta deleted the RunNotebooksInTox branch January 20, 2025 20:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add additional environment to `tox` to run notebooks #35

ENH: Add additional environment to `tox` to run notebooks #35

jhlegarreta commented Dec 22, 2024 •

edited

Loading

codecov bot commented Dec 22, 2024 •

edited

Loading

oesteban commented Jan 14, 2025

jhlegarreta commented Jan 18, 2025 •

edited

Loading

jhlegarreta commented Jan 18, 2025

jhlegarreta commented Jan 18, 2025

oesteban commented Jan 20, 2025

jhlegarreta commented Jan 20, 2025

jhlegarreta commented Jan 20, 2025

jhlegarreta commented Jan 20, 2025

ENH: Add additional environment to tox to run notebooks #35

ENH: Add additional environment to tox to run notebooks #35

Conversation

jhlegarreta commented Dec 22, 2024 • edited Loading

codecov bot commented Dec 22, 2024 • edited Loading

Codecov Report

oesteban commented Jan 14, 2025

jhlegarreta commented Jan 18, 2025 • edited Loading

jhlegarreta commented Jan 18, 2025

jhlegarreta commented Jan 18, 2025

oesteban commented Jan 20, 2025

jhlegarreta commented Jan 20, 2025

jhlegarreta commented Jan 20, 2025

jhlegarreta commented Jan 20, 2025

ENH: Add additional environment to `tox` to run notebooks #35

ENH: Add additional environment to `tox` to run notebooks #35

jhlegarreta commented Dec 22, 2024 •

edited

Loading

codecov bot commented Dec 22, 2024 •

edited

Loading

jhlegarreta commented Jan 18, 2025 •

edited

Loading