Imviz parser

[pllim]

Fix #510

I cannot test this until #540 is resolved because jwst is not compatible with Windows.

Reference: https://github.com/glue-viz/glue/blob/master/glue/core/data_factories/fits.py

Proof-of-concept

Successfully loaded the following locally in notebook:

# JWST/NIRCAM ASDF-in-FITS
imviz.load_data('/path/to/jw1069001001_01203_00002_nrca1_level2_cal.fits')  # SCI
imviz.load_data('/path/to/jw1069001001_01203_00002_nrca1_level2_cal.fits[DQ,1]')
imviz.load_data('/path/to/jw1069001001_01203_00002_nrca1_level2_cal.fits[VAR_POISSON]')

# HST/ACS Drizzled FITS
imviz.load_data('/path/to/jbt7a3020_drz.fits')  # SCI
imviz.load_data('/path/to/jbt7a3020_drz.fits[WHT,1]')
imviz.load_data('/path/to/jbt7a3020_drz.fits[CTX]')

As you can see, JWST filenames are a little long, so I need to scroll to get the full screenshot(s).

TODO

• Tom R, how do I know if the parser used the ASDF logic?
• Tom R, how to tell it to not load extensions that don't make sense?
• Tom R, why is it not auto-displaying? How to make it?
• POs, where are the list of images to test against as acceptance criteria? See Imviz parser #541 (comment)
• Did I break circular region selection in Tom's example notebook? (No, I didn't. See [BUG] Imviz: NotImplementedError: ROIs of type EllipticalROI are not yet supported #569)
• Do we need test in CI? How? (Yes, use remote_data and stuff from PO.)

[pllim]

I don't have anything official to test with, so using one from my own stash: https://stsci.box.com/s/hwrc5reqygmmv2rl3yvvz90l7ryjac6h (jw1069001001_01203_00002_nrca1_level2_cal.fits)

[astrofrog]

Do we need test in CI? How?

If you want a shortcut, the easiest would be to actually call parse_data with app being a MagicMock and then passing an HDUList for the file object - that way you can check that add_data and so on are called correctly and you can check that the data object seems ok.

[eteq]

Re: which images to treat as acceptance criteria - @pllim, I think you hit the two main items at the outset! I.e.

1. A JWST level 2 image
2. A drizzled HST image (if you really want we could also do in flt with the distortions and all, but not mandatory)

I think ideally we check both of those in CI following @astrofrog's suggestion in #541 (comment).

That leaves open the question where to get the images. I think the answer for now is to try to pick a file for 1 that we're already using in some notebook (like maybe https://data.science.stsci.edu/redirect/JWST/jwst-data_analysis_tools/stellar_photometry/jw01072001001_01101_00001_nrcb1_cal.fits ? although I'm not 100% sure that's level 2, so might have to find another one) and for 2 get some convenient drz file on MAST - for example https://mast.stsci.edu/api/v0.1/Download/file?bundle_name=MAST_2021-04-21T1828.sh&uri=mast:HST/product/jclj01010_drz.fits would do just fine.

[astrofrog]

Looks good to me so far! Just a couple small comments below

[pllim]

I have no idea how these got into the original notebook from Tom R. But when I update it, these are removed by jupyter. 🤷

[astrofrog]

Maybe a jupyter version difference? Fine to remove though :)

[pllim]

The two test files given by Erik are not exactly small; hopefully not enough to trigger quota limits. 😬 🤞

• jw01072001001_01101_00001_nrcb1_cal.fits 113 MB
• jclj01010_drz.fits 208 MB

[pllim]

After this is merged, if no further changes, @duytnguyendtn needs to update branch protection rules for CI, since I changed the job name a little. FYI.

[pllim]

FWIW, the new tests passed locally...

[astrofrog]

Looks good to me! Thanks for adding the tests 😀

[pllim]

Great! @duytnguyendtn , any other approval needed for merge?

[pllim]

OK, bug fixed. Should be ready ready now.

[rosteen]

Looks good to me. I did some testing from the notebook (including loading the JWST file) as well as running the tests locally. I hadn't looked at the Imviz example notebook lately, Imviz is looking good so far!

Is there an issue open for the CI branch protection rule change you mentioned @pllim , or was that already addressed?

[pllim]

I can't really open issue about branch protection until this is merged.

[rosteen]

Wait, that last test needs to pass before I can merge this. But that test can't pass until @duytnguyendtn updates the branch protection rules for CI before the test will run? Is that right?

[pllim]

No, it is now called "Python 3.8 with coverage checking, all deps, and remote data" but only after this PR is merged. We're stuck with chicken and egg situation.

[duytnguyendtn]

Updated the test name. Thanks for your patience!