Cubeviz spectral extraction through plugin (#2827)

CHANGES.rst

@@ -10,6 +10,12 @@ New Features
 Cubeviz
 ^^^^^^^
 
+- Automatic spectral extraction now goes through the logic of the spectral extraction plugin for
+  self-consistency.  This results in several breaking changes to data-labels and ``get_data``
+  (the extracted spectra are now given dedicated data-labels instead of referring to them by
+  the label of the flux cube) as well as to several plugins: model fitting, gaussian smooth,
+  line analysis, and moment maps. [#2827]
+
 Imviz
 ^^^^^
 
@@ -28,6 +34,14 @@ API Changes
 Cubeviz
 ^^^^^^^
 
+- ``get_data`` no longer supports ``function`` or ``spatial_subset`` as arguments.  To access
+  an extracted 1D spectrum, use the Spectral Extraction plugin or the automatic extraction of
+  spatial subsets, and refer to the data-label assigned to the resulting 1D spectrum. [#2827]
+
+- Several plugins that take 1D spectra replace ``spatial_subset`` with referring to the 1D
+  spectrum in ``dataset``.  This affects: model fitting, gaussian smooth, line analysis,
+  and moment maps. [#2827]
+
 Imviz
 ^^^^^
 

docs/cubeviz/export_data.rst

@@ -17,25 +17,15 @@ Spatial Regions
     :ref:`Export Spatial Regions <imviz_export>`
         Documentation on how to export spatial regions.
 
-Since Specviz can be accessed from Cubeviz, the following line of code
-can be used to extract the *spectrum* of a spatial subset named "Subset 1":
+Any cube (or extracted spectrum) can be extracted from cubeviz:
 
 .. code-block:: python
 
-    subset1_spec1d = cubeviz.specviz.get_spectra(spectral_subset="Subset 1")
+    subset1_spec1d = cubeviz.get_data("Spectrum (Subset 1, sum)")
 
-An example without accessing Specviz:
 
-.. code-block:: python
-
-    subset1_spec1d = cubeviz.get_data(data_label=flux_data_label, 
-                                      spatial_subset="Subset 1",
-                                      function="mean")
-
-Note that in the above example, the ``function`` keyword is used to tell Cubeviz
-how to collapse the flux cube down to a one dimensional spectrum - this is not 
-necessarily equivalent to the collapsed spectrum in the spectrum viewer, which 
-may have used a different collapse function.
+To use a ``function`` other than sum, use the :ref:`Spectral Extraction <spectral-extraction>` plugin
+first to create a 1D spectrum and then refer to it by label in ``get_data``.
 
 To get all subsets from the spectrum viewer:
 
@@ -58,11 +48,12 @@ To access the spatial regions themselves:
     :ref:`Export Spectra <specviz-export-data>`
         Documentation on how to export data from the ``spectrum-viewer``.
 
-The following line of code can be used to extract a spectral subset named "Subset 2":
+The following line of code can be used to extract 1D spectrum either automatically extracted
+or extracted manually through the :ref:`Spectral Extraction <spectral-extraction>` plugin:
 
 .. code-block:: python
 
-    subset2_spec1d = cubeviz.specviz.get_spectra("Subset 2")
+    subset2_spec1d = cubeviz.get_data("Spectrum (Subset 2, sum)")
 
 3D Data Cubes
 =============
@@ -85,16 +76,6 @@ where the mask (if available) is as defined in
 
     mydata.write("mydata.fits", format="jdaviz-cube")
 
-Data can also be accessed directly from ``data_collection`` using the following code:
-
-.. code-block:: python
-
-    cubeviz.app.data_collection[0]
-
-Which is returned as a `~glue.core.data.Data` object. The
-`~glue.core.data_collection.DataCollection` object
-can be indexed to return all available data (i.e., not just using 0 like in the
-previous example).
 
 .. _cubeviz-export-model:
 

docs/cubeviz/plugins.rst

@@ -292,9 +292,6 @@ Spectral Extraction
 
 .. image:: ../img/cubeviz_spectral_extraction.png
 
-.. note::
-
-    Spectral Extraction requires at least version 5.3.2 of astropy.
 
 The Spectral Extraction plugin produces a 1D spectrum from a spectral
 cube. The 1D spectrum can be computed via the sum, mean, minimum, or

jdaviz/app.py

@@ -152,7 +152,7 @@ def to_unit(self, data, cid, values, original_units, target_units):
                     eqv = u.spectral_density(spec.spectral_axis)
 
         else:  # spectral axis
-            eqv = u.spectral()
+            eqv = u.spectral() + u.pixel_scale(1*u.pix)
 
         return (values * u.Unit(original_units)).to_value(u.Unit(target_units), equivalencies=eqv)
 
@@ -412,6 +412,8 @@ def __init__(self, configuration=None, *args, **kwargs):
         self._get_object_cache = {}
         self.hub.subscribe(self, SubsetUpdateMessage,
                            handler=self._on_subset_update_message)
+        self.hub.subscribe(self, SubsetDeleteMessage,
+                           handler=self._on_subset_delete_message)
 
         # Store for associations between Data entries:
         self._data_associations = self._init_data_associations()
@@ -423,8 +425,7 @@ def __init__(self, configuration=None, *args, **kwargs):
                            handler=self._on_layers_changed)
         self.hub.subscribe(self, SubsetCreateMessage,
                            handler=self._on_layers_changed)
-        self.hub.subscribe(self, SubsetDeleteMessage,
-                           handler=self._on_layers_changed)
+        # SubsetDeleteMessage will also call _on_layers_changed via _on_subset_delete_message
 
     def _on_plugin_table_added(self, msg):
         if msg.plugin._plugin_name is None:
@@ -433,7 +434,7 @@ def _on_plugin_table_added(self, msg):
         key = f"{msg.plugin._plugin_name}: {msg.table._table_name}"
         self._plugin_tables.setdefault(key, msg.table.user_api)
 
-    def _update_live_plugin_results(self, trigger_data_lbl=None, trigger_subset=None):
+    def _iter_live_plugin_results(self, trigger_data_lbl=None, trigger_subset=None):
         trigger_subset_lbl = trigger_subset.label if trigger_subset is not None else None
         for data in self.data_collection:
             plugin_inputs = data.meta.get('_update_live_plugin_results', None)
@@ -455,18 +456,34 @@ def _update_live_plugin_results(self, trigger_data_lbl=None, trigger_subset=None
                                for attr in data_subs]):
                     # trigger parent data of subset does not match subscribed data entries
                     continue
+            yield (data, plugin_inputs)
+
+    def _update_live_plugin_results(self, trigger_data_lbl=None, trigger_subset=None):
+        for data, plugin_inputs in self._iter_live_plugin_results(trigger_data_lbl, trigger_subset):
             # update and overwrite data
             # make a new instance of the plugin to avoid changing any UI settings
             plg = self._jdaviz_helper.plugins.get(data.meta.get('Plugin'))._obj.new()
             if not plg.supports_auto_update:
                 raise NotImplementedError(f"{data.meta.get('Plugin')} does not support live-updates")  # noqa
             plg.user_api.from_dict(plugin_inputs)
+            # keep auto-updating, even if the option is hidden from the user API
+            # (can remove this line if auto_update is exposed to the user API in the future)
+            plg.add_results.auto_update_result = True
             try:
                 plg()
             except Exception as e:
                 self.hub.broadcast(SnackbarMessage(
                     f"Auto-update for {plugin_inputs['add_results']['label']} failed: {e}",
                     sender=self, color="error"))
+                # TODO: should we delete the entry (but then any plot options, etc, are lost)
+                # self.vue_data_item_remove({'item_name': data.label})
+
+    def _remove_live_plugin_results(self, trigger_data_lbl=None, trigger_subset=None):
+        for data, plugin_inputs in self._iter_live_plugin_results(trigger_data_lbl, trigger_subset):
+            self.hub.broadcast(SnackbarMessage(
+                f"Removing {data.label} due to deletion of {trigger_subset.label if trigger_subset is not None else trigger_data_lbl}",  # noqa
+                sender=self, color="warning"))
+            self.vue_data_item_remove({'item_name': data.label})
 
     def _on_add_data_message(self, msg):
         self._on_layers_changed(msg)
@@ -478,6 +495,10 @@ def _on_subset_update_message(self, msg):
         if msg.attribute == 'subset_state':
             self._update_live_plugin_results(trigger_subset=msg.subset)
 
+    def _on_subset_delete_message(self, msg):
+        self._remove_live_plugin_results(trigger_subset=msg.subset)
+        self._on_layers_changed(msg)
+
     def _on_plugin_plot_added(self, msg):
         if msg.plugin._plugin_name is None:
             # plugin was instantiated after the app was created, ignore
@@ -2123,7 +2144,14 @@ def set_data_visibility(self, viewer_reference, data_label, visible=True, replac
 
             data = self.data_collection[data_label]
 
-            viewer.add_data(data, percentile=95, color=viewer.color_cycler())
+            # set the original color based on metadata preferences, if provided, and otherwise
+            # based on the colorcycler
+            # NOTE: this is intentionally not a single line to avoid incrementing the color-cycler
+            # unless it is used
+            color = data.meta.get('_default_color')
+            if color is None:
+                color = viewer.color_cycler()
+            viewer.add_data(data, percentile=95, color=color)
 
             # Specviz removes the data from collection in viewer.py if flux unit incompatible.
             if data_label not in self.data_collection:
@@ -2315,13 +2343,6 @@ def _on_data_deleted(self, msg):
             if data_item['name'] == msg.data.label:
                 self.state.data_items.remove(data_item)
 
-        # TODO: Fix bug with DataCollectionDeleteMessage not working with
-        #  a handler in cubeviz/plugins/viewers.py. This code is a temporary
-        #  workaround for that.
-        if self.config == 'cubeviz':
-            viewer = self.get_viewer(self._jdaviz_helper._default_spectrum_viewer_reference_name)
-            viewer._check_if_data_removed(msg=msg)
-
         self._clear_object_cache(msg.data.label)
 
     def _create_data_item(self, data):
@@ -2479,7 +2500,7 @@ def _create_viewer_item(self, viewer, vid=None, name=None, reference=None,
             'layer_options': "IPY_MODEL_" + viewer.layer_options.model_id,
             'viewer_options': "IPY_MODEL_" + viewer.viewer_options.model_id,
             'selected_data_items': {},  # noqa data_id: visibility state (visible, hidden, mixed), READ-ONLY
-            'visible_layers': {},  # label: {color, label_suffix}, READ-ONLY
+            'visible_layers': {},  # label: {color}, READ-ONLY
             'wcs_only_layers': wcs_only_layers,
             'reference_data_label': reference_data_label,
             'canvas_angle': 0,  # canvas rotation clockwise rotation angle in deg
@@ -2686,7 +2707,7 @@ def compose_viewer_area(viewer_area_items):
         for name in config.get('tray', []):
             tray = tray_registry.members.get(name)
 
-            tray_item_instance = tray.get('cls')(app=self)
+            tray_item_instance = tray.get('cls')(app=self, tray_instance=True)
 
             # store a copy of the tray name in the instance so it can be accessed by the
             # plugin itself

jdaviz/components/viewer_data_select.vue

@@ -208,12 +208,8 @@ module.exports = {
       } else if (this.$props.viewer.config === 'cubeviz') {
         if (this.$props.viewer.reference === 'spectrum-viewer') {
           if (item.meta.Plugin === undefined) {
-            // then the data can be a cube (auto-collapsed) as long as its the flux data
-            // if this logic moves to python, we could check directly against reference data instead
-            return (item.name.indexOf('[FLUX]') !== -1 || item.name.indexOf('[SCI]') !== -1) && this.dataItemInViewer(item, returnExtraItems)
-          } else if (item.meta.Plugin === 'GaussianSmooth') {
-            // spectrally smoothed would still be a collapsible cube
-            return item.ndims === 3 && this.dataItemInViewer(item, returnExtraItems)
+            // then only allow 1d spectra (not cubes or images)
+            return item.ndims === 1
           } else {
             // filter plugin results to only those that are spectra
             return item.ndims === 1 && this.dataItemInViewer(item, returnExtraItems)

jdaviz/components/viewer_data_select_item.vue

@@ -178,8 +178,6 @@ module.exports = {
           return ['IVAR', 'ERR'].indexOf(extension) !== -1
         } else if (this.$props.viewer.reference === 'mask-viewer') {
           return ['MASK', 'DQ'].indexOf(extension) !== -1
-        } else if (this.$props.viewer.reference === 'spectrum-viewer') {
-          return ['SCI', 'FLUX'].indexOf(extension) !== -1
         }
       } else if (this.$props.viewer.config === 'specviz2d') {
         if (this.$props.viewer.reference === 'spectrum-2d-viewer') {

jdaviz/configs/cubeviz/cubeviz.yaml

@@ -25,13 +25,13 @@ tray:
   - g-markers
   - cubeviz-slice
   - g-unit-conversion
+  - cubeviz-spectral-extraction
   - g-gaussian-smooth
   - g-collapse
   - g-model-fitting
   - g-line-list
   - specviz-line-analysis
   - cubeviz-moment-maps
-  - cubeviz-spectral-extraction
   - imviz-aper-phot-simple
   - export
 viewer_area:

jdaviz/configs/cubeviz/helper.py

@@ -5,6 +5,7 @@
 from specutils import Spectrum1D
 from specutils.io.registers import _astropy_has_priorities
 
+from jdaviz.core.events import SnackbarMessage
 from jdaviz.core.helpers import ImageConfigHelper
 from jdaviz.configs.default.plugins.line_lists.line_list_mixin import LineListMixin
 from jdaviz.configs.specviz import Specviz
@@ -79,6 +80,27 @@ def load_data(self, data, data_label=None, override_cube_limit=False, **kwargs):
 
         super().load_data(data, parser_reference="cubeviz-data-parser", **kwargs)
 
+        if 'Spectral Extraction' not in self.plugins:  # pragma: no cover
+            msg = SnackbarMessage(
+                "Automatic spectral extraction requires the Spectral Extraction plugin to be enabled",  # noqa
+                color='error', sender=self, timeout=10000)
+            self.app.hub.broadcast(msg)
+        else:
+            try:
+                self.plugins['Spectral Extraction']._obj._extract_in_new_instance(auto_update=False, add_data=True)  # noqa
+            except Exception:
+                msg = SnackbarMessage(
+                    "Automatic spectrum extraction for the entire cube failed."
+                    " See the spectral extraction plugin to perform a custom extraction",
+                    color='error', sender=self, timeout=10000)
+            else:
+                msg = SnackbarMessage(
+                    "The extracted 1D spectrum was generated automatically for the entire cube."
+                    " See the spectral extraction plugin for details or to"
+                    " perform a custom extraction.",
+                    color='warning', sender=self, timeout=10000)
+            self.app.hub.broadcast(msg)
+
     @deprecated(since="3.9", alternative="select_wavelength")
     def select_slice(self, slice):
         """
@@ -120,26 +142,21 @@ def specviz(self):
             self._specviz = Specviz(app=self.app)
         return self._specviz
 
-    def get_data(self, data_label=None, spatial_subset=None, spectral_subset=None, function=None,
+    def get_data(self, data_label=None, spatial_subset=None, spectral_subset=None,
                  cls=None, use_display_units=False):
         """
         Returns data with name equal to ``data_label`` of type ``cls`` with subsets applied from
-        ``spatial_subset`` and/or ``spectral_subset`` using ``function`` if applicable.
+        ``spectral_subset``, if applicable.
 
         Parameters
         ----------
         data_label : str, optional
             Provide a label to retrieve a specific data set from data_collection.
         spatial_subset : str, optional
-            Spatial subset applied to data.
+            Spatial subset applied to data.  Only applicable if ``data_label`` points to a cube or
+            image.  To extract a spectrum from a cube, use the spectral extraction plugin instead.
         spectral_subset : str, optional
             Spectral subset applied to data.
-        function : {True, False, 'minimum', 'maximum', 'mean', 'median', 'sum'}, optional
-            Ignored if ``data_label`` does not point to cube-like data.
-            If True, will collapse according to the current collapse function defined in the
-            spectrum viewer.  If provided as a string, the cube will be collapsed with the provided
-            function.  If False, None, or not passed, the entire cube will be returned (unless there
-            are values for ``spatial_subset`` and ``spectral_subset``).
         cls : `~specutils.Spectrum1D`, `~astropy.nddata.CCDData`, optional
             The type that data will be returned as.
 
@@ -149,20 +166,8 @@ def get_data(self, data_label=None, spatial_subset=None, spectral_subset=None, f
             Data is returned as type cls with subsets applied.
 
         """
-        # If function is a value ('sum' or 'minimum') or True and spatial and spectral
-        # are set, then we collapse the cube along the spatial subset using the function, then
-        # we apply the mask from the spectral subset.
-        # If function is any value other than False, we use specviz
-        if (function is not False and spectral_subset and spatial_subset) or function:
-            return self.specviz.get_data(data_label=data_label, spectral_subset=spectral_subset,
-                                         cls=cls, spatial_subset=spatial_subset, function=function)
-        elif function is False and spectral_subset:
-            raise ValueError("function cannot be False if spectral_subset"
-                             " is set")
-        elif function is False:
-            function = None
         return self._get_data(data_label=data_label, spatial_subset=spatial_subset,
-                              spectral_subset=spectral_subset, function=function,
+                              spectral_subset=spectral_subset,
                               cls=cls, use_display_units=use_display_units)
 
     # Need this method for Imviz Aperture Photometry plugin.