[FEAT] Support panel dashboarding library

[kszlim]

Right now the resampler kind of works with Panel in the sense that you can embed a FigureResampler based figure into a Panel dashboard, but the dynamically resampling doesn't work.

So upon load with verbose on, I can see the resampling occur, but as you zoom in or out, there's no dynamic resampling.

Alternatively, it'd be nice if you just gave a simple solution for hooking it into a panel based dashboard.

[jonasvdd]

Hi @kszlim,

Plotly-resampler leverages the Dash ecosystem to create dynamic aggregation callbacks. So integrating this natively into panel is not that straightforward.

A quick online search pointed me toward "Embed your Dash app", which suggests that this could be embedded with an iframe.

Please let me know if this helps you any further.
Kind regards,
Jonas

[kszlim]

Ah yeah, I reached out to the holoviz discord and was told it would involve

Basically Panel could display the "normal" plotly figure. Listen for a relayout event. Send the relayout_data to the server. Run the `construct_update_data` https://github.com/predict-idlab/plotly-resampler/blob/2b79498eaa79cba891310dfcec3860936af3207b/plotly_resampler/figure_resampler/figure_resampler_interface.py#L1347 and send updated traces to the frontend.

So there probably would have to be an implementation just for panel.

[skemp117]

I would also like this feature. For now, using pn.pane.Plotly(fig) is still useful to get a fully interactable down sampled plotly object. The problem is just that the dynamic resampling doesn't work. I played around with trying to use iframes or some other method to embed as a dash app, but never ended up with a good solution. I'm also pretty new to this space, so it could've easily just been due to me being new.

[kszlim]

I would also like this feature. For now, using pn.pane.Plotly(fig) is still useful to get a fully interactable down sampled plotly object. The problem is just that the dynamic resampling doesn't work. I played around with trying to use iframes or some other method to embed as a dash app, but never ended up with a good solution. I'm also pretty new to this space, so it could've easily just been due to me being new.

Yeah, I don't think there is a good solution without direct support. I tried using it as an ipywidget, but that is somewhat broken too.

[skemp117]

Ok I've got something working on v0.9.1. Here is a running example. If you want to turn this into a more solid contribution @kszlim or @jonasvdd just lmk. I tested briefly with v0.9.2 but the construct_update_data had changed and I didn't want to work on it anymore today lol. The major issue I had was with the way the pn.pane.Plotly object redraws the figure, it always calls an autosize event, so if you didn't have your y-axis range set firmly, it would be reset.

from __future__ import annotations

from typing import List, Tuple

import plotly.graph_objects as go
from plotly.basedatatypes import BaseFigure

from plotly_resampler.aggregation import (
    AbstractAggregator,
    AbstractGapHandler,
    MedDiffGapHandler,
    MinMaxLTTB,
)
from plotly_resampler.figure_resampler.figure_resampler_interface import AbstractFigureAggregator
from plotly_resampler.figure_resampler.utils import is_figure, is_fr

import panel as pn

import re

class FigureResamplerPanel(AbstractFigureAggregator, go.Figure):
    """Data aggregation functionality for ``go.Figures``."""

    def __init__(
        self,
        figure: BaseFigure | dict = None,
        convert_existing_traces: bool = True,
        default_n_shown_samples: int = 1000,
        default_downsampler: AbstractAggregator = MinMaxLTTB(),
        default_gap_handler: AbstractGapHandler = MedDiffGapHandler(),
        resampled_trace_prefix_suffix: Tuple[str, str] = (
            '<b style="color:sandybrown">[R]</b> ',
            "",
        ),
        show_mean_aggregation_size: bool = True,
        convert_traces_kwargs: dict | None = None,
        verbose: bool = False,
    ):
        """Initialize a dynamic aggregation data mirror using a dash web app.

        Parameters
        ----------
        figure: BaseFigure
            The figure that will be decorated. Can be either an empty figure
            (e.g., ``go.Figure()``, ``make_subplots()``, ``go.FigureWidget``) or an
            existing figure.
        convert_existing_traces: bool
            A bool indicating whether the high-frequency traces of the passed ``figure``
            should be resampled, by default True. Hence, when set to False, the
            high-frequency traces of the passed ``figure`` will not be resampled.
        default_n_shown_samples: int, optional
            The default number of samples that will be shown for each trace,
            by default 1000.\n
            !!! note
                - This can be overridden within the [`add_trace`][figure_resampler.figure_resampler_interface.AbstractFigureAggregator.add_trace] method.
                - If a trace withholds fewer datapoints than this parameter,
                  the data will *not* be aggregated.
        default_downsampler: AbstractAggregator, optional
            An instance which implements the AbstractAggregator interface and
            will be used as default downsampler, by default ``MinMaxLTTB`` with
            ``MinMaxLTTB`` is a heuristic to the LTTB algorithm that uses pre-selection
            of min-max values (default 4 per bin) to speed up LTTB (as now only 4 values
            per bin are considered by LTTB). This min-max ratio of 4 can be changed by
            initializing ``MinMaxLTTB`` with a different value for the ``minmax_ratio``
            parameter. \n
            !!! note
                This can be overridden within the [`add_trace`][figure_resampler.figure_resampler_interface.AbstractFigureAggregator.add_trace] method.
        default_gap_handler: AbstractGapHandler, optional
            An instance which implements the AbstractGapHandler interface and
            will be used as default gap handler, by default ``MedDiffGapHandler``.
            ``MedDiffGapHandler`` will determine gaps by first calculating the median
            aggregated x difference and then thresholding the aggregated x delta on a
            multiple of this median difference.  \n
            !!! note
                This can be overridden within the [`add_trace`][figure_resampler.figure_resampler_interface.AbstractFigureAggregator.add_trace] method.
        resampled_trace_prefix_suffix: str, optional
            A tuple which contains the ``prefix`` and ``suffix``, respectively, which
            will be added to the trace its legend-name when a resampled version of the
            trace is shown. By default a bold, orange ``[R]`` is shown as prefix
            (no suffix is shown).
        show_mean_aggregation_size: bool, optional
            Whether the mean aggregation bin size will be added as a suffix to the trace
            its legend-name, by default True.
        convert_traces_kwargs: dict, optional
            A dict of kwargs that will be passed to the [`add_trace`][figure_resampler.figure_resampler_interface.AbstractFigureAggregator.add_trace] method and
            will be used to convert the existing traces. \n
            !!! note
                This argument is only used when the passed ``figure`` contains data and
                ``convert_existing_traces`` is set to True.
        verbose: bool, optional
            Whether some verbose messages will be printed or not, by default False.
        """
        # Parse the figure input before calling `super`
        if is_figure(figure) and not is_fr(figure):
            # A go.Figure
            # => base case: the figure does not need to be adjusted
            f = figure
        else:
            # Create a new figure object and make sure that the trace uid will not get
            # adjusted when they are added.
            f = self._get_figure_class(go.Figure)()
            f._data_validator.set_uid = False

            if isinstance(figure, BaseFigure):
                # A base figure object, can be;
                # - a go.FigureWidget
                # - a plotly-resampler figure: subclass of AbstractFigureAggregator
                # => we first copy the layout, grid_str and grid ref
                f.layout = figure.layout
                f._grid_str = figure._grid_str
                f._grid_ref = figure._grid_ref
                f.add_traces(figure.data)
            elif isinstance(figure, dict) and (
                "data" in figure or "layout" in figure  # or "frames" in figure  # TODO
            ):
                # A figure as a dict, can be;
                # - a plotly figure as a dict (after calling `fig.to_dict()`)
                # - a pickled (plotly-resampler) figure (after loading a pickled figure)
                # => we first copy the layout, grid_str and grid ref
                f.layout = figure.get("layout")
                f._grid_str = figure.get("_grid_str")
                f._grid_ref = figure.get("_grid_ref")
                f.add_traces(figure.get("data"))
                # `pr_props` is not None when loading a pickled plotly-resampler figure
                f._pr_props = figure.get("pr_props")
                # `f._pr_props`` is an attribute to store properties of a
                # plotly-resampler figure. This attribute is only used to pass
                # information to the super() constructor. Once the super constructor is
                # called, the attribute is removed.

                # f.add_frames(figure.get("frames")) TODO
            elif isinstance(figure, (dict, list)):
                # A single trace dict or a list of traces
                f.add_traces(figure)

        super().__init__(
            f,
            convert_existing_traces,
            default_n_shown_samples,
            default_downsampler,
            default_gap_handler,
            resampled_trace_prefix_suffix,
            show_mean_aggregation_size,
            convert_traces_kwargs,
            verbose,
        )

        if isinstance(figure, AbstractFigureAggregator):
            # Copy the `_hf_data` if the previous figure was an AbstractFigureAggregator
            # and adjust the default `max_n_samples` and `downsampler`
            self._hf_data.update(
                self._copy_hf_data(figure._hf_data, adjust_default_values=True)
            )

            # Note: This hack ensures that the this figure object initially uses
            # data of the whole view. More concretely; we create a dict
            # serialization figure and adjust the hf-traces to the whole view
            # with the check-update method (by passing no range / filter args)
            with self.batch_update():
                graph_dict: dict = self._get_current_graph()
                update_indices = self._check_update_figure_dict(graph_dict)
                for idx in update_indices:
                    self.data[idx].update(graph_dict["data"][idx])   

    def parse_relayout_panel(self, event, update_data):
        layout_update = {}
        figure_object = None

        if not event:
            return layout_update
        for object in event:
            if type(object) is PanelFigureManager:
                figure_object = object
                continue
        if not figure_object:
            return layout_update
                
        print('Viewport')
        print(figure_object.viewport)

        y_matches = self._re_matches(
            re.compile(r"[y]axis\d*\.range",  re.IGNORECASE), 
            figure_object.viewport.keys()
        )

        for range_change_axis in y_matches:
            axis = range_change_axis.split(".")[0]

            # Handle y-axis rescaling caused by self.param.trigger('object') which causes an autosize relayout
            if f'{axis}.autorange' in update_data and  'xaxis.autorange' in update_data:
                if update_data[f'{axis}.autorange'] and update_data['xaxis.autorange']:
                    layout_update = update_data
            else:
                print('Here 3')
                print( figure_object.viewport[f'{axis}.range'])
                layout_update[f'{axis}.range'] = figure_object.viewport[f'{axis}.range']

        return layout_update

class PanelFigureManager(pn.pane.Plotly):
    """Using data aggregation in Holoviz Panel."""
    def __init__(self, *args,**kwargs):
        """Initialize a dynamic aggregation data mirror using a dash web app.
        Parameters
        ----------
        figure: FigureResamplerPanel
            The figure to be used in a similar manner as panel.pane.Plotly
        """
        super().__init__(*args,**kwargs)
        self.param.watch(self.on_relayout, 'relayout_data')
        self.self_call = True


    def on_relayout(self, event):
        if self.self_call:
            print('**********************************Self call')
            self.self_call = False
            return
        
        self.autosize_last = False
        # Process the event to construct new data
        # Assuming self.object is a FigureResamplerPanel instance
        update_data = self.object.construct_update_data(event.new)
        
        if not self.object._is_no_update(update_data):  # when there is an update
            with self.object.batch_update():
                # Update layout  
                self.object.layout.update(self.object.parse_relayout_panel(event, update_data[0]), overwrite=True)          
                self.object.layout.update(self.object._parse_relayout(update_data[0]), overwrite=True)

                # Update data
                for updated_trace in update_data[1:]:
                    trace_idx = updated_trace.pop("index")
                    self.object.data[trace_idx].update(updated_trace)
           
            self.self_call = True
            self.param.trigger('object')

            # Fix the annying self call issue where panel updates the figure calling an autosize event
            y_matches = self.object._re_matches(
            re.compile(r"[y]axis\d*",  re.IGNORECASE), 
                self.object.layout
            )
            for axis in y_matches:
                self.object.layout.update({f'{axis}.autorange': False}, overwrite=True)
                

import panel as pn
import plotly.graph_objs as go
import numpy as np
from plotly.subplots import make_subplots

x = np.arange(1_000_000)
y = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000
y2 = - ((3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000)

# Create a subplot with two y-axes
fig = make_subplots(specs=[[{"secondary_y": True}]])

# Create a basic Plotly figure
fig = FigureResamplerPanel(fig)
fig.add_trace(go.Scatter(mode='lines'), hf_x=x, hf_y=y)
fig.add_trace(go.Scatter(mode='lines'), hf_x=x, hf_y=y2, secondary_y = True)

# fig = FigureResamplerPanel(go.Figure())
# fig.add_trace(go.Scatter(mode='lines'), hf_x=x, hf_y=y)
# fig.add_trace(go.Scatter(mode='lines'), hf_x=x, hf_y=y2)

# Create and show a Panel layout
fig_panel = PanelFigureManager(fig)
layout = pn.Column(fig_panel)
layout.servable()

[kszlim]

@skemp117 I'd definitely like to see this make it in, but I'm not a maintainer of this package 😄

[jonasvdd]

@kszlim @skemp117, everybody, who is willing to put up the work, can be a maintainer of this package :)