latest API proposal

[kushalkolar]

I think we finally have enough from mesmerize-core and fastplotlib, as well as user experience, to make an elegant API for mesmerize-viz (which I can probably write in a weekend).

Basically, we make DataFrame and Series extensions for visualizations, to get rid of the common plotting code that is all over the current mesmerize-core demo nbs.

For example for mcorr:

df.iloc[i].mcorr.viz(["raw", "mcorr"], image_widget_kwargs={...})
# plots a ImageWidget of raw and mcorr

Similar for cnmf:

row.cnmf.viz([["mcorr", "rcm", "temporal_stack"], ["temporal"]],... kwargs)

plots this (and the RCM too): https://user-images.githubusercontent.com/9403332/210027199-6e4ac193-6096-4d18-80d5-a41591ea4d4f.gif

Contours are automatically put on the rcm plot etc. and all contours and temporal are linked by default. Use "good" contours by default.
There are checkboxes allowing to choose if you want to view good, bad, or no contours. If they want to change colors or other stuff they have to access the graphics themselves.

We can also allow these to be called at the DataFrame level.
This makes a grid of all items in the dataframe:

df.mcorr.viz("mcorr")

# or a subset of items
df.iloc[5:10].mcorr.viz("mcorr")

# if a list is provied we could do image widget gridplot where each gridplot row is from one dataframe row?
df.iloc[5:10].mcorr.viz(["mcorr", "mean"])
# providing mcorr and mean would also require changes to ImageWidget to allow non-matching dims

This plots only one item but provides a dataframe widget to go through multiple batch items:

df.mcorr.viz(["raw", "mcorr"]) 

Potential dataframe widgets:
widgetti/ipyvuetify#71
https://github.com/mwouts/itables

❗ Need to determine if these also work with vaex and/or polars if we migrate away from pandas.

Signatures:

row.mcorr.viz

• data
  • list of str, "raw" | "mcorr"
• input_movie_kwargs: dict
  • passed to caiman.get_input_movie()
• image_widget_kwargs: dict
  • passed to fastplotlib.ImageWidget

row.cnmf.viz

• data
  • list of str, or list of lists of str
    • each str must belong to: "contours", "mcorr", "rcm", "rcb", "residuals", "temporal", "temporal-stack", "temporal-hm" (temporal heatmap)
    • If it's a list of list, multiple gridplots are made which are then VBox stacked.
    • perhaps also allow showing things like "rcm" or "temporal" multiple times with different component_ixs args (see below), could be done like "rcm.0", "rcm.1".
• component_ixs: str, np.ndarray, or dict
  • if str or np.ndarray, used for get_temporal(), get_contours(), get_rcm(), etc.
  • if dict, must be like {"rcm": "all", "contours": "good"}. If there are multiple of the same data, can do {"rcm.0": "good", "rcm.1": "bad"}
• input_movie_kwargs: dict
  • passed to caiman.get_input_movie()
• image_widget_kwargs: dict
  • passed to fastplotlib.ImageWidget

[clewis7]

I think this is great...would need to make sure that row.mcorr.viz.close() or row.cnmf.viz.close() is implemented for plot closing, especially when people having many plots open at a time could be problematic for their devices

[kushalkolar]

In the implementation I actually made it so that mcorr.viz() returns a fastplotlib.ImageWidget, so they can close the ImageWidget. I think it's better to return an ImageWidget instance which you can call show() on because then you can modify the ImageWidget, like window functions, cmaps, etc.

[kushalkolar]

For showing the dataframe we can instead use something like this instead of just list widgets: widgetti/ipyvuetify#71

[RuiOSilva]

Hi all,

I slightly tested this and the latest API seems already quite good. Here are my notes as you asked Kushal:

1 - I tested it and for an initial visualization it worked well. However, I believe this could be improved also by adding some numeric labelling. In examples where more than 100 neurons are present, the bookeh is still more user friendly for verifying that all 'good' neurons are well deconvoluted. I think it could be very useful to also add a numeric slider for the neurons visualization.

2 - Also, I had the idea that could be interesting adding a feature of neuron removal/changing from good to bad indexes by visualization and deletion.

3 - I think it could also be useful to add a feature to open images other than created by mesmerize core also. I did not understood very well how to do that. In my current code I have been using mesmerize core lazy imports for things other than calcium analysis and to couple it with fastplotlib. Maybe it could be useful to separate the lazy imports in another small package by itself to use it as a dependency of fastplotlib?

I have several other small points but that would be dependent on changes that I did to CaImAn itself so if I find them useful for overall users I'll post later.

[kushalkolar]

Hi all,

I slightly tested this and the latest API seems already quite good. Here are my notes as you asked Kushal:

1 - I tested it and for an initial visualization it worked well. However, I believe this could be improved also by adding some numeric labelling. In examples where more than 100 neurons are present, the bookeh is still more user friendly for verifying that all 'good' neurons are well deconvoluted. I think it could be very useful to also add a numeric slider for the neurons visualization.

You could already do this yourself by creating a slider to go through components and use link() to set the present feature of contours or temporal graphics. But I do plan to add this visualization: nel-lab/mesmerize-core#181

2 - Also, I had the idea that could be interesting adding a feature of neuron removal/changing from good to bad indexes by visualization and deletion.

Yes, haven't gotten around to doing this yet

3 - I think it could also be useful to add a feature to open images other than created by mesmerize core also. I did not understood very well how to do that. In my current code I have been using mesmerize core lazy imports for things other than calcium analysis and to couple it with fastplotlib. Maybe it could be useful to separate the lazy imports in another small package by itself to use it as a dependency of fastplotlib?

Yes we've thought of this, if it happens it won't be for a few months but I think we're leaning in that direction now. I think there's a general use for a simple array interface for lazy loading or computing which is simpler than Dask (which is overkill for this purpose).

[kushalkolar]

@RuiOSilva

feature of neuron removal/changing from good to bad indexes by visualization and deletion.

Do you have any ideas for the kind of API you would like for this purpose?

For example this would produce the follwing viz:

row.cnmf.viz(
    [
        ["mcorr", "temporal_stack"], 
        ["temporal"]
    ],
    ... kwargs
)

[RuiOSilva]

@kushalkolar I do not have a clear idea in mind yet, but in my mind, when I am using this visualization, I sometimes see bad components detected as good and most frequently good components detected as bad, even after tuning the quality parameters selection (because it depends on the recordings also).

I would try to start adding some kind of buttons that would go bold also when clicking on neurons wither good or bad. When you clicked that button it would induce a deletion of that number from one of the arrays into the other.

Coding wise, I would have to see if this is the easiest way, but it would be very straight forward. I can foresee problems in people that mistakenly click on buttons, so I would add a line of code to resave after visualization if the user wants. This way, it would only change the saved files if people actually trusted what they did.

In summary:

• widget buttons events synched with clicked buttons
• clicked widget buttons events synched with cnmf object function to change indexes

[kushalkolar]

The basics of this was done in #23 , leaving open until it's documented

[kushalkolar]

The API as merged in #28 is more or less final.