CaImAn Notebook Interface (beta)

Introduction

The CaImAn Web Interface is a web-based interface to the Calcium Imaging Analysis (CaImAn) suite of analysis algorithms. The goal is to allow users with limited or no programming experience to be able to automate their Calcium imaging analysis with the CaImAn package as well as let all users visualize and interactively inspect the results of CNMF.

CaImAn is a toolbox consisting of several different algorithms and workflows and also includes some behavior analysis. The web interface is designed to cover the most common analysis workflows, but it does not expose the entire functionality of the underlying CaImAn package. The interface is in beta and may have bugs, please file GitHub issues accordingly. The following is a list of features included in the web interface:

• Rigid and non-rigid motion correction of TIF and AVI video files
• Constrained Non-Negative Matrix Factorization (CNMF) algorithms for automated ROI detection and
• spatial and temporal extraction
• Calcium signal deconvolution
• Support for both 1-photon and 2-photon microscopy videos
• The following features are not currently supported in the web interface:
• OnACID online CNMF algorithm
• Volumetric (3D) data processing
• Behavior Analysis The interface is under active development with bug fixes and new features added periodically.

Installation

While the interface is aimed at users with limited programming experience, the installation currently requires entering a few text commands into a command line prompt (e.g. Terminal on Mac). We are actively exploring ways of making the interface a standalone, downloadable Desktop application.

1. Navigate your browser to https://github.com/flatironinstitute/CaImAn
2. Scroll down to find the installation instructions for your operating system using Git. Install using the developer option, i.e. using ”pip install e .”. You must have the conda package manager installed already
3. Change your directory to the git directory containing CaIman, then find the "nb_interface" folder.
4. Launch a jupyter notebook (i.e. enter jupyter notebook in Terminal)
5. You will find a single jupyter notebook file called caiman interface.ipynb. Open this notebook and run all the cells, the interface will appear within the notebook. You can directly launch this notebook by using the command jupyter notebook caiman_interface.ipynb if you're in the nb_interface folder.
6. Additional instructions are present within the notebook itself.

Analysis

In this section we will cover the basic workflow for motion correcting and analyzing a raw microscopy video of GCaMP fluorescing cells. The CaImAn package includes a demo TIF file for the analysis called demoMovie.tif Steps:

1. Main. In the ”Main” tab, set your working directory to whichever folder on your computer contains the files you wish to analyze.
2. Also in the Main tab, you should set some basic movie parameters including the movie frame rate, Ca2+ transient decay time and the microscopy type (i.e. whether it’s a 1-photon or 2-photon microscope).
3. Motion Correction. Proceed to the Motion Correction tab. By default, the path to the demo movie (a 2-photon movie) is loaded into the text box. To load your own file, you can either type (or copy/paste) the path to a folder (if you wish to analyze a batch of files) or a single file, OR you can press the Browse button which will pop up a file dialog where you can easily find and select the folders or files you want.
4. Press the Load Files button to load the files.
5. There are two groups of settings for motion correction labeled Basic and Advanced. The basic settings should definitely be verified before running motion correction, however, the advanced settings may be left at their default values in many cases. The meaning of each of the setting parameters is outlined in the following section.
6. Select Group or Independent mode. If you’re attempting to motion correction multiple files, you can either motion correct them separately (Independent mode) or if they are all part of the same recording session, you can run them together producing a single, large motion corrected file at the end (Group mode).
7. Select Rigid or Non-Rigid. Rigid motion correction is fast but does not correct for elastic (non-rigid) deformations of the video. Since the brain is a compressible organ, it may have stretching or compressing movements during the recording that need to be corrected. If only translational movement occurs then rigid mode will work fine.
8. Click ”Run Motion Correction” to begin motion correction. Various logs will appear in the black outlined box below. The Status bar at the top will indicate while the algorithm is running and when it has completed. This process will produce a motion corrected video file with the ”.mmap” file extension.
9. Once motion correction has finished, proceed to the MC Results tab. This tab includes a button that will popup a window playing the original movie (left) side-by-side with the motion corrected movie (right). It also shows a plot of the shifts made to correct the motion. In rigid mode, the shifts apply to the whole movie, whereas in non-rigid mode the movie is broken down into a number of overlapping patches and shifts are applied to each patch independently. The shifts plot for non-rigid mode will plot the shifts mean and variance for each patch for the x (horizontal) and y (vertical) shifts made.
10. CNMF. If the motion correction results are satisfactory, proceed to the CNMF tab. If not, go back to the Motion Correction tab and re-run with different settings. In the CNMF tab, find the ”.mmap” file created by the motion correction step and click the ”Load Files” button. If you came immediately from motion correction step, the file path will already be populated and you just need to click Load Files.
11. After the file has been loaded, click the ”Inspect Correlation Plot” button. A window will pop up showing the correlation plot of the movie. You can adjust the VMIN/VMAX settings below to find the right settings that appear to capture all the ROIs and minimize background. This window is only for guiding your setting selection and does not actually set any parameters itself.
12. Select Patches or Single FOV (field of view) mode. Patches mode is more efficient since it will break the movie into overlapping patches and analyze them in parallel, recombining the results at the end. Sometimes, however, this can result in some artifacts. Alternatively run Single field of view mode, which will analyze the entire movie but is less efficient. Set the Min Corr and Min SNR settings according to your results in the Inspect Correlation Plot step. See the ”Settings and Parameters” section of this document for more information about the rest of the settings in this step.
13. Click the Run CNMF button. The status bar will change to ”Running CNMF...” and will indicate when it has finished. Proceed to the CNMF Results tab.
14. CNMF Results. Once CNMF has finished and you’re in the CNMF results tab, click the View/Refine CNMF Results button at the top. After a few short moments, 4 plots below should appear with the analyze data. The plot in the top left shows a correlation plot (i.e. similar to a Z-projection of the movie) with red dots overlayed indicating the spatial location of detected ROIs. You can click the red dots or slide the ROI# slider to view that particular ROIs Ca2+ trace and the isolated spatial extent in the plots below.
15. If the algorithm detected too few or too many ROIs, you can refine the results by clicking the ”EDIT” button, modify the parameters, and click Update. The detected ROIs (i.e. red dots) will update. If you cannot get the results you want, you may need to re-run the CNMF algorithm using different settings.
16. To download the Ca2+ signal traces and deconvolved signal traces for the detected ROIs, click the Down- load button and the files will be downloaded to wherever your working directory has been set. Select the ”dF/F” checkbox before Downloading if you want to normalize the signals using the change in fluorescence over the baseline.

Settings and Parameters

Motion Correction

Basic Settings

• Grouped? If your folder contains movies from the same experiment, i.e. if your recording software produces multiple segments of a single recording, then select ”Group” so these will be combined in the analysis to a single movie. If your folder contains movies from different recordings, select ”Independent” so they will be run separately.
• Downsample factors are percentages from 0 to 1. If you enter 0.5 for height and width, then the movie resolution will be halved. E.g. if you have a 500px by 500px movie, it will be downsampled to 250px by 250px. You can also downsample in the time domain, i.e. if your movie has 1000 frames, entering 0.5 will downsample the frames to 500 frames. This should only be done if you’re recording at a high frame rate.
• ”High Pass” setting should be left at default value.
• Motion Correction has two modes: rigid and non-rigid. Rigid is much faster. Non-rigid should be used when the movie has non-rigid (elastic) motions/deformations. This can sometimes happen given with in vivo brain recordings since the brain is an elastic organ.

Advanced Settings

• Num iterations (rigid): The number of iterations to run the rigid motion correction algorithm. More iterations may result in better performance but at the cost of a longer processing time.
• Num splits (parallelization): The algorithm can split the data into multiple parts and process them independently, combining at the end. The more ”splits” the fasting the algorithm can run in principle.
• Strides: For non-rigid motion correction, the movie is divided into overlapping patches. Strides will control the number of patches.
• Overlaps: How many pixels each patch overlaps (non-rigid only).
• Upsample Factor: Upsample factor of shifts per patches to avoid smearing when merging patches (non- rigid only)
• Max Deviation (rigid): This controls the maximum deviation allowed for patch with respect to rigid shift.

Motion Correction Results

• Here you can view a side-by-side of the original movie with the motion corrected movie and the applied shifts needed to correct the motion. The movies will pop-up in a separate window. Press the Q key on your keyboard to close the movie window. You should make sure the algorithm didn’t apply any dramatic shifts.

CNMF

Basic Settings

• Patches? CNMF-E can run in two modes: patches or single field of view (FOV). Patches mode will break up each frame into multiple pieces and process them independently. On computers with multiple processors, this can be much faster. However, patches mode can produce unwanted reconstruction artifacts when the pieces are poorly re-assembled.
• You can downsample in the CNMF-E stage, spatially (height and width) and temporally (frames/time). However, if you already downsampled during motion correction, you probably should not downsample again.
• K is the estimated number of neurons in your field of view. This parameter helps the algorithm detect the correct number of neurons. It does not need to be highly accurate; an over-estimate is better.
• gSig This is the estimated half-width (radius) of each neuron in the movie, in pixel values. The default value will work in most cases.
• gSiz This is related to gSig and typically the default value will work fine.
• min corr: minimal correlation peak for 1-photon imaging initialization
• min pnr: minimal peak to noise ratio for 1-photon imaging initialization

Advanced Settings

• P (AR order): order of the autoregressive process used to estimate deconvolution
• Num global background components: The background is modeled as a simple, low-complexity image with
• a few components, usually set to 2 for the background + blood vessel.
• Max Correlation (Merge Thresh): merging threshold, max correlation allowed
• Analyze dendrites?: If checked, will initialize components to be able to capture fine dendritic processes, otherwise will only capture cell body and major branches.
• Min SNR: adaptive way to set threshold (will be equal to min SNR)
• Spatial Correlation Threshold (rval thr): all r values above this are accepted (spatial consistency metric)
• CNN threshold: all samples with probabilities larger than this are accepted

CNMF Results

• This interface allows you to look at a ”Z-projection” (i.e. flattened video) of the data to see how well the CNMF-E algorithm did at detecting neurons and avoiding noise. In the top left is a the Z-projected image with red dots denoting the neurons it detected.
• In general, CNMF-E will think some noise is a neuron. In that case, you need to look through the traces and at the image panels to see which cells are noise and press the ‘Delete ROI‘ button. This will not modify the ‘context‘. This just excludes those ROIs if you download the data as a file.
• Check the ‘Use dF/F‘ checkbox if you want to convert the signals to delta fluorescence / fluorescence. This is a necessary step if you want to be able to compare the signals between animals and recording sessions.

Troubleshooting

• If you have any issues, first try to restart the application by restarting the Jupyter Kernel and re-running all of the cells.
• Make sure all files are loaded properly and have the correct file extension.
• Any other issues should be filed at the GitHub for CaIman ⟨ https://github.com/flatironinstitute/CaImAn ⟩