align-and-compare.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utilizing Hierarchical Structure for Audio-Based Music Similarity\n",
    "#### Christos Plachouras, ISMIR 2021 Late-Break Demo, [Code](https://github.com/chrispla/align-and-compare), [Paper](https://archives.ismir.net/ismir2021/latebreaking/000057.pdf), [Poster](https://s3.eu-west-1.amazonaws.com/production-main-contentbucket52d4b12c-1x4mwd6yn8qjn/1c8eb7df-d463-4bc5-b33e-ac615490788c.pdf), [Video](https://youtu.be/0yJkVQqi67U)\n",
    "\n",
    "\n",
    "This notebook provides code for demonstrating how the method works for comparing two audio files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### > Library importing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for audio processing\n",
    "import librosa\n",
    "# for plotting\n",
    "import librosa.display\n",
    "import matplotlib.pyplot as plt \n",
    "# for matrix processing\n",
    "import numpy as np\n",
    "import scipy\n",
    "# for spectral clustering\n",
    "import sklearn.cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### > Paths and Parameters\n",
    "Choose paths for the two audio files, configure parameters for structure analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Audio file paths\n",
    "path1 = '/enter/path/here'\n",
    "path2 = '/enter/path/here'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Constant-Q Transform parameters\n",
    "BINS_PER_OCTAVE = 12 * 3\n",
    "N_OCTAVES = 7\n",
    "\n",
    "# Spectral clustering parameters (spectral clustering component translates to number of segment types)\n",
    "kmin = 2  # needs to be >=2\n",
    "kmax = 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### > Hierarchical structure analysis\n",
    "\n",
    "Code modified from [Brian McFee - Laplacian Segmentation (MIT License)](https://github.com/bmcfee/lsd_viz)\n",
    "\n",
    "Brian McFee, Dan Ellis. \"Analyzing Song Structure with Spectral Clustering\". ISMIR 2014.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### >> Compute Laplacian\n",
    "First, we compute the Constant-Q Transform of the audio file and beat-synchronize it. Then, we compute an affinity matrix encoding the self-similarity of each frame of the CQT, and combine it with a sequence matrix encoding the local consistency of MFCCs. Finally, we compute the Laplacian to shift it to its matrix representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_laplacian(path, BINS_PER_OCTAVE, N_OCTAVES):\n",
    "    \"\"\"\n",
    "    Compute the Laplacian matrix from an audio file using\n",
    "    its Constant-Q Transform.\n",
    "\n",
    "    Input\n",
    "    -----\n",
    "    path: filepath (str)\n",
    "\n",
    "    Output\n",
    "    -----\n",
    "    y: audio samples (np.array) - returned for plotting\n",
    "    C: Constant-Q Transform (np.array) - returned for plotting\n",
    "    beat_times: beats in seconds - returned for format_segments()\n",
    "    L: Laplacian matrix (np.array)\n",
    "    \"\"\"\n",
    "\n",
    "    # load audio\n",
    "    y, sr = librosa.load(path, sr=22050, mono=True)\n",
    "\n",
    "    # Compute Constant-Q Transform in dB\n",
    "    C = librosa.amplitude_to_db(np.abs(librosa.cqt(y=y,\n",
    "                                                   sr=sr,\n",
    "                                                   bins_per_octave=BINS_PER_OCTAVE,\n",
    "                                                   n_bins=N_OCTAVES * BINS_PER_OCTAVE)),\n",
    "                                                   ref=np.max)\n",
    "\n",
    "    # beat tracking\n",
    "    tempo, beats = librosa.beat.beat_track(y=y, sr=sr, trim=False)\n",
    "    # get beat times in seconds\n",
    "    beat_times = librosa.frames_to_time(librosa.util.fix_frames(beats, x_min=0, x_max=C.shape[1]), sr=sr)\n",
    "\n",
    "    # beat synchronize the CQT\n",
    "    Csync = librosa.util.sync(C, beats, aggregate=np.median)\n",
    "\n",
    "    # stack 4 consecutive frames\n",
    "    Cstack = librosa.feature.stack_memory(Csync, 4)\n",
    "\n",
    "    # compute weighted recurrence matrix\n",
    "    R = librosa.segment.recurrence_matrix(Cstack, width=3, mode='affinity', sym=True)\n",
    "\n",
    "    # enhance diagonals with a median filter\n",
    "    df = librosa.segment.timelag_filter(scipy.ndimage.median_filter)\n",
    "    Rf = df(R, size=(1, 7))\n",
    "    Rf = librosa.segment.path_enhance(Rf, 15)\n",
    "\n",
    "    # compute MFCCs\n",
    "    mfcc = librosa.feature.mfcc(y=y, sr=sr)\n",
    "    # beat synchronize them\n",
    "    Msync = librosa.util.sync(mfcc, beats)\n",
    "\n",
    "    # build the MFCC sequence matrix\n",
    "    path_distance = np.sum(np.diff(Msync, axis=1)**2, axis=0)\n",
    "    sigma = np.median(path_distance)\n",
    "    path_sim = np.exp(-path_distance / sigma)\n",
    "    R_path = np.diag(path_sim, k=1) + np.diag(path_sim, k=-1)\n",
    "\n",
    "    # get the balanced combination of the MFCC sequence matric and the CQT\n",
    "    deg_path = np.sum(R_path, axis=1)\n",
    "    deg_rec = np.sum(Rf, axis=1)\n",
    "    mu = deg_path.dot(deg_path + deg_rec) / np.sum((deg_path + deg_rec)**2)\n",
    "    A = mu * Rf + (1 - mu) * R_path\n",
    "\n",
    "    # compute the normalized Laplacian\n",
    "    L = scipy.sparse.csgraph.laplacian(A, normed=True)\n",
    "\n",
    "    return y, C, beat_times, L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### >> Segment and cluster\n",
    "\n",
    "We eigendecompose the Laplacian and make sets of its first integer k in [kmin, kmax) eigenvectors. We then perform spectral clustering on each set using the respective number k of clusters. We also compute the Euclidean self-distance of the eigenvectors across time for each set to create \"approximations\" of the Laplacian for visualization purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def segment(L, kmin, kmax):\n",
    "    \"\"\"\n",
    "    Decompose Laplacian and make sets of its first integer k in [kmin, kmax] eigenvectors.\n",
    "    For each set, compute its Euclidean self distance over time, and do spectral clustering.\n",
    "\n",
    "    Input\n",
    "    -----\n",
    "    L: Laplacian matrix (np.array)\n",
    "    kmin: first eigenvector index\n",
    "    kmax: last eigenvector index\n",
    "\n",
    "    Output\n",
    "    -----\n",
    "    distances: self distance matrix of each set of first eigenvectors (np.array, shape=(kmax-kmin, beats, beats))\n",
    "    segments: estimated structural segments for each set of first eigenvectors (np.array, shape=(kmax-kmin, beats))\n",
    "    \"\"\"\n",
    "\n",
    "    # eigendecomposition\n",
    "    evals, evecs = scipy.linalg.eigh(L)\n",
    "\n",
    "    # eigenvector filtering\n",
    "    evecs = scipy.ndimage.median_filter(evecs, size=(9, 1))\n",
    "\n",
    "    # normalization\n",
    "    Cnorm = np.cumsum(evecs**2, axis=1)**0.5\n",
    "\n",
    "    # initialize sets\n",
    "    distances = []\n",
    "    segments = []\n",
    "\n",
    "    for k in range(kmin, kmax):\n",
    "        # create set using first k normalized eigenvectors\n",
    "        Xs = evecs[:, :k] / Cnorm[:, k-1:k]\n",
    "\n",
    "        # get self distance over time of the set\n",
    "        distances.append(scipy.spatial.distance.squareform(scipy.spatial.distance.pdist(Xs, metric='euclidean')))\n",
    "\n",
    "        # cluster each set k using k clusters to get k segment types\n",
    "        KM = sklearn.cluster.KMeans(n_clusters=k, n_init=50, max_iter=500)\n",
    "        segments.append(KM.fit_predict(Xs))\n",
    "\n",
    "    return np.asarray(distances), np.asarray(segments)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### >> Run and plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute Laplacian using the Constant-Q Transform\n",
    "y1, C1, beat_times1, L1 = compute_laplacian(path1, BINS_PER_OCTAVE, N_OCTAVES)\n",
    "y2, C2, beat_times2, L2 = compute_laplacian(path2, BINS_PER_OCTAVE, N_OCTAVES)\n",
    "\n",
    "# Compute Laplacian approximations and hierarchical structural analysis \n",
    "d1, s1 = segment(L1, kmin, kmax)\n",
    "d2, s2 = segment(L2, kmin, kmax)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get relative scale of beat vectors for ploting\n",
    "cqt_scale1 = min(1, C1.shape[1]/C2.shape[1])\n",
    "cqt_scale2 = min(1, C2.shape[1]/C1.shape[1])\n",
    "seg_scale1 = min(1, s1.shape[1]/s2.shape[1])\n",
    "seg_scale2 = min(1, s2.shape[1]/s1.shape[1])\n",
    "\n",
    "# plot Constant-Q Transform\n",
    "for path, C, scale in zip([path1, path2], [C1, C2], [cqt_scale1, cqt_scale2]):\n",
    "    fig1, ax1 = plt.subplots(1, figsize=(15*scale, 5))\n",
    "    librosa.display.specshow(C, y_axis='cqt_hz', sr=22050,\n",
    "                                bins_per_octave=BINS_PER_OCTAVE,\n",
    "                                x_axis='s', ax=ax1)\n",
    "    ax1.set_title(path+\" - Constant-Q Transform\")\n",
    "\n",
    "# plot Laplacian approximations\n",
    "for path, d in zip([path1, path2], [d1, d2]):\n",
    "    fig2, ax2 = plt.subplots(1, kmax-kmin, figsize=(15, 3), sharey='all')\n",
    "    plt.set_cmap('viridis')\n",
    "    for k in range(kmax-kmin):\n",
    "        ax2[k].imshow(d[k], origin='lower', cmap='viridis', interpolation='none')\n",
    "        ax2[k].set_title(\"k = \" + str(kmin+k))\n",
    "    fig2.suptitle(path+\" - Approximations of Laplacian\")\n",
    "    ax2[0].set(xlabel='beats', ylabel='beats')\n",
    "\n",
    "# plot hierarchical structure\n",
    "y_ticks = np.arange(0, kmax-kmin)\n",
    "y_tick_labels = np.arange(kmin, kmax).astype(str)\n",
    "for path, s, scale in zip([path1, path2], [s1, s2], [seg_scale1, seg_scale2]):\n",
    "    fig3, ax3 = plt.subplots(1, 1, figsize=(15*scale, 10))\n",
    "    ax3.imshow(s, interpolation='none', aspect=20)\n",
    "    ax3.set_title(path+' - Hierarchical Structure')\n",
    "    ax3.set(xlabel='beats', ylabel='k')\n",
    "    ax3.set(yticks = y_ticks, yticklabels = y_tick_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### > Format segments and boundaries\n",
    "\n",
    "To facilitate searching for pairs of segments with the same number of beats, we will convert the segments to the format `[start, length]`. To easily go back and forth between the different sampling indices, we will do this in beats, seconds, and frames."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_segments(s, beat_times):\n",
    "\n",
    "    \"\"\"\n",
    "    At each level of the hierarchy, format each segment as [start, length].\n",
    "    Do this for starting time and length in time, starting beat and lenght \n",
    "    in beats, and starting frame and length in frames.\n",
    "\n",
    "    Input\n",
    "    -----\n",
    "    s: hierarchical structure, as returned by segment()\n",
    "    beat_times: beats in seconds, as returned by compute_laplacian()\n",
    "\n",
    "    Output\n",
    "    -----\n",
    "    f_beats: hierarchical structure with segment formatting in beats (np.array)\n",
    "    f_times: hierarchical structure with segment formatting in seconds (np.array)\n",
    "    f_frames: hierarchical structure with segment formatting in frames (np.array)\n",
    "    \"\"\"\n",
    "\n",
    "    f_beats = []\n",
    "    f_times = []\n",
    "    f_frames = []\n",
    "    \n",
    "    #traverse hierarchies\n",
    "    for i in range(s.shape[0]):\n",
    "\n",
    "        # Locate segment boundaries from the label sequence\n",
    "        bound_beats = 1 + np.flatnonzero(s[i][:-1] != s[i][1:])\n",
    "        # Count beats 0 as a boundary\n",
    "        bound_beats = librosa.util.fix_frames(bound_beats, x_min=0)\n",
    "        \n",
    "        # Convert beat indices to frames\n",
    "        bound_times = beat_times[bound_beats]\n",
    "        # Tack on the end-time\n",
    "        bound_times = list(np.append(bound_times, beat_times[-1]))\n",
    "\n",
    "        # format as [beat_start_position, segment_length_in_beats]\n",
    "        beat_poslen = []\n",
    "        # format as [time_start_position, segment_length_in_time]\n",
    "        time_poslen = []\n",
    "\n",
    "        #traverse beats\n",
    "        for idx in range(len(bound_beats)-1):\n",
    "            beat_poslen.append([bound_beats[idx], bound_beats[idx+1]-bound_beats[idx]])\n",
    "            time_poslen.append([bound_times[idx], bound_times[idx+1]-bound_times[idx]])\n",
    "\n",
    "        f_beats.append(beat_poslen)\n",
    "        f_times.append(time_poslen)\n",
    "        f_frames.append(librosa.time_to_frames(time_poslen, hop_length=1))\n",
    "\n",
    "    return np.asarray(f_beats), np.asarray(f_times), np.asarray(f_frames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### > Find segment pairs\n",
    "\n",
    "Traverse the hierarchies of both songs and find all pairs of segments across them that have the same number of beats. Store each pair in the format `[song1_hierarchy_index, song1_segment_index, song2_hierarchy_index, song2_segment_index]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def find_pairs(f_beats1, f_beats2):\n",
    "    \"\"\"\n",
    "    Given two hierarchical structures with [start_in_beats, length_in_beats]\n",
    "    segment formatting, return all pairs of segments with the same number of beats.\n",
    "\n",
    "    Input\n",
    "    -----\n",
    "    f_beats1: formatted hierarchical structure in beats of first song (np.array)\n",
    "    f_beats2: formatted hierarchical structure in beats of second song (np.array)\n",
    "\n",
    "    Output\n",
    "    -----\n",
    "    pairs: all matched pairs, in the format \n",
    "           [song1_k_index, song1_segment_index, song2_k_index, song2_segment_index]\n",
    "    \"\"\"\n",
    "\n",
    "    pairs = []\n",
    "\n",
    "    # not the prettiest, but easier to understand loop\n",
    "    for k1 in range(f_beats1.shape[0]):  # traverse hierarchies, song 1\n",
    "        for segs1 in range(len(f_beats1[k1])):  #traverse segments, song 1\n",
    "            for k2 in range(f_beats2.shape[0]):  #traverse hierarchies, song 2\n",
    "                for segs2 in range(len(f_beats2[k2])):  #traverse segments, song 2\n",
    "                    # store if segment duration matches\n",
    "                    if f_beats1[k1][segs1][1] == f_beats2[k2][segs2][1]:\n",
    "                        pairs.append([k1,segs1,k2,segs2])\n",
    "    \n",
    "    return np.asarray(pairs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### > Visualize matches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### >> Run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# format segments\n",
    "f_beats1, f_times1, f_frames1 = format_segments(s1, beat_times1)\n",
    "f_beats2, f_times2, f_frames2 = format_segments(s2, beat_times2)\n",
    "\n",
    "# find all pairs\n",
    "pairs = find_pairs(f_beats1, f_beats2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### >> Plot merged pairs over Constant-Q Transform\n",
    "\n",
    "We will plot all segments that have been matched. In order to have less clutter, we will add a vertical axis indicating how many times that particular segment has been matched. This has the side effect that we're potentially loosing the full information about matches for subsegments that belong to a larger segment. We will plot this over the Constant-Q Transform of each song, to see the structural boundaries on it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "segment_matches1 = np.zeros(y1.shape[0])\n",
    "segment_matches2 = np.zeros(y2.shape[0])\n",
    "\n",
    "for pair in pairs:\n",
    "\n",
    "    # get starting and ending frame indices\n",
    "    start, end = (f_frames1[pair[0]][pair[1]][0], \n",
    "                  f_frames1[pair[0]][pair[1]][0] + f_frames1[pair[0]][pair[1]][1])\n",
    "    # make a samples array with 1s during the segment\n",
    "    seg1 = np.zeros(y1.shape[0])\n",
    "    seg1[start:end] = 1\n",
    "    # add segment to segment_matches\n",
    "    segment_matches1 += seg1\n",
    "\n",
    "    # get starting and ending frame indices\n",
    "    start, end = (f_frames2[pair[2]][pair[3]][0], \n",
    "                  f_frames2[pair[2]][pair[3]][0] + f_frames2[pair[2]][pair[3]][1])\n",
    "    # make a samples array with 1s during the segment\n",
    "    seg2 = np.zeros(y2.shape[0])\n",
    "    seg2[start:end] = 1\n",
    "    # add segment to segment_matches\n",
    "    segment_matches2 += seg2\n",
    "\n",
    "# plot CQT and segment matches on top\n",
    "fig, ax = plt.subplots(2, 1, figsize=(15, 10))\n",
    "\n",
    "librosa.display.specshow(C1, y_axis='cqt_hz', sr=22050,\n",
    "                                bins_per_octave=BINS_PER_OCTAVE,\n",
    "                                x_axis='s', ax=ax[0])\n",
    "ax0 = ax[0].twinx()  # twin axis\n",
    "segment_matches1 = segment_matches1[::22050]  # subsample to seconds indexing\n",
    "ax0.plot(segment_matches1, color=\"white\", linewidth=3)\n",
    "ax0.set_ylabel(\"Number of matches\")\n",
    "ax0.set_ylim(ymin=0)\n",
    "ax[0].set(title='Valid segments of ' + path1)\n",
    "\n",
    "\n",
    "librosa.display.specshow(C2, y_axis='cqt_hz', sr=22050,\n",
    "                                bins_per_octave=BINS_PER_OCTAVE,\n",
    "                                x_axis='s', ax=ax[1])\n",
    "ax1 = ax[1].twinx()  # twin axis\n",
    "segment_matches2 = segment_matches2[::22050]  # subsample to seconds indexing\n",
    "ax1.plot(segment_matches2, color=\"white\", linewidth=3)\n",
    "ax1.set_ylabel(\"Number of matches\")\n",
    "ax1.set_ylim(ymin=0)\n",
    "ax[1].set(title='Valid segments of ' + path2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### >> Matches of 3 largest segments\n",
    "\n",
    "In the paper I propose some further filtering we can do to the matches depending on the knowledge of our dataset. One of these is selecting only the 3 largest unique segments that have been paired. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get 3 pair with the largest segments\n",
    "first = second = third = pairs[0]\n",
    "for pair in pairs:\n",
    "    # get segment size in frames from k and structure index\n",
    "    # ensure they are unique segments across hierarchies\n",
    "    if f_frames1[pair[0]][pair[1]][1] > f_frames1[first[0]][first[1]][1]:\n",
    "        third = second\n",
    "        second = first\n",
    "        first = pair\n",
    "    elif (f_frames1[pair[0]][pair[1]][1] > f_frames1[second[0]][second[1]][1]):\n",
    "        if (f_frames1[pair[0]][pair[1]][1] != f_frames1[first[0]][first[1]][1]):\n",
    "            third = second\n",
    "            second = pair\n",
    "    elif (f_frames1[pair[0]][pair[1]][1] > f_frames1[third[0]][third[1]][1]):\n",
    "        if (f_frames1[pair[0]][pair[1]][1] != f_frames1[second[0]][second[1]][1]):\n",
    "            if (f_frames1[pair[0]][pair[1]][1] != f_frames1[first[0]][first[1]][1]):\n",
    "                third = pair\n",
    "                \n",
    "# arrays to store arrays of samples for 3 largest segments\n",
    "segments1 = []\n",
    "segments2 = []\n",
    "\n",
    "for pair in [first, second, third]:\n",
    "\n",
    "    # get starting and ending frame indices\n",
    "    start, end = (f_frames1[pair[0]][pair[1]][0], \n",
    "                  f_frames1[pair[0]][pair[1]][0] + f_frames1[pair[0]][pair[1]][1])\n",
    "    # make a samples array with 1s during the segment\n",
    "    seg1 = np.zeros(y1.shape[0])\n",
    "    seg1[start:end] = 1\n",
    "    # append each segment sample array individually\n",
    "    segments1.append(seg1)\n",
    "\n",
    "    # get starting and ending frame indices\n",
    "    start, end = (f_frames2[pair[2]][pair[3]][0], \n",
    "                  f_frames2[pair[2]][pair[3]][0] + f_frames2[pair[2]][pair[3]][1])\n",
    "    # make a samples array with 1s during the segment\n",
    "    seg2 = np.zeros(y2.shape[0])\n",
    "    seg2[start:end] = 1\n",
    "    # append each segment sample array individually\n",
    "    segments2.append(seg2)\n",
    "\n",
    "# plot CQT and 3 segments with different colors on top\n",
    "fig, ax = plt.subplots(2, 1, figsize=(15, 10))\n",
    "\n",
    "librosa.display.specshow(C1, y_axis='cqt_hz', sr=22050,\n",
    "                                bins_per_octave=BINS_PER_OCTAVE,\n",
    "                                x_axis='s', ax=ax[0])\n",
    "ax0 = ax[0].twinx()  # twin axis\n",
    "for individual_segment in segments1:\n",
    "    individual_segment_sec = individual_segment[::22050]  # subsample to seconds indexing\n",
    "    ax0.plot(individual_segment_sec, linewidth=3)\n",
    "ax0.set_ylim(ymin=0)\n",
    "ax[0].set(title='Unique, largest 3 matched segments of ' + path1)\n",
    "\n",
    "librosa.display.specshow(C2, y_axis='cqt_hz', sr=22050,\n",
    "                                bins_per_octave=BINS_PER_OCTAVE,\n",
    "                                x_axis='s', ax=ax[1])\n",
    "ax1 = ax[1].twinx()  # twin axis\n",
    "for individual_segment in segments2:\n",
    "    individual_segment_sec = individual_segment[::22050]  # subsample to seconds indexing\n",
    "    ax1.plot(individual_segment_sec, linewidth=3)\n",
    "ax1.set_ylim(ymin=0)\n",
    "ax[1].set(title='Unique, largest 3 matched segments of ' + path2)\n"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "5fa8e7a0e7c7188de72acea4ae1bc222d1770499c4c3d36ce32843ef46b20053"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}