root_note_detection/RootNoteDemo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Demo: using the root-note detection model\n",
    "\n",
    "In this notebook we demonstrate how a user can use our trained model for root-note prediction tasks on new pieces of music."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "import pandas as pd\n",
    "from collections import Counter\n",
    "\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "from music21 import *\n",
    "from music21.analysis.discrete import KrumhanslSchmuckler, SimpleWeights, AardenEssen, BellmanBudge, TemperleyKostkaPayne"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Step 1: load the model\n",
    "\n",
    "In the main notebook `root_note_detection.ipynb` we have experimented with multiple models for root-note detection, and saved the best model. We used code like this to save it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_filename = \"./models/SMOTE_DS_RandomForestClassifier.h5\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "```python\n",
    "rf = RandomForestClassifier()\n",
    "rf.fit(Xtrain, ytrain)\n",
    "pickle.dump(rf, model_filename)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now, a user does not need to retrain the model. We can load in the model (which is in our GitHub repo with the filename shown above), using code like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "rf = pickle.load(open(model_filename, \"rb\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Step 2: load a piece of music to be analysed\n",
    "\n",
    "Next, we'll load a piece of music where we want to detect the key. We'll choose the first piece in the *O'Neill's 1850* collection, which is in the `music21` corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "tune = corpus.parse(\"oneills1850/0001-0050.abc\", number=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The Enchanted Valley'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tune.metadata.title"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Step 3: extract features\n",
    "\n",
    "We'll use `music21` to extract the necessary features from this piece.\n",
    "\n",
    "The features we need are:\n",
    "\n",
    "```\n",
    "'Krumhansl-Shmuckler', 'simple weights', 'Aarden Essen',\n",
    "'Bellman Budge', 'Temperly Kostka Payne'\n",
    "'final_note', 'freq note', 'freq weighted acc'\n",
    "```\n",
    "\n",
    "Notice we do not include the `as_transcribed` feature: most datasets where root-note detection is required will not have a partially-reliable `as_transcribed` feature.\n",
    "\n",
    "We'll define a function to extract the features in the expected order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def feature_extractor(tune):\n",
    "    \n",
    "    # get all the predictions from the Krumhansl-Schmuckler and similar algorithms\n",
    "    histogram_predictions = [\n",
    "        method().getSolution(tune).getTonic().pitchClass\n",
    "        for method in\n",
    "            (KrumhanslSchmuckler, SimpleWeights, AardenEssen, BellmanBudge, TemperleyKostkaPayne)\n",
    "    ]\n",
    "    \n",
    "    # get a few features from heuristics. first, get the raw pitch\n",
    "    # class sequence as integers\n",
    "    tunef = tune.flatten()\n",
    "    pitch_classes = [note.pitch.pitchClass for note in tunef.notes]\n",
    "    \n",
    "    final_note = pitch_classes[-1]\n",
    "    freq_note = Counter(pitch_classes).most_common()[0][0]\n",
    "    \n",
    "    accentNotes = [0.0] * 12\n",
    "    thresh = 0.5 # this will get only beats stronger than 0.5, ie one note per beat\n",
    "    s = tune[1] # tune[1] is the stream\n",
    "    for idx, note in enumerate(s.recurse().notes):\n",
    "        if note.isNote and float(note.beatStrength) >= thresh:\n",
    "            pitchClass = note.pitch.pitchClass\n",
    "            dur = note.duration.quarterLength\n",
    "            accentNotes[pitchClass] += dur\n",
    "            \n",
    "    freq_weighted_acc = max(range(12), key=lambda i: accentNotes[i])\n",
    "    \n",
    "    heuristics = [\n",
    "        final_note,\n",
    "        freq_note,\n",
    "        freq_weighted_acc\n",
    "    ]\n",
    "    \n",
    "    # concatenate the two sets of predictions\n",
    "    return histogram_predictions + heuristics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now let's run our function on the tune we have extracted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "features = feature_extractor(tune)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[7, 7, 7, 7, 7, 7, 7, 7]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Well, it looks like this was an \"easy\" tune: all the key-detection algorithms (Krumhansl-Schmuckler and variants) and all of our heuristics, gave the same result. They predicted pitch-class 7, ie G. This doesn't always happen (and that's what makes the problem interesting, and motivates an ensemble model).\n",
    "\n",
    "Anyway, let's pass this feature vector into our model.\n",
    "\n",
    "### Step 4: run the model with these features to obtain a prediction\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf.predict([features])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "As shown, and as expected, the random forest model makes the prediction that the root note is pitch-class 7. In this case, the random forest doesn't add any value! In other tunes, it does -- as an ensemble algorithm it's a bit better than the sum of its parts. Detailed evaluation of this is in a forthcoming paper."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Conclusion\n",
    "\n",
    "To summarise our workflow:\n",
    "\n",
    "1. Load the pretrained RF model using `pickle`\n",
    "2. Load a piece of music using `music21`\n",
    "3. Extract features using `feature_extractor`\n",
    "4. Run the RF model with these features to obtain a prediction"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}