introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-incremental-training-highlevel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# End-to-End Incremental Training Image Classification Example\n",
    "1. [Introduction](#Introduction)\n",
    "2. [Prerequisites and Preprocessing](#Prequisites-and-Preprocessing)\n",
    "  1. [Permissions and environment variables](#Permissions-and-environment-variables)\n",
    "  2. [Prepare the data](#Prepare-the-data)\n",
    "3. [Training the model](#Training-the-model)\n",
    "   1. [Training with SageMaker Training](#Training-with-sagemaker-training)\n",
    "      1.  [Training parameters](#Training-parameters)\n",
    "      2.  [Input data specification](#Input-data-specification)\n",
    "      3.  [Start the training](#Start-the-training)\n",
    "   2.  [Training with Automatic Model Tuning (HPO)](#Training-with-automatic-model-tuning-hpo)\n",
    "4. [Inference](#Inference)\n",
    "   1. [Download test image](#Download-test-image)\n",
    "   2. [Evaluation](#Evaluation)\n",
    "   3. [Clean up](#Clean-up)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "Welcome to our end-to-end example of incremental training using Amazon Sagemaker image classification algorithm. In this demo, we will use the Amazon sagemaker image classification algorithm to train on the [Caltech-256 dataset](https://paperswithcode.com/dataset/caltech-256). First, we will run the training for few epochs. Then, we will use the generated model in the previous training to start another training to improve accuracy further without re-training again. \n",
    "\n",
    "To get started, we need to set up the environment with a few prerequisite steps, for permissions, configurations, and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prequisites and Preprocessing\n",
    "\n",
    "### Permissions and environment variables\n",
    "\n",
    "Here we set up the linkage and authentication to AWS services. There are three parts to this:\n",
    "\n",
    "* The roles used to give learning and hosting access to your data. This will automatically be obtained from the role used to start the notebook\n",
    "* The S3 bucket that you want to use for training and model data\n",
    "* The Amazon sagemaker image classification docker image which need not be changed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install --upgrade sagemaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "print(role)\n",
    "\n",
    "sess = sagemaker.Session()\n",
    "bucket = sess.default_bucket()\n",
    "prefix = \"ic-fulltraining\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import image_uris\n",
    "\n",
    "training_image = image_uris.retrieve(region=sess.boto_region_name, framework=\"image-classification\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data preparation\n",
    "Download the data and transfer to S3 for use in training. In this demo, we are using Caltech-256 dataset, which contains 30608 images of 256 objects. For the training and validation data, we follow the splitting scheme in this MXNet [example](https://github.com/apache/incubator-mxnet/blob/master/example/image-classification/data/caltech256.sh). In particular, it randomly selects 60 images per class for training, and uses the remaining data for validation. The algorithm takes `RecordIO` file as input. The user can also provide the image files as input, which will be converted into `RecordIO` format using MXNet's [im2rec](https://mxnet.incubator.apache.org/how_to/recordio.html?highlight=im2rec) tool. It takes under a minute to convert the entire Caltech-256 dataset (~1.2GB) on a p3.2xlarge instance. However, for this demo, we will use record io format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "s3_client = boto3.client(\"s3\")\n",
    "\n",
    "\n",
    "def upload_to_s3(channel, file):\n",
    "    s3 = boto3.resource(\"s3\")\n",
    "    data = open(file, \"rb\")\n",
    "    key = channel + \"/\" + file\n",
    "    s3.Bucket(bucket).put_object(Key=key, Body=data)\n",
    "\n",
    "\n",
    "# caltech-256\n",
    "s3_client.download_file(\n",
    "    \"sagemaker-sample-files\",\n",
    "    \"datasets/image/caltech-256/caltech-256-60-train.rec\",\n",
    "    \"caltech-256-60-train.rec\",\n",
    ")\n",
    "s3_client.download_file(\n",
    "    \"sagemaker-sample-files\",\n",
    "    \"datasets/image/caltech-256/caltech-256-60-val.rec\",\n",
    "    \"caltech-256-60-val.rec\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Two channels: train, validation\n",
    "s3train = \"s3://{}/{}/train/\".format(bucket, prefix)\n",
    "s3validation = \"s3://{}/{}/validation/\".format(bucket, prefix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# upload the rec files to train and validation channels\n",
    "!aws s3 cp caltech-256-60-train.rec $s3train --quiet\n",
    "!aws s3 cp caltech-256-60-val.rec $s3validation --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Once we have the data available in the correct format for training, the next step is to actually train the model using the data. After setting training parameters, we kick off training, and poll for status until training is completed.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the model\n",
    "\n",
    "Now that we are done with all the setup that is needed, we are ready to train our object detector. \n",
    "\n",
    "Training can be done by either calling SageMaker Training with a set of hyperparameters values to train with, or by leveraging SageMaker Automatic Model Tuning ([AMT](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html)). AMT, also known as hyperparameter tuning (HPO), finds the best version of a model by running many training jobs on your dataset using the algorithm and ranges of hyperparameters that you specify. It then chooses the hyperparameter values that result in a model that performs the best, as measured by a metric that you choose.\n",
    "\n",
    "In this notebook, both methods are used for demonstration purposes, but the model that the HPO job creates is the one that is used as the base one for incremental training. You can instead choose to use the model created by the standalone training job by changing the below variable `deploy_amt_model` to False."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deploy_amt_model = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training with SageMaker Training\n",
    "\n",
    "To begin, let us create a ``sageMaker.estimator.Estimator`` object. This estimator will launch the training job.\n",
    "\n",
    "#### Training parameters\n",
    "There are two kinds of parameters that need to be set for training. The first one are the parameters for the training job. These include:\n",
    "\n",
    "* **Training instance count**: This is the number of instances on which to run the training. When the number of instances is greater than one, then the image classification algorithm will run in distributed settings. \n",
    "* **Training instance type**: This indicates the type of machine on which to run the training. Typically, we use GPU instances for these training \n",
    "* **Output path**: This the s3 folder in which the training output is stored"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_output_location = \"s3://{}/{}/output\".format(bucket, prefix)\n",
    "\n",
    "ic = sagemaker.estimator.Estimator(\n",
    "    training_image,\n",
    "    role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.p3.2xlarge\",\n",
    "    volume_size=50,\n",
    "    max_run=360000,\n",
    "    input_mode=\"File\",\n",
    "    output_path=s3_output_location,\n",
    "    sagemaker_session=sess,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apart from the above set of parameters, there are hyperparameters that are specific to the algorithm. These are:\n",
    "\n",
    "* **num_layers**: The number of layers (depth) for the network. We use 18 in this samples but other values such as 50, 152 can be used.\n",
    "* **image_shape**: The input image dimensions,'num_channels, height, width', for the network. It should be no larger than the actual image size. The number of channels should be same as the actual image.\n",
    "* **num_classes**: This is the number of output classes for the new dataset. Imagenet was trained with 1000 output classes but the number of output classes can be changed for fine-tuning. For caltech, we use 257 because it has 256 object categories + 1 clutter class.\n",
    "* **num_training_samples**: This is the total number of training samples. It is set to 15240 for caltech dataset with the current split.\n",
    "* **mini_batch_size**: The number of training samples used for each mini batch. In distributed training, the number of training samples used per batch will be N * mini_batch_size where N is the number of hosts on which training is run.\n",
    "* **epochs**: Number of training epochs.\n",
    "* **learning_rate**: Learning rate for training.\n",
    "* **top_k**: Report the top-k accuracy during training.\n",
    "\n",
    "\n",
    "After setting training parameters, we kick off training, and poll for status until training is completed, which in this example, takes 2 to 3 minutes per epoch on a p3.2xlarge machine. The network typically converges after 10 epochs. However, to save the training time, we set the epochs to 2 but please keep in mind that it may not be sufficient to generate a good model.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ic.set_hyperparameters(\n",
    "    num_layers=18,\n",
    "    image_shape=\"3,224,224\",\n",
    "    num_classes=257,\n",
    "    num_training_samples=15420,\n",
    "    mini_batch_size=256,\n",
    "    epochs=2,\n",
    "    learning_rate=0.1,\n",
    "    top_k=2,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Input data specification\n",
    "Set the data type and channels used for training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = sagemaker.inputs.TrainingInput(\n",
    "    s3train,\n",
    "    distribution=\"FullyReplicated\",\n",
    "    content_type=\"application/x-recordio\",\n",
    "    s3_data_type=\"S3Prefix\",\n",
    ")\n",
    "validation_data = sagemaker.inputs.TrainingInput(\n",
    "    s3validation,\n",
    "    distribution=\"FullyReplicated\",\n",
    "    content_type=\"application/x-recordio\",\n",
    "    s3_data_type=\"S3Prefix\",\n",
    ")\n",
    "\n",
    "data_channels = {\"train\": train_data, \"validation\": validation_data}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Start the training\n",
    "\n",
    "Start training by calling the fit method in the estimator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "ic.fit(inputs=data_channels, logs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training with Automatic Model Tuning ([HPO](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html)) <a id='AMT'></a>\n",
    "***\n",
    "As mentioned above, instead of manually configuring our hyper parameter values and training with SageMaker Training, we'll use Amazon SageMaker Automatic Model Tuning. \n",
    "        \n",
    "The code sample below shows you how to use the HyperParameterTuner. For recommended default hyparameter ranges, check the [Amazon SageMaker Image Classification HPs documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/IC-Hyperparameter.html). \n",
    "\n",
    "The tuning job will take 15 to 20 minutes to complete.\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from sagemaker.tuner import IntegerParameter, ContinuousParameter\n",
    "from sagemaker.tuner import HyperparameterTuner\n",
    "\n",
    "job_name = \"DEMO-ic-mul-\" + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
    "print(\"Tuning job name: \", job_name)\n",
    "\n",
    "# Image Classification tunable hyper parameters can be found here https://docs.aws.amazon.com/sagemaker/latest/dg/IC-tuning.html\n",
    "hyperparameter_ranges = {\n",
    "    \"beta_1\": ContinuousParameter(1e-6, 0.999, scaling_type=\"Auto\"),\n",
    "    \"beta_2\": ContinuousParameter(1e-6, 0.999, scaling_type=\"Auto\"),\n",
    "    \"eps\": ContinuousParameter(1e-8, 1.0, scaling_type=\"Auto\"),\n",
    "    \"gamma\": ContinuousParameter(1e-8, 0.999, scaling_type=\"Auto\"),\n",
    "    \"learning_rate\": ContinuousParameter(1e-6, 0.5, scaling_type=\"Auto\"),\n",
    "    \"mini_batch_size\": IntegerParameter(8, 64, scaling_type=\"Auto\"),\n",
    "    \"momentum\": ContinuousParameter(0.0, 0.999, scaling_type=\"Auto\"),\n",
    "    \"weight_decay\": ContinuousParameter(0.0, 0.999, scaling_type=\"Auto\"),\n",
    "}\n",
    "\n",
    "# Increase the total number of training jobs run by AMT, for increased accuracy (and training time).\n",
    "max_jobs = 6\n",
    "# Change parallel training jobs run by AMT to reduce total training time, constrained by your account limits.\n",
    "# if max_jobs=max_parallel_jobs then Bayesian search turns to Random.\n",
    "max_parallel_jobs = 2\n",
    "\n",
    "\n",
    "hp_tuner = HyperparameterTuner(\n",
    "    ic,\n",
    "    \"validation:accuracy\",\n",
    "    hyperparameter_ranges,\n",
    "    max_jobs=max_jobs,\n",
    "    max_parallel_jobs=max_parallel_jobs,\n",
    "    objective_type=\"Maximize\",\n",
    ")\n",
    "\n",
    "\n",
    "# Launch a SageMaker Tuning job to search for the best hyperparameters\n",
    "hp_tuner.fit(inputs=data_channels, job_name=job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare for incremental training\n",
    "Now, we will use the best model created by the tuning job (or the model created by the standalone training job) to start another training with the same dataset. This new training will start with higher accuracy as it uses the model generated in the previous training. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the location of the model data from the tuning job's best training (or the previous standlone training)\n",
    "\n",
    "model_data = (hp_tuner.best_estimator() if deploy_amt_model else ic).model_data\n",
    "print(ic.model_data)\n",
    "\n",
    "# Prepare model channel in addition to train and validation\n",
    "model_data_channel = sagemaker.inputs.TrainingInput(\n",
    "    model_data,\n",
    "    distribution=\"FullyReplicated\",\n",
    "    s3_data_type=\"S3Prefix\",\n",
    "    content_type=\"application/x-sagemaker-model\",\n",
    ")\n",
    "\n",
    "data_channels = {\"train\": train_data, \"validation\": validation_data, \"model\": model_data_channel}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start another training\n",
    "We use the same non-tunable hyperparameters as before. When the model channel is present, the use_pretrained_model parameter is ignored. The number of classes, input image shape and number of layers should be the same as the previous training since we are starting with the same model. Other parameters, such as learning_rate, mini_batch_size, etc., can vary. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "incr_ic = sagemaker.estimator.Estimator(\n",
    "    training_image,\n",
    "    role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.p3.2xlarge\",\n",
    "    volume_size=50,\n",
    "    max_run=360000,\n",
    "    input_mode=\"File\",\n",
    "    output_path=s3_output_location,\n",
    "    sagemaker_session=sess,\n",
    ")\n",
    "incr_ic.set_hyperparameters(\n",
    "    num_layers=18,\n",
    "    image_shape=\"3,224,224\",\n",
    "    num_classes=257,\n",
    "    num_training_samples=15420,\n",
    "    mini_batch_size=128,\n",
    "    epochs=2,\n",
    "    learning_rate=0.01,\n",
    "    top_k=2,\n",
    ")\n",
    "\n",
    "incr_ic.fit(inputs=data_channels, logs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see from the logs, the training starts with the previous model and hence the accuracy for the first epoch itself is higher. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inference\n",
    "\n",
    "***\n",
    "\n",
    "We can now use the trained model to perform inference. You can deploy the created model by using the deploy method in the estimator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ic_classifier = incr_ic.deploy(initial_instance_count=1, instance_type=\"ml.m4.xlarge\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download test image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_name = \"/tmp/test.jpg\"\n",
    "s3_client.download_file(\n",
    "    \"sagemaker-sample-files\",\n",
    "    \"datasets/image/caltech-256/256_ObjectCategories/008.bathtub/008_0007.jpg\",\n",
    "    file_name,\n",
    ")\n",
    "# test image\n",
    "from IPython.display import Image\n",
    "\n",
    "Image(file_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation\n",
    "\n",
    "Evaluate the image through the network for inteference. The network outputs class probabilities and typically, one selects the class with the maximum probability as the final class output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import numpy as np\n",
    "from sagemaker.serializers import IdentitySerializer\n",
    "\n",
    "with open(file_name, \"rb\") as f:\n",
    "    payload = f.read()\n",
    "\n",
    "ic_classifier.serializer = IdentitySerializer(\"image/jpeg\")\n",
    "result = json.loads(ic_classifier.predict(payload))\n",
    "# the result will output the probabilities for all classes\n",
    "# find the class with maximum probability and print the class index\n",
    "index = np.argmax(result)\n",
    "object_categories = [\n",
    "    \"ak47\",\n",
    "    \"american-flag\",\n",
    "    \"backpack\",\n",
    "    \"baseball-bat\",\n",
    "    \"baseball-glove\",\n",
    "    \"basketball-hoop\",\n",
    "    \"bat\",\n",
    "    \"bathtub\",\n",
    "    \"bear\",\n",
    "    \"beer-mug\",\n",
    "    \"billiards\",\n",
    "    \"binoculars\",\n",
    "    \"birdbath\",\n",
    "    \"blimp\",\n",
    "    \"bonsai-101\",\n",
    "    \"boom-box\",\n",
    "    \"bowling-ball\",\n",
    "    \"bowling-pin\",\n",
    "    \"boxing-glove\",\n",
    "    \"brain-101\",\n",
    "    \"breadmaker\",\n",
    "    \"buddha-101\",\n",
    "    \"bulldozer\",\n",
    "    \"butterfly\",\n",
    "    \"cactus\",\n",
    "    \"cake\",\n",
    "    \"calculator\",\n",
    "    \"camel\",\n",
    "    \"cannon\",\n",
    "    \"canoe\",\n",
    "    \"car-tire\",\n",
    "    \"cartman\",\n",
    "    \"cd\",\n",
    "    \"centipede\",\n",
    "    \"cereal-box\",\n",
    "    \"chandelier-101\",\n",
    "    \"chess-board\",\n",
    "    \"chimp\",\n",
    "    \"chopsticks\",\n",
    "    \"cockroach\",\n",
    "    \"coffee-mug\",\n",
    "    \"coffin\",\n",
    "    \"coin\",\n",
    "    \"comet\",\n",
    "    \"computer-keyboard\",\n",
    "    \"computer-monitor\",\n",
    "    \"computer-mouse\",\n",
    "    \"conch\",\n",
    "    \"cormorant\",\n",
    "    \"covered-wagon\",\n",
    "    \"cowboy-hat\",\n",
    "    \"crab-101\",\n",
    "    \"desk-globe\",\n",
    "    \"diamond-ring\",\n",
    "    \"dice\",\n",
    "    \"dog\",\n",
    "    \"dolphin-101\",\n",
    "    \"doorknob\",\n",
    "    \"drinking-straw\",\n",
    "    \"duck\",\n",
    "    \"dumb-bell\",\n",
    "    \"eiffel-tower\",\n",
    "    \"electric-guitar-101\",\n",
    "    \"elephant-101\",\n",
    "    \"elk\",\n",
    "    \"ewer-101\",\n",
    "    \"eyeglasses\",\n",
    "    \"fern\",\n",
    "    \"fighter-jet\",\n",
    "    \"fire-extinguisher\",\n",
    "    \"fire-hydrant\",\n",
    "    \"fire-truck\",\n",
    "    \"fireworks\",\n",
    "    \"flashlight\",\n",
    "    \"floppy-disk\",\n",
    "    \"football-helmet\",\n",
    "    \"french-horn\",\n",
    "    \"fried-egg\",\n",
    "    \"frisbee\",\n",
    "    \"frog\",\n",
    "    \"frying-pan\",\n",
    "    \"galaxy\",\n",
    "    \"gas-pump\",\n",
    "    \"giraffe\",\n",
    "    \"goat\",\n",
    "    \"golden-gate-bridge\",\n",
    "    \"goldfish\",\n",
    "    \"golf-ball\",\n",
    "    \"goose\",\n",
    "    \"gorilla\",\n",
    "    \"grand-piano-101\",\n",
    "    \"grapes\",\n",
    "    \"grasshopper\",\n",
    "    \"guitar-pick\",\n",
    "    \"hamburger\",\n",
    "    \"hammock\",\n",
    "    \"harmonica\",\n",
    "    \"harp\",\n",
    "    \"harpsichord\",\n",
    "    \"hawksbill-101\",\n",
    "    \"head-phones\",\n",
    "    \"helicopter-101\",\n",
    "    \"hibiscus\",\n",
    "    \"homer-simpson\",\n",
    "    \"horse\",\n",
    "    \"horseshoe-crab\",\n",
    "    \"hot-air-balloon\",\n",
    "    \"hot-dog\",\n",
    "    \"hot-tub\",\n",
    "    \"hourglass\",\n",
    "    \"house-fly\",\n",
    "    \"human-skeleton\",\n",
    "    \"hummingbird\",\n",
    "    \"ibis-101\",\n",
    "    \"ice-cream-cone\",\n",
    "    \"iguana\",\n",
    "    \"ipod\",\n",
    "    \"iris\",\n",
    "    \"jesus-christ\",\n",
    "    \"joy-stick\",\n",
    "    \"kangaroo-101\",\n",
    "    \"kayak\",\n",
    "    \"ketch-101\",\n",
    "    \"killer-whale\",\n",
    "    \"knife\",\n",
    "    \"ladder\",\n",
    "    \"laptop-101\",\n",
    "    \"lathe\",\n",
    "    \"leopards-101\",\n",
    "    \"license-plate\",\n",
    "    \"lightbulb\",\n",
    "    \"light-house\",\n",
    "    \"lightning\",\n",
    "    \"llama-101\",\n",
    "    \"mailbox\",\n",
    "    \"mandolin\",\n",
    "    \"mars\",\n",
    "    \"mattress\",\n",
    "    \"megaphone\",\n",
    "    \"menorah-101\",\n",
    "    \"microscope\",\n",
    "    \"microwave\",\n",
    "    \"minaret\",\n",
    "    \"minotaur\",\n",
    "    \"motorbikes-101\",\n",
    "    \"mountain-bike\",\n",
    "    \"mushroom\",\n",
    "    \"mussels\",\n",
    "    \"necktie\",\n",
    "    \"octopus\",\n",
    "    \"ostrich\",\n",
    "    \"owl\",\n",
    "    \"palm-pilot\",\n",
    "    \"palm-tree\",\n",
    "    \"paperclip\",\n",
    "    \"paper-shredder\",\n",
    "    \"pci-card\",\n",
    "    \"penguin\",\n",
    "    \"people\",\n",
    "    \"pez-dispenser\",\n",
    "    \"photocopier\",\n",
    "    \"picnic-table\",\n",
    "    \"playing-card\",\n",
    "    \"porcupine\",\n",
    "    \"pram\",\n",
    "    \"praying-mantis\",\n",
    "    \"pyramid\",\n",
    "    \"raccoon\",\n",
    "    \"radio-telescope\",\n",
    "    \"rainbow\",\n",
    "    \"refrigerator\",\n",
    "    \"revolver-101\",\n",
    "    \"rifle\",\n",
    "    \"rotary-phone\",\n",
    "    \"roulette-wheel\",\n",
    "    \"saddle\",\n",
    "    \"saturn\",\n",
    "    \"school-bus\",\n",
    "    \"scorpion-101\",\n",
    "    \"screwdriver\",\n",
    "    \"segway\",\n",
    "    \"self-propelled-lawn-mower\",\n",
    "    \"sextant\",\n",
    "    \"sheet-music\",\n",
    "    \"skateboard\",\n",
    "    \"skunk\",\n",
    "    \"skyscraper\",\n",
    "    \"smokestack\",\n",
    "    \"snail\",\n",
    "    \"snake\",\n",
    "    \"sneaker\",\n",
    "    \"snowmobile\",\n",
    "    \"soccer-ball\",\n",
    "    \"socks\",\n",
    "    \"soda-can\",\n",
    "    \"spaghetti\",\n",
    "    \"speed-boat\",\n",
    "    \"spider\",\n",
    "    \"spoon\",\n",
    "    \"stained-glass\",\n",
    "    \"starfish-101\",\n",
    "    \"steering-wheel\",\n",
    "    \"stirrups\",\n",
    "    \"sunflower-101\",\n",
    "    \"superman\",\n",
    "    \"sushi\",\n",
    "    \"swan\",\n",
    "    \"swiss-army-knife\",\n",
    "    \"sword\",\n",
    "    \"syringe\",\n",
    "    \"tambourine\",\n",
    "    \"teapot\",\n",
    "    \"teddy-bear\",\n",
    "    \"teepee\",\n",
    "    \"telephone-box\",\n",
    "    \"tennis-ball\",\n",
    "    \"tennis-court\",\n",
    "    \"tennis-racket\",\n",
    "    \"theodolite\",\n",
    "    \"toaster\",\n",
    "    \"tomato\",\n",
    "    \"tombstone\",\n",
    "    \"top-hat\",\n",
    "    \"touring-bike\",\n",
    "    \"tower-pisa\",\n",
    "    \"traffic-light\",\n",
    "    \"treadmill\",\n",
    "    \"triceratops\",\n",
    "    \"tricycle\",\n",
    "    \"trilobite-101\",\n",
    "    \"tripod\",\n",
    "    \"t-shirt\",\n",
    "    \"tuning-fork\",\n",
    "    \"tweezer\",\n",
    "    \"umbrella-101\",\n",
    "    \"unicorn\",\n",
    "    \"vcr\",\n",
    "    \"video-projector\",\n",
    "    \"washing-machine\",\n",
    "    \"watch-101\",\n",
    "    \"waterfall\",\n",
    "    \"watermelon\",\n",
    "    \"welding-mask\",\n",
    "    \"wheelbarrow\",\n",
    "    \"windmill\",\n",
    "    \"wine-bottle\",\n",
    "    \"xylophone\",\n",
    "    \"yarmulke\",\n",
    "    \"yo-yo\",\n",
    "    \"zebra\",\n",
    "    \"airplanes-101\",\n",
    "    \"car-side-101\",\n",
    "    \"faces-easy-101\",\n",
    "    \"greyhound\",\n",
    "    \"tennis-shoes\",\n",
    "    \"toad\",\n",
    "    \"clutter\",\n",
    "]\n",
    "print(\"Result: label - \" + object_categories[index] + \", probability - \" + str(result[index]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean up\n",
    "\n",
    "\n",
    "When we're done with the endpoint, we can just delete it and the backing instances will be released. Uncomment and run the following cell to delete the endpoint and model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ic_classifier.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "notice": "Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 4
}