Applications of Deep Learning

Setup

• Clone repo and make sure to keep the folder structure intact
• Follow guide to obtaining datasets
• Delete the README.md files in all the folders
• Download to trained models weights
• Populate the Test folder with your own images according to category
• Install all the dependencies in the requirements

Run demos

• Download the saved pickle files
• Set the DEMO flag to true

Retrain

• Set the DEMO flag to false
• Run and Wait :)

Requirements

• Tensorflow >= 1.7 (it is recommended to use the version with GPU support)
• Keras >= 2.1
• Numpy >= 1.10
• Scikit Image >= 0.13.1
• Scikit Learn >= 0.19.1
• Pillow >= 5.0
• Pickle >= 0.7.1

The conda environment file or requirements provide the full list of packages

Datasets

Image Classification

• Obtain a a copy of the Kaggle Dogs vs Cats dataset and split the training images keeping equal ratio of labels into the folders Dataset/cats_dogs_large/Train/dog, Dataset/cats_dogs_large/Train/cat and similar folders for validation and testing. We have set asside 2000 images fro training
• The folder Dataset/abstract_classification was populated with two categories of approximately 1200 images hand picked from the Flickr 8k dataset. We can not redistribute this, but you can select several examples that depict close-up shoots of people or scenery and place them in the respective folders of training, validation and test

Image Captioning

• Obtain a copy of the Flickr 8k dataset from Department of Computer Science, University of Illinois at Urbana–Champaign and place it in the folder Datasets/flickr8k
• Download the word embeddings from GloVe and place them in Datasets/glove

Trained models

Image classification

• Download the cats and dogs classification weights and place them in Models/cats_and_dogs_large/final
• Download the cats and dogs classification small weights and place them in Models/cats_and_dogs_small/final
• Download the cats and dogs classification small augmented weights and place them in Models/cats_and_dogs_small_augmented/final
• Download the abstract classification weights and place them in Models/abstract_classification/final

Image captioning

• Download the image captioning weights and place them in Models/abstract_classification/final

Pickle files

• You can also download the pickle files from the release distribution and place them in the appropiate folder as needed or regenerate them by running the code

Test files

Image classification

• place some images in the folder Test/imagenet to observ the VGG16 predictions and explore the activations with quiver
• place some cats and dogs images in the folder Test/cats_and_dogs_large for the prediction of the retrained model on the full dataset
• place some cats and dogs images in the folder Test/cats_and_dogs_small for the prediction of the retrained model on the reduced dataset with and without augmentation
• place some random images in the folder Test/abstract_classification for the prediction of the model trained to distinguish between scenery and photos of persons

Image captioning

• place some random images in the folder Test/image_caption for generating captions

Misc

• The Presentation folder contains some images which may be subject to copyright and we will not distribute them. Some sections of the notebook will not rerun (usually the markdown sections of the code)

Enjoy and have fun (deep) learning