Skip to content

Home

Jump to bottom
Mu Li edited this page Jul 7, 2019 · 4 revisions

How to setup the running environment

The notebooks are based on both a developing branch of d2l-ai and a developing branch of mxnet. The instructions to setup the environments are slightly more complicated.

Running on macos

Assume you have both recent python (>=3.6) and pip (>=9) installed.

  1. first install mxnet with deepnumpy: 
    # For python 3.7
pip install https://apache-mxnet.s3-accelerate.amazonaws.com/dist/python/numpy/latest/mxnet-1.5.0-cp37-cp37m-macosx_10_11_x86_64.whl
# For python 3.6
pip install https://apache-mxnet.s3-accelerate.amazonaws.com/dist/python/numpy/latest/mxnet-1.5.0-cp36-cp36m-macosx_10_11_x86_64.whl
  2. install a developing version of d2l 
    pip install git+https://github.com/d2l-ai/d2l-en@numpy2

Now use jupyter notebook to start the notebooks.

Note that macos version doesn't support GPU.

Running from a fresh linux

We will use Ubuntu 16.04 on AWS as an example for setting up a GPU-enabled version on a fresh Linux.

  1. sudo apt-get update && sudo apt-get install build-essential

  2. Install the recent CUDA, which is CUDA 10.1 patch 1, and setup the bash environment

    echo "export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib64" >>.bashrc

  3. install python and pip

    sudo apt-get install python3-pip
pip3 install --upgrade pip

  4. install mxnet with deepnumpy API

    pip install https://apache-mxnet.s3-us-west-2.amazonaws.com/dist/python/numpy/latest/mxnet_cu101mkl-1.5.0-py2.py3-none-manylinux1_x86_64.whl --user

  5. install the developing version of d2l

    pip install git+https://github.com/d2l-ai/d2l-en@numpy2 --user

Now use jupyter notebook to start the notebooks.

Using AWS Sagemaker

Sagemaker provides MXNet environments under conda_mxnet_p*. It comes with mxnet-cu100mkl 1.4.0. We can change it by add the following script during creating a notebook:

#!/bin/bash
sudo -u ec2-user -i <<'EOF'

env_name=1day

# Create a new environment
conda create -n $env_name -y
source /home/ec2-user/anaconda3/bin/activate $env_name

# Add kernel to jupyter notebook
sudo /home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin/python -m ipykernel install --name $env_name

cd ~/SageMaker
rm -rf *

# Get notebooks from d2l-en.zip
wget http://numpy.d2l.ai/d2l-en.zip
unzip d2l-en.zip
rm d2l-en.zip

# Set notebooks' default kernel to $env_name
for f in chapter*/*ipynb; do
    sed -i s/\"language_info\":\ {/\"kernelspec\":\ {\"display_name\":\ \"$env_name\",\"language\":\ \"python\",\ \"name\":\ \"$env_name\"\},\"language_info\":\ {/g $f
done

# Get notebooks from 1day-notebooks git
git clone https://github.com/mli/1day-notebooks.git

# Set notebooks' default kernel to $env_name
for f in 1day-notebooks/notebooks-*/*ipynb; do
    sed -i s/\"display_name\":\ \"Python\ 3\"/\"display_name\":\ \"$env_name\"/g $f
    sed -i s/\"name\":\ \"python3\"/\"name\":\ \"$env_name\"/g $f
done

source /home/ec2-user/anaconda3/bin/deactivate

EOF
Clone this wiki locally