PyTorch-官方教程.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PyTorch是什么?\n",
    "\n",
    "它是一个基于 Python 的科学计算包, 其主要是为了解决两类场景:\n",
    "\n",
    "- NumPy 的替代品, 以使用 GPU 的强大加速功能。\n",
    "- 个深度学习研究平台, 提供最大的灵活性和速度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一. 新手入门\n",
    "\n",
    "### Tensors (张量)\n",
    "\n",
    "Tensor 是 PyTorch 中重要的数据结构，可认为是一个高维数组。它可以是一个数（标量）、一维数组（向量）、二维数组（矩阵）以及更高维的数组。Tensor 和 Numpy 的 ndarrays 类似，但 Tensor 可以使用 GPU 进行加速。Tensor 的使用和 Numpy 及 Matlab 的接口十分相似，下面通过几个例子来看看 Tensor 的基本使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch as t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.构建5*3的矩阵，只是分配了空间，未初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1.6395e-43  0.0000e+00  9.2486e-43\n",
       " 0.0000e+00  7.0065e-45  4.8617e+30\n",
       " 4.4280e-37  0.0000e+00  0.0000e+00\n",
       " 0.0000e+00  2.7552e-40  1.7749e+28\n",
       " 1.6604e-36  0.0000e+00  0.0000e+00\n",
       "[torch.FloatTensor of size 5x3]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x=t.Tensor(5,3)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.FloatTensor"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.使用 [0,1] 均匀分布随机初始化二维数组"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 0.8483  0.4879  0.9843\n",
       " 0.9719  0.6333  0.3482\n",
       " 0.3585  0.9425  0.2510\n",
       " 0.2901  0.8676  0.8539\n",
       " 0.7581  0.5903  0.1601\n",
       "[torch.FloatTensor of size 5x3]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = t.rand(5,3)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([5, 3])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.torch.Size 是 tuple 对象的子类，支持 tuple 的所有操作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.size()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4.加法运算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1.1953  1.2616  1.8893\n",
       " 1.8912  0.7641  0.9240\n",
       " 0.9479  1.4463  0.6354\n",
       " 1.1362  1.5495  0.9203\n",
       " 1.6701  1.5588  1.1289\n",
       "[torch.FloatTensor of size 5x3]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#第一种写法\n",
    "y=t.rand(5,3)\n",
    "x+y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1.1953  1.2616  1.8893\n",
       " 1.8912  0.7641  0.9240\n",
       " 0.9479  1.4463  0.6354\n",
       " 1.1362  1.5495  0.9203\n",
       " 1.6701  1.5588  1.1289\n",
       "[torch.FloatTensor of size 5x3]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#第二种写法\n",
    "t.add(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1.1953  1.2616  1.8893\n",
       " 1.8912  0.7641  0.9240\n",
       " 0.9479  1.4463  0.6354\n",
       " 1.1362  1.5495  0.9203\n",
       " 1.6701  1.5588  1.1289\n",
       "[torch.FloatTensor of size 5x3]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#第三种写法，指定加法结果的输出目标。需要预先分配空间\n",
    "result = t.Tensor(5,3)\n",
    "t.add(x,y,out=result)\n",
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5.函数名后面带下划线 _ 的函数会修改 Tensor 本身."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最初的y\n",
      "\n",
      " 0.3470  0.7737  0.9050\n",
      " 0.9193  0.1309  0.5758\n",
      " 0.5894  0.5038  0.3844\n",
      " 0.8461  0.6819  0.0665\n",
      " 0.9120  0.9685  0.9688\n",
      "[torch.FloatTensor of size 5x3]\n",
      "\n",
      "使用add加法之后，y的结果未改变\n",
      "\n",
      " 0.3470  0.7737  0.9050\n",
      " 0.9193  0.1309  0.5758\n",
      " 0.5894  0.5038  0.3844\n",
      " 0.8461  0.6819  0.0665\n",
      " 0.9120  0.9685  0.9688\n",
      "[torch.FloatTensor of size 5x3]\n",
      "\n",
      "使用add_加法之后，y的结果改变\n",
      "\n",
      " 1.1953  1.2616  1.8893\n",
      " 1.8912  0.7641  0.9240\n",
      " 0.9479  1.4463  0.6354\n",
      " 1.1362  1.5495  0.9203\n",
      " 1.6701  1.5588  1.1289\n",
      "[torch.FloatTensor of size 5x3]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('最初的y')\n",
    "print(y)\n",
    "print('使用add加法之后，y的结果未改变')\n",
    "y.add(x)\n",
    "print(y)\n",
    "print('使用add_加法之后，y的结果改变')\n",
    "y.add_(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6.Tensor 还支持很多操作，包括数学运算、线性代数、选择、切片等等，其接口设计与 Numpy 极为相似。更详细的使用方法，会在后面系统讲解。Tensor 和 Numpy 的数组之间的互操作非常容易且快速。对于 Tensor 不支持的操作，可以先转为 Numpy 数组处理，之后再转回 Tensor。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 0.4879\n",
       " 0.6333\n",
       " 0.9425\n",
       " 0.8676\n",
       " 0.5903\n",
       "[torch.FloatTensor of size 5]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "把 Tensor 转换为 Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      "[torch.FloatTensor of size 5]\n",
      " [ 1.  1.  1.  1.  1.]\n"
     ]
    }
   ],
   "source": [
    "a = t.ones(5)\n",
    "b = a.numpy()\n",
    "print(a,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "把 Numpy 转换为 Tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  1.  1.  1.  1.] \n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      "[torch.DoubleTensor of size 5]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.ones(5)\n",
    "b = t.from_numpy(a)\n",
    "print(a,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensor 和 numpy 对象共享内存，所以他们之间的转换很快，而且几乎不会消耗什么资源。但这也意味着，如果其中一个变了，另外一个也会随之改变。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 2.  2.  2.  2.  2.] \n",
      " 2\n",
      " 2\n",
      " 2\n",
      " 2\n",
      " 2\n",
      "[torch.DoubleTensor of size 5]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "b.add_(1)\n",
    "print(a,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7.Tensor 可以通过 .cuda 方法转换为 GPU 的 Tensor，从而享受 GPU 带来的加速运算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 在不支持CUDA的机器下，下一步不会运行\n",
    "if t.cuda.is_available():\n",
    "    x = x.cuda()\n",
    "    y = y.cuda()\n",
    "    x + y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二. Autograd: 自动微分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "深度学习的算法本质上是通过反向传播求导数，而 PyTorch 的 Autograd 模块则实现了此功能。在 Tensor 上的所有操作，Autograd 都能为它们自动提供微分，避免了手动计算导数的复杂过程。\n",
    "autograd.Variable 是 Autograd 中的核心类，它简单封装了 Tensor ，并支持几乎所有 Tensor 有的操作。Tensor 在被封装为 Variable 之后，可以调用它的 .backward 实现反向传播，自动计算所有梯度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variable（变量）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "autograd.Variable 是包的核心类，它包装了张量, 并且支持几乎所有的操作. 一旦你完成了你的计算, 你就可以调用 .backward() 方法, 然后所有的梯度计算会自动进行.你还可以通过 .data 属性来访问原始的张量, 而关于该 variable（变量）的梯度会被累计到 .grad 上去."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Variable主要包含三个属性：\n",
    "- data： 保存 Variable 所包含的 Tensor\n",
    "- grad:  保存 data 对应的梯度，grad 也是个 Variable\n",
    "- grad_fn: 指向一个 Function 对象，这个 Function 用来反向传播计算输入的梯度，具体细节会在后面讲解"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你想计算倒数, 你可以在 Variable 上调用 .backward() 方法. 如果 Variable 是标量的形式（例如, 它包含一个元素数据）, 你不必指定任何参数给 backward(), 但是, 如果它有更多的元素. 你需要去指定一个 grad_output 参数, 该参数是一个匹配 shape（形状）的张量."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.使用 Tensor 创建一个 Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 1  1\n",
       " 1  1\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = Variable(t.ones(2,2),requires_grad = True)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 4\n",
       "[torch.FloatTensor of size 1]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = x.sum()\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<SumBackward0 at 0x78917b8>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.grad_fn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#方向传播，计算梯度\n",
    "y.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 1  1\n",
       " 1  1\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# y = x.sum() = (x[0][0] + x[0][1] + x[1][0] + x[1][1])\n",
    "# 每个值的梯度都为1\n",
    "x.grad "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.grad 在反向传播过程中是累加的( accumulated )，这意味着每一次运行反向传播，梯度都会累加之前的梯度，所以反向传播之前需把梯度清零。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 2  2\n",
       " 2  2\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 3  3\n",
       " 3  3\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 0  0\n",
       " 0  0\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#以下划线结束的函数是inplace操作，就像前面的add_\n",
    "x.grad.data.zero_()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 1  1\n",
       " 1  1\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.Variable 和 Tensor 具有近乎一致的接口，在实际使用中可以无缝切换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable containing:\n",
      " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
      " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
      " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
      " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
      "[torch.FloatTensor of size 4x5]\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\n",
       " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
       " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
       " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
       " 0.5403  0.5403  0.5403  0.5403  0.5403\n",
       "[torch.FloatTensor of size 4x5]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = Variable(t.ones(4,5))\n",
    "y = t.cos(x)\n",
    "x_tensor_cos = t.cos(x.data)\n",
    "print(y)\n",
    "x_tensor_cos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三. 神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 神经网络可以使用 torch.nn 创建\n",
    "\n",
    "Autograd 实现了反向传播功能，但是直接用来写深度学习的代码在很多情况下还是稍显复杂，torch.nn 是专门为神经网络设计的模块化接口。nn 构建于  Autograd 之上，可用来定义和运行神经网络。nn.Module 是 nn 中最重要的类，可把它看成是一个网络的封装，包含网络各层定义以及 forward 方法，调用 forward(input) 方法，可返回前向传播的结果。下面就以最早的卷积神经网络：LeNet 为例，来看看如何用 nn.Module 实现。LeNet 的网络结构如图所示。\n",
    "![imge](http://pytorch.org/tutorials/_images/mnist.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是一个基础的前向传播( feed-forward )网络: 接收输入，经过层层传递运算，得到输出。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个典型的神经网络训练过程如下：\n",
    "\n",
    "- 定义具有一些可学习参数(或权重)的神经网络\n",
    "- 迭代输入数据集\n",
    "- 通过网络处理输入\n",
    "- 计算损失(输出的预测值与实际值之间的距离)\n",
    "- 将梯度传播回网络\n",
    "- 更新网络的权重，通常使用一个简单的更新规则: weight = weight - learning_rate * gradient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.定义网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "定义网络时，需要继承 nn.Module ，并实现他的 forward 方法，把网络中具有学习参数的层次放在构造函数 __init__ 中，如果某一层(如 relu )不具有可学习的参数，则既可放在构造函数中，也可以不放，但不建议放在其中，而在 forward 中使用 nn.funtional 代替。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (conv1): Conv2d (1, 6, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (conv2): Conv2d (6, 16, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (fc1): Linear(in_features=400, out_features=120)\n",
      "  (fc2): Linear(in_features=120, out_features=84)\n",
      "  (fc3): Linear(in_features=84, out_features=10)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        ##nn.Module子类的函数必须在构造函数中执行父类的构造函数\n",
    "        #下式等价于nn.Module.__init__(self)\n",
    "        super(Net,self).__init__()\n",
    "        \n",
    "        #卷积层'1'表示输入图片为单通道,'6'表示输出通道数，'5'表示卷积核为5*5\n",
    "        self.conv1 = nn.Conv2d(1,6,5)\n",
    "        #卷积层\n",
    "        self.conv2 = nn.Conv2d(6,16,5)\n",
    "        #仿射层/全连接层，y = Wx + b\n",
    "        self.fc1 = nn.Linear(16*5*5,120)\n",
    "        self.fc2 = nn.Linear(120,84)\n",
    "        self.fc3 = nn.Linear(84,10)\n",
    "    \n",
    "    def forward(self,x):\n",
    "        #卷积--激活--池化\n",
    "        x = F.max_pool2d(F.relu(self.conv1(x)),(2,2))\n",
    "        x = F.max_pool2d(F.relu(self.conv2(x)),2)\n",
    "        #reshape '-1'表示自适应\n",
    "        x = x.view(x.size()[0],-1)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "net = Net()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "只要在 nn.Module 的子类中定义了 forward 函数，backward 函数就会自动被实现(利用 Autograd ) 在 forward 函数中可以使用任何 Variable 支持的函数，还可以使用 if , for 循环， print 、log 等 Python 语法，写法和标准的Python写法一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "网络的学习参数可以通过 net.parameters() 返回， net.named_parameters 可同时返回学的参数以及名称。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10\n"
     ]
    }
   ],
   "source": [
    "params = list(net.parameters())\n",
    "print(len(params))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "conv1.weight : torch.Size([6, 1, 5, 5])\n",
      "conv1.bias : torch.Size([6])\n",
      "conv2.weight : torch.Size([16, 6, 5, 5])\n",
      "conv2.bias : torch.Size([16])\n",
      "fc1.weight : torch.Size([120, 400])\n",
      "fc1.bias : torch.Size([120])\n",
      "fc2.weight : torch.Size([84, 120])\n",
      "fc2.bias : torch.Size([84])\n",
      "fc3.weight : torch.Size([10, 84])\n",
      "fc3.bias : torch.Size([10])\n"
     ]
    }
   ],
   "source": [
    "for name,parameters in net.named_parameters():\n",
    "    print(name,':',parameters.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "forward 函数的输入和输出都是 Variable ，只有 Variable 才具有自动求导功能，而 Tensor 是没有的，所以在输入时，需把 Tensor 封装成 Variable。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 10])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input = Variable(t.randn(1,1,32,32))\n",
    "out = net(input)\n",
    "out.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#所有参数的梯度清零\n",
    "net.zero_grad()\n",
    "#反向传播\n",
    "out.backward(Variable(t.ones(1,10)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要注意的是，torch.nn 只支持 mini-batches ，不支持一次只输入一个样本，即一次必须是一个 batch 。但如果只想输入一个样本，则用 input.unsqueeze(0) 将 batch_size 设为１。例如 nn.Conv2d 输入必须是4维的，形如 nSamples×nChannels×Height×Width 。可将 nSample 设为 1 ，即 1×nChannels×Height×Width。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在继续之前, 让我门回顾一下迄今为止所有见过的类.\n",
    "\n",
    "概括:\n",
    "- torch.Tensor - 一个 多维数组.\n",
    "- autograd.Variable - 包装张量并记录应用于其上的历史操作. 具有和 Tensor 相同的 API ,还有一些补充, 如 backward(). 另外 拥有张量的梯度.\n",
    "- nn.Module - 神经网络模块. 方便的方式封装参数, 帮助将其移动到 GPU , 导出, 加载等。\n",
    "- nn.Parameter - 一种变量, 当被指定为 Model 的属性时, 它会自动注册为一个参数.\n",
    "- autograd.Function - 实现 * autograd 操作的向前和向后定义*. 每个 Variable 操作, 至少创建一个 Function 节点, 连接到创建 Variable 的函数, 并 编码它的历史.\n",
    "\n",
    "在这一点上，我们涵盖:\n",
    "- 定义一个神经网络\n",
    "- 处理输入并反向传播\n",
    "还剩下:\n",
    "\n",
    "- 计算损失函数\n",
    "- 更新网络的权重"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "nn 实现了神经网络中大多数的损失函数，例如 nn.MSELoss 用来计算均方误差，nn.CrossEntropyLoss 用来计算交叉熵损失。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 28.7705\n",
       "[torch.FloatTensor of size 1]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output = net(input)\n",
    "target = Variable(t.arange(0,10))\n",
    "criterion = nn.MSELoss()\n",
    "loss = criterion(output,target)\n",
    "loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果对 loss 进行反向传播溯源(使用 gradfn 属性)，可看到它的计算图如下：\n",
    "\n",
    "    input  -> conv2d -> relu -> maxpool2d -> conv2d -> relu -> maxpool2d  \n",
    "         -> view -> linear -> relu -> linear -> relu -> linear \n",
    "         -> MSELoss\n",
    "         -> loss\n",
    "当调用 loss.backward() 时，该图会动态生成并自动微分，也即会自动计算图中参数( Parameter )的导数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "反向传播之前 conv1.bias的梯度\n",
      "Variable containing:\n",
      " 0\n",
      " 0\n",
      " 0\n",
      " 0\n",
      " 0\n",
      " 0\n",
      "[torch.FloatTensor of size 6]\n",
      "\n",
      "反向传播之后 conv1.bias的梯度\n",
      "Variable containing:\n",
      "1.00000e-02 *\n",
      " -1.3395\n",
      "  1.0183\n",
      " -5.0718\n",
      "  4.1131\n",
      "  2.1308\n",
      " -7.4020\n",
      "[torch.FloatTensor of size 6]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#运行.backward()，观察调用之前和调用之后的grad\n",
    "net.zero_grad()#把net中所有可以学习参数的梯度清零\n",
    "print('反向传播之前 conv1.bias的梯度')\n",
    "print(net.conv1.bias.grad)\n",
    "loss.backward()\n",
    "print('反向传播之后 conv1.bias的梯度')\n",
    "print(net.conv1.bias.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.优化器（更新权重）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在反向传播计算完所有参数的梯度后，还需要使用优化方法来更新网络的权重和参数，例如随机梯度下降法( SGD )的更新策略如下：\n",
    "\n",
    "    weight = weight - learning_rate * gradient\n",
    "手动实现如下：\n",
    "\n",
    "    learning_rate = 0.01\n",
    "    for f in net.parameters():\n",
    "        f.data.sub_(f.grad.data * learning_rate)# inplace 减法\n",
    "torch.optim 中实现了深度学习中绝大多数的优化方法，例如 RMSProp、Adam、SGD 等，更便于使用，因此大多数时候并不需要手动写上述代码。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "#新建一个优化器，指定要调整的参数和学习率\n",
    "optimizer = optim.SGD(net.parameters(),lr = 0.01)\n",
    "\n",
    "#在训练过程中\n",
    "#先将梯度清零（与net.zero_grad()效果一样）\n",
    "optimizer.zero_grad()\n",
    "\n",
    "#计算损失\n",
    "output = net(input)\n",
    "loss = criterion(output,target)\n",
    "\n",
    "#反向传播\n",
    "loss.backward()\n",
    "\n",
    "#更新参数\n",
    "optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四. 训练一个分类器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "是这个, 你已经看到了如何定义神经网络, 计算损失并更新网络的权重.\n",
    "\n",
    "现在你可能会想,\n",
    "\n",
    "### 数据呢?\n",
    "一般来说, 当你不得不处理图像, 文本, 音频或者视频数据时, 你可以使用标准的 Python 包将数据加载到一个 numpy 数组中. 然后你可以将这个数组转换成一个 torch.*Tensor.\n",
    "\n",
    "- 对于图像, 会用到的包有 Pillow, OpenCV .\n",
    "- 对于音频, 会用的包有 scipy 和 librosa.\n",
    "- 对于文本, 原始 Python 或基于 Cython 的加载, 或者 NLTK 和 Spacy 都是有用的.\n",
    "- 特别是对于 vision, 我们已经创建了一个叫做 torchvision, 其中有对普通数据集如 Imagenet, CIFAR10, MNIST, 等和用于图像数据的转换器, 即, torchvision.datasets 和 torch.utils.data.DataLoader.\n",
    "\n",
    "这提供了巨大的便利, 避免了编写重复代码."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在深度学习中数据加载及预处理是非常复杂繁琐的，但 PyTorch 提供了一些可极大简化和加快数据处理流程的工具。同时，对于常用的数据集，PyTorch 也提供了封装好的接口供用户快速调用，这些数据集主要保存在 torchvison 中。\n",
    "\n",
    "torchvision 实现了常用的图像数据加载功能，例如 Imagenet、CIFAR10、MNIST 等，以及常用的数据转换操作，这极大地方便了数据加载，并且代码具有可重用性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 一个小的案例：CIFAR-10 分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们来尝试实现对 CIFAR-10 数据集的分类，步骤如下:\n",
    "\n",
    "- 1.使用 torchvision 加载并预处理 CIFAR-10 数据集\n",
    "- 2.定义一个卷积升级网络\n",
    "- 3.定义一个损失函数和优化器\n",
    "- 4.训练网络并更新网络参数\n",
    "- 5.测试网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.CIFAR-10 数据加载及预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[CIFAR-10](http://www.cs.toronto.edu/~kriz/cifar.html) 是一个常用的彩色图片数据集，它有10个类别: 'airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'。每张图片都是3×32×323×32×32，也即3-通道彩色图片，分辨率为32×3232×32。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torchvision as tv\n",
    "import torchvision.transforms as transforms\n",
    "from torchvision.transforms import ToPILImage\n",
    "show = ToPILImage() #可以把Tensor转成Image，方便可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "# 第一次运行程序torchvision会自动下载CIFAR-10数据集，\n",
    "# 大约100M，需花费一定的时间，\n",
    "# 如果已经下载有CIFAR-10，可通过root参数指定\n",
    "\n",
    "# 定义对数据的预处理,transforms.Normalize  归一化\n",
    "transforms = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "#训练集\n",
    "trainset = tv.datasets.CIFAR10(\n",
    "    root='/cy/',\n",
    "    train = True,\n",
    "    download = True,\n",
    "    transform = transforms)\n",
    "\n",
    "trainloader = t.utils.data.DataLoader(\n",
    "    trainset,\n",
    "    batch_size = 4,\n",
    "    shuffle = True,\n",
    "    num_workers =2)\n",
    "\n",
    "#测试集\n",
    "testset = tv.datasets.CIFAR10(\n",
    "    '/cy/',\n",
    "    train = False,\n",
    "    download = True,\n",
    "    transform = transforms)\n",
    "\n",
    "testloader = t.utils.data.DataLoader(\n",
    "    testset,\n",
    "    batch_size = 4,\n",
    "    shuffle = False,\n",
    "    num_workers = 2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset 对象是一个数据集，可以按下标访问，返回形如( data,label )的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ship\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAIAAAD/gAIDAAALVElEQVR4nO1cW3MVxxGe2d1zk46E\nhEASlpCEBaEo43K5UqmUKz8jpCo/MQ/Jj0j5JQllLGIw2NxsK4iLERJH17PXPEz316OZxdLoeb4X\nWruzvbPDfNOX6Tn64cuRUkopVde1OomqEbmsaqcZhIKbFTVJVVV5jRsWRGdRlaRc8d2Gbmtu3/AD\nTdM4Ql4m0tXavYs+NI1mVepjX9pUckUXlXMX7RMVcWbEwQpAppkCEACttMgsJiw1uOK1EYHvJWht\nvQWqUhY0s0FrJqbGYy5V00S6Bwjf5RpdSZKU/vaorRrpldau2oRfFGdWAOJgBSBLZMLy5OS/Ey1D\nCakRXqAZBDZJunEayRVrkguNEpe3iSwO3DkxWCCv9R0ed1J0hvsO9uFtYLTSNg3d5QhsjTMrAHGw\nApBZfHJnvj2zwYtaYTLz5GzQ5qQiS2w8U6tsHmm3WeL1wmK9e+VEJ+C7ijlkN5VvZQluJM5HqTbv\nF0Y6zqwAxMEKQBysAGRwWH0XO7GMqCxVWNr4AawFvivQ1O6CaLvdWpNLXXNEnYgn7boC0r0Ga6ul\nSvrAjgVPg6pkj58vQUOt8S2W68APIhiIHvx5EAcrAJk18wkWc+ygF3EsLnLuSey9qwNudC0+hJU5\nEp/Ddf0tEvqeDFwHiztYHBpXgzg0csv3VERV43EzevDnQRysAEg+S8JgvldbbEgSmBKka+mWzHyh\nFWJyNIJOC4hs4VJLPotttJDPzZbZy0cj7V2zqFJXuZh7lmp7zqAPSMYpV4g4HXGwAiCBdIt32tjW\nENlk7d/FE87f2mOHnSxqGtCQA1rtPqi8KxBOmjD0wPOQvTy45fm2JA1ASY3eeHyMOB1xsAKQiY/n\n08q68BteX4vQss9DQm3lfxEb4lErGhVb62mSXLXVP5ek1h0v2e0pP/HtkuJ2I9w4swIQBysAWcPT\nrvIqAM6I1CcdiMIzuYBHmGR4MEFWl1mQ8pNlUzhv0QolCOzxCotV3fD/OvIwKLNgVbVmd9oLRWtR\nLouD1q6vHGdWAOJgBUBI0VKMcDYknt8olUaNn8axd114ejeunRKC1O5mkrWDeyI6dL7DN82+oYQ1\nTKz8rV8A4Wd7Ik5HHKwAZFq5+/26JegTaEmQIg/jjnjLhEey0dr8hJXBbgjebO3XukkebM3aqkBk\nLQkc3PXKJrxMqd3hunYXkxQ+s/tVER9HHKwAZLLm+5nEVrRUyzGJavdOW+7UripgS4dSPNGIJCZt\nKaZI47CG1OonzC7KpPx9B+isGrwOHRUaViBpTZ5qmqb8FRFnRhysAMTBCoCUSYK24H9b4liWNvHF\nUZXfkrjlhYP5n1mrQ8aBcCUmPOFuUbMclf68q4QOpFYVJ+JorINwgLAdlXiOQtW6THsZ7RhInwdx\nsAJgbbJaRbzm38r2tgE/Nhan2fWMMZNB0IP9D9C0vf3OCEXB2StW1ZuYcl47nBxSr/hQTpL15TO4\n82VJroZfoCDuiFecUdvBAF/WnBqLZ3fOgzhYAcjatm1cwYb2JrNVe8R/s+1Dk4QLfp/98BCq7t69\na4TxeGyEPCc+Fg1Zyi++/NIIn9++bQTQcHK2B1U4QqekNApW3k1tV6UbFdi5A1hPmGZvszXiDIiD\nFYAsscp86N/W3BPguay1xmRmTR5/Gy6xXbh0ERdXlz+hFzEdtt+/N0JeEw0zVvr4+wdGuH79Bt86\n8Qb+CPSKbTrTFoF3guJcvlLZpcbMOkmXt5Q2RpyGOFgBaNndCd/fYQ1ylo6Jyf8X+TGZuV5X3njz\nxroRpqbIBf3mm3tG6A5njXBwdER9YtZfnL3g99M6g4cKRWTZvEIoT0r8PLhStXcsPs6sAMTBCoCY\nlMqLqqQw1vb6pJCBnTdVOQ+CAjjR8fbtKyN8d/9b6Dw+PjbC5i+/GCHNiKTXrpOw9XLLCF999Sfu\nFPWqKqQeIvUOi9f8OR22ffiZCvldB8mQW7UOqPzDOHBqO86sAMTBCkBWeT+XIlV6lt2QX3GQ/U9q\nX1aF00YOjLEvOneZrJvqiDVMFQV3U3Nz1GyOXNa8yo2w9YpoOL+wyMq5JMi22rUwivopd9wtnFq5\nYeOJPSfv5EyTRGsYjjhYAcgQOllzklDVYiPQLFMwgkix8hlLsaL0f3BhetoIPzx5YoT5K8vQeXBw\nYISpGaLh/v6+EV5vEfuevPjJCH/7+z+M8Jc7fzVCryuZUuvnlOhKXoBE2hFg2cUVtew+fNESzWKt\nwzkQBysAcbACkB0XpXNJ9kUsM4/cccXubJmT/52mXW5BQ//zTz8b4e3bX42wf3hohPxEJRScD96w\n6Q2MsLh01QhXr103wmBIy193YpJ7YvWZ/Ymyoe6N+St6aYe/y1udJeQQVVhwk9oNSOLMCkAcrABk\n9+7/10jwtuEldKzcU6/DfnNN/vrkgPzvJCEaNglduXdvwwgbG/eNsLu3Z4SF1TXoXF4mN+Lp06dG\nmGNXfmVlxQjrN24aYW2Nkl9vft02wrgQHoJZ45w2ipBTyziQxg6TtfdLRCtKey1q4SZpcC9EfBxx\nsAKQvf+wa6TBgCxRxkmlzLKGmoPJNSbIzDTlgvsDqkJ49uJ/dGuGMr/r69eMsDMi13x6fhE6//Xv\n/xhhc3PTCCWnqO7c+bMRZmcptH786LER3rwmGua2OWQTdshmt9MhIwinPpX9Hg6k4dNbNMTeKtYl\nv4Y64nTEwQpABpNSHNAEnp2l3FOv30W7hUt0scPcHI12jbC3T/Gw4jNqv7tJlmtpiUi3u0c03DnM\nofOPf/i9Eb74/DNqtks6+/zqmRnyRY8OaJvnYH/EfWeiWdVRiIgrzohhdwe0bbyAv2yj4W9UL0Wc\njjhYAcgSnszb22Rl9njCPzvaQbseVwpcmiVepFLaQCPe53I9mNGq5NxQ2bJBsrJ8hVRxVT4MMRzj\nfEz28ZPFy0bY3KRUV29yILqYUKMRkTTPmYZcnIsMV8qVvzCCRdFCQ+tcbsxnhSMOVgCyhmfdxUs0\nz1EOW42lWLbhY9mDASVzUQePCp5KUZuDQ7KPBVfyjXMOPGsxYTnzGDSE3cmYKSknWLocga6vXnUe\nV0qV7HlWnDhqeM8JDNOpe1K8kjNDkjgqeenAmlDHFM05EAcrABkog1mHdAccQqWULjkvyns5OVfN\n9jPKzHSEO8je8OOY+aX1Yww1NjvlPdyM+ctv2d+jDmRMzP60dC/nOG5+boaUF2TT9yoUPXT4HbKB\nRVcSoXQxphdVXAQMWxlnVgDiYAUgO2YaznEyBDwBv5RSyyuU1ex1aTI/evS9EV5uvTHCYEhbCUh4\ndlLyG3WXnUxl5yS50LxyDWuGA6mcGtIDEsbwNot9UcQBYMo1VDOTE0Y4PqRDL3VO2VosF3ND3h9Z\nmIcq1Dq8eU0PVtXgRHcjzoI4WAGIgxWAbOEy0fWIyzQS9iFu3/4M7VaWKTO1NyLmT0xQNvnwmIz0\n0xfPjfDkx2eknVUhRzbJJ+GU5a9P8PrS4aieM2MSig/6tHCguPKoOIYq/KbTaIeC//l5itKHvJIO\np+gtV68sGGHpCn17t2M5NLwX++7dB/5k+sA4swIQBysAGfI+MMljrtPf2JDK4offkYBULJJWq2tr\nRrh165YRUGb14AEduHn+nBi6s7MLnb0eu/68EwNh0KFb3Q7Fz91u12lTWbWNSUqdQeHFCgf8K4ur\nRri6St7PBU6E9bFzbKnCNm2vR+m50ZAS7nFmBSAOVgAyJGum+QDN+JBouPVqE+0O93aNAIp1mBf/\n/PprI3Q9WoE7S0tLRsjzH6ETaazhkExkxldqjl1hm0bcAcTkCJ6VUkfHtIZ8yiVKO2wWYaw7XVI+\n9SkRM0mQ/hYavt+mF/X7ZD3n5siUx5kVgDhYAfg/pQ4eZ65sAxcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=100x100 at 0x897D320>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(data, label) = trainset[100]\n",
    "print(classes[label])\n",
    "\n",
    "# (data + 1) / 2是为了还原被归一化的数据\n",
    "show((data + 1) / 2).resize((100, 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Dataloader 是一个可迭代的对象，它将 dataset 返回的每一条数据拼接成一个 batch ，并提供多线程加速优化和数据打乱等操作。当程序对 dataset 的所有数据遍历完一遍之后，相应的对 Dataloader 也完成了一次迭代。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       bird       truck       plane        bird\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAABkCAIAAAAnqfEgAAA5OklEQVR4nO19V5Bl13XduuG9+1K/\n0LknoWcADAYYRAIgQIJJJEVKtmhaLtmyykqWZalK/nCo8q//7B9/uCRXuaxS2bLNsq1Ulk2LESIJ\nAkwQkQYYYHLqns7p5XiDP+5ap3u6e/Tf9Nk/vfu9e88999zz3ttrh7UBK1asWLFixYoVK1asWLFi\nxYoVK1asWLFixYoVK1as/P8lzsGXfvnf/UqqVEpFHuT0AWSK5fTfKIpTpVYZS5V4NEyVUEq+mE+V\nxPFSZWO7x7MCnnVssgagvrOe/jvsNVOlUOGFYifHV5IaL93IpsqtD5Z5TJhJlerUVKq0hyGA6dlJ\nTmA4SpVS6Xiq1LtrqfLBre/xTk/yTncafCtJeI/LSxup8v3/8I09i4Tfe+V13mkuShV31EkVL/ZT\nZVTkbNtOwINjvhKGXKjIcwC4vBqyTpIqmSTkaC7f6iUcNnb5UsxjoUMQa9qu4wMIQ77T5EOAl3C2\nBZ0T+Bw2GfGKMThupN3hOTzLgQMg0nVDzcBxeOg/e+E53CuX3nhNU+IkwiFHG45CAAMthavROt1u\nqqxscW8k2qhFnytZKnBvVMsl3lEu0GQAwMtwY3g+FW1bxC5Ha3a4J2PN39zIaDjcdyMvv/IagO0W\nd+nM9ESqnDlxIlXmZ+dSJZP3943f7Q1S5cTMTKo885FP7h38937n76ZK1ucqZXzOJIo4k7EC73Rs\nzHwwuZJJwqUbDnjwcBQDGGnzjwacSaVcSRXf51aMYq7LaMSDXYebw9VOM8uSy+cABAEXP583SoFH\nup7G57IPBrz3sj7XpVKVo+V4lp/xAPiZ7L7rBs99HveKCytWrFg5IuIffGlh/XaqHHNnUyWOhwBO\nVWnmHD9OU6VV306VfJ4/bpBNsd3gW+sbVBpt/hr0q+OpUq5lADg8A3nZUwj4TY8Op+ft8Jt40OKv\nwWDE0aanjqVKbZIW1kQmAJCAg6xtr6TK3OzZVEkwnSrtDf4IZPVr8MjDj6SKMbVWR5s4TDIe5zYO\n/qb1lt9JlW6TptbY+Rc5fknrHGkRMvf8iDkyOnL68S/5XI04ouFTT/iKG/c5B91j6HIRBwl/3/g8\nZJ7Frkwh8IBYv6JJRs9ONl0iCyuRuWeumI4nMw6esdAPsdQpG9ttKptcyfoOjZQwigAMBlyuTMJR\nZqdphriy8SpVWuXlHFcym+Vq6D4wkqXgZQIAw1hzChO9zlXa3NpJlS+//M1UWVjn484GXA3zI9/r\n0jZptLoAEr1+aXEpVd67djNVjmsHZn0es3CXOGBru54qTz9+HofJhtYkyPAGg4CD5LJ8ZDljCsnE\nzQW8I19nZQMim9TUGvS4tn3tUjNsVqcYwyrW5gxHtImM4WksOMQ9AOGAg/Q7nFtLdronO3ow4J7p\ndLkBHnr4wVTZWb+TKo4OzgYZAJ6MPs8/5HspFWthWbFi5ciI/cKyYsXKkZFDTC/XpbHd7tFE3Npu\nAZgYp21cLPKApcV6qhjHZz5Poy5FkQCKRdrYJ0+dTJUhaNkuNNYBlAs0YkOZppke7c8TwSlOaUhE\nUBCSyhW2UiVbpAOy2ab12+1tAzh56oH039UlTvuRh3iD41VOqVunT7ddFy4d0S94/Axx6/GZaRwm\nfiwQ1yG+uPvjr6TKYIf4IhO1UiU4eYZn5empLU0QPji5EoDQkbPc5dxigbt+aPzBvPcxIY6c3Mgd\nYbdQNrabRACGsTCjse0FOc0DGvUEFnQusN+Zbxz/ruMAcIQJ9wDB+2LC999/L1VWVrgsgWDL8RPH\nAUxN0vNQLfBRTozJMZzdv50cR8EKIb6RQK7BtvXeEMCrP/hB+u/6Nh9QVo7hpUUitdVmg6PJQx82\nurohLbtuLePnAfRDLamUwQ433toGYZ0TE1wPFUYYCpa+9uZ7OEy2Gy1dxSBBoTx53xuaWz7gtPPC\nxVktVMaEGrwY2IX5nS5RXq3KIw14NKDaRCdMxMnztCW0GoHvAQgCf9/lfG8/JGzIKRSGfY3BZ+fr\njoZDfjZ7vXS28kVoAx4Ua2FZsWLlyIj9wrJixcqRkUMg4exUNVVGMszKpQqAZpMW7+XLH/AAJXcM\nhrQM81l+A04oh6uXpVGaEdAIPBqltfEigMUlRls2FXN86sTjqfLWa9dS5dR0fu9MAFQrDFkOFESb\nm2XsMk1B8j1O6YH5+VTZ2FjWAbThG8Ju5Qpx38oCre7JWQaq4A5wmGQivn79TeYZ+ds3eF89Wuz4\n4Gvp36WLQkx5Djv9xIdSJahNApg7/nD6bzLGoOdORICTgGb/ZEKL+rTyWbIhV3KpTXiSNZa57wNo\ndohb6wrZmChhSU+qkPBcfzc4yMmGBmmaGFGcAHBNbHA3I+y+RvyHn3yMc5gnwDdAL18sAggyHC6r\nC2d3wSmxSahLDhMF7xQm3hSY6o64f969eAl7QNalG9dTpSik6eh+/AxBYkZTasmxIAyEjGKLiB0A\nrqB6EiptTa/4+jQlAkpRyEkaxNw/kN7F+zI5Yiam6fGcUIOEAmguDCRXOA8mgKiQX5IAiPTpaOww\nVOd6BMjFAs81GWcmMJrAOChMlJDTy/o+gKLSr0xQ1WRdGWwYJ1y3kfIHOwOOVi7zwxsoqSDIegAy\numUzk4NiLSwrVqwcGbFfWFasWDkycggkrFWpLK0x029yag5AXmZzt0fcNF5RrUCRprVwBnylfjnG\n1oWKQka0HouZHIB+lgCneowW+/plQqrmOk8uPF3T+Dz3ZJVBwJU6gaqp55g/9SCA9XUizVt330qV\nwOFs/YSDTJQZn3JztJwrVYYyt7cVlsrIqr9Xig4t6lurxK15JUYWhJSDEbHztGoplhfupsqVhVu8\ndNEDkHvxpfTf0vwzqTI2ywzDYpEoshSqxKHP8fNZDjvIGsymaXseAEdZgg1TZaO/WSHBmsxyX08K\nMshHeppDoZ5Wtw8gUX1PJKjg4L6QsKLAcSEwKF4FJU4CICM04YQCgIrAGsg5ULitIcT0xrvvp8rr\nb76bKjttboAgmwfw4INMU+ybmKeCqhkTWFQGZtwXptZboRCfCZR5jgMg1iDGYeIkJp4oSKVFFn7d\nrWaL77NOgwEnEJkPjsMJREM+90IgJGtWTBjK81TRpdmmaC6KOQMDpVsDDutnlBUsAB4rMTWKzZ7f\nH/wdDSPsQamuao8SVeSYhGqzbqsbBKE91X5VKtylGRVa5QIfQGBijjZx1IoVKz8Bcsg3WU7u6okx\nudbcAYCgQDMqKMjzrYqckvIpchn+5tf7+jWT3y5f4rBxRzZRPwFQqvJLs5RwtC+9+v1U+dzP/mqq\nHJ+iPdVQTo3rcuZjeV5xqDSuy9c+ADA1xYyniQm6sYcD/qQU8/ypf3aG1aoffMBK5so4b3lHNa6Q\n8bhPZlRt88Df/HiqfPf3v807zZicF/2+Kafm1jp9ugs7/JGpVkMAzgwXp9ugCeat0IIozvLeCxVG\nFfyiwgs53ruMGORVihGNfABjHg+oKnHGFAA7StEaaN08/dy5+nF05HSP5OFO4hh7SmEdWVh/jZfU\n2BSJyfcJeLPtXh/AxhaXuqiSWnPOwjLjJBcu0VbdaNF5fPPOYqp0R/JAq7BjPPABZHMM1Iw0yVaP\nu7Sao61tEo6MqxiyWSJjfMlvHeSyADy9nggxGBd1DOMz1vqYMnHjxr5PvlokZ3m5xKcwoWokz69q\napykMXB6Mgx9U+OilLn0gXi6XqjHvbHF0EQpz/VRauOumWwIC0bR/ltLS6Z92VOuqq/d3SIenpLA\nzIRKZMIUWtJRbBbZARCa0hxjnh0Qa2FZsWLlyIj9wrJixcqRkUMgYVm8V2PKpfJGDoBaQCQ4Ehwr\n5Wm1hrK6MyqPqI3Rumt0afBvN4jmfFmn5UoFQKLqgcs/Zg13khCFlcdZH9PvEGRVxojmhkoe8ZUn\nVawJjbaaAHp9Zh41GmICqnC0M6oa73bopi0t8q2R6tQDnwZzPxROuVcyIjBod1kkND5HENpb4lul\nHM9tqKi92RRAFpKtuNsAnI3b6b+5EauF4johobvFHLG12lOp8tSnSFjWk980HPGKPnizflIA4Gpt\nC0I6sUl1ifiAOoJUQ1nsjti5TB5WLMCSnuweKPRx/fva8JGnSq8+13Z1lelvr/3orwBcubOa/hvs\nFutztLUdUX30+aQMMZYrx0WiOAwMT0AYQcxNABzfABwlNAmSQNEDX8eYIiQfBhuK6w0OAFdxDJMw\nFboKEWhZTMqho/FxULlXTp2id+LhB+jBmBXfVkZIudvnubfu0G+wvsSV7IuHzjyH1I1tIOiu11/3\nHqm0yEQ2PN185GhJ5croKibQa48AFDP8d6akQh+Tt+aYK2qRJcaXb7z7yb00ZMlfU5IjsRaWFStW\njozYLywrVqwcGTkEEpbExOooh6rgBADK+nYbyloeyVr2xd8QK35R1CCRMkPioWjnFDpJ0/r9mODr\n0juEBklMduNI9rOjQQaCbCbYkssZIjceU61UAfSUrFQV72B9h5i02aynSrvFmFqQ52w3t4jvcvkq\nlfLhUUIvUh1Gl6MlbdnnmyrWLyqhqc+FKo2Z1CauxngxC6DbMgEU5VjlOLdRh1OKWkxPW/s2UfbY\n6SdTpTBBkJu4LH/pOWMAXAHAks817+0mBHG5hgIRQ6UPVfRWJmF1i0mMGsADECsPyJAaB4bT74B8\n793LqfLWRUb6rt1kDlpnNALgBlzhnrwHptAn0fwdZZw5mlsk2GWikCYNapjEADKG7UAbI1SoTuFi\nBNpGkcHUSjYM5PfoD8S52OsCyGq/ZYS+TDJaJCjt7EYYTXTV1PGInPJeOf8o2SVnx0VMIhaTgT5T\nnlK/qtVqqrQb3CQDkeQNhbb8NJ57IFSXy4pIWh+uJDGRRE5mTKmYTUUhL11ismGz2wVQFAr++FOs\nu8oIThpouEv5YXiWd6OofMWEO9ODM4L5JgnuoFgLy4oVK0dG7BeWFStWjowcAgkdGcNQOC/tq1Gv\n0cjLq9dF0lQxhJLoTCmAIdSGa1L1RT8g8zhBCGB9gUDg0lsLqfLix9l5pVpTsxyZ1t0ObWBDS1AU\ngZ/p9XL5+jUAJ9TL5MwZEiHcvk3u7XabVxyqVsBVKYArarFYqZKBX8ShYko0xP6eUVnMzAmS891c\nYZAr0bCVCV4oP+RsJ0oO9sSVYqVxNsTynsnyQrmQqZItVR1t3SQlee8EAUXx2KdSpTDzHIB8kQuY\n8ag0VZ8UqVYjp6qOSGhrPL4nYRhAVzui72QARKaIX4/SPzz2BQB/+Odf5h31hIZMlLlUhBIyAQzM\nxtDjNgmEsUnXNImvghWuQpnmvTCJsCfrtSoQ1xETw0CIxjDoR3I1eAa/aIOZFjLpI4p0F2NmWNMA\nyXBLGHClTRKIlm9wnzhYYZeqkFPaqnO2O6KdqFWqqWKK5OoiUHGE2QMRwI9GCQBfxS6ucLH5XBrF\nxExzovQLNH5ni16IdpeQeWFpGcD5B1nT1hOUzikz2Y8Vb/X2o1ETHDwoKQYcOvt5Iw6KtbCsWLFy\nZMR+YVmxYuXIyCGQ0GTr5RSjGcuUAGzUZdGpL9OcKKWLZZWqKa2urSiYiUYlQ8UWZUKP5fIA3nrt\ndvrv0gKvu7BE7GYo/c6cZD2dJ0O9rnBeIjvTUduilBu+rzTFVp0BlFMaZFPtOUNFOdd3GKA0JfKe\nIj6NbbHx3SuZkNwMrVXy9pVKJG6HaYHZZVzSEVlCIOiXUdulcBsAMmXVqSuMGNSIoAcC5kmbS9pX\nymJb7TkH7at86zrR4nD26wCC2UfTf93ZZ1OlXGFYJ8kRt3YMdYGrrpbKGwzVgs3Aq7LjYQ8AEd08\nhu59rf2ecg53OQb0lpvE2IP3Xb2zW35oWAEcE2kSrjwEW6leL3YBjERLUFRRod8itooVQTZU7pPq\nwltvEGSZfQW1UEP2nsmP5E/I6iXTMdSU6fVEsDGlFOixEhNE3375nqmbykRDlbG2wQ3WFiNCucSM\n7ndVWXn5OrkJTx0jRiuoTnPoOgBiT9mh3i5DIK+jFRzpS6AwVuWN6Nb6qr50dykVHewJvGYMdjMt\ndeVhMGm0eiyk+gBQlXNJXzC7GaT89/4ZpNbCsmLFypGRwxqprtHcODnNopBSxgXQvcyvyhee/Giq\nVMv8on3jGmmCgwn5/NQje7qi/t2R3H4yQLrdIYDr79MMgfqqN+TGHmqQ7Xpdx6hOXd/9I1lSJlfr\n1KlTAHpKfdqpy3HY4Q8sZEY1ldnU6/OtvDJz2m1eMXHbOFRUOpPol7Ba+3Cq/PDdV1OltcU55JXs\nVJMXeYJlTijXitjDGlbf4H3113mnpXGeolw3DGTYDJXC5nS5dMWYN+I1NgF0ZKt2yz9OlVGZTA+5\n0y+kSnCGZM3ZAq2MRI75gcfxs/rFy49iAJ5Ib7dkau3cj+cJuy5243M1fUbzGQ9AR4aJ6ei5e6oh\n7TXOdil7PLjKPNKwLjLYEzYpKc/Ll3f5wVPzqXJmnkbxsrjMWm1OpqOe7xlZ7qkVYHz8GXEnuKpb\nOvcQs+GOHeOeX1pkHOmheVr3Y8qu+q/33mmQpWW0ukHocHeZDYA9mTMXrzAZ6k1ltLXU5LWmj0NR\nnMUpw3Kk2Q7F6G2YGCITTHCMic1Xuipr62kRGiLJcP0sgKyIPcpicPFgTGCNavKwdKFuj8OGJqii\no1Pj2hhWB3eCEWthWbFi5ciI/cKyYsXKkZFDIGFhJA+6LLZi7gyApyfo8yv2VVD+AA3dc/O0Km9t\nsHulO1RlSYbGZFvtQoI8rd8rF1YALN7a4HVrHH9tjbQNK6uXUqVaIXFwJmE+0fJdWs75HMednuOs\nGvUNAL4CAp7cjYsLtKhnpln602/VeYzc2IYDoCsbu1RRv8l7Ze3qt3g7TZqvN9aIvx7/yM+kyvUr\n5J9bfPNiqnTUdbWjWpxirQFAk0VVyoj3h555RpNiQFSGy86Qb1XE4xzJ0+zUAwAF+cKzoDO+36bS\n3eRsW5ffTpVj82xWVJ07zTtqcvw3bjGwEEQAcP6JF9N/w+MEQTv3LaXYdY7v0v7dS2ZgfO2eYwCg\nOYDnTk3RO5HJEYwMBVsGIuw2bWa8JAYwpmzBDz/9dKo8+vgjqeIqT+qDywxW3N1keMcUeJlEv114\n4vsAwtj48vk4TkyRw/r5x86lSn8kp744xCtKjzolXsl9YlpJXbmmllSmvifhnd40JA1qgeNrJzTb\nvKLJDgs8D4BYF9BW26RMhvu5KbhXFPNyFHKQHfWXXZc7ZW2nzlnFIfZUC5nlMoZPYlgA9RAN33HG\ndP3Zk02X/iVINHx+98/YshaWFStWjozYLywrVqwcGTkEEj58jJ1junna2FffrwO48/o76b+f+pnP\npsrcOdrADx1jyKnbJ5rbErl4I2TSU+SzDL21zIt+43+/A6BYNuR5jHDtrDC28sZfMFnlwTmSEGwP\nmCZTEnZaukloc/wkIWGQKwBYF3d1r19PlTDk+DeucUq9LkMnkbKiCgEt9maLx0wLH+2TxjpRkttS\nNyBll1ROk2nvpz73G6my+CaDdO++8pVUadeJFsfmqgD6DV5uU71dRZePVp1g4f010QaYCoYKUc+x\nSR7dk0Gec/MAcmIozPUJZErqr1N2uT47K++kynCZz25BjHF/eZGI4Hs3GUSrVMcAfO1rhMOf+Ye/\nnSqTj30Y95GCAkk9xXNhco5SSrwDPOfJAap4U4PiDohoSuLnK6krp2+Y1Ed9AONl7rcpcQ8MFIv9\nsz/6n6liysVcZeQZNohAaMXMLqVFdzWIo9qpM8cZEzQtDrYGhGwFBcQn1DZppqrw8L1y4zb3fFuP\nbKvJ7bqyzi2xI9ynKyMUdGrorLIq1QrZAMBIAXGTWhWqgEwZkzg1yYUauFz2UlGc9yJHNLyJ/cEQ\nwKr6afUV4S2qmscEVXeDwnqrEFAxjWBNWmX6xF1v/7kHxVpYVqxYOTJiv7CsWLFyZOQQSJiIHqDg\n0lZ0SrMAullChqhCO/DVN76eKg8eeyJVTs8RGty5woP7rhLwGjQ433uF1u/p408DmH+Jkabvvcay\n/tEqDdGbbzFKePnim6lSO0nze21xQzegmONVEsVValUAtxfJbVCvi5Mvw/vq9Oqp0hKBX6lKY74f\n8q3+iMrdtVs4TEKfSHZijobu6x8Q5blq3RkViV8e+ARB4sknGFHqd7gs41kA6C6+l/5740dMOl1S\nhLEX08jvOrTD17YZt7q1SFgHJTEer/AeT54AgGNVLs4DFSKRWHMb9HnujbsqtDolsFPT+i/xrYrB\nX34CwES6RotsYnpKNPkHJSuiuzDDhYpUtpUCDRM8GiaCuib90GQSCk6OFBzs7JgETh6bU+7l1FgB\nQE5w7M5N1q+ceZxlSV/84t/WDRBcXfiA63/rNvet4SzMCAOm/H95XWVC4Gta/bjMTAqq+Dn9JBkW\nZ6YYmPbuw3T4w3cupMrkOBf36jVOe6NnAnzivBejQ89MUuFCQ6tQKmQBwBDSKzDqKso5JsoKXwVk\nHSUMj/p8IobfIlCvhpT4v6sIY0fsKeaWTSzYwD0TJfQEOU15UGSSyR1377mWrcGKFSs/CXKIhbW4\nyXSPs/OsmE0KWQCf/nn+G5WYy7O8RGXxdf4u/drP0c38+Wd+PVXevvpXqdJboV3wxU/QZz9x8hiA\nP/nyl9J/G0oRevgTpLKKlEbyykV63z/ifipVuhv8ao/1ZdyUybC5EwDw5OrzfBXidHhAf6jUMLV0\nDT2mgAWiqXLkjl1XqcQ+iUo0mi5cf4UzGVPr1ilWolZj2gVuh2ba1iJ/Ni+8Q0/q2tItAJt3WOhz\nepa/22/doc91IFLd0yf4sJ49x/F3XueN9LIsuLlwlw/idrsDYEpcXi88UU2V+UmaWk2VUhuDertF\npaXe6Cst4x03iVEegOk2V9K/zga0q+KJPijG1TpQYlEYivA6AoCsf1+nu6mbdeSnnxR5VkUtncbL\nfHY5Y8oNewAma7QX4gF91VevMxHvEx/7ZKoUNezJ2WqqLNxlsY4pRglUO504EfbYcdM1Pu6BrIxB\nl8qjZ0nBVq5wboY47H5dcy7e4IN7Wk7rsw8z2rP4Bs3YgawkxWN262wMs1u7b3KpXACxb5KteNKJ\nGu3BQJmSy3cYWtkAwyMLTa7/2jofdKTx06dprGbs2lN8ITbMXLqi+lshVppVJJ96KFs7lytibxNW\nW/xsxYqVnwCxX1hWrFg5MnJY15w5Na2JlAUT3gYQJLQPvYhIavY0re6VmGlEv/9f/mOq/Oav/qNU\n+YUXfz1Vtmfp23PlnPzWha8DWO4RDf3i73yBw56tctgB3cwXfkQkdeXdd1Jl8W167rNj9Ac/+TxZ\nBzbX2wCGqrYZV9ecQEwMW5v1VFldI/g9XWPQoKcKho1NOvVL1cMpks9+7G+kyql5LtfOiBghX6Ix\nPC7o9OpfMP3qT/8XlbffUxWCAwBVteaZ+VnOdr1D8/jiImerxiX4Wz9FSLi5zdlOnyck9ARP2p0O\ngFaTB7i3ifLW6upauk3sEHTl+FT5SCcvRgSejYyAWG8nBrA+5OI8miNS6DQOB87ALggqqGpkIJdt\nCg1GagJqGAtMGo6hSD4xw7V96iG6C46LSmRukus/JhaqeqsBIKfbOakEvfXrTJ371ndeSZWffuFD\n+8afm2QsJTFkBvI5+G4CoNchVFxe5kp6KiA6f55OfUNM3Je/PMhzAe8HdTpD3vLlm9zYn/gIcxsn\nr3Lay9vi6hIBRtY3K8ZxekM+ka1WHcAg5n545uR8qsyIr6q5zoe40+Mpq6Cy3OQjMwU9xgme3mxe\nHYkCk1plWuoeUEyTWhNuMOljBvqldGDO3gqf+4i1sKxYsXJkxH5hWbFi5cjIIZBwvU+jt7VB67Q0\n7QLwRGW3sUSkM66ihLEpjnNlkTHB//Md2v8vnf54qqw0ib8Wlhj1uL1zCcBHf44pPIUcTcetNaZQ\nDUGL9OkXeczoGEsQkh7R4p3bBCONFhOXHM8DkJgSCo+YrizwWMjNp0pTrVXPniN33dYWX9kQbJyc\nqabKVdwjozJf/8s3eV/vXiKL4T/9V4QGG+JivnCRc3vqC38/VWZf4jjf+OorANqqhvcyvMEhzX94\nHqfdVcRnVRZ7q0Mb/sKr3+HNmgQmJNhjY3dUn9Qu8ZVHVc2zLV6KDYVxB4KEZx6iwT+R5xWfOj6G\nPXGxOWVs5ZP70jUMBryj2hjjUy+9+DTf6rQAZES8d+wYsVte4M6k8FTGiJnHyyoBUZqP4TkwLULL\nxRyAnboJCnMXnT3B8a/dJjfe17/7/VT5B7/4RR68zegtRB5p+oyuba1jT9+msqKTteNE6KG4v3sd\nAvCgwL230+EcbizexWFiUN7yFh/8rTu3U+X5J0gy8ZVXWeBlOLKVfYVY3aqGarY6SvoAfI/PZaSy\nHmR4pHqk4s46r7ih8GNDZAwmnOcKm2dcB0BFCYY55WdFJs9LSNzAPYPrjWIkI+ycHmyihAePNGIt\nLCtWrBwZsV9YVqxYOTJyCCRcUZzipUeJ5pz8AACU+JcNZPqKqjxUytxjzzB2E3aYCPfHrxEkFqq8\n1ikx7T17/lEAbZcWaaSuk2N5YofpvGn/SRu7/AjHf/B3fj5Vrl8iNnR8011yFcDOKic57NBqvXib\n1Gif/9xHUqU0yxscmyHsmnmIVyzN0egtjzMI9YP/8Sr2SLNNRPDyyyTnGwmD/OhbP+JqPMmmNQ+9\n9PlUefoLhB6dlnqUDgBg4W1WZiDDlWwOlFyn5MCsWp9mDNW66V7ZV1zPAOEE2BOXiUyCX5Fr23OZ\nmHruqWqqHN/kK4Go6V74JJdu8SZn+9SJLgBjr3c7XHw3um9YZ3acS/qRZ1inMj9NAF4MMgBiJVUa\ngJBTPNGUaORFhJAIe5rKft8zgSq1TQpDAHlRDqyt306VuePkm+zNEPJcVNeZt99nEdhHn+IjW1Hb\n3Yb6mIajCMDsLN0gjmBL37C/C5MOFFz74fcJOW+vrElZx2HiqBppKBvi3UsMDr74IYawywXe0ap6\nrJrmwQa7DYRk434EYLLKXXrhDmOaMwEXoVMnVF/RVtwUdo70reBmudPySmcNnBDAlLg2Y7Gzj4zp\n4+x/mqbgxgR/TU6pedBpTulABViG4OGgWAvLihUrR0bsF5YVK1aOjBwCCU+UCIIaiuuhVwIQq+Yr\nr5BNEjAIEsQ0j/MaLxqnnfzgR0RrneFZJV8FVuU8gOE27dvEpf0Zi6bdcdRHU1XdPSW5VUs0Jh95\nlsVfnb7aOm07AB4/w3qu61dp0r/xFqN4Ow3G7OJJsch3mERXLTBvMCcC+yh/yBIBiAJiq0KNACcI\nWZN1/e0fpsrEPHkHCzNM7AwDrsZwR5z0mSGAakm9V1WKNVAGoCMQVBBnuatjHJOkp9xFtdFESqyW\n6PWukjPvqmFaXxz7z3+IQahHHhDtXMLVmPMYWurVuNopY0FfWY49RdCyu+hzvzyknlfHJpRYq7TD\nXJAF4GtjmNhQq82ZDMUPt7FOpOMJJBbGuJKLS3yaQZ7jlKvjALICaHeWGBOsq3zy5Dz3xolZkmcs\n3GSx58eFW7fVQLcoEF2aHsOeNR8pLmY6qt64cTtV3r/CkPK779ML4WVVpre0isPEcNppRbEtev63\nP+DcqmPck5tNfmQc37gCBM1Uqpj6Bgpa2/nH5lNloI52yw0ucl0xwVBt0DzhU9c3/BAm+9cHMKka\nyZEekCOUZ0KlWaUBG9oGgw13Wd4FG1N3UJw50IT1gFgLy4oVK0dGDjEfZmbp2N5U8n6hlgNQExdt\nOS/enIwIs/RNCVEaNZv8cZ6pcrSREj8a6lHq+n0Aiccv760d/lQORQF7cpoZLq64e2IxAcVqIB6r\nk3t3wJ/EDGYAjFdJ33zuuSupMvVXLH/Pqv6+O+QplRJ//BP1VY/VBd51D08JqXv0WdZ9Hum3GT2Y\nbNMVfffyO6lSNHxS01q6bfp03Z1rAKYLXJPZEifw4jn+ZC2pdH5a7exrYzxGVMa7tklo7DLej9iv\n5BzNyR509Oj/+Ks0B547w7mdPcGzl7Y4rOE/yh8DgLxsmVBMEtHuRjI8yJRPqmQqFGvCSPTZvWiA\nPY8jJ6UsJoZYbUeNOzaSObC9Xeckl+nP3haN8vpOE9i1VVbu0LM+VKHMP/8X/zJVTsxwc373O4yc\nvP0Wox/lIo2I5RUaaBNRBKCmpvY7HW6exbs84Kvf/CZvUMseivNjbZP5fT2T/nSvZH2VahkbQsGE\nFWVmnZ0nd/nMOD93TXUM8kWVVVARVdqZJiebJVAjn1aDo3VMVx5tnqyunDM2kZpIZWXYzo3XAIwp\nvyyU+e+YDSYjPzFxADXjMf1TjbjGH++42OOM/2vEWlhWrFg5MmK/sKxYsXJk5BBI2KzTfDXVLXcW\n7wDwldTvKP1/4NK+nZaJiNgUTFDpd9TqQ8UbReXUbNTXAfS6PCAvztleyEP7ciEaXCZHJzoDmZeJ\nalnEBptPxgHcvEHvZqdCI/+xp9iNtTvgW9mKWqT4nJIhDAjFVNEQftknIxXKH3uYmTtXPyDtxNYm\nHeozd3jpjbXbqbJ8+ZVUOT3DOzlbXgZw/EEVo7R4O59/Wg1FslyWkmIdl+8QcnoO1z+bkYkO4xJO\nW7xwtiZLakF5QNmEVwxU57H2lnKF2ow81Osi2FUz0bPTHoDKBM+dPEmQUs6bK9zAvTIl73hU4CLH\nwhFp/ZDS7zAUwDEVRsb5agpxMgJK86ISnn/oLGerzjF/8F++BGBignDv+Q+z7evr32Mm3VtvMDfw\nzIM8t9vmue+9T6Ltn/7Mp1PlL/7zf0uVURIB+KVf+eX035e//d1UWd1g/t3Nu1zAygSdJy0N21Vl\nTHSf4MQpMQiuXuUgGRkTPRWsNJXteFrFQBeu0x8/0CelHHBtH5ytAfCUEeZ1CWC3uvS1b6paqK89\nk88pGCJcmTUJUwr4PDA7jT1e821xKMf6mNSqvBFX8zedbgej/SQN7m5vXQfYbZxrne5WrFj5SRD7\nhWXFipUjI4clGYlxLaPKj/Z6G8BAUZ5Q9mFXBm5dlmelTLBQEGtXTjEI08oxUspJcawMYGuLxqqr\n+EJRIGIg3DcUy3giIFZXV5tGg9QOpTxzasp+BcAbV2j2Bw/TxvbLxE2uQ17z1Q2G6rpCuydmmDQU\na5I7jToOE0dpXx9+9qOpcvHLX0uVK7dodT//CFfsmXnauu2Eox3nLaIyCwCFDI/sKvsmUnGCA66P\n79C0bilKOD3BMGEry3vf3iRaRC8B4MvGTlwl7OiRdRLa8JEp3xdaeX2RVxwYEu6Qs7m2PQQQlHjv\nH50mi0bZMQGg/ZAwiThb0+g0NrvOAYCMocozuTxKCDLFHKbXTEb7yjHcbzomo0jW+uoqgJs3CJf+\n/b/916ny0nP0CXzzGwzn/cmP2Y3pU59mn4GqksXaihffUB+dfjgA8Jba2yyvcl8trNID4OWI2dc3\nuYDDkWKmu+T4h9cwPfkIwenWDk9Zb1LpKddpp0FfzclZQs5amduopfKaCa3PqYIPIFNirNOU7NSF\n5kIZK66hS9QkdQh85XlNqMRnrJwHsKFGO3c26V2pt5jeVVrhp+yRY0xCPC7wHiUKUGqTV8bkZXJi\n3Hdt7hFrYVmxYuXIiP3CsmLFypGRQyBhXpZ5rEyyk/MnANSqRFLOsJ4qszNEIqMtBkoGHb5lIGEg\nSKhaejQ7tB4LU+MAxicY8mj3OMjIo/k67NBGzKtqfCDGgmaf+GhMUQkvYkVRmjwZJmprqvaQEW6n\nSgkk2Ov1VHmgcNt2i1Zrd0iM1lfwcZ/84Kv/ieMr2lKeYDrfgjp0XbnC2qbnn5HVrRKZXpOvTFcT\nABkxFuRC3Wmd4NcVIojqnOTDJ2l1z9a4tpdF0G4wSJKiOSE1Rwm3nsz92OUVh4oFy/zHqGewAf+6\niuulO6Lf5V1861uMlI2P3TflLxESDHdBHBU/4wEwiNAEj7LiBtht96S/QcD1MVmInljsFNdFt74N\n4PYd+goSrfljj7AiZ0J75o/+9M9T5RsvfyNVNuqE8//4t38rVZ547rlUqVUKgEnJ3SUmjzTJVpPP\nfbcRbLT/lnGfGqahWp997AW2In77KklTvLtUegrwdVuEnA9PE6ltOdy3jyraePr4SQDrDQL/m8sE\nyG0lrkaakkkKzZtGreKMVHAYJbWw6yMCcLdBJLjYEDF8gzvQVYFaQyv5iWeeT5WOsgsaqpE6Pq2s\n7CTCnuCgjRJasWLlJ0EOa1VvFP1mFfJVAH357bKRvKfqkZgr0d3Y0I9M1hGxb8/kSZniA8qt63cA\n5PM03Ezx89YqXXpFh8PmVIQSqhN3q84L+SJp9fq0AmozNQBj05zk8qbaaneZgnTqJL/y/T7Hv7tx\nV3OjXRMUxM1UUP3LvbJ568ucpH6OTrHyB7dXVZkhS2Gg+qFA5L9Bjb9mIdoAYi1pOOApKwv8yRrP\ncMGG+kkvTjBPanWJv2bLK1xk0z0lfXLyue9WR5vWMpv6uY6Uh5VR31rftE2V3SfeM6SN7ofqOR7K\nhR+q4OOgOIqlxHJj726w+J4XPFnlyQGHuiMrYDgSR7apEh4oqUf786lzDwN4QM1ysuoTs721oXM5\n7Gc+81McRIbb1UUa5ndWaEScPMPneny6CuCGfPm5nCi6zCRlK5o6FVPVZarQ4/v0g3n3ImuDnjzH\nOEZR037uKbbsXV3m3LyIe+P4JL3v81Kmatwbl1e2AUQiUytNsqKosbSmuamtVIWfgrEyIYJ2BNRW\nGBtCTv16G8CWGLebotCKlSk50treFT3coojCd/SZNQTI5+bV7ckFdu1RRAeKeIxYC8uKFStHRuwX\nlhUrVo6MHFaaI6KCnCkBz48DCB1adE6i7hprzPrJVejY7iSy9kS3VHF58KhP/19oOlMmPoCR6u/X\nxfLjdoURJokmLl4ng+2UJ/w44syXEmbBTGRo1W/sbABYadGkL5VZPrKTsFbDy4hwWQRDq02O1hPx\nlhOKqSc83EuaTwipYlE+lNXQ5YmzpqqGsOVluk2hmgcc00LNjRUAjDtcpYJa49TOcxFKJZUltblu\nK4IeNzdpkBuC2rFxKrUgALC2wslnc4RLU5M0+3du89xilm8VAkGnBpFgSf1simIfq7e6APKB+qCo\nRYrhhDrYAjMjutusSv9jwyQ1GADwTEKQFAPZXJ1iXkkMp5JwxUhI07zyO7/1WwAS4dahqkYM5cOO\n6p86wjxjIsJ+SPTcOWFnk8HUarQBBPpQmM3j+mRrSMRqbYiDd13HSl10ksNNhJ/7LNsozU9XOcnN\nH1DZ4g4/OUW4p1VHoGqnfov469KCJgMAGNcHM1I0ZqrGT5CnJa2N85WWAllZ1egcF1/Y3Alit8jp\nAxjleDuLa/wUmE/xblKdcsTubBGEbq7ykxIr03O7RWxbzt8TtDGMzwfFWlhWrFg5MmK/sKxYsXJk\n5DBIqBLzWPGxUqWGPS1MYgVxCmpM4ioeMq5wXjcmftwMaYvWRMa2bno63tsbw49NtIj2YUtsZ5G+\nWDOiVcsqO6qviElhTDb8oAmgWKlyNNm3swEv124zCjldpaE7OM5bbjb7GkSAYnS4dRoPOFoU85Qw\npHJskqZ7a43HLG3xju7eZqBqZoLT/qUvPAPg5hrN/qkSzeYHTvFO+8pKy4xxJh/c4c1vqZpmQtx+\nz36I0Gb1bh/A6goX38vwAT311PlUWVz+HmcywyBRLHLbbfVnnTmmsJEAXxoErsra7wltmfr+bS2g\nETccaRC159RwaQpVViAr1gYwTH6hjmx3+YBMmHU3r0l5ZEMdHPW7AGKVBPV13bo2dttgwxH31aYK\nrfIiVi5ocyYiQhh1hwAqRQKosaLilQoBGy7Jg+FOw3yN+2QYVVUg9diDjBImn+W+/cGbZAFpKxSb\ncfkpW1ihT6aptz79MVJTnH/kAQCJYvJ3F+khmapyh7S3ee6WCuBuL7EIKZ/jKx97iWVnzz3/Md5j\nAgB9pbb9m9/93VT5/ptvp8p4VXV1Kqe7scxLt+smW1BBc+XQzdSq2GM9ZQ3V9wGxFpYVK1aOjNgv\nLCtWrBwZOQQSZgtqbyOQ5SZ9AE6s6J5seNNQpKmUsDEVrC8Ldg0VEagoJ7PTpfnadocAMoqXlWSE\nF2vMgrslq7UkOvkJtbHcvEHzta0+l5lxjj/o7wCoKcCxKkrsaRWyJG1hk44KVgT7CoGpRuHNjkaH\n2/CdlrJnFfcxkY3tDQHMTZq+eZ/LUhW1W1v84lFhCsBawrv4y+/WU8URyjYpdL6w7ZL48vsjrtgD\nqgrKqvKivtME4CpvUHeDGzdvp0oimvyclr3VHenSlNk5Tvv2Hc4qTQ/0Mzxk0FAuce7w9FrsSQI0\nlBt7uqd4APp63YT5HMXUIsExXyDLUAkeTC7tK0W5Xq9jT+/YroKSq2qEYwq86sL7/WR/qqfJ9R2J\npMT1PAAF0fPPTrJa5ZjyePsdpR+b9jNKpxTg2wW5++T4KbI1hC7rrp58hvmiD4rI4dol7vn/+5Vv\npcqlq7dT5Wc//7lU+cVf+HupMndsAnsAaGunniq+1i1W1D7K0/lQFyn+9UvEd888yTkUVK43VpsA\nMFAiwbMPs8otp09xpHW7e5elUZkyz81PEo1WS9yu5onfWFwC0JCzKOPd15CyFpYVK1aOjNgvLCtW\nrBwZOQQSuoYJXJ1Th2EfwJhMR9Pjc6B+XIHCFm1BksGAAK2n+EthkqwMxdA0De0BKAtNuLKWW4o9\nldWvsSTTutGkVe8JlUyoQ5fJP9xuLAPoq1+WJ+66XMIp7fQ4yHiW4wdCHCUVRS6uEo0WC2UcJqOe\nKXNTJEilUKHiYmURtw+7CpXWaAw3trgsP3r3EoB+j5N/Xx1DBzEDMZECYpMKPm51ee+G6XAo2NJt\nmWL8BIAjSobRkKdcvkrOQkMuPhD5Yqhi+iBrIJsCYTDdMX0Arm7ZN82pTBTsgBjsZnqRGRmFIYBE\n5+byRc1tuE8xe9KwHjhalkRKRmmu2VwAoC042VRMsCvYsi4PRkvrZmgthoobDkN1J1BOYyWfB3Di\nmBhKtIK+MjDPHGP1YrMtN4Xa35XLjC1ubXFfvfvOW3uX4smnSdLQ63NKJso5OV5Nle3JeqrMzHEX\nPZMl/8Rv/NLfSZVTM9wkccqOonUzwB+K2nu6r9w4d/ik/BU7C9wkzS3eyMQUb23QagNoyg/zhc+Q\n+PALn/18qjQ6xJULiyy6NHSAJnPYdGBrqGPulWs3ANy8Q8ybqEj5oFgLy4oVK0dGDrGwfJWJh4Zg\nqNvFns4lu5lJMb+tZ0uselkI6Xeslmk3PaCvbb8oD3ef4xSTAoBcXrwFoqBS80eM9OOZlxe2WFCW\nh34oHDWViTQtxxkA6I5oOJgu6tkhYwitkHUMs+McJNbP6UD9JrOyueRe3C9ZGS+mXbiyxzA5aQjF\n1I1VXt6RGoVPTvOuR80NAHeX5UtWQAOmhiOUC79V553KzOiNaIq25UFvifUhrSgyNo2paBkqv8xY\nRMZDbGwWwz48HJgfOhlfQRZ77CmTmrdrlR+QXUYHOb93276GIwCBOtcbW6mj9jDGRe2q4sOEEfZw\nJxlzj8dMjFcBDNVpNasctKwuNNAPuOnAPjdNw6TT5FmeYgXVcpXHTI4DmDCscLrBcbV9feYx9k8a\n7mZmyboXjNje5t77/T/8EvaIscVG2q6ZjKrQNJMTp8kb8emf/lSqrGzQnKmVePp2nQl9/TDBnty3\nULQKFVnlWTXBHdZ5y6bHa67Ez+w777PJ7rSoSNIet7VxkUMobW13q+khPv7YOdwrZqdFuz1WeckP\nv/ACgIZmEioO8wff/if7BrEWlhUrVo6M2C8sK1asHBk5BBJGKk83jH25XAFAq07701X9jecZn6tQ\novo4jgZ1KoItaw4dbPlClWe1BwCG8omaRq27Ll4hjpwAWrdNf3mgivValb787Q1eqFQtACgJ19Rk\nsU/5dJdGQqmBmH87a0pBMoatEEevt7/WhHNzxGkhDOca/KVqIVcoNRDZdKmo6IRcqsdOxNjT8fT2\nGocNhVKLahQE9RpxDXYTUEpyvMe+cuU8pw6gXNHkBJdaorI9MUcQ5AxEq6boxC50UumP6VXTHwwB\n9EzJlJ57u3Mf5Lx3klrtPfRsDvZ45fv9/UttwKPpmmM4k6MDLUldz0y7C8CLOKVMIgCorXhKjAie\nhnv6PNHcmkjyNtdEdKf9mVItR/JFBGKOLCp3z+DWSCmHB8n6CrNTB14DgJ7yFrtKNNtukuKjUqZT\nvFblIztz7ulUOX5WFUXT9MkksTLvmk0AGZVbmRDBkhhWwjp59ep10UCu1zkHffAvXCYkXNnk9B47\n9xCAQpHelWld19AZBqb3qjgRMz434VAFPSbPzpxVmMoDmJtlqZzj74/PGLEWlhUrVo6M2C8sK1as\nHBk5LA8roS3nqLNLJh8BKBZokY58E2miebzRIwkBVOziB0yP2lSaz1zEV4KEJmJ31AQwLsqHUCjJ\n2YWcNK1LWXHG75CfLFPlaH6Rb/XXaOIWxwsAjiv/y1NgcUXnQsijrV4y2TyP6bdUryAQMlE7vOik\nooJy3yfSCXL89g+HQrJZnhvDBGs4bDXDt6JWG4AX0TY22NBzOOypYwrrQOldIuFbb+wnuvNchhSL\n+QDAmfNn0n/fvsikGE/Y0DCtma4ziaBTXylmzR01hhHSjBMPe4KDc8KV5x8nCcR//7Pv4F7ZJeo2\nHXBMv5koBBDHJu3LLOl+tgPHtMYR33h/sL/Qx0i/1wfgKCZVUpTQUzdQVwVkBQW5AhEQTooNoqwi\nMMMtMeoPADSb9GxUK2IQ0ZImjom3mgjmbkege//ulws/Jl3f9YUlDits9bnPfDxV4j5vpFqkByAo\n0Lnh6h5Nc6oiA/eaiUyTocKF/Q5R3uYSI/uz83zLVLONn5xPFXPXFy5fB9Dv0bFQVOqc2erFEhcw\nL19JWbmNQz2yMUUqp6YIkPPFIoC8PstZ3ftBsRaWFStWjozYLywrVqwcGTkEEnqmzkZJdG7oAphQ\n5cFgSIvRGOoJaP5NT1VTZUdM5zmZr54Y73aDQXEMIJI1buznnNDWUGlvXZmgnvIqDSPaxibDOlOz\nNI97SQt7LFJTmTFSRf5IhTI5X7hMcauOLtRT6pqnJMB9cmySkKE/omk9GNBszmlVswqLxDCNSHmP\ngVDJA+eeALDaopH86vs/4iSFROamuIAnRaf38HlyvF29RmN+4Tbh8OqKGsF2HAAXLzLKs72tFFkF\n6BbuMAqm4ZFVNPJDTzPlLxFtXqRCqF/7zd/AniL7UonTnp3jahyEhCbeaux8g4pSjGBQkucZQkee\nYvJRDbG6eUAjveLuQlrluw4TAI4azbnKqvW1AUJtCUPcPtSzKyiN2TdFLQaeeA4ATyFU16zXbnau\necHXrTuaG/bd0T65du1aqiwqinfixIlUufTeB5pJTopcJWMiVlHpT14EKml3NfN5CYx3wkTehb9O\nPkQ2CE9w8qwmua0+rI7e6ra7APpqqdfrc4cbLk9fre1MmmiouGosXvldK8m0anNdAMO+iefaNl9W\nrFixYsWKFStWrFixYsWKFStWrFixYsWKFStWrFixYsWKlaMm/w++OwUHkBXpuQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=400x100 at 0xA964E80>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next() # 返回4张图片及标签\n",
    "print(' '.join('%11s'%classes[labels[j]] for j in range(4)))\n",
    "show(tv.utils.make_grid((images+1)/2)).resize((400,100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.定义一个卷积神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "拷贝上面的 LeNet 网络，修改 self.conv1 第一个参数为 3 通道，因 CIFAR-10 是 3 通道彩图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (conv1): Conv2d (3, 6, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (conv2): Conv2d (6, 16, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (fc1): Linear(in_features=400, out_features=120)\n",
      "  (fc2): Linear(in_features=120, out_features=84)\n",
      "  (fc3): Linear(in_features=84, out_features=10)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5) \n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)  \n",
    "        self.fc1   = nn.Linear(16*5*5, 120)  \n",
    "        self.fc2   = nn.Linear(120, 84)\n",
    "        self.fc3   = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x): \n",
    "        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) \n",
    "        x = F.max_pool2d(F.relu(self.conv2(x)), 2) \n",
    "        x = x.view(x.size()[0], -1) \n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)        \n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.定义损失函数和优化器(loss 和 optimizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch import optim\n",
    "criterion = nn.CrossEntropyLoss()#交叉熵损失函数\n",
    "optimizer = optim.SGD(net.parameters(),lr = 0.001,momentum = 0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.训练网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是事情开始变得有趣的时候. 我们只需循环遍历数据迭代器, 并将输入提供给网络和优化器。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所有网络的训练流程都是类似的，不断地执行如下流程：\n",
    "\n",
    "- 1.输入数据\n",
    "- 2.前向传播+反向传播\n",
    "- 3.更新参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.181\n",
      "[1,  4000] loss: 1.855\n",
      "[1,  6000] loss: 1.670\n",
      "[1,  8000] loss: 1.554\n",
      "[1, 10000] loss: 1.515\n",
      "[1, 12000] loss: 1.479\n",
      "[2,  2000] loss: 1.408\n",
      "[2,  4000] loss: 1.359\n",
      "[2,  6000] loss: 1.365\n",
      "[2,  8000] loss: 1.325\n",
      "[2, 10000] loss: 1.311\n",
      "[2, 12000] loss: 1.290\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "t.set_num_threads(8)\n",
    "for epoch in range(2):  \n",
    "    \n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        \n",
    "        # 输入数据\n",
    "        inputs, labels = data\n",
    "        inputs, labels = Variable(inputs), Variable(labels)\n",
    "        \n",
    "        # 梯度清零\n",
    "        optimizer.zero_grad()\n",
    "        \n",
    "        # forward + backward \n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()   \n",
    "        \n",
    "        # 更新参数 \n",
    "        optimizer.step()\n",
    "        \n",
    "        # 打印log信息\n",
    "        running_loss += loss.data[0]\n",
    "        if i % 2000 == 1999: # 每2000个batch打印一下训练状态\n",
    "            print('[%d, %5d] loss: %.3f' \\\n",
    "                  % (epoch+1, i+1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.测试网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此处仅训练了 2 个 epoch（遍历完一遍数据集称为一个 epoch ），来看看网络有没有效果。将测试图片输入到网络中，计算它的 label ，然后与实际的 label 进行比较。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "实际的label:       cat     ship     ship    plane\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAABkCAIAAAAnqfEgAAA0bklEQVR4nO19WZMc6XXdqcysvbq6\nem/0ABgAg2UwxAyHo5mhRIkSJdohWrbssOWwFXaEIxzhFz/4wb9DP8ARDlNW2H6wZYclh+RNFmmS\nokmKnN2zYkCgATS6G71UV1fXmpWLH/KcW9Xd1SNRCke45e8+ALezsjK//PKrzHvuci7gxIkTJ06c\nOHHixIkTJ06cOHHixIkTJ06cOHHixImT/78kd3rTb/3TV/RZkimFIACQ87zszzAcZkqUjLhDoZAp\nccKvpEnKg3hxpujbSKOqjh8DyBcG2Z8+An0l1dGiTBlFPGyS2IADjYFbhlJy3DPR0XIaNkcbRzqR\nLtADBxnqWx2eGb2QH/3Gv7uPCdnf3+cAIu6ay02ZzJ9UfrKDpCeV8QYv+5MbvNQ7uUdO8yMlhU0g\nd05T2/tPGKTtuby8fOKj3/rWOrWYE7W/u50pw8EAwLXnrmd/NmbrmZL3OYBC3qdiW7SMgpwWSdTP\nlFo1r6/nAAQ+B+l7PMjBQTNTZmZmuGc+r6NxH1stURJmiq1b/pnj371uj98NuJxKpVKmhCG/G+mX\nUi6VdXyeqDFTmjzs13/zn/EqFm/yKz5/U/WZWqYcDbkUu+19jU2/C93XQMMtB0UAJT/QuHUr7dZp\nQ5zEJ7Yk2jI+rK7R83xMWwC5nP3e7acan9qH3yoWi5lS8Io6dRFArsDJ6e1/lCk//8u/duIgHpw4\nceLknEhwelNoL9iE7y4kCYAiaBl54IMwCE5aTzJZkAu4aWhvG71bgoQfZU9/7Yic7DVEQ51IT/rE\n19j4XooDPpvDUB9Fno4TA8jJOisV9E7WdXmBvZx1RnDnFPZa4Nsg8KY/033fn7r9zyl/NjMtp7fZ\n2CLycgASe5+mGm0qM0qvXDMzJ779Z7ewTkutwjvlpVxswy63JGEPQKnAo1XL3CHQ4W0BFLVKyrqb\nnoY9jG0fro1C3gMnAACCQGaa7DUvd/Lai4IIsuTQ7Y10IkoGI1Itfk8nyMv6MHttNBzqQjRs2RQ4\n4/4mKQcf+XM8SJ4/t9inheXlZWH1O5mSxl2dmscZptxn5CUABpo3/VwQjghoPC3gfo8/c1vSdiEG\nSjyPSpqEADwzeDVvUaQVaE+AnD0lOD9zc7y0YnlGh+WNSLwUQK7I88adGs4QZ2E5ceLk3Ih7YDlx\n4uTcyBRImApMAcPJLTnhskQoz68Ihcm0Nh+fudwKBZp5UZLXR/7kPmZM5uSn9/QYzXk0OFOPBnM/\noWm7vUcbtRPyW53OSJcUA6iVBAQ0ttkKHZ/lEi8w8eRYFXTyZfDndSFhMh3sGAj6DDT0Z5A/zdHG\niMx2Hhvi9kl2IQLmI15yYOAh1i3LnT5jcmrLnyCfMewgx1Mbviv4PH7eiwEUPYF32y5/+bBPx7bv\n876XAt7E0VDQxuB8xC1pLgAQC+EW8vyKIUEIN1l4IZY7otfjGfd3dzNlZZFAJvPH+wWuDF/HtwnM\n6+0fCC0OFWewCMBoZD+uY+Kl3B5rbLGCIXGO11Wa4akXnl3htw4PMqXWI0gMB/zNxrUSgGS2kf05\nI9xtJ/IshjYMNQk8Y6nE2R5PmO5vtvZsBdpBIl1XYmtHS7EQcMmVy4o8wCA5pz1BDCAx++lsx4iz\nsJw4cXJuxD2wnDhxcm5kCiQMElqVlrvhJSNMmO4TURyFbE6F0iJDUhZMUSrK6sqtTGm39gDs7dMI\nzwcFnU4RQCVM9VHJlA/XaaijtMB9PH4U1ggbm+0mgI2nrezPWpEHSba55fIqT7RQM6BhKWC8xoIs\n2/hUOgn3lNX650y/+nMhSp05NnyqZLQoSQCMBLc/vc8kspVV5kkZrl+aJ+QpKUaT/ORD+oxJKCgR\nL4nkSRAQyHsJgLz+9GKuhEJekMSPtWcoRXczJziv5RoNFC70qwAGusCKXAG+BQ4Nt+hKuwPCrjff\nfCtTRkKjc/XXeNiiB0DYDjmD4VohniGd1KLbgqsWxk2mQ8IICpyBizMRQB4qwutLqSrmV6/olr31\no0wJd4kNL7x4C0Bulz+KYY4xx5ou4KjPCGNJwy6mPJq3oLikooQWPB1WSgCCkeDwSEercraLh4eZ\nElx6IVN6jVkOUpg91o0oJbzYXJoC8GLFauMzDSlnYTlx4uTciHtgOXHi5NzIFEhoSCMXNKh4OQhl\nAPCEm0IZ+YUCbdR4nD92MgOzoCDKF//SX86UN7/3fQCbB3vZn10BwCiiRfpwYydT7j95kimluQuZ\ncnH1CrcUWM8RClEWassAogFt472dzUypNOYzZaPzNFMGuqKVGo3hiooD4pCI4Kwn+uko4f+l0pzP\nxIwKbuZVGqW80H5nCKB1SLP/6R6rUsozhAYLKk+xWhMLmVmxztTxjc/6p5OCHAipLi1vZSLxEICv\nMF8uJrjLK1Y7MhAhqOvXDUQo6Vc1NElkML4IoNNuZX/VKoRFnmbSKmYCBYNbCg4221TKSrkMheHC\nUQIgKNh9VxQv5kgi/Rysdq0gV0OqlZbE0z0M4x+dRfFSqyRT0qeAWU7YbZBTjVGscrclQv7eUQhg\ndP8Tjk2ek0QVQd1AF6axFSKVFj1WmnGo+i0Fjge1EgB/wD8DXjGGFzik/pbqn3JL/O7sIi9EJxp5\nFlflVSdpAsCX9yDwzlzzzsJy4sTJuZEpFtbQ42P7sMenchwNAczV+EiuqyInkGfd/KnjysrkZLpH\nr8uckW/+/n/KlKcHAwBPO9zh4SZ3WN98zOOXaGpFPs2oWp1P66BS0z58LZT08C15FQB7MpHWLl7O\nlIFsrvv3aWE1W8rlWePRri5TyQd6t5zhJbXKjDT5CQyOdDxBx7aPE1tOWVixJtXKuX29aa1yYne/\nnSntLq+oP4wBdHsqciryVnb7vFO1iswNjaQwHsyfcBU/kS1ZzFliEWcyr/rYLJdqnEiV6HbkVKPj\nncxj8nOqEZE5ZlNppfgxRgA6R5yTR5axJaPJjKNLdU6LZV298+57mfL5O5/LlMSSwuIQQCm1dELO\nZL8nnKE1E42UPhbw+CNVyA+HPUyTWJZXovy41IwJ/cpCy9XSiWaPNBvLzMwqLz/LMaSH2Zi4w+Iq\nR5tXPfM2K6ihipyuwmLpCiNaeVXRDQSYqjNVAOERr2KoyQnK8pdrBQYLtPVyeZmiKU3FGS0fX4Zb\nlMsDyHlKEsSZdW/OwnLixMm5EffAcuLEybmRKZBwt0d7rBk1MuXb/+vbAG7foGPvl+4Ql81ZsbWl\nooiSwVNFjhU9yLeLBw+ZE9TsFQGkFfrCvZqyP+aPMqWkDI5QaTKh1dnMced6jaN6urWVKe1WE0Bd\nxnBJPtdHAoD5Ou3nna2HmVJ7yjNeqKt8x7MIgDE6HJNuz6gsZGPLtDYuMF88AaYYbZBhQy859s6w\nIiEDZh0hGvO+l+XKHageYkuQcOeASsbTMBLe6x0RDu/I+77xhNP1wo1rmfLclYscrdKIxv5+o9PK\nTfw7Ubrhne2I9+U4TwSUPDkQ+odtABBKSkUJ4Iu2oWCEazaBI4YRYsNWsT4ae/dDAN0uE4KePuWe\n1XpNJxI21EyGHe5TUvhot9XKlLfeJ0isFn0A169xugJB0WGPi6csFpBkyLURKw4QG9YZtDFVLKXO\nuKjGmYz6SLAxL5RdvPcpj/rGdzIlev2L+lYRQJoSkxYEHgfglda2eIG+mCSSqiqWUsVwRvzWzEKD\np36yDwAdLqf8Ct1HeEyAGWiSB7ucN1/em+QmM7MG4njwcubvzwEItFzTs6M+zsJy4sTJuRH3wHLi\nxMm5kWmlOY2rmdLb4+NsVFgC0BRU7IUEWfWCUmDGoTRDQ7RFBxFB1q5M+N22Ig6NBQBzy4zidRJa\ny4sqxLEIYKh8j4Gq0vuKAT2riEZPGHAnHAAIlJbVaurEGmRfBq1f4Imethmg3GoTvzy7KGx7hnXa\n6jNKVauI1zAwFGxsENrbwiKGBMdEesffGaeyura3mIM2P0/sXC7x0oYDXlqlyC2rS0TrGd9xt8fL\nqcoIDwdia9OFdcQ2F42LjRRaGqeA2UeTVzNJDoGzpGRsedrJIGExjQHUFGadNW48pY8VhY9KBo+E\nxD1d+5hmNxYXdjsEMFPl9jnN24MNUjPff0zlk3t/mCmtvVamdAYcWy98P1MCKLuq1wZw5yYpjP/G\nX/tapjyjFTgscbSDLscfdnmieqqkpP4RpkneV1mMJsHChYkcL8ZAWTvg8aMN5hjWC/ylHG3yjGFp\nFkAqwsvcFlMaq89wuYZ14S9wkZQ7Sh9rcZADVU1Fe3QgFAYhgKhNuF1sMnw/6guPlwmZWw8Y6y+U\nCQlnLjCC6SsXLFXi1RApgEgLL0zOxITOwnLixMm5EffAcuLEybmRKZDw1ksMNGz8gHn9tdklAK9/\nidsr3nqmhIqPGBrKiV8tRiNTZoT43nmPEY1agyb0M1fuAEhl0hcMYA5YrNNTZYBnTTs04A/fezdT\nZkvcUlGQsVapAdjcpm1sBRueQOK80gVbB7R4D5pU7m/R1l1bZqJdoFGdkKDOq4iF6UaKkEJBHFMs\n9GPVIYaP0uMppOPooRSrIzEOAMO2DdXZjFQ9DyGLSm0GE5Aw5xv/gTqXlHXLrE+M4rjjGM2pwWSQ\nP3/y88/ChI/X1zVIzuRRm8smHg0BPFHd1YHoIrod4v3lBaK5WpUowlfScmiUhAXx8+n+dgc9AAMb\ntLjkH21yXT3YYKi0F4quQ4FjVHh8IxW3aq3tR58A2BTm+qPvfDdTbt98LlOWGsRH/U6LI1F7m9Ft\nMpR0RLl3QorCdKnuIIw0RZDZk9JRlVvn1c9nSj34KV7REed25Gcs6UZEqQhjmSfqxqK7kCtgJI6E\nvFZyXwz6lsfZj2MAvQ7PUtXRBtqzqJ/h/AxZQGI9HDpaclDyankk/r9cbuJCMTp7OTkLy4kTJ+dG\n3APLiRMn50amQMLKLMHOs9cYEMmKpZ69yoaXiyNihtZ9Jl6OBFLiiGjr9V/4m5ly+dqrmXL1xfVM\nefNtorm52iqAzR0a6oHYvEoKaakmHB3l9R0e0MaeU9dMsx2tFnxxcRHAQFX2e8oAtFKyWk1RSGWH\nhgo53X+8kSnLcwQaNy4qNe64fP1f/Rse1hJHZfrOqEfm9auEw6+99ILOyK9bcmkWiUsNv8g+jzSl\nFuQqFIUajABDWY4Lc8pZtQ5shQIm2AKQl+muoraWAqMtMa4dHbYyZWQ5sQrwLShv8Mb1awDyVqFm\n3TknQOMJ+c73fpApRtVvRZG9QQfA+vYT7UCxWZpT5nBVgdGizpNXKmmgvEdPbb56gxBAoLasqeDw\nVlPE5wrfVmoNnVMEJB2r9eOZrAS1XqsD+OlXX8r+7B42tQNx96NHnNJ79+7xIwXPH+5zSvsKIJ6Q\napXrLdKVjmK7C0RzRpeSEwour3B+2urqunvI0eZ8H0CoZmUFC8C1uGck5F9Ujndba7JkHQ2MLlE+\njWFWnaq2DId9zZvwa0V1jjMXL2WKbx6GcWc53eBxPnIKjNdTcnbmqLOwnDhxcm5kioXlF+k2e7L9\nYaZ84dXXAFRnaZj4R8biIONCr9wfb9Ab93NzTOZChQUfM1VVPwQ8frlQwUQxhPmS19bo8P7wHot4\nCnJJtuVTvHqJ1t/N51lV32yqg0g9B2Bzm4knnt4SDfFhtcSUZDZXudLIlL7K0D99pPKgwvRn+kD+\n7LCv8nSZM51DXbq2xLef57dSI/YVL22hjAlTZUx2LFNrdp4pPGMiB+t3YvwNskmtACrhvzzausqh\nnuxwWpr7tFX7fdWRDPW2FKODUQtcvESf9OVLFwFUC7ZsLHRwpoX1zl2euqISDbvRg6gHoDHP3DG7\ny6GMmp2O5laXXCtx7UUyFT1rrarWSl5QBVDoqhvoiC78ZrN5Yth2a0MRRbR1RqsGu7zIZTM/fwET\nFT/7B5zJhQbP++rnuRQ3NmmntwecqI9UuXKaTJwXKD96eYYX2FHKYaBVGltClipaPC2nRMliOV+x\nCM/HhLN8JPKSstomGbwwW9V87bHm1nrwRCqJy5dzABKlvBnJnfE65CM1NrZMQ323FNsql+VpnNXI\nYeJ25M5eTs7CcuLEybkR98By4sTJuZEpkDBfordyMDBoMAKQVyFLpWquUDr/iqJbnQloQ/7WP/8X\nmfKrf/ef8LAqUygUzYaPAFy99kz2506TzteBHJ+rywQLRlw7VCuUa9cZAXjuOrHh4dvsd9I96gBo\ni/Q2iszFSyO/oXyZpEWre3ZO3Axy1fseL2RjcwfT5O/8rV/jkOSirp7qE1kWdDLO4XZbbAoigcgH\nJQCB8llS2ed9ZS2liXLQhCby8u4HZsznrdDnGKK0fJaBaA+MsWCu0ciUWCyAJZ/jb+0T9Ww8Wc+U\n64q3+F6ACdzqC6V+RmnOkSXCmatb2LDilQBcvMQ8pnDIkext8yt7TQZkVpfJBldapCu32drXUblz\nfY64tVScAzAQy0Yv4pyXKrrvEe/7uLerHPbm3Ih6HO3rP3UnU24+uwZgENJr/uDH/Mq9Tz7IlJ95\n7cVMuXSZS/rRe4xKmb88OaPopKBsr4LyChPR3ZUVMInEgHjUVutTEYSUZolbV6qKEaUJjrUsFQOi\nbBRf3oNxZOaUpCoPMkgY+ykmGBA9KQVDnzrsUOSLRtMS6NpjTfu49VQSYKJwzSgqT4uzsJw4cXJu\nxD2wnDhxcm5kijWYU3FAT9Bs0OsDyKu95dG+akRUiJMHQcSFBi3DTz9kKsrmBhX0iPgePl7PlC+s\nfhHAM88yJri2Q6V7jzvMFxqZUm8QG96//4AnWqPV3RLIGsl8fbqzDyAxqgRZvD3F9bxTDAxW1mMB\nrIJ4zsL9bUyTRMloYxtbH9UKrJgplzhjfdG29UacuvX7vMZCoQzg8lUWsj94zPr73/uv38iUSNGc\nko5WMUVAslEn2GnMEhF84QsvAVhaZHnEcxc5XV5OnIKy1C0SZGGj/jLxxdqFBpVn2Kwo45DrKbtn\njILPfvHlRcy/uMwxWOB1b+8xgE5XBAYqzbBksYaYyNeu3ciU+iyvqL5IkLin6HAi7JxVofTUKLSn\ncFsYKrNJJAQFY3kMeMsKYmpfXuWULs1RKeU9AEsCnnWlL+0/JO57+OP1TFlV3LO1/X0edp6jDc/A\nX4F4C3w1iC3pZ9jaYXCz2SFlwu4Wo5BzM0yZvPMC0ai1K874D0aKx1lU2parNSUwV0NuDPC5czwO\nR1o8L/vIvqtqm/F31VBHZ7QlZzvnlRmXt2BgCgCeEG58dlqfs7CcOHFybsQ9sJw4cXJuZJqBarUm\nCg1cWFzABBL55rtEeXMKAN2Yp7FXKljYhfhrd4cgLhm2MuXydVJ8+aUigEqdRv7iClNM91VC0VJw\n0PgBl5eZRRkIn1oJjpXvZ9FAK22xDMOBIoyRWMYXBSvgqQmryMxKinFEYhM8Ib/7e3/AsYn32lPy\nXk3h1BkhtSs3eGlLC8RHCxdYtTO/uAygJDaC1kfEF+9//ChT+sKvBibsvsyIrv765SuZ8qUvvsLj\nV2cAVH3V0MjEDjVdkdpk9awiRw1ByzpsoyG+/G02RtvbawIoq45kZZUTWKko+/eUNATnJwqhRMIH\nD0BT5HnttlIljfNbKO/RBgdQb/O79dkGdxYdXE/5rshFmKwvqfB2lCpWxGMAhzNZ0z6B2pdeWuC1\nG1tDt90CEAlgGp/9VcHVjz76cabcvPW8js/Z3lQqaUmFVifE4Jh1BkiE1I6ULL27S+/EQZNH++S9\nH2bKx+8Se16/ziKwK9dvA5hbFAuFQJaxSxpPv6Ev3+hGtC0Y9yI41mtuoh2sgo/a08LFpzsNm4yD\nj2POkuws9lOd3lsPzsJy4sTJOZJpeVh6WNZretPOlDHRUrQtsqC9Fp93i3UepypPZKQOKOvK5VmZ\nb2TKs3oJZJkyP3zzo+zPJ1v0ns7UaHPllYHywaePNDorPVG6hx7GnS7fvXML8wAi7bCtGp1anQMI\n5HSv6L1qRSEImfiT9DiY1eXpxc8/evt/Z0pZNEzDkJ71vJzKP/3Tr2fKwyeki92n2xR3PscyjkK5\nBKA3pHWQlxn7yiukOhqoGap5iG9cY9nT58SytLbIS6tXaPskgxDA4232B905EAf0Hrd0O/RJt1Qc\nHo7UKV4nsnJrq8EajSIAlQbn5A54FbOz02cJQHC8JhkTL8msXNnCI4FqtmxLocTDLi7R61+r8QJL\nCjgEGmSQ543IctBSFYJY36NZ5aB51u1JnFCB1bgMlZqnMus04rTE8RBAqNKTvi6nMsO0xIfb9I5/\ncJ/Wt9X3jFT2lLbPTHrKxEyVkvjBn5e9dv02oxa9I973D95i7uFbb9DC+s531jPlww/fB3Dr9svZ\nnzdu3c6UxlwjU2w5+f5Jw0qVXZNbtACSGBNZhCZWrBPLmE/GKWBnypgVLudjooouSs7M63MWlhMn\nTs6NuAeWEydOzo1MY2uQg+3C8gXt5AFIlLBz4SIhyY821zOlBTVrCegmbyzSLTdbp6Fu+ThXBAlr\nswsA/uVv/uvsz56O3+6z6qKn8hrzN682RJXV5Km7RTsRvaQffbwJYEe0BObKbXgcZL0haCAWpCCk\nMR/0mAY1XxGOKE03aXcfE6XOy8a+eJEe6Bc+z2qhvGDF++/8MccvO78mkqOdvS0A1TphxUKdO/z1\nr/08B6kcp9lZ7rO4wOybZpMT9eAh6acPW4Sl7cMjAEeKWhyIhqmpfieRQhAFebgLgvNGYlGvc/xW\nxzO3PAOgaFC6LGoBUVaclnmRTSehebh5oiTqAyiIZWF5ZS1Tcqo9KiiryMBpSZUrvgZptBbG/pzl\nBFmiWa+rQhxjgJI/PhU27B1yJp+scyabyhFqqKvrykIDQEl0EeYYTgOi+EClP3tqZnNxjUuupmtv\nD6a7k61kx1oRp55tkWNbmVmNBdYn/dxXuOSuX+dP8rvf+lam3F/fANB7WywUYih58SW6Gi5d4kEC\nRWbiyBi9rZBI12jO9DTFRD9gIxCx5k/GdTXuA2ttay29y+qTxk53D0CSnsSVp8VZWE6cODk34h5Y\nTpw4OTcyBRIa8W59jsZ8FAcAijJ9b4r590dvEFsdFljNn4Dm98pFHvmDDxm/+Nmv/MNM+b44c7vd\nNoCRAnM7WydDgZ1I8SNht4ZH7PZMmdjncJc2fOQ3MmV1pYEJa9YqcgbiQe70xMWsjqrRgGVDywFD\njWuiUR5GVs9xTJ7cZY1+W7GnX/3lf5wpX/vaVzPlD7/JaNGy+CGW1XW1rFSgUi4BsCI+3xkpJSVD\nRbLGDRZFSmPZ/oTDfrTDNKVQ7XOCUhXAzAyzfpYFZEbhyfhOXkjQSuRNmZlhkK5en9FHOQAdEfI+\nfcp7Z3N7WioCSpGnsJqSzhr1ZQDJmAaS96Vc4+lSq+oQbElSbTlFs5uaggRApBsXxRxbe19k3Hbt\ngoSdQwZPN9XCZ3Ve1U5Vpv5lPZwSQdFIh7Fw5DMCWbduMtPw5Reo3L3PMPHb732EaZITEvTEZeyJ\n+CTvW6GMsqIUxfMUGL1xk8TNiX4ym9v/AUBzj+A0UXHY0ycfZ8pzNxg3vP05fnd5RS4g/dKjkfia\nlcwYpzEm7ssUamzh7tMkfGOWx/HF2pdSYIwwxxU/p8RZWE6cODk34h5YTpw4OTcyBRJWa4Qtc4uM\ncWQ97weqXynVZC0rePToEYsGvvw62c4GHSVn1mlsb22wnuDTu3d52CjEuDEHOuJdqM8TilppTkMp\nrLdu8fg/fIeW7Vsfr/PUX/mVTMmIBu/f+/TEQSzXdKDqisurRHMl5VvOzwuMiJIwCqfnsA16jLu9\n+HkWyv/SV38pUxbUKfZnv6hIn6DHjCqK6ppkv1DCRDdQi1sZS7c1CqrLUE9EDHFNs7F8kXHJ5gHn\ncKbRADASWskJLxlvt4WlrOlLR9G0VC1SjFb88RYTXgf9HoCRUHasEo1K9czSHIPktQrn1vDdzu4+\ngLYyVy1f9PotJkYa3bufNzRExXBxqIYtPVHr9Yc9AJHyeD2VHCVD7lkTCjaa/3JBJV+KfzXkE5gV\nyXo4HALoaZBGN+ipoGROcL4iisqNxyy0EqrD556/gWlihP3+WJErwOqIrHQmORZcAxAK6V+8dCVT\nrly5CuANaycszsKdnRYVocWPPmIXq6tXObbnnqOyssJU1RklxyKXBzBQW9ZYv4684LyFAi1x1Cpz\nUuOxHIutzxwmi4Qcp7sTJ07+Aoh7YDlx4uTcyBRImETEULPzREzdfgygJ3xhUaTLl0lCcPd9orzD\nnpIDq4wkXiZhN9Y/Wc+UzU3ii5/50msAusId9TXmDc6vMbbyqEnc11NL1UKVNvzsMo//Sp1j2N1j\nDGh9fQNAR7X7LbWWXF6i2V8HjeErNWK35brI0UEcESrGVD2DS+za8y9nyq//g3/EQcYEGp/cY8wu\nyYnEQpHEkTLimi3Vuyc9ALG6ZipGhATEL0dtFuv7T2n2byondihUkigdsaoo5P1PNwA8eKTAq1Ix\nFxYX9F2l6aqR6p4mEEogHDMdSqlVygAaJZ7FOAX7nemxVEzkozb3OOz7YmqPkiGARoOlo2trpBYI\nVao2Cgknk5RDaguJ9/rG5DHUaIWh8h4mcF9J3BJl5YuaTyBRuK0qBkdDZAVV2Nlqz8KpRi6Y80/G\n7Eai4d/YZ+Vmr9vKFCuoXL1wEdPEF1wyBToRcgrsjtMsT9X66SOrQKzP1IHJuk0pRrGvyHu7yfvy\n9h7x4wfv/ihT5lX/u7rKn9vq2hUApZLynBcYWFxaoRvH0nftlkXyMFjr1nHiqOWdJh4mWBzSM5jv\n4SwsJ06cnCOZYmEdiVKgLA/xcBBCnS0wkZi/NM/X9V15zneafAHu6Z3cmOGj9/aLdEnef8iclIzA\nypziN2/Qc3zjKq2y9c1WpmSl5wD294xfQd1f9G7ceJ/m2NZ+G0BOIQJfFf+rF2m4XdFT+rJ8+cZ+\nNZQplyR5DXJ6LcWv/f2/xwGs8p357vuMKpgHNBy3CVG9hVy25lbM+prE9m6xHp/jVwm3hHo37u3R\ngrNUI7OEGmKJylzRzX01Rpe/dm9PjULFFxzJ6W7FOtY5pqK26SUlH3mRDyC0jjRqf1JWatVpaal+\naGuThm1VlT3Pv/AigAWxklUU+hiI3fjggGl3xiTRE61CRXlqs3Wu0qp61pcLeQCBbKVYTvcsyANg\nJKLqgXV2GXP+iqVXeEKZbQj8AoBULVcHQyr7uzQY9/YZX7JqMGPCMF6QokiNT0guNQuLW8xFnZOp\nYtwGExUxVMzn3e/QHt/e2gKwtUWjqX2oCjkZhjMK+9RklJXEO2IUchvbXNJ319kNdzCIAUQxD7K4\nRFR05w7r7W7eYDLa0hJva32WkZNimU+AFFot+oHQpjfabud0d+LEyV8AcQ8sJ06cnBuZAgnv36P5\nd1nJ+yUvBJAIRARmQ5qHT07lmkiBn3+eqTR/+Af/JVN6LVqnlQX6Vu9t7AC4eJH+vKu3SO9bFCR5\n7ln2kmk1W5ny4Yf07icCIxsHtPPbfdn5cRFAu0WkubxKG/VRk1vmLzYyZV+QB+qV0pK/OVVDoEEy\nxDR55503MuW9997JFE+GrifT2koczOcKGCMCjeqg4GFiJgtjogLx+SpFy0v5Ub1IL7UnAozIt2tX\n+lgKAAUhkUgsgL2WRRV0XVasIxQaynsdqW1SRzioUggALIvALxAuK5xZSoH5Zd7ueWEEYwHOFtKR\nCqSOOup4WsxraOLVkxv+mRVGToq6d771jlUxVnfQBzBQsKIlXGmQbTDgGW/fJjdeXhmFE3zBJ1v4\nDLtHADa26dDY2aWvOhSUNnIRyywryFXS0TV+4xvfwFRRMldiOVaR6mOEFq0PVM5X0pMglS83/Ltv\nvckztnYALCiJ7PEWr72uZLG8VrgF2epKPQvEjlIITnpgOl4HwH6LgZr1ByxQax1wWt56QwtYpJiX\nL9MVsyZa8Atr/EmurXBLtTYHIFcW5YN3Zlqfs7CcOHFybsQ9sJw4cXJuZAokfOdTBqEu3yEleYIu\ngJzFy2S1ttXPo9VioGRh/uVM+ZWv/WKmvPx5Wt2//R9/J1NyKvWenZ0D8Mwao2zGue5HDBLNr3J4\na9eICFrCIG+/806mbHXEvZ2nrTu7ughg8Tr/9AXHYpnUd9UI59620rv03LY6la6uNUpsir6FCfmj\nb/+PTOmJGq2Q52HLqkGx6fVTVfZbG8u8QcIcgFLxJMouiF8hUOpZSW1lDWgodgevZC8ehV3CEMBA\nZTEjBcgs88heVcF4i65UkHy2RujRqHBLreIDKAT8Sl4pQrl4OnDGRGcUS9oKBHuTDOxYDYoyniz1\nrSTcN+hy/P1DLrm+uq8GBZtSEcXFEYBPPiRaebi+zpEI+BuSWrvANKJ5kSP2BetMOZA7otnaB9AL\nLf/L6EC4xXr62s2oCFttqbZpW7UyJ2QkhG4h5lwk2gZDi9o5VQqVhRQ7Cg4O+jzOrZsvAHjl5dey\nP994jy0I/vhHzLE6FKl/rLWxvMqQ35e//OVMCXTL1tUs9vs/+D6Az71ALv+6XEA7uq5tJQlaTHZV\nJBBXr17hGRUT7x4d6opSAHm1sx2c4hQxcRaWEydOzo24B5YTJ07OjUyBhHfbBCN7sagL8gMAXij7\nLRH/lrLs1tRQ88tfYqSvlGfc6uqzLPj+q3/71zPl3//Of86U3e1DAJuHRhvA/qwFWbzNHpV7YoOA\nIjLpIpHm3DJHa2AnSxlNSrZdJGRCsoeRijbGiZG0rbsezfuR4l5pMt06XVmiMbzVZ/wljluZUlez\nzEClOe091moctWmHj2KLfw0xtRZBHGaFMufWMG805njj+6ai1q1VkazHWVauHVb8ATkBqJJwX1mT\nMK8U3EtSLq4xJCcgjsHgCICndrOBMEmjXj45fsndTz7MlBfuEEfYtGej8xSaS1TDYbCiJwb6QY8R\nagsXWqPca6IzX15mgmJW+REoVjsr9kQ7ryXlWvLnRx9/kilGUGHOAcuizBZYR0mhfXEWGiQM1avN\nOOMfPeXasAzS+IwGVuO2o2P2dP5vJHlCzEgEEi2oWVY4+Mtf+ao+8TDB137zZbp3XvwpKgqujuff\negVcu8bM7UAzduUGSf7WLt8CUC7zdlufARu/9Rkw3Le8xNRxo3zwhZQ9eWniZAhgpCtNctNnCc7C\ncuLEyTmSKRbWJ2qP+rvfpaPuC88uAlgtqHm3XiAXVvnsvLDIl9hz11TbqRKKrV0+cb/+b2lYvfU2\nX7lZxc9E6YucpnLXxSUeNjY3s/zl1ic18tSI/PilDELrPmJ9t2kn+LI7UtUMR7LO8lY6Y0lJ4fQq\ngXSkEvEq30JH1jUz5kv4+du0KZILtLl29zgbO6Lr7bRiTLylYyVSJRGPVg34Xnr+JfJQb8q5uyt/\nfz88+drPSn+KKq6q6pY1qpyuJTX7WV3jTbyu2uOVEqeuo8SofdXHZh7uquIAtRm+aRcW5nCGjJT0\nNOhwtJ6sy8yKMHos63h67y7tnSMLaOidnFdQwhrfJ1aqbWW9cQpgcYGDNHuqN7aeqDx69PjEPqZY\np/ieCrAPWy0A3T21yw1s2Lwco+jqKk0pUo1RPO7tPt126PdpQvpKHwtEBh3qpxQp9zDSldphjd3M\nqneiOMJEM5tQ1uva5au6QhWHSfFEmvbgETPX+qGhFrFmz16dPN3BofpOaTaq9Su6UNX5H/LSNp82\nNVqOsqj6uayyKFdTdfrBmU2YnIXlxImTcyPugeXEiZNzI1MgYUd22jfeYh3Mp/fuA/grr7Ig+7k1\ngpQH90lD/POvkau3lKer+EiI7Lf/G/M+3v6Axfq9SAUxQQmAJzfwuJeknO6pZz45fjQUZBtpS07J\nNUMdNjM3g+AkuKtUZH/KtI4NQ2ge7ESR2mQWlB12QvY3Wcgej2i+9mXt96zHqjpfLolAKj8kZCuL\nYKHvpwDS1ICxsIP8jr0+weOXXyfAvHObpMyPHjE7Zu+ATn0rE8kc2oabSjrdkiBVQ8xZsc64vcej\nfSJeJMjnWl8mvKrM1gFUZvjdebFr1eR8PS1l3YhQiMxCHFnQxpMzuSDcak16LDJQk1PZU2ZQRRcS\nKWfn7sek62g39wG0VA2TWLtcHS3QkiiJ5MD4LnrC9bsi7TJI6HsBgDmth1B79pQSFokEIhkDwJO0\nCrncdBPh29/+nxx8RMLiipKSElE/G/nHGIQmlhopwmgLESQRAE9IbSBwl4z5sKz+RlGXBmMstZqu\nUT147Dv00Ht2B5UEZwmGenpY0MMbE4+cuvZxUmAMAFUdRIGs0+IsLCdOnJwbcQ8sJ06cnBuZAgkX\nFmkZNptEJVutAwDfU6OaePSs9qXVtyQSO/i02H/4Bin3fv+b38uUYVLVOcVq4B17XMaWYyVD0Zqh\nGieslddYjCZnBSXGkeD5mMj1mDH227H5OjpxtEQkCmZar64S49TrVN7AMVlV4G/jEbFhNLTsGCoP\nFO06VJ6UXXBX6V3daAQgiQ0SiodaIGI4IOJ467v/PVN+scoruqMr6os+wUJmWR3VwCJcAhE7exzt\nQxEW7/UY9hoIHpUEAOeVXlea5fj9cgEChgCKwpU5f8pC4iVrkAZGPNVmZaMd6AItx6pkeTpSLMAX\nNulYeGw0x8ZZbKHeoICJkqygZEfjkELh/Y4oPSxuaB1hLTZc0vhH/RDASIwCFpC1AJ/5NCxzKlKi\nYhob7J0edC5phURCgoFqwoLCrEaiuKQ5T8b83epVYyCRa83CiOHx7ZOXaI2UbA9ROap3VDhQ6dXx\n32ykJrgGzC030NNoPZzEjyZhx259BGCgz0vBHs4QZ2E5ceLk3Ih7YDlx4uTcyBRLPpAdmy+IQmxQ\nBPBgx+okmPn5C6+Qpa/cYEF2W5zov/HHhFAD2agjGfxFMXtlJrTlTJr4MiZPW8/F00jQdlYtTrlU\nxkQmm5GyH6nhipVHDAVSZhus6li9QKUmHNETI8UJuXyTJGTtLiFVd8PsWHG/Ceg1daKCqmpChQXj\njLHbYLAdIrW4Erfce4/x1sdHnMklT+1XlS4Yy+rueAmA7ZRo5Z6ikxtiBegZAcNl1uivXiGbWkm1\nLDCgJ8q9Wq0GoKIonqfE1PSM4BeAtpg8ekoc3dkUB8MgBBArRdYoJUaCbHZd1kI0L6KIcRRYit3x\nbMKMm2HQURxZRAtHh1QsNludUVKxJjAdKTAtFsMsr/VQ3YYMCcbKyTRi+OTU3TSCitwYsh0TyxPu\ndBjwrcjFYceyTsAWCgwjG5syLT2LG0YAQmPpEPeDJZ2Ow4WG2cdI08alORT+zb5lZXATFWXxCcUa\nqXqnfsf2USAgORpFAHpzXFcXLs7gDHEWlhMnTs6NuAeWEydOzo1MbaRqPT5lKwYlAGFEu3ynQ/vz\nzY8ZsvmVHm28o5QAarNJpaQgXdSzHDYRhFcqAAIZq1Yfn9OoLKxgMcFUHAbGhGfFZZ2Qww6jLgQM\nMVH+bgCwO6ChWxMSnFtmPZ2RiH/8gCHRfGK27jGpNxhKW1phKG1LkNAsYKvMH8pOtp5RsXo3xTgJ\nH04M2w43Egbp7jGtzis2MsUX68CmTvQOhgDuCUB1apy32kUW/S2qbe2irr2o5MwxF5+gTVEM9H7g\nA/CtyaiF87TltGw//FQHO1kBl0XTAjG4Ww/OnHUzzRMWVUSOmDuFXyIVhHYiRRKHEYBElXFjAjz1\n+yoUGYlbfoaT0BVcbR9QsaZn6TgKmcMEgZ+xOKTpyTtl2DBvRAu6y73edA/DxiPSDX66zfNWldQa\nCEVG45XFGYv1UaKgc36chj2CKgoB6NLHLgZrEGtd+8YxR9tH99dmO2OkSOKT8VBPvo6cGErG5PRa\nRafmCR3l9MZzFQDPvMQmEnUlFJwWZ2E5ceLk3Mi09JkxZY96cnh5AIn1mJSZs77D18XXf5utcb76\nlVcz5f4mrYDeONdJvnxVV/iFAoCK3pkF2Up9FVWYvzyVcZQv8dS+3vnmoLUt2aO9b3k6uhzboSHj\naGGVsYLdPdaRt1SV0npEu+D6NVW3H5eSOtYUdTn2covlr5XfHFHu5JSOi/azneztY/vpLZdK6egt\n95Fe8rNqqPPxgKzW74tdulmvAJi/xMGvXSErWWON116ocPzWhHWksQWya3wpgd722Rt1bCLl7AV7\n5pvPT5SmFJu7VzZLdjRL2EntLc3vDsW8HIkbI9GcTvAfUMzpnnUVNesgkKkVaxWViqpYUu3RwR7t\nmq5iLHmtdt+6ew6HmOhhYybweBK0kq3jaUlLriPaiV73ENPEg1aRLYTYiKRlAdkk+5pJGVDWHtWK\nsbI5tilNlftmk5sadDCCCuvBI/7uWJ+NzJTz8wBS61Rk5F1mnVnb1/H8KI6h8EgkMuu6mEIuvngT\nQJDj7WjdfR9niLOwnDhxcm7EPbCcOHFybmQKJJxXU0mrmehGIYCCOi9aKoenRK3v/PC9TFnfpBv+\nsEcbuykPvTJCUBUYySoMijqI4Y6S/OW+zHL7yGzUSEAvZ749mbhxOMJEBkpZSHNxntQC80tEgqHA\nwlB1/H1j7xWgyLpynhYrlO+qWH+mwRMNugQyxv0QyyqOxwa/xm8W9HFJhX1SJUN1lWLzXXFVPxKF\n9H5FuUgrzA5bvbgE4NoiowoLCi94mvyuAKDVQwTywtaMOVo7B0qdK5UrAIqa0rzIOT5DzNU9ZgFW\n+lOa5ACkikSMkaa+ay722Nz8QqnFohwLWiTG+pDy4OZv1u1Q1CJU+lhP6UXd4zUiAHIFHnagPMFs\n/FoyY0xvkNC2GBtEGvLUB/vE7KPwjOVkpJXaYSSsbh9BxTqWg5gIf3maWyPqS9IIkzBcnpmCrt3w\nZZIeQ+iYyMMajeSrNy97mmKiFe6YhcI8C6nc/7mTP9WRqC7nbpKC+eJVlvQNnu4A+LH4NsoipDwt\nzsJy4sTJuRH3wHLixMm5kSmQcCgQpE4rGCYjAHlxM0RmFRv/gUJm65vkALA6+yg0y9a646hZaa+D\nidiKlexUraFLhdjQk8FZMP42oRWrvN9tijEaESZKN+aU1LG60KCyykhZS9it3WI9REfdTRrqfLO3\nM71wPFQxhF+gxTu3xBON1H80UrhwZJE4494WJMyuzDJ3cqeCg1BVRyDeu1FZpS2zHORzsysaNqtq\navUAwIygYlGVRgOr6rB4pdHaiT9vjBY0hrwgeRZpzWtPS8hKz6AqBzAIrfTfIlbH0nw8XaCRu9uS\nmIB7wiCWPWSwKzm5wLJamZHSCX2t55FwX6zDVrUUDQl6RpLRV7HL8T43yal4riVkBQLINi3Np/w5\njIaM3uZOQn+JBQBF5+ApXphXTA3x+IfHnRV5H5M2mIshzQEoCdg26mK4twvRsM25YT+Zgm732Pmj\n72WRRPtK58hi8Tqs7mbbgs6LPPXlmzczZX6eDoqNj9goa//efUxknJUKZ02Ts7CcOHFyfsQ9sJw4\ncXJuZCokpDFcFAbJCvsTxS9yFqSwou2xchIJpslJ0z0dl3onmLD/Dw6I6ZpiSa/XWJAxK4BW92hM\nJurtGSVDXYkgQMkHMBTlmDGIB7KWo566MPW4T6fF7luJyAyM7XsQTH+mB0oTbSwQnNbESB0PxWom\nKGgNoNIxmZlRC3iYQCLWm9aI0AJBzrKyEOs1VZbUSO1WE1NFTbA6s+pDcfJ1NNqeAQHjNRcrQEGI\nzACgAbEx/kpTAKGK7AsFKUo1PC35ovE1KnPYPAmehwmmh3FwcJxmezKwCEUSLQJrlWSRQloZsX1f\nSDDuq5hGUcKqvlKeZeDY+OdGKtvyToE3onWL/I4bnlKrCq522/QwtJUvaojZ7jtUlcLt5mcx8nWV\nSKXim/RVkWOKDXLM22d1NrkUE5yIvaCtAYxBob4rX42mJQjtbnqnvnVMIo3Nzmvp5fVlFYHduqZj\n8UQf//AHmTLc4e/Oj2NMVAslZ7SbhbOwnDhx4sSJEydOnDhx4sSJEydOnDhx4sSJEydOnDhx4sSJ\nEyf/z8r/Ab+8NWulkQjIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.Image.Image image mode=RGB size=400x100 at 0x7DD1EF0>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next() \n",
    "# 一个batch返回4张图片\n",
    "print('实际的label: ', ' '.join('%08s'%classes[labels[j]] for j in range(4)))\n",
    "show(tv.utils.make_grid(images / 2 - 0.5)).resize((400,100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接着计算网络预测的 label ："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "预测结果:    cat plane  ship plane\n"
     ]
    }
   ],
   "source": [
    "# 计算图片在每个类别上的分数\n",
    "outputs = net(Variable(images))\n",
    "# 得分最高的那个类\n",
    "_, predicted = t.max(outputs.data, 1)\n",
    "\n",
    "print('预测结果: ', ' '.join('%5s'\\\n",
    "            % classes[predicted[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "已经可以看出效果，准确率 50%，但这只是一部分的图片，再来看看在整个测试集上的效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000张测试集中的准确率为: 51 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0 # 预测正确的图片数\n",
    "total = 0 # 总共的图片数\n",
    "for data in testloader:\n",
    "    images, labels = data\n",
    "    outputs = net(Variable(images))\n",
    "    _, predicted = t.max(outputs.data, 1)\n",
    "    total += labels.size(0)\n",
    "    correct += (predicted == labels).sum()\n",
    "\n",
    "print('10000张测试集中的准确率为: %d %%' % (100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练的准确率远比随机猜测(准确率 10% )好，证明网络确实学到了东西。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 在GPU训练\n",
    "就像之前把 Tensor 从 CPU 转到 GPU 一样，模型也可以类似地从 CPU 转到 GPU 。\n",
    "\n",
    "请记住，您必须将输入和目标每一步都发送到 GPU。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if t.cuda.is_available():\n",
    "    net.cuda()\n",
    "    images = images.cuda()\n",
    "    labels = labels.cuda()\n",
    "    output = net(Variable(images))\n",
    "    loss = criterion(output,Variable(labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果发现在 GPU 上并没有比 CPU 提速很多，实际上是因为网络比较小，GPU 没有完全发挥自己的真正实力。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 在多个GPU上进行训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你希望使用所有 GPU 来看更多的 MASSIVE 加速, 请查看可选[Optional: Data Parallelism](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/data_parallel_tutorial.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "\n",
    "对PyTorch的基础介绍至此结束。总结一下，本节主要包含以下内容。\n",
    "\n",
    "- 1.Tensor: 类似 Numpy 数组的数据结构，与 Numpy 接口类似，可方便地互相转换。\n",
    "- 2.autograd/Variable: Variable 封装了 Tensor ，并提供自动求导功能。\n",
    "- 3.nn: 专门为神经网络设计的接口，提供了很多有用的功能(神经网络层，损失函数，优化器等)。\n",
    "- 4.神经网络训练: 以 CIFAR-10 分类为例演示了神经网络的训练流程，包括数据加载、网络搭建、训练及测试。\n",
    "\n",
    "通过本节的学习，相信读者可以体会出 PyTorch 具有接口简单、使用灵活等特点。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 我下一步去哪里？\n",
    "\n",
    "- [训练神经网络玩电子游戏](http://pytorch.apachecn.org/cn/tutorials/intermediate/reinforcement_q_learning.html)\n",
    "- [在 imagenet 上培训最先进的 ResNet 网络](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id11)\n",
    "- [利用生成对抗网络训练人脸生成器](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id13)\n",
    "- [使用 Recurrent LSTM 网络训练单词语言模型](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id15)\n",
    "- [更多的例子](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id17)\n",
    "- [更多教程](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id19)\n",
    "- [在论坛上讨论 PyTorch](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id21)\n",
    "- [与 Slack 上与其他用户聊天](http://pytorch.apachecn.org/cn/tutorials/beginner/blitz/cifar10_tutorial.html#id23)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}