3.0-Instalación Spark.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Instalando spark sin dolores de cabeza: docker\n",
    "---\n",
    "\n",
    "<img src='img/spark.png' width=300>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<img src='img/docker.png'>\n",
    "\n",
    "---\n",
    "\n",
    "# Docker\n",
    "\n",
    "A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. Available for both Linux and Windows based apps, containerized software will always run the same, regardless of the environment. Containers isolate software from its surroundings, for example differences between development and staging environments and help reduce conflicts between teams running different software on the same infrastructure.\n",
    "\n",
    "[https://www.docker.com/](https://www.docker.com/)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Docker es un sistema de virtualización que permite ejecutar software utilizando los recursos de la maquina anfitriona.\n",
    "\n",
    "Utilizaremos este sistema para las sesiones sobre spark.\n",
    "\n",
    "Para poder usarlo, instalalo según tu sistema operativo siguiendo las instrucciones en:\n",
    "\n",
    "https://www.docker.com/community-edition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Insalación en **ubuntu** por terminal:\n",
    "\n",
    "```sh\n",
    "sudo apt-get update\n",
    "sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D\n",
    "sudo apt-add-repository 'deb https://apt.dockerproject.org/repo ubuntu-xenial main'\n",
    "sudo apt-get update\n",
    "sudo apt-get install -y docker-engine\n",
    "sudo systemctl status docker\n",
    "```\n",
    "Después del último comando, veremos una sálida tipo:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Una vez instalado, abre una terminal y ejecuta el siguiente comando:\n",
    "\n",
    "```sh\n",
    "docker run hello-world\n",
    "```\n",
    "\n",
    "si todo se ha instalado correctamente, veremos aparecer:\n",
    "\n",
    "\n",
    ">Hello from Docker!\n",
    "This message shows that your installation appears to be working correctly.\n",
    "To generate this message, Docker took the following steps:\n",
    " 1. The Docker client contacted the Docker daemon.\n",
    " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\n",
    "    (amd64)\n",
    " 3. The Docker daemon created a new container from that image which runs the\n",
    "    executable that produces the output you are currently reading.\n",
    " 4. The Docker daemon streamed that output to the Docker client, which sent it\n",
    "    to your terminal.\n",
    "\n",
    ">To try something more ambitious, you can run an Ubuntu container with:\n",
    "> $ docker run -it ubuntu bash\n",
    "\n",
    ">Share images, automate workflows, and more with a free Docker ID:\n",
    " https://cloud.docker.com/\n",
    "\n",
    ">For more examples and ideas, visit:\n",
    "> https://docs.docker.com/engine/userguide/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ejecutar jupyter notebooks con python  y spark usando docker\n",
    "\n",
    "1. Dar permisos a la carpeta para que el usuario 1000 pueda acceder a ella\n",
    "```sh\n",
    "sudo chown 1000 /path/to/folder/\n",
    "```\n",
    "2. Colocarnos en la carpeta dónde queremos lanzar los notebooks y ejecutar:\n",
    "\n",
    "```sh\n",
    "sudo docker run --rm --user root -e NB_UID=1000 -it -p 8888:8888 -p 4040:4040 -e GRANT_SUDO=yes -v $PWD:/home/jovyan/MADM  jupyter/all-spark-notebook\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si es la primera vez que ejecutamos el comando, nos bajara e instalará todo lo necesario para poder trabajar con python y spark en nuestro ordenador desde el repositorio de imágenes de Docker. Si no es la primera vez, ejecutará la imagen que tiene guardada en nuestro local.\n",
    "\n",
    "Una vez lanzado el jupyter via docker, sólo tenemos que ir al navegador y usarlo como hasta ahora.\n",
    "\n",
    "\n",
    "Para más información, visita:\n",
    "\n",
    "https://github.com/jupyter/docker-stacks/tree/master/all-spark-notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}