Implement playing audio through discord voice connections. (#180)

* Implement playing audio through discord voice connections in Nostrum.

* Improve voice api. Make switching voice channels painless

* Improve docs

* Add helpful functions for checking voice connections

* Update guild cache on voice_state_update. Add logger config options, update docs.

* Add support for youtube-dl

* Fix typo and make youtube-dl be quiet

* Add example bot for using voice channels.

* Shore up docs

* Format everything. Raise exception if execs not found. Warn on start if configured and not found.

* cleanup

* Detect end of input for pipe/youtubedl playing. Emit Voice Speaking Update events on start/stop.

* Fix conflict

config/config.exs

@@ -2,8 +2,12 @@ import Config
 
 config :nostrum,
   token: "",
-  num_shards: :auto
+  num_shards: :auto,
+  ffmpeg: "ffmpeg",
+  youtubedl: "youtube-dl"
 
-config :logger, :console, metadata: [:shard]
+config :logger, :console, metadata: [:shard, :guild, :channel]
+
+config :porcelain, :driver, Porcelain.Driver.Basic
 
 if File.exists?("config/secret.exs"), do: import_config("secret.exs")

docs/static/Intro.md

@@ -1,7 +1,7 @@
 # Intro
 Nostrum is a an Elixir library that can be used to interact with Discord.
 
-Nostrum currently supports the latest stable version of Elixir, v. 1.7.
+Nostrum currently supports versions of Elixir at or above v. 1.9.
 
 With a platform like Discord, there are many moving parts and an attempt was made
 to break these parts into smaller logical pieces.
@@ -13,6 +13,7 @@ the following -
  * [State](state.html) - `Caches` that hold the state of Discord that your bot can see.
  * [Events](events.html) - How you can handle real time events that your bot can see.
  * [Custom Consumers](consumers.html) - Information on defining custom consumer processes.
+ * [Voice](voice.html) - Playing audio through Discord voice channels.
 
 ### Why Elixir?
 From the Elixir website -
@@ -76,6 +77,8 @@ you can omit the field and it will default to 1. You can also set this option to
 
 The following fields are also supported:
 
+ - `ffmpeg` - Specifies the path to the `ffmpeg` executable for playing audio. Defaults to `"ffmpeg"`.
+ - `youtubedl` - Specifies the path to the `youtube-dl` executable for playing audio with youtube-dl support. Defaults to `"youtube-dl"`.
  - `gateway_intents` - This field takes a list of atoms representing gateway intents for Nostrum to subscribe to from the Discord API. More information can be found in the [gateway intents](gateway-intents.html) documentation page.
  - `dev` - This is added to enable Nostrum to be run completely stand alone for
  development purposes. `true` will cause Nostrum to spawn its own event consumers.
@@ -108,12 +111,14 @@ config :logger,
 ```
 
 Nostrum exposes the following metadata fields through logger:
- - `shard` - Id of the shard on which the event occured
+ - `shard` - Id of the shard on which the event occurred
+ - `guild` - Name of the guild on which the voice connection event occurred
+ - `channel` - Name of the channel on which the voice connection event occurred
 
 To enable this metadata, logger can be configured as such:
 ```Elixir
 config :logger, :console,
-  metadata: [:shard]
+  metadata: [:shard, :guild, :channel]
 ```  
 
 For more information on how this works, please see the Logger
@@ -125,3 +130,5 @@ A very simple example bot can be found
 [here](https://github.com/Kraigie/nostrum/blob/master/examples/event_consumer.ex).
 
 A more complex bot can be found [here](https://github.com/jchristgit/bolt).
+
+An example bot that plays audio through voice channels can be found [here](https://github.com/Kraigie/nostrum/blob/master/examples/audio_player_example.ex).

docs/static/Voice.md

@@ -0,0 +1,30 @@
+## Voice Channels
+Discord voice channels allow audio data to be sent to the voice servers over UDP.
+A bot is able to connect to up to one voice channel per guild. One websocket
+connection will be opened and maintained for each voice channel the bot joins.
+The websocket connection should reconnect automatically the same way that the 
+main Discord gateway websocket connections do. For available voice functions and
+usage see the `Nostrum.Voice` module.
+
+## FFmpeg
+Nostrum uses the powerful [ffmpeg](https://ffmpeg.org/) command line utility to
+encode any audio (or video) file for sending to Discord's voice servers.
+By default Nostrum will look for the executable `ffmpeg` in the system path.
+If the executable is elsewhere, the path may be configured via
+`config :nostrum, :ffmpeg, "/path/to/ffmpeg"`.
+The function `Nostrum.Voice.play/3` allows sound to played via files, local or 
+remote, or via raw data that gets piped to `stdin` of the `ffmpeg` process.
+When playing from a url, the url can be a name of a file on the filesystem or a url
+of file on a remote server - [ffmpeg supports a ton of protocols](https://www.ffmpeg.org/ffmpeg-protocols.html),
+the most common of which are probably `http` or simply reading a file from the filesystem.
+
+## youtube-dl
+With only `ffmpeg` installed, Nostrum supports playing audio/video files or raw, piped
+data as discussed in the section above. Nostrum also has support for `youtube-dl`, another
+powerful command line utility for downloading audio/video from online video services.
+Although the name implies support for Youtube, `youtube-dl` supports downloading from
+[an immense list of sites](https://github.com/ytdl-org/youtube-dl/blob/master/docs/supportedsites.md).
+By default Nostrum will look for the executable `youtube-dl` in the system path. If the
+executable is elsewhere, the path may be configured via `config :nostrum, :youtubedl, "/path/to/youtube-dl"`.
+When `Nostrum.Voice.play/3` is called with `:ytdl` for the `type` parameter, `youtube-dl` will be
+run with options `-f bestaudio -q -o -`, which will attempt to download the audio at the given url and pipe it to `ffmpeg`.

examples/audio_player_example.ex

@@ -0,0 +1,105 @@
+# This file can be placed somewhere in ./lib, and it can be started
+# by running iex -S mix then calling AudioPlayerSupervisor.start_link([]).
+defmodule AudioPlayerSupervisor do
+  use Supervisor
+
+  def start_link(args) do
+    Supervisor.start_link(__MODULE__, args, name: __MODULE__)
+  end
+
+  @impl true
+  def init(_init_arg) do
+    children = [AudioPlayerConsumer]
+
+    Supervisor.init(children, strategy: :one_for_one)
+  end
+end
+
+defmodule AudioPlayerConsumer do
+  use Nostrum.Consumer
+
+  alias Nostrum.Api
+  alias Nostrum.Cache.GuildCache
+  alias Nostrum.Voice
+
+  require Logger
+
+  # Soundcloud link will be fed through youtube-dl
+  @soundcloud_url "https://soundcloud.com/fyre-brand/level-up"
+  # Audio file will be fed directly to ffmpeg
+  @nut_file_url "https://brandthill.com/files/nut.wav"
+
+  def start_link do
+    Consumer.start_link(__MODULE__)
+  end
+
+  def get_voice_channel_of_msg(msg) do
+    msg.guild_id
+    |> GuildCache.get!()
+    |> Map.get(:voice_states)
+    |> Enum.find(%{}, fn v -> v.user_id == msg.author.id end)
+    |> Map.get(:channel_id)
+  end
+
+  def do_not_ready_msg(msg) do
+    Api.create_message(msg.channel_id, "I need to be in a voice channel for that.")
+  end
+
+  def handle_event({:MESSAGE_CREATE, msg, _ws_state}) do
+    case msg.content do
+      # The bot will search through the guild cache's voice states to find
+      # the voice channel that the message author is in to join.
+      "!summon" ->
+        case get_voice_channel_of_msg(msg) do
+          nil ->
+            Api.create_message(msg.channel_id, "Must be in a voice channel to summon")
+
+          voice_channel_id ->
+            Voice.join_channel(msg.guild_id, voice_channel_id)
+        end
+
+      "!leave" ->
+        Voice.leave_channel(msg.guild_id)
+
+      # Following play song/nut commands check if connected
+      # and will let the user know in case of failure.
+      "!play song" ->
+        if Voice.ready?(msg.guild_id) do
+          Voice.play(msg.guild_id, @soundcloud_url, :ytdl)
+        else
+          do_not_ready_msg(msg)
+        end
+
+      "!play nut" ->
+        if Voice.ready?(msg.guild_id) do
+          Voice.play(msg.guild_id, @nut_file_url, :url)
+        else
+          do_not_ready_msg(msg)
+        end
+
+      # Following commands don't check anything so they'll
+      # fail quietly if nothing is playing/paused or in channel.
+      "!pause" ->
+        Voice.pause(msg.guild_id)
+
+      "!resume" ->
+        Voice.resume(msg.guild_id)
+
+      "!stop" ->
+        Voice.stop(msg.guild_id)
+
+      _ ->
+        :noop
+    end
+  end
+
+  def handle_event({:VOICE_SPEAKING_UPDATE, payload, _ws_state}) do
+    Logger.debug("VOICE SPEAKING UPDATE #{inspect(payload)}")
+  end
+
+  # Default event handler, if you don't include this, your consumer WILL crash if
+  # you don't have a method definition for each event type.
+  def handle_event(_event) do
+    :noop
+  end
+end

examples/cache.ex

@@ -1,6 +1,6 @@
 # To get this example going, run `iex -S mix` and `ExampleSupervisor.start_link`.
 # This will start the event consumer under a supervisor.
-defmodule ExampleSupervisor do
+defmodule CacheExampleSupervisor do
   use Supervisor
 
   def start_link(args \\ []) do
@@ -16,7 +16,7 @@ defmodule ExampleSupervisor do
 end
 
 # The event consumer that will be handling all incoming events.
-defmodule ExampleConsumer do
+defmodule CacheExampleConsumer do
   use Nostrum.Consumer
 
   def start_link do

lib/nostrum/api.ex

@@ -110,7 +110,7 @@ defmodule Nostrum.Api do
     - `type` - The type of status to show. 0 (Playing) | 1 (Streaming) | 2 (Listening) | 3 (Watching)
     - `stream` - URL of twitch.tv stream
   """
-  @spec update_shard_status(pid, status, String.t(), integer, String.t()) :: :ok
+  @spec update_shard_status(pid, status, String.t(), integer, String.t() | nil) :: :ok
   def update_shard_status(pid, status, game, type \\ 0, stream \\ nil) do
     Session.update_status(pid, to_string(status), game, stream, type)
     :ok
@@ -121,7 +121,7 @@ defmodule Nostrum.Api do
 
   See `update_shard_status/5` for usage.
   """
-  @spec update_status(status, String.t(), integer, String.t()) :: :ok
+  @spec update_status(status, String.t(), integer, String.t() | nil) :: :ok
   def update_status(status, game, type \\ 0, stream \\ nil) do
     Supervisor.update_status(status, game, stream, type)
     :ok
@@ -136,7 +136,7 @@ defmodule Nostrum.Api do
 
   To disconnect from a channel, `channel_id` should be set to `nil`.
   """
-  @spec update_voice_state(Guild.id(), Channel.id(), boolean, boolean) :: no_return | :ok
+  @spec update_voice_state(Guild.id(), Channel.id() | nil, boolean, boolean) :: no_return | :ok
   def update_voice_state(guild_id, channel_id, self_mute \\ false, self_deaf \\ false) do
     Supervisor.update_voice_state(guild_id, channel_id, self_mute, self_deaf)
   end

lib/nostrum/application.ex

@@ -3,16 +3,20 @@ defmodule Nostrum.Application do
 
   use Application
 
+  require Logger
+
   @doc false
   def start(_type, _args) do
     check_token()
+    check_executables()
     setup_ets_tables()
 
     children = [
       Nostrum.Api.Ratelimiter,
       Nostrum.Shard.Connector,
       Nostrum.Cache.CacheSupervisor,
-      Nostrum.Shard.Supervisor
+      Nostrum.Shard.Supervisor,
+      Nostrum.Voice.Supervisor
     ]
 
     if Application.get_env(:nostrum, :dev),
@@ -37,4 +41,26 @@ defmodule Nostrum.Application do
 
   defp check_token(_invalid_format),
     do: raise("Invalid token format, copy it again from the `Bot` tab of your Application")
+
+  defp check_executables do
+    ff = Application.get_env(:nostrum, :ffmpeg)
+    yt = Application.get_env(:nostrum, :youtubedl)
+
+    cond do
+      is_binary(ff) and is_nil(System.find_executable(ff)) ->
+        Logger.warn("""
+        #{ff} was not found in your path. Nostrum requires ffmpeg to use voice.
+        If you don't intend to use voice, configure :nostrum, :ffmpeg to nil to suppress.
+        """)
+
+      is_binary(yt) and is_nil(System.find_executable(yt)) ->
+        Logger.warn("""
+        #{yt} was not found in your path. Nostrum supports youtube-dl for voice.
+        If you don't require youtube-dl support, configure :nostrum, :youtubedl to nil to suppress.
+        """)
+
+      true ->
+        :ok
+    end
+  end
 end

lib/nostrum/cache/guild/guild_server.ex

@@ -119,6 +119,11 @@ defmodule Nostrum.Cache.Guild.GuildServer do
     call(guild_id, {:update, :emoji, guild_id, emojis})
   end
 
+  @doc false
+  def voice_state_update(guild_id, voice_state) do
+    call(guild_id, {:update, :voice_state, guild_id, voice_state})
+  end
+
   def handle_call({:select, fun}, _from, state) do
     {:reply, fun.(state), state, :hibernate}
   end
@@ -195,6 +200,23 @@ defmodule Nostrum.Cache.Guild.GuildServer do
     {:reply, ret, %{state | roles: new}, :hibernate}
   end
 
+  def handle_call({:update, :voice_state, guild_id, vsu}, _from, %{voice_states: vs_list} = state) do
+    # Remove heavy and duplicate data from voice state
+    vsu = vsu |> Map.delete(:member)
+    vs_list = vs_list |> Enum.reject(fn v -> v.user_id == vsu.user_id end)
+
+    new_voice_states =
+      if is_nil(vsu.channel_id) do
+        # Leaving
+        vs_list
+      else
+        # Joining, changing, or updating
+        [vsu | vs_list]
+      end
+
+    {:reply, {guild_id, new_voice_states}, %{state | voice_states: new_voice_states}, :hibernate}
+  end
+
   def handle_call({:update, :emoji, guild_id, emojis}, _from, state) do
     old_emojis = state.emojis
     {:reply, {guild_id, old_emojis, emojis}, %{state | emojis: emojis}, :hibernate}

lib/nostrum/constants.ex

@@ -131,4 +131,34 @@ defmodule Nostrum.Constants do
     {k, _} = Enum.find(opcodes(), fn {_, v} -> v == opcode end)
     k |> String.downcase() |> String.to_atom()
   end
+
+  # Voice Gateway has a separate set of opcodes
+  def voice_opcodes do
+    %{
+      "IDENTIFY" => 0,
+      "SELECT_PROTOCOL" => 1,
+      "READY" => 2,
+      "HEARTBEAT" => 3,
+      "SESSION_DESCRIPTION" => 4,
+      "SPEAKING" => 5,
+      "HEARTBEAT_ACK" => 6,
+      "RESUME" => 7,
+      "HELLO" => 8,
+      "RESUMED" => 9,
+      "UNDOCUMENTED_10" => 10,
+      "UNDOCUMENTED_11" => 11,
+      "CLIENT_CONNECT" => 12,
+      "CLIENT_DISCONNECT" => 13,
+      "CODEC_INFO" => 14
+    }
+  end
+
+  def voice_opcode_from_name(event) do
+    voice_opcodes()[event]
+  end
+
+  def atom_from_voice_opcode(opcode) do
+    {k, _} = Enum.find(voice_opcodes(), fn {_, v} -> v == opcode end)
+    k |> String.downcase() |> String.to_atom()
+  end
 end

lib/nostrum/consumer.ex

@@ -22,7 +22,7 @@ defmodule Nostrum.Consumer do
 
   alias Nostrum.Shard.Stage.Cache
   alias Nostrum.Struct.{Channel, WSState}
-  alias Nostrum.Struct.Event.{MessageDelete, MessageDeleteBulk}
+  alias Nostrum.Struct.Event.{MessageDelete, MessageDeleteBulk, SpeakingUpdate}
   alias Nostrum.Util
 
   @doc """
@@ -160,6 +160,7 @@ defmodule Nostrum.Consumer do
           {:USER_UPDATE,
            {old_user :: Nostrum.Struct.User.t() | nil, new_user :: Nostrum.Struct.User.t()},
            WSState.t()}
+  @type voice_speaking_update :: {:VOICE_SPEAKING_UPDATE, SpeakingUpdate.t(), WSState.t()}
   @type voice_state_update :: {:VOICE_STATE_UPDATE, map, WSState.t()}
   @type voice_server_update :: {:VOICE_SERVER_UPDATE, map, WSState.t()}
   @type webhooks_update :: {:WEBHOOKS_UPDATE, map, WSState.t()}
@@ -200,6 +201,7 @@ defmodule Nostrum.Consumer do
           | typing_start
           | user_settings_update
           | user_update
+          | voice_speaking_update
           | voice_state_update
           | voice_server_update
           | webhooks_update