diff --git a/guides/intro/intro.md b/guides/intro/intro.md index 369cd3f0b..642b47e60 100644 --- a/guides/intro/intro.md +++ b/guides/intro/intro.md @@ -35,55 +35,73 @@ config :nostrum, ## Configuration options -Apart from the fields mentioned above, the following fields are also supported: - - - `num_shards` - A fixed number of shards to run, or `:auto` to have Nostrum - determine it automatically. Defaults to `:auto`. - - `ffmpeg` - Specifies the path to the `ffmpeg` executable for playing audio. - Defaults to `"ffmpeg"`. - - `youtubedl` - Specifies the path to the `youtube-dl` executable for playing - audio with youtube-dl support. Defaults to `"youtube-dl"`. - - `streamlink` - Specifies the path to the `streamlink` executable for playing - livestream audio with streamlink support. Defaults to `"streamlink"`. - - `gateway_intents` - This field takes a list of atoms representing gateway - intents for Nostrum to subscribe to from the Discord API. More information - can be found in the [gateway intents](gateway-intents.html) documentation - page. - - `audio_timeout` - Milliseconds that input must begin generating audio by - upon invoking `play`. More information about this option can be found in the - [voice](voice.html) documentation page. Defaults to `20_000` (20s). - - `audio_frames_per_burst` - Number of opus frames to send at a time while - playing audio. More information about this option can be found in the - [voice](voice.html) documentation page. Defaults to `10`. - - `voice_auto_connect` - This will determine if Nostrum automatically connects - to voice websockets gateways upon joining voice channels. If set to `false` - but you still wish to connect to the voice gateway, you can do so manually - by calling `Nostrum.Voice.connect_to_gateway/1` after joining a voice - channel. Defaults to `true`. - - `dev` - This is added to enable Nostrum to be run completely stand alone for - development purposes. `true` will cause Nostrum to spawn its own event - consumers. If you have the dev flag set to true while running Nostrum - alongside your application some of your events will be consumed. Defaults to - `false`. - - `log_full_events` - This will log the full payload received over the - websocket. This is included primarily for debugging and testing purposes. - Defaults to `false`. - - `log_dispatch_events` - This will log dispatch events as they are received - from the gateway. This is included primarily for debugging and testing - purposes. Defaults to `false`. - - `request_guild_members` - This will perform member chunking to retrieve a - complete list of members for all guilds. This will increase start up time - and memory usage by quite a bit. Defaults to `false`. - - `fullsweep_after_default` - Sets the `fullsweep_after` flag for processes - that can have irregularly high memory usage due to Discord payloads. This - options will dramatically reduce the amount of memory used by some processes - at the cost of increased CPU usage. This is useful if you're running your - application under a memory constrained environment. This comes at the cost - of increased CPU usage. By default, this option will only affect some - processes. You can set this flag for *all* processes using environment - variables or by [setting the system flag - yourself](http://erlang.org/doc/man/erlang.html#system_flag-2). Defaults to - whatever your system recommends, which is probably `65535`. +Apart from the `token` field mentioned above, the following fields are also supported: + +- `num_shards` - the amount of shards to run. Can be one of the following: + - `:auto`: use the suggested amount of shards as provided by Discord. + - *`num`*: a number of shards to run. nostrum will warn if this is not the + recommended amount. + - `{lowest, highest, total}`: start shards `lowest` to `highest`. `total` + should contain the total amount of shards that your bot is expected to have. + Useful for splitting a single bot across multiple servers, but see also [the + multi-node documentation](../advanced/multi_node.md). +- `gateway_intents` - a list of atoms representing gateway intents for Nostrum + to subscribe to from the Discord API. More information can be found in the + [gateway intents](./gateway_intents.md) documentation page. +- `request_guild_members` - perform member chunking to retrieve a complete list + of members for all guilds at startup. Depending on your [cache + backend](../advanced/pluggable_caching.md), this may increase startup time + and, memory usage by quite a bit. Defaults to `false`. + + +### Voice-specific + +- `ffmpeg` - Specifies the path to the `ffmpeg` executable for playing audio. + Defaults to `"ffmpeg"`. +- `youtubedl` - Specifies the path to the `youtube-dl` executable for playing + audio with youtube-dl support. Defaults to `"youtube-dl"`. +- `streamlink` - Specifies the path to the `streamlink` executable for playing + livestream audio with streamlink support. Defaults to `"streamlink"`. +- `audio_timeout` - Milliseconds that input must begin generating audio by + upon invoking `play`. More information about this option can be found in the + [voice](./voice.html) documentation page. Defaults to `20_000` (20s). +- `audio_frames_per_burst` - Number of opus frames to send at a time while + playing audio. More information about this option can be found in the + [voice](./voice.html) documentation page. Defaults to `10`. +- `voice_auto_connect` - This will determine if Nostrum automatically connects + to voice websockets gateways upon joining voice channels. If set to `false` + but you still wish to connect to the voice gateway, you can do so manually + by calling `Nostrum.Voice.connect_to_gateway/1` after joining a voice + channel. Defaults to `true`. + + +### Development & debugging + +- `log_full_events` - This will log the full payload received over the + websocket. Defaults to `false`. +- `log_dispatch_events` - This will log dispatch events as they are received + from the gateway. Defaults to `false`. +- `fullsweep_after_default` - Sets the `fullsweep_after` flag for processes + that can have irregularly high memory usage due to Discord payloads. This + options will dramatically reduce the amount of memory used by some processes + at the cost of increased CPU usage. This is useful if you're running your + application under a memory constrained environment. This comes at the cost + of increased CPU usage. By default, this option will only affect some + processes. You can set this flag for *all* processes using environment + variables or by [setting the system flag + yourself](http://erlang.org/doc/man/erlang.html#system_flag-2). Defaults to + whatever your system recommends, which is probably `65535`. + + +### Internal options + +The following options are only used for testing nostrum itself. + +- `dev` - This is added to enable Nostrum to be run completely stand alone for + development purposes. `true` will cause Nostrum to spawn its own event + consumers. If you have the dev flag set to true while running Nostrum + alongside your application some of your events will be consumed. Defaults to + `false`. ## Logging diff --git a/lib/nostrum/shard.ex b/lib/nostrum/shard.ex index 7ac4c4db9..7f77ad2c7 100644 --- a/lib/nostrum/shard.ex +++ b/lib/nostrum/shard.ex @@ -5,7 +5,7 @@ defmodule Nostrum.Shard do alias Nostrum.Shard.Session - def start_link([_, shard_num] = opts) do + def start_link([_, shard_num, _total] = opts) do Supervisor.start_link(__MODULE__, opts, name: :"Nostrum.Shard-#{shard_num}") end diff --git a/lib/nostrum/shard/payload.ex b/lib/nostrum/shard/payload.ex index f3339662a..09d96a2a7 100644 --- a/lib/nostrum/shard/payload.ex +++ b/lib/nostrum/shard/payload.ex @@ -1,7 +1,7 @@ defmodule Nostrum.Shard.Payload do @moduledoc false - alias Nostrum.{Constants, Shard.Intents, Util} + alias Nostrum.{Constants, Shard.Intents} @large_threshold 250 @@ -24,7 +24,7 @@ defmodule Nostrum.Shard.Payload do }, "compress" => false, "large_threshold" => @large_threshold, - "shard" => [state.shard_num, Util.num_shards()], + "shard" => [state.shard_num, state.total_shards], "intents" => Intents.get_enabled_intents() } |> build_payload("IDENTIFY") diff --git a/lib/nostrum/shard/session.ex b/lib/nostrum/shard/session.ex index 8324c0e9a..c259e4c91 100644 --- a/lib/nostrum/shard/session.ex +++ b/lib/nostrum/shard/session.ex @@ -48,15 +48,17 @@ defmodule Nostrum.Shard.Session do GenServer.call(pid, :get_ws_state) end - def start_link([gateway, shard_num]) do - GenServer.start_link(__MODULE__, [gateway, shard_num], spawn_opt: [Util.fullsweep_after()]) + def start_link([gateway, shard_num, total]) do + GenServer.start_link(__MODULE__, [gateway, shard_num, total], + spawn_opt: [Util.fullsweep_after()] + ) end - def init([_gateway, _shard_num] = args) do + def init([_gateway, _shard_num, _total] = args) do {:ok, nil, {:continue, args}} end - def handle_continue([gateway, shard_num], nil) do + def handle_continue([gateway, shard_num, total_shards], nil) do Connector.block_until_connect() Logger.metadata(shard: shard_num) @@ -73,6 +75,7 @@ defmodule Nostrum.Shard.Session do conn_pid: self(), conn: worker, shard_num: shard_num, + total_shards: total_shards, stream: stream, gateway: gateway <> @gateway_qs, last_heartbeat_ack: DateTime.utc_now(), diff --git a/lib/nostrum/shard/supervisor.ex b/lib/nostrum/shard/supervisor.ex index f0dbac256..0fd84acb7 100644 --- a/lib/nostrum/shard/supervisor.ex +++ b/lib/nostrum/shard/supervisor.ex @@ -44,33 +44,34 @@ defmodule Nostrum.Shard.Supervisor do require Logger - def start_link(_args) do - {url, gateway_shard_count} = Util.gateway() - - num_shards = - case Application.get_env(:nostrum, :num_shards, :auto) do - :auto -> - gateway_shard_count + defp cast_shard_range(gateway_shards, :auto), do: {1, gateway_shards, gateway_shards} + defp cast_shard_range(gateway_shards, gateway_shards), do: {1, gateway_shards, gateway_shards} + + defp cast_shard_range(gateway_shards, count) when is_integer(count) and count > 0 do + Logger.warn( + "Configured shard count (#{count}) " <> + "differs from Discord Gateway's recommended shard count (#{gateway_shards}). " <> + "Consider using `num_shards: :auto` option in your Nostrum config." + ) - ^gateway_shard_count -> - gateway_shard_count + {1, count, count} + end - shard_count when is_integer(shard_count) and shard_count > 0 -> - Logger.warn( - "Configured shard count (#{shard_count}) " <> - "differs from Discord Gateway's recommended shard count (#{gateway_shard_count}). " <> - "Consider using `num_shards: :auto` option in your Nostrum config." - ) + defp cast_shard_range(_gateway_shards, {lowest, highest, total} = range) + when is_integer(lowest) and is_integer(highest) and is_integer(total) and + lowest <= highest and highest <= total do + range + end - shard_count + def start_link(_args) do + {url, gateway_shard_count} = Util.gateway() - value -> - raise ~s("#{value}" is not a valid shard count) - end + value = Application.get_env(:nostrum, :num_shards, :auto) + shard_range = cast_shard_range(gateway_shard_count, value) Supervisor.start_link( __MODULE__, - [url, num_shards], + [url, shard_range], name: __MODULE__ ) end @@ -101,16 +102,17 @@ defmodule Nostrum.Shard.Supervisor do end @doc false - def init([url, num_shards]) do - children = for i <- 0..(num_shards - 1), do: create_worker(url, i) + def init([url, {lowest, highest, total}]) do + shard_range = lowest..highest + children = for num <- shard_range, do: create_worker(url, num - 1, total) Supervisor.init(children, strategy: :one_for_one, max_restarts: 3, max_seconds: 60) end @doc false - def create_worker(gateway, shard_num) do + def create_worker(gateway, shard_num, total) do Supervisor.child_spec( - {Shard, [gateway, shard_num]}, + {Shard, [gateway, shard_num, total]}, id: shard_num ) end diff --git a/lib/nostrum/struct/ws_state.ex b/lib/nostrum/struct/ws_state.ex index a3e092fe3..571e12f93 100644 --- a/lib/nostrum/struct/ws_state.ex +++ b/lib/nostrum/struct/ws_state.ex @@ -5,6 +5,7 @@ defmodule Nostrum.Struct.WSState do defstruct [ :shard_num, + :total_shards, :seq, :session, :shard_pid, @@ -23,6 +24,15 @@ defmodule Nostrum.Struct.WSState do @typedoc "The shard number" @type shard_num :: non_neg_integer + @typedoc """ + The highest shard number for this bot. + + This may not be started locally, it is just used by nostrum to inform the + gateway which events we are interested in. + """ + @typedoc since: "0.8.0" + @type total_shards :: non_neg_integer() + @typedoc "The sequence number of the last event" @type seq :: integer | nil @@ -71,6 +81,7 @@ defmodule Nostrum.Struct.WSState do @type t :: %__MODULE__{ shard_num: shard_num, + total_shards: total_shards, seq: seq, session: session, shard_pid: shard_pid, diff --git a/lib/nostrum/util.ex b/lib/nostrum/util.ex index ba420165d..c06d4773d 100644 --- a/lib/nostrum/util.ex +++ b/lib/nostrum/util.ex @@ -53,19 +53,38 @@ defmodule Nostrum.Util do end @doc """ - Returns the number of shards. + Returns the total amount of shards as per the configuration. - This is not the number of currently active shards, but the number of shards specified - in your config. + ## Return value + + - If you specified your shards as `:auto`, the return value will be the + recommended number of shards as given by the gateway. + + - If you explicitly specified your shard numbers as an integer, it will be + the given number. + + - If you specified your shards in the form `{lowest, highest, total}` to + start a specific range of the total shards you want to start, this will be + the `total` value. + + Should Discord not supply us with any shard information, this will return + `1`. + + Note that this is not the number of currently active shards, but the number + of shards specified in your config. """ - @spec num_shards() :: integer + @spec num_shards() :: pos_integer() def num_shards do num = with :auto <- Application.get_env(:nostrum, :num_shards, :auto), - {_url, shards} <- gateway(), - do: shards + {_url, shards} <- gateway() do + shards + end - if num == nil, do: 1, else: num + case num do + {_lowest, _highest, total} -> total + nil -> 1 + end end @doc false