Nostrum Application Crash Loop

[hukl]

Over night our error log started filling up with these messages where the Nostrum app goes into a crash loop when it fails to parse Guilds.Models.IncidentsData

Stacktrace:

[error] GenServer #PID<0.9674.0> terminating
** (Protocol.UndefinedError) protocol Enumerable not implemented for %{__struct__: Guilds.Models.IncidentsData, dm_spam_detected_at: nil, dms_disabled_until: nil, invites_disabled_until: nil, raid_detected_at: nil} of type Guilds.Models.IncidentsData (a struct)
    (elixir 1.14.5) lib/enum.ex:1: Enumerable.impl_for!/1
    (elixir 1.14.5) lib/enum.ex:166: Enumerable.reduce/3
    (elixir 1.14.5) lib/enum.ex:4307: Enum.reduce/3
    (nostrum 0.8.0) lib/nostrum/util.ex:153: Nostrum.Util.safe_atom_map/1
    (nostrum 0.8.0) lib/nostrum/util.ex:153: anonymous fn/2 in Nostrum.Util.safe_atom_map/1
    (stdlib 4.3.1) maps.erl:411: :maps.fold_1/3
    (nostrum 0.8.0) lib/nostrum/util.ex:153: Nostrum.Util.safe_atom_map/1
    (nostrum 0.8.0) lib/nostrum/util.ex:153: anonymous fn/2 in Nostrum.Util.safe_atom_map/1
    (stdlib 4.3.1) maps.erl:411: :maps.fold_1/3
    (nostrum 0.8.0) lib/nostrum/util.ex:153: Nostrum.Util.safe_atom_map/1
    (nostrum 0.8.0) lib/nostrum/shard/event.ex:13: Nostrum.Shard.Event.handle/3
    (nostrum 0.8.0) lib/nostrum/shard/session.ex:103: Nostrum.Shard.Session.handle_info/2
    (stdlib 4.3.1) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3.1) gen_server.erl:1200: :gen_server.handle_msg/6
    (stdlib 4.3.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:gun_ws, #PID<0.9676.0>, #Reference<0.2461590911.3585343490.220956>, {:binary, <<236, 125, 11, 124, 20, 213, 213, 248, 36, 44, 73, 72, 66, 178, 64, 64, 65, 133, 65, 107, 5, 36, 113, 94, 251, 138, 80, 54, 201, 134, 16, 75, 96, 73, 2, 40, 98, 151, 217, 221, 217, 236, 192, 238, ...>>}}
State: %Nostrum.Struct.WSState{shard_num: 0, total_shards: 1, seq: 1, session: "7849acc5c0ce6e80fb58cc770b9c6238", shard_pid: nil, conn: #PID<0.9676.0>, conn_pid: #PID<0.9674.0>, stream: #Reference<0.2461590911.3585343490.220956>, gateway: "gateway.discord.gg/?compress=zlib-stream&encoding=etf&v=10", last_heartbeat_send: ~U[2023-06-15 10:44:23.340796Z], last_heartbeat_ack: ~U[2023-06-15 10:44:23.489435Z], heartbeat_ack: true, heartbeat_interval: 41250, heartbeat_ref: {:once, #Reference<0.2461590911.3585343490.220960>}, zlib_ctx: #Reference<0.2461590911.3585474562.220958>}

I dumped the payload before parsing in session.ex:103 and the only reference to to this struct looks as follows

incidents_data: %{
      __struct__: Guilds.Models.IncidentsData,
      dm_spam_detected_at: nil,
      dms_disabled_until: nil,
      invites_disabled_until: nil,
      raid_detected_at: nil
    },

Any ideas what's happening or how to fix this?

[jchristgit]

#351 is a similar problem. This needs to be fixed on Discord's side. Not sure if we can make this more crash-resistant in nostrum. Upstream issue is at discord/discord-api-docs#6230

[lyninx]

Heya! Just an FYI; I have a bot using nostrum that randomly started falling over a couple of days ago due to the same issue. It looks like Guilds.Models.RecurrenceRule is being leaked in my case.

[error] GenServer #PID<0.527.0> terminating
** (Protocol.UndefinedError) protocol Enumerable not implemented for %{__struct__: Guilds.Models.RecurrenceRule, by_month: nil, by_month_day: nil, by_n_weekday: nil, by_weekday: [1], by_year_day: nil, count: nil, end: nil, frequency: 2, interval: 1, start: "2023-12-19T00:00:00+00:00"} of type Guilds.Models.RecurrenceRule (a struct). This protocol is implemented for the following type(s): DBConnection.PrepareStream, DBConnection.Stream, Date.Range, Ecto.Adapters.SQL.Stream, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Jason.OrderedObject, List, Map, MapSet, Postgrex.Stream, Range, Stream
    (elixir 1.14.3) lib/enum.ex:1: Enumerable.impl_for!/1
    (elixir 1.14.3) lib/enum.ex:166: Enumerable.reduce/3
    (elixir 1.14.3) lib/enum.ex:4307: Enum.reduce/3
    (nostrum 0.7.0) lib/nostrum/util.ex:132: Nostrum.Util.safe_atom_map/1
    (nostrum 0.7.0) lib/nostrum/util.ex:132: anonymous fn/2 in Nostrum.Util.safe_atom_map/1
    (stdlib 4.3) maps.erl:411: :maps.fold_1/3
    (nostrum 0.7.0) lib/nostrum/util.ex:132: Nostrum.Util.safe_atom_map/1
    (elixir 1.14.3) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
    (nostrum 0.7.0) lib/nostrum/util.ex:132: anonymous fn/2 in Nostrum.Util.safe_atom_map/1
    (stdlib 4.3) maps.erl:411: :maps.fold_1/3
    (nostrum 0.7.0) lib/nostrum/util.ex:132: Nostrum.Util.safe_atom_map/1
    (nostrum 0.7.0) lib/nostrum/util.ex:132: anonymous fn/2 in Nostrum.Util.safe_atom_map/1
    (stdlib 4.3) maps.erl:411: :maps.fold_1/3
    (nostrum 0.7.0) lib/nostrum/util.ex:132: Nostrum.Util.safe_atom_map/1
    (nostrum 0.7.0) lib/nostrum/shard/event.ex:12: Nostrum.Shard.Event.handle/3
    (nostrum 0.7.0) lib/nostrum/shard/session.ex:100: Nostrum.Shard.Session.handle_info/2
    (stdlib 4.3) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3) gen_server.erl:1200: :gen_server.handle_msg/6
    (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:gun_ws, #PID<0.528.0>, #Reference<0.3883579898.983040001.130556>, {:binary, <<236, 125, 9, 120, 36, 101, 153, 112, 39, 233, 220, 231, 12, 51, 160, 32, 80, 136, 202, 53, 105, 234, 62, 34, 66, 206, 153, 140, 147, 201, 196, 36, 128, 3, 236, 54, 95, 85, 125, 149, 174, 73, 119, 87, ...>>}}
State: %Nostrum.Struct.WSState{shard_num: 0, seq: 2, session: "16ae16936d51729e231d024b53096377", shard_pid: nil, conn: #PID<0.528.0>, conn_pid: #PID<0.527.0>, stream: #Reference<0.3883579898.983040001.130556>, gateway: "gateway.discord.gg/?compress=zlib-stream&encoding=etf&v=10", last_heartbeat_send: ~U[2023-12-18 01:11:00.834439Z], last_heartbeat_ack: ~U[2023-12-18 01:11:00.913502Z], heartbeat_ack: true, heartbeat_interval: 41250, heartbeat_ref: {:once, #Reference<0.3883579898.983040002.131081>}, zlib_ctx: #Reference<0.3883579898.983171076.130054>}

To fix things on my end, I've pinned my app to use a ref containing the fix above (#512) until it gets rolled into the next release. Cheers!

[Th3-M4jor]

Thanks for the report, I'll open an issue on the discord api docs.

[jchristgit]

Are we missing safe_atom_map somewhere? I thought we were resilient against this now..

[Th3-M4jor]

@jchristgit the fix was merged after the most recent hex release