Cardinality of snmp_collection_duration histograms

[candlerb]

(Moved from #950)

The /metrics endpoint carries snmp_collection_duration histograms. Since the separation of module and auth parameters, these now suffer from an NxM cardinality, because every combination of (auth,module) is listed - even those which are never used.

root@prometheus:~# curl -fsS 'localhost:9116/metrics' | grep '^snmp_packet_duration' | wc -l
18
root@prometheus:~# curl -fsS 'localhost:9116/metrics' | grep '^snmp_collection_duration' | wc -l
1540

I propose that this be changed to grouping on just (module). If you happen to use different auth credentials for different devices, it doesn't necessarily have a meaningful relationship to different classes of device.

(Aside: I wonder if passing an explicit "class" scrape parameter, which defaults to the module name, might be a useful addition. You could then collect statistics specifically for a given target or group of targets, which are meaningful to you. But the same effect could be achieved by simply duplicating a particular module under a different name)

snmp_exporter version

snmp_exporter, version 0.24.0 (branch: HEAD, revision: c9fd025309572b81eaca65100de4ca38ea406da9)
  build user:       root@fb4e1a1f31ed
  build date:       20230829-12:20:02
  go version:       go1.21.0
  platform:         linux/amd64
  tags:             netgo

[SuperQ]

I thought the auth+module combination would be useful, but maybe you're right, if you really need that much granularity you can use scrape_duration_seconds from Prometheus.

[candlerb]

if you really need that much granularity you can use scrape_duration_seconds from Prometheus.

Also, snmp_exporter itself returns some per-scrape metrics from the /snmp endpoint:

# HELP snmp_scrape_duration_seconds Total SNMP time scrape took (walk and processing).
# TYPE snmp_scrape_duration_seconds gauge
snmp_scrape_duration_seconds{module="if_mib"} 0.344920395
# HELP snmp_scrape_packets_retried Packets retried for get, bulkget, and walk.
# TYPE snmp_scrape_packets_retried gauge
snmp_scrape_packets_retried{module="if_mib"} 0
# HELP snmp_scrape_packets_sent Packets sent for get, bulkget, and walk; including retries.
# TYPE snmp_scrape_packets_sent gauge
snmp_scrape_packets_sent{module="if_mib"} 22
# HELP snmp_scrape_pdus_returned PDUs returned from get, bulkget, and walk.
# TYPE snmp_scrape_pdus_returned gauge
snmp_scrape_pdus_returned{module="if_mib"} 446
# HELP snmp_scrape_walk_duration_seconds Time SNMP walk/bulkwalk took.
# TYPE snmp_scrape_walk_duration_seconds gauge
snmp_scrape_walk_duration_seconds{module="if_mib"} 0.308051709