[Standby-Support][FEAT] Add support for smartctl --nocheck= to the collector

[danie1k]

The smartctl tool has a bunch of useful arguments, Scrutiny's collector could make use of them, similarly to Telegraf.
I suggested only the nocheck argument, because for me personally it's the most important one because of "standby" option, so the drive is not woken up constantly.

Describe the solution you'd like
Extend collector's config file with more options, covering smartctl's arguments.

Excerpt from smartctl manual

-n POWERMODE, --nocheck=POWERMODE

[ATA only] Specifies if smartctl should exit before performing any checks when the device is in a low-power mode. It may be used to prevent a disk from being spun-up by smartctl. The power mode is ignored by default. A nonzero exit status is returned if the device is in one of the specified low-power modes (see RETURN VALUES below).

Note: If this option is used it may also be necessary to specify the device type with the '-d' option. Otherwise the device may spin up due to commands issued during device type autodetection.

The valid arguments to this option are:

never - check the device always, but print the power mode if '-i' is specified.

sleep - check the device unless it is in SLEEP mode.

standby - check the device unless it is in SLEEP or STANDBY mode. In these modes most disks are not spinning, so if you want to prevent a disk from spinning up, this is probably what you want.

idle - check the device unless it is in SLEEP, STANDBY or IDLE mode. In the IDLE state, most disks are still spinning, so this is probably not what you want.

-- https://linux.die.net/man/8/smartctl

[danie1k]

After getting more familiar with the repo... Will that be supported by the command in following section of the config file?

scrutiny/example.scrutiny.yaml

Lines 85 to 88 in 060ac7b

	#collect:
	# metric:
	# enable: true
	# command: '-a -o on -S on'

[AnalogJ]

Yeah, I eventually want to make the collector command flags completely customizable.
I'll add this as an enhancement, but it'll have to wait until this database upgrade is complete.

[AnalogJ]

apologies, I should have updated & closed this issue earlier. This functionality has been available since v0.4.9

You can update the collector config files and specify the command flags you'd like to use when Scrutiny triggers smartctl.

# example to show how to override the smartctl command args globally
commands:
  metrics_scan_args: '--scan --json' # used to detect devices
  metrics_info_args: '--info --json' # used to determine device unique ID & register device with Scrutiny
  metrics_smart_args: '--xall --json' # used to retrieve smart data for each device.

Closing this issue for now, feel free to comment/reopen if you have any concerns.

[boomam]

Closing this issue for now, feel free to comment/reopen if you have any concerns.

Are there any plans to add these options as env variables in a future release of the collector?

[AnalogJ]

You should already be able to do that @boomam COLLECTOR_COMMANDS_METRICS_SCAN_ARGS etc

[boomam]

You should already be able to do that @boomam COLLECTOR_COMMANDS_METRICS_SCAN_ARGS etc

Cool, i didn't see it documented anywhere (unless i egregiously missed it....) , so i thought I'd ask.

 
For my initial test, adding COLLECTOR_COMMANDS_METRICS_SCAN_ARGS as an env variable on the collector container, and setting it to -n standby or --nocheck=standby did not work.
I'll try and work out the syntax for stopping drives spinning up/similar and post a PR for the collectors doc page. :-)

[kocane]

@boomam did you ever figure this out? trying to do the same thing

[boomam]

No, i gave up and decided to use a different method to get the same data.

[kocane]

No, i gave up and decided to use a different method to get the same data.

Would you share this method with me :D?

[WhoTheHeck]

I also couldn't get it to work that way. And now use a workaround. I have multiple hosts sending data to scrutiny. Therefore, each of them has the collector running and the container itself has no drives handed over:

version: '3.5'

services:
  scrutiny:
    container_name: scrutiny
    image: ghcr.io/analogj/scrutiny:master-omnibus
    cap_add:
      - SYS_RAWIO
    ports:
      - "8080:8080"
      - "8086:8086"
    volumes:
      - /run/udev:/run/udev:ro
      - /pathto/config:/opt/scrutiny/config
      - /pathto/influxdb:/opt/scrutiny/influxdb
    devices:
      - /dev/null:/dev/null
    restart: unless-stopped

On the host with sleeping drives I use 2 config files. One that checks all drives and one that has the drives that go to sleep ignored with:

# drives to ignore:
devices:
  - device: /dev/sdc
    ignore: true
  - device: /dev/sdd
    ignore: true
  - device: /dev/sde
    ignore: true
# scrutiny endpoint:
api:
  endpoint: 'http://interalip:port'

I use a script that checks the drives state first. This doesn't wake them up. Based on that I run the collector with one or the other config:

#!/bin/bash

# Function to check if any of the drives are in standby
check_standby() {
  for drive in "sdc" "sdd" "sde"; do
    state=$(smartctl -i -n standby /dev/$drive | grep "ACTIVE or IDLE")
    if [[ -z $state ]]; then
      return 1
    fi
  done
  return 0
}

# Check if any drive is in standby and execute the corresponding scrutiny collector
if check_standby; then
  /opt/scrutiny/bin/scrutiny-collector-metrics-linux-amd64 run --config /opt/scrutiny/bin/alldrives.yaml
else
  /opt/scrutiny/bin/scrutiny-collector-metrics-linux-amd64 run --config /opt/scrutiny/bin/nowakeup.yaml
fi

...but it would be much easier if scrutiny could be configured to just not force-wake disks.

[datenzar]

Hi there,

I'm facing the same challenge and want to prevent my disks from staying active. I already set the ENV variables correspondingly but without success.

COLLECTOR_COMMANDS_METRICS_SCAN_ARGS: "--scan --json -n idle"
COLLECTOR_COMMANDS_METRICS_INFO_ARGS: "--info --json -n idle"
COLLECTOR_COMMANDS_METRICS_SMART_ARGS: "--xall --json -n idle"

My guess is that the code is missing the respective binding so I created a PR. It's in draft mode as I haven't had the chance to test it.

[datenzar]

An update from my testing efforts show bad results. Unfortunately, neither my above solution nor the configuration keeps the disk in standby mode. I guess there's another command somewhere executed in the code which isn't covered and causes the disks to spin up but above solution.

I used the solution #221 (comment) which works (thanks @WhoTheHeck!) and will use it. Not very elegant but working.

[c0ldtech]

@AnalogJ is there any documentation how it should work to change the parameters so we could (for example) include the --nocheck=standby option of smartctl?

I would really like to avoid spinning up my drives unnecessarily and it would be nice if there was an "official" way of achieving this without adding custom scripts or changing existing code in the container.

[AnalogJ]

I think c.SetEnvPrefix("COLLECTOR") is missing.

https://github.com/AnalogJ/scrutiny/pull/619/files

Can someone make a PR (and test that change)?

[datenzar]

I think c.SetEnvPrefix("COLLECTOR") is missing.

https://github.com/AnalogJ/scrutiny/pull/619/files

Can someone make a PR (and test that change)?

Hi @AnalogJ,

sorry for the late reply. I was able to complete the fix. Please see my PR. Thx