"Number of files by Instance" metrics doesn't shows current state

[AdamStawarz]

Installation details
Panel Name: Number of files by Instance
Dashboard Name: OS Metrics
Scylla-Monitoring Version: 3.10.0
Scylla-Version: 2021.1

Number of files by Instance is always different from the actual value on node and does not change.
I've already checked it on few clusters.

[~]$ sudo find /var/lib/scylla/ | wc -l
364
[~]$ sudo lsof /var/lib/scylla/ | wc -l
46
[~]$ sudo lsof | wc -l
4153

[AdamStawarz]

@amnonh @tarzanek

[fee-mendes]

The formula should likely be:
sum(node_filesystem_files{mountpoint="$mount_point", instance=~"$node"} - node_filesystem_files_free{mountpoint="$mount_point", instance=~"$node"}) by ([[by]])

node_filesystem_files basically describes the maximum number of inodes, whereas node_filesystem_files_free is the number of unallocated inodes. total - free == used

[fee-mendes]

@AdamStawarz also keep in mind that there is no relation between lsof and the number of inodes in a filesystem. So only the first find command is what's of interest in your output. ;-)