Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

consul_raft_leader does not reflect correct state of leader #208

Open
Abhimanyu-Jana opened this issue Jul 30, 2021 · 0 comments
Open

consul_raft_leader does not reflect correct state of leader #208

Abhimanyu-Jana opened this issue Jul 30, 2021 · 0 comments

Comments

@Abhimanyu-Jana
Copy link

What did you do?
setup monitoring for presence/absence of cluster leader using consul_raft_leader metric

What did you expect to see?
When external queries to consul cluster fail with HTTP 500 or "No cluster leader" error, we expect to see consul_raft_leader value change from 1 to 0

What did you see instead? Under which circumstances?
consul_raft_leader value still remains 1 despite there being obvious issues with cluster health. We can confirm based on logs that show the "No cluster leader" errors, as well as using "consul operator raft list-peers" command

Environment
Linux

  • consul_exporter version:
    0.7.1

  • Consul version:
    Consul v1.8.3

  • Prometheus version:
    N/A

  • Prometheus configuration file:
    N/A

  • Logs:

Error getting peers: Failed to retrieve raft configuration: Unexpected response code: 500 (No cluster leader)
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant