Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Restructure collecting of prometheus metrics and bump dependencies #606

Merged
merged 3 commits into from
Oct 19, 2022

Conversation

paulfantom
Copy link
Contributor

@paulfantom paulfantom commented Oct 13, 2022

What this PR does / why we need it

This PR aims to reduce learning curve and simplify UX by configuring prometheus instances shipped with tobs to gather all PrometheusRules, ServiceMonitors, PodMonitors, and Probes that are injected into the cluster. This way the observability platform can collect data from the whole cluster.

To opt-out a resource from being used by prometheus shipped with tobs, you need to label it with tobs/excluded label (value doesn't matter).

This PR is also disabling scraping metrics using prometheus label mechanism and forcing users to use prometheus-operator CRs. This should prevent data duplication in larger clusters and align us closer to kube-prometheus project (jsonnet one). Feature can be re-enable if needed by the user.

Checklist

@paulfantom paulfantom changed the title Collect all prometheus-operator resources by default Restructure collecting of prometheus metrics Oct 13, 2022
@paulfantom paulfantom marked this pull request as ready for review October 13, 2022 14:39
@paulfantom paulfantom requested a review from a team as a code owner October 13, 2022 14:39
@paulfantom paulfantom force-pushed the collect-all branch 2 times, most recently from cd3e06e to 297efc3 Compare October 17, 2022 12:06
@paulfantom paulfantom self-assigned this Oct 17, 2022
@paulfantom paulfantom changed the title Restructure collecting of prometheus metrics Restructure collecting of prometheus metrics and bump dependencies Oct 17, 2022
@paulfantom paulfantom enabled auto-merge October 17, 2022 12:32
@paulfantom paulfantom disabled auto-merge October 18, 2022 18:24
@paulfantom paulfantom marked this pull request as draft October 19, 2022 08:47
@paulfantom paulfantom force-pushed the collect-all branch 2 times, most recently from 655dc44 to 4f0da98 Compare October 19, 2022 14:25
@paulfantom
Copy link
Contributor Author

I think I found the main issues preventing merge of this PR:

  • timescaledb-single chart was in incorrect version and promscale couldn't connect
  • grafana test pod was tried to be scraped for metrics, which resulted in test failure

@paulfantom paulfantom marked this pull request as ready for review October 19, 2022 14:31
@paulfantom paulfantom requested a review from nhudson October 19, 2022 14:31
@paulfantom paulfantom enabled auto-merge October 19, 2022 14:38
@paulfantom paulfantom merged commit 240a7fe into timescale:main Oct 19, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants