Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Reimplement teraslice-exporter using new foundation promMetrics api #3600

Closed
godber opened this issue Apr 30, 2024 · 2 comments
Closed

Reimplement teraslice-exporter using new foundation promMetrics api #3600

godber opened this issue Apr 30, 2024 · 2 comments

Comments

@godber
Copy link
Member

godber commented Apr 30, 2024

When this (https://github.com/terascope/teraslice/pull/3594/files) is merged, we should move the external teraslice-exporter project (found here: https://github.com/terascope/teraslice-exporter) into the core of the Teraslice master and adapt it to use the promMetrics API. It should expose the same set of metrics but no longer gather it's information by querying the Teraslice api.

godber pushed a commit that referenced this issue May 7, 2024
This PR makes the following changes:

- Move exporter metrics from the external teraslice exporter to new
internal metrics
- These metrics live and are only set on the `cluster_master` as the old
exporter was hitting the cluster_master api to get those specific
metrics
- Added methods within the `cluster_master`, `execution_controller`, and
`worker` class that allows for the addition of metrics specific to the
assignment
- Updated prom metrics documentation
- Bump **@terascope/job-components** from `v0.73.0` to `v0.73.1`

Ref to issue #3600
@godber
Copy link
Member Author

godber commented May 9, 2024

@busma13 can close this when he's validated internally.

@godber
Copy link
Member Author

godber commented May 9, 2024

We want to try the following tests in a test environment:

  • Teraslice v1.5.0, built-in promMetrics disabled, run job, see master resource usage
  • Teraslice v1.5.0, built-in promMetrics enabled, run job, see master,worker,exc resource usage
  • Teraslice v1.5.0, built-in promMetrics enabled, run job, scrape with prometheus, see master,worker,exc resource usage

Other things may be required in the job, but in order to target the job with prometheus, at the moment, we need to label the job manually like this on the job definition:

    ...
    "labels": {
        "scrape-target": "true",
        "scrape-prometheus-instance": "my-prom"
    },
   ...

@godber godber changed the title Reimplement teraslice-expoerter using new foundation promMetrics api Reimplement teraslice-exporter using new foundation promMetrics api May 10, 2024
@busma13 busma13 closed this as completed May 13, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

2 participants