Skip to content

Avoid parsing error from bogus Azure Flexible Server custom GUC #587

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 1 commit into from
Feb 9, 2022

Conversation

mbanck
Copy link
Contributor

@mbanck mbanck commented Nov 11, 2021

Azure Flexible Server (currently in preview but slated to become generally available before the end of the year) apparently added a custom GUC with type int, but their default value is Disabled, breaking postgres_exporter:

=> SELECT name, setting, vartype, COALESCE(unit, ''), short_desc FROM pg_settings WHERE vartype IN ('bool', 'integer', 'real') AND name = 'sync_commit_cancel_wait';                                   
          name           | setting  | vartype | coalesce |                                    short_desc                                     
-------------------------+----------+---------+----------+-----------------------------------------------------------------------------------
 sync_commit_cancel_wait | Disabled | integer |          | Allow cancellation of waiting for replication of a transaction committed locally.

This is the error:

panic: Error converting setting "sync_commit_cancel_wait" value "Disabled" to float: strconv.ParseFloat: parsing "Disabled": invalid syntax
goroutine 43 [running]:
main.(*pgSetting).metric(0xc0003bd860, 0xc00021b260, 0x5, 0x5)
    /app/cmd/postgres_exporter/pg_setting.go:80 +0x4f9
main.querySettings(0xc000200480, 0xc00029e210, 0x0, 0x0)
    /app/cmd/postgres_exporter/pg_setting.go:49 +0x305
main.(*Server).Scrape(0xc00029e210, 0xc000200480, 0xc00029e200, 0x0, 0x0)
    /app/cmd/postgres_exporter/server.go:121 +0x1a5
main.(*Exporter).scrapeDSN(0xc0002d6000, 0xc000200480, 0xc0002d4000, 0x78, 0x2, 0x0)
    /app/cmd/postgres_exporter/datasource.go:116 +0xed
main.(*Exporter).scrape(0xc0002d6000, 0xc000200480)
    /app/cmd/postgres_exporter/postgres_exporter.go:728 +0x145
main.(*Exporter).Collect(0xc0002d6000, 0xc000200480)
    /app/cmd/postgres_exporter/postgres_exporter.go:617 +0x39
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
    /go/pkg/mod/github.com/prometheus/client_golang@v1.11.0/prometheus/registry.go:448 +0x154
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
    /go/pkg/mod/github.com/prometheus/client_golang@v1.11.0/prometheus/registry.go:538 +0xe4d

I have reported this to Microsoft, but fixing this would require changing the source and making a release, so I do not anticipate a quick turnaround time for a fix.
In order to use postgres_exporter with Azure Flexible Server in the meantime, we use this hot-fix which I am also submitting here, but I can understand if you do not want to carry this work-around in the official release.

@SuperQ
Copy link
Contributor

SuperQ commented Nov 20, 2021

This needs a DCO sign-off. You can use git commit -s --amend to add it.

Signed-off-by: Michael Banck <michael.banck@credativ.de>
@AlexDCraig
Copy link

Any update on this? I'm hitting this myself.

@Aaron-ML
Copy link

Aaron-ML commented Feb 9, 2022

@mbanck @dotansimha Do we have an ETA for release?, we are currently seeing this issue.

@sysadmind sysadmind merged commit 3880df4 into prometheus-community:master Feb 9, 2022
@graudeejs
Copy link

For the documentation purposes, link to issue in azure docs:
https://github.com/MicrosoftDocs/azure-docs/issues/89931

@ianmuge ianmuge mentioned this pull request May 25, 2022
SuperQ added a commit that referenced this pull request Jul 28, 2022
NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added #556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector #556
* [FEATURE] Add pg_database collector #613
* [ENHANCEMENT] Add pg_database_size_bytes metric #613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC #587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. #599
* [BUGFIX] Sanitize setting values because of Aurora irregularity #620

Signed-off-by: SuperQ <superq@gmail.com>
@SuperQ SuperQ mentioned this pull request Jul 28, 2022
ademidoff pushed a commit to percona/postgres_exporter that referenced this pull request Oct 12, 2022
* Update build

* Update to Go 1.18.
* Update minimum Go version to 1.17.
* Update Go modules for 1.17 format.
* Bump Go modules
* Enable dependabot.
* Update Prometheus common files.
* Fixup yamllint.

Signed-off-by: SuperQ <superq@gmail.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files (prometheus-community#650)

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update readme to include Postgres 14 support

It looks like postgres 14.1 was added to CI here:

prometheus-community@fcb2535

See also: prometheus-community#651 (comment)

Signed-off-by: Austin Godber <godber@uberhip.com>

* Bump github.com/prometheus/common from 0.34.0 to 0.35.0

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.34.0 to 0.35.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Release v0.11.0

NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added prometheus-community#556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector prometheus-community#556
* [FEATURE] Add pg_database collector prometheus-community#613
* [ENHANCEMENT] Add pg_database_size_bytes metric prometheus-community#613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC prometheus-community#587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. prometheus-community#599
* [BUGFIX] Sanitize setting values because of Aurora irregularity prometheus-community#620

Signed-off-by: SuperQ <superq@gmail.com>

* fix for exporter issue 633

fix for exporter issue 633: prometheus-community#633

"Scan error on column index 2, name \"checkpoint_write_time\": converting driver.Value type float64 (\"6.594096e+06\") to a int: invalid syntax prometheus-community#633"

Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>

* Fix checkpoint_sync_time value type

Error:
sql: Scan error on column index 3, name \"checkpoint_sync_time\": converting driver.Value type float64 (\"1.876469e+06\") to a int: invalid syntax

See also:
prometheus-community#633
prometheus-community#666

Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>

* Release 0.11.1

* [BUGFIX] Fix checkpoint_write_time value type prometheus-community#666
* [BUGFIX] Fix checkpoint_sync_time value type prometheus-community#667

Signed-off-by: SuperQ <superq@gmail.com>

* PMM-10820 missing metric restore

* PMM-10820 missing metric drop

* PMM-10820 merge fix

* PMM-10820 PR review fix

Signed-off-by: SuperQ <superq@gmail.com>
Signed-off-by: prombot <prometheus-team@googlegroups.com>
Signed-off-by: Austin Godber <godber@uberhip.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Co-authored-by: SuperQ <superq@gmail.com>
Co-authored-by: prombot <prometheus-team@googlegroups.com>
Co-authored-by: Austin Godber <godber@uberhip.com>
Co-authored-by: Joe Adams <github@joeadams.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Co-authored-by: Nicolas Rodriguez <nico@nicoladmin.fr>
ritbl pushed a commit to heniek/postgres_exporter that referenced this pull request Mar 19, 2023
Avoid parsing error from bogus Azure Flexible Server custom GUC
ritbl pushed a commit to heniek/postgres_exporter that referenced this pull request Mar 19, 2023
NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added prometheus-community#556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector prometheus-community#556
* [FEATURE] Add pg_database collector prometheus-community#613
* [ENHANCEMENT] Add pg_database_size_bytes metric prometheus-community#613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC prometheus-community#587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. prometheus-community#599
* [BUGFIX] Sanitize setting values because of Aurora irregularity prometheus-community#620

Signed-off-by: SuperQ <superq@gmail.com>
ritbl pushed a commit to heniek/postgres_exporter that referenced this pull request Mar 19, 2023
* Update build

* Update to Go 1.18.
* Update minimum Go version to 1.17.
* Update Go modules for 1.17 format.
* Bump Go modules
* Enable dependabot.
* Update Prometheus common files.
* Fixup yamllint.

Signed-off-by: SuperQ <superq@gmail.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files (prometheus-community#650)

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update readme to include Postgres 14 support

It looks like postgres 14.1 was added to CI here:

prometheus-community@fcb2535

See also: prometheus-community#651 (comment)

Signed-off-by: Austin Godber <godber@uberhip.com>

* Bump github.com/prometheus/common from 0.34.0 to 0.35.0

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.34.0 to 0.35.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Release v0.11.0

NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added prometheus-community#556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector prometheus-community#556
* [FEATURE] Add pg_database collector prometheus-community#613
* [ENHANCEMENT] Add pg_database_size_bytes metric prometheus-community#613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC prometheus-community#587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. prometheus-community#599
* [BUGFIX] Sanitize setting values because of Aurora irregularity prometheus-community#620

Signed-off-by: SuperQ <superq@gmail.com>

* fix for exporter issue 633

fix for exporter issue 633: prometheus-community#633

"Scan error on column index 2, name \"checkpoint_write_time\": converting driver.Value type float64 (\"6.594096e+06\") to a int: invalid syntax prometheus-community#633"

Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>

* Fix checkpoint_sync_time value type

Error:
sql: Scan error on column index 3, name \"checkpoint_sync_time\": converting driver.Value type float64 (\"1.876469e+06\") to a int: invalid syntax

See also:
prometheus-community#633
prometheus-community#666

Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>

* Release 0.11.1

* [BUGFIX] Fix checkpoint_write_time value type prometheus-community#666
* [BUGFIX] Fix checkpoint_sync_time value type prometheus-community#667

Signed-off-by: SuperQ <superq@gmail.com>

* PMM-10820 missing metric restore

* PMM-10820 missing metric drop

* PMM-10820 merge fix

* PMM-10820 PR review fix

Signed-off-by: SuperQ <superq@gmail.com>
Signed-off-by: prombot <prometheus-team@googlegroups.com>
Signed-off-by: Austin Godber <godber@uberhip.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Co-authored-by: SuperQ <superq@gmail.com>
Co-authored-by: prombot <prometheus-team@googlegroups.com>
Co-authored-by: Austin Godber <godber@uberhip.com>
Co-authored-by: Joe Adams <github@joeadams.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Co-authored-by: Nicolas Rodriguez <nico@nicoladmin.fr>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants