Postgresql-setup script updates

[rdlrt]

Description

The postgresql-setup script (as well as docker restore functionality) seems to be only used/tested in a standard configuration and old version of postgresql. With the new options available to dbsync configuration, the setup fails over. This PR hopes to add a few minor updates that are hopefully easy to introduce as non-breaking changes and would largely assist us sticking to official images and not have to maintain a fork.

Additionally , there is a bump of references to postgresql 14 to 17 as pg14 is ancient and misses out on lot of value additions (eg: being able to add better compression on tx_cbor, parallel index maintainance without locks)

• docker-compose*.yml: Bump postgresql version to 17, node version to 10.1.4 and dbsync version to 13.6.0.4
• postgresql-setup.sh:
  • Parse psql output using csv instead of manipulating white chars or table seperators
  • Use strict checks for ${PGDATABASE} as there can be another database with a prefix/suffix attached, giving back false outputs
  • When creating/restoring a snapshot, existence of ledger_file should be optional, as there are dbsync config options that don't require lstate files
  • When restoring a snapshot (eg: across postgres versions or for external systems that may have views/stored functions on database being restored), there will be cases where an informational error may be expected. The presence of --exit-on-error flag prevents use of these snapshots, especially when run from docker created by nix-ops (where it isnt as straightforward to override scripts)

Checklist

• Commit sequence broadly makes sense
• Commits have useful messages
• New tests are added if needed and existing tests are updated
• Any changes are noted in the changelog
• Code is formatted with fourmolu on version 0.10.1.0 (which can be run with scripts/fourmolize.sh)
• Self-reviewed the diff

Migrations

• The pr causes a breaking change of type a,b or c
• If there is a breaking change, the pr includes a database migration and/or a fix process for old values, so that upgrade is possible
• Resyncing and running the migrations provided will result in the same database semantically

If there is a breaking change, especially a big one, please add a justification here. Please elaborate
more what the migration achieves, what it cannot achieve or why a migration is not possible.

[erikd]

@sgillespie We might need your help on the Nix stuff.

[sgillespie]

@sgillespie We might need your help on the Nix stuff.

I think we just need to update nixpkgs/haskell.nix

[sgillespie]

Unfortunately, we will not be able to use postgresql_17 until we update the nixpkgs pin for haskell.nix, currently in progress: input-output-hk/haskell.nix#2307

[rdlrt]

Will re-base and amend once #1936 is merged

[sgillespie]

Will re-base and amend once #1936 is merged

It's done, you should now be able to rebase from master

[rdlrt]

It's done, you should now be able to rebase from master

Thanks a lot @sgillespie , updated this PR

[Cmdv]

🎉 Thanks for the contribution @rdlrt

[sgillespie]

Won't quoting this make it add "" to the list of arguments? I suspect this will result in:

pg_restore: error: too many command-line arguments (first is "")

[sgillespie]

Thanks for putting this together @rdlrt . I have one concern about the CLI arguments to pg_restore that I mentioned above, otherwise LGTM. Once you address that I'll add my approval

[rdlrt]

Thanks @sgillespie - Done

[sgillespie]

Can you also add another shellcheck exception?

In ./scripts/postgresql-setup.sh line 208:
            $( [ -z "${SKIP_RESTORE_ERROR:-}" ] && echo --exit-on-error ) \
            ^-- SC2046 (warning): Quote this to prevent word splitting.

[rdlrt]

Can you also add another shellcheck exception?

In ./scripts/postgresql-setup.sh line 208:
            $( [ -z "${SKIP_RESTORE_ERROR:-}" ] && echo --exit-on-error ) \
            ^-- SC2046 (warning): Quote this to prevent word splitting.

That line [albeit surprisingly] does not seem to pop up a warning anymore on latest shellcheck v0.9.0-1, but added an exclusion regardless

[sgillespie]

Can you also add another shellcheck exception?

In ./scripts/postgresql-setup.sh line 208:
            $( [ -z "${SKIP_RESTORE_ERROR:-}" ] && echo --exit-on-error ) \
            ^-- SC2046 (warning): Quote this to prevent word splitting.

That line [albeit surprisingly] does not seem to pop up a warning anymore on latest shellcheck v0.9.0-1, but added an exclusion regardless

Interesting, I'll have to revisit next time we update nixpkgs