Skip to content

feat: multi-ext-versions-pgrcron #1549

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 41 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
a6ea70f
feat: multi-ext-versios-pgrcron
samrose Apr 15, 2025
036732a
feat: add version to drv and patch instead of postPatch rewrite
samrose Apr 15, 2025
5ebea89
chore: newline
samrose Apr 15, 2025
864b0a1
feat: auto create multi version
samrose Apr 15, 2025
1cfdab0
chore: do not re-intro maintainers here not needed
samrose Apr 15, 2025
bf0f7f4
chore: bump version
samrose Apr 16, 2025
d662c77
feat: pg_cron version switcher in pkg and prestart
samrose Apr 24, 2025
b25a8ea
test: a tmp test for this branch to test older versions
samrose Apr 24, 2025
525dbb9
test: temp test for ext handling
samrose Apr 24, 2025
ed45c8b
test: instead of patch, add prior to start of machine in testinfra
samrose Apr 24, 2025
2dc78f2
test: only run on pg 15 for this test, as 1.3.1 limited to 15
samrose Apr 24, 2025
f465cfd
feat: use jq from nixpkgs
samrose Apr 24, 2025
c6d497c
test: handle braces properly
samrose Apr 24, 2025
642444f
test: propagate errors from any failure
samrose Apr 25, 2025
c52c1fc
test: more logging for healthcheck
samrose Apr 25, 2025
e1bc16e
test: adding even more logging
samrose Apr 25, 2025
320c4b6
test: extend logging more to see what happens when pg starts
samrose Apr 25, 2025
bdcaede
test: handle lib extension names per system
samrose Apr 28, 2025
aa2da1b
chore: bump versions
samrose Apr 28, 2025
6008d31
test: more debugging
samrose Apr 28, 2025
9147e5e
test: move logging here
samrose Apr 28, 2025
108c2cc
test: try direct
samrose Apr 28, 2025
30e8ae5
test: use the right alias on machine
samrose Apr 28, 2025
4b3daef
test: do not unpack result
samrose Apr 29, 2025
44d18ba
test: reorg and print logs while waiting continue on other checks whe…
samrose Apr 29, 2025
d39e7a1
test: restructure checks to avoid race
samrose Apr 29, 2025
fcb2976
Update nix/ext/pg_cron.nix
samrose May 1, 2025
8107172
chore: refactor based on review
samrose May 2, 2025
e8b7707
fix: fixing broken elements of script
samrose May 5, 2025
4597490
test: new flag and other fixes to testing
samrose May 13, 2025
23d6272
test: pushing commit to register for nix build and testing
samrose May 19, 2025
c917ba8
feat: prestart detect version
samrose May 19, 2025
12abec0
feat: provide jq correct
samrose May 19, 2025
762f4b9
test: rollback logging on test
samrose May 20, 2025
386bf9f
test: trying with systemd restart
samrose May 20, 2025
609744e
chore: cleanup type
samrose May 20, 2025
1181804
feat: fixing symlink issues, prestart bugs
samrose May 20, 2025
1472a8e
fix: drill down in aliases to change core alias
samrose May 21, 2025
5eb9ef5
chore: bump test vers.
samrose May 21, 2025
57687fc
feat: handling symlinks
samrose May 21, 2025
3646bff
chore: bump vers.
samrose May 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/testinfra-ami-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,13 @@ jobs:
df -h / # Display available space

- name: Run tests
timeout-minutes: 10
timeout-minutes: 30
env:
AMI_NAME: "supabase-postgres-${{ steps.random.outputs.random_string }}"
run: |
# TODO: use poetry for pkg mgmt
pip3 install boto3 boto3-stubs[essential] docker ec2instanceconnectcli pytest pytest-testinfra[paramiko,docker] requests
pytest -vv -s testinfra/test_ami_nix.py
pytest -vvvv -s testinfra/test_ami_nix.py

- name: Cleanup resources on build cancellation
if: ${{ cancelled() }}
Expand Down
79 changes: 79 additions & 0 deletions ansible/files/postgres_prestart.sh.j2
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
#!/bin/bash

set -x # Print commands

log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
}

check_orioledb_enabled() {
local pg_conf="/etc/postgresql/postgresql.conf"
if [ ! -f "$pg_conf" ]; then
Expand All @@ -26,7 +32,78 @@ update_orioledb_buffers() {
fi
}

check_extensions_file() {
local extensions_file="/root/pg_extensions.json"
if [ ! -f "$extensions_file" ]; then
log "extensions: No extensions file found, skipping extensions versions check"
return 1
fi
if [ ! -r "$extensions_file" ]; then
log "extensions: Cannot read extensions file"
return 1
fi
return 0
}

get_pg_cron_version() {
if ! check_extensions_file; then
return
fi

local version
version=$(jq -r '.pg_cron // empty' "/root/pg_extensions.json")
if [ -z "$version" ]; then
log "pg_cron: Not specified in extensions file"
return
fi

if ! [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
log "pg_cron: Invalid version format: $version"
return
fi

# Log the version but don't include it in the output
log "pg_cron: Found version $version in extensions file" >&2
printf "%s" "$version"
}

switch_pg_cron_version() {
local version="$1"
local switch_script="/var/lib/postgresql/.nix-profile/bin/switch_pg_cron_version"

if [ ! -x "$switch_script" ]; then
log "pg_cron: No version switch script available at $switch_script"
return 1
fi

log "pg_cron: Switching to version $version"
# Run directly as root since we're already running as root
"$switch_script" "$version"
local exit_code=$?
if [ $exit_code -eq 0 ]; then
log "pg_cron: Version switch completed successfully"
else
log "pg_cron: Version switch failed with exit code $exit_code"
fi
return $exit_code
}

handle_pg_cron_version() {
local version
version=$(get_pg_cron_version)
if [ -n "$version" ]; then
# Don't fail if version switch fails
switch_pg_cron_version "$version" || log "pg_cron: Version switch failed but continuing"
fi
}

main() {
log "Starting prestart script"

# 1. pg_cron version handling
handle_pg_cron_version

# 2. orioledb handling
local has_orioledb=$(check_orioledb_enabled)
if [ "$has_orioledb" -lt 1 ]; then
return 0
Expand All @@ -35,6 +112,8 @@ main() {
if [ ! -z "$shared_buffers_value" ]; then
update_orioledb_buffers "$shared_buffers_value"
fi

log "Prestart script completed"
}

# Initial locale setup
Expand Down
2 changes: 2 additions & 0 deletions ansible/files/postgresql_config/postgresql.service.j2
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ TimeoutStopSec=90
TimeoutStartSec=86400
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal
OOMScoreAdjust=-1000
EnvironmentFile=-/etc/environment.d/postgresql.env
LimitNOFILE=16384
Expand Down
6 changes: 6 additions & 0 deletions ansible/tasks/stage2-setup-postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,12 @@
shell: |
sudo -u postgres bash -c ". /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh && nix profile install github:supabase/postgres/{{ git_commit_sha }}#{{postgresql_version}}_src"
when: stage2_nix

- name: Install jq from nix binary cache
become: yes
shell: |
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh && nix profile install nixpkgs#jq --profile /nix/var/nix/profiles/default
when: stage2_nix

- name: Set ownership and permissions for /etc/ssl/private
become: yes
Expand Down
6 changes: 3 additions & 3 deletions ansible/vars.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ postgres_major:

# Full version strings for each major version
postgres_release:
postgresorioledb-17: "17.0.1.085-orioledb"
postgres17: "17.4.1.035"
postgres15: "15.8.1.092"
postgresorioledb-17: "17.0.1.067-orioledb-pgcron-8"
postgres17: "17.4.1.017-pgcron-8"
postgres15: "15.8.1.074-pgcron-8"

# Non Postgres Extensions
pgbouncer_release: "1.19.0"
Expand Down
Loading
Loading