Skip to content

chore(deploy): configure PostgreSQL shared buffers per environment - #1747

Merged
rdimitrov merged 1 commit into
mainfrom
codex/configure-postgres-buffers
Oct 26, 2026
Merged

rdimitrov merged 1 commit into
mainfrom
codex/configure-postgres-buffers

Conversation

@rdimitrov

@rdimitrov rdimitrov commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Production incremental-sync traffic is causing heavy shared-buffer churn and BufferMapping contention with PostgreSQL's default 128 MB pool. Add an optional per-stack postgresSharedBuffers setting and exercise 512MB in staging before a separate production promotion. Unset stacks retain the PostgreSQL default; node capacity, CPU/memory resources, connection limits, and Grafana are unchanged.

Document memory budgeting, the singleton database restart, rollback, and workload-aware verification. The production target is a buffer-only experiment; it is not a claim that buffers alone resolve CPU contention.

Validation: make -C deploy build, git diff --check, independent review, and all CI checks passed. Full refreshed previews succeeded, including the committed staging head: staging changes only the intended registry database parameter plus deployment housekeeping; production has no database or node change. Production promotion waits for staging restart, SHOW shared_buffers, and readiness verification.

@rdimitrov
rdimitrov merged commit 054140b into main Oct 26, 2026
9 checks passed
@rdimitrov
rdimitrov deleted the codex/configure-postgres-buffers branch October 26, 2026 17:53
rdimitrov added a commit that referenced this pull request Oct 26, 2026
Set production registry PostgreSQL `shared_buffers` to `512MB` using the
per-stack setting from #1747. Current incremental-sync traffic causes
substantial normal-buffer evictions and BufferMapping contention with
the default 128 MB pool; the servers relation footprint including
indexes and TOAST is about 299 MiB.

This isolates the buffer change. Production nodes remain e2-standard-4,
the app stays on 1.8.5, and CPU/memory resources, connection limits,
indexes, and cache policy are unchanged. It adds 384 MiB of configured
buffers and restarts the singleton database; rollback also restarts it.

Validation: all CI checks and independent review passed. The full
refreshed preview of head 83e34bf shows only `shared_buffers=512MB` plus
the acknowledged app deployment annotation, admission jobs, and Grafana
RBAC normalization. Staging deployment 40372902793 succeeded; SQL
confirms `512MB` with zero pending-restart settings, both app replicas
are ready, and direct listing, incremental-sync, and search smoke
requests returned 200. Staging nodes remain e2-standard-2.

Production memory peak is about 1.1 GiB against a 4 GiB limit with no
OOM events and ample node memory. The recent successful backup was
previously verified. After recovery/warm-up, compare 15 minutes of
workload-specific query rates, buffer evictions, waits, CPU pressure,
queues, errors, and listing latency. Restart-cleared backlog alone is
not evidence of a sustained fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant