infra/home/ansible/postgresql-upgrade-manual-steps.md

Manual PostgreSQL 15 to 17 Upgrade Steps

Prerequisites

• SSH into Proxmox host: ssh root@10.0.16.231 (or appropriate host)
• Access PostgreSQL container: pct enter 300

Step 1: Pre-upgrade Checks

# Check current version
sudo -u postgres psql -t -c "SELECT version();"

# List current clusters
pg_lsclusters

# Check disk space
df -h /var/lib/postgresql

# List databases and sizes
sudo -u postgres psql -t -c "SELECT pg_database.datname, pg_size_pretty(pg_database_size(pg_database.datname)) FROM pg_database WHERE datname NOT IN ('template0', 'template1') ORDER BY pg_database_size(pg_database.datname) DESC;"

# Check active connections
sudo -u postgres psql -t -c "SELECT count(*) as connections, datname FROM pg_stat_activity GROUP BY datname ORDER BY connections DESC;"

Step 2: Backup Current Data

# Create backup directory
mkdir -p /var/backups/postgresql-upgrade-$(date +%s)
cd /var/backups/postgresql-upgrade-$(date +%s)

# Backup all databases
sudo -u postgres pg_dumpall > all_databases.sql

# Backup configuration files
cp /etc/postgresql/15/main/postgresql.conf postgresql.conf.backup
cp /etc/postgresql/15/main/pg_hba.conf pg_hba.conf.backup

# Note important settings from postgresql.conf
grep -E "listen_addresses|max_connections|shared_buffers|effective_cache_size|work_mem" /etc/postgresql/15/main/postgresql.conf

Step 3: Install PostgreSQL 17

# Add PostgreSQL APT repository
apt-get update
apt-get install -y wget ca-certificates
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
echo "deb http://apt.postgresql.org/pub/repos/apt/ bookworm-pgdg main" > /etc/apt/sources.list.d/pgdg.list
apt-get update

# Install PostgreSQL 17
apt-get install -y postgresql-17 postgresql-client-17 postgresql-contrib-17

# If using PostGIS
apt-get install -y postgresql-17-postgis-3

Step 4: Stop Services

# Stop all PostgreSQL services
systemctl stop postgresql

# Verify services are stopped
systemctl status postgresql

Step 5: Perform the Upgrade

# Check clusters again
pg_lsclusters

# Drop the empty PostgreSQL 17 cluster (if it was auto-created)
pg_dropcluster 17 main --stop

# Upgrade the cluster
pg_upgradecluster 15 main

# This will:
# - Create new cluster with PostgreSQL 17
# - Migrate all data
# - Update ports (old cluster moves to 5433, new stays on 5432)

Step 6: Configure PostgreSQL 17

# Edit postgresql.conf
nano /etc/postgresql/17/main/postgresql.conf

# Update these settings:
# listen_addresses = '*'
# max_connections = 200
# shared_buffers = 2GB
# effective_cache_size = 6GB
# work_mem = 16MB

# Edit pg_hba.conf for network access
nano /etc/postgresql/17/main/pg_hba.conf

# Add these lines:
# host    all             all             10.0.19.0/24            scram-sha-256
# host    all             all             10.0.0.0/8              scram-sha-256

Step 7: Start and Verify

# Start PostgreSQL 17
systemctl start postgresql@17-main
systemctl enable postgresql@17-main

# Check status
systemctl status postgresql@17-main

# Verify version
sudo -u postgres psql -p 5432 -t -c "SELECT version();"

# Check clusters
pg_lsclusters

# Test connection
sudo -u postgres psql -l

Step 8: Post-upgrade Tasks

# Update PostGIS extension (if used)
sudo -u postgres psql -d forgejo -c "ALTER EXTENSION postgis UPDATE;"

# Run ANALYZE on all databases for optimizer statistics
for db in $(sudo -u postgres psql -t -c "SELECT datname FROM pg_database WHERE datname NOT IN ('template0', 'template1');"); do
  echo "Analyzing $db..."
  sudo -u postgres psql -d $db -c 'ANALYZE;'
done

# Test remote connectivity
# From another host:
# psql -h 10.0.19.5 -U forgejo -d forgejo

Step 9: Cleanup (After Verification)

# Once everything is confirmed working:

# Stop old cluster
systemctl stop postgresql@15-main

# Optional: Remove old cluster (only after thorough testing!)
# pg_dropcluster 15 main

# Remove old packages (only after thorough testing!)
# apt-get remove postgresql-15 postgresql-client-15 postgresql-contrib-15

Important Notes

• The old PostgreSQL 15 cluster will remain on port 5433
• The new PostgreSQL 17 cluster will be on port 5432
• Ensure all applications are tested before removing the old cluster
• Keep backups for at least 30 days after upgrade
• Monitor logs: tail -f /var/log/postgresql/postgresql-17-main.log

Rollback Plan

If issues occur:

# Stop PostgreSQL 17
systemctl stop postgresql@17-main

# Change PostgreSQL 15 back to port 5432
sed -i 's/port = 5433/port = 5432/' /etc/postgresql/15/main/postgresql.conf

# Start PostgreSQL 15
systemctl start postgresql@15-main

# Update applications to point back to PostgreSQL 15