Error while using pg_dump and pg_restore to transfer data from one RDS database to another

257 Views Asked by At
  • I have a TWO databases on RDS with master user postgres, master database postgres and a password set on each
  • First one has a custom user, let's call it myuser and a custom database, let's call it mydb
  • I am using a custom port 26189 if that changes anything
  • myuser has permissions to createdb and login and mydb is owned by myuser on the first database
  • I want to be able to restore the user and database INTO the second database
  • I did a pg_dump command to dump all globals and also the database
  • Postgres version = 14.9 R1 on both

This is what my backup script roughly looks like

POSTGRES_ENCODING="UTF8"
POSTGRES_BACKUP_GLOBAL_FILE_NAME="something_something_global"
POSTGRES_BACKUP_DATABASE_FILE_NAME="something_something_db"
POSTGRES_HOST="[email protected]"
POSTGRES_PORT=26189
POSTGRES_ROOT_USERNAME=postgres
POSTGRES_DATABASE_NAME=mydb
JOBS=2

pg_dumpall \
    --globals-only \
    --no-owner \
    --no-password \
    --no-role-passwords \
    --encoding="${POSTGRES_ENCODING}" \
    --file="${POSTGRES_BACKUP_GLOBAL_FILE_NAME}" \
    --host="${POSTGRES_HOST}" \
    --port="${POSTGRES_PORT}" \
    --username="${POSTGRES_ROOT_USERNAME}"

pg_dump \
    --blobs \
    --create \
    --no-password \
    --dbname="${POSTGRES_DATABASE_NAME}" \
    --encoding="${POSTGRES_ENCODING}" \
    --file="${POSTGRES_BACKUP_DATABASE_FILE_NAME}" \
    --format="directory" \
    --host="${POSTGRES_HOST}" \
    --jobs="${JOBS}" \
    --port="${POSTGRES_PORT}" \
    --username="${POSTGRES_ROOT_USERNAME}"

Now when restoring, I want to basically import all the globals I dumped with the pg_dumpall command, then I want to create the mydb database which belongs to myuser and then restore all tables inside mydb

This is what my restore script looks like

POSTGRES_ENCODING="UTF8"
POSTGRES_BACKUP_GLOBAL_FILE_NAME="something_something_global"
POSTGRES_BACKUP_DATABASE_FILE_NAME="something_something_db"
POSTGRES_HOST="[email protected]"
POSTGRES_PORT=26189
POSTGRES_ROOT_USERNAME=postgres
POSTGRES_DATABASE_NAME=mydb
JOBS=2
TARGET_DATABASE_HOST="[email protected]"
TARGET_POSTGRES_PASSWORD="somePASSWORD"
TARGET_POSTGRES_PORT=26189
POSTGRES_SSL_ROOT_CERT_PATH="home/ec2-user/cert.pem"
TARGET_POSTGRES_ROOT_USERNAME="postgres"
TARGET_POSTGRES_DATABASE_NAME="mydb"

psql \
    --quiet \
    --file="${POSTGRES_BACKUP_GLOBAL_FILE_NAME}" \
    "host=${TARGET_POSTGRES_HOST} password=${TARGET_POSTGRES_PASSWORD} port=${TARGET_POSTGRES_PORT} sslrootcert=${POSTGRES_SSL_ROOT_CERT_PATH} sslmode=verify-full user=${TARGET_POSTGRES_ROOT_USERNAME}"

PGPASSWORD="${TARGET_POSTGRES_PASSWORD}" \
createdb \
    --encoding="UTF8" \
    --host="${TARGET_POSTGRES_HOST}" \
    --owner="${TARGET_POSTGRES_USERNAME}" \
    --port="${TARGET_POSTGRES_PORT}" \
    --username="${TARGET_POSTGRES_ROOT_USERNAME}" \
    "${TARGET_POSTGRES_DATABASE_NAME}"

PGPASSWORD="${TARGET_POSTGRES_PASSWORD}" \
pg_restore \
    --disable-triggers \
    --exit-on-error \
    --format=directory \
    --dbname="${TARGET_POSTGRES_DATABASE_NAME}" \
    --host="${TARGET_POSTGRES_HOST}" \
    --jobs="${JOBS}" \
    --port="${TARGET_POSTGRES_PORT}" \
    --username="${TARGET_POSTGRES_USERNAME}" \
    "/home/${POSTGRES_BACKUP_DATABASE_FILE_NAME}"

When I run this restore script, it is not working. IT gives me a lot of errors in the first step when I run the psql command from restore script

psql:globals.dump:16: ERROR:  role "ec2-user" already exists
psql:globals.dump:17: ERROR:  must be superuser to alter superuser roles or change superuser attribute
psql:globals.dump:18: ERROR:  role "postgres" already exists
psql:globals.dump:19: ERROR:  must be superuser to alter superuser roles or change superuser attribute
psql:globals.dump:20: ERROR:  role "rds_ad" already exists
psql:globals.dump:21: ERROR:  cannot alter "rds_ad"
DETAIL:  "rds_ad" is a protected role.
psql:globals.dump:22: ERROR:  role "rds_iam" already exists
psql:globals.dump:23: ERROR:  cannot alter "rds_iam"
DETAIL:  "rds_iam" is a protected role.
psql:globals.dump:24: ERROR:  role "rds_password" already exists
psql:globals.dump:25: ERROR:  cannot alter "rds_password"
DETAIL:  "rds_password" is a protected role.
psql:globals.dump:26: ERROR:  role "rds_replication" already exists
psql:globals.dump:27: ERROR:  cannot alter "rds_replication"
DETAIL:  "rds_replication" is a protected role.
psql:globals.dump:28: ERROR:  role "rds_superuser" already exists
psql:globals.dump:29: ERROR:  cannot alter "rds_superuser"
DETAIL:  "rds_superuser" is a protected role.
psql:globals.dump:30: ERROR:  role "rdsadmin" already exists
psql:globals.dump:31: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:32: ERROR:  role "rdsrepladmin" already exists
psql:globals.dump:33: ERROR:  cannot alter "rdsrepladmin"
DETAIL:  "rdsrepladmin" is a protected role.
psql:globals.dump:34: ERROR:  role "rdstopmgr" already exists
psql:globals.dump:35: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:44: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:49: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:54: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:59: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:64: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:69: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:74: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:79: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:84: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:89: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:94: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:99: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:104: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:109: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:114: ERROR:  cannot alter "rdsadmin"
DETAIL:  "rdsadmin" is a protected role.
psql:globals.dump:123: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:128: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:133: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:138: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:143: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:148: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:153: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:158: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:163: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:168: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:173: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:178: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:183: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:190: ERROR:  cannot alter "rds_superuser"
DETAIL:  "rds_superuser" is a protected role.
psql:globals.dump:191: ERROR:  cannot alter "rdstopmgr"
DETAIL:  "rdstopmgr" is a protected role.
psql:globals.dump:192: ERROR:  cannot alter "rds_superuser"
DETAIL:  "rds_superuser" is a protected role.
psql:globals.dump:193: ERROR:  cannot alter "rds_superuser"
DETAIL:  "rds_superuser" is a protected role.
psql:globals.dump:194: ERROR:  cannot alter "rds_superuser"
DETAIL:  "rds_superuser" is a protected role.
psql:globals.dump:195: ERROR:  must be superuser to set grantor
psql:globals.dump:202: ERROR:  cannot create rds_temp_tablespace

The second line from restore which is the createdb command also fails

createdb: error: database creation failed: ERROR: must be member of role "myuser"

All I want to do is transfer globals from database 1 to datbase 2 and create the user and create the database assigned to this user and be able to login with this database and user into the second database

0

There are 0 best solutions below