postgresql - 使用 AWS 数据迁移服务 (DMS) 从 Heroku PostgreSQL 迁移到 AWS RDS 不工作

Question

我正在尝试使用 AWS DMS 将数据从 Heroku PostgreSQL 数据库复制和复制到 AWS RDS PostgreSQL 实例，但它不起作用(更多信息见下文)。

在 DMS 日志中，我可以看到以下消息:

2017-10-25T17:43:35 [SOURCE_CAPTURE ]E: RetCode: SQL_ERROR SqlState: 42501 NativeError: 1 Message: ERROR: must be superuser or replication role to use replication slots;, Error while executing the query [1022502] (ar_odbc_stmt.c:2421)

Heroku PostgreSQL 的用户不是 super 用户或具有复制角色，但我在 2 天前使用相同的源数据库进行了实验并且它工作正常(我停止了实验以创建更大的 RDS 实例目标)。我想这次我忘记了一些配置。

• 来源:Heroku PostgreSQL standard-2 (9.5.8)
• 目标:位于 sa-east-1 区域且 rds.logical_replication=1 的 AWS RDS PostgreSQL (9.6.3)
• DMS:us-east-1 区域的实例

完整的 Cloudwatch 日志:

2017-10-25T17:43:27 [TASK_MANAGER ]I: Task 'WKJYBNDMNIRS3AEA77T5BP5RN4' running full load and CDC in resume mode after recoverable error, retry #3 (replicationtask.c:1078)
2017-10-25T17:43:27 [TASK_MANAGER ]I: Task Id: bb955197-648a-4177-8653-677b5daf5a6e (replicationtask.c:2682)
2017-10-25T17:43:30 [TASK_MANAGER ]I: Creating threads for all components (replicationtask.c:1664)
2017-10-25T17:43:31 [TASK_MANAGER ]I: Threads for all components were created (replicationtask.c:1832)
2017-10-25T17:43:31 [TASK_MANAGER ]I: Task initialization completed successfully (replicationtask.c:2741)
2017-10-25T17:43:31 [TARGET_APPLY ]I: Going to connect to ODBC connection string: PROTOCOL=7.4-0;DRIVER={PostgreSQL};SERVER=XXXXX.rds.amazonaws.com;DATABASE=XXXXX;PORT=5432;sslmode=verify-ca;pqopt=sslrootcert=/rdsdbdata/rds-metadata/certificates/QQSFSDJTBLFXQLQ6VRVDDAYKCM/dms-ca-cert.pem;UID=USERID; (odbc_endpoint_imp.c:2236)
2017-10-25T17:43:31 [TARGET_APPLY ]I: Target endpoint 'PostgreSQL' is using provider syntax 'PostgreSQLLinux' (provider_syntax_manager.c:589)
2017-10-25T17:43:31 [TARGET_APPLY ]I: Transaction size is 10000. Array size is 1000. (odbc_endpoint_imp.c:837)
2017-10-25T17:43:31 [TARGET_APPLY ]I: CSV files will not be kept (odbc_endpoint_imp.c:1519)
2017-10-25T17:43:31 [TARGET_APPLY ]I: Error CSV files will not be kept (odbc_endpoint_imp.c:1528)
2017-10-25T17:43:31 [TARGET_APPLY ]I: Working in transactional apply mode (endpointshell.c:1436)
2017-10-25T17:43:31 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=SERVER_AT_HEROKU.compute-1.amazonaws.com;Port=5432;Database=XXXX;UseDeclareFetch=1;Fetch=10000;Uid=USERID;Pwd=PASSWORD;sslmode=require (postgres_endpoint_util.c:252)
2017-10-25T17:43:31 [SOURCE_CAPTURE ]I: Source endpoint 'PostgreSQL Source' is using provider syntax '<default>' (provider_syntax_manager.c:583)
2017-10-25T17:43:34 [TASK_MANAGER ]I: All stream components were initialized (replicationtask.c:2547)
2017-10-25T17:43:34 [SOURCE_CAPTURE ]I: Initial position is set to 'now' (postgres_endpoint_util.c:343)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]I: Replication slot 'wkjybndmnirs3aea_00016385_bb955197_648a_4177_8653_677b5daf5a6e' does NOT exist. (postgres_endpoint_capture.c:293)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]I: Initial positioning requested is 'now' (postgres_endpoint_capture.c:313)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]E: Failed to execute statement: 'SELECT xlog_position FROM pg_create_logical_replication_slot('wkjybndmnirs3aea_00016385_bb955197_648a_4177_8653_677b5daf5a6e', 'test_decoding')' [1022502] (ar_odbc_stmt.c:2415)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]E: RetCode: SQL_ERROR SqlState: 42501 NativeError: 1 Message: ERROR: must be superuser or replication role to use replication slots;, Error while executing the query [1022502] (ar_odbc_stmt.c:2421)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]E: Creating replication slot 'wkjybndmnirs3aea_00016385_bb955197_648a_4177_8653_677b5daf5a6e' failed. [1022502] (postgres_endpoint_capture.c:334)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]E: Error executing source loop [1022502] (streamcomponent.c:1581)
2017-10-25T17:43:35 [TASK_MANAGER ]E: Task error notification received from subtask 0, thread 0 [1022502] (replicationtask.c:2375)
2017-10-25T17:43:35 [TASK_MANAGER ]E: Stream component failed at subtask 0, component st_0_QVUF6NET6KSB26SCIIRH4UQZAQ [1022502] (subtask.c:1350)
2017-10-25T17:43:35 [TASK_MANAGER ]W: Task 'WKJYBNDMNIRS3AEA77T5BP5RN4' encountered a recoverable error (repository.c:4749)
2017-10-25T17:43:35 [SOURCE_CAPTURE ]E: Stream component 'st_0_QVUF6NET6KSB26SCIIRH4UQZAQ' terminated [1022502] (subtask.c:1513)
2017-10-25T17:43:40 [TASK_MANAGER ]I: Subtask #0 ended (replicationtask_util.c:937)
2017-10-25T17:43:40 [TASK_MANAGER ]I: Task management thread terminated (replicationtask.c:3210)

Answer 1

The prereqs for DMS声明源数据库的用户需要具有 super 用户属性； Heroku 不允许这样做。

https://devcenter.heroku.com/articles/heroku-postgresql