AWS DMS task cannot start because of missing premigration assessments

Terraform Providers AWS
1

Hi,
I’m trying to use the terraform-aws-modules/terraform-aws-dms module to export rows from an RDS database into an S3 bucket as CSV files.

After a couple of false starts I’m now at the point where all of the resources specified in that module have been created successfully, however the task cannot be started because:

│ Error: starting DMS Replication Task (rds-to-s3): operation error Database Migration Service: StartReplicationTask, https response error StatusCode: 400, RequestID: 8e51c85c-b04f-4445-aade-e98222dc5297, InvalidResourceStateFault: Test connection for replication instance poc-dms and endpoint source-database should be successful for starting the replication task
│ 
│   with module.database_migration_service.aws_dms_replication_task.this["s3_export"],
│   on .terraform/modules/database_migration_service/main.tf line 386, in resource "aws_dms_replication_task" "this":
│  386: resource "aws_dms_replication_task" "this" {
│ 
╵
Error: Process completed with exit code 1.

Example code

module "database_migration_service" {
  source  = "terraform-aws-modules/dms/aws"
  version = "~> 2.0"

   [...]lines omitted for brevity[...]

  replication_tasks = {
    s3_export = {
      migration_type      = "full-load-and-cdc"
      replication_task_id = "rds-to-s3"

      replication_task_settings = jsonencode({
        FullLoadSettings = {
          TargetTablePrepMode = "DO_NOTHING",
        },
      })

      source_endpoint_key    = "rds_source"
      start_replication_task = true

      table_mappings = jsonencode({
        rules = [{
          object-locator = {
            schema-name = "identityiq"
            table-name  = "spt_audit_event"
          }
          rule-action = "explicit"
          rule-id     = "1"
          rule-name   = "1"
          rule-type   = "selection"
        }]
      })

      target_endpoint_key = "s3_target"
    }
  }
}

I checked the source code for this module to see if there was a variable/flag to control creating a premigration assessment; there doesn’t seem to be one. The start_replication_task value only feeds directly into the start_replication_task value on the aws_dms_replication_task resource which does not document anything about a premigration assessment being required.

I also checked the provider docs, expecting to potentially see a specific aws_dms_premigration_assessment resource; there doesn’t seem to be one.

My gut feeling is that this functionality is not often used and is currently broken due to this requirement to run premigration assessments, and that either

  • the aws_dms_replication_task resource needs to do an extra API call when start_replication_task is true in order to create the premigration assessment and actually allow the task to start on successful completion

or

  • there needs to be an extra resource type to create the premigration assessment
    • the downside of a second resource is then the dependencies would mean that start_replication_task could no longer be used because this would fail (the task resource would try to start before marking as Created, but the assessment resource would need the task resource to exist first to retrieve ARNs etc). So this would likely also need a third resource in order to trigger starting an existing task, which would only run after task creation and assessment creation+completion

Any tips how I might be able to work around this? My last resort would be to set start_replication_task to false and then use local_exec provisioners to run AWS CLI commands manually, but (for obvious reasons) this is definitely not my preferred solution.

Thanks!

2

Actually, I think I solved this…

Because the error only appeared after the task was created, and the console showed the missing premigration assessments, my confirmation bias meant I focused on that.

The error is actually related to the endpoint, but because the endpoints are not used until the task is created, the error only presents when the task is created even though the config error is not with the task.

Ultimately, the information I needed was on the Endpoints page in the console, where the error message shows that I have an incorrect security group membership/rule and the DMS instance is timing out when connecting to RDS. So I need to solve this first before then trying to start the task again - where the premigration assessment may not be an issue.