Terraform Moved block chaining issue

[deitChi]

I’ve been advised that there is an unofficial feature of Terraform which allows for chained moves.

i.e.

moved {
  from = aws_route_table_association.tn
  to   = aws_route_table_association.cust
}

moved {
  from = aws_route_table_association.cust
  to   = aws_route_table_association.private
}

However I’ve been struggling to implement it, as Terraform appears to ignore my first ‘move’ in the chain.

For context, both resources had for_each set (I’ve seen a GH issue being resolved with regards to moving from a non iterated resource to a count (Cyclic dependency error when both the parent and the child module have moved block. · Issue #30208 · hashicorp/terraform · GitHub))

These are non-overlapping resources.

aws_route_table_association.tn has awsenpt.eu-west-1, transitgw.eu-west-1
aws_route_table_association.cust has foo.eu-west-1, bar.eu-west-1

I would expect the chain of custody for the resources to pass along, however Terraform successfully moves cust -> private, but ignores tn -> cust.

This can be worked around, by moving single elements directly:

moved {
  from = aws_route_table_association.tn["awsenpt.eu-west-1"]
  to   = aws_route_table_association.private["awsenpt.eu-west-1"]
}

But it would be way neater if I could combine them in one.

If it’s worth mentioning - this called from within a module.

modules/vpc
	route_table.tf
	route_table-moved.tf <-- here

Terraform 1.7.2
Provider version latest v4 AWS.

Tried:
TF 1.7.5, 1.8.x
Provider version v5

[deitChi]

Plan Output

# module.foo.aws_route_table_association.cust["foo.eu-west-1"] has moved to module.foo.aws_route_table_association.private["foo.eu-west-1"]
    resource "aws_route_table_association" "custom" {
        id             = "xxxxx"
        # (3 unchanged attributes hidden)
    }

# module.foo.aws_route_table_association.cust["bar.eu-west-1"] has moved to module.foo.aws_route_table_association.private["bar.eu-west-1"]
    resource "aws_route_table_association" "custom" {
        id             = "xxxxx"
        # (3 unchanged attributes hidden)
    }

# module.foo.aws_route_table_association.private["awsendpt.eu-west-1"] will be created
+   resource "aws_route_table_association" "private" {
+       id             = (known after apply)
+       route_table_id = "xxxxx"
+       subnet_id      = (known after apply)
    }

  # module.foo.aws_route_table_association.private["transitgw.eu-west-1"] will be created
+   resource "aws_route_table_association" "private" {
+       id             = (known after apply)
+       route_table_id = "xxxxx"
+       subnet_id      = (known after apply)
    }

  # module.foo.aws_route_table_association.tn["awsendpt.eu-west-1"] will be destroyed
  # (because aws_route_table_association.tn is not in configuration)
-   resource "aws_route_table_association" "tn" {
-       id             = "xxxxx" -> null
-       route_table_id = "xxxxx" -> null
-       subnet_id      = "xxxxx" -> null
        # (1 unchanged attribute hidden)
    }

  # module.foo.aws_route_table_association.tn["transitgw.eu-west-1"] will be destroyed
  # (because aws_route_table_association.tn is not in configuration)
-   resource "aws_route_table_association" "tn" {
-       id             = "xxxxx" -> null
-       route_table_id = "xxxxx" -> null
-       subnet_id      = "xxxxx" -> null
        # (1 unchanged attribute hidden)
    }

[deitChi]

I did have an interesting error when trying just about anything:

Cyclic dependency in move statements
The following chained move statements form a cycle, and so there is no final location to move objects to:
  - .terraform/modules/awwe-vpc-cbsotevisionnetsvcs-edge-001/modules/virtual_private_cloud/route_table-moved.tf:1,1: module.foo[*].aws_route_table_association.tn[*] → module.foo[*].aws_route_table_association.cust[*]
  - .terraform/modules/foo/modules/virtual_private_cloud/route_table-moved.tf:6,1: module.foo[*].aws_route_table_association.cust[*] → module.foo[*].aws_route_table_association.private[*]
  - vpc_edge-import.tf:1,1: module.foo.aws_route_table_association.tn["awsendpt.eu-west-1"] → module.foo.aws_route_table_association.private["awsendpt.eu-west-1"]
...
A chain of move statements must end with an address that doesn't appear in any other statements, and which typically also refers to an object still declared in the configuration.

To achieve this - I added an explicit moved block outside of the module, and it duplicated.

So Terraform is ‘trying’ to move the resources, and it orders the glob’d resources first - however still failing to move them.

[apparentlymart]

Hi @deitChi,

I’m not 100% sure from the information given, but it seems like your state already contains resource instances for both “tn” and “cust”. Is that right?

When the from and to addresses are both referring to a whole resource rather than to an individual instance of the resource, Terraform understands that as “this resource block was renamed” and so wants to move all of the instances together, as a single unit, to the new address. The chaining rule only works in this case if all of the existing instances belong to the same resource address, because otherwise Terraform will move “cust” to “private” and then try to move “tn” to “private” but fail because “private” is now already occupied by the resource formerly named “cust”, and so there is no vacant resource address to move to.

I guess you were probably hoping that Terraform would move each instance individually, rather than moving the resources as a whole. You mentioned that this works when you specify the individual instance keys separately, and indeed that causes Terraform to make a different interpretation: one individual resource instance object moved to a new resource instance address, and so as long as that target instance address is vacant the move can succeed.

In current Terraform there is no way to move individual instances of a resource without naming them explicitly. It seems like you want to be able to write a systematic rule like “for each existing instance of this resource, move it individually to be an instance of another resource with the same instance key”. If so, that’s what issue #33236 is about, although it remains unclear how exactly Terraform can support that since it currently needs to resolve all moves before evaluating any expressions and yet the use-cases described there have all tended to generalize to need some kind of expression evaluation during the move-resolution step. We intend to keep researching it to find a feasible design, though.

[deitChi]

Hi @apparentlymart,

It’s exactly as you described, we have 2 instances, ‘tn’ & ‘cust’. Each has a ‘for_each’ with non-overlapping keys (let’s blame my predecessor, or a deitChi without appreciation for well structured code).

I would like to migrate them both to an instance ‘private’, hence my approach would be to move tn → cust & cust → private.

I think terraform needs to inspect the keys of the resource instance ‘tn’, and inspect the keys of ‘cust’, if any match - raise an exception. If not merge, and follow the chain of moves.

Although since this move chaining doesn’t support multiple resources being combined, why not just do a similar operation for tn → private & cust → private.

Anyway, I’m glad we got to the bottom of it, and ultimately the feature is not supported yet.

I look forward to seeing the outcome!