Terraform destroys resources before updating dependent resources

Terraform
1

Terraform destroys resources before updating dependent resources

Hi, suppose you have the folowing desired state:

resource "foo" "my_foo_1" {
  name = "foo1"
}

resource "foo" "my_foo_2" {
  name = "foo2"
}

resource "bar" "my_bar" {
  foo = foo.my_foo_1.id
}

You do terraform apply and my_foo_1, my_foo_2 and my_bar are created.

Then you change your desired state to this:

# Removed my_foo_1

resource "foo" "my_foo_2" {
  name = "foo2"
}

resource "bar" "my_bar" {
  foo = foo.my_foo_2.id # Reference changed
}

If you do terraform apply now, Terraform should:

  • Update my_bar, pointing to my_foo_2 instead of my_foo_1 (note: assume this is an in-place update and not a replacement)
  • Delete my_foo_1

But the order if these actions is crucial: the update has to be done before the delete, otherwise the backend API will complain that my_foo_1 is referenced by some other resource (i.e. my_bar) so it cannot be deleted.

What happens, though, is that Terraform starts by destroying my_foo_1 and obviously fails.

When I noticed this behavior I was really surprised, because my_bar is clearly dependent on my_foo_1, so I don’t understand why Terraform destroys my_foo_1 before updating my_bar.

My best guess is that there are situations where this behavior makes sense.

But I think that the scenario I’m describing is pretty common and reasonable. After all, it’s just moving from a valid desired state to another perfectly valid desired state.

Of course I could solve the issue by introducing an intermediate state where my_foo_1 is still present but my_bar doesn’t reference it anymore:

resource "foo" "my_foo_1" {
  name = "foo1"
}

resource "foo" "my_foo_2" {
  name = "foo2"
}

resource "bar" "my_bar" {
  foo = foo.my_foo_2.id # Reference changed
}

But this it’s just a workaround and it kinda breaks the declarative paradigm by forcing me to tell Terraform how to order the updates to my infrastructure.

Is there a way to make Terraform behave properly that doesn’t involve breaking the transition in two steps / two terraform apply executions?

Assume that I’m the developer of the provider defining foo and bar resources, so I accept solutions that involve modifying the provider.

2

Sorry, I’m realizing that this is a duplicate of:

I initially ignored that topic because it mentioned lifecycle.create_before_destroy, which according to the documentation seems to serve a different purpose:

By default, when Terraform must change a resource argument that cannot be updated in-place due to remote API limitations, Terraform destroys the existing object and then create a new replacement object with the new configured arguments. Use the create_before_destroy rule to instruct Terraform to create a replacement resource before destroying the current resource.

In my case there are no resources that need to be destroyed and then recreated. Instead, I have:

  • a resource to be updated in-place
  • a resource to be destroyed

…and the resource to be updated in-place is currently dependent on the resource to be destroyed (but won’t be anymore after the update). So, in my case, I need the update to be executed before the destroy.

For some reason, though, lifecycle.create_before_destroy makes Terraform behave as I expect.

I think it would be useful to add something like this to the documentation:

An additional situation in which create_before_destroy may be useful is when resource A references resource B but you want to destroy B and remove A’s reference to B. For some providers, this only works if A is updated before destroying B. By default, though, Terraform destroys B first. By using create_before_destroy on A you can make Terraform update A before destroying B.

Do you think this would be a correct way to put it?

Also, would it be possible to provide a dedicated lifecycle rule for this use case? Something with a better name, e.g. update_before_destroying_dependencies.

3

The name of the option has been thought about many times over the past years, but nothing else has really stuck. You can’t omit that it will cause replacement to be create-then-destroy since that the most noticeable change, so other options have always just ended up being long and awkward (create_and_update_dependencies_before_destroy), or just another arbitrary name that needs to be learned (invert_destroy_order).

The registration pattern is well-known by existing providers, and the resources need to document that the options is required, so in the end the name almost doesn’t matter if the user doesn’t know when it needs to be used.

A bigger change would be to allow providers to set create_before_destroy directly when using this registration pattern, and that option could be documented/named as needed.