Why update values in the state when they are ignored?

Terraform
1

I’m unsure if this is a provider specific implementation detail or more general. I would like to be able to ignore_changes and expect that not only are the ignored attributes not updated in future applies, but that the values are not written back to the state.

For instance, if I create a parameter:

resource "aws_ssm_parameter" "some_secret" {
  name  = "something"
  type  = "SecureString"
  value = "value-not-set"

  lifecycle {
    ignore_changes = all
  }
}

If I then I update the value in aws, I’d like the state to simply ignore this change. I don’t follow why it needs to be written back to the state. In this way, we could create parameters, but expect the sensitive data not to be in the state. These values could just be left at their initials values, and nothing would change functionally, and it would allow instances like these to keep some sensitive data out of the state.

2

Hi @theherk,

The state data for a managed resource serves a few different purposes:

  1. To compare the prior state with the current configuration to determine if any changes are needed.
  2. To expose data for expressions elsewhere in the module to refer to.
  3. To allow a provider to track identifying information between runs so that e.g. it can remember the server-issued identifier for a previously-created object.

The ignore_changes setting only affects the behavior of purpose 1: it tells Terraform to ignore situations where the configuration no longer matches the state, and to just keep whatever value is in the prior state.

Note that ignore_changes is a plan-time idea, but you seem to be more concerned about the “refresh” operation where Terraform updates the prior state to match the real remote objects. Terraform must update the state during the refresh step regardless of ignore_changes because of purposes 2 and 3:

  • If any expressions refer to an attribute of this resource instance then they will often need to adapt to the changed value
  • If the provider uses any subset of the attributes in the state as a source of record – for example, if one of them tracks a remote identifier that isn’t recorded in the configuration – then the provider expects to recieve its most recently returned value in the next call to plan changes.

In this particular case we can use our human intuition to realize that cases 2 and 3 are not important, but Terraform doesn’t have enough information to know that.

A future addition to Terraform’s provider protocol could potentially allow a provider to give Terraform more information to understand that (3) is not important, in principle.

However, I think (2) ends up being the most difficult requirement, because if Terraform doesn’t save a particular attribute value in the prior state and then a future configuration change introduces a new reference to that attribute from elsewhere in the configuration Terraform would suddenly now need to fetch and save an attribute it didn’t save previously, or else there would be no value to refer to.

My sense is that in order to meet your requirement of being able to omit certain attributes from the state the following would both need to be true:

  • There is some new language mechanism which has a similar syntax to ignore_changes but makes the stronger assertion that the value of that attribute will not currently or at any future time be referred to in expressions elsewhere in the configuration.
  • There is some way for a provider to distinguish between the subset of attributes that are just copies of data it can look up from the remote system at any time vs. attributes where the value in the state is the source of record, which is typically whatever information the provider would need to request the values from the first subset.

If both mechanisms described above existed, for any attribute that is both not a source of record and is declared as “not to be used elsewhere in the configuration”, Terraform could safely leave its value set as null in the state. Since Terraform would have no record of what value those attributes have in the remote system, they would get the same effective behavior as ignore_changes too, just because there would be no state to compare with and so Terraform would have no option but to ignore the configuration for those attributes.

I think the big question is whether having Terraform just create an object and totally ignore it thereafter (and not propagate its results to any other part of the configuration) is really that useful. At that point you could get an equivalent result by just using the vendor’s imperative CLI instead, because the typical reason to use Terraform is to manage changes to objects over time and/or to wire objects together by routing outputs from one into arguments of another. I expect there would still be some uses to that, but they’d need to be compelling enough to justify all of the work to change Terraform’s architecture to make the above conditions both be true.

3

As expected, a very thoughtful and thorough explanation. As it stands, ignore_changes means don’t update even if different. What I suggest for addition, as you mention is a new mechanism, is some similar flag that really means to ignore changes, in all senses of the phrase.

The challenges presented are really clear; it makes sense why this may not be a good idea. However, I still think it can be useful. A lot of times we’re wiring things together by reference rather than by specific configuration of the resource. In the example, I use an ssm parameter, because it is common for me to create a parameter and pass its path to tooling that should reference configuration in that parameter, but not by referencing the value in the state. I can give lambda environment variables with paths to ssm parameters and expect the lambda to go get that data with its own set of permissions.

I suppose implementing something like this should be based on how many others would see value in it; I can only say for sure that I would.