How to ignore local_file data objects

Terraform
1

Hi.

Take the following module code:

resource "random_string" "awscli_output_temp_file_name" {
  length  = 16
  special = false
}

resource "local_file" "awscli_results_file" {
  depends_on           = [random_string.awscli_output_temp_file_name]
  filename             = "${path.module}/temp/${random_string.awscli_output_temp_file_name.result}.json"
  directory_permission = "0777"
  file_permission      = "0666"
}

data "external" "awscli_program" {
  depends_on = [local_file.awscli_results_file]
  program    = ["${path.module}/scripts/awsWithAssumeRole.sh"]
  query = {
    assume_role_arn    = var.assume_role_arn
    role_session_name  = var.role_session_name
    aws_cli_commands   = join(" ", var.aws_cli_commands)
    aws_cli_query      = var.aws_cli_query
    output_file        = local_file.awscli_results_file.filename
    debug_log_filename = var.debug_log_filename
  }
}

data "local_file" "awscli_results_file" {
  depends_on = [data.external.awscli_program]
  filename   = data.external.awscli_program.query.output_file
  ignore_changes = true
}

Has been working just fine. I run the aws cli command (to get data not yet available in Terraform via the AWS Provider) and then present that data to Terraform. For example, identify which DB instance in a cluster is the reader, or how many EC2 instances are currently running in an ASG so we can launch the new ASG with the same number of instances and have no drop in capacity. I can assume a role. Or not. All wonderful. Even I say so myself (https://registry.terraform.io/modules/digitickets/cli/aws/latest).

Since Terraform 0.15.4 (maybe 0.15.3 … not sure … skipped that version), we’ve been getting (correctly so this is not a bug but a new and valid and useful feature) the following output in our plans:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # module.rds_writer.local_file.awscli_results_file has been deleted
  - resource "local_file" "awscli_results_file" {
      - directory_permission = "0777" -> null
      - file_permission      = "0666" -> null
      - filename             = ".terraform/modules/rds_writer/temp/iUc1oABQ4XYczzx3.json" -> null
      - id                   = "da39a3ee5e6b4b0d3255bfef95601890afd80709" -> null
    }

Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

So, my question, is how do I get Terraform to not “worry” about this resource? The code is run in a pipeline, so there’s no file at the beginning of the run. I know I could enforce the name of the file in some way but the file would never be in the repo so it would always be missing.

2

Hi @rquadling,

Unfortunately what you’re seeing here is an extension of the caveat included in the local_file documentation:

When working with local files, Terraform will detect the resource as having been deleted each time a configuration is applied on a new machine where the file is not present and will generate a diff to re-create it. This may cause “noise” in diffs in environments where configurations are routinely applied by many different users or within automation systems.

Terraform isn’t really designed for managing objects on the local system and so its design makes the assumption that all objects should persist from one run to the next. The local_file resource is there as a pragmatic way to let you create local files when you need to, but since it doesn’t fit within Terraform’s assumptions its behavior is often sub-optimal, and that’s true here too.

If you can’t find a different way to write your configuration such that it doesn’t need to generate a temporary file on disk then unfortunately this new messaging is not avoidable in your case. In principle you could entirely disable Terraform’s refresh step using -refresh=false, but in addition to the usual caveats that wouldn’t help in this case because then Terraform wouldn’t detect that file is missing, and would just assume the file is still there from last time.

3

Thank you very much for that feedback. I’m glad to find the “edge” of a system.

One of the values I’m attempting to read is a dynamic value from a Terraform created resource. The desired capacity is dynamic and in a blue/green deployment, you need to know the current number of EC2 instances running so you can launch the same number and swap over.

Is there any way to read a value from a resource that Terraform is going to replace (in the same plan/apply cycle)?

4

Hi @rquadling,

I’m afraid that follow-up is a bit outside of my expertise since I work on Terraform Core rather than the AWS provider, and so my AWS knowledge is pretty rusty. It sounds like you have some other system dynamically changing desired_capacity outside of Terraform but yet you want Terraform to respect that change when making future updates to the system.

I don’t have a ready solution for that situation, unfortunately; typically I would advise that each part of your system be managed exclusively by only one system, rather than trying to manage the same attributes with both Terraform and some other mechanism. But I understand that this sort of general advice is not super helpful for solving your specific problem, so hopefully someone else in this forum with more recent AWS experience can offer a more concrete suggestion.