File modified by template, different etag calculation between plan/apply

I have a template file read as data, which I want to save in a S3 bucket. The web page file will contain the dynamic address of the deployed API.

data "template_file" "index_html" {
  template = file("${path.module}/../web/html/index.html")
  vars = {
  api_endpoint = "${aws_api_gateway_deployment.my_deployment.invoke_url}${aws_api_gateway_stage.my_stage_test.stage_name}"
  }
}
resource "aws_s3_bucket_object" "my_bucket_html" {
  bucket = aws_s3_bucket.my_bucket.bucket
  content = data.template_file.index_html.rendered
  key = "index.html"
  etag = md5(data.template_file.index_html.rendered)
  content_type = "text/html"
}

Every time I run “terraform plan” it will believe it was modified, but “terraform apply” will correctly notice it wasn’t modified. That’s annoying, even though the end result is right. I mean, it’s somehow obvious that plan cannot know the replaced value - so I assume it calculates the etag with the unchanged file, then apply has the right value and notices the etag didn’t change.

Any ideas how I could do this in a different way, which lets “terraform plan” not report changes when there are none? Thank you!

Hi @sorin-costea,

It looks like you have an interesting situation here where you know more than the AWS provider does, and so you can see that nothing actually needs to change when you replace one of those objects (either aws_api_gateway_deployment.my_deployment or aws_api_gateway_stage.my_stage_test), but the provider itself is waiting to get confirmation of that from the remote API during the apply step.

I’m not up-to-date on the details of these specific resource types and so I’m mostly going to just make general observations here, but I did notice in some quick GitHub searching that it seems like AWS provider release v1.16.0 included PR #3469, which hopefully allows you to simplify that expression to just refer to the stage’s invoke_url directly, assuming I’ve correctly understood your intent of concatenating those two together:

data "template_file" "index_html" {
  template = file("${path.module}/../web/html/index.html")
  vars = {
    api_endpoint = aws_api_gateway_stage.my_stage_test.invoke_url
  }
}

This might solve your problem, if it happened to be the aws_api_gateway_deployment object’s replacement that was causing the template result to be unknown. If you can avoid re-creating the “stage” object then invoke_url should stay known on future plans and thus allow the template result to also be known.

Reviewing the AWS provider source code I was interested to see that it’s actually the provider code itself that’s building those URLs, rather than the remote AWS API:

Given that, it might be interesting to open a feature request issue in the AWS provider repository to discuss the possibility of having the provider fill in that invoke_url attribute at the plan stage rather than at the apply stage, if the provider can see that the rest API id and stage name are both already known at that point. There might be some technical blockers for that which I’m not aware of, but I think still worth discussing it because it would make you less likely to hit this rough edge.

In the immediate term though, with the AWS provider as it exists today, I was mainly interested in looking that up to see if it would be possible to construct that invoke_url manually, using information you already have available in the configuration. It seems like the regional domain name is the only thing missing, which you could potentially construct via the aws_region data source:

data "aws_region" "current" {}

locals {
  aws_region_domain = "${data.aws_region.current.name}.amazonaws.com"
}

You can then construct the URL out of some component parts that Terraform should already know during planning:

    api_endpoint = "https://${aws_api_gateway_stage.my_stage_test.rest_api_id}.${local.aws_region_domain}/${aws_api_gateway_stage.my_stage_test.name}"

Not an ideal solution, but I think it ought to work.


By the way, the template_file data source exists primarily for use with Terraform v0.11 and earlier. Since you are using a newer version of Terraform, I’d suggest using the built-in templatefile function instead, because it’s integrated directly into the Terraform language and so it avoids installing the hashicorp/template provider plugin and it avoids some quirks that data source has as a result of being implemented outside of the main Terraform language.

You can typically replace a data "template_file" block with a local value that gives the same effect, like this:

locals {
  index_html = templatefile("${path.module}/../web/html/index.html", {
    api_endpoint = "https://${aws_api_gateway_stage.my_stage_test.rest_api_id}.${local.aws_region_domain}/${aws_api_gateway_stage.my_stage_test.name}"
  })
}

Then instead of data.template_file.index_html.rendered you can refer to local.index_html instead.

I don’t expect that this change would make any significant difference to the problem you were asking about here, though, because the same constraint applies that Terraform would need to know the value of api_endpoint in order to be able to predict the result of rendering the template.

1 Like

Thank you, I used the newer templatefile as you suggested and guess what, no changes reported anymore! So it helped :slight_smile:
(the AWS provider was already version 3.35 so fixed looong time ago, probably there’s a difference between templatefile and template_file when in the cycle they apply changes)