Console format broken in 0.14?

Terraform
1

We use the below console command to retrieve some Terraform variable values in our deployment pipeline. When upgrading to Terraform 0.14, this stopped working, and now just returns nothing at all. I cannot find any information in the upgrade doc about console or functions being changed or deprecated, so I’m not sure why this stopped working in 0.14. So far I’ve tried 0.14.5 and 0.14.8 (latest at the time of this writing).

In the example below, this variable holds the AWS account ID, but any variable you use behaves the same way.

echo 'format("var.account_id=%s", var.account_id)' | terraform console

The below command works fine, which seems to point to the “format” function specifically:

echo 'var.account_id' | terraform console

Any help would be greatly appreciated.

2

would it work to output the variables and use terraform output (Command: output - Terraform by HashiCorp) to get the value after apply in you case? do you need it before apply?

3

We do need it before apply - mostly for the very first build, when it won’t exist in output data yet. We spin up new environments, which are like a clean/blank start each time, so no outputs would exist for the new environment until an apply is completed.

I know we could define these things in shell scripts or other ways too, but we are trying to avoid defining the same variable in multiple places.

4

How do you pass input variables to your TF? Environment? Command line arguments? .tfvars file?

I was thinking that with a little reorganisation, you might be able use .tfvars json input file to feed that to TF and jq to read values from the same file in your pipeline scripts (Input Variables - Configuration Language - Terraform by HashiCorp).

5

Ya, that might work, but we have a lot of variables and some are maps and arrays, so editing the tfvars file gets a lot harder when it’s json. Currently we have regular (non-json) tfvars.

I’m now looking for a way to programmatically convert the standard tfvars file into json.

But ideally if the console/format function didn’t break with 0.14, none of these workarounds would even be needed. So my hope is that there is a more direct workaround that can enable the original console command to work with a small tweak - instead of refactoring our entire solution to work around the breakage.

6

The console approach seems riskier to me, I don’t think there is a stability guarantee. It might keep happening. Can you inject GitHub - tmccombs/hcl2json: Convert hcl2 to json into your build agent?

just tried this:

ohmer@luigi ~/w/t/vars> hcl2json/hcl2json < test.tfvars | jq .
{
  "var1": "value1",
  "var2": [
    "item1",
    "item2"
  ]
}
ohmer@luigi ~/w/t/vars> hcl2json/hcl2json < test.tfvars | jq .var1
"value1"
ohmer@luigi ~/w/t/vars> hcl2json/hcl2json < test.tfvars | jq -r .var1
value1
ohmer@luigi ~/w/t/vars> cat test.tfvars
var1 = "value1"
var2 = ["item1", "item2"]
ohmer@luigi ~/w/t/vars> hcl2json/hcl2json < test.tfvars | jq -r .var2
[
  "item1",
  "item2"
]
7

There isn’t any built-in way to do this in Terraform. But here’s a really simple command-line tool that should do that for you: tfvars-json

It renders the JSON version of your .tfvars file to standard out, so it’s well-suited for this use case. Combining it with jq as @ohmer suggested should work well, I hope!

8

I should also add that the original issue you describe here sounds like a bug in Terraform. I haven’t been able to reproduce it locally, so if you can please consider filing an issue on GitHub.

9

I feel unsure about this because this use of terraform console isn’t supported in the way you intend to use it, but it should still be producing some sort of output explaining that.

The reason it can’t work exactly like this is because terraform console renders values derived from the state snapshot created by the latest apply, not values from a particular plan. Since root variable values are a property of a plan rather than something in the configuration or in the state, terraform console can’t “see” variable values, and so var.foo-type expressions will return “unknown value” placeholders.

However, terraform console is intended to produce a specialized error message for the situation where the result value is unknown, rather than returning nothing at all, and so if it isn’t returning an error message then I would consider that to be a bug.

Some earlier versions of Terraform would, arguably incorrectly, pick up default values for variables in terraform console and assume those would always be the values to use in expressions, but that changed several versions ago (I think it was in v0.12) because that behavior was unreliable, inconsistent, and not intended, and folks frequently asked questions that implied that they were confused by it.

One thing we have considered before is a special mode for terraform console where you can ask it to evaluate expressions in the context of a particular saved plan rather than in the context of the prior state snapshot, which could perhaps be a different way to get a result like you wanted here, although it’s not trivial to implement and it’s been difficult to prioritize it when terraform console isn’t really intended to be part of any routine Terraform workflow – it’s primarily there as a command to help with development and debugging.

1 Like