Terraform Test on Object Doesn't Match

I have the following assertion in a test:

  assert {
    condition = module.helper_acm_validation.certificate_domains == { "test-app.constr.manual.foobar.com" = [] }
    error_message = "var.subdomains was not included in the certificate"
  }

I’ve run the module with terraform plan to check the real value of module.helper_acm_validation.certificate_domains, and it is indeed { "test-app.constr.manual.foobar.com" = [] }. Running this test however, results in the assertion failure:

│ Error: Test assertion failed
│ 
│   on tests/unit.tftest.hcl line 23, in run "application_workspaces":
│   23:     condition = module.helper_acm_validation.certificate_domains == { "test-app.constr.manual.foobar.com" = [] }
│     ├────────────────
│     │ module.helper_acm_validation.certificate_domains is map of list of string with 1 element
│ 
│ var.subdomains was not included in the certificate

If I change the assertion to test for:

condition     = keys(module.helper_acm_validation.certificate_domains)[0] == "test-app.constr.manual.foobar.com"

it results in a successful test.

Similarly, testing for the value of module.helper_acm_validation.certificate_domains["test-app.constr.manual.foobar.com"] to be [] is also a success.

So the constituent parts match, but when I compare the entire object it’s a fail.

Hi @afrazo,

The error message suggests that certificate_domains has type map(list(string)), but the expression you are comparing it to is using an object constructor with a tuple constructor inside it, and so has the following type:

object({
  "test-app.constr.manual.foobar.com" = tuple([])
})

The == operator has no context to use to infer what types you intend and so, unlike some other language features, it can only work if the two values have identical types and identical values.

I don’t think it’s really possible to use the == operator in this case because there is no way to describe an empty list(string) value directly as a constant expression. The closest would be tolist([]) but that doesn’t specify the element type, so it would produce a list of an unknown element type and thus would also not match.

I do tend to recommend writing more specific conditions like your second example anyway, because then it’s clearer exactly what characteristic of the value is being tested. But I suppose one tricky way to create an empty list of string in an expression would be something like this:

slice(tolist([""]), 0, 0)

Since this function starts with a list of strings and then effectively discards all of the elements, Terraform should be able to infer that the result has string as the element type even though it has length zero. I don’t love this answer, but it’s something I expect to work in today’s Terraform.

Makes sense. I’ve written a bunch of assertions testing every value, and I’m kind of okay with this, since I agree it can be useful to know exactly which part isn’t matching.

It would still be nice if there was a way to compare values in the way that we expect based on experience with setting them. i.e. When we say foo = { "bar" = [] } in a resource declaration, it looks like we’re assigning a map of lists to “foo”, especially when the type of foo is explicitly set to map(list(string)) (otherwise it should throw an error, no?).

Hi @afrazo,

Indeed, in many other contexts we can just imagine that { ... } produces maps and [ ... ] produces lists because Terraform will convert automatically when it has enough information to do so.

In particular, if you are assigning to an argument in a resource block then the provider specifies what type it’s expecting, and if you are assigning to an argument in a module block then the corresponding variable block often specifies what type it’s expecting, and so in both cases Terraform just quietly takes care of it for you, assuming that the element types are valid for the type constraint.

Although this implicit typing serves its intended purpose of making it more convenient to do the most common thing in Terraform – configuring resources – it does have these knock-on consequences on other parts of the language, and the == operator is a particularly tricky case because (as I was saying before) it gives absolutely no information about what types are expected on either side of it, and so it must rely on the author to perform any type conversions manually.

I do agree that there’s some room for improving the language here, but I’m not sure yet what’s best to do. The most general solution would be to implement a convert function that lets you just write out exactly the type you’re expecting:

  # INVALID: this is just a hypothetical language design idea
convert(value, map(list(string)))

…and during earlier Terraform v0.12 development I did implement such a function – its implementation is even still present in the HCL repository – but it ran into Terraform’s historical decision to assume that any unrecognized symbol is intended to be a resource type name and the start of a reference to a resource, and so Terraform forces interpreting the string in the above as the start of a reference to a hypothetical resource "string" "something" and so rejects it as invalid before the function even gets to run. I would like to fix that someday, but have not yet found a good way to do it without breaking the Terraform v1.x compatibility promises.

Another possibility we considered is something condition-block-specific – where assert is an example of a “condition block”, but also precondition, postcondition, etc, for dealing with the common case of comparing a result to an expected value, which could potentially optionally allow also specifying the type that’s expected so that Terraform can explicitly check that, and can convert the want value to that type if both are specified:

  # INVALID: this is just a hypothetical language design idea
  assert {
    # "match" could be used instead of "condition"
    match {
      got       = module.helper_acm_validation.certificate_domains
      want      = { "test-app.constr.manual.foobar.com" = [] }
      want_type = map(list(string))
    }
    error_message = "Invalid value for certificate_domains output."
  }

I would imagine the rule for the above being that Terraform will first test whether got’s type matches want_type, returning an error immediately if not, but would then convert want to that type and test whether that converted value is equal to got, using the same rules as the == operator but with the types matching.

It would also be valid to use want_type alone to check that the type is what was expected without considering the value, or to use want alone in which case it would be like want_type = any and you’d get the same situation you encountered with your original question, but it would work fine in the simple case of just comparing two primitive-typed values, like two strings.

This condition-block-specific answer is at least not blocked by historical language design problems, so has a more likely path to success but does still need to overcome the usual concern of it overlapping with part of the functionality of an existing pattern and thus creating more than one way to do something, which causes uncertainty for new authors as they try to figure out which one to use. It’s probably worth it for providing an alternative to this common gotcha, though.