Wrong type-checking with conditionals?

Terraform
1

Hello here !

I have struggled against what looks like an unexpected behavior if not a bug. Can you give me your opinion if I should raise a bug?

Here is a small reproducible case with latest terraform 1.13.3 (same with 1.11.4 and 1.12.2) main.tf:

locals {
  dbk = null
  config_by_env_name = {
    dev = local.dbk != null ? {} : null
  }
}

module "sub" {
  source = "./sub"

  config_by_env_name = local.config_by_env_name
}

Then sub module sub/main.tf is just:

variable "config_by_env_name" {
  type = map(object({
    option = string
  }))
}

This mini code does nothing but should be passing through terraform apply.
Unfortunately, I’m hitting this ugly type-checking error (would have expected that null is a valid value for the expected input):

│ Error: Invalid value for input variable
│
│   on main.tf line 28, in module "sub":
│   28:   config_by_env_name = local.config_by_env_name
│
│ The given value is not suitable for module.sub.var.config_by_env_name
│ declared at sub\variables.tf:1,1-30: element "dev": attribute "option" is
│ required.

Do note that local.config_by_env_nameis equals to {dev = null}.
And when replacing config_by_env_name = {dev = local.dbk != null ? {} : null}by its real value {dev = null}it’s working as exepected.

Any way I can circumvet this?

2

The error is because you are assigning a map key of dev with a null value, but the type of that value is an object with required attributes. If you truly want a null value within the map, you could define the object such that the attributes are optional.

If you don’t want that key to be used within the map at all, the solution is to filter out the null values, so that the key is never seen, maybe something like

config_by_env_name = { for k, v in local.config_by_env_name : k=>v if v != null }

There’s no way in a variable type declaration to define whether a particular value is nullable or not, other than as an attribute, so within the map it seems that the implementation ended up making the object required if any attributes are required.

3

Hello @jbardin

Thanks for your reply.
However I do need this key to be present to know that there is no defined value.

For me, null is a valid value for any definition of object.

I have tested the exact same code by adding an optional attribute within the object and the same error appeared.

variable "config_by_env_name" {
  type = map(object({
    mandatory_attr = string
    optional_attr  = optional(string)
  }))
}

brings to (after a terraform apply):

│ Error: Invalid value for input variable
│
│   on main.tf line 27, in module "sub":
│   27:   config_by_env_name = local.config_by_env_name
│
│ The given value is not suitable for module.sub.var.config_by_env_name
│ declared at sub\variables.tf:1,1-30: element "dev": attribute
│ "mandatory_attr" is required.
4

Adding an optional attribute doesn’t change the behavior when there is still a required attribute in the object definition. Your conditional does not result in the same value as your literal; if you pass in the literal equivalent of the conditional using {} you get the same error

config_by_env_name {dev = {}}

This is generally a problem with using {} which is an explicit empty object, so it’s typically not going be useful for anything unless you are passing it into a context where it can be automatically converted into a map.

If you do need a type specified, I would probably suggest defining that type fully in an input variable, and using that as the optional value in the conditional statement so that it matches the expected into within the module.

5

May also want to read Optional object attributes whithin a module and Object keys are silently filtered from variable if not in type definition · Issue #29204 · hashicorp/terraform · GitHub. What I’ve found is that there are still some cases where, while an object seems cleaner, doing a map and then doing custom validation ends up being better for certain use cases.

6

@wyardley : I wouldn’t like to go out of the type management. It’s never a good idea whatever the language.

@jbardin : I’m not sure to get your proposition. Would it be possible for you to provide some pseudo code to guide me?

While trying to overcome this issue, I finally found a hack to make it “work”.
The first operand in the ternary condition should raise an error when the condition evaluates to false. It’s really ugly but it makes this working as I would expect it…

Here is an example nearer to the reality main.tf:

variable "dbk" {
  type = object({
    mandatory_attr = string
    optional_attr  = optional(string)
  })
  default = null
}
locals {
  config_by_env_name = {
    dev = var.dbk != null ? merge(var.dbk, {/*...*/}) : null
  }
}
module "sub" {
  source = "./sub"

  config_by_env_name = local.config_by_env_name
}

And sub module sub/main.tf:

variable "config_by_env_name" {
  type = map(object({
    mandatory_attr = string
    optional_attr  = optional(string)
  }))
}

By modifying the code, it works whatever the value of the dbk var (nullor {mandatory_attr = “something”}):

locals {
  config_by_env_name = {
    dev = var.dbk != null ? (var.dbk == null ? regex("foo", "bar") : merge(var.dbk, {})) : null
  }
}
7

Having required attributes in an object type is going to require that you are as strict as possible with the object types in the configuration. Using merge like this inherently makes things dynamic, because it returns a different type based on its inputs. Your workaround is interesting, because what you’ve essentially done is obfuscate the result to the point that Terraform ends up with a dynamic type because the type information is lost from the true side of the expression. With the added idea of the merge which throws out the general possibility of strict typing, I’m not sure of a good idea to represent this in config.

EDIT, I may have had a typo in my POC, I cannot replicate now what I have below
I’ll leave this here for further thought though in case it does reproduce. The typed-null assignment within the map can still be quite unintuitive when types are inferred.

> local.config_by_env_name
{
  "dev" = null
}

> type(local.config_by_env_name["dev"])
object({
    mandatory_attr: string,
    opptional_attr: string,
})

but the assignment is still not allowed, because the map value type is failing the required attr check before the null value is checked. At a minimum this is something which appears inconsistent from the user’s perspective and should be looked into, and better type declarations may not help either.

-----------

As for how one would do this in Terraform now, I think it would typically be done by not trying to make use of the null value at all as I mentioned earlier. You need to restructure things slightly, and it may feel somewhat repetitive as a workaround, but listing the desired envs separately from their config values would break up the type dependency between them. Just having a list(string) or set(string) for env names, and then lookup the value from the map as needed would allow you filter out the null values entirely and avoid the type discrepancies.