Default to null on dynamic blocks

Terraform
1

Hi guys,

I’m trying to import several s3 buckets created by hand and ansible with different configurations in a single terraform module.

There’s any way to ignore some attributes with dynamic blocks?

I have a variable file mostly looking like this:

variable "buckets" {
    type = map(object({
        website = object({
            index_document = string
            error_document = string
        })
        cors_rule = object({
            allowed_headers = set(string)
            allowed_methods = set(string)
            allowed_origins = set(string)
            expose_headers = set(string)
            max_age_seconds = string
        })
    }
}

My tfvars is similar to this:

buckets = {
    test123 = {
        website = {
            index_document = "index.html"
            error_document = "error.hml" ** # Trying to ommit this for example**
        }
    } 
    test456 = {
        website = null **# Trying to set null as default value**
    }
}

And my module looks like this:

resource "aws_s3_bucket" "bucket" {
for_each = var.buckets
bucket = each.key

dynamic "website" {
    for_each = each.value.website.*
    content {
        index_document = each.value.website.index_document
        error_document = each.value.website.error_document **# tring to ommit this if not defined**
    }
}

test123 will be configured as website and test456 won’t as expected.

Is there any way to put website = null as the default if not defined to not have to declare it on test456? (besides the website, I have logging, cors, etc …) and it is possible to declare any number of elements inside the content block instead of all of them without setting each unwanted one as null at the tfvars file?

2

You cannot assign a null default to a nested field in an object-type variable (there’s a feature request open for this), but you might be able to achieve a similar outcome with some empty strings and a conditional.

It’s a bit verbose, there’s probably a nicer way to accomplish the same thing, but this worked for me (I removed the cors_rule object from your variable definition for brevity):

buckets = {
    test123 = {
        website = {
            index_document = "index.html"
            error_document = ""
        }
    } 
    test456 = {
        website = "" 
    }
}

resource "aws_s3_bucket" "bucket" {
  for_each = var.buckets
  bucket   = each.key

  dynamic "website" {
    for_each = each.value.website.*
    content {
      index_document = each.value.website.index_document == "" ? null : each.value.website.index_document
      error_document = each.value.website.error_document== "" ? null : each.value.website.index_document
    }
  }
}

And here’s the plan output using that tfvars file as proof:

Terraform will perform the following actions:

  # aws_s3_bucket.bucket["test123"] will be created
  + resource "aws_s3_bucket" "bucket" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "test123"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }

      + website {
          + index_document = "index.html"
        }
    }

  # aws_s3_bucket.bucket["test456"] will be created
  + resource "aws_s3_bucket" "bucket" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "test456"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }

      + website {}
    }

I hope this helps!

3

Thanks @mildwonkey,

But the code you shared has exactly the same behavior that I’m having with the difference that an empty value is provided instead of null.

I’m trying to omit the website variable entirely inside the bucket variable and omit objects inside the website variable if set (like error_document)

4

Hi @fanfoni,

As with website = null, the way to leave error_document unset while still meeting the type constraint is to explicitly set it to null:

buckets = {
    test123 = {
        website = {
            index_document = "index.html"
            error_document = null
        }
    } 
    test456 = {
        website = null
    }
}

The type constraint says that error_document must be present, but we can set it to null to explicitly state that there is no error_document for this particular website. You can then use each.value.website.error_document in the definition of the website block argument, where it will interpret null the same way as it being unset.

As @mildwonkey noted, the ability to declare an object type attribute as optional in a type constraint is an open feature request, but in current versions of Terraform that isn’t implemented and so explicitly setting to null is the current way to represent leaving an optional argument unset.

5

Thanks for the clarification @apparentlymart .

I was thinking about creating a dummy variable or local and merge it with the provided ones to fulfil the missing objects with null values but I haven’t found a way to do it yet.

There’s any way to bring the variable block bellow into the module if let’s say cors_rule is set and allowed_origins is unset and try to merge it with some dummy values later without triggering the dynamic block if cors_rule is null? (like declaring it as any and then recreating this map locally inside the module with all missing values filled with null)

variable "buckets" {
    type = map(object({
        website = object({
            index_document = string
            error_document = string
        })
        cors_rule = object({
            allowed_headers = set(string)
            allowed_methods = set(string)
            allowed_origins = set(string)
            expose_headers = set(string)
            max_age_seconds = string
        })
    }
}
6

Hi @fanfoni,

I personally would just do what you showed and have the callers write the explicit null expressions in, because the cost of writing those explicit nulls tends to be relatively low compared to the complexity of doing the normalization inside the module.

With that said, you can write a local value expression to normalize what you recieved from an input variable. I normally suggest this pattern for interpreting the results of jsondecode or yamldecode to normalize for quirks of externally-defined formats, but it also works for input variables with the any type constraint:

locals {
  buckets = tomap({
    for key, bucket in var.buckets : key => {
      website = try(bucket.website, null) == null ? null : {
        index_document = tostring(try(bucket.website.index_document, null))
        error_document = tostring(try(bucket.website.error_document, null))
      }
      cors_rule = try(bucket.cors_rule, null) == null ? null : {
        allowed_headers = toset(try(bucket.cors_rule.allowed_headers, ["X-Custom-Header"]))
        allowed_methods = toset(try(bucket.cors_rule.allowed_methods, ["GET"]))
        allowed_origins = toset(try(bucket.cors_rule.allowed_origins, ["*"]))
        expose_headers  = toset(try(bucket.cors_rule.expose_headers, null))
        max_age_seconds = tostring(try(bucket.cors_rule.max_age_seconds, null))
      }
    }
  })
}

Here I’ve used various type conversion functions (tostring, toset, tomap, try) to convert the input into consistent data types in a similar way to what Terraform would normally do automatically as part of applying a variable type constraint, but with additional conditional logic to substitute default values when some attributes are not set.

If you take this approach, you may wish to also write some Custom Variable Validation rules for your variable, so that Terraform can still give error feedback in the context of the caller’s expression when the input is invalid, even though Terraform will no longer be enforcing the type constraints directly there.

7

Thank you @apparentlymart , it worked like a charm and your solution is extremally cool IMHO.

The bucket module is just one of the modules in this situation, but we’ll have to import a lot of unstandardized stuff into Terraform.

This will be part of a CI/CD process and everybody will be notified if the job fails for some reason, so I’m trying to at least avoid simple things like missing values.

The cool part of your solution is that it will be possible to release new versions of the module with new parameters without having to worry too much about the existing variable file if something is added.