Mourning the loss of defaults()

After a almost 2 year long experiment starting in v0.14 all the way through v1.2.9, the optional()/1 & defaults() syntax has been suddenly replaced with a new optional()/2 syntax that is not usable for dynamic values, and is not usable to traverse data structures.

While the new optional()/2 syntax is convenient for extremely simple structures and values, it is really a slight improvement over the old defaults = syntax, with the only benefits being readability and ability to set values within a structure.

But defaults(), while not being super intuitive, was very powerful while traversing data structures and injecting dynamic values with conditional logic; which I find to be a requirement when Terraform’ing anything more than very simple resources.

I’m really bummed that, with no overlapping experiment time, I have to now rewrite a lot of code by adding convoluted logic and difficult to read nested calls to coalesce(), merge() wrapped in conditionals in order to emulate what was being done with defaults(), in order to continue working with v1.3.0.

Why did you guys let that experiment stagnate for almost 2 years, with no changes, then suddenly pull it with no replacement??? Maddening.

Rob

Could you give some examples?

Hi Katy - thanks for the quick response; apologies for my delayed response :slight_smile:

Here is one example, which is parsing a config (in this case a YAML config) for GCP security policy rules. Much of our Terraform design is gitops based flows with YAML configs so our engineers can PR infrastructure change requests. We try to drive desirable behavior by via thoughtful choice of defaults - ie: allow for config to achieve common behavior via very simple declarative structure; but opt into more custom behavior by specifying more details, often in the form of additional map key/values. In this case, the priority of the rule corresponds to the index in the list.

  security_policy_rules = [
    for rule in [
      for idx, rule in var.security_policy_rules : defaults(
        rule,
        { 
          action   = "deny(403)"
          preview  = false,
          priority = idx,
        },
      )
    ] : merge(
      rule.versioned_expr != null ? { src_ip_ranges = ["*"] } : {},
      rule,
    )
  ]

Here is a snippet from the config we use to provision Terraform Cloud workspaces via a YAML config, implementing an opinionated convention of workspace naming.

  config_workspaces = merge(flatten([
    for name, workspace in {
      for name, workspace in var.config_workspaces : name => defaults(
        workspace,
        {
          allow_destroy_plan = false
          auto_apply         = true
          file_triggers_enabled = length(
            regexall("__", name)
          ) > 0
          global_remote_state = false
          identifier          = "stordco/${split("__", name)[0]}"
          terraform_version   = var.terraform_version
          queue_all_runs      = true
        },
      )
      } : [
      for region in coalescelist(workspace.regions, [""]) : {
        join("--", compact([name, region])) = workspace
      }
    ]
  ])...)

…and for users, setting a default (but overridable) email coupled to username:

    for username, member in var.config_members : username => defaults(
      member,
      {
        email = "${username}@stord.com",
      },
    )
  }

Another use case is where we have a single variable structure that we ingest; but we then use transformations to process it in different contexts (ie: would not be suitable to place in the variable definition). Use case is Auth0 config rule scripts.

    scripts = merge(
      flatten([
        for idx, script in var.config_rules.scripts : [
          for script_id, script_args in script : {
            (script_id) = merge(
              defaults(
                script_args,
                {
                  enabled = true
                  name    = script_id
                },
              ),
              { order = idx + 1 },
            )
          }
        ]
      ])...
    )

Here is another pattern where in our input variables we have a “special” hash key (_DEFAULTS) that we can use to set our org-wide repo defaults in the same map that is processed to create repos. Context is GitHub config.

locals {
  config_repos = {
    for name, repo in var.config_repos : name => merge(
      defaults(
        repo,
        {
          for setting in [
            "allow_auto_merge",
            "allow_merge_commit",
            "allow_rebase_merge",
            "allow_squash_merge",
            "archived",
            "auto_init",
            "branch_protection",
            "default_branch",
            "delete_branch_on_merge",
            "description",
            "dismiss_stale_reviews",
            "enforce_admins",
            "has_downloads",
            "has_issues",
            "has_projects",
            "has_wiki",
            "homepage_url",
            "is_template",
            "require_code_owner_reviews",
            "required_linear_history",
            "required_status_checks_strict",
            "visibility",
            "vulnerability_alerts",
          ] : (setting) => lookup(var.config_repos["_DEFAULTS"], setting, null)
        },
      ),
      {
        autolink_references = merge(
          coalesce(var.config_repos["_DEFAULTS"].autolink_references, {}),
          coalesce(repo.autolink_references, {}),
        ),
      },
      {
        environments = {
          for env_name, env in coalesce(repo.environments, {}) : env_name => merge(
            defaults(
              env,
              { protected_branches = false },
            ),
            {
              reviewers = {
                members = coalesce(
                  lookup(
                    coalesce(env.reviewers, {}),
                    "members",
                    [],
                  ),
                  [],
                )
                teams = coalesce(
                  lookup(
                    coalesce(env.reviewers, {}),
                    "teams",
                    [],
                  ),
                  [],
                )
              }
              secrets = coalesce(env.secrets, {})
            }
          )
        }
      },
      {
        teams_admin : concat(
          repo.teams_admin == null ? var.config_repos["_DEFAULTS"].teams_admin : repo.teams_admin,
          ["Repo Admins"]
        )
      }
    )
    if length(regexall("^_", name)) == 0
  }
}

The idea with this transformation is to keep all the logic in one place, which lets our resource definition be very simple; the for loop in the defaults arg lets us maintain a very clean and simple list of settings that we accommodate. Here’s the corresponding resource definition for context:

resource "github_repository" "this" {
  for_each = local.config_repos

  allow_auto_merge       = each.value.allow_auto_merge
  allow_merge_commit     = each.value.allow_merge_commit
  allow_rebase_merge     = each.value.allow_rebase_merge
  allow_squash_merge     = each.value.allow_squash_merge
  archived               = each.value.archived
  auto_init              = each.value.auto_init
  delete_branch_on_merge = each.value.delete_branch_on_merge
  description            = each.value.description
  has_downloads          = each.value.has_downloads
  has_issues             = each.value.has_issues
  has_projects           = each.value.has_projects
  has_wiki               = each.value.has_wiki
  homepage_url           = each.value.homepage_url
  is_template            = each.value.is_template
  name                   = each.key
  topics                 = each.value.topics
  visibility             = each.value.visibility
  vulnerability_alerts   = each.value.vulnerability_alerts

  dynamic "template" {
    for_each = toset(compact([each.value.template]))

    content {
      owner      = split("/", template.value)[0]
      repository = split("/", template.value)[1]
    }
  }

  lifecycle {
    ignore_changes = [branches, pages]
  }
}

regards,

Rob