Usage of provider::terraform::encode_expr

[naftulikay]

I am using the tfe provider to configure Terraform Cloud workspaces, and I need to encode HCL. To this end, I have a module that does something like this:

locals {
  value = {
    a = "a"
    b = "b"
  }
}

provider terraform {}
provider tfe {}

resource tfe_variable default {
  key = "config"
  value = provider::terraform::encode_expr(local.value)
  category = "terraform"
  hcl = true
  # omitted for brevity
}

This is within a module that is being published to my private Terraform Cloud registry. It passes terraform validate and tflint, but when Terraform Cloud attempts to import it into the registry, it fails with the following error:

Error: error loading the module: Missing newline after argument: An argument definition must end with a newline. (in r.vars.tf on line 21)

This is the exact use-case mentioned in the docs for this function, but I can’t seem to figure out why it refuses to import the module.

Am I using it wrong?

[apparentlymart]

Hi @naftulikay!

Can you share which line from the snippet you shared is line 21 of r.vars.tf? I’d like to narrow down where the syntax error is being raised.

Thanks!

[naftulikay]

The verbatim copy is here:

resource tfe_variable tfc_aws_provider_config {
  key             = "tfc_aws_provider_config"
  value           = provider::terraform::encode_expr(local.aws_provider_config)
  category        = "terraform"
  hcl             = true
  description     = "Terraform provider configuration for AWS"
  variable_set_id = tfe_variable_set.aws.id
}

Line 21 is the line of value = ....

And local it is using is here:

locals {
  aws_provider_config = {
    account_id     = var.aws_account.id
    default_region = "us-east-2"
    default_tags   = local.aws_tags
    sts_region     = "us-east-2"
  }

  aws_tags = {
    account = var.env.name
    account_id = var.aws_account.id
    account_short = var.env.name_short
    account_name = var.aws_account.name
    env = var.env.name
    env_short    = var.env.name_short
  }
}

[apparentlymart]

Actually, on further pondering I have a theory: the registry might not have been updated to the latest version of HCL yet, and so it wouldn’t understand the syntax for namespaced providers. I’ll check on that internally to see if that’s a correct hunch.

[apparentlymart]

Thanks for raising this, @naftulikay!

It does indeed seem like we neglected to upgrade the version of the HCL library that Terraform Registry uses, and so it doesn’t understand this new syntax. I’ve let the relevant teams know and so hopefully we can get it updated soon, and then I expect it will accept your module.

I’ll post an update here once the registry upgrade is complete.

[naftulikay]

Fantastic, thanks for looking into it, I wasn’t aware it was a new feature, I’ll be sure to update my minimum required Terraform version.

Also, instead of a function, wouldn’t it maybe make more sense to just add a terraform_hcl data provider to the terraform provider alongside terraform_remote_state? That seems a lot more straightforward to me, i.e.:

data terraform_hcl aws_config {
  value = {
    a = "b"
    b = ["a", "b", "c"]
    c = 128
    d = {
      e = "f"
    }
  }
}

resource tfe_variable aws_config {
  key             = "tfc_aws_provider_config"
  value           = data.terraform_hcl.aws_config.rendered
  category        = "terraform"
  hcl             = true
  variable_set_id = tfe_variable_set.default.id
}

This strategy could also prevent breakage of language servers and IDE insight into code: my IDE (JetBrains-based) does not understand provider::terraform::encode_expr(...) and shows errors in my source-code.

[apparentlymart]

First of all, an update on the original problem: the Terraform Registry team tells me they’ve just finished deploying the upgraded HCL parser, so I expect you should be able to publish your module now. Sorry for the unexpected blocker!

---

Regarding the followup question:

I think the desired end state is for the tfe_variable resource type to take an arbitrary value directly and apply the necessary encoding itself, so you wouldn’t need to explicitly encode it at all, but this is an older provider that was designed before such a thing was possible and so it’s not clear yet how best to get there without breaking existing modules. This solution using the explicit encoding function to produce HCL expression syntax is just something that became possible with the recent addition of provider-defined functions and so we put it in the documentation in the hope it would be useful in the meantime.

For the more general question of “why not just use data sources?”, we got lots of feedback over the years that folks found the syntactic overhead of data resources inconvenient or otherwise distasteful for this sort of thing where there aren’t actually any network requests being made and it’s all just local compute anyway. It being a function also means that Terraform can evaluate it during terraform validate rather than only in terraform plan, which admittedly doesn’t help much in this case, but helps in others.

You’re right that there’s no technical reason that it couldn’t have been a data source, but based on the aforementioned feedback the intention is for all of these “compute-only” situations to be presented as functions rather than as data sources in future, and to reserve data sources for the situation they were originally intended for: retrieving some data from elsewhere, typically over the network, where it’s important to control the ordering of those requests in relation to other actions.

There are some other examples in non-builtin providers, such as the hashicorp/aws provider’s new function arn_parse.

[naftulikay]

Thank you so much for your response and the fix. My module is publishing now.