Terraform String Encode HCL Configuration as String

Terraform
,
1

I am setting up Vault using Terraform Helm, so I have the following.

resource "helm_release" "vault" {
    # ...
    values = [
    yamlencode({
      server = {
        ha = {
          raft = {
            config = <<-EOT
              # ... more stuff here
              seal "awskms" {
                kms_key_id = "${aws_kms_key.vault[0].key_id}"
              }
            EOT
          }
        }
    })
  ]
}

As you can see, I am using <<-EOT to have a multi-line string of an HCL configuration value. Ideally, I prefer to have a full HCL syntax. For example:

resource "helm_release" "vault" {
  # ...
  values = [
    yamlencode({
      server = {
        ha = {
          raft = {
            config = madeupstringencode({
              seal "awskms" {
                kms_key_id = "${aws_kms_key.vault[0].key_id}"
              }
            })
          }
        }
      })
    ]
}

Is that possible today?

Functions - Configuration Language | Terraform | HashiCorp Developer, but I could not find any function that would do this (my apologies if I missed anything).

2

No.

I wouldn’t expect it to ever be possible really - HCL is a language toolkit more than a language in some ways, and Terraform-flavoured HCL isn’t necessarily processed in exactly the same way as Vault-flavoured HCL.

Just carry on doing it the way you already are, or move the file content to an external file and use the templatefile function for substitutions.

1 Like
3

As @maxb noted, it isn’t really meaningful to just take an arbitrary data structure and “HCL encode” it like you might with JSON or YAML, because HCL is a schema-driven configuration language toolkit rather than just a data structure serialization format. Encoding it tends to require knowledge of a specific language built in terms of HCL, rather than just the HCL grammar.

One way this could potentially work is for the hashicorp/vault provider to offer a data source which accepts structured data as its arguments and generates valid Vault policy language structure based on that. It would then be Vault-specific and so could directly implement Vault’s policy language rather than just the generic HCL grammar.

I don’t think the provider has such a thing today though, and it might be a little tricky in your situation where you aren’t otherwise using that provider because everything else that provider does requires working credentials for an already-running Vault server. Might be worth opening a feature request in the Vault provider’s repository to note the use-case and discuss the design trade-offs.

You could also investigate whether Vault supports a JSON-based variant of its policy language. If so, you could generate that with jsonencode instead.

4

Prompted by @apparentlymart 's answer I can reply back with some more info to build on it further:

The current hashicorp/vault provider does actually already have a vault_policy_document data source, but the reason it’s inapplicable here is that this use case is actually for Vault’s server configuration file rather than policy syntax.

Vault does technically permit a JSON-formatted server configuration file, but this use case is using the official Vault Helm chart, which always puts the config in a file with an .hcl file extension, so I’m not sure whether that would actually work out.

I don’t think a custom Vault config file HCL-building data source would make sense, as the Vault config file makes use of HCL blocks, which can’t be part of a Terraform value, so you’d end up needing to invent a special custom arrangement of Terraform values just to pass the desired data over to the provider… at which point, given the relative simplicity of Vault server config files, you really are better off just using the existing textual templating support in Terraform.

5

Oh whoops I don’t know how I missed vault_policy_document when I peeped in there earlier!

Thanks for pointing out that this isn’t a vault policy. I should’ve looked more closely before I answered. Indeed, generating a full Vault configuration is presumably more involved – the configuration language is quite a bit larger than the policy language – and so it would probably be a significant maintenance burden to describe a mirror of the full Vault config language as a Terraform data source schema.

From what you’ve said here it seems like one possible change would be to change that Helm chart to be able to generate a file with a name that Vault will recognize as being in JSON format, though I imagine the JSON form of the full Vault configuration language is far less ergonomic than the HCL-based form and so templating the native syntax (HCL-based) might be the best compromise indeed.

6

Thus far, I am happy to use templatefile and I managed to change a bit how I did things where it is not that painful to maintain.

server:
  ha:
    raft:
      config: |
        ui = true
        listener "tcp" {
          address = "[::]:8200"
          cluster_address = "[::]:8201"
          tls_disable = false
          tls_cert_file = "/vault/userconfig/server-tls/tls.crt"
          tls_key_file = "/vault/userconfig/server-tls/tls.key"
          tls_client_ca_file = "/vault/userconfig/ca-tls/tls.crt"
        }
        storage "raft" {
          path = "/vault/data"
          retry_join {
            leader_api_addr = "https://vault-0.vault-internal:8200"
            leader_ca_cert_file = "/vault/userconfig/ca-tls/tls.crt"
            leader_client_cert_file = "/vault/userconfig/server-tls/tls.crt"
            leader_client_key_file = "/vault/userconfig/server-tls/tls.key"
          }
          retry_join {
            leader_api_addr = "https://vault-1.vault-internal:8200"
            leader_ca_cert_file = "/vault/userconfig/ca-tls/tls.crt"
            leader_client_cert_file = "/vault/userconfig/server-tls/tls.crt"
            leader_client_key_file = "/vault/userconfig/server-tls/tls.key"
          }
          retry_join {
            leader_api_addr = "https://vault-2.vault-internal:8200"
            leader_ca_cert_file = "/vault/userconfig/ca-tls/tls.crt"
            leader_client_cert_file = "/vault/userconfig/server-tls/tls.crt"
            leader_client_key_file = "/vault/userconfig/server-tls/tls.key"
          }
          autopilot {
            cleanup_dead_servers = true
            min_quorum = 3
          }
        }
        service_registration "kubernetes" {}

Some pain points with the retry_join config (easy to make a mistake and forget to add things based on the scaling), and still, string value in YAML.

But it works :person_shrugging:

I am following the conversation, but it goes above my head a few things …

What would you recommend as the path forward regarding the Helm Chart and Vault? Should we bring the situation up to them?

7

Use a for directive to automatically repeat the block the appropriate number of times, then: Strings and Templates - Configuration Language | Terraform | HashiCorp Developer

IMO you should move your YAML back into the Terraform file and yamlencode() it, and just have the server config HCL as the template.