Using Vault provider to get `kms_key_id` from Vault and using it in `terraform init` `-backend-config=` option in `Jenkinsfile`

Terraform
1

Hi,

We are trying to utilize vault in our Jenkins pipelines in a declarative pipeline Jenkinsfile, but aside from that, we want to run the typical bash command terraform plan and apply and use the -backend-config.

we could run this command:

terraform init -backend-config=backend_nonprod.tfvars

And in that backend_nonprod.tfvars file we have a bucket name for the S3 bucket storing the tfstate file, the key that is the name of that tfstate object/file, and then a dynamodb_table for holding a lock (if that’s the right terminology) on the tfsate file during terraform plan and applys. The issue is we also have a kms_key_id that we use to encrypt the tfstate file as well. This is an AWS kms key, and we can’t have it in plain text in the backend_nonprod.tfvars file like we could the bucket, key(s3 object for tfstate file), and dynamodb_table name.

we have Vault provider setup in the providers.tf, and so now I want a way to 1) access the kms_key_id in our Vault workspace, and 2) pass it to the -backend-config option during the terraform init command in addition to passing the tfvars file, meaning something like:

terraform init -backend-config=backend_nonprod.tfvars -backend-config kms_key_id=<something here>

I’ve went into the vars file to declare a path to the kv store in vault, and also to access that kms key id:

# vars.tf

variable "terraform_backend_secrets_vault_path_nonprod" {
  type = string
  description = "kv path in Vault for terraform backend secrets for nonprod"
  default = "kv/yadayada/nonprod" # location of `kms_key_id` key-value pair secret
}

variable "kms_key_id_nonprod" {
  type = string
  description = "AWS kms key id used for encrypting terraform state file for nonprod"
  default = data.terraform_backend_secrets_vault_path_nonprod.data["kms_key_id"] # secret is called `kms_key_id` in the kv store "kv/yadayada/nonprod" in Vault
}

In the main.tf so far I have:

# main.tf
data "vault_terraform_backend_nonprod" "nonprod" {
  path = var.terraform_backend_secrets_vault_path_nonprod
}

I guess my question is, how to I utilize that "kms_key_id_nonprod" in the terraform init bash command and pass it in as a -backend-config= option?

For context, we have Jenkins Vault approle for using Vault in the Jenkinsfile, and also we have some other Jenkinsfiles using Vault TF provider. I want to get out of us using a mix of those two patterns, and have us purely use TF Vault provider for vaulting in our pipelines, since we do use Vault provider for something else in our TF code and getting a particular secret already. We just haven’t had to use those secrets in terraform init and having to pass those in on the command line. Perhaps I’m right there and the fix is easy.

2

Hi @aaa,

Fetching the credentials to initialize this configuration must be done outside of Terraform, because before you initialize the working directory Terraform cannot run (aside from the init step).

A typical design for this is to run Terraform in some kind of automation, such as GitHub Actions, and then have a separate step before running Terraform that prepares the environment that Terraform will run in, including fetching the arguments necessary to initialize the backend.

1 Like
3

Thank you, this makes sense. We can use something from Terraform if it has not been terraform inited yet…

It looks like there is a way to get secrets out of Vault with withVault wrapped around the terraform init and plan and so on, in a Jenkinsfile, and then set them as TF vars to be made available in a configuration (like here but that doesn’t set TF_Vars). I may post a code sample later.

I may do this and remove the TF Vault provider entirely and let Jenkins provide the variables to Terraform instead.