One significant change is that I’m using Vault for KMS instead of Azure KeyVault. I have no trouble deploying the required infrastructure into Azure, but I am encountering an error when attempting to configure Boundary. The Terraform returns:
module.configure_base_boundary.boundary_scope.global: Creating…
╷
│ Error: error calling read scope: {“kind”:“Unauthenticated”, “message”:“Unauthenticated, or invalid token.”}
│
│ with module.configure_base_boundary.boundary_scope.global,
│ on …......\modules\common\hashicorp\boundary_config\scopes.tf line 1, in resource “boundary_scope” “global”:
│ 1: resource “boundary_scope” “global” {
Are you able to use that Transit key with the Vault CLI from the environment you’re running Terraform in, to encrypt and decrypt data? How are you passing the Transit key config to the Boundary TF provider?
Yes I have confirmed the transit key works with the recovery token via the Vault CLI from the boundary controller VM.
Here is the slightly edited Terraform showing the provider setup with vault resources.
provider "vault" {
alias = "my_vault"
address = "https://my.vault.server"
auth_login {
path = "auth/approle/login"
parameters = {
role_id = var.login_approle_role_id
secret_id = var.login_approle_secret_id
}
}
}
provider "boundary" {
alias = "boundary_config"
addr = data.terraform_remote_state.regional_infrastructure.outputs.boundary_url
# TODO: Remove tls_insecure once we get a proper certificate for this
tls_insecure = true
auth_method_id = data.vault_kv_secret_v2.auth.data["auth_method_id"]
password_auth_method_login_name = data.vault_kv_secret_v2.auth.data["password_auth_method_login_name"]
password_auth_method_password = data.vault_kv_secret_v2.auth.data["password_auth_method_password"]
recovery_kms_hcl = <<-END
# Recovery KMS block: configures the recovery key for Boundary
# Use a production KMS such as AWS KMS for production installs
kms "transit" {
purpose = "recovery"
address = "https://my.vault.server"
token = "${data.vault_kv_secret_v2.tokens.data["recovery"]}"
disable_renewal = "false"
// Key configuration
key_name = "boundary-recovery"
mount_path = "transit/"
}
END
}
data "vault_kv_secret_v2" "auth" {
provider = vault.my_vault
mount = "rg_services"
name = "boundary/provider_auth"
}
# The Vault Terraform Provider has a bug that errors when creating tokens. Until this is fixed I'm storing a token in
# Vault at rg_services/boundary/transit_tokens to get around this.
# Bug Report: https://github.com/hashicorp/terraform-provider-vault/issues/1424
# TODO: Remove the vault_kv_secret_v2.tokens and uncomment the following vault_token sections once the Vault Provider is fixed.
# TODO: Remember to update the token references to point to the created tokens.
data "vault_kv_secret_v2" "tokens" {
provider = vault.my_vault
mount = "rg_services"
name = "boundary/transit_tokens"
}
...
I’m currently storing some long use tokens in Vault for this because of the bug mentioned above. But basically it’s all retrieved from Vault at runtime.
I don’t usually post source code to help answer questions but in this case it’s probably helpful:
// It must be before the current time. This means someone can't create
// one way in the future and keep using it. We fudge this by 1 minute to
// account for time discrepancies between systems.
if info.CreationTime.After(time.Now().Add(FutureLeeway)) {
return nil, errors.New("recovery token creation time is invalid")
}
Basically the timestamp is probably too far into the future compared to the current time on the Boundary box. The leeway time is one minute. Likely there is over a minute of clock drift between some of your boxes, might be worth ensuring NTP or similar is up and running.
Time drift was absolutely the issue here. The machine executing the Terraform code had drifted by more than 2 minutes. After getting the clocks synced back up everything ran as expected. Thank you for pointing this one out!