Could not load backend configuration

Vault
1

Moving backwards from the error, the agent-init container errors with the following

2023-02-22T09:28:58.253Z [INFO]  auth.handler: authenticating
2023-02-22T09:28:58.264Z [ERROR] auth.handler: error authenticating:
  error=
  | Error making API request.
  | 
  | URL: PUT http://vault.vault.svc:8200/v1/auth/kubernetes/login
  | Code: 500. Errors:
  | 
  | * could not load backend configuration
   backoff=1m47.4s

I have annotations on the pod as follows:

      "vault.hashicorp.com/agent-inject"               = "true"
      "vault.hashicorp.com/role"                       = "foo"
      "vault.hashicorp.com/agent-inject-secret-domain" = "bar/domain"

The following Terraform creates the roles, auth_backend and policies

resource "vault_auth_backend" "kubernetes" {
  type = "kubernetes"
}

data "vault_policy_document" "document" {
  rule {
    path         = "kvv2/bar/*"
    capabilities = ["read", "list"]
    description  = "Allow read on bar secrets"
  }
}

resource "vault_policy" "policy" {
  name   = "home-assistant"
  policy = data.vault_policy_document.document.hcl
}

resource "vault_kubernetes_auth_backend_role" "role" {
  backend                          = vault_auth_backend.kubernetes.path
  role_name                        = "foo"
  bound_service_account_names      = ["foo"]
  bound_service_account_namespaces = ["foo"]
  token_ttl                        = 3600
  token_policies                   = [vault_policy.policy.name]
}

Vault looks like it’s listening on http://vault.vault.svc:8200

If I curl from the foo namespace. I get the following (which is expected). So I’m not sure what’s going on with the init container.

curl -X PUT http://vault.vault.svc:8200/v1/auth/kubernetes/login
{"errors":["missing role"]}
2

Faced with a mystery error like

I find it’s usually most helpful to go look in the source code to find out what it really means.

Sure enough, https://github.com/hashicorp/vault-plugin-auth-kubernetes/blob/main/path_login.go reveals it means that the Kubernetes auth backend configuration is not set… which makes sense, as there’s nothing in your Terraform code to configure it.

https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/kubernetes_auth_backend_config

3

AH, that worked a treat. I did think that it might be misconfigured, but also it was accessing vault at what I thought was the correct address :slight_smile:

Thanks very much for helpful pointers :+1: