Authentication issues with GCP secrets engine impersonated account path

Vault
1

So I have a bit of curiosity here as to how authentication works behind the impersonated accounts feature of the GCP secrets engine, and I’m wondering if I can get some clarification here.

What am I doing?
I am looking to use service account impersonation to generate an OAuth2 token that will be used downstream by a service to access GCP Artifact Registry. The config for the provisioned endpoint is like so:

{
  "name": "artifact-registry",
  "service_account_email": "artifact-registry@MY_PROJECT.iam.gserviceaccount.com",
  "ttl": "1800s",
  "token_scopes": [
    "https://www.googleapis.com/auth/cloud-platform"
  ]
}

What are my problems?
My issues arise when trying to manually hit the endpoint via the Vault CLI. In running the command :

VAULT_TOKEN=my_access_token VAULT_SKIP_VERIFY=true 
vault read gcp/impersonated-account/artifact-registry/token

The go library returns the error from GCP:

* unable to generate token source: impersonate: unable to generate access token: 
Post "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/artifact-registry@MY_PROJECT.iam.gserviceaccount.com:generateAccessToken": Forbidden

What have I tried?
First, I validated the GCP endpoint by sending an HTTP request to the Google IAM API generateAccessToken function. This was using an OAuth2 token tied to my GCP user account, where the role Service Account Token Creator has been enabled. This returned the token I was expecting.

This led me to looking at the source code for the impersonated account implementation and gave me a few questions to ponder over. What account is being used when making the Vault request?

In the file vault-plugin-secrets-gcp/plugin/path_impersonated_account_secrets.go at 187ca51b3db9053da95ab58235a016f3c1558d48 · hashicorp/vault-plugin-secrets-gcp · GitHub , the function utilizes GCP credentials. If these credentials are not supplied, default credentials are used (vault-plugin-secrets-gcp/plugin/backend.go at 187ca51b3db9053da95ab58235a016f3c1558d48 · hashicorp/vault-plugin-secrets-gcp · GitHub).

Looking up what this default would be, it states:

It looks for credentials in the following places, preferring the first location found:

A JSON file whose path is specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable.

I’ve since set this environment variable to a service account key, but to no success.

Perhaps someone could help point me to the thing I’m missing here…

Cheers!

2

Facing same issue for static account as well.I am not able to create keys for json as key generation is restricted at Org level. Can someone help how this could be achieve using workload identity ?

3

Hi @vikas.khatri2009 - this is from an old tutorial, but these steps used to work to generate keys:

  1. Enable the GCP secrets engine.

    $ vault secrets enable gcp

Success! Enabled the gcp secrets engine at: gcp/

  2. Configure the GCP secrets engine to use the VaultServiceAccountKey.json credentials.

    $ vault write gcp/config \
    ttl="2m" \
    max_ttl="10m" \
    credentials=@$TUTORIAL_TEMP/VaultServiceAccountKey.json

    Example output:

    Success! Data written to: gcp/config

  3. Define the bindings to be used by Vault in a file named gcpbindings.hcl.

    $ tee $TUTORIAL_TEMP/gcpbindings.hcl <<EOF
 resource "//cloudresourcemanager.googleapis.com/projects/$GCP_PROJECT_ID" {
        roles = ["roles/viewer"]
      }
EOF

    The roles/viewer role is used in this tutorial for simplicity. You can use any
    predefined or custom role
    that provides the necessary permissions for the application.

  4. Create a Vault roleset named edu-app-key.

    $ vault write gcp/roleset/edu-app-key \
    project=$GCP_PROJECT_ID \
    secret_type="service_account_key"  \
    bindings=@$TUTORIAL_TEMP/gcpbindings.hcl

    Example output:

    Success! Data written to: gcp/roleset/edu-app-key

  5. Read the edu-app roleset to generate a new key.

    $ vault read gcp/roleset/edu-app-key/key

Key                 Value
---                 -----
lease_id            gcp/roleset/edu-app-key/key/ra4SvjyYlp9udTrkCBnsK3Ky.RfmoS
lease_duration      2m
lease_renewable     true
key_algorithm       KEY_ALG_RSA_2048
key_type            TYPE_GOOGLE_CREDENTIALS_FILE
private_key_data    ewogICJ0eXBlIjogInNlcnZpY2VfY...snip...dmF0ZV9rZXlfaWQiOiAiMWIyNzViOG

    The lease_duration is set to 2m because the TTL defined in the secrets engine configuration
    was set to 2m.

    They private_key_data value is a base64 encoded blob that has a properly formatted JSON file
    which the service can use to authenticate with GCP.

  6. Generate a new key and base64 decode the private_key_data field to reveal the JSON formatted credential file.

    $vault read --field private_key_data gcp/roleset/edu-app-key/key | base64 --decode

    Example output:

    {
  "type": "service_account",
  "project_id": "abc-123DEFb547c4b85ab06e16606d",
  "private_key_id": "16f43c11931007f0131a2148a4add81ccd889ffd",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADCBKcwggSj...snip.../zL\nMmXo1ma5v8V4rPTAvkHNWbM=\n-----END PRIVATE KEY-----\n",
  "client_email": "vaultedu-app-1677764256@abc-123DEF1b547c4b85ab06e16606d.iam.gserviceaccount.com",
  "client_id": "109361234567891786547",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/vaultedu-app-1677764256%40abc-123DEF1b547c4b85ab06e16606d.iam.gserviceaccount.com"
}

  7. Return to the GCP console.

  8. Expand the hamburger menu and navigate to IAM & Admin >> Service Accounts.

  9. A service account has been created in GCP based on the Vault role name edu-app-key.

  10. Click the vaultedu-app-key123456789@… service account and click the KEYS tab.

  11. The key ID associated with the service account key you created will be listed.

As far as workload identity federation goes, that is more for Vault to connect to GCP without long lived credentials.