Vault generated AWS credentials via Jenkins Pipeline

Hi all

I have just started testing Vault integration into my Jenkins Pipeline for retrieving AWS credentials in order to control AWS services & resources via Terraform.

I have setup Vault in my Kubernetes lab and have it all configured correctly to retrieve / generate AWS credentials, this all works fine.

This is only a home lab / dev environment so I can do as I please.

At the moment I am using the following method to inject AWS creds into my pipeline stages which I really don’t like at all, but I wanted to test that Jenkins could successfully retrieve AWS credentials:

      echo "Exporting VAULT details..."
      export VAULT_ADDR=<<kubernetes_vault_endpoint>>
      export SECRET_ID=$(vault write -field=secret_id -f auth/approle/role/jenkins/secret-id)
      export VAULT_TOKEN=$(vault write -field=token auth/approle/login role_id=${ROLE_ID} secret_id=${SECRET_ID})
      vault read aws/creds/jenkins >> credentials
      echo
      echo "credentials file contains the following..."
      cat ./credentials
      echo
      echo "Parsing credentials file and exporting AWS key details..."
      export AWS_ACCESS_KEY_ID=`cat ./credentials | grep access_key | awk '{print $2}'`
      export AWS_SECRET_ACCESS_KEY=`cat ./credentials | grep secret_key | awk '{print $2}'`
      echo
      rm ./credentials

…I really don’t like this method at all, is there a better way that I can do this? Ideally a plugin which will automatically populate the AWS environment variables with key / key_id in any stage within my Jenkinsfile.

Would appreciate any help as I am just starting out with this.

Thanks

There is a Vault plugin for Jenkins. I don’t have any real experience with it other than I know it exists but it may help you out a bit: GitHub - jenkinsci/hashicorp-vault-plugin: Jenkins plugin to populate environment variables from secrets stored in HashiCorp's Vault.

I actually wouldn’t recommend the Vault plugin for Jenkins. It requires you to input a static secret_id, which kinda defeats the purpose of having a just-in-time OTP. I don’t see anything wrong with what OP posted initially. It may seem like a lot of steps but it’s all coded in the pipeline steps and should happen relatively quickly.

Having said that, instead of output the AWS credentials as is, I would do something like:

vault read -format=json aws/creds/jenkins | jq -r ".access_key, .secret_key" > ./credentials

so then you don’t have to do a series of greps and awks; you just read the first line as access_key and second line as secret_key from the resulting file

1 Like

What do you mean by “requires you to input a static secret_id”? We use the Jenkins plugin extensively and it works pretty well for us, both to pull out static secrets (from the k/v secret engine) and dynamic ones during build & deployment pipelines.

The part where it says “You enter your role_id and secret_id there” and you even see it in the screenshot where there’s a field for the admin/user to enter in the secret_id just like if it was a username/password Jenkins credential.

See: Vault Plugin for Jenkins

EDIT: there’s also the part earlier when describes how an AppRole works and mentions “How long should the secret_id live (can be indefinite)”. And while the secret_id certainly can be made to live forever, that’s not best practice. Why even use AppRole if you’re going to do that? Why not just set up a userpass auth account for Vault?

EDIT2: there are multiple config/usage types for the Vault plugin and I’m just referring to the AppRole one in particular. The Vault Token from File seems to be decent but only if you run a Vault Agent on your Jenkins host to manage that token’s lifecycle…but then if you’re going to do that, you can just as easily add a step in your pipeline to read from that same file to get your token:

i.e.

VAULT_TOKEN=$(cat /path/to/token-file.txt) vault kv get secrets/mysecret

And if you’re not using a Vault Agent to rotate that token then that’s also not best practice

Ahh, so you are meaning the initial credentials for Jenkins to authenticate to Vault. This requirement exists for whatever mechanism you are using. The original script still needs something similar - I’m assuming some sort of Vault login occurs before the script runs (or an existing login token is being used via the environment).

In our case we do use the AppRole within Jenkins, which is updated every time a deployment happens (so fairly regularly as Jenkins versions get updated).

The advantage of the plugin over just running the Vault CLI is the integration. You don’t need to worry about being on a node with an up to date copy of the Vault CLI (or keeping it updated) and the usage looks just the same as any other Jenkins pipeline step from a plugin (rather than mixing your Gorrvy / declarative with lots of shell scripting).

One thing I’m not quote getting from the original script is the login using AppRole. As you are already logged into Vault (to be able to generate a new secret ID for the “jenkins” role) what is the rational for then logging in again (presumably ROLE_ID is set to the value for the “jenkins” role again) - why not just use the credentials you already have?