Boundary credential store token

Boundary
1

Hi,

I want to allow users to get postgres credentials from boundary.
Everything works fine, but the way I create the token for vault is manually and after a period of time (TTL) the credentials disappears and I should create again a credentials store.
How can I use Boundary to authenticate with Vault with a permanent token or by k8s service account without creating new tokens again and again?

1 Like
2

When you say the credentials disappear, what are you seeing when you try to connect to your postgres target after the credential library stops working? Are you creating the Vault token as a renewable token and does the Vault token have policies attached in Vault that allow it to renew itself?

3

I created the token for period of 20m and set it in the credentials store.
Does Boundary renew the token even though I’m not using the credentials?

4

Hello @ofir.kuperiu,
Thanks for your report and trying to make Boundary a better product!

The Boundary controller should be running a periodic job that monitors all your credential store Vault tokens, and renews them at about half way to their expiry regardless of having credentials attached to the token. If the only credential store you have is that one with the 20m token, I would expect it to be renewed every 10 minutes. This does, however, have a requirement that your Boundary controller and your Vault server are up and running consistently to be able to perform the renewal.

We have received another report similar to yours, I have not been able to reproduce an issue with token renewal on my side yet. I was planning to spend some cycles next week looking into this.

5

Hello,

i have been experiencing the same issue. I have created Vault token for boundary using terraform (fulfilling the token requirements i.e. renewable, period set etc), boundary does auto renew the token but after sometime token get expired. i could only found below error in logs but this doesnt make sense as same vault permission works on recreation.

{"log":"{"id":"GpS5sB6nQo","source":"https://hashicorp.com/boundary/boundary-5c5fbd6874-6wzwl/controller+worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"vault.(TokenRenewalJob).renewToken","data":{"credential store id":"csvlt_pnJAPVLNni","msg":"Vault credential store current token has expired"}},"datacontentype":"application/cloudevents","time":"2023-01-05T07:02:51.713638399Z"}n","stream":"stderr","time":"2023-01-05T07:02:51.713822812Z"}


{"log":"{"id":"PxMIlDXKRw","source":"https://hashicorp.com/boundary/boundary-5c5fbd6874-6wzwl/controller+worker","specversion":"1.0","type":"error","data":{"error":"vault.(client).renewToken: vault: https://vault.xxx.com: unknown: error #0: Error making API request.nnURL: PUT https://vault.xxx.com/v1/auth/token/renew-selfnCode: 403. Errors:nn* permission denied","error_fields":{"Code":0,"Msg":"vault: https://vault.xxx.com","Op":"vault.(client).renewToken","Wrapped":{"HTTPMethod":"PUT","URL":"https://vault.xxx.com/v1/auth/token/renew-self","StatusCode":403,"RawError":false,"Errors":["permission denied"],"NamespacePath":""}},"id":"e_89l5wEzMZ5","version":"v0.1","op":"vault.(client).renewToken"},"datacontentype":"application/cloudevents","time":"2023-01-05T07:02:51.711629689Z"}n","stream":"stderr","time":"2023-01-05T07:02:51.711872334Z"}
6

Hi @louisruch
We have deployed boundary and vault on Kubernetes PODs . So it is inevitable for us to keep it up all time as PODs are ephemeral . Recently we too have faced this issue in which vault credential token didnot get renewed by Boundary on time . After manually setting the token in Boundary , it worked for some days but then few of the tokens expired while some others were still working .Any suggestion to tackle this issue while running Boundary and Vault on PODs ? Will the controller’s periodic job still run after POD restarts ?

1 Like
7

Does the vault policy has “renew”?
For us it works (once created, we don’t need to manually renew the token)

8

We have these policies + another “read” capability for our specific role in vault -

path "auth/token/lookup-self" {
  capabilities = ["read"]
}

path "auth/token/renew-self" {
  capabilities = ["update"]
}

path "auth/token/revoke-self" {
  capabilities = ["update"]
}

path "sys/leases/renew" {
  capabilities = ["update"]
}

path "sys/leases/revoke" {
  capabilities = ["update"]
}

path "sys/capabilities-self" {
  capabilities = ["update"]
}
9

Hi @louisruch
There is one more doubt , if boundary job cannot renew token on half of the set period (lets say there was some down time ), when will boundary try it again ? Will it retry before token gets expired ?

For example , in the above case where token period has been set to 20m and if boundary job cannot renew around 10m , then only 10m are remaining for the token to be active . Post that it’s lease expires .