Database Dynamic Secrets - users/leases being expired before max TTL

elnoxgdl · May 24, 2022, 11:09pm

Hi Everyone,

I’m facing a problem why my dynamic secrets while using databases specifically MySQL when a user logs into the UI and requests a new credential it’s valid for about a week 6-7 days before Vault decides to revoke the lease. I have all my TTLs pretty high so I really don’t understand why is revoking my leases too soon?

The only thing I can think of is the auth method for my users which is OIDC where the token could be expiring soon and that triggers a “cascade” revocation of leases associated with the auth somehow? My auth TTL’s are more than 7 days so that makes no sense, at this point i’m not sure what’s causing this any help will be great!

The problematic role that suppose to be providing credentials with a TTL of 90 days is the following the UI still shows 1 hour but that’s apparently a bug? If I use the CLI it shows the values correctly.

Vault CLI role results:

aram · May 25, 2022, 8:38am

Well your default lease TTL is set to 9 days so unless someone renews that token it is going to expire.

There is something odd going on because max_ttl is a global setting and you’re getting two different values here, so I would check your configuration to see what is going on.

maxb · May 25, 2022, 10:21am

There is a global maximum lease TTL, but there are also options to override it per mount and sometimes per role within a particular mount.

elnoxgdl · May 25, 2022, 8:36pm

Hi Aram,

The default lease TTL is 7776000 is seconds I believe and if I translate to hours is 2160 and both matches 90 days. If its 9 days then that might be the issue however I don’t see anything that matches 9 days ?

Yes, maxb is correct I have a global and a max TTL per mount in this case the dynamic secrets for my database mount is overriden.

maxb · May 26, 2022, 12:14am

Vault will revoke leases if the token used to obtain them expires or is revoked.

Try logging in and using the vault token lookup CLI command (or auth/token/lookup-self API path) to confirm the token TTL you are working with.

Also, if you grant yourself access:

path "sys/leases/lookup/" {
  capabilities = ["list", "sudo"]
}

path "sys/leases/lookup/*" {
  capabilities = ["list", "sudo"]
}

you can also explore the leases present in Vault:

$ vault list sys/leases/lookup
Keys
----
auth/

$ vault list sys/leases/lookup/auth/
Keys
----
userpass/

$ vault list sys/leases/lookup/auth/userpass
Keys
----
login/

$ vault list sys/leases/lookup/auth/userpass/login
Keys
----
admin/

$ vault list sys/leases/lookup/auth/userpass/login/admin
Keys
----
h510f73ec01ad059bdfaacd25cd254123bee9748fd8c3ae71bfcc7afb858c834f
h7d08eb46a87b6b04a5811a3777fff26c1be047368ea41d6030db4132c7ba6b4d
he004132214c8b03de7b5496bc2c4a43f8ec32cd2994817cf3a540f33048a0d66
heaa89c8bf710d6dbb83720175263f23a7de377757239ed785776b9c7998a4b83

and ask for specific expiry information about a particular lease:

$ vault write sys/leases/lookup lease_id=auth/userpass/login/admin/h510f73ec01ad059bdfaacd25cd254123bee9748fd8c3ae71bfcc7afb858c834f
Key             Value
---             -----
expire_time     2022-06-25T01:02:07.808451918+01:00
id              auth/userpass/login/admin/h510f73ec01ad059bdfaacd25cd254123bee9748fd8c3ae71bfcc7afb858c834f
issue_time      2022-05-26T01:02:07.808455266+01:00
last_renewal    <nil>
renewable       true
ttl             719h56m2s

In my example above I only have auth/ leases in my test Vault, but if you have active database leases they’d show up in that API and you could check when they are due to expire.

aram · May 26, 2022, 7:51am

I don’t know what maxb said, I have them blocked.

Your “database” connection has it’s own TTL, although all child tokens are related in this case the TTL from the database connection is not overriden – otherwise you would never be able to create a short term dynamic user.

if I create a db role that has a 60 seconds TTL … it’ll expire in 60 seconds no matter what my auth token is set to or how much time I have left on my auth token (if it’s less than both will expire).

maxb · May 28, 2022, 6:09pm

Gosh, well, it’s nice to find that out. I guess I can stop attempting to have a productive interaction with @aram when I respond to something they said.

I joined this site a bit less than 3 months ago, and in that time, my only interaction with @aram has been in 14 public topics (best as I can find in my activity history), often disagreeing with them or correcting factual points:

And the only response I ever got in all of those from @aram was:

To be blocked simply for challenging incorrect information, is really quite shocking to me!

EDIT: It seems some community member or members decided to flag my post, made to defend my reputation in the face of an implied accusation of having done something worthy of being blocked, as “off-topic” - which seems like an abuse of the flags system to me. I invite anyone who has a problem with my posts to engage with me in reasonable discussion - rather than seeking to suppress my messages using false flags.