Rate limit on dynamic secret mount

Vault
1

Hi,
I am trying to use the rate limit feature for aws mount with no success.
According to documentation the path needs to be mount or namespace. I tried setting the rate limit path to “aws” but it’s not working.
Is it not supposed to work on dynamic secrets mount? Couldn’t find anything about it in documentation.

Many thanks,
Adi

2

Can you explain what you’re trying to achieve? It would also help if you provided the payload and command that you ran.

3

Hi aram,
We have services creating aws users (using vault’s aws/creds API) that sometimes go into a bad restart-loop state. That causes them to create hundreds of users in a very short time.
AWS has a hard limit on the number of users that can be created so when the services are acting up and we reach the limit we are unable to issue more users.
We are working with teams to switch to assumed_role from iam_users so this won’t happen again, but that takes time.
In the mean time I wanted to protect us from this use case by creating rate limit on the aws mount - So that after X requests to aws/creds the quota limit will be exceeded. Given that the rate limit is on the mount I thought that setting “aws” as the rate limit path should work.

Here is the PUT request I used to create it:

 curl \
 --request POST --header "X-Vault-Token: ..." \
 --data '{"path": "aws", "rate": 2}' \
 https://<vault_address>/v1/sys/quotas/rate-limit/aws

It didn’t block any request.
Tested with

curl \
    --header "X-Vault-Token: ..." \
    --request GET \
    "https://<vault_address>/v1/aws/creds/<name>"

Thanks,
Adi

4

Ah ok.

I did this and it worked:

# curl -X POST -H "X-Vault-Token: $VAULT_TOKEN" --data '{ "path": "aws/", "rate": 1, "interval": "1s" }' ${VAULT_ADDR}/v1/sys/quotas/rate-limit/aws-limiter
$ for i in {1..20}; do ./aws_req.sh; done 
// 1 successful message, creds created ... // 
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
{"errors":["request path \"aws/creds/root-role\": rate limit quota exceeded"]}
5

What is your test with? This implies the rate limit is 2 per second, which from what I’ve seen is faster than AWS will satisfy the request anyways.

See /sys/quotas/rate-limit - HTTP API | Vault | HashiCorp Developer
where you should set interval, unless 2/sec is what you want and you’re not seeing that work.

6

Thanks @aram and @mikegreen for the quick replies.
The 2 per second was only to test the rate limit (in reality we will have much less of course to actually be lower than aws’s rate limit).
Want to update that it did work for us. I initially tested this on vault with version 1.5.0. After @aram showed it worked for him I saw there were some fixes related to rate limit in later vault versions. After upgrading to latest it did work.

Thanks :slight_smile: