KV V2 Secret - 403 Forbidden - ONLY for exact string

Vault
1

Tried to put in a new secret today and got a Permission denied error for a very specific string and couldn’t figure out why. It’s happening in all interfaces, the UI, HTTP API, and CLI:

$ export secret_path="test/app"
$ vault kv put -format=json secret/$secret_path special_key='/ONE-2023?retryWrites=true&w=majority'

Error writing data to secret/<path here>: Error making API request.

URL: PUT <vault addr>/v1/secret/data/$secret_path
Code: 403. Raw Message:

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
</body>
</html>

I found that if I remove just the substring ‘ONE-2023’ from the secret it works though. Can anyone explain what may be going on here?

$ curl --request POST -d "@payload.json" -H "X-Vault-Token: $VAULT_TOKEN"  "$VAULT_ADDR/v1/secret/data/$secret_path"
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
</body>
</html>

$ cat payload.json
{
	"data": {
		"mongodb_uri": "/ONE-2023?retryWrites=true&w=majority"
	}
}

I don’t think it’s a special character issue since the substring I remove is simple and technically the same if it needed to have URL encoding. Really odd that it works after removing ‘ONE-2023’ but get a 403 forbidden when it’s included.

2

Could this be coming from some load balancer or security appliance in front of Vault?

The actual Vault API does not return HTML error responses as far as I know.

1 Like
3

Ah! I bet that’s it. Thank you for that prompt. it’s definitely the application firewall in between.

4

Sure enough… changed the rule from BLOCK to COUNT for:
AWSManagedRulesCommonRuleSet#CrossSiteScripting_BODY

Secret update worked after that.