HMAC generation differences with SHA1 between Vault and Python

Vault
1

Hello,

I am currently working on seting up a Vault (Community Edition) server, which my company would like to use to generate HMAC values.

For this I had set up a test server with the Transit secrets engine enabled, then imported a key into it. The server is currently running on a Raspberry Pi 4 and outside of dev mode. For test purposes TLS is disabled since the server is only available internally in the companies network.

I tested sending a request for HMAC generation using a python script. An example script using test data can be seen below. This script is both sending a request to Vault to generate an HMAC and generating one by using Pythons hmac and hashlib modules. Both generations are based on the same test input (a binary type object, which gets converted for Vault) and are using the SHA1 algorithm.

import requests
import base64
import json
import hmac
import hashlib

vault_url=".../v1/transit/hmac/test-key"
vault_token="..."

challenge=b'example'

#preparing vault request data
encoded_challenge=base64.b64encode(challenge).decode()

header={
        'X-Vault-Token': vault_token,
        'Content-Type': 'application/json'
}

payload={
        'input':encoded_challenge,
        'algorithm': 'sha1'
}

#sending request
response = requests.post(vault_url, headers=header, data=json.dumps(payload), verify=False)

#encoding result to be shown as bytes
hmac_result =response.json()['data']['hmac']
hmac_result = hmac_result.replace('vault:v1:','')
hmac_result = base64.b64decode(hmac_result)

#generating hmac via python
key = bytes.fromhex("10e27289adab1826b70b369a3bd4acca973e6165ad58e49425eb4312a433e42c")
python_hmac=hmac.digest(key, challenge, hashlib.sha1)

print('vaults hmac:',hmac_result)
print('pythons hmac:',python_hmac)

As shown in the screenshot below, the results from Vault and Python are different.

vault_support_result
vault_support_result518×131 2.18 KB

I already tried examining Vaults logs (both server logs on debug level and audit logs), where I didn’t find any hints on this problem. I also tried using Vaults debug command, but I don’t have enough experience with Vaullt to be able to use it properly.

Therefore I want to ask if somebody could assist in this problem and might have an idea about the reason why there could be this differences and how I could avoid them.

Best regards,
Tim Lange

2

It doesn’t look like Transit HMAC supports sha1, so it’s likely defaulting to sha2-256. Can you try again with one of the supported algorithms?

4

How are you importing the secret key into Vault?

After some time digging through Vault’s documentation and source code I found that each transit key has an independent, random 256-bit HMAC secret key.

Even when you import a key (rsa, aes, ed25519, ecdsa) Vault will generate a new independent, random 256-bit HMAC secret key. I’m not sure what type of key you are importing, but I believe this is why you are getting different results, as your HMAC secret keys are still different.

There seems to be an undocumented key type specifically for HMAC secret keys. It is not listed under types on the import key documentation. (I’ll work on my end to see if we can get that updated)

Can you try the following? Be sure your HMAC secret key fits the length requirement for Vault (32 - 512 bytes)

# generate 32 byte HMAC secret key
head -c 32 /dev/urandom | base64 > hmac_secret_key.b64

# enable transit (if not enabled)
vault secrets enable transit

# import HMAC secret key, using type=hmac
vault transit import transit/keys/example @hmac_secret_key.b64 type=hmac

I was able to get Vault and Python HMAC to match, using most of your initial code

vaults hmac: b':\xf9\x8c3\x0c\xcb\xba\x0f\xb4\x86\xf4vfTet\xd9O\x9f2z\xd2\xfat\x91\xfc\xcaj\x93\x99B\x8b'
pythons hmac: b':\xf9\x8c3\x0c\xcb\xba\x0f\xb4\x86\xf4vfTet\xd9O\x9f2z\xd2\xfat\x91\xfc\xcaj\x93\x99B\x8b'
5

I tried your suggestion and this worked. The only missing thing was the type specification when I uploaded the key.
Thank you for your efforts^^