Vault + SPNEGO (MIT Kerberos) + openldap fails

I put together a dumb kerberos realm with a dump openldap configuration trying to test out Kerberos+LDAP sso in Vault. But I am not able to get curl SPNEGO to work, vault login with kerberos keytab works though.

This is my openldap domain,

ldapadd -x -D 'ou=users,dc=test,dc=domain' -W
dn: dc=test,dc=domain
objectClass: dcObject
objectClass: organization
dc: users
description: Users OU

ldapadd -x -D 'cn=Manager,dc=test,dc=domain' -W
dn: dc=test,dc=domain
objectClass: dcObject
objectClass: organization
dc: test
o: test
description: Manager


dn: cn=Manager,dc=test,dc=domain
objectClass: organizationalRole
cn: Manager
description: Directory Manager


dn: ou=groups,dc=test,dc=domain
objectclass:organizationalunit
ou: groups
description: generic groups branch

dn: cn=itpeople,ou=groups,dc=test,dc=domain
objectclass: groupofnames
cn: itpeople
description: IT security group
member: cn=John Doe,ou=users,dc=test,dc=domain


dn: ou=users,dc=test,dc=domain
objectclass:organizationalunit
ou: users
description: generic users branch

dn: cn=John Doe,ou=users,dc=test,dc=domain
objectclass: inetOrgPerson
cn: John Doe
sn: johndoe
uid: johndoe
userPrincipalname=johndoe@TEST.DOMAIN
userpassword: testuser
ou: users

dn: cn=bind2,dc=test,dc=domain
objectclass: inetOrgPerson
sn: bind2
uid: bind2
userpassword: bind2

dn: cn=John Doe,ou=users,dc=test,dc=domain
changetype: delete

dn: cn=John Doe,ou=users,dc=test,dc=domain
objectclass: inetOrgPerson
cn: John Doe
sn: johndoe
uid: johndoe
userpassword: testuser
ou: users
roomNumber: johndoe@TEST.DOMAIN

krb5.conf

[libdefaults]
        default_realm = TEST.DOMAIN

[realms]
# use "kdc = ..." if realm admins haven't put SRV records into DNS
        TEST.DOMAIN = {
                admin_server = kdc.test.domain
                kdc = kdc.test.domain
        }
[logging]

Vault config

vault auth enable -passthrough-request-headers=Authorization -allowed-response-headers=www-authenticate kerberos
vault write auth/kerberos/config keytab=@vault.keytab.base64 service_account="HTTP/vault.test.domain"
vault write auth/kerberos/config/ldap \
    binddn="cn=bind2,dc=test,dc=domain"\
    bindpass=bind2 \
    userdn="ou=users,dc=test,dc=domain" \
    userattr=roomNumber \
    userfilter="(uid={{.Username}})" \
    insecure_tls=false \
    url=ldap://localhost
    groupattr=member \
    groupdn="ou=groups,DC=TEST,DC=DOMAIN"  \
    groupfilter="(&(objectClass=groupofnames)(member:={{.dn}}))"

Curl

2022-05-06T13:18:16.463Z [INFO]  auth.kerberos.auth_kerberos_8b1d6426: 127.0.0.1:8080 - SPNEGO validation error: defective token detected: [Root cause: Decrypting_Error] Decrypting_Error: error decrypting encpart of service ticket provided: KRB Error: (45) KRB_AP_ERR_NOKEY Service key not available - Could not get key from keytab: matching key not found in keytab. Looking for [HTTP vault.test.domain] realm: TEST.DOMAIN kvno: 1 etype: 18

Logs with vault login

> vault login -method=kerberos username=johndoe service=HTTP/vault.test.domain realm=TEST.DOMAIN keytab_path=./user.keytab krb5conf_path=/etc/krb5.conf

2022-05-06T13:24:04.354Z [INFO]  auth.kerberos.auth_kerberos_8b1d6426: 127.0.0.1:8080 johndoe@TEST.DOMAIN - SPNEGO authentication succeeded
2022-05-06T13:24:04.354Z [DEBUG] auth.kerberos.auth_kerberos_8b1d6426: identity: &{username:johndoe displayName:johndoe realm:TEST.DOMAIN cname:{NameType:1 NameString:[johndoe]} keytab:0xc000cf7020 password: attributes:map[] validUntil:{wall:0 ext:63787526644 loc:<nil>} authenticated:true human:true authTime:{wall:354788192 ext:63787440244 loc:<nil>} groupMembership:map[] sessionID:3073a370-0507-12fb-6276-e347f7011670}
2022-05-06T13:24:04.355Z [DEBUG] auth.kerberos.auth_kerberos_8b1d6426: compiling search filter: search_filter="(uid={{.Username}})"
2022-05-06T13:24:04.355Z [DEBUG] auth.kerberos.auth_kerberos_8b1d6426: discovering user: userdn="ou=users,dc=test,dc=domain" filter="(uid=johndoe)"
2022-05-06T13:24:04.356Z [DEBUG] auth.kerberos.auth_kerberos_8b1d6426: auth/ldap: User BindDN fetched: username=johndoe binddn="cn=John Doe,ou=users,dc=test,dc=domain"
2022-05-06T13:24:04.356Z [WARN]  auth.kerberos.auth_kerberos_8b1d6426: groupdn is empty, will not query server
2022-05-06T13:24:04.356Z [DEBUG] auth.kerberos.auth_kerberos_8b1d6426: auth/ldap: Groups fetched from server: num_server_groups=0 server_groups=[]

For the life of me I can’t figure out why HTTP/vault.test.domain turns into
‘HTTP vault.test.domain’ when using Curl but vault login with kerberos works. Notice the space there.

I figured this out. My kerberos setup was broken. From,

Kerberos by default resolves the hostname to an IP address and then back to a hostname. It's likely that something is causing the IP address to resolve to local instead of server.local.