Kerberos not working correctly

Vault
1

We have spent weeks trying to get Kerberos and I am at a loss for why it is not working and figured I would turn to the forum to see if I can get the Issue Resolved.

We are running Vault v1.6.3+ent

We are wanting to use Kerberos authentication with keytabs.

My problem is my UPNDOMAIN is different from my Kerberos REALM.

I get the following error when trying to log in.

vault login -method=kerberos
username=user service=“HTTPS/thv.domain.com”
realm=CORP.DOMAIN.COM
keytab_path="/export/home/account/user.keytab"
krb5conf_path="/etc/krb5.conf"
disable_fast_negotiation=false

Error authenticating: couldn’t log in: [Root cause: KRBMessage_Handling_Error] KRBMessage_Handling_Error: AS Exchange Error: AS_REP is not valid or client password/keytab incorrect < KRBMessage_Handling_Error: KDC did not respond appropriately to FAST negotiation

I set my ldap config up and it is working using LDAP auth. So I did my kerberos ldap the same way as it recommends.

if I set disable_fast_negotiation=true I get an error in my journalctl and also a debug message that shows the SPNEGO token was successful and it is pulling my valid credentials from my kdc server.
[INFO] auth.kerberos.auth_kerberos_0e58d037: 10.164.104.140:8080 user@CORP.DOMAIN.COM - SPNEGO authentication succeeded

[DEBUG] auth.kerberos.auth_kerberos_0e58d037: identity: &{username:user displayName:user realm:CORP.DOMAIN.COM cname:{NameType:1 NameString:[user]} keytab:0xc000fdfee0 password: attributes:map[gokrb5AttributeKeyADCredentials:{EffectiveName:user FullName:user UserID:212775 PrimaryGroupID:513 LogOnTime:{wall:918075300 ext:63753415503 loc:} LogOffTime:{wall:709551516 ext:68937867273 loc:} PasswordLastSet:{wall:648669600 ext:63750206191 loc:} GroupMembershipSIDs:
It pulls all of them

Kerberos config put in as
vault read /auth/kerberos/config
Key Value

service_account svc_account

LDAP config is
$ vault read /auth/kerberos/config/ldap
Key Value

anonymous_group_search false
binddn CN=svc_account,OU=Service Accounts,OU=Other,DC=corp,DC=domain,DC=com
case_sensitive_names false
certificate n/a
deny_null_bind true
discoverdn false
groupattr sAMAccountName
groupdn OU=Restricted Groups,DC=corp,DC=domain,DC=com
groupfilter (&(objectClass=group)(|(samaccountname=Valid_Group)(samaccountname=hash*)(samaccountname=vault*)))
insecure_tls false
starttls false
tls_max_version tls12
tls_min_version tls12
token_bound_cidrs
token_explicit_max_ttl 0s
token_max_ttl 0s
token_no_default_policy false
token_num_uses 0
token_period 0s
token_policies
token_ttl 0s
token_type default
upndomain upndomain.com
url ldap://ldapvip.corp.domain.com
use_token_groups false
userattr cn
userdn CN=Users,DC=corp,DC=domain,DC=com

My kerberos realm is CORP.DOMAIN.COM

My user user@corp.domain.com has a UserPrincipalName=user@upndomain.com. (Different and I know the instructions say to set the upndomain = kerberos realm (LDAP does not work when I do this)
user is in the Valid_Group so it should find it in the search as it works with normal ldap.

I can pull a service ticket from HTTPS/thv.domain.com doing $kvno HTTPS/thv.domain.com
and it returns a kvno 2 which is what I have my keytab kvno number set to for my service account that the service account keytab is using and is put into the config.

Keytab name: FILE:vault.keytab
KVNO Timestamp Principal

2 04/02/2021 08:51:02 svc_account@CORP.DOMAIN.COM (aes256-cts-hmac-sha1-96)

user keytab kvno = 52 and that is in its keytab
slot KVNO Principal

1 1 user@CORP.DOMAIN.COM (aes256-cts-hmac-sha1-96)
2 1 user@CORP.DOMAIN.COM (aes256-cts-hmac-sha1-96)
3 52 user@CORP.DOMAIN.COM (aes256-cts-hmac-sha1-96)
4 0 user@CORP.DOMAIN.COM (aes256-cts-hmac-sha1-96)

I can do a klist -Ae and shoe that I can pull the tgt ticket and service tickets
klist -Ae
Ticket cache: KCM:8877:92714
Default principal: user@CORP.DOMAIN.COM

Valid starting Expires Service principal
04/07/2021 13:04:01 04/07/2021 23:03:45 HTTPS/thv.domain.com@CORP.DOMAIN.COM
Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
04/07/2021 13:03:45 04/07/2021 23:03:45 krbtgt/CORP.DOMAIN.COM@CORP.DOMAIN.COM
renew until 04/14/2021 13:03:45, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
04/07/2021 13:04:07 04/07/2021 23:03:45 HTTPS/thv.domain.com:8200@CORP.DOMAIN.COM
Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

Ticket cache: KCM:8877
Default principal: svc_account@CORP.DOMAIN.COM

Valid starting Expires Service principal
04/07/2021 13:03:22 04/07/2021 23:03:22 krbtgt/CORP.DOMAIN.COM@CORP.DOMAIN.COM
renew until 04/14/2021 13:03:22, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

I will provide any addition configurations setup obfuscated with non-company identifiable info.

Any help will be greatly appreciated. Thanks

2

This is how I got things to work:

Set the upndomain value to the empty string
Set the userattr to “samaccountname”
Make sure you can kinit with your client-side keytab. E.g., kinit -k -t /path/to/client/keytab user@CORP.DOMAIN.COM. It seems you can, so that’s good.

You can use “curl --negotiate -u: …” or you can use “vault login -method=kerberos…” but if the latter gives you a token error, then you may be running into a possible bug where the vault command is not including a needed X-Vault-Namespace header in the request, even if you have the environment variable set or are specifying the namespace on the command line. I’m still seeing this in 1.8.0. You can check this by adding “–output-curl-string” to your “vault login” command. If the client is happy and can generate the SPNEGO token, it will spew a curl command that you can execute. I run something like “vault login -method=kerberos -output-curl-string … | sh -v” to see this. You can then tweak the curl command line as needed.

I found that getting the login to work with (a modern version of) curl to be most instructive when debugging things.

Also, after you’ve run the curl command, make sure that there’s an HTTP/hostname service principal name in your ccache, the SPN associated with svc_account@CORP.DOMAIN.COM.

Also trying to use a service account from a trusted realm (assuming you have some kind cross-realm trust going in your environment) has a high probability of ending in tears.

Give these things a try. Let us know how you get on.

jd