Vault-AutoAuth minikube vault-agent randomly not able to authenticate

Following :slight_smile:

2021-02-15T10:18:29.915Z [INFO] sink.server: starting sink server
2021/02/15 10:18:29.915615 [DEBUG] (runner) final config: {“Consul”:{“Address”:"",“Namespace”:"",“Auth”:{“Enabled”:false,“Username”:"",“Password”:""},“Retry”:{“Attempts”:12,“Backoff”:250000000,“MaxBackoff”:60000000000,“Enabled”:true},“SSL”:{“CaCert”:"",“CaPath”:"",“Cert”:"",“Enabled”:false,“Key”:"",“ServerName”:"",“Verify”:true},“Token”:"",“Transport”:{“DialKeepAlive”:30000000000,“DialTimeout”:30000000000,“DisableKeepAlives”:false,“IdleConnTimeout”:90000000000,“MaxIdleConns”:100,“MaxIdleConnsPerHost”:5,“TLSHandshakeTimeout”:10000000000}},“Dedup”:{“Enabled”:false,“MaxStale”:2000000000,“Prefix”:“consul-template/dedup/”,“TTL”:15000000000,“BlockQueryWaitTime”:60000000000},“DefaultDelims”:{“Left”:null,“Right”:null},“Exec”:{“Command”:"",“Enabled”:false,“Env”:{“Denylist”:,“Custom”:,“Pristine”:false,“Allowlist”:},“KillSignal”:2,“KillTimeout”:30000000000,“ReloadSignal”:null,“Splay”:0,“Timeout”:0},“KillSignal”:2,“LogLevel”:“DEBUG”,“MaxStale”:2000000000,“PidFile”:"",“ReloadSignal”:1,“Syslog”:{“Enabled”:false,“Facility”:“LOCAL0”,“Name”:""},“Templates”:[{“Backup”:false,“Command”:"",“CommandTimeout”:30000000000,“Contents”:"{{- with secret “secret/data/myapp/config” }}\n{{ .Data.data.password }}\n{{ end }}\n",“CreateDestDirs”:true,“Destination”:"/etc/secrets/keyfile",“ErrMissingKey”:false,“Exec”:{“Command”:"",“Enabled”:false,“Env”:{“Denylist”:,“Custom”:,“Pristine”:false,“Allowlist”:},“KillSignal”:2,“KillTimeout”:30000000000,“ReloadSignal”:null,“Splay”:0,“Timeout”:30000000000},“Perms”:0,“Source”:"",“Wait”:{“Enabled”:false,“Min”:0,“Max”:0},“LeftDelim”:"",“RightDelim”:"",“FunctionDenylist”:,“SandboxPath”:""},{“Backup”:false,“Command”:"",“CommandTimeout”:30000000000,“Contents”:"{{- with secret “secret/data/myapp/config” }}\n{{ .Data.data.username }}={{ .Data.data.password }}\n{{ end }}\n",“CreateDestDirs”:true,“Destination”:"/tmp/keyfiledir/keyfile",“ErrMissingKey”:false,“Exec”:{“Command”:"",“Enabled”:false,“Env”:{“Denylist”:,“Custom”:,“Pristine”:false,“Allowlist”:},“KillSignal”:2,“KillTimeout”:30000000000,“ReloadSignal”:null,“Splay”:0,“Timeout”:30000000000},“Perms”:0,“Source”:"",“Wait”:{“Enabled”:false,“Min”:0,“Max”:0},“LeftDelim”:"",“RightDelim”:"",“FunctionDenylist”:,“SandboxPath”:""}],“Vault”:{“Address”:“http://192.168.93.59:8200”,“Enabled”:true,“Namespace”:"",“RenewToken”:false,“Retry”:{“Attempts”:12,“Backoff”:250000000,“MaxBackoff”:60000000000,“Enabled”:true},“SSL”:{“CaCert”:"",“CaPath”:"",“Cert”:"",“Enabled”:false,“Key”:"",“ServerName”:"",“Verify”:false},“Transport”:{“DialKeepAlive”:30000000000,“DialTimeout”:30000000000,“DisableKeepAlives”:false,“IdleConnTimeout”:90000000000,“MaxIdleConns”:100,“MaxIdleConnsPerHost”:5,“TLSHandshakeTimeout”:10000000000},“UnwrapToken”:false},“Wait”:{“Enabled”:false,“Min”:0,“Max”:0},“Once”:false,“BlockQueryWaitTime”:60000000000}
2021/02/15 10:18:29.915652 [INFO] (runner) creating watcher
2021-02-15T10:19:29.916Z [ERROR] auth.handler: error authenticating: error=“context deadline exceeded” backoff=2.277351381
2021-02-15T10:19:32.193Z [INFO] auth.handler: authenticating
2021-02-15T10:20:32.194Z [ERROR] auth.handler: error authenticating: error=“context deadline exceeded” backoff=2.62356052
2021-02-15T10:20:34.818Z [INFO] auth.handler: authenticating
2021-02-15T10:21:15.018Z [ERROR] auth.handler: error authenticating: error=“Put “http://192.168.93.59:8200/v1/auth/kubernetes/login”: EOF” backoff=1.66253271
2021-02-15T10:21:16.680Z [INFO] auth.handler: authenticating
2021-02-15T10:21:16.681Z [ERROR] auth.handler: error authenticating: error=“Put “http://192.168.93.59:8200/v1/auth/kubernetes/login”: dial tcp 192.168.93.59:8200: connect: connection refused” backoff=2.766047824
2021-02-15T10:21:19.447Z [INFO] auth.handler: authenticating
2021-02-15T10:21:19.447Z [ERROR] auth.handler: error authenticating: error=“Put “http://192.168.93.59:8200/v1/auth/kubernetes/login”: dial tcp 192.168.93.59:8200: connect: connection refused” backoff=2.427155335
2021-02-15T10:21:21.875Z [INFO] auth.handler: authenticating
2021-02-15T10:22:21.875Z [ERROR] auth.handler: error authenticating: error=“context deadline exceeded” backoff=2.104642714
2021-02-15T10:22:23.980Z [INFO] auth.handler: authenticating
2021-02-15T10:23:23.981Z [ERROR] auth.handler: error authenticating: error=“context deadline exceeded” backoff=2.32387425
2021-02-15T10:23:26.305Z [INFO] auth.handler: authenticating
2021-02-15T10:24:26.305Z [ERROR] auth.handler: error authenticating: error=“context deadline exceeded” backoff=1.9061328039999998
2021-02-15T10:24:28.212Z [INFO] auth.handler: authenticating

have enable reverse port forwarding from minikube.
It was working very recently.
I am able to do curl seal-status from minikube vm to vault process running on windows host.

If both vault and minikube knows each other with good ip it wont happen