Vault HA all three pods are in standby mode. I have upgraded my Kubernetes cluster to 1.25.10-gke.1200 version during upgrade the vault agent is not able to connect to the vault server for authentication. Can’t relate this to Kubernetes version because another vault instance is running on the same cluster. For both vault instances the configuration is same.
Vault version : v1.13.1
Kuberentes version : 1.25.10-gke.1200
Agent log:
2023-08-01T07:58:00.738Z [ERROR] agent.auth.handler: error authenticating: error=“Put "https://{vault_service_url}/v1/auth/kubernetes/login": dial tcp {service_id}:443: i/o timeout” backoff=4m6.49s
2023-08-01T08:02:07.235Z [INFO] agent.auth.handler: authenticating
2023-08-01T08:02:07.251Z [ERROR] agent.auth.handler: error authenticating:
error=
| Error making API request.
|
| URL: PUT https://{address}:8200/v1/auth/kubernetes/login
| Code: 500. Errors:
|
| * local node not active but active cluster node not found
backoff=4m49.94s
Vault pod log:
2023-08-01T07:40:51.626Z [ERROR] core: forward request error: error=“error during forwarding RPC request”
2023-08-01T07:41:11.573Z [ERROR] core: error during forwarded RPC request: error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing remote error: tls: internal error"”
2023-08-01T07:41:11.573Z [ERROR] core: forward request error: error=“error during forwarding RPC request”
2023-08-01T07:41:13.471Z [ERROR] core: error during forwarded RPC request: error=“rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing remote error: tls: internal error"”
------------------------------------------ && ---------------------------------- from leader pod --------------------------
2023-07-31T22:23:15.941Z [WARN] core.cluster-listener: no TLS config found for ALPN: ALPN=[“req_fw_sb-act_v1”]
2023-07-31T22:24:43.298Z [WARN] core.cluster-listener: no TLS config found for ALPN: ALPN=[“req_fw_sb-act_v1”]
2023-07-31T22:25:10.941Z [WARN] core.cluster-listener: no TLS config found for ALPN: ALPN=[“req_fw_sb-act_v1”]
2023-07-31T22:26:33.297Z [WARN] core.cluster-listener: no TLS config found for ALPN: ALPN=[“req_fw_sb-act_v1”]
2023-07-31T22:27:00.319Z [WARN] core.cluster-listener: no TLS config found for ALPN: ALPN=[“req_fw_sb-act_v1”]