Http: TLS handshake error from <IP:PORT> remote error: tls: bad certificate

Hi.
I upgraded our AWS EKS cluster to version 1.25. After that, our Agent injectors refused to work properly showing the following messages:

 2023-10-05T12:05:00.434Z [ERROR] handler: http: TLS handshake error from 10.31.164.232:36034: remote error: tls: bad certificate                                                                               │
│ 2023-10-05T12:05:00.434Z [ERROR] handler: http: TLS handshake error from 10.31.164.232:36018: remote error: tls: bad certificate

I made many attempts to fix it but it seems that nothing works.

Here is our values.yaml file

injector:
  # Mount Path of the Vault Kubernetes Auth Method.
  authPath: "auth/qa-us"
server:
  ha:
    config: |
      listener "tcp" {
        tls_disable = 1
        address = "[::]:8200"
        cluster_address = "[::]:8201"
      }
      storage "dynamodb" {
        ha_enabled = "true"
        region     = "us-east-1"
        table      = "vault-dynamodb-table"
      }
      seal "awskms" {
        region     = "us-east-1"
        kms_key_id = "<KMS_ID>"
      }

Can anyone please let me know what I can do to fix it?

have you identified what those IP’s are?

It’s the cluster endpoint
Name: kubernetes
│ Namespace: default
│ Labels: endpointslice.kubernetes.io/skip-mirror=true
│ Annotations:
│ Subsets:
│ Addresses: 10.31.117.146,10.31.164.232
│ NotReadyAddresses:
│ Ports:
│ Name Port Protocol
│ ---- ---- --------
│ https 443 TCP
│ Events:

Well from the Vault config file shared earlier, it appears that TLS has been disabled in the Vault listener config. But the Kube cluster IP is trying to access Vault via HTTPS. That could be the core of the issue. Please check config of how Vault is being accessed.