UDP probes failed, network may be misconfigured

Consul
1

We have a consul server cluster setup directly on VMs and we are now trying to setup agents on K8s running on different VMs. Consul agent is able to join the cluster but we have below issue

2023-03-09T15:32:57.931Z [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to Consul_Server_1 over TCP but UDP probes failed, network may be misconfigured
2023-03-09T15:32:58.932Z [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to Consul_Server_1 over TCP but UDP probes failed, network may be misconfigured
2023-03-09T15:32:59.932Z [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to Consul_Server_1 over TCP but UDP probes failed, network may be misconfigured

Here is our Deployment, We are not using any service as of now. no portforwarding was done.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: consul-deployment
  labels:
    app: consul
spec:
  selector:
    matchLabels:
      app: consul
  replicas: 1
  template:
    metadata:
      labels:
        app: consul
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: DC
                operator: In
                values:
                - dc1
      containers:
      - name: consul-container
        image: consul_consul_test:0.2
        ports:
          - containerPort: 8500
            name: ui-port
            protocol: TCP 
          - containerPort: 8400
            name: alt-port
          - containerPort: 53
            name: udp-port
            protocol: UDP
          - containerPort: 53
            name: udp-port
            protocol: TCP  
          - containerPort: 8443
            name: https-port
          - containerPort: 8080
            name: http-port
          - containerPort: 8301
            protocol: UDP
            name: serflan
          - containerPort: 8301
            protocol: TCP
            name: serflan            
          - containerPort: 8302
            name: serfwan
            protocol: TCP
          - containerPort: 8302
            name: serfwan
            protocol: UDP         
          - containerPort: 8600
            name: consuldns
            protocol: UDP
          - containerPort: 8600
            name: consuldns
            protocol: TCP  
          - containerPort: 8300
            name: server
        volumeMounts:
        - name: consul-data
          mountPath: /data
      volumes:
        - name: consul-data
          emptyDir:
            sizeLimit: 5Gi