Consul Service Mesh with Apisix Gateway

Hi everyone. I am implementing microservices and using consul as service mesh. I am struggling to install apisix as API Gateway. Current guides only include configuration for service discovery, not service mesh. I barely injected apisix to consul mesh, but I have to put this annotations for it because two services connected to one pods:

       consul.hashicorp.com/connect-inject: 'true'
       consul.hashicorp.com/transparent-proxy: 'false'
       consul.hashicorp.com/connect-service: 'aapisix-gateway, apisix-admin'
       consul.hashicorp.com/connect-service-port: '80,9180'
       consul.hashicorp.com/enable-metrics: 'false'

for ingress controller:

    consul.hashicorp.com/connect-inject: 'true'
    consul.hashicorp.com/transparent-proxy: 'false'
    consul.hashicorp.com/connect-service: 'ingress-controller,ingress-controller-apisix-gateway'
    consul.hashicorp.com/connect-service-port: '80,80'
    consul.hashicorp.com/enable-metrics: 'false'

But apisix gateway can not connect to my application servers in mesh. I thought there was problem with

    consul.hashicorp.com/transparent-proxy: 'false'

So i added upstreams:

      consul.hashicorp.com/transparent-proxy-exclude-outbound-cidrs: "10.96.0.1/32"
      consul.hashicorp.com/connect-service-upstreams: "integration-service.svc:8081"

But upstream only created for one service other service left without upstream. Now, I am wondering is there any proper guide to integrate apisix gateway with consul mesh in kubernetes?

Full apisix helm chart values :

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

global:
  # e.g.
  # imagePullSecrets:
  #   - my-registry-secrets
  #   - other-registry-secrets
  # -- Global Docker registry secret names as an array
  imagePullSecrets: []
  # -- Secrets to be created with apisix
  secrets: []
    # - name: secret-name
    #   values:
    #     - name: secret-key
    #       value: secret=value
service:
  type: LoadBalancer
podAnnotations:
    consul.hashicorp.com/connect-inject: 'true'
    consul.hashicorp.com/transparent-proxy: 'false'
    consul.hashicorp.com/connect-service: 'aapisix-gateway, apisix-admin'
    consul.hashicorp.com/connect-service-port: '80,9180'
    consul.hashicorp.com/enable-metrics: 'false'
    consul.hashicorp.com/transparent-proxy-exclude-outbound-cidrs: "10.96.0.1/32"
    consul.hashicorp.com/connect-service-upstreams: "integration-service.svc:8080"
apisix:
  # -- Enable or disable Apache APISIX itself
  # Set it to false and ingress-controller.enabled=true will deploy only ingress-controller
  enabled: true 
  # -- Enable nginx IPv6 resolver
  enableIPv6: true

  # -- Whether the APISIX version number should be shown in Server header
  enableServerTokens: true

  enableControl: true
  # -- Use Pod metadata.uid as the APISIX id.
  setIDFromPodUID: false
  # -- Add custom [lua_shared_dict](https://github.com/openresty/lua-nginx-module#toc88) settings,
  # click [here](https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix/values.yaml#L27-L30) to learn the format of a shared dict
  customLuaSharedDicts: []
    # - name: foo
    #   size: 10k
    # - name: bar
    #   size: 1m
  # -- Whether to add a custom lua module
  luaModuleHook:
    enabled: false
    # -- extend lua_package_path to load third party code
    luaPath: ""
    # -- the hook module which will be used to inject third party code into APISIX
    # use the lua require style like: "module.say_hello"
    hookPoint: ""
    # -- configmap that stores the codes
    configMapRef:
      # -- Name of the ConfigMap where the lua module codes store
      name: ""
      # mounts decides how to mount the codes to the container.
      mounts:
        # -- Name of the ConfigMap key, for setting the mapping relationship between ConfigMap key and the lua module code path.
        - key: ""
        # -- Filepath of the plugin code, for setting the mapping relationship between ConfigMap key and the lua module code path.
          path: ""

  # -- Defines how apisix handles routing:
  # - radixtree_uri: match route by uri(base on radixtree)
  # - radixtree_host_uri: match route by host + uri(base on radixtree)
  # - radixtree_uri_with_parameter: match route by uri with parameters
  httpRouter: radixtree_host_uri
   

      # Integration service discovery registry. E.g eureka\dns\nacos\consul_kv
      # reference:
      # https://apisix.apache.org/docs/apisix/discovery/#configuration-for-eureka
      # https://apisix.apache.org/docs/apisix/discovery/dns/#service-discovery-via-dns
      # https://apisix.apache.org/docs/apisix/discovery/consul_kv/#configuration-for-consul-kv
      # https://apisix.apache.org/docs/apisix/discovery/nacos/#configuration-for-nacos
      # https://apisix.apache.org/docs/apisix/discovery/kubernetes/#configuration
      #
      # an eureka example:
      # ```
      # eureka:
      #   host:
      #     - "http://${username}:${password}@${eureka_host1}:${eureka_port1}"
      #     - "http://${username}:${password}@${eureka_host2}:${eureka_port2}"
      #   prefix: "/eureka/"
      #   fetch_interval: 30
      #   weight: 100
      #   timeout:
      #     connect: 2000
      #     send: 2000
      #     read: 5000
      # ```
      #
      # the minimal Kubernetes example:
      # ```
      # kubernetes: {}
      # ```
      #
      # The prerequisites for the above minimal Kubernetes example:
      #  1. [Optional] Set `.serviceAccount.create` to `true` to create a dedicated ServiceAccount.
      #     It is recommended to do so, otherwise the default ServiceAccount "default" will be used.
      #  2. [Required] Set `.rbac.create` to `true` to create and bind the necessary RBAC resources.
      #     This grants the ServiceAccount in use to List-Watch Kubernetes Endpoints resources.
      #  3. [Required] Include the following environment variables in `.nginx.envs` to pass them into
      #     nginx worker processes (https://nginx.org/en/docs/ngx_core_module.html#env):
      #      - KUBERNETES_SERVICE_HOST
      #      - KUBERNETES_SERVICE_PORT
      #     This is for allowing the default `host` and `port` of `.discovery.registry.kubernetes.service`.
  # -- Enable Proxy Protocol
  proxyProtocol:
    enabled: false
    # -- The port with proxy protocol for http, it differs from node_listen and admin_listen.
    listenHttpPort: 9181
    # -- The port with proxy protocol for https
    listenHttpsPort: 9182
    # -- Enable the proxy protocol for tcp proxy, it works for stream_proxy.tcp option
    tcp: true
    # -- Enable the proxy protocol to the upstream server
    upstream: true

  # -- Enable Data Encryption
  data_encryption:
    enabled: false
    # -- An array of 16 character strings used to encrypt/decrypt fields with AES-128-CBC
    keyring: []

  # -- Enable full customized config.yaml
  enableCustomizedConfig: false
  # -- If apisix.enableCustomizedConfig is true, full customized config.yaml.
  # Please note that other settings about APISIX config will be ignored
  customizedConfig: {}

  image:
    # -- Apache APISIX image repository
    repository: apache/apisix
    # -- Apache APISIX image pull policy
    pullPolicy: IfNotPresent
    # -- Apache APISIX image tag
    # Overrides the image tag whose default is the chart appVersion.
    tag: 3.8.0-debian

  # -- Use a `DaemonSet` or `Deployment`
  kind: Deployment
  # -- kind is DaemonSet, replicaCount not become effective
  replicaCount: 1

  # -- Set [priorityClassName](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#pod-priority) for Apache APISIX pods
  priorityClassName: ""
  # -- Annotations to add to each pod
  # -- Set the securityContext for Apache APISIX pods
  podSecurityContext: {}
    # fsGroup: 2000
  # -- Set the securityContext for Apache APISIX container
  securityContext: {}
    # capabilities:
    #   drop:
    #   - ALL
    # readOnlyRootFilesystem: true
    # runAsNonRoot: true
    # runAsUser: 1000

  # -- See https://kubernetes.io/docs/tasks/run-application/configure-pdb/ for more details
  podDisruptionBudget:
    # -- Enable or disable podDisruptionBudget
    enabled: false
    # -- Set the `minAvailable` of podDisruptionBudget. You can specify only one of `maxUnavailable` and `minAvailable` in a single PodDisruptionBudget.
    # See [Specifying a Disruption Budget for your Application](https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget)
    # for more details
    minAvailable: 90%
    # -- Set the maxUnavailable of podDisruptionBudget
    maxUnavailable: 1

  # -- Set pod resource requests & limits
  resources: {}
    # -- Use the host's network namespace

    # We usually recommend not to specify default resources and to leave this as a conscious
    # choice for the user. This also increases chances charts run on environments with little
    # resources, such as Minikube. If you do want to specify resources, uncomment the following
    # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
    # limits:
    #   cpu: 100m
    #   memory: 128Mi
    # requests:
    #   cpu: 100m
    #   memory: 128Mi
  hostNetwork: false

  # -- Node labels for Apache APISIX pod assignment
  nodeSelector: {}
  # -- List of node taints to tolerate
  tolerations: []
  # -- Set affinity for Apache APISIX deploy
  affinity: {}
  # -- Topology Spread Constraints for pod assignment spread across your cluster among failure-domains
  # ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/#spread-constraints-for-pods
  topologySpreadConstraints: []

  # -- timezone is the timezone where apisix uses.
  # For example: "UTC" or "Asia/Shanghai"
  # This value will be set on apisix container's environment variable TZ.
  # You may need to set the timezone to be consistent with your local time zone,
  # otherwise the apisix's logs may used to retrieve event maybe in wrong timezone.
  timezone: ""

  # -- extraEnvVars An array to add extra env vars
  # e.g:
  # extraEnvVars:
  #   - name: FOO
  #     value: "bar"
  #   - name: FOO2
  #     valueFrom:
  #       secretKeyRef:
  #         name: SECRET_NAME
  #         key: KEY
  extraEnvVars: []

  # -- Set the readinessProbe for Apache APISIX pods
  readinessProbe:
    failureThreshold: 6
    initialDelaySeconds: 10
    periodSeconds: 10
    successThreshold: 1
    tcpSocket:
      port: &gatewayHttpContainerPort 9080
    timeoutSeconds: 1

nameOverride: ""
fullnameOverride: ""

  #serviceAccount:
  #  create: true
  #  annotations: {}
  #  name: "apisix-gateway"

rbac:
  create: false

deployment:
  # -- Apache APISIX deployment mode
  # Optional: traditional, decoupled, standalone
  #
  # ref: https://apisix.apache.org/docs/apisix/deployment-modes/
  mode: traditional

  # -- Deployment role
  # Optional: traditional, data_plane, control_plane
  #
  # ref: https://apisix.apache.org/docs/apisix/deployment-modes/
  role: "traditional"

gateway:
  enabled: false
  # -- Apache APISIX service type for user access itself
  service:
    type: LoadBalancer
  # -- Setting how the Service route external traffic
  # If you want to keep the client source IP, you can set this to Local.

  # ref: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip
  externalTrafficPolicy: Cluster
  # type: LoadBalancer
  annotations: {}
  #   service.beta.kubernetes.io/aws-load-balancer-type: nlb
  externalIPs: []
  # -- Apache APISIX service settings for http
  http:
    enabled: true
    servicePort: 80
    containerPort: *gatewayHttpContainerPort
    # -- Support multiple http ports, See [Configuration](https://github.com/apache/apisix/blob/0bc65ea9acd726f79f80ae0abd8f50b7eb172e3d/conf/config-default.yaml#L24)
    additionalContainerPorts: []
      # - port: 9081
      #   enable_http2: true          # If not set, the default value is `false`.
      # - ip: 127.0.0.2               # Specific IP, If not set, the default value is `0.0.0.0`.
      #   port: 9082
      #   enable_http2: true
  # -- Apache APISIX service settings for tls
  tls:
    enabled: false
    servicePort: 443
    containerPort: 9443
    # -- Support multiple https ports, See [Configuration](https://github.com/apache/apisix/blob/0bc65ea9acd726f79f80ae0abd8f50b7eb172e3d/conf/config-default.yaml#L99)
    additionalContainerPorts: []
      # - ip: 127.0.0.3           # Specific IP, If not set, the default value is `0.0.0.0`.
      #   port: 9445
      #   enable_http2: true
    # -- Specifies the name of Secret contains trusted CA certificates in the PEM format used to verify the certificate when APISIX needs to do SSL/TLS handshaking with external services (e.g. etcd)
    existingCASecret: ""
    # -- Filename be used in the gateway.tls.existingCASecret
    certCAFilename: ""
    http2:
      enabled: true
    # -- TLS protocols allowed to use.
    sslProtocols: "TLSv1.2 TLSv1.3"
    # -- Define SNI to fallback if none is presented by client
    fallbackSNI: ""
  # -- Proxy Protocol Configuration
  proxyProtocol:
    # -- If you enable proxy protocol, you must use this port to receive http request with proxy protocol
    http:
      enabled: false
      # - Specify NodePort (only if gateway.type is NodePort)
      # nodePort:
      # - Define a Service Port on which the gateway is listening
      servicePort: 9181
      # - Gateway Service Port to use as target
      containerPort: 9181
    # -- The port with proxy protocol for https
    https:
      enabled: false
      # - Specify NodePort (only if gateway.type is NodePort)
      # nodePort:
      # - Define a Service Port on which the gateway is listening
      servicePort: 9182
      # - Gateway Service Port to use as target
      containerPort: 9182
  # -- Apache APISIX service settings for stream. L4 proxy (TCP/UDP)
  stream:
    enabled: false
    tcp: []
    udp: []
  # -- Using ingress access Apache APISIX service
  ingress:
    enabled: true
    # -- Ingress annotations
    annotations: {}
      # kubernetes.io/ingress.class: nginx
      # kubernetes.io/tls-acme: "true"
    hosts:
      - host: apisix.local
        paths: []
    tls: []
  #  - secretName: apisix-tls
  #    hosts:
  #      - chart-example.local
  # -- Override default labels assigned to Apache APISIX gateway resources
  # labelsOverride:
  #   app.kubernetes.io/name: "{{ .Release.Name }}"
  #   app.kubernetes.io/instance: '{{ include "apisix.name" . }}'
admin:
  # -- Enable Admin API
  enabled: true
  # -- admin service type
  type: ClusterIP
  # loadBalancerIP: a.b.c.d
  # loadBalancerSourceRanges:
  #   - "143.231.0.0/16"
  # -- IPs for which nodes in the cluster will also accept traffic for the servic
  externalIPs: []
  # -- which ip to listen on for Apache APISIX admin API. Set to `"[::]"` when on IPv6 single stack
  ip: 0.0.0.0
  # -- which port to use for Apache APISIX admin API
  port: 9180
  # -- Service port to use for Apache APISIX admin API
  servicePort: 9180
  # -- Admin API support CORS response headers
  cors: true
  # -- Admin API credentials
  credentials:
    # -- Apache APISIX admin API admin role credentials
    admin: edd1c9f034335f136f87ad84b625c8f1
    # -- Apache APISIX admin API viewer role credentials
    viewer: 4054f7cf07e344346cd3f287985e76a2

    # -- The APISIX Helm chart supports storing user credentials in a secret.
    # The secret needs to contain two keys, admin and viewer, with their respective values set.
    secretName: ""
    # -- Name of the admin role key in the secret, overrides the default key name "admin"
    secretAdminKey: ""
    # -- Name of the viewer role key in the secret, overrides the default key name "viewer"
    secretViewerKey: ""

  allow:
    # -- The client IP CIDR allowed to access Apache APISIX Admin API service.
    ipList:
      - 0.0.0.0/0
  # -- Using ingress access Apache APISIX admin service
  ingress:
    enabled: false
    # -- Ingress annotations
    annotations: {}
      # kubernetes.io/ingress.class: nginx
      # kubernetes.io/tls-acme: "true"
    hosts:
      - host: apisix-admin.local
        paths:
          - "/apisix"
    tls: []
  #  - secretName: apisix-tls
  #    hosts:
  #      - chart-example.local

nginx:
  workerRlimitNofile: "20480"
  workerConnections: "10620"
  workerProcesses: auto
  enableCPUAffinity: true
  user: root
  envs: []
  # -- allow customize meta in `nginx_config` section
  meta: |
    lua_shared_dict:
      prometheus-metrics: 15m

# -- Customize the list of APISIX plugins to enable. By default, APISIX's default plugins are automatically used. See [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml)
plugins: []
# -- Customize the list of APISIX stream_plugins to enable. By default, APISIX's default stream_plugins are automatically used. See [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml)
stream_plugins: []

# -- Set APISIX plugin attributes, see [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml#L376) for more details
pluginAttrs: {}

extPlugin:
  # -- Enable External Plugins. See [external plugin](https://apisix.apache.org/docs/apisix/next/external-plugin/)
  enabled: false
  # -- the command and its arguements to run as a subprocess
  cmd: ["/path/to/apisix-plugin-runner/runner", "run"]

wasmPlugins:
  # -- Enable Wasm Plugins. See [wasm plugin](https://apisix.apache.org/docs/apisix/next/wasm/)
  enabled: false
  plugins: []

# -- customPlugins allows you to mount your own HTTP plugins.
customPlugins:
  # -- Whether to configure some custom plugins
  enabled: true
  # -- the lua_path that tells APISIX where it can find plugins,
  # note the last ';' is required.
  luaPath: "/opts/custom_plugins/?.lua"
  plugins:
    # -- plugin name.
    - name: "plugin-name"
      # -- plugin attrs
      attrs: {}
      # -- plugin codes can be saved inside configmap object.
      configMap:
        # -- name of configmap.
        name: "configmap-name"
        # -- since keys in configmap is flat, mountPath allows to define the mount
        # path, so that plugin codes can be mounted hierarchically.
        mounts:
          - key: "the-file-name"
            path: "mount-path"

# -- Update strategy for apisix deployment
updateStrategy: {}
  # type: RollingUpdate

# -- Additional `volume`, See [Kubernetes Volumes](https://kubernetes.io/docs/concepts/storage/volumes/) for the detail.
extraVolumes: []
# - name: extras
#   emptyDir: {}

# -- Additional `volume`, See [Kubernetes Volumes](https://kubernetes.io/docs/concepts/storage/volumes/) for the detail.
extraVolumeMounts: []
# - name: extras
#   mountPath: /usr/share/extras
#   readOnly: true

# -- Additional `initContainers`, See [Kubernetes initContainers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) for the detail.
extraInitContainers:
  - name: init-sysctl
    image: busybox:1.28
    command:
      - /bin/sh
      - -c
      - |
        sysctl -w net.core.somaxconn=65535
        sysctl -w net.ipv4.ip_local_port_range="1024 65535"
        sysctl -w net.ipv4.tcp_max_syn_backlog=8192
        sysctl -w fs.file-max=1048576
        sysctl -w fs.inotify.max_user_instances=16384
        sysctl -w fs.inotify.max_user_watches=524288
        sysctl -w fs.inotify.max_queued_events=16384
    securityContext:
      privileged: true
# - name: init-myservice
#   image: busybox:1.28
#   command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"]

# access log and error log configuration
logs:
  # -- Enable access log or not, default true
  enableAccessLog: true
  # -- Access log path
  accessLog: "/dev/stdout"
  # -- Access log format
  accessLogFormat: '$remote_addr - $remote_user [$time_local] $http_host \"$request\" $status $body_bytes_sent $request_time \"$http_referer\" \"$http_user_agent\" $upstream_addr $upstream_status $upstream_response_time \"$upstream_scheme://$upstream_host$upstream_uri\"'
  # -- Allows setting json or default characters escaping in variables
  accessLogFormatEscape: default
  # -- Error log path
  errorLog: "/dev/stderr"
  # -- Error log level
  errorLogLevel: "warn"

dns:
  resolvers:
    - 127.0.0.1
    - 172.20.0.10
    - 114.114.114.114
    - 223.5.5.5
    - 1.1.1.1
    - 8.8.8.8
  validity: 30
  timeout: 5

initContainer:
  # -- Init container image
  image: busybox
  # -- Init container tag
  tag: 1.28
  # -- Set the securityContext for the init container
  securityContext: {}
    # capabilities:
    #   drop:
    #   - ALL
    # readOnlyRootFilesystem: true
    # runAsNonRoot: true
    # runAsUser: 1000

autoscaling:
  enabled: false
  # -- HPA version, the value is "v2" or "v2beta1", default "v2"
  version: v2
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  targetMemoryUtilizationPercentage: 80

# -- Custom configuration snippet.
configurationSnippet:
  main: |

  httpStart: |

  httpEnd: |

  httpSrv: |

  httpAdmin: |

  stream: |

# -- Observability configuration.
# ref: https://apisix.apache.org/docs/apisix/plugins/prometheus/
serviceMonitor:
  # -- Enable or disable Apache APISIX serviceMonitor
  enabled: false
  # -- namespace where the serviceMonitor is deployed, by default, it is the same as the namespace of the apisix
  namespace: ""
  # -- name of the serviceMonitor, by default, it is the same as the apisix fullname
  name: ""
  # -- interval at which metrics should be scraped
  interval: 15s
  # -- path of the metrics endpoint
  path: /apisix/prometheus/metrics
  # -- prefix of the metrics
  metricPrefix: apisix_
  # -- container port where the metrics are exposed
  containerPort: 9091
  # -- @param serviceMonitor.labels ServiceMonitor extra labels
  labels: {}
  # -- @param serviceMonitor.annotations ServiceMonitor annotations
  annotations: {}
  # -- @param serviceMonitor.metricRelabelings MetricRelabelConfigs to apply to samples before ingestion.
  # ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs
  metricRelabelings: []

# -- etcd configuration
# use the FQDN address or the IP of the etcd
etcd:
  # -- install etcd(v3) by default, set false if do not want to install etcd(v3) together
  enabled: true
  # -- if etcd.enabled is false, use external etcd, support multiple address, if your etcd cluster enables TLS, please use https scheme, e.g. https://127.0.0.1:2379.
  host:
    # host or ip e.g. http://172.20.128.89:2379
    - http://etcd.host:2379
  # -- if etcd.enabled is false, username for external etcd. If etcd.enabled is true, use etcd.auth.rbac.rootPassword instead.
  user: ""
  # -- if etcd.enabled is false, password for external etcd. If etcd.enabled is true, use etcd.auth.rbac.rootPassword instead.
  password: ""
  # -- apisix configurations prefix
  prefix: "/apisix"
  # -- Set the timeout value in seconds for subsequent socket operations from apisix to etcd cluster
  timeout: 30
  # -- Name of the existing secret containing user and password for external etcd, overrides etcd.user and etcd.password
  existingSecret: ""
  # -- Name of key containing username to be retrieved from the existing secret, has a value of user by default
  existingSecretUserKey: ""
  # -- Name of key containing password to be retrieved from the existing secret, has a value of password by default
  existingSecretPasswordKey: ""

  # -- if etcd.enabled is true, set more values of bitnami/etcd helm chart
  auth:
    rbac:
      # -- No authentication by default. Switch to enable RBAC authentication
      create: false
      # -- root password for etcd. Requires etcd.auth.rbac.create to be true.
      rootPassword: ""
      # -- Name of the existing secret containing credentials for the root user
      existingSecret: ""
      # -- Name of key containing password to be retrieved from the existing secret
      existingSecretPasswordKey: ""
    tls:
      # -- enable etcd client certificate
      enabled: false
      # -- name of the secret contains etcd client cert
      existingSecret: ""
      # -- etcd client cert filename using in etcd.auth.tls.existingSecret
      certFilename: ""
      # -- etcd client cert key filename using in etcd.auth.tls.existingSecret
      certKeyFilename: ""
      # -- whether to verify the etcd endpoint certificate when setup a TLS connection to etcd
      verify: true
      # -- specify the TLS Server Name Indication extension, the ETCD endpoint hostname will be used when this setting is unset.
      sni: ""

  # -- ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container
  # -- added for backward compatibility with old kubernetes versions, as seccompProfile is not supported in kubernetes < 1.19
  containerSecurityContext:
    enabled: false

  service:
    port: 2379

  replicaCount: 3

dashboard:
  enabled: true
  service:
    type: LoadBalancer
  config:
    conf:
      etcd:
        # -- Supports defining multiple etcd host addresses for an etcd cluster
        endpoints:
          - apisix-etcd:2379
        # -- apisix configurations prefix
        prefix: "/apisix"
        # -- Specifies etcd basic auth username if enable etcd auth
        username: ~
        # -- Specifies etcd basic auth password if enable etcd auth
        password: ~

# -- Ingress controller configuration
ingress-controller:
  nameOverride: ingress-controller
  fullnameOverride: ingress-controller
  serviceAccount:
  # -- Specifies whether a ServiceAccount should be created
    create: false
  podAnnotations:
    consul.hashicorp.com/connect-inject: 'true'
    consul.hashicorp.com/transparent-proxy: 'false'
    consul.hashicorp.com/connect-service: 'ingress-controller,ingress-controller-apisix-gateway'
    consul.hashicorp.com/connect-service-port: '80,80'
    consul.hashicorp.com/enable-metrics: 'false'
    consul.hashicorp.com/transparent-proxy-exclude-outbound-cidrs: "10.96.0.1/32"
    consul.hashicorp.com/connect-service-upstreams: "integration-service.svc:8081"
  enabled: true
  config:
    apisix:
      adminAPIVersion: "v3"
      serviceFullname: "apisix-admin.consul.svc"
      discovery: {}

vault:
  # -- Enable or disable the vault integration
  enabled: false
  # -- The host address where the vault server is running.
  host: ""
  # -- HTTP timeout for each request.
  timeout: 10
  # -- The generated token from vault instance that can grant access to read data from the vault.
  token: ""
  # -- Prefix allows you to better enforcement of policies.
  prefix: ""

Service accounts:

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: consul
  name: ingress-controller
automountServiceAccountToken: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: consul
  name: ingress-controller-apisix-gateway
automountServiceAccountToken: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: consul
  name: apisix-admin
automountServiceAccountToken: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: consul
  name: apisix-gateway
automountServiceAccountToken: true

service account secrets:

apiVersion: v1
kind: Secret
metadata:
  name: ingress-controller
  namespace: consul
  annotations:
    kubernetes.io/service-account.name: ingress-controller
type: kubernetes.io/service-account-token
---
apiVersion: v1
kind: Secret
metadata:
  name: ingress-controller-apisix-gateway
  namespace: consul
  annotations:
    kubernetes.io/service-account.name: ingress-controller-apisix-gateway
type: kubernetes.io/service-account-token
---
apiVersion: v1
kind: Secret
metadata:
  name: apisix-admin
  namespace: consul
  annotations:
    kubernetes.io/service-account.name: apisix-admin
type: kubernetes.io/service-account-token
---
apiVersion: v1
kind: Secret
metadata:
  name: apisix-gateway
  namespace: consul
  annotations:
    kubernetes.io/service-account.name: apisix-gateway
type: kubernetes.io/service-account-token

Thank you in advance.

Hi @Sharofiddin ,

Welcome to the HashiCorp Forums!

I think making the following changes would make it work for you.

  1. set consul.hashicorp.com/transparent-proxy to true
  2. Once you do step 1, you don’t have to explicitly set the connect-service-upstreams annotation.
  3. Remove transparent-proxy-exclude-outbound-cidrs annotation.
  4. Add exclude-inbound-ports annotation and exclude the API Gateway listener port (so that it can be bypassed from transparent proxy iptables redirection).
  5. You will also have to set dialedDirectly proxy defaults
  6. Create intentions for the gateway to talk to upstream services.

The following documentation would be helpful I believe: Configure Ingress Controllers for Consul on Kubernetes | Consul | HashiCorp Developer

Now I am getting this error

                 ----     ------        ----                   ----                   -------
  Warning  FailedCreate  105s (x16 over 4m29s)  replicaset-controller  Error creating: admission webhook "consul-connect-injector.consul.hashicorp.com" denied the request: multi port services are not compatible with transparent proxy

My full values.yaml file:

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

global:
 # e.g.
 # imagePullSecrets:
 #   - my-registry-secrets
 #   - other-registry-secrets
 # -- Global Docker registry secret names as an array
 imagePullSecrets: []
 # -- Secrets to be created with apisix
 secrets: []
   # - name: secret-name
   #   values:
   #     - name: secret-key
   #       value: secret=value
service:
 type: LoadBalancer
podAnnotations:
   consul.hashicorp.com/connect-inject: 'true'
   consul.hashicorp.com/transparent-proxy: 'true'
   consul.hashicorp.com/connect-service: 'apisix-gateway,apisix-admin'
   consul.hashicorp.com/connect-service-port: '80,9180'
   consul.hashicorp.com/exclude-inbound-ports: '30720'
apisix:
 # -- Enable or disable Apache APISIX itself
 # Set it to false and ingress-controller.enabled=true will deploy only ingress-controller
 enabled: true 
 # -- Enable nginx IPv6 resolver
 enableIPv6: true

 # -- Whether the APISIX version number should be shown in Server header
 enableServerTokens: true

 enableControl: true
 # -- Use Pod metadata.uid as the APISIX id.
 setIDFromPodUID: false
 # -- Add custom [lua_shared_dict](https://github.com/openresty/lua-nginx-module#toc88) settings,
 # click [here](https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix/values.yaml#L27-L30) to learn the format of a shared dict
 customLuaSharedDicts: []
   # - name: foo
   #   size: 10k
   # - name: bar
   #   size: 1m
 # -- Whether to add a custom lua module
 luaModuleHook:
   enabled: false
   # -- extend lua_package_path to load third party code
   luaPath: ""
   # -- the hook module which will be used to inject third party code into APISIX
   # use the lua require style like: "module.say_hello"
   hookPoint: ""
   # -- configmap that stores the codes
   configMapRef:
     # -- Name of the ConfigMap where the lua module codes store
     name: ""
     # mounts decides how to mount the codes to the container.
     mounts:
       # -- Name of the ConfigMap key, for setting the mapping relationship between ConfigMap key and the lua module code path.
       - key: ""
       # -- Filepath of the plugin code, for setting the mapping relationship between ConfigMap key and the lua module code path.
         path: ""

 # -- Defines how apisix handles routing:
 # - radixtree_uri: match route by uri(base on radixtree)
 # - radixtree_host_uri: match route by host + uri(base on radixtree)
 # - radixtree_uri_with_parameter: match route by uri with parameters
 httpRouter: radixtree_host_uri
  

     # Integration service discovery registry. E.g eureka\dns\nacos\consul_kv
     # reference:
     # https://apisix.apache.org/docs/apisix/discovery/#configuration-for-eureka
     # https://apisix.apache.org/docs/apisix/discovery/dns/#service-discovery-via-dns
     # https://apisix.apache.org/docs/apisix/discovery/consul_kv/#configuration-for-consul-kv
     # https://apisix.apache.org/docs/apisix/discovery/nacos/#configuration-for-nacos
     # https://apisix.apache.org/docs/apisix/discovery/kubernetes/#configuration
     #
     # an eureka example:
     # ```
     # eureka:
     #   host:
     #     - "http://${username}:${password}@${eureka_host1}:${eureka_port1}"
     #     - "http://${username}:${password}@${eureka_host2}:${eureka_port2}"
     #   prefix: "/eureka/"
     #   fetch_interval: 30
     #   weight: 100
     #   timeout:
     #     connect: 2000
     #     send: 2000
     #     read: 5000
     # ```
     #
     # the minimal Kubernetes example:
     # ```
     # kubernetes: {}
     # ```
     #
     # The prerequisites for the above minimal Kubernetes example:
     #  1. [Optional] Set `.serviceAccount.create` to `true` to create a dedicated ServiceAccount.
     #     It is recommended to do so, otherwise the default ServiceAccount "default" will be used.
     #  2. [Required] Set `.rbac.create` to `true` to create and bind the necessary RBAC resources.
     #     This grants the ServiceAccount in use to List-Watch Kubernetes Endpoints resources.
     #  3. [Required] Include the following environment variables in `.nginx.envs` to pass them into
     #     nginx worker processes (https://nginx.org/en/docs/ngx_core_module.html#env):
     #      - KUBERNETES_SERVICE_HOST
     #      - KUBERNETES_SERVICE_PORT
     #     This is for allowing the default `host` and `port` of `.discovery.registry.kubernetes.service`.
 # -- Enable Proxy Protocol
 proxyProtocol:
   enabled: false
   # -- The port with proxy protocol for http, it differs from node_listen and admin_listen.
   listenHttpPort: 9181
   # -- The port with proxy protocol for https
   listenHttpsPort: 9182
   # -- Enable the proxy protocol for tcp proxy, it works for stream_proxy.tcp option
   tcp: true
   # -- Enable the proxy protocol to the upstream server
   upstream: true

 # -- Enable Data Encryption
 data_encryption:
   enabled: false
   # -- An array of 16 character strings used to encrypt/decrypt fields with AES-128-CBC
   keyring: []

 # -- Enable full customized config.yaml
 enableCustomizedConfig: false
 # -- If apisix.enableCustomizedConfig is true, full customized config.yaml.
 # Please note that other settings about APISIX config will be ignored
 customizedConfig: {}

 image:
   # -- Apache APISIX image repository
   repository: apache/apisix
   # -- Apache APISIX image pull policy
   pullPolicy: IfNotPresent
   # -- Apache APISIX image tag
   # Overrides the image tag whose default is the chart appVersion.
   tag: 3.8.0-debian

 # -- Use a `DaemonSet` or `Deployment`
 kind: Deployment
 # -- kind is DaemonSet, replicaCount not become effective
 replicaCount: 1

 # -- Set [priorityClassName](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#pod-priority) for Apache APISIX pods
 priorityClassName: ""
 # -- Annotations to add to each pod
 # -- Set the securityContext for Apache APISIX pods
 podSecurityContext: {}
   # fsGroup: 2000
 # -- Set the securityContext for Apache APISIX container
 securityContext: {}
   # capabilities:
   #   drop:
   #   - ALL
   # readOnlyRootFilesystem: true
   # runAsNonRoot: true
   # runAsUser: 1000

 # -- See https://kubernetes.io/docs/tasks/run-application/configure-pdb/ for more details
 podDisruptionBudget:
   # -- Enable or disable podDisruptionBudget
   enabled: false
   # -- Set the `minAvailable` of podDisruptionBudget. You can specify only one of `maxUnavailable` and `minAvailable` in a single PodDisruptionBudget.
   # See [Specifying a Disruption Budget for your Application](https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget)
   # for more details
   minAvailable: 90%
   # -- Set the maxUnavailable of podDisruptionBudget
   maxUnavailable: 1

 # -- Set pod resource requests & limits
 resources: {}
   # -- Use the host's network namespace

   # We usually recommend not to specify default resources and to leave this as a conscious
   # choice for the user. This also increases chances charts run on environments with little
   # resources, such as Minikube. If you do want to specify resources, uncomment the following
   # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
   # limits:
   #   cpu: 100m
   #   memory: 128Mi
   # requests:
   #   cpu: 100m
   #   memory: 128Mi
 hostNetwork: false

 # -- Node labels for Apache APISIX pod assignment
 nodeSelector: {}
 # -- List of node taints to tolerate
 tolerations: []
 # -- Set affinity for Apache APISIX deploy
 affinity: {}
 # -- Topology Spread Constraints for pod assignment spread across your cluster among failure-domains
 # ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/#spread-constraints-for-pods
 topologySpreadConstraints: []

 # -- timezone is the timezone where apisix uses.
 # For example: "UTC" or "Asia/Shanghai"
 # This value will be set on apisix container's environment variable TZ.
 # You may need to set the timezone to be consistent with your local time zone,
 # otherwise the apisix's logs may used to retrieve event maybe in wrong timezone.
 timezone: ""

 # -- extraEnvVars An array to add extra env vars
 # e.g:
 # extraEnvVars:
 #   - name: FOO
 #     value: "bar"
 #   - name: FOO2
 #     valueFrom:
 #       secretKeyRef:
 #         name: SECRET_NAME
 #         key: KEY
 extraEnvVars: []

 # -- Set the readinessProbe for Apache APISIX pods
 readinessProbe:
   failureThreshold: 6
   initialDelaySeconds: 10
   periodSeconds: 10
   successThreshold: 1
   tcpSocket:
     port: &gatewayHttpContainerPort 9080
   timeoutSeconds: 1

nameOverride: ""
fullnameOverride: ""

 #serviceAccount:
 #  create: true
 #  annotations: {}
 #  name: "apisix-gateway"

rbac:
 create: false

deployment:
 # -- Apache APISIX deployment mode
 # Optional: traditional, decoupled, standalone
 #
 # ref: https://apisix.apache.org/docs/apisix/deployment-modes/
 mode: traditional

 # -- Deployment role
 # Optional: traditional, data_plane, control_plane
 #
 # ref: https://apisix.apache.org/docs/apisix/deployment-modes/
 role: "traditional"

gateway:
 enabled: false
 # -- Apache APISIX service type for user access itself
 service:
   type: LoadBalancer
 # -- Setting how the Service route external traffic
 # If you want to keep the client source IP, you can set this to Local.

 # ref: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip
 externalTrafficPolicy: Cluster
 # type: LoadBalancer
 annotations: {}
 #   service.beta.kubernetes.io/aws-load-balancer-type: nlb
 externalIPs: []
 # -- Apache APISIX service settings for http
 http:
   enabled: true
   servicePort: 80
   containerPort: *gatewayHttpContainerPort
   # -- Support multiple http ports, See [Configuration](https://github.com/apache/apisix/blob/0bc65ea9acd726f79f80ae0abd8f50b7eb172e3d/conf/config-default.yaml#L24)
   additionalContainerPorts: []
     # - port: 9081
     #   enable_http2: true          # If not set, the default value is `false`.
     # - ip: 127.0.0.2               # Specific IP, If not set, the default value is `0.0.0.0`.
     #   port: 9082
     #   enable_http2: true
 # -- Apache APISIX service settings for tls
 tls:
   enabled: false
   servicePort: 443
   containerPort: 9443
   # -- Support multiple https ports, See [Configuration](https://github.com/apache/apisix/blob/0bc65ea9acd726f79f80ae0abd8f50b7eb172e3d/conf/config-default.yaml#L99)
   additionalContainerPorts: []
     # - ip: 127.0.0.3           # Specific IP, If not set, the default value is `0.0.0.0`.
     #   port: 9445
     #   enable_http2: true
   # -- Specifies the name of Secret contains trusted CA certificates in the PEM format used to verify the certificate when APISIX needs to do SSL/TLS handshaking with external services (e.g. etcd)
   existingCASecret: ""
   # -- Filename be used in the gateway.tls.existingCASecret
   certCAFilename: ""
   http2:
     enabled: true
   # -- TLS protocols allowed to use.
   sslProtocols: "TLSv1.2 TLSv1.3"
   # -- Define SNI to fallback if none is presented by client
   fallbackSNI: ""
 # -- Proxy Protocol Configuration
 proxyProtocol:
   # -- If you enable proxy protocol, you must use this port to receive http request with proxy protocol
   http:
     enabled: false
     # - Specify NodePort (only if gateway.type is NodePort)
     # nodePort:
     # - Define a Service Port on which the gateway is listening
     servicePort: 9181
     # - Gateway Service Port to use as target
     containerPort: 9181
   # -- The port with proxy protocol for https
   https:
     enabled: false
     # - Specify NodePort (only if gateway.type is NodePort)
     # nodePort:
     # - Define a Service Port on which the gateway is listening
     servicePort: 9182
     # - Gateway Service Port to use as target
     containerPort: 9182
 # -- Apache APISIX service settings for stream. L4 proxy (TCP/UDP)
 stream:
   enabled: false
   tcp: []
   udp: []
 # -- Using ingress access Apache APISIX service
 ingress:
   enabled: true
   # -- Ingress annotations
   annotations: {}
     # kubernetes.io/ingress.class: nginx
     # kubernetes.io/tls-acme: "true"
   hosts:
     - host: apisix.local
       paths: []
   tls: []
 #  - secretName: apisix-tls
 #    hosts:
 #      - chart-example.local
 # -- Override default labels assigned to Apache APISIX gateway resources
 # labelsOverride:
 #   app.kubernetes.io/name: "{{ .Release.Name }}"
 #   app.kubernetes.io/instance: '{{ include "apisix.name" . }}'
admin:
 # -- Enable Admin API
 enabled: true
 # -- admin service type
 type: ClusterIP
 # loadBalancerIP: a.b.c.d
 # loadBalancerSourceRanges:
 #   - "143.231.0.0/16"
 # -- IPs for which nodes in the cluster will also accept traffic for the servic
 externalIPs: []
 # -- which ip to listen on for Apache APISIX admin API. Set to `"[::]"` when on IPv6 single stack
 ip: 0.0.0.0
 # -- which port to use for Apache APISIX admin API
 port: 9180
 # -- Service port to use for Apache APISIX admin API
 servicePort: 9180
 # -- Admin API support CORS response headers
 cors: true
 # -- Admin API credentials
 credentials:
   # -- Apache APISIX admin API admin role credentials
   admin: edd1c9f034335f136f87ad84b625c8f1
   # -- Apache APISIX admin API viewer role credentials
   viewer: 4054f7cf07e344346cd3f287985e76a2

   # -- The APISIX Helm chart supports storing user credentials in a secret.
   # The secret needs to contain two keys, admin and viewer, with their respective values set.
   secretName: ""
   # -- Name of the admin role key in the secret, overrides the default key name "admin"
   secretAdminKey: ""
   # -- Name of the viewer role key in the secret, overrides the default key name "viewer"
   secretViewerKey: ""

 allow:
   # -- The client IP CIDR allowed to access Apache APISIX Admin API service.
   ipList:
     - 0.0.0.0/0
 # -- Using ingress access Apache APISIX admin service
 ingress:
   enabled: false
   # -- Ingress annotations
   annotations: {}
     # kubernetes.io/ingress.class: nginx
     # kubernetes.io/tls-acme: "true"
   hosts:
     - host: apisix-admin.local
       paths:
         - "/apisix"
   tls: []
 #  - secretName: apisix-tls
 #    hosts:
 #      - chart-example.local

nginx:
 workerRlimitNofile: "20480"
 workerConnections: "10620"
 workerProcesses: auto
 enableCPUAffinity: true
 user: root
 envs: []
 # -- allow customize meta in `nginx_config` section
 meta: |
   lua_shared_dict:
     prometheus-metrics: 15m

# -- Customize the list of APISIX plugins to enable. By default, APISIX's default plugins are automatically used. See [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml)
plugins: []
# -- Customize the list of APISIX stream_plugins to enable. By default, APISIX's default stream_plugins are automatically used. See [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml)
stream_plugins: []

# -- Set APISIX plugin attributes, see [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml#L376) for more details
pluginAttrs: {}

extPlugin:
 # -- Enable External Plugins. See [external plugin](https://apisix.apache.org/docs/apisix/next/external-plugin/)
 enabled: false
 # -- the command and its arguements to run as a subprocess
 cmd: ["/path/to/apisix-plugin-runner/runner", "run"]

wasmPlugins:
 # -- Enable Wasm Plugins. See [wasm plugin](https://apisix.apache.org/docs/apisix/next/wasm/)
 enabled: false
 plugins: []

# -- customPlugins allows you to mount your own HTTP plugins.
customPlugins:
 # -- Whether to configure some custom plugins
 enabled: true
 # -- the lua_path that tells APISIX where it can find plugins,
 # note the last ';' is required.
 luaPath: "/opts/custom_plugins/?.lua"
 plugins:
   # -- plugin name.
   - name: "plugin-name"
     # -- plugin attrs
     attrs: {}
     # -- plugin codes can be saved inside configmap object.
     configMap:
       # -- name of configmap.
       name: "configmap-name"
       # -- since keys in configmap is flat, mountPath allows to define the mount
       # path, so that plugin codes can be mounted hierarchically.
       mounts:
         - key: "the-file-name"
           path: "mount-path"

# -- Update strategy for apisix deployment
updateStrategy: {}
 # type: RollingUpdate

# -- Additional `volume`, See [Kubernetes Volumes](https://kubernetes.io/docs/concepts/storage/volumes/) for the detail.
extraVolumes: []
# - name: extras
#   emptyDir: {}

# -- Additional `volume`, See [Kubernetes Volumes](https://kubernetes.io/docs/concepts/storage/volumes/) for the detail.
extraVolumeMounts: []
# - name: extras
#   mountPath: /usr/share/extras
#   readOnly: true

# -- Additional `initContainers`, See [Kubernetes initContainers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) for the detail.
extraInitContainers:
 - name: init-sysctl
   image: busybox:1.28
   command:
     - /bin/sh
     - -c
     - |
       sysctl -w net.core.somaxconn=65535
       sysctl -w net.ipv4.ip_local_port_range="1024 65535"
       sysctl -w net.ipv4.tcp_max_syn_backlog=8192
       sysctl -w fs.file-max=1048576
       sysctl -w fs.inotify.max_user_instances=16384
       sysctl -w fs.inotify.max_user_watches=524288
       sysctl -w fs.inotify.max_queued_events=16384
   securityContext:
     privileged: true
# - name: init-myservice
#   image: busybox:1.28
#   command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"]

# access log and error log configuration
logs:
 # -- Enable access log or not, default true
 enableAccessLog: true
 # -- Access log path
 accessLog: "/dev/stdout"
 # -- Access log format
 accessLogFormat: '$remote_addr - $remote_user [$time_local] $http_host \"$request\" $status $body_bytes_sent $request_time \"$http_referer\" \"$http_user_agent\" $upstream_addr $upstream_status $upstream_response_time \"$upstream_scheme://$upstream_host$upstream_uri\"'
 # -- Allows setting json or default characters escaping in variables
 accessLogFormatEscape: default
 # -- Error log path
 errorLog: "/dev/stderr"
 # -- Error log level
 errorLogLevel: "warn"

dns:
 resolvers:
   - 127.0.0.1
   - 172.20.0.10
   - 114.114.114.114
   - 223.5.5.5
   - 1.1.1.1
   - 8.8.8.8
 validity: 30
 timeout: 5

initContainer:
 # -- Init container image
 image: busybox
 # -- Init container tag
 tag: 1.28
 # -- Set the securityContext for the init container
 securityContext: {}
   # capabilities:
   #   drop:
   #   - ALL
   # readOnlyRootFilesystem: true
   # runAsNonRoot: true
   # runAsUser: 1000

autoscaling:
 enabled: false
 # -- HPA version, the value is "v2" or "v2beta1", default "v2"
 version: v2
 minReplicas: 1
 maxReplicas: 100
 targetCPUUtilizationPercentage: 80
 targetMemoryUtilizationPercentage: 80

# -- Custom configuration snippet.
configurationSnippet:
 main: |

 httpStart: |

 httpEnd: |

 httpSrv: |

 httpAdmin: |

 stream: |

# -- Observability configuration.
# ref: https://apisix.apache.org/docs/apisix/plugins/prometheus/
serviceMonitor:
 # -- Enable or disable Apache APISIX serviceMonitor
 enabled: false
 # -- namespace where the serviceMonitor is deployed, by default, it is the same as the namespace of the apisix
 namespace: ""
 # -- name of the serviceMonitor, by default, it is the same as the apisix fullname
 name: ""
 # -- interval at which metrics should be scraped
 interval: 15s
 # -- path of the metrics endpoint
 path: /apisix/prometheus/metrics
 # -- prefix of the metrics
 metricPrefix: apisix_
 # -- container port where the metrics are exposed
 containerPort: 9091
 # -- @param serviceMonitor.labels ServiceMonitor extra labels
 labels: {}
 # -- @param serviceMonitor.annotations ServiceMonitor annotations
 annotations: {}
 # -- @param serviceMonitor.metricRelabelings MetricRelabelConfigs to apply to samples before ingestion.
 # ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs
 metricRelabelings: []

# -- etcd configuration
# use the FQDN address or the IP of the etcd
etcd:
 # -- install etcd(v3) by default, set false if do not want to install etcd(v3) together
 enabled: true
 # -- if etcd.enabled is false, use external etcd, support multiple address, if your etcd cluster enables TLS, please use https scheme, e.g. https://127.0.0.1:2379.
 host:
   # host or ip e.g. http://172.20.128.89:2379
   - http://etcd.host:2379
 # -- if etcd.enabled is false, username for external etcd. If etcd.enabled is true, use etcd.auth.rbac.rootPassword instead.
 user: ""
 # -- if etcd.enabled is false, password for external etcd. If etcd.enabled is true, use etcd.auth.rbac.rootPassword instead.
 password: ""
 # -- apisix configurations prefix
 prefix: "/apisix"
 # -- Set the timeout value in seconds for subsequent socket operations from apisix to etcd cluster
 timeout: 30
 # -- Name of the existing secret containing user and password for external etcd, overrides etcd.user and etcd.password
 existingSecret: ""
 # -- Name of key containing username to be retrieved from the existing secret, has a value of user by default
 existingSecretUserKey: ""
 # -- Name of key containing password to be retrieved from the existing secret, has a value of password by default
 existingSecretPasswordKey: ""

 # -- if etcd.enabled is true, set more values of bitnami/etcd helm chart
 auth:
   rbac:
     # -- No authentication by default. Switch to enable RBAC authentication
     create: false
     # -- root password for etcd. Requires etcd.auth.rbac.create to be true.
     rootPassword: ""
     # -- Name of the existing secret containing credentials for the root user
     existingSecret: ""
     # -- Name of key containing password to be retrieved from the existing secret
     existingSecretPasswordKey: ""
   tls:
     # -- enable etcd client certificate
     enabled: false
     # -- name of the secret contains etcd client cert
     existingSecret: ""
     # -- etcd client cert filename using in etcd.auth.tls.existingSecret
     certFilename: ""
     # -- etcd client cert key filename using in etcd.auth.tls.existingSecret
     certKeyFilename: ""
     # -- whether to verify the etcd endpoint certificate when setup a TLS connection to etcd
     verify: true
     # -- specify the TLS Server Name Indication extension, the ETCD endpoint hostname will be used when this setting is unset.
     sni: ""

 # -- ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container
 # -- added for backward compatibility with old kubernetes versions, as seccompProfile is not supported in kubernetes < 1.19
 containerSecurityContext:
   enabled: false

 service:
   port: 2379

 replicaCount: 3

dashboard:
 enabled: false
 service:
   type: LoadBalancer
 config:
   conf:
     etcd:
       # -- Supports defining multiple etcd host addresses for an etcd cluster
       endpoints:
         - apisix-etcd:2379
       # -- apisix configurations prefix
       prefix: "/apisix"
       # -- Specifies etcd basic auth username if enable etcd auth
       username: ~
       # -- Specifies etcd basic auth password if enable etcd auth
       password: ~

# -- Ingress controller configuration
ingress-controller:
 nameOverride: ingress-controller
 fullnameOverride: ingress-controller
 serviceAccount:
 # -- Specifies whether a ServiceAccount should be created
   create: false
 podAnnotations:
   consul.hashicorp.com/connect-inject: 'true'
   consul.hashicorp.com/transparent-proxy: 'true'
   consul.hashicorp.com/connect-service: 'ingress-controller,ingress-controller-apisix-gateway'
   consul.hashicorp.com/connect-service-port: '80,80'
   consul.hashicorp.com/exclude-inbound-ports: '80'
 enabled: true
 config:
   apisix:
     adminAPIVersion: "v3"
     serviceFullname: "apisix-admin.cc-dev.svc"
     discovery: {}

vault:
 # -- Enable or disable the vault integration
 enabled: false
 # -- The host address where the vault server is running.
 host: ""
 # -- HTTP timeout for each request.
 timeout: 10
 # -- The generated token from vault instance that can grant access to read data from the vault.
 token: ""
 # -- Prefix allows you to better enforcement of policies.
 prefix: ""

I configured dialedDirectly with fllowing yaml:

apiVersion: consul.hashicorp.com/v1alpha1
kind: Mesh
metadata:
  name: mesh
spec:
  transparentProxy:
    dialedDirectly: true

Any updates according to this issue?