API Gateway unable to add or remove HTTPRoutes

HashiCorp Cloud Platform (HCP) HCP Consul
,
1

I am currently trying to set up the API gateway, and I’m finding it very complicated and difficult. I suspect there is something simple wrong with my configuration.

To start with, I managed to get one HTTP route working, however I am unable to add further routes, and if I delete the original route, it still continues to work.

The helm chart has the following configuration for the gateway:

apiGateway:
  enabled: true
  image: "hashicorp/consul-api-gateway:0.3.0"
  logLevel: debug
  managedGatewayClass:
    enabled: true
    serviceType: LoadBalancer
    useHostPorts: true
    copyAnnotations:
      service:
        annotations: |
          - external-dns.alpha.kubernetes.io/hostname
          - external-dns.alpha.kubernetes.io/ttl

global tls is configured:

  tls:
    enabled: true
    enableAutoEncrypt: true
    verify: true
    serverAdditionalDNSSANs:
      ## Add the K8s domain name to the consul server certificate
      - consul-server.consul-system.svc.cluster.local

Interestingly when I look in the logs for the gateway controller I see a lot of the following error:

[WARN]  grpc/logging.go:55: consul-api-gateway-server.sds-server: [core]grpc: Server.Serve failed to complete security handshake from "X.X.X.X:54640": tls: failed to verify client certificate: x509: certificate has expired or is not yet valid: current time 2022-08-02T16:11:51Z is after 2022-07-17T17:24:23Z

also

[ERROR] watch/plan.go:95: consul-api-gateway-server.cert-manager.watch: Watch errored: type=connect_roots error="Unexpected response code: 500 (rpc error making call: i/o deadline reached)" retry=5s

When I do a describe on the api-gateway I get the following:

Spec:
  Gateway Class Name:  consul-api-gateway
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  All
    Name:      https
    Port:      8443
    Protocol:  HTTPS
    Tls:
      Certificate Refs:
        Group:
        Kind:   Secret
        Name:   consul-server-cert
      Mode:     Terminate
Status:
  Addresses:
    Type:   IPAddress
    Value:  X.X.X.X
  Conditions:
    Last Transition Time:  2022-08-02T15:17:57Z
    Message:               Ready
    Observed Generation:   1
    Reason:                Ready
    Status:                True
    Type:                  Ready
    Last Transition Time:  2022-08-02T15:17:57Z
    Message:               Scheduled
    Observed Generation:   1
    Reason:                Scheduled
    Status:                True
    Type:                  Scheduled
    Last Transition Time:  2022-08-02T15:17:57Z
    Message:               InSync
    Observed Generation:   1
    Reason:                InSync
    Status:                True
    Type:                  InSync
  Listeners:
    Attached Routes:  1
    Conditions:
      Last Transition Time:  2022-08-02T15:17:57Z
      Message:               NoConflicts
      Observed Generation:   1
      Reason:                NoConflicts
      Status:                False
      Type:                  Conflicted
      Last Transition Time:  2022-08-02T15:17:57Z
      Message:               Attached
      Observed Generation:   1
      Reason:                Attached
      Status:                False
      Type:                  Detached
      Last Transition Time:  2022-08-02T15:17:57Z
      Message:               Ready
      Observed Generation:   1
      Reason:                Ready
      Status:                True
      Type:                  Ready
      Last Transition Time:  2022-08-02T15:17:57Z
      Message:               ResolvedRefs
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
    Name:                    https
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute

I’m not sure whether the fact that the server is unable to update routes is related to the server cert error, or whether more information is needed.