Handling traffic with Ingress GW in a Consul Connect mesh with Nomad scheduling

Consul
1

Hello !

I’ve successfully setup a non Cloud cluster with Consul (connect enabled), Nomad and Vault. It’s done with 3 Nomad/Consul servers and 3 Nomad/Consul clients nodes, all on bare metal.

If I have well understood, the Ingress Gateway will handle the traffic from outside the cluster, to the service mesh of my nomad job services.

Do you have a reference architecture about managing the ingress gateway on Nomad hosts ?

  • Does I need an only one named Ingress Gateway instance which listen on port 80 for each client node in my cluster ? In my case 3 ingress gateway. But in this case, that’s means all the Ingress gateways are configured in one Consul Config Entry for all my Consul services. If I’ve hundred of services It’s complicated to manage this file

  • An other possibility would be to have an ingress gateway dedicated for each services that I want to be available outiside my service mesh, and so separated Consul Config Entries for each IG. As the IG do port mapping to the host, I cannot have 2 Ingress gateways listening on the same port (like the 80) of the Consul/Nomad node. Maybe putting a second stage reverse proxy (Traefik for example) in front of all the Ingress gateway with non static defined ports ? But I think I loose the Ingress Gateway avantages, because that add a second stage, and if I need TLS, the second proxy will have the role of terminating TLS connection

So I really enjoy the hashistack but I don’t really know how to deal with inbound client traffic into my Nomad/Consul service mesh cluster. It would be so great if the documentation would cover this use case

Thanks in advance

1 Like
2

Hi @timotheenicolas

I posted something similar yesterday in the nomad channel:

Hope it helps

1 Like
3

@timotheenicolas After reading a lot of documents and do some research with K8s, finally I dicided to follow bellow model.

Traefik -> Ingress Gateway -> Service Mesh -> Terminating Gateway -> DB...

I have to use Traefik as edge proxy because of powerful routing criteria, automatic Let’s Encrypt SSL…

Ingress Gateway will listen to localhost, Traefik will forwad traffic to Ingress Gateway. It will be a bit slower because of adding extra proxy but have much more flexibility.

4

Thanks a lot for you answer :slight_smile:

I think I will go with the same architecture that you did in fact.

And to be clear, you configure Traefik dynamically with service tags in the Ingress Gateway job ?

5

That is correct @timotheenicolas

In my traefik config, I use this to scrape consul and find all the services that I tagged:

# Enable Consul Catalog configuration backend.
[providers.consulCatalog]
  refreshInterval = "30s"
  prefix = "traefik"
  constraints = "Tag(`traefik.tags=service`)"
  [providers.consulCatalog.endpoint]
address = "http://127.0.0.1:8500"

Traefik will then find the service, get an TLS cert via Lets Encrypt and setup the rest.

    tags = [
      "traefik.tags=service",
      "traefik.http.routers.example.rule=Host(`example.com`)",
      "traefik.http.routers.example.tls=true",
      "traefik.http.routers.example.tls.certresolver=resolver",
      "traefik.http.routers.example.tls.domains[0].main=example.com",
      "traefik.http.routers.example.tls.domains[0].sans=www.example.com",
      "traefik.http.routers.example.entrypoints=websecure",
    ]