Nomad with Consul Connect - GET 502 BAD GATEWAY

Nomad
1

TLDR:

  • NGINX Frontend -proxy_pass-> Sidecar Proxy → Sidecar Proxy → NGINX Backend
  • Complete job file + container logs
  • Backend serves HTML on Port 8888
  • Frontend listens on port 8080 and serves proxy_pass to connect proxy on port 7777
  • I cannot see any processes on port 7777 and get a BAD GATEWAY 502 on port 8080
  • Frontend HealthCheck fails accordingly but the frontend proxy says it is connected:

consul_service2
consul_service21133×453 31.7 KB

consul_service
consul_service1124×794 87.8 KB

I did run the counter-api tutorial on exactly the same node (both services running on a single node) - so I do not think that there is a setup issue (firewall, etc.).

There seems to be an issue with the job file - but I have been through this file many times now and made a few adjustments based on discussions here and on GH. But no chance.

Does someone spot the issue or has a similar job file that does work that I can try? I am currently clueless on how to debug this any further :confused:

2

Just to verify - building this service without Connect works as expected =>

Just removed the connect stanza and am now hosting the backend on port 7777:

network {
      mode = "host"
      port "http" {
          static = "7777"
      }
    }

    service {
        name = "backend-http"
        port = "http"
        provider = "nomad"

        check {
            name     = "HTTP Health"
            port = "http"
            path     = "/"
            type     = "http"
            protocol = "http"
            interval = "30s"
            timeout  = "2s"
        }
    }

And pointed the frontend directly to the backend service:

server {
    listen 8080;
    listen [::]:8080;
    server_name _;
    location / {
      proxy_pass http://127.0.0.1:7777;
    }
}

Now the service is accessible both on port 7777 and 8080:

curl localhost:8080 // or :7777
<!DOCTYPE html>
<html>
<head>
    <title>Hello Nomad</title>
</head>
<body>
    <h1>No Consul Connect...</h1>
    <p>¯\_(ツ)_/¯</p>
</body>
</html>