Exposing upstream services

Hi there, I’m new to this forum. I’ve been having some struggles getting Nomad and Consul to work with upstreams. I’ve done a search on this forum and from what I can tell what I’ve done should be working. I’ve set up a simple project for testing, one service on a static port 3000 which sets service_b as an upstream on local port 8080.

For some reason I just cannot get this to work? All the health checks on Consul pass, I can curl the services using the ports given by Consul etc. I just cannot do curl localhost:3000/upstream where the /upstream endpoint is calling some endpoint on service B.

Service A nomad file:

job "job-a" {
  datacenters = ["dc"]
  type        = "service"

  group "service-a" {
    network {
      mode = "bridge"

      port "service-a" {
        to = 3000
        static = 3000
      }
    }

    service {
      name = "service-a"
      port = "service-a"

      connect {
        sidecar_service {
          proxy {
            upstreams {
              destination_name = "service-b"
              local_bind_port  = 8080
            }
          }
        }
      }

      check {
        type     = "http"
        port     = "service-a"
        interval = "20s"
        timeout  = "20s"
        path     = "/status"
      }
    }

    task "service-a" {
      driver = "docker"

      config {
        image = "service_a:local"
        ports = ["service-a"]
      }
    }
  }
}

Service B nomad file:

job "job-b" {
  datacenters = ["dc"]
  type        = "service"

  group "service-b" {
    network {
      mode = "bridge"

      port "service-b" {
        to = 3000
      }
    }

    service {
      name = "service-b"
      port = "service-b"

      connect {
        sidecar_service {}
      }

      check {
        type     = "http"
        port     = "service-b"
        interval = "20s"
        timeout  = "20s"
        path     = "/status"
      }
    }

    task "service-b" {
      driver = "docker"

      config {
        image = "service_b:local"
        ports = ["service-b"]
      }
    }
  }
}

I’m sure there’s something very simple and obvious that I’ve missed here but so far I am still struggling and hoping someone can help. Thank you!

Having read this: I don't understand networking between services - #2 by Clivern

I think the gist of it is basically the only way to make it work is to bind to an interface when the jobs are distinct. Ie. tasks in the same group only work on the loopback interface?

I’m on linux, and for some reason the only way I could get the sidecar setup to work was to pass my network interface to the nomad agent startup. But I shouldn’t have to do this right as it is apparently only required on Windows/Mac (Frequently Asked Questions | Nomad | HashiCorp Developer)?

ie. sudo nomad agent -dev -bind=0.0.0.0 -network-interface=enp0s31f6 -config=dev.hcl

Can anyone help explain why this is the case?

Hi @rwmorton,

When you run a dev agent for Consul Connect, the -dev-connect argument will take care of this.

You should run sudo nomad agent -dev-connect instead of just -dev

-dev-connect: Start the agent in development mode, but bind to a public network interface rather than localhost for using Consul Connect. It may be used with -dev-consul to configure default workload identities for Consul. This mode is supported only on Linux as root.
Commands: agent | Nomad | HashiCorp Developer

The Nomad Connect integration document also has this. ref: Consul Service Mesh | Nomad | HashiCorp Developer

I hope this helps.

Thank you I did try this I believe but let me try tear it all down and start again. Will report back!

Thank you it did the trick :muscle: