Getting traffic into cluster using traefik

Consul
1

I was hoping i might be able to get some guidance.

My current setup for testing is 3 consul/nomad servers and 3 client servers . I currently have a healthy cluster, with no errors. I’ve been able to run a few job specs without errors. But when I start to increase the amount of services in the job spec, then things aren’t working as expected.

I’ve been following tutorials and reading docs and would like to get to a point where my cluster looks like this: Application load balancing photo

But I’m struggling to set up the ingress from traefik (nginx in the photo) to the containers.

I currently have traefik (but I’m open to switch to fabio, nginx or haproxy) set up as a system type running on my 3 client nodes using this spec from traefik’s website.

job "traefik" {
  datacenters = ["dc1"]
  type        = "system"

  group "traefik" {
    count = 1

    network {
      port  "http"{
         static = 80
      }
      port  "admin"{
         static = 8080
      }
    }

    service {
      name = "traefik-http"
      provider = "consul"
      port = "http"
    }

    task "server" {
      driver = "docker"
      config {
        image = "traefik"
        ports = ["admin", "http"]
        args = [
          "--api.dashboard=true",
          "--api.insecure=true", ### For Test only, please do not use that in production
          "--entrypoints.web.address=:${NOMAD_PORT_http}",
          "--entrypoints.traefik.address=:${NOMAD_PORT_admin}",
          "--providers.consulcatalog=true",
	  "--providers.consulcatalog.endpoint.address=http://192.168.50.188:8500",
        ]
      }
    }
  }
}

I’ve also deployed this job spec, hoping that a routing rule gets set up for count.example.com

job "countdash" {
  datacenters = ["dc1"]

  group "api" {
    count = 2
    network {
      mode = "bridge"
    }

    service {
      name = "count-api"
      port = "9001"

      connect {
        sidecar_service {}
      }
    }

    task "web" {
      driver = "docker"

      config {
        image = "hashicorpdev/counter-api:v3"
      }
    }
  }

  group "dashboard" {
    network {
      mode = "bridge"

      port "http" {
        static = 9002
        to     = 9002
      }
    }

    service {
      name = "count-dashboard"
      port = "http"

      connect {
        sidecar_service {
          proxy {
            upstreams {
              destination_name = "count-api"
              local_bind_port  = 8080
            }
          }
        }
      }
      tags = [
        "traefik.enable=true",
        "traefik.http.routers.count.rule=Host(`count.example.com`)"
        ]
    }

    task "dashboard" {
      driver = "docker"

      env {
        COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}"
      }

      config {
        image = "hashicorpdev/counter-dashboard:v3"
      }
    }
  }
}

But what I’ve noticed, is that when I run this job spec, the service count-api gets registered in traefik, and the tags I have set up for the dashboard service don’t get registered in traefik. I get an error in the traefik: Router defined multiple times with different configurations

Without the tag, both services get registered in traefik. I’m able to browse to the host:port and access the app without issue, so I know the routing is working. With the example set up without the tag, traefik does load balance between the two api containers that get created.

So to get this to work, am I supposed to do a job spec with the count-dashboard and the count-api in separate job specs, or do I just use something like nginx/fabio/haproxy, or am I just doing this completely wrong?