Cannot get Traefik load-balancer to work properly with Nomad

[johnson]

Hello,

I am trying to configure Traefik on our nomad servers and was able to deploy on AWS. I was able to access the traefik dashboard via locally using curl but if I go through my AWS provided load-balancer which gets forwarded to Traefik it does not work externally.

Please see below for my traefik.nomad config. I have also followed the load-balancing documentation: Load Balancing with Traefik | Nomad - HashiCorp Learn

I was able to access internally but not externally. The health check for port 8080 seems to fail as well with 404 error. This does not seem to be an issue with AWS firewall or security group settings.

job "traefik" {
  datacenters = ["us-east-1a"]
  type        = "service"

  group "traefik" {
    network {
      port "http" {
        static = 8080
      }

      port "api" {
        static = 8081
      }
    }

    service {
      name = "traefik"

      check {
        name     = "alive"
        type     = "tcp"
        port     = "http"
        interval = "10s"
        timeout  = "2s"
      }

      tags = [
        "traefik.enable=true",
        "traefik.http.routers.dashboard.rule=Host(`xxxx.xxxx.xxxx`) && (PathPrefix(`/dashboard`) || PathPrefix(`/api`))",
        "traefik.http.routers.dashboard.service=api@internal",
        "traefik.http.routers.dashboard.entrypoints=http",
      ]
    }

    task "traefik" {
      driver = "docker"

      config {
        image        = "traefik:v2.4"
        force_pull   = true
        network_mode = "host"

        volumes = [
          "local/traefik.toml:/etc/traefik/traefik.toml",
        ]
      }

      template {
        data = <<EOF
[entryPoints]
  [entryPoints.http]
    address = ":80"

[api]
    dashboard = true
    insecure  = true

# Enable Consul Catalog configuration backend.
[providers.consulCatalog]
    prefix           = "traefik"
    exposedByDefault = false
    [providers.consulCatalog.endpoint]
      address = "127.0.0.1:8500"
      scheme  = "http"
EOF

        destination = "local/traefik.toml"
      }

      resources {
        cpu    = 100
        memory = 128
      }
    }
  }
}

[lgfa29]

Hi @johnson

@johnson

I was able to access the traefik dashboard via locally using curl but if I go through my AWS provided load-balancer which gets forwarded to Traefik it does not work externally.

How did you access the Traefik API in your test? Would you mind sharing the curl command that you used?

@johnson

The health check for port 8080 seems to fail as well with 404 error.

This seems be because you have your http port set to 8080 while your Traefik config is listening on port 80.

Would you mind changing them so they match and trying again?

@johnson

This does not seem to be an issue with AWS firewall or security group settings.

If I recall correctly, AWS load balancers need a health check in order to send traffic to a machine. Could you check if the port being used is actually working and that your load balancer shows your Nomad clients as healthy in the AWS dashboard?

[johnson]

Thank you, I seem to found the issue! It was definitely the port that was setup incorrectly.