Nomad pattern for load balancers

Nomad
1

Hi there

I’m looking for some suggestions for running my load balancer task in nomad?

I’m trying to think what’s best, so I wondered how other people are running theirs in prod?

I’ve seen some people suggest running their load balancer as a task on their server nodes

Some suggest using host mode but an equal number saying that’s bad practice.

I’m thinking of using a static port for the lb and a dynamic port for the web pool. I’ll just have to manage assigning static ports to each separate lb instance.

I appreciate there are loads of ways it can be done, just looking for best practice or patterns to avoid

Thanks

Gary

2

Hi @username-is-already (Gary),

The first thing that stands out is that I would suggest against running your load balancer on the servers. I believe the most common practice is to run the load balancer(s) on a subset of clients; or potentially dedicated clients.

A common pattern is to run the load balancer as a system job with a static port; therefore one LB is placed on each appropriate client giving HA. Dynamic ports for the web pool sounds good.

The HashiCorp Learn site has a Nomad Load Balancing guide which may be useful as reference material.

Thanks,
jrasell and the Nomad team

3

Shameless plug from a previous life: :grinning:
https://www.velotio.com/engineering-blog/simplified-cloud-deployments

TL;DR
The LB could be an ASG (or equivalent) for LB-HA and the services behind the LB pool.

Also plug:

NOTE: This is an old article, and thus the job files haven’t been updated to recent 1.0 GA syntax.

4

We run on a subset of clients.

Keep in mind, if you want to access connect-enabled services (Consul), your lb needs to support this in som way (or you need to configure it yourself).

This example below is using using traefik 2.5-RC (which supports connect):

# https://traefik.io/blog/traefik-labs-and-hashicorp-extends-partnership-with-integration-of-traefik-proxy-and-consul-connect/
# https://doc.traefik.io/traefik/v2.5/reference/static-configuration/env/
# https://atodorov.me/2021/03/27/using-traefik-on-nomad/

locals {
  image = "traefik:v2.5"
  
  cpu = 100
  cpu_restrict = false
  ram = 32
  ram_max = 64
  
  sidecar_cpu = 100
  sidecar_ram = 64
}

job "traefik" {
  datacenters = ["dc1"]

  constraint {
    attribute = "${node.class}"
    value     = "lb"
  }

  group "traefik" {
    
    network {
      mode = "bridge"
      
      port "traefik" {
        to = 8080
        static = 8080
      }

      port "http" {
        to = 80
        static = 80
      }

      port "https" {
        to = 443
        static = 443
      }

      port "tcp" {
        to = 5000
        static = 5000
      }
    }

    service {
      name = "traefik"
      port = "traefik"
      
      connect {
        sidecar_task {
          resources {
            cpu    = local.sidecar_cpu
            memory = local.sidecar_ram
          }
        }
        sidecar_service {}
      }
    }

    task "traefik" {
      driver = "docker"
      
      resources {
        cpu = local.cpu
        memory = local.ram
        memory_max = local.ram_max
      }
      
      config {
        image = local.image
        memory_hard_limit = local.ram_max
        cpu_hard_limit = local.cpu_restrict
        ports = ["traefik","http","https","tcp"]
      }

      env {
        TRAEFIK_ACCESSLOG = "true"
        TRAEFIK_ACCESSLOG_FORMAT = "json"
        TRAEFIK_LOG = "true"
        TRAEFIK_LOG_FORMAT = "json"
        TRAEFIK_METRICS_PROMETHEUS = "true"
        TRAEFIK_METRICS_PROMETHEUS_ADDENTRYPOINTSLABELS = "true"
        TRAEFIK_METRICS_PROMETHEUS_ADDSERVICESLABELS = "true"
        TRAEFIK_PING = "true"
        TRAEFIK_API = "true"
        TRAEFIK_API_INSECURE = "true"
        TRAEFIK_PILOT_DASHBOARD = "false"
        # ENTRYPOINTS
        TRAEFIK_ENTRYPOINTS_HTTP = "true"
        TRAEFIK_ENTRYPOINTS_HTTPS = "true"
        TRAEFIK_ENTRYPOINTS_TCP = "true"
        TRAEFIK_ENTRYPOINTS_HTTP_ADDRESS = ":80"
        TRAEFIK_ENTRYPOINTS_HTTPS_ADDRESS = ":443"
        TRAEFIK_ENTRYPOINTS_TCP_ADDRESS = ":5000"
        # PROVIDER: CONSUL CATALOG (CONNECT-AWARE)
        TRAEFIK_PROVIDERS_CONSULCATALOG = "true"
        TRAEFIK_PROVIDERS_CONSULCATALOG_ENDPOINT_ADDRESS = "172.17.0.1:8500"
        TRAEFIK_PROVIDERS_CONSULCATALOG_ENDPOINT_SCHEME = "http"
        TRAEFIK_PROVIDERS_CONSULCATALOG_ENDPOINT_TLS_CAOPTIONAL = "true"
        TRAEFIK_PROVIDERS_CONSULCATALOG_CONNECTAWARE = "true"
        TRAEFIK_PROVIDERS_CONSULCATALOG_CONNECTBYDEFAULT = "true"
        TRAEFIK_PROVIDERS_CONSULCATALOG_EXPOSEDBYDEFAULT = "false"
        TRAEFIK_PROVIDERS_CONSULCATALOG_REFRESHINTERVAL = "30"
        TRAEFIK_PROVIDERS_CONSULCATALOG_SERVICENAME = "traefik"
        TRAEFIK_PROVIDERS_CONSULCATALOG_DEFAULTRULE = "Host(`{{ normalize .Name }}.example.com`)"
      }
    }
  }
}