Reference architecture for service mesh

Nomad
1

Is there any reference architecture for combination of Nomad + Consul Service Mesh + Ingress Gateway + Traefik?

I setup Nomad + Consul Service Mesh successfully but I dont know how to public traffic to internet.

Currently I used Traefik + Consul Catalog to public traffic.

1 Like
2

I want to use Traefik as edge proxy because of lacking features of Consul Ingress Gateway but I don’t know how to setup Consul Service Mesh correctly.

Traefik -> Ingress Gateway -> Service Mesh

I’m using Consul 1.8.4, Nomad 0.12.5.

3

Hi @tunh

Personally, I settled on a docker container that I could deploy via nomad to do the ingress gateway for me:

DockerFile:

FROM envoyproxy/envoy-alpine:v1.14.2

RUN apk add -u bash curl &&\
    wget https://releases.hashicorp.com/consul/1.8.4/consul_1.8.4_linux_amd64.zip -O /tmp/consul.zip &&\
    unzip /tmp/consul.zip -d /tmp &&\
    mv /tmp/consul /usr/local/bin/consul

COPY ingress-proxy/entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

Entrypoint.sh

#!/bin/bash
echo "Starting"
export hostip=$(hostname -i)

# Wait until Consul can be contacted
until curl -s ${CONSUL_HTTP_ADDR}/v1/status/leader | grep 8300; do
  echo "Waiting for Consul to start"
  sleep 1
done


# register any config from individual files
if [ ! -z "$CONFIG_FILE" ]; then
  IFS=';' read -r -a configs <<< ${CONFIG_FILE}

  for file in "${configs[@]}"; do
    echo "Writing central config $file"
    consul config write $file

    exit_status=$?
    if [ $exit_status -ne 0 ]; then
      echo "### Error writing central config: $file ###"
      cat $file
      echo ""
      exit 1
    fi
  done
fi

# register any central config from a folder
if [ ! -z "$CENTRAL_CONFIG_DIR" ]; then
  for file in `ls -v $CENTRAL_CONFIG_DIR/*`; do
    echo "Writing central config $file"
    consul config write $file
    echo ""

    exit_status=$?
    if [ $exit_status -ne 0 ]; then
      echo "### Error writing central config: $file ###"
      cat $file
      echo ""
      exit 1
    fi
  done
fi


# If we do not need to register a service just run the command
if [ ! -z "$SERVICE_NAME" ]; then
  # register the service with consul
  echo "Registering service with consul $SERVICE_NAME"
  consul connect envoy -gateway=ingress -register -service ${SERVICE_NAME} -address ${hostip}:${SERVICE_BIND}

  exit_status=$?
  if [ $exit_status -ne 0 ]; then
    echo "### Error writing service config: $file ###"
    cat $file
    echo ""
    exit 1
  fi
  
  # make sure the service deregisters when exit
  export service_id=$(curl http://${CONSUL_HTTP_ADDR}/v1/catalog/service/${SERVICE_NAME}| jq --raw-output '.[0].ServiceID')
  trap "consul services deregister -id=${service_id}" SIGINT SIGTERM EXIT
fi

I would then have the following nomad job, to deploy the ingress gateway:

job.hcl

job "traefik-ingress" {
  datacenters = ["GENERAL"]

  group "traefik-ingress" {
    network {
      mode = "host"
    }
    task "traefik-ingress" {
      driver = "docker"
      env {
        CONSUL_HTTP_ADDR = "$${NOMAD_IP_http}:8500"
        CONSUL_GRPC_ADDR = "$${NOMAD_IP_http}:8502"
        SERVICE_NAME = "traefik-ingress"
        SERVICE_BIND = "8780"
        CONFIG_FILE = "/tmp/config.hcl"
      }
      service {
        name = "ingress-web"
        tags = [
          "traefik.tags=service",
          "traefik.http.routers.example.rule=Host(`example.com`)",
          "traefik.http.routers.example.tls=true",
          "traefik.http.routers.example.tls.certresolver=resolver",
          "traefik.http.routers.example.tls.domains[0].main=example.com",
          "traefik.http.routers.example.tls.domains[0].sans=www.example.com",
          "traefik.http.routers.example.entrypoints=websecure",
        ]
        port = "http"
      }
      template {
        data = <<EOF
Kind = "ingress-gateway"
Name = "traefik-ingress"
Listeners = [
  {
    Port = 18081
    Protocol = "tcp"
    Services = [
      {
       Name = "web"
      }
    ]
  }
]
EOF
        destination = "local/config.hcl"
      }
      config {
        image = "custom.docker:latest"
        force_pull = "true"
        volumes = [
          "local/config.hcl:/tmp/config.hcl"
        ]
      }
      resources {
        cpu = 100
        memory = 128
        network {
          port "http" {
            static = 18081
          }
          port "envoy" {
            static = 8780
          }
        }
      }
    }
  }
}

Hope it helps you. This was also something that took me a while to get right.

Cheers
@CarelvanHeerden

2 Likes
4

@CarelvanHeerden Thank you for the reference!

1 Like