Hello,
I would like to setup a very simple nomad single node cluster where:
- most of the workloads will be private (bind to localhost)
- few ones could be public like the load balancer which get request from internet and route it to private workloads (e.g. nginx)
client {
enabled = true
host_network "public" {
cidr = "MY_PUBLIC_ADDRESS/24"
}
host_network "private" {
cidr = "127.0.0.1/8"
}
}
data_dir = "/var/lib/nomad"
datacenter = "dc1"
consul {
address = "127.0.0.1:8500"
}
# Enable the server
server {
enabled = true
bootstrap_expect = 1
}
The goal here is to start on the same host 1 server and 1 client with 2 networks: public and private.
Then I can follow the following tutorial Load Balancing with NGINX | Nomad - HashiCorp Learn to deploy a nginx public load balancer serving private demo app service.
nginx configuration is:
job "nginx" {
datacenters = ["dc1"]
group "nginx" {
count = 1
network {
mode = "host"
port "http" {
static = 8080
host_network = "public"
}
}
service {
name = "nginx"
port = "http"
}
task "nginx" {
driver = "docker"
config {
image = "nginx"
ports = ["http"]
volumes = [
"local:/etc/nginx/conf.d",
]
}
template {
data = <<EOF
upstream backend {
{{ range service "demo-webapp" }}
server {{ .Address }}:{{ .Port }};
{{ else }}server 127.0.0.1:65535; # force a 502
{{ end }}
}
server {
listen 8080;
location / {
proxy_pass http://backend;
}
}
EOF
destination = "local/load-balancer.conf"
change_mode = "signal"
change_signal = "SIGHUP"
}
}
}
}
demo app configuration is:
job "demo-webapp" {
datacenters = ["dc1"]
group "demo" {
count = 1
network {
mode = "host"
port "http" {
to = -1
host_network = "private"
}
}
service {
name = "demo-webapp"
port = "http"
check {
type = "http"
path = "/"
interval = "2s"
timeout = "2s"
}
}
task "server" {
env {
PORT = "${NOMAD_PORT_http}"
NODE_IP = "${NOMAD_IP_http}"
}
driver = "docker"
config {
image = "hashicorp/demo-webapp-lb-guide"
ports = ["http"]
}
}
}
}
This configuration apparently works as expected:
- both workloads are running
- nginx workload listens on [MY_PUBLIC_ADDRESS]:8080
- demo workload listens on 127.0.0.1:[DYNAMIC_PORT]
- nginx template is properly rendered:
nomad alloc fs d20c1772 nginx/local/load-balancer.conf
upstream backend {
server 127.0.0.1:22741;
}
server {
listen 8080;
location / {
proxy_pass http://backend;
}
}
If I request the demo workload manually from the host, it works fine:
curl 127.0.0.1:22741
Welcome! You are on node 127.0.0.1:22741
but sadly it does not work through the load balancer:
curl [MY_PUBLIC_ADDRESS]:8080
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.21.3</center>
</body>
</html>
it appears the problem comes from the fact nginx workloard does not have network access to the demo workloard:
nomad alloc exec -i -t -task nginx d20c1772 sh
# curl 127.0.0.1:22741
curl: (7) Failed to connect to 127.0.0.1 port 22741: Connection refused
I suppose it is normal but I would be happy to have your feeling and advice about the best way to handle this use case which seems pretty simple.
I suppose using Consul Connect | Nomad by HashiCorp can allow me to do what I want, but I would like to keep the config as simple as possible so I prefer to be sure to need it before to go forward.
your confirmation or any help and recommendations would be much appreciated.
Thank you !
Notice: I know and I understand this is not recommended to run nomad cluster as single node in production. However, in my case, nomad is used much as an abstraction layer to install/schedule the same set of services on different servers (prod / preprod …) than as a real clustering manager. Its configuration is done thanks to terraform so I can loose my data dir and simply reapply the configuration if needed.