Hello.
I want to set up the nginx configuration using a template. But I can’t get network variables from the main task.
Job:
job "backend" {
datacenters = ["dc1"]
type = "service"
group "backend" {
count = 1
constraint {
operator = "distinct_hosts"
value = "true"
}
network {
port "http" {
static = 8085
}
port "backend" {
static = 8090
}
}
volume "cored" {
type = "host"
read_only = true
source = "cored"
}
restart {
attempts = 2
interval = "5m"
delay = "15s"
mode = "fail"
}
ephemeral_disk {
size = 300
}
task "backend" {
user = "cored"
driver = "exec"
volume_mount {
volume = "cored"
destination = "/usr/local/cored-d/"
read_only = true
}
config {
command = "/usr/local/cored-d/bin/linux/cored"
args = []
}
resources {
cpu = 200
}
}
task "web" {
driver = "docker"
config {
image = "nginx"
volumes = [
"local/default.conf:/etc/nginx/conf.d/default.conf",
]
}
template {
data = <<EOF
server {
listen 8085;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://${NOMAD_IP_backend}:${NOMAD_PORT_backend};
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
EOF
destination = "local/default.conf"
change_mode = "signal"
change_signal = "SIGHUP"
}
resources {
cpu = 200
}
}
service {
name = "cored"
tags = ["http"]
port = "http"
check {
name = "alive"
type = "http"
path = "/"
interval = "10s"
timeout = "2s"
}
}
}
}
default.conf in the container:
server {
listen 8085;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://${NOMAD_IP_backend}:${NOMAD_PORT_backend};
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
I also tried to use $$ instead of $.
I also tried to use {{env NOMAD_IP_backend}} and {{NOMAD_IP_backend}} but then I get the following error:
Template failed: (dynamic): parse: template: :6: function "NOMAD_IP_backend" not defined
Did I do something wrong?
Hey! Thanks for sharing the code in first place, this is the correct way to ask for help here.
Having said that, I don’t want to be rude but there are a lot of things wrong in the file. I would suggest you to slice it in smaller pieces, otherwise it’s too hard to debug.
First, create one task and its service with the nomad service discovery (if you are not using consul). You could check if the service was created correctly by the nomad CLI nomad service list.
Then create two tasks in the same group one of them with a service.
And then try to query the service from the template stanza of the other task.
It’s too long to answer these questions all together. I would suggest to ask more direct questions.
I am also doubting what you are trying to achieve. It seems the nginx settings point to itself. I might be wrong too, but I would suggest you to explain without techie vocal your use case.
I will be waiting your answer!
Hello. Thank you for your response.
Okay, the backend uses a web framework that has a widespread problem with memory leaks due to dead connections. So, to solve this issue, I have to set up the worker connection manager. For example, Nginx uses this one. I can set up worker connection numbers, and dead connections will be deleted.
Currently, I use static ports, but in the future, I want to use dynamic ports. and I don’t understand how I can set these ports in the Nginx configuration in Nomad. I know I can set up two services and in the local nginx get ports by SRV resolver. But then the alive check for the backend service will send requests right to the backend, and I will still have trouble with memory leaks.
And yes, you’re right. The Nginx settings point at themselves. The backend and Nginx are in the same host.
So, I tried to separate tasks.
nginx:
job "nginx-job" {
datacenters = ["dc1"]
type = "service"
group "cored-group" {
count = 1
network {
port "http" {
static = 8085
}
}
restart {
attempts = 2
interval = "5m"
delay = "15s"
mode = "fail"
}
task "cored-web" {
driver = "docker"
config {
image = "nginx"
volumes = [
"local/default.conf:/etc/nginx/conf.d/default.conf",
]
}
template {
data = <<EOF
server {
listen 8085;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://${NOMAD_IP_backend}:${NOMAD_PORT_backend};
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
EOF
destination = "local/default.conf"
change_mode = "signal"
change_signal = "SIGHUP"
}
resources {
cpu = 200
}
}
service {
name = "cored"
tags = ["http"]
port = "http"
check {
name = "alive"
type = "http"
path = "/"
interval = "10s"
timeout = "2s"
}
}
}
}
Backend web app:
job "cored-job" {
datacenters = ["dc1"]
type = "service"
group "cored-group" {
count = 1
network {
mode = "bridge"
port "backend" {
to = 8090
}
volume "cored" {
type = "host"
read_only = true
source = "cored"
}
restart {
attempts = 2
interval = "5m"
delay = "15s"
mode = "fail"
}
ephemeral_disk {
size = 300
}
task "cored-job" {
user = "cored"
driver = "exec"
volume_mount {
volume = "cored"
destination = "/usr/local/cored-d/"
read_only = true
}
config {
command = "/usr/local/cored-d/bin/linux/cored"
args = []
}
resources {
cpu = 200
}
}
}
}
}
I think it is a simple question: we have two tasks: the backend and the web server. How can I transfer the host: port to the Nginx configuration, For example, in Docker, I could use links in the port/network configuration.
Solved.
My first mistake was using ${NOMAD_IP_backend} because it is just an environment variable and Nginx doesn’t know what it is.
My second mistake was using {{env NOMAD_IP_backend}}. I forgot the double quotes.
The solution:
data = <<EOF
server {
listen 8085;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://{{env "NOMAD_ADDR_backend"}};
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
EOF
But thank you for your attention
My bad. I’m glad to hear you found the solution! Thanks for sharing! 