Hi there, I’m trying to do something which should be relatively straightforward. I’m converting a Docker compose file to a Nomad job. Here is the compose file:
version: "3.9"
services:
axeapi:
build: .
restart: always
ports:
- "8000:8000"
links:
- redis
axerunner:
build: ./cron
restart: always
environment:
- SCRIPT_SLEEP_SECONDS=600
- REQUEST_DELAY_MILLISECONDS=50
- REDIS_HOST=redis
- REDIS_PORT=6379
- LOG_LEVEL=DEBUG
- NUMBER_OF_DAYS=60
links:
- redis
redis:
image: redis
restart: always
ports:
- "6379:6379"
The axerunner container requires internet access to periodically query an API, while as the axeapi is internal only. I’m testing this locally on a Fedora 35 workstation by running Nomad and Consul in dev mode e.g.
sudo nomad agent -dev-connect
and
consul agent -dev
I’ve converted the compose file above into the following job file:
job "axe" {
datacenters = ["dc1"]
type = "service"
group "axeapi" {
network {
mode = "bridge"
port "http" {
static = 8000
to = 8000
}
}
service {
name = "axeapi"
port = "http"
connect {
sidecar_service {
proxy {
upstreams {
destination_name = "redis"
local_bind_port = 6379
}
}
}
}
}
task "axeapi" {
driver = "docker"
config {
image = "127.0.0.1:5000/axeapi:v2"
}
}
}
group "axerunner" {
network {
mode = "bridge"
dns {
servers = []
options = []
searches = []
}
}
service {
name = "axerunner"
connect {
sidecar_service {
proxy {
upstreams {
destination_name = "redis"
local_bind_port = 6379
}
}
}
}
}
task "axerunner" {
driver = "docker"
config {
image = "127.0.0.1:5000/axerunner:v2"
}
env {
SCRIPT_SLEEP_SECONDS = "600"
REQUEST_DELAY_MILLISECONDS = "50"
REDIS_HOST = "127.0.0.1"
REDIS_PORT = "6379"
LOG_LEVEL = "DEBUG"
}
}
}
group "redis" {
network {
mode = "bridge"
}
service {
name = "redis"
port = "6379"
connect {
sidecar_service {}
}
}
task "redis" {
driver = "docker"
config {
image = "redis:3.2"
}
}
}
}
This almost works, except I can’t seem to get the axerunner container to be able to query the external API, I get a “Temporary failure in name resolution”, e.g. DNS issues. The resolv.conf inside the container in question looks ok, as shown here:
nameserver 10.159.0.1
nameserver 8.8.8.8
nameserver 8.8.4.4
I’ve omitted the usual resolv conf commentary for brevity.
I can also confirm that running this image manually works ok. The DNS settings in the Nomad file above I’ve included as I read a previous discussion which said entering those settings with empty lists causes the container to use host defaults. So resolv.conf looks good, but strangely if I run docker inspect and vimdiff the manual run of the container and the Nomad run of the container I can see the following differences.
Left pane is the manually run container and the right pane is the container run by Nomad.
I’m running Nomad version v1.2.3 and Consul version v1.11.2.
Struggling at what to do next here, any help would be greatly appreciated.