@jrasell I’m running Nomad on Rocky Linux, on which I have currently turned off firewalld, so there shouldn’t be any interference from that. I also have a Unifi Dream Machine Pro that manages my network, but again, there aren’t any settings there that would seem to affect this. I used to run the same containers that I’m now migrating to Nomad with docker-compose on the same host, and they never had this kind of connection issues.
Here is an example job to bring up a container that I just exec into:
job "network_multitool" {
datacenters = ["local"]
type = "batch"
group "network_multitool" {
network {
mode = "bridge"
}
task "network_multitool" {
driver = "docker"
config {
image = "praqma/network-multitool:alpine-extra"
entrypoint = ["sleep", "infinity"]
}
}
}
}
Here’s my nomad config for reference:
data_dir = "/opt/nomad/data"
bind_addr = "0.0.0.0"
datacenter = "local"
acl {
enabled = true
token_ttl = "30s"
policy_ttl = "60s"
}
server {
# license_path is required as of Nomad v1.1.1+
# license_path = "/etc/nomad.d/nomad.hcl"
enabled = true
bootstrap_expect = 1
}
client {
enabled = true
servers = ["127.0.0.1"]
}
plugin "docker" {
config {
allow_caps = ["audit_write", "chown", "dac_override", "fowner", "fsetid", "kill", "mknod",
"net_bind_service", "setfcap", "setgid", "setpcap", "setuid", "sys_chroot", "sys_admin", "sys_rawio"]
gc {
image = true
image_delay = "3m"
container = true
dangling_containers {
enabled = true
dry_run = false
period = "5m"
creation_grace = "5m"
}
}
volumes {
enabled = true
#selinuxlabel = "z"
}
}
}
consul {
address = "127.0.0.1:8500"
server_service_name = "nomad-server"
client_service_name = "nomad-client"
auto_advertise = true
server_auto_join = true
client_auto_join = true
token = "redacted"
}
On the host the nomad interface is this:
55: nomad: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 72:29:24:47:18:61 brd ff:ff:ff:ff:ff:ff
inet 172.26.64.1/20 brd 172.26.79.255 scope global nomad
valid_lft forever preferred_lft forever
inet6 fe80::b436:43ff:fe36:4534/64 scope link
valid_lft forever preferred_lft forever
In the container ip a lists this:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0@if4828: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 8a:0e:c0:dd:33:ec brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.26.64.11/20 brd 172.26.79.255 scope global eth0
valid_lft forever preferred_lft forever
Not sure if this is any indication of the cause, but in the network_multitool container it says that the ping and traceroute operations are not permitted, even if the user is root.
bash-5.1# ping google.com
ping: socket: Operation not permitted
bash-5.1# traceroute google.com
traceroute: socket(AF_INET,3,1): Operation not permitted
bash-5.1# whoami
root
Also, here is my docker daemon.json:
{
"dns": ["172.26.64.1"],
"dns-search": ["consul"],
"insecure-registries": ["http://192.168.1.100:8082"],
"registry-mirrors": ["http://192.168.1.100:8082"],
"bridge": "none"
}