I just found that Nomad seems to be executing a iptables command when starting a task which uses an invalid option flag.
When the task starts I get this error in the Nomad UI:
failed to setup alloc: pre-run hook "network" failed: failed to configure networking for alloc: failed to configure network: plugin type="portmap" failed (add): unable to create chain CNI-HOSTPORT-SETMARK: running [/usr/bin/iptables -t nat -C CNI-HOSTPORT-SETMARK -m comment --comment CNI portfwd masquerade mark -j MARK --set-xmark 0x2000/0x2000 --wait]: exit status 2: iptables v1.8.8 (legacy): unknown option "--set-xmark" Try `iptables -h' or 'iptables --help' for more information.
I’ve tried downgrading iptables from 1.8.8 to 1.8.3 where it seems that the flag exists, but the error persists.
I’ve also checked that my kernel modules are loaded:
❯ sudo lsmod | rg iptable
iptable_filter 16384 1
iptable_nat 16384 1
nf_nat 57344 2 iptable_nat,xt_MASQUERADE
iptable_mangle 16384 0
ip_tables 36864 3 iptable_filter,iptable_nat,iptable_mangle
x_tables 57344 8 xt_conntrack,iptable_filter,xt_addrtype,xt_comment,ip_tables,iptable_nat,xt_MASQUERADE,iptable_mangle
I’m on Arch Linux running Nomad 1.4.3 and this just appeared today out of nowhere.