How to configure CNI Macvlan in Nomad

Dear board members,

I’m trying to configure macvlan using CNI in Nomad.

Currently I have got it working via Docker networking (created a macvlan network on each nomad client):

docker network create -d macvlan
–subnet=192.168.137.0/24
–gateway=192.168.137.1
-o parent=eth0 ingress

Then using this job config will launch a docker container on a nomad client which is reachable from the outside with the specified IP address, moving the container around will also move the IP to the new host which is what I need:

job “docs” {
datacenters = [“dc1”]

group “example” {
count = 1

network {
port “http” {
to = “8181”
}
}

task “server” {
driver = “docker”

 service {
   name = "http-echo"
   port = "http"
   address_mode = "driver"
   check {
     name     = "http-echo-check"
     type     = "tcp"
     interval = "10s"
     timeout  = "2s"
   }
 }

 resources {
   cpu    = 600
   memory = 128
 }

 config {
   image = "hashicorp/http-echo"
   network_mode = "ingress"
   ipv4_address = "192.168.137.232"
   ports = ["http"]
   args = [
     "-listen",
     ":8181",
     "-text",
     "hello world",
   ]
 }

}
}
}

However, this is the docker way, not the Nomad CNI way… I did configure a macvlan interface via CNI (/opt/cni/config/ingress.conflist):

{
“cniVersion”: “0.4.0”,
“name”: “ingress”,
“plugins”: [
{
“type”: “macvlan”,
“master”: “eth0”,
“ipam”: {
“type”: “host-local”,
“subnet”: “192.168.137.224/27”,
“gateway”: “192.168.137.1”
}
},
{
“type”: “portmap”,
“capabilities”: { “portMappings”: true },
“snat”: true
}
]
}

However when I use this by adding this into the network stanza of the job config above Nomad won’t assign the IP to the docker container…

mode = “cni/ingress”

I’ve been looking everywhere online but could not find a good help resource. I hope that someone here is able to help out :slight_smile: