Networking assistance

Nomad
2

I believe I have figured out my issue, which was twofold.

  1. My CNI definition was incomplete
  2. I am assuming that Nomad will not correctly report the intended IP address of the allocation. This might be by “design” if I interpreted this correctly…

For the CNI definition, here is what I’ve settled on:

{
  "cniVersion": "1.0.0",
  "name": "netbootxyz",
  "plugins": [
    {
      "type": "macvlan",
      "master": "vlan.100",
      "mode": "bridge",
      "ipam": {
        "type": "static",
        "addresses": [
          {
            "address": "192.168.100.15/25",
            "gateway": "192.168.100.1"
          }
        ],
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "dns": {
          "nameservers": ["192.168.108.11", "192.168.108.10"],
          "domain": "example.com",
          "search": ["example.com"]
        }
      }
    },
    {
      "type": "tuning",
      "mac": "7a:44:45:00:00:00"
    }
  ]
}

The job:

job "netbootxyz" {
  datacenters = ["lab"]
  type        = "service"

  group "netbootxyz" {
    network {
      mode = "cni/netbootxyz"
      port "ui" {
        static = 3000
      }
      port "tftp" {
        static = 69
      }
    }

    volume "truenas-nfs" {
      type            = "csi"
      source          = "truenas-nfs"
      read_only       = false
      attachment_mode = "file-system"
      access_mode     = "multi-node-multi-writer"
    }

    task "netbootxyz" {
      driver = "podman"

      config {
        image      = "netbootxyz/netbootxyz:0.7.1-nbxyz3"
        ports      = ["ui", "tftp"]
        privileged = true
      }

      volume_mount {
        volume      = "truenas-nfs"
        destination = "/assets"
        read_only   = false
      }

      volume_mount {
        volume      = "truenas-nfs"
        destination = "/config"
        read_only   = false
      }

      resources {
        cpu        = 1000
        memory     = 1024
      }
    }
  }
}

From here, the alloc/container participates in the physical network with all its intended requirements. But the UI reports that the alloc/container is bound to the node’s IP:

nomad alloc status -json 30c76495 | jq '.Resources.Networks'
[
  {
    "CIDR": "",
    "DNS": null,
    "Device": "",
    "DynamicPorts": null,
    "Hostname": "",
    "IP": "192.168.100.13",
    "MBits": 0,
    "Mode": "cni/netbootxyz",
    "ReservedPorts": [
      {
        "HostNetwork": "default",
        "Label": "ui",
        "To": 0,
        "Value": 3000
      },
      {
        "HostNetwork": "default",
        "Label": "tftp",
        "To": 0,
        "Value": 69
      }
    ]
  }
]

Since I am very much a Nomad newb, if my interpretation is incorrect, please do correct me.

Thanks