Using a new private host network. how to see values or refer to them?

hi there.

I’d appreciate some pointers on setting up some nodes I have properly, as I think as I’ve been learning, I’ve done a few things wrong.

My situation now:

I have a few nodes in a network which are accessible via a public IP.

I have since set up a private network that I would like the nodes to be communicating on instead for internal traffic.

However, I’d still like the option of being able to advertise a public IP for some of these machines, rather than have a single ingress point that proxies to the every private machine.

I can see the appeal of having this and can imagine us migrating there later.

I’d like to use nomad to manage the various services they expose, and this is where I’d like some help.

Moving from hardcoded /static ip ports to referring to hthe proper variables

I have a job like this for one job running on one 'monitoring box, where other machines are sending statsd metrics to over the private network via UDP.

This job is designed to only run on the monitoring box, which I’m using the constraints stanza to so.

It looks like this:


job "statsd_exporter" {
  datacenters = ["dc1"]

  type = "service"

  group "statsd_exporter" {

    count = 1

    network {
      mode = "host"
      port "stats_d_exporter" {

        // exporter_port
        static = 9102
      }
    }

    task "statsd_exporter" {
      driver = "raw_exec"

      config {
        command = "statsd_exporter"
        args    = ["--web.listen-address", "10.0.0.2:9102"]
      }

      resources {
        cpu    = 250 # 250 MHz
        memory = 64  # 64MB
      }

      constraint {
        attribute = "${meta.role}"
        value     = "o11y"
      }

    }
  }
}


I don’t want to hardcode the ip address, and as I mentioned before, I have a private network on a different interface that I’d like to listen to for UDP packets from other nodes in the system.

Assume the internal network is something like 10.0.0.0/24, and the other nodes are on the same network, and can reach each others IP address on this network, so we have 10.0.0.4, 10.0.0.7 and so on. The monitoring box’s internal would address be somethign like 10.0.0.2

How do I represent this private host network in my client stanza?

I’ve tried this and in the nomad UI, while I’ve been able to see new “meta” info I have added to a client like defined in the constraint role above, but I can’t see the network in the UI.

client {
  host_network "my-provider-1" {
    interface = "ens10"
    cidr      = "10.0.0.0/24"
  }
}

I’m also not sure how to refer to any internal IP address, of 10.0.0.2 using variables.

I’d like to have something like this in this job


config {
        command = "statsd_exporter"
        args = ["--web.listen-address",  "$(CLIENT_NODE_PRIVATE_IP)" "$(NOMAD_PORT_stats_d_exporter)"]
      }

But when I’m looking in the new docs below, I can’t see a simple way to reach this.

How do I do this?

Do I need to clear out the local state of any machines that connected using a public network and reconnect, being explicit about using the private network IP now?

And if this is the case, if I want to make the node reachable from it’s public IP, how would I do this?

Thanks, and apologies for asking such basic questions.

Ah, I think I figured it out.

this bit here

config {
        command = "statsd_exporter"
        args = ["--web.listen-address",  "$(CLIENT_NODE_PRIVATE_IP)" "$(NOMAD_PORT_stats_d_exporter)"]
      }

I needed to define the network like so in my client:

client {
  host_network "my_named_network" {
    interface = "ens10"
    cidr      = "10.0.0.0/24"
  }
}

then refer to it in the job

network {
  mode = "host"
  port "statsd_exporter" {
    static       =  9102
    host_network = "my_named_network"
  }

}

and then refer to port like so:

config {
        command = "statsd_exporter"
        args = [
          "--web.listen-address",  "${NOMAD_ADDR_stats_d_exporter}"
        ]
      }

i now get the private IP port combo I wanted

Hi @mrchrisadams,

Glad you managed to figure it out and thanks for posting the detail which will be useful to others.

Thanks,
jrasell and the Nomad team

1 Like