Pointing nomad to a remote docker daemon over TCP

Nomad
, ,
1

I need help with pointing nomad to the docker daemon running at "tcp://10.0.2.2:5732"

Here’s the Nomad agent config I am currently using: hashiqube/nomad.sh at master · servian/hashiqube · GitHub

It’s at /etc/nomad/server.conf

I need the docker driver to point to another docker host - the docker daemon running at "tcp://10.0.2.2:5732"

Is there a way not to modify the /etc/nomad/server.conf but override the docker plugin stanza to:

plugin "docker" {
  config {
    endpoint = "tcp://10.0.2.2:5732"
  }
}

Given that I already have only one agent running (this is for testing), should I:

  1. Kill the running Nomad agent: sudo kill ${NOMAD_AGENT_PID}
  2. Restart the Nomad agent: sudo nomad agent -dev-connect -config/etc/nomad/server.conf -config=nomad_pointing_to_external_docker_daemon.hcl
    (I will actually redo the block hashiqube/nomad.sh at master · servian/hashiqube · GitHub with only the above changed config arg)

where nomad_pointing_to_external_docker_daemon.hcl contains:

plugin "docker" {
  config {
    endpoint = "tcp://10.0.2.2:5732"
  }
}

Will this work?

2

You can pass in more than one configuration file using the -config flag and they are applied in the order you supply them. If the -config flag points to a directory, they are applied in dictionary order.

I use this technique for Nomad host_volumes, plugin "raw_exec", & my vault stanza configurations.
I use a single -config pointing at the directory. Then I can comment them out of my configuration during tests by renaming them from .hcl to .hcl.off

🔍so my configuration dir looks like this

host_networks.hcl

client {
  host_network "private" {
    cidr = "11.0.0.0/24"
  }
}

nomad.hcl

data_dir = "/opt/nomad/data"
datacenter = "dc1"
enable_debug = true
leave_on_interrupt = true
log_level = "DEBUG"

client {
  enabled = true
  cni_config_dir = "/opt/cni/config"
}

consul {
  token = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

vault.hcl

vault {
  enabled = true
  address = "http://active.vault.service.consul:8200"
  task_token_ttl = "10m"
}

plugin-docker.hcl

plugin "docker" {
  config {
    auth { config = "/etc/nomad.d/docker_auth.json.txt" }
    allow_privileged = true
    volumes {
      enabled = true
    }
  }
}

plugin-raw_exec.hcl

plugin "raw_exec" {
  config {
    enabled = true
  }
}

telemetry.hcl

telemetry {
  publish_allocation_metrics = true
  publish_node_metrics       = true
  prometheus_metrics	     = true
}

volumes.hcl

client {
  host_volume "container-test" {
    path = "/opt/nomad/volumes/container-test"
    read_only = false
  }
  host_volume "scratch" {
    path      = "/opt/nomad/scratch"
    read_only = false
  }
  host_volume "docker-registry" {
    path = "/opt/volumes/docker-registry"
    read_only = false
  }
  host_volume "zk1" {
    path = "/opt/nomad/volumes/zk1"
    read_only = false
  }
}

Hopefully this gives you some ideas. Also, given the connection you linked, you shouldn’t need the -dev-connect flag. The -dev-connect flag exists to override the -dev agent’s default behavior of binding to localhost.

2 Likes