Confused about mTLS, systemd units, ports, agent instances


in my small cluster I run several Nomad clients & on three of them I also have the Nomad server portion active. So far I do not have encryption active. I’m using Hashicorp’s packages for Ubuntu.

At the moment I’m trying to follow the tutorial for enabling mTLS, and I’m running into several confusing questions that I cannot seem to find the answer to, or the best common practice.

First of all, certificates have been created for the client, the server & the CLI. No problem.

Now, when trying to configure both the client & the server to use them, it seems that I must use two different configuration files for them as both use a top-level tls { … } block with different content, referring to a role-specific certificate (client vs server).

This in turn means I cannot use the default systemd unit file nomad.service that comes with the Ubuntu packages as that unit starts the nomad binary as nomad agent … -config /etc/nomad.d, a directory, meaning it’ll read all files from that directory — no way to distinguish between “this tls block is only for the client, that other one only for the server, and both in the same agent instance, please”.

I can deploy two custom systemd unit files, both starting an agent with -config /etc/nomad.d/client.hcl and -config /etc/nomad.d/server.hcl respectively. However, in that case I must also re-configure the port numbers as both processes will otherwise try to open 4646, which fails for the second unit, whichever that may be.

In my basic setup I’m currently running I have both client & server running as a single agent instance with a single configuration file, enabling both parts. Yes, I’m aware that the documentation recommends against it but it has the advantage of being easy to get running. And the other way (running multiple agent instances, one for client, one for server) isn’t really documented well.

What’s the best way to go about it?