Possibilities for protecting /v1/metrics endpoint

Hi there! We’ve been pretty happy with our Nomad setup so far. One thing we’ve not been able to find a solution for is how to shield access to the /v1/metrics endpoint.

We’ve set up ACLs and would like to allow public access to the Nomad UI on port 4646 for authenticated users. We noticed this means that the /v1/metrics endpoint will also be available, even for requests that don’t contain an ACL token. We’re looking for some options to protect access to this endpoint.

An ideal solution for us would be if we can configure Nomad to serve the /v1/metrics endpoint on a different port from the UI. Then we’d only expose the UI port from outside our private network. I don’t believe it’s possible to configure a separate metrics port on Nomad though, is that right?

Another option would be to use a path-rule on the entrypoint into our Nomad cluster, to disallow http requests to /v1/metric from the public internet. We currently use an AWS NLB as the entry-point to the private network our Nomad cluster is running in though, a layer-4 load balancer that does not support path-based rules. It’d be a bummer to have to add an additional layer-7 load balancer solely for the metrics endpoint.

We’re currently running Nomad version 1.7.3.

Have folks come up with any other approaches to protecting access to the /v1/metrics endpoint. We’d be grateful for any ideas!

1 Like