UID/GID ownership of template output in NOMAD_SECRETS_DIR for Postgres

[harningt]

@eikenb

@harningt

If you set the uid of the cert/key, then the file is re-rendered every ~3-5 minutes … sometimes multiple times in a row. The final content is the same - but it is a tad annoying to have signals send so frequently to reload configuration changes.

Ownership (UID/GID) is one of the things checked for at render time and it will trigger a re-render. It wants the files generated to have the same UID/GID as the running process. For consul-template this makes sense as you’d be running consul-template as the postgres user. Not sure how this is supposed to work with nomad.
…

Looking at the code - it looks like it is supposed to be doing a check on whether or not the UID and GID match up with what is configured for both Nomad and consul-template. The whole render-path setup for Nomad looks to be directly through consul-template … which checks to see if the recorded render time is >= the stored render time.

@eikenb

@harningt

Scenario that seems to work okay:

use two templates w/ secret to render the private key and public key

Note that this will also result in getting a new PKI cert/key whenever you restart the (nomad?) process as secret works only with the data from vault which is keeps in memory to know its state. It cannot read data back from the local files as pkiCert does.

@harningt

pkiCert template that works ok for non-nomad:

• Single template that writes out the certificate and has a writeFile to key
  … does not work well for nomad because it requires unlocking file writing capabilities
  … also still requires the script to change permissions

This is consul-template’s current recommended way to work with pkiCert in cases where you need to split them out into separate files.

Got it - yeah - it feels like it’d add a ton of complexity to handle it any other way.

@eikenb

Sorry I’m not sure if any of that helps that much.

I’ve been thinking about other ways of addressing this but haven’t come up with much. You need both the PKI Key and the Cert as they cannot be re-fetched without generating a new one. But there is no place to store them except for the target file and there is only 1 of those. Cases like this are one of the reasons we added writeToFile.

For writeToFile the listed concern is that you have control to write freely on the filesystem. Would it be reasonable to add a restriction on writeToFile to honor the path sandbox that file reading does? Looking at things it seems like it’s a simple operation since the majority of the work is dealt with by the file utility.

I think the writeToFile solution will be what I adopt - I’ve used it for writing out PKI when I don’t have Nomad helping out (but the process owner being an unprivileged user and writeToFile not restricted). In the short-term, I’ll just live with the unlikely chance that something calls writeToFile in a way that breaks the sandbox.

So the plan I think will work right:

• Set a template output to a file that has everything (keys, cert, ca…) w/ restrictive owner and not changing UID
• Add writeToFile items in the template that write out the separate certificate and key files using the stored template data and sets the appropriate owner
• Unlock the writeToFile capability in Nomad.
• See about writing an pull-request for writeToFile honoring the sandbox.