Securing Postgresql (pg) backend credentials

Hi folks,

I’m new to Terraform, therefore apologies in case I’m asking something obvious.

I am trying to use the pg backend in a secure way, and that means not storing the PostgreSQL connection string (containing username/password) on the file system.

So I’m trying to use environment variables. I know they are not perfect, but better than permanently storing credentials on the file system.

After a bit of struggle, I managed to figure out how to pass the connection string as an environment variable to the cli, which looks something like this:

TF_CLI_ARGS="-backend-config='conn_str=postgres://some_host/some_db'" \
        terraform init

…and this would be all right, but then I found out that .terraform/terraform.tfstate still contains the connection string in clear text under backend.config.conn_str.

Am I missing something? Is there a better, more secure way?

Thanks!

Hi @salomvary,

It sounds like you’re on the right track here that the typical answer is to put the sensitive values in the environment (in a backend-specific way) and put non-sensitive values in the configuration, which can either mean literally in configuration files (a backend block in a .tf file) or by overriding that configuration on the terraform init command line.

I think where I lost the trail of your question is the relationship between the fact that you put the credentials in the environment and the fact that the connection string appeared in the filesystem. That sounds like the expected result to me: the .terraform/terraform.tfstate would track the connection string but not the credentials.

Are you saying that the credentials from the environment variables are also ending up in the file on disk? If so, that isn’t expected, but we can work on figuring out why it happened and how to stop it.

I’m sorry if I’m missing something crucial here; I am admittedly coming from a position of knowing about how backends in general work rather than specifically how the pg backend works, so perhaps there’s something special about that one that I’m missing but I can go read up a bit if this turns out to be something specific to that backend.

I have the same question. Documentation seems to imply that sensitive credentails (PG password) can be provided via env vars, but it does not seem to be the case. It seems you have to put the password into conn_str (whether hardcoded in the .tf file, or passed via -backend-config, either on command line or through TF_CLI_ARGS env var). In all those cases, the password, being part of the config string, ends up being stored on disk.

I’d like to make sure password never touches disk - and instead is only ever read from env var. How can this be done? It appears that you cannot use variables (which could be set with env vars) in backend config…
@apparentlymart, is there any way at all to have Terraform read the password dynamically from some env var each time?

Thanks.

Hi @nirvana-msu!

Most of the available backends are maintained by folks outside of the Terraform Core team and so I’m afraid I’m not very familiar with the pg backend’s design.

From what I can tell from reading the source code, it seems that you are right: this particular backend expects all of its settings to be packed into a single URI-like connection string, and doesn’t seem to have any separate mechanisms for passing credentials.

The typical design principle for a backend is for it to accept credentials in the same ways that the official CLI tool (or similar) for the target system would, so that you can configure credentials just once in that standard way and have it work for both the official CLI and for Terraform.

For the pg backend in particular I suppose that means that the backend should obtain credentials the same way that the pg CLI does. If that isn’t true today then I would suggest opening a feature request against the backend to describe which credentials-passing strategies are not currently working, and then the maintainers can decide how to respond to that.

I looked in the Terraform source code to determine which underlying Go library the code is using to talk to PostgreSQL.

Then, looking up the documentation for that library at https://pkg.go.dev/github.com/lib/pq:

Most environment variables as specified at PostgreSQL: Documentation: 14: 34.15. Environment Variables supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters.

The pgpass mechanism as described in PostgreSQL: Documentation: 14: 34.16. The Password File is supported, but on Windows PGPASSFILE must be specified explicitly.