Recommended best practices for storing API credentials?

What are your recommendations for safely storing API credentials, secret keys, etc. The getting started guide asks you to put your AWS credentials directly into your code:

Indeed, the Getting Started guide is using that technique because it’s the simplest for just getting the provider setup done and moving on to the focus of the guide, but it is unfortunate that the first thing we introduce is actually not something most users should be doing.

Our general recommendation is that anything relating to who is running Terraform or where Terraform is running should live outside of the Terraform configuration. That could be environment variables, system-specific configuration files, etc, but we have generally preferred to use non-Terraform-specific existing mechanisms where possible, so that users can e.g. just set up AWS credentials once on their machine and have them shared between the AWS CLI, Terraform, and any other tool using the AWS SDKs.

The provider configuration, on the other hand, is for any settings related to what Terraform is deploying to. For example, it makes sense to set region in the configuration (or indirectly through an input variable/data source) because it’s about what Terraform is supposed to create/manage, not who is running Terraform or where Terraform is running.

The rule of thumb I like to follow is: your average Terraform configuration should ideally be runnable with no input variables and do something reasonable. It should fully describe what is to be created and where it is to be created (possibly overridable by input variables, but with defaults), but a different person should be able to run that same configuration in a different context and get the same result.

This gives flexbility as needs change. To start we might run Terraform on individual laptops with credentials in ~/.aws/credentials, but later we might switch to running Terraform on an EC2 instance with an instance profile, or in Terraform Enterprise with AWS credentials set via environment variables. Ideally we should be able to move the configuration around like this without changing its source code at all.

There is a yellow warning box in the getting started guide that alludes to this, but I think we could be more explicit that the technique used in the Getting Started guide is only for convenience in the tutorial, and not a practice that should be carried forward into real use.

6 Likes

I would suggest using AWS roles for terraform which voids the purpose of storing credentials locally. Storing credentials in ~/.aws/credentials is not a good idea since its clear text.

2 Likes

I agree with the EC2 role recommendation, and would go on to say that a simple multi-account switch role example (walkthrough) would be good to have in such a case.

The typical scenario is that the TF machine could be managing infra of many different AWS accounts.

Since the original question was very general, I was trying to keep things broad in my original answer and just used ~/.aws/credentials as an example of some place credentials might come from.

Indeed, if you are using AWS – particularly in a larger, decomposed architecture – using multiple accounts and assume_role can be very helpful in keeping things isolated while still running Terraform centrally. The s3 backend documentation has a multi-account AWS architecture guide describing one way to set that up, although of course with AWS IAM there are lots of different ways to model things with different tradeoffs, so best to treat that as an example and adapt its details to your needs.

What is the recomended way to achieve setting of credentials for a service like app.terraform.io? How would I hook in aws secrets manager?

Should I be supplying a ami key, for a user to assume a role, that then gets secrets from the manager? It seems at present there is no clear way to escape credentials at some level being in clear text and/or limiting those that can view that text.

Hi @staticcat,

In Terraform Cloud the common approach is to use stored environment variables to set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to some credentials with at least enough access to do whatever other actions need to happen. You can mark the secret as “Sensitive” in that UI, in which case Terraform Cloud will make it available only to the running Terraform processes and won’t allow it to be retrieved in the editing UI.

Since Terraform Cloud is a non-interactive environment, the usual concerns of being able to switch between different profiles to apply different configurations don’t apply: each workspace has its own set of environment variables.

It’s still possible to use an assume_role approach there, by providing credentials that have access only to call sts:AssumeRole to obtain other access.

When using Terraform Cloud it’s mandatory to use its own state storage, so credentials for the S3 backend are no longer important and the problem is just to make suitable credentials available to the AWS provider.