Recommended best practices for storing API credentials?

What are your recommendations for safely storing API credentials, secret keys, etc. The getting started guide asks you to put your AWS credentials directly into your code:

Indeed, the Getting Started guide is using that technique because it’s the simplest for just getting the provider setup done and moving on to the focus of the guide, but it is unfortunate that the first thing we introduce is actually not something most users should be doing.

Our general recommendation is that anything relating to who is running Terraform or where Terraform is running should live outside of the Terraform configuration. That could be environment variables, system-specific configuration files, etc, but we have generally preferred to use non-Terraform-specific existing mechanisms where possible, so that users can e.g. just set up AWS credentials once on their machine and have them shared between the AWS CLI, Terraform, and any other tool using the AWS SDKs.

The provider configuration, on the other hand, is for any settings related to what Terraform is deploying to. For example, it makes sense to set region in the configuration (or indirectly through an input variable/data source) because it’s about what Terraform is supposed to create/manage, not who is running Terraform or where Terraform is running.

The rule of thumb I like to follow is: your average Terraform configuration should ideally be runnable with no input variables and do something reasonable. It should fully describe what is to be created and where it is to be created (possibly overridable by input variables, but with defaults), but a different person should be able to run that same configuration in a different context and get the same result.

This gives flexbility as needs change. To start we might run Terraform on individual laptops with credentials in ~/.aws/credentials, but later we might switch to running Terraform on an EC2 instance with an instance profile, or in Terraform Enterprise with AWS credentials set via environment variables. Ideally we should be able to move the configuration around like this without changing its source code at all.

There is a yellow warning box in the getting started guide that alludes to this, but I think we could be more explicit that the technique used in the Getting Started guide is only for convenience in the tutorial, and not a practice that should be carried forward into real use.

5 Likes

I would suggest using AWS roles for terraform which voids the purpose of storing credentials locally. Storing credentials in ~/.aws/credentials is not a good idea since its clear text.

2 Likes

I agree with the EC2 role recommendation, and would go on to say that a simple multi-account switch role example (walkthrough) would be good to have in such a case.

The typical scenario is that the TF machine could be managing infra of many different AWS accounts.

Since the original question was very general, I was trying to keep things broad in my original answer and just used ~/.aws/credentials as an example of some place credentials might come from.

Indeed, if you are using AWS – particularly in a larger, decomposed architecture – using multiple accounts and assume_role can be very helpful in keeping things isolated while still running Terraform centrally. The s3 backend documentation has a multi-account AWS architecture guide describing one way to set that up, although of course with AWS IAM there are lots of different ways to model things with different tradeoffs, so best to treat that as an example and adapt its details to your needs.