Best Practices for `aws_db_instance` redacting required master password from plans and state

Terraform
1

I have an aws_db_instance with a password property that is required, and was already created long ago with the Random provider via:

resource "random_password" "my_db_password" {
  length  = 16
  special = false
}

I read in the latest documentation that will Terraform 1.11.0 the password_wo property is available to use in place of password.

So currently the password my_db_password is stored in state, and that was created by the Terraform configuration shown above. It appears that what password_wo allows us to do is prevents us from creating the password via our Terraform configuration and let’s us pull it in during the terraform plan and/or terraform apply operation - say fetching that value from vault, but as the documentation states, “Note that this may show up in logs, and it will be stored in the state file.” So it doesn’t get us what we fully need.

Is our best option then to use the ephemeral { } block such as detailed here?

So it seems that we would be destroying our resource "random_password" "my_db_password" by removing that block of code from our Terraform configuration, satisfying the password property of the aws_db_instance by using password_wo in place of plain password property, and also then saving the password from being stored in plan files and/or state files by using the ephemeral block.

or should we use ephemeral is an argument/property for a variable such as detailed here?

Note that the actual DB password I do want to use is grabbed from somewhere else, so I’m just trying to fulfill the requirements of the aws_db_instance while not giving access to the DB instance via any credential/password created via code.

Just trying to find the cleanest/best way to use it given the context.

Thank you!

2

Hi @aaa,

I think you have the gist of it here. The only way to ensure data is not captured in the state is to use an ephemeral value along with a write-only attribute. Since you said the password is coming from an external source; supplying that value via an ephemeral variable is the simplest means of using the feature if that is possible within your workflow. Otherwise you must use an ephemeral resource to supply the value.

1 Like
3

Thank you @jbardin .

I am trying to implement the solution basically as lined out here using an ephemeral block:

ephemeral "random_password" "db_password" {
  length           = 16
  override_special = "!#$%&*()-_=+[]{}<>:?"
}

resource "aws_db_instance" "example" {
  instance_class       = "db.t3.micro"
  allocated_storage    = "5"
  engine               = "postgres"
  username             = "example"
  skip_final_snapshot  = true
  publicly_accessible  = true
  db_subnet_group_name = aws_db_subnet_group.example.name
  password_wo          = ephemeral.random_password.db_password.result
  password_wo_version  = 1
}

https://developer.hashicorp.com/terraform/language/manage-sensitive-data#use-the-ephemeral-block

However, upon terraform plan I get this error:

Error: on main.tf line 23 in ephemeral “random_password” “db_password”:
232: ephemeral “random_password” “db_password {

The provider hashicorp/random does not support ephemeral resource “random_password”

I run terraform version and it shows 1.14.4 so I should be good there. Know what may give?

In any case, please let me know if I need to do something at the provider level for Random or other.

EDIT: Ah, we are on random provider version 3.6.3 and documentation says that this page doesn’t exist for the random provider. I need to upgrade that, let me try. Yes it works, but now it is asking for a password_wo_version. That brings me to another question.

Another Question

password_wo should remove the initial DB random password from both plan and state files, but technically, it is a password that could hypothetically be used to log into the DB correct? I know it is only generated during the Terraform operation be it a plan or an apply but it is not available to Terraform that, so very briefly is that password available.

It looks like password_wo_version is required if I use password_wo because my plan now asks for it with this error:

Error: …
“password_wo”: all of `password_wo,password_wo_version` must be specified

https://developer.hashicorp.com/terraform/language/manage-sensitive-data/write-only#update-write-only-arguments-with-versions

Terraform does not store write-only arguments in state files, so Terraform has no way of knowing if a write-only argument value has changed. Because Terraform cannot track write-only argument values, it sends write-only arguments to the provider during every operation.

So it looks like AWS receives the value of that ephemeral write only password password_wo, is that correct?

Now, upon each terraform apply does that ephemeral "random_password" "db_password" change in value, or do I have to increment password_wo_versionfrom 1 to 2 to 3 and so on to get that value to change? And I guess if I keep it at 1, then only AWS stores that value securely and from then on only AWS knows about it and Terraform never does (or does it always general a dummy random value for terraform plan or apply and sends up that new value to AWS?

4

Yes, during each plan and apply the provider may receive a different ephemeral value for password_wo, because the lifetime of an ephemeral value is only single phase of operation.

The provider just chooses to ignore password_wo during every non-create plan, unless there is a corresponding change to password_wo_version. Changing password_wo_version is your signal to the provider to use a new password_wo value.

5

I had one more question at this point, when we try to make this change and go from a resource to ephemeral it shows that original initial DB password getting destroyed, “will be destroyed”, for resource “random_password” “db_password”, and password attribute getting removed from the aws_db_instance and password_wo_version getting added to the aws_db_instance.

My question is, despite the above destruction of the resource, the actual password that was created should still be available for use on the aws_db_instance to authenticate since it is still configured for use on that AWS RDS instance due to the initial creation of that master DB password for that RDS instance.

I hope this is making sense, I see Terraform will destroy that resource, but I presume i will still in fact exist in the real world and can be used to still authenticate to the AWS RDS instance. Is that correct?

6

I’m not entire sure what you’re asking, but random_password does not represent anything in the real world, or within your infrastructure. The random provider does not connect to any external services. Since random_password isn’t a thing outside of Terraform, destroying that resource only consists of removing from the state.

7

Pardon I was not clear on what the original configuration was, which is the above.

resource “random_password" "db_password" was generated and used as the original password as shown above. So our original password was created using the resource.random_password.db_password and assigned to the password attribute of the resource "aws_db_instance”. So it is originally configured to use it.

So the question is, when we go to ephemeral "random_password" "db_password" it shows the resource “random_password" getting destroyed. I think that password is still configured for the AWS DB instance and in use despite it getting removed from TF state (which is what we want)