How to extract only a specific key from Vault secret / not save unrelated key in TF state

Terraform Providers
1

I am disturbed to find that if I want to use a specific key from a secret (say an SSH public_key field) Terraform will store the private key (which is not used) in the Terraform state file.

data "vault_generic_secret" "ssh_key" {
  path = "secret/ssh/${var.key_name}"
}

resource "something" "else" {
  public_key = data.vault_generic_secret.ssh_key.data["public_key"]
....

I’ve looked at the kv_secret_v2 resource and it doesn’t provide an option to retrieve only a single key either. Is there any mechanism to avoid doing this, other than by putting the public and private keys in different Vault secrets?

2

Hi @jorhett,

I am admittedly not a Vault expert and so hopefully someone else will contribute if they have more detail on this, but my sense is that Vault’s design is that information with different security properties should typically be stored at separate paths in Vault so that e.g. Vault’s ACL system can apply different rules to each.

Given that, my first idea here would’ve been the very thing you suggested you don’t want to do: putting the public and private keys at separate paths in Vault.

I don’t know of any way to avoid the behavior you saw because from Vault’s standpoint both the public key and the private key are grouped together as a single object with how you currently have it populated, and therefore always retrieved together due to how the underlying API is designed.

1 Like
3

Thanks for the reply @apparentlymart. While I do understand what you mean, there’s a difference between access policy (which you are correct about) and unintended leakage. Let me explain.

For example, only my team can access a given SSH key. There’s neither need nor desire to give anyone else any kind of access. Thus Vault configuration is as expected and works as desired.

Vault absolutely allows for retrieval of singular key/value pairs of a secret. So I can retrieve just one key and provide it to Ansible, Puppet, (name your tool of choice) without giving them every key.

When I go to retrieve the public key from this secret, I am willfully choosing to allow this information to be stored in Terraform state (as it must be). That does not present a problem.

What I am not doing is intending to expose the private key of this secret. This is a side effect of the data provider. It is not necessary for Terraform, nor does it provide value for Terraform. This is unintended leakage, not a design decision by Vault.

I grasp the workaround you suggested: we could go split up every key of each secret into separate secrets to work around this limitation. But this isn’t a Vault design principle, this doesn’t serve any of Vault’s design goals or policy frameworks. As a matter of point, denying use of multiple key/value pairs in a secret is fighting against Vault’s design goals.

This is really, truly, simply an unintended leakage problem in the current data provider.

4

I think that’s a bit of a contrived way to look at it.

From the data provider’s perspective, the whole secret is being accessed:

It’s only elsewhere in your Terraform code, that you choose what keys to access:

You say that:

except it doesn’t … it’s all or nothing. Sure, you can throw away some of the data after you retrieve it, but that’s not the same thing.

You also say that:

but to me, the fact that Vault’s ACL policy layer is defined solely on the path of secrets, and doesn’t allow restricting access to subkeys within a secret differently, strongly suggests that it is a Vault design principle that data of different confidentiality levels should be stored in separate secrets.

Ultimately, though, there is a core point we do agree upon: non-Terraform Vault workflows make it easy to discard retrieved data that you do not wish to persist, but terraform-provider-vault lacks this capability. That would probably be a fine thing to raise a GitHub issue, or even a PR, about.

Meanwhile, @apparentlymart already described the best approach available today.

5

Yes, retaining data that you don’t need and writing it out in plaintext on local and remote storage is exactly the definition of data leakage

But it isn’t data of a different confidentiality level. There’s no difference in the confidentiality level of this data. If there was a confidentiality level difference, it would make perfect sense to separate the secrets.

The issue is that the data provider leaks data that is not used by Terraform. And that the only way to prevent that, is to separate inseparably-related keys of the same confidentiality level. In short, design Vault for Terraform rather than design by Vault guidance policy.

This would also require that some lookups in question for us with than just pub/private keys would become 40-60 separate secrets, requiring that same number of distinct Vault GET requests for apps that require every key, only so that Terraform can retrieve a small handful of them without exposing all of them.

6

Issue created Don't leak Vault keys not necessary to create resources · Issue #1555 · hashicorp/terraform-provider-vault · GitHub