I’m currently thinking how to deploy properly Vault in multiple environments: dev, staging, and production. Each environment has its own cluster. Here are multiple options:
- One global vault (Vault Cloud)
- One vault per environment (running in cluster)
- One global vault (Vault Cloud) and one vault per environment (running in cluster)
Many people recommend having one vault per environment. However, what about external resources (S3, DB, …) specific to this environment ? If we destroy / recreate cluster with terraform, we will lose secrets whereas we didn’t destroy external resources so I’m not sure the right place to store credentials. Here are some use cases:
- Stripe API credentials (secret shared by multiple env): If shared by staging and prod for eg, I guess it should be in global Vault ? Maybe it should be “synced” from global to local ?
- MySQL as a Service user credentials (external secret): If we store credentials in environment vault, we lose credentials every time we rebuild cluster whereas MySQL as a service is “external” to cluster. But maybe it’s what we should expect ?
- Redis operator K8S (local secret): As service only exist on environment cluster, I guess credentials should stick to environment vault
- Dev S3 bucket (external secret): Same as MySQL database, if we store credentials in Dev cluster then we lose credentials everytime we deploy with terraform. Maybe we should expect this or store in global vault ?
I didn’t find documentation on this topic. What are your advices ?