Hey all.
I’ve recently tried to implement a multi-region set up for vault on kubernetes.
I used a global table for dynamodb, encrypted unseal keys in both region’s s3 bucket encrypted with respective region’s kms keys.
I thought I was a genius, hacking my way into a multi-region vault set up. I believe it worked for like a month up till the point of when I turned on “disable_cache” since secrets were stored inside the pod until it’s forced to read. I then hit this point.
time="2020-05-06T00:57:56Z" level=info msg="Failed to request new Vault token Error making API request.\n\nURL: PUT http://vault.vault:8200/v1/auth/kubernetes/login\nCode: 500. Errors:\n\n* Post https://172.20.0.1/apis/authentication.k8s.io/v1/tokenreviews: x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"kubernetes\")"
time=“2020-05-06T00:57:58Z” level=fatal msg=“failed to create vault clienttimeout [10s] during waiting for Vault token” app=vault-env
I think what happened was, both vaults were simultaneously overwriting each other’s K8 certificate authority in dynamodb. Who knows what else it was overwriting… I just wanted to know if my approach is possible to do? Or am I subjected to stay one region.
Thanks for your time.