I get the following error when enabling a plugin. I appreciate help:
plugin tls init: error=“error during token unwrap request: Put https://10.244.0.25:8200/v1/sys/wrapping/unwrap: x509: certificate is valid for 127.0.0.1, not 10.244.0.25” timestamp=2022-09-23T11:53:44.843Z
This is indeed a fairly complex one to understand, unless you’ve read a couple of specific areas in the Vault source code!
Somewhat surprisingly, when Vault starts up a plugin, the initial negotiation of the communications channel between Vault and the plugin, requires the plugin to make an HTTP request to the api_addr of the Vault server!
This is what is failing here.
You have set the api_addr in the configuration to the all-zeros address. Vault has responded to this by updating it to a specific IP address for the host, since the all-zeros address cannot be directly connected to.
The TLS certificate you are using does not contain an IP address Subject Alternative Name for this IP address, so when the plugin comes to attempt to make this request, the TLS negotiation fails.
To remedy this, you could:
Arrange for your TLS certificate to contain an appropriate IP SAN.
Change your api_addr to include a hostname, and include that hostname in your TLS certificate.
An IP Address Subject Alternative Name. Like the 127.0.0.1 that you have already got in your certificate.
However, since you’ve now revealed you’re in Kubernetes, using an IP address isn’t going to work, since it will change whenever the pod is rescheduled.
plugin tls init: error="error during token unwrap request: Put https://10.244.1.54:8200/v1/sys/wrapping/unwrap: x509: certificate is valid for 127.0.0.1, not 10.244.1.54"
btw, I would also try also to run with mTLS and to do that I used: tls_require_and_verify_client_cert = true option . however, then I get another error which seems to be not related to the plugin: http: TLS handshake error from 127.0.0.1:39212: tls: client didn't provide a certificate
You have not said how you are deploying Vault to Kubernetes… but if you are using the HashiCorp Vault helm chart, it sets an environment variable, making whatever you write in your config file be ignored:
So you should remove the ignored setting from your config, and set it via Helm values instead.
Thanks again for your help! I appreciate your help in configuring mutual tls. I used the following settings (adding tls_require_and_verify_client_cert to the above settings) and I get http: TLS handshake error from 127.0.0.1:59792: tls: client didn't provide a certificate error Thanks again!
OK thanks will try to use ha instead of standalone.
IIUC in k8s vault with tls will work only with ha? is there a documentation how to configure vault tls in k8s?
I used Standalone Server with TLS | Vault | HashiCorp Developer as a guide but it seems to not work on k8s. Thanks again
Thanks again for your help!!! The following configuration which enables mutual tls finally worked for me:
server:
ha:
enabled: true
replicas: 1
# Set the api_addr configuration for Vault HA
# See https://www.vaultproject.io/docs/configuration#api_addr
# If set to null, this will be set to the Pod IP Address
apiAddr: "https://vault.fybrik-system.svc.cluster.local:8200"
# Set the cluster_addr confuguration for Vault HA
# See https://www.vaultproject.io/docs/configuration#cluster_addr
# If set to null, this will be set to https://$(HOSTNAME).{{ template "vault.fullname" . }}-internal:8201
clusterAddr: "https://vault.fybrik-system.svc.cluster.local:8201"
config: |
plugin_directory = "/usr/local/libexec/vault"
ui = true
listener "tcp" {
address = "[::]:8200"
cluster_address = "[::]:8201"
tls_disable = false
tls_cert_file = "/vault/userconfig/vault-server-tls/tls.crt"
tls_key_file = "/vault/userconfig/vault-server-tls/tls.key"
tls_client_ca_file = "/vault/userconfig/vault-server-tls/ca.crt"
tls_require_and_verify_client_cert = true
}
storage "file" {
path = "/vault/data"
}
disable_mlock = true
Another question: what the difference between tls_disable_client_certsand tls_require_and_verify_client_cert
Many thanks!