I’m trying to setup a Vault 1.6.0 HA cluster with Internal Storage. I use a Helm Chart to deploy it on a K8S cluster. But when I deploy the Helm Chart it makes 5 different clusters instead of One!
NAME READY STATUS RESTARTS AGE
vault-0 1/1 Running 0 13m
vault-1 1/1 Running 0 13m
vault-2 1/1 Running 0 13m
vault-3 1/1 Running 0 13m
vault-4 1/1 Running 0 13m
vault-agent-injector-6f76cf6-vls9j 1/1 Running 0 13m
[useradmin@pc-001]$ kubectl -n vault exec -it vault-0 vault operator raft list-peers
Node Address State Voter
vault-0 vault-0.vault-internal:8201 leader true
[useradmin@pc-001]$ kubectl -n vault exec -it vault-1 vault operator raft list-peers
Node Address State Voter
vault-1 vault-1.vault-internal:8201 leader true
[useradmin@pc-001]$ kubectl -n vault exec -it vault-2 vault operator raft list-peers
Node Address State Voter
vault-2 vault-2.vault-internal:8201 leader true
[useradmin@pc-001]$ kubectl -n vault exec -it vault-3 vault operator raft list-peers
Node Address State Voter
vault-3 vault-3.vault-internal:8201 leader true
[useradmin@pc-001]$ kubectl -n vault exec -it vault-4 vault operator raft list-peers
Node Address State Voter
vault-4 vault-4.vault-internal:8201 leader true
[useradmin@pc-001]$ kubectl -n vault exec -it vault-0 vault status
Key Value
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.6.0
Storage Type raft
Cluster Name vault-cluster-dd3a541d
Cluster ID a142a180-056e-030b-e73a-79bb3878cac8
HA Enabled true
HA Cluster https://vault-0.vault-internal:8201
HA Mode active
Raft Committed Index 53
Raft Applied Index 53
[useradmin@pc-001]$ kubectl -n vault exec -it vault-1 vault status
Key Value
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.6.0
Storage Type raft
Cluster Name vault-cluster-a03e36d0
Cluster ID 041efc7e-0cd2-ac37-72b4-6334d176720f
HA Enabled true
HA Cluster https://vault-1.vault-internal:8201
HA Mode active
Raft Committed Index 49
Raft Applied Index 49
[useradmin@pc-001]$ kubectl -n vault exec -it vault-2 vault status
Key Value
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.6.0
Storage Type raft
Cluster Name vault-cluster-5c34e2b9
Cluster ID 30ac93eb-3cb2-805a-29a8-42484668151a
HA Enabled true
HA Cluster https://vault-2.vault-internal:8201
HA Mode active
Raft Committed Index 42
Raft Applied Index 42
[useradmin@pc-001]$ kubectl -n vault exec -it vault-3 vault status
Key Value
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.6.0
Storage Type raft
Cluster Name vault-cluster-01b5e4c4
Cluster ID 61187cff-6589-3480-8c35-1c8ac32aad2f
HA Enabled true
HA Cluster https://vault-3.vault-internal:8201
HA Mode active
Raft Committed Index 36
Raft Applied Index 36
[useradmin@pc-001]$ kubectl -n vault exec -it vault-4 vault status
Key Value
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.6.0
Storage Type raft
Cluster Name vault-cluster-2afafb9a
Cluster ID 6efc9e19-a89d-f661-88af-942105c7f2de
HA Enabled true
HA Cluster https://vault-4.vault-internal:8201
HA Mode active
Raft Committed Index 34
Raft Applied Index 34
This is a snippet of my helmchart:
Run Vault in “HA” mode. There are no storage requirements unless audit log
persistence is required. In HA mode Vault will configure itself to use Consul
for its storage backend. The default configuration provided will work the Consul
Helm project by default. It is possible to manually configure Vault to use a
different HA backend.
ha:
enabled: true
replicas: 5
# Set the api_addr configuration for Vault HA
# See https://www.vaultproject.io/docs/configuration#api_addr
# If set to null, this will be set to the Pod IP Address
# apiAddr: null
# Enables Vault's integrated Raft storage. Unlike the typical HA modes where
# Vault's persistence is external (such as Consul), enabling Raft mode will create
# persistent volumes for Vault to store data according to the configuration under server.dataStorage.
# The Vault cluster will coordinate leader elections and failovers internally.
raft:
# Enables Raft integrated storage
enabled: true
# Set the Node Raft ID to the name of the pod
setNodeId: true
# Note: Configuration files are stored in ConfigMaps so sensitive data
# such as passwords should be either mounted through extraSecretEnvironmentVars
# or through a Kube secret. For more information see:
# https://www.vaultproject.io/docs/platform/k8s/helm/run#protecting-sensitive-vault-configurations
config: |
disable_mlock = true
ui = true
listener "tcp" {
tls_disable = 0
address = "[::]:8200"
cluster_address = "0.0.0.0:8201"
#tls_client_ca_file = "/vault/userconfig/vault-tls/ca/crt"
tls_cert_file = "/vault/userconfig/vault-tls/tls.crt"
tls_key_file = "/vault/userconfig/vault-tls/tls.key"
}
storage "raft" {
path = "/vault/data"
retry_join {
leader_api_addr = "https://vault-0.vault-internal:8200"
#leader_ca_cert_file = "/vault/userconfig/vault-tls/ca.crt"
leader_client_cert_file = "/vault/userconfig/vault-tls/tls.crt"
leader_client_key_file = "/vault/userconfig/vault-tls/tls.key"
}
retry_join {
leader_api_addr = "https://vault-1.vault-internal:8200"
#leader_ca_cert_file = "/vault/userconfig/vault-tls/ca.crt"
leader_client_cert_file = "/vault/userconfig/vault-tls/tls.crt"
leader_client_key_file = "/vault/userconfig/vault-tls/tls.key"
}
retry_join {
leader_api_addr = "https://vault-2.vault-internal:8200"
#leader_ca_cert_file = "/vault/userconfig/vault-tls/ca.crt"
leader_client_cert_file = "/vault/userconfig/vault-tls/tls.crt"
leader_client_key_file = "/vault/userconfig/vault-tls/tls.key"
}
retry_join {
leader_api_addr = "https://vault-3.vault-internal:8200"
#leader_ca_cert_file = "/vault/userconfig/vault-tls/ca.crt"
leader_client_cert_file = "/vault/userconfig/vault-tls/tls.crt"
leader_client_key_file = "/vault/userconfig/vault-tls/tls.key"
}
retry_join {
leader_api_addr = "https://vault-4.vault-internal:8200"
#leader_ca_cert_file = "/vault/userconfig/vault-tls/ca.crt"
leader_client_cert_file = "/vault/userconfig/vault-tls/tls.crt"
leader_client_key_file = "/vault/userconfig/vault-tls/tls.key"
}
}
service_registration "kubernetes" {}
# config is a raw string of default configuration when using a Stateful
# deployment. Default is to use a Consul for its HA storage backend.
# This should be HCL.
# Note: Configuration files are stored in ConfigMaps so sensitive data
# such as passwords should be either mounted through extraSecretEnvironmentVars
# or through a Kube secret. For more information see:
# https://www.vaultproject.io/docs/platform/k8s/helm/run#protecting-sensitive-vault-configurations
# config: null
# A disruption budget limits the number of pods of a replicated application
# that are down simultaneously from voluntary disruptions
disruptionBudget:
enabled: true
# maxUnavailable will default to (n/2)-1 where n is the number of
# replicas. If you'd like a custom value, you can specify an override here.
maxUnavailable: null
Definition of the serviceAccount used to run Vault.
These options are also used when using an external Vault server to validate
Kubernetes tokens.
serviceAccount:
# Specifies whether a service account should be created
create: true
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
# Extra annotations for the serviceAccount definition. This can either be
# YAML or a YAML-formatted multi-line templated string map of the
# annotations to apply to the serviceAccount.
annotations: {}
Settings for the statefulSet used to run Vault.
statefulSet:
# Extra annotations for the statefulSet. This can either be YAML or a
# YAML-formatted multi-line templated string map of the annotations to apply
# to the statefulSet.
annotations: {}