Vault (raft) not reaching out to each other

Vault
,
1

Hello. I am trying to setup a Vault cluster using integrated-storage feature, meaning using RAFT as the backend. I have 3 machines in same network (running Ubuntu 20.04). Lets assume they have the following name:

  1. r1.gs.com
  2. r2.gs.com
  3. r3.gs.com

My config in 1st machine is as follows:

cluster_addr  = "r1.gs.com:8201"
api_addr      = "r1.gs.com:8200"
disable_mlock = true

listener "tcp" {
  address            = "0.0.0.0:8200"
  cluster_address    = "r1.gs.com:8201"
  tls_cert_file      = "/opt/vault/tls/server.crt"
  tls_key_file       = "/opt/vault/tls/server.key"
}

storage "raft" {
  path    = "/opt/vault/data"
  node_id = "1"

  retry_join {
    leader_api_addr         = "r2.gs.com:8200"
    leader_client_cert_file = "/opt/vault/tls/server.crt"
    leader_client_key_file  = "/opt/vault/tls/server.key"
    leader_ca_cert_file     = "/opt/vault/tls/ca.crt"
  }
  retry_join {
    leader_api_addr         = "r3.gs.com:8200"
    leader_client_cert_file = "/opt/vault/tls/server.crt"
    leader_client_key_file  = "/opt/vault/tls/server.key"
    leader_ca_cert_file     = "/opt/vault/tls/ca.crt"
  }
}

ui = true

My config in 2nd machine is as follows:

cluster_addr  = "r2.gs.com:8201"
api_addr      = "r2.gs.com:8200"
disable_mlock = true

listener "tcp" {
  address            = "0.0.0.0:8200"
  cluster_address    = "r2.gs.com:8201"
  tls_cert_file      = "/opt/vault/tls/server.crt"
  tls_key_file       = "/opt/vault/tls/server.key"
}

storage "raft" {
  path    = "/opt/vault/data"
  node_id = "1"

  retry_join {
    leader_api_addr         = "r1.gs.com:8200"
    leader_client_cert_file = "/opt/vault/tls/server.crt"
    leader_client_key_file  = "/opt/vault/tls/server.key"
    leader_ca_cert_file     = "/opt/vault/tls/ca.crt"
  }
  retry_join {
    leader_api_addr         = "r3.gs.com:8200"
    leader_client_cert_file = "/opt/vault/tls/server.crt"
    leader_client_key_file  = "/opt/vault/tls/server.key"
    leader_ca_cert_file     = "/opt/vault/tls/ca.crt"
  }
}

ui = true

My config in 3rd machine is as follows:

cluster_addr  = "r3.gs.com:8201"
api_addr      = "r3.gs.com:8200"
disable_mlock = true

listener "tcp" {
  address            = "0.0.0.0:8200"
  cluster_address    = "r3.gs.com:8201"
  tls_cert_file      = "/opt/vault/tls/server.crt"
  tls_key_file       = "/opt/vault/tls/server.key"
}

storage "raft" {
  path    = "/opt/vault/data"
  node_id = "1"

  retry_join {
    leader_api_addr         = "r1.gs.com:8200"
    leader_client_cert_file = "/opt/vault/tls/server.crt"
    leader_client_key_file  = "/opt/vault/tls/server.key"
    leader_ca_cert_file     = "/opt/vault/tls/ca.crt"
  }
  retry_join {
    leader_api_addr         = "r2.gs.com:8200"
    leader_client_cert_file = "/opt/vault/tls/server.crt"
    leader_client_key_file  = "/opt/vault/tls/server.key"
    leader_ca_cert_file     = "/opt/vault/tls/ca.crt"
  }
}

ui = true

My vault service starts up, and I unseal each machine, and each machine becomes the Leader. They are not talking to each other.

TLS is enabled. I am using a private ca.crt. I generated the server.crt and server.key files myself. If I give a command to r2.gs.com to join the raft on r1.gs.com, it returns successful, but the peer list on r1.gs.com doesn’t show it.

Any idea where am I wrong?

2

First, you must add the ‘https’ in all your leader_api_addr, cluster_addr, and api_addr
ex, leader_api_addr = “https://r2.gs.com:8200”