Backup old consul (v0.6.4)

Consul
1

Hello there! I have an old vault and consul as a storage backend cluster that I need to backup and restore on a newer one.

Every Consul documentation about this topic suggests the use of snapshot subcommand, but this consul version doesn’t have such command.

Which other approach should I take?

Vault version: v0.5.2
Consul version: v0.6.4

Thanks!

2

I’d setup a new Vault/Consul cluster and use a Python script (or whatever language you prefer) and the HTTP API to read all keys from Consul and all secrets from Vault and copy them over to the new cluster. Depending on the amount of data, this can get a bit challenging, I know. :wink:

This also is a good opportunity to make sure all ACLs, secret engine configurations, roles, etc. can be applied in an automated way (e.g. using Terraform, Ansible, etc.).

1 Like
3

Thank you for your answer, I’ll give it a try!

4

You may also be able to use consul-backinator to assist with this backup and restore process.

I’ve never used this myself, and its not officially endorsed by HashiCorp, but it might be worth trying before you go down the route of writing your own KV backup client.

3 Likes
5

Thank you, blake. I should receive all keys that I have on this consul cluster using the following request, correct?

curl "http://<IP>:<port>/v1/kv/?keys=true"

But it returns an empty list. Now I have doubts if the vault cluster really uses the consul as storage backend, the vault.hcl file has a slightly different configuration from what I commonly used to:

backend "consul" {
 address = "127.0.0.1:8500"
 path = "vault"
 scheme = "http"
       token = "<token>"
       advertise_addr = "https://vault-2:8200"
}

listener "tcp" {
 address = "0.0.0.0:8200"
 tls_cert_file = "vault.cert"
 tls_key_file = "vault.key"
}

Can someone confirm that this vault cluster really uses consul as storage backend: :smiley:
Thanks!

6

Are ACLs enabled in your Consul environment? If so, you’ll want to issue the query with an token that has privileges to read the KV paths that you are trying to export. You can then provide that token in the API request using one of the supported authentication headers or as a query argument (https://www.consul.io/api-docs#authentication). For example:

curl --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" "$CONSUL_HTTP_ADDR/v1/kv/?keys"

That command will return an array of key names which are stored in Consul. If you want to include the values, you’ll want to just use the ?recurse query argument.

The process of configuring storage backends in Vault has changed in recent releases. However, the config you shared was the correct way to configure Consul as a storage backend in Vault version 0.5.2 (see https://github.com/hashicorp/vault/blob/v0.5.2/website/source/docs/config/index.html.md.

7

Thank you Blake. I set the consul http token and consul http addr correctly, but I didn’t get the list of keys.

I took another approach and was able to back up the files using the vault backend migrator tool.

I appreciate all the help :slight_smile: