How to import k/v pairs from a file

Consul
1

I want to use files to import key value pairs into a folder in Consul.

e.g.
I have a config folder in consul k/v tab.
I have a file (json/txt) with key/value pairs separated by colon
[{key : value}, {key, value}…]
I import file into config folder
consul kv import(or put) config/ @file.json
expected outcome
config folder now has all the key/ value pairs from the file separated into their own key/ value pairs.

I understand that I might not be explaining perfectly, so feel free to ask any questions. If there is already a way to do it with consul alone, please let me know. I know there are 3rd party libraries (git2consul etc) that can do that, but I want to know if there is a best way to do that (preferably not using a 3rd party library, but if that is the only way then that is fine).

2

You can do something like you want with a shell script using the jq utility to parse the json.

With a simple example json file…

$ cat in.json
[{"k1":"v1"},{"k2":"v2"},{"k3":"v3"}]

You’d import the data into consul something like…

$ cat in.json | jq -r '.[] | to_entries[] | "config/"+.key+ " "+ .value' | xargs consul kv put
Success! Data written to: config/k1
Success! Data written to: config/k2
Success! Data written to: config/k3
$ consul kv get config/k1
v1
...

Hope this helps.

1 Like
3

I remember seeing that somewhere. I was hoping that we can do it without a third party library, but I had a feeling that won’t be possible. Thank you for your help!

4

Here’s a slightly different way to do this using consul kv import.

$ cat in.json | jq -r '[.[] | to_entries[] | .["key"] = "config/"+.key | .["value"] = (.value | @base64)]' | consul kv import -
Imported: config/k1
Imported: config/k2
Imported: config/k3
$ consul kv get config/k1
v1
...

This might be more efficient than calling consul kv put for each entry.