Consul-template - mergeMap using values from parseYAML

Consul
1

Hi all.

Following the example of:

{{ (tree "env/defaults" | explode)
   | mergeMap (tree "env/my-env" | explode)
   | mergeMap (tree "env/my-env/my-service" | explode)
   | toYAML }}

I would like to create a merge of yaml docs.

e.g.

{{ (key "env/defaults" | parseYAML)
   | mergeMap (key "env/my-env" | parseYAML )
   | mergeMap (key "env/my-env/my-service" | parseYAML)
   | toYAML }}

i.e - env/defaults and env/my-env and env/my-env/my-service are all valid YAML documents which I expect to merge recursively.

However - I get this error:

wrong type for value; expected map[string]interface {}; got map[interface {}]interface {}

Help?

I guess it’s around that parseYAML and tree or ls do not return the same thing - how can I bridge this gap?

EDITED:
here is the simplest reproduction example I could come with:

merged:
{{ print "foo: overriden\nbaz: taz" | parseYAML 
  | mergeMap (print "foo: bar\nbak: rak" | parseYAML ) 
  | toYAML | indent 2 }}

instead of the error - we should expect something like:

merged:
  bak: rak
  baz: taz
  foo: bar

I tried explode, explodeMap, but I’m not seem to be in the right direction…

2

eh… ehum… nobody here?

3

The problem is that parseYAML returns Go maps where the map keys can be of varying data types, whilst most of the rest of the consul-template functions that work with maps expect to work with Go maps where the keys are restricted to being strings only.

This typing difference is determined statically when the map is being created, so it doesn’t matter whether the keys in the YAML just happen to all be strings - it’s still a type difference that the Go type system will error on.

Ultimately, consul-template is pretty far away from being a complete programming language, so there’s no way to solve this with the built-in functionality of the language today.

I can see two/three options:

  • Attempt to use consul-template’s plugin support to work around this … I’m not convinced this is a good option since plugin input uses command line parameters - so this would imply passing whole YAML documents in command line parameters … it feels like a quoting mishap or length limitation would mess things up sooner or later. Also the security implications of exposing input to all processes executing on the same OS.

  • Add a new function to the consul-template project, and build a custom version (and maybe try to get it merged upstream)

    • Perhaps parseYAMLAssumingStringKeysOnly

    • Or convertMapHavingOnlyStringKeys