Consul-template how to query value from specific key of service meta

Consul
#1

Hi,
I’m trying to find a way how to query a specific key of my service meta.
Let’s say I have a simple ctmpl:

{{- range services -}}
  {{- if and (.Name | contains "sidecar" | not) (or (.Tags | contains "development") (.Tags | contains "azure")) }}
      {{ range service .Name }}
      {{.Name}}:{{.ServiceMeta}}
    {{- end -}}
  {{- end -}}
{{- end -}}

The above template produces the following output:

      nginx:map[]

      static-client:map[nginx-enabled:true pod-name:static-client]

      static-server:map[path:staticserver pod-name:static-server-68898fbff4-lcrp4 nginx-enabled:true]
      static-server:map[path:staticserver pod-name:static-server-68898fbff4-vsmbm nginx-enabled:true]
      static-server:map[nginx-enabled:true path:staticserver pod-name:static-server-68898fbff4-xzkxk]

And here I’m have two questions:

  • How to query only a value of nginx-enabled key
  • How to avoid a repeating service name. If service in catalog has more than one instance then api function service returns all of instances, but for my case I only need to get one. This probably can be somehow figured out using index by taking first element in array, but unfortunately i still not able to construct a working template.

So as result I would like to see something like:

static-client: true
static-server: true

Thank you in advance.

#2

@andriktr,

You can use the following template to print the names of pods which have the key nginx-enabled defined in the service metadata, and the corresponding value of that key.

{{- $allowedTags := parseJSON `["azure","development"]` -}}
{{ range services }}
  {{- if and (.Name | contains "sidecar" | not) (containsAny $allowedTags .Tags) }}
    {{- range service .Name }}
      {{- $nginxEnabled := (index .ServiceMeta "nginx-enabled") -}}
      {{- if $nginxEnabled -}}
        {{ printf "%s:%s\n" .Name $nginxEnabled -}}
       {{- end -}}
    {{- end -}}
  {{- end -}}
{{ end }}

I have a question about your desire to not repeat the service name. Say you have two pods with the nginx-enabled metadata. One has that key set of a value of true and the other false. Is that a situation which might be encountered in your environment? If so, how should the template handle that scenario?

1 Like
#3

@blake Thanks for response.

Say you have two pods with the nginx-enabled metadata. One has that key set of a value of true and the other false . Is that a situation which might be encountered in your environment?

This is not expected in our case.

Actually what we try to achieve is to dynamically create ngnix.conf file which is based on NGINX Plus and Consul integration.

In general to achieve this we need few things:

  • First we need a decision is it a nginx enabled service (this might be achieved by searching specific tag/tags like ngninx-enabled or by specific meta key value)
  • We also need a service name (and we need it only once)
  • And the last one is we need to pass a path for location block and the idea here was to use service meta. For example we set service meta path=consultest then the value from this meta goes to location block in nginx.conf.

Here is a short example:

server {
   listen 80;

   location /test { # `test` should come from service meta key value
      proxy_pass http://consultest; # `consultest` comes from service name
      health_check;
   }
}

upstream web { 
  zone upstream_consultest 128k; # `consultest` comes from service name
  server service.consul service=consultest resolve; # `consultest` comes from service name
}
resolver 127.0.0.1:8600 valid=5s;
resolver_timeout 2s;

Thanks in advance

#4

@andriktr,

That makes sense. This is an interesting use case. :slight_smile:

I put together this template based on what you described. You may have a tweak it a bit to work in your environment, but I believe it gets close to what you want.

{{- $allowedTags := parseJSON `["azure","development","k8s"]` -}}
{{- range services -}}
  {{- if and (.Name | contains "sidecar" | not) (containsAny $allowedTags .Tags) -}}

    {{- /* Group services by the metadata 'nginx-enabled' */ -}}
    {{- $groupedServices := (service .Name | byMeta "nginx-enabled") -}}

    {{- /* Map also contains services not matching this metadata.
        Fetch only services with 'nginx-enabled' set to 'true'. */ -}}
    {{- $nginxEnabledSvc := (index $groupedServices "true") -}}

    {{- range $nginxEnabledSvc -}}
      {{- /* Fetch path key for service */ -}}
      {{- $path := index .ServiceMeta "path" -}}

      {{- if $path -}}
        {{- /* If path exists, set <path>=<service name> in paths map
            only if no previous entry exists */ -}}
        {{- scratch.MapSetX "paths" $path .Name -}}
      {{- end -}}

    {{- end -}}
  {{- end -}}
{{- end -}}

server {
  listen 80;

{{ range $path, $service := scratch.Get "paths" }}
  location /{{ $path }} {
    proxy_pass http://{{ $service }};
    health_check;
  }
{{ end }}
}

{{ range $index, $service := scratch.MapValues "paths" }}
upstream {{ $service }} {
  zone upstream_{{ $service }} 128k;
  server service.consul service={{ $service }} resolve;
}
{{ end }}

resolver 127.0.0.1:8600 valid=5s;
resolver_timeout 2s;
3 Likes
#5

Wow, @blake. Thank you very much for your help.
I really appreciate it :slight_smile:

#6

Hey @blake,
I have one additional question regarding template you have provided. How it would be possible to additional add an if statement according which if meta key path is not defined for service then location directive should stay unchanged like

 location / {
    proxy_pass http://{{ $service }};
    health_check;
  }
{{ end }}

Thank you.

#7

Hi @andriktr,

Is there use case where multiple services have this as undefined, or would there only be one service like this which hosts the root endpoint? If its the latter, you could simply change your service registrations so that the path’s contain the leading slash (e.g., path="/users") and then remove the leading slash from the location block in the template.