Consul Connect dynamic upstreams

Hi,

We have an application split up in several microservices.

  1. One “central” microservice must be able to realtime discover all microservices filtered by a specific tag (let’s say a tag “plugin”).

  2. Then this “central” microservice must be able to call (http) these “plugin” microservices when needed

For the “discovery”, no problem; we’re using the Consul registry/discovery API and it’s perfectly working.

For the “calls”, we want to use Consul Connect and route the calls through the Mesh using Envoy.

Currently, if we want the “central” microservice to be able to call a “plugin” microservice, we need to:

  • update its Proxy configuration to add the “plugins” upstreams
  • update the app conf to indicate that plugin x is reachable using the envoy port xx

It’ absolutely not dynamic… :frowning_face:

Our goal is to deploy plugins without any action on the Central Service

Below a simplified example of our issue:

{
  "ID": "app-central-1-proxy",
  "Name": "app-central-proxy",
  "Kind": "connect-proxy",
  "Proxy": {
    "DestinationServiceName": "app-central",
    "DestinationServiceId": "app-central-1",
    "LocalServicePort": 7000,
    "Upstreams": [
      {
        "DestinationType": "service",
        "DestinationName": "app-plugin-x",
        "LocalBindPort": 8001
      },
      {
        "DestinationType": "service",
        "DestinationName": "app-plugin-y",
        "LocalBindPort": 8002
      },
      {
        "DestinationType": "service",
        "DestinationName": "app-plugin-z",
        "LocalBindPort": 8003
      },
      {
       ...
      }
    ]
  },
  "Port": 21000
}

This kind of use-case looks pretty common; what’s the best way to deal with?