I have been trying to get Consul and Envoy to talk to each other, specifically setting up the dynamic CDS to provide me with the third party endpoints.
Currently I have Consul containing details of these endpoints and I’ve been looking at the go-control-plane example in the Envoy code to convert these into a set of clusters, however I came across Consul Connect and this felt like a much cleaner way to achieve this.
I know will have to register the services as third party ones but that looks to be straight foward. The bit I’m stuggling with is how to setup Envoy and Consul configuration so that Envoy will use Consul as a CDS at all. I have attempted to follow the step by step instructions for but my knowleadge of Consul is not great and I don’t appear to be able to get any of the tutorials I’ve seen to work.
Can someone point me at an ‘Explain Like I’m 5’ level of walkthrough that will get me started. At the very least I need help to explain how to tell Consul to provide CDS endpoint for Envoy.
I think I’ve managed to get Consul to start with xds enabled (apparently that was as easy as setting the grpc port to a value) and the external service registartion thing also appears to work, it tells me it won’t be able to result the dns but that isn’t something I care about right now.
However envoy, upon starting and attempting to connect with Consul I get told that “Cluster not available” and the connection is closed
Looks like the issues it that I don’t know how to tell consul to expose the xDS service, just setting the port in the configuration wasn’t enough, there isn’t anything listening.
Could you clarify what you mean by third-party endpoints? Are these services which are external to your data center, or services which are internal to the DC but are running on hosts which do not have a Consul client installed?
It sounds like you’re setting up Consul outside of Kubernetes (please correct me if I’m wrong). If so, the configuration files in this tutorial might be a good resource in helping you understand how to setup and configure Consul service mesh for the first time: Getting Started with Consul Service Mesh for Virtual Machines.
Consul was really designed to be a complete control plane for Envoy proxies, providing all of the CDS, RDS, EDS, etc info (via ADS) for proxies in a service mesh. I’m not aware of anyone who has attempted to just consume CDS info from Consul, so I’m not certain whether you’ll run into additional issues as you attempt this configuration.
I’d like to better understand the problem you’re trying to solve, and desired setup so that I can help you determine whether there’s a better way to architect this given Consul’s native feature set.
These are services that are running external to the data center where we run envoy and consul. For these I was/am planning to use consul to provide configuration for envoy to connect upstream to but they would be registered by another service rather than being part of a mesh.
The use case here is that I have a number of connection being routed by envoy to these upstream services as needed, currently I’m serving this with a static configuration but that is only useful in the short term. CDS was suggested as the solution to this problem on the envoy slack and as I am already using Consul for the majority of the shared configuration it seemed like it would be a good choice.
I do appear to be running into a number of issues though, once I finally managed to get what I think is Consul providing a grpc service I ran into errors with the fact I was using Consul 1.9 and envoy 1.18.2, I have since upgraded to the 1.10 beta and it’s giving me not implemented status 2 responses. I’m assuming I need to dig through the configuration to work out what I have failed to specify. However this is progress as at least they are talking…
I’m not sure if you’re aware, but Consul provides native support for connecting to egress services through proxies called terminating gateways. Have you considered whether this gateway will work for your use case?
I was not aware of this and it might be helpful, that said it appears that it would just act as a node for services to route through to a known target, returning the configuration data to a given envoy node based on what services that sidecar has registered as upstream.
In my case the finally routing destination is caculated by the envoy filter and not the service calling it, similar to the way a load balancer will forward your connection to one of many endpoints but with more internal logic that just balancing. This it turn means it needs a cluster list that has all the registered external services.
I have only just noticed that Consul returns the configuration based on the cluster and id set in the envoy configuration. This might be why I was having so much trouble.
I am having similar problem as yours. I am trying to use consul to provide control plane for envoy.
I have been stuck with ‘streamAggregatedResources gRPC config stream closed: 14, upstream connect error or disconnect/reset before headers. reset reason: connection failure’ from envoy.
Would it be possible for you to share your envoy config file just to see what I am missing ?
I got that error before I switch to the 1.10 beta version of consul, I’m assuming you have the 1.8.2 version of envoy as well. One of the other import things to note is that these talk over DELTA_GRPC rather than GRPC in the envoy configuration.
I should note however that I couldn’t get Consul to do what I wanted so I’m now using the go-control-plane example from envoy hooked up to Etcd instead, I only really needed the key/value store part when using that control plane.