Connect agent running on vm with cluster running on k8s

Consul
1

Hi,

im trying to run a consul cluster in k8s and have agents running on VMs with legay applications connect to it.
Since k8s does not allow mixed protocol Loadbalancers i am not sure on how to acomplish communication between the agents and the cluster. I might use nodeports and have the agents connect against the node diretly but that seems to be a maintenance nightmare with changing nodes.
Can anyone point me towards a solution?

Thanks,
Johannes

2

Is this an odd szenario? I thought this would be a usual setup :slight_smile: Nobody any hints?

3

Hey @SpatialJ,

Thanks for this question and apologies for delayed response!

It sounds like you have a Consul server cluster on k8s and Consul client agents running outside of k8s on VMs. Without diving too deep into it, have you already seen these docs on how to connect external clients to your k8s server cluster? It recommends using k8s cloud auto-join provider, however, it only supports joining via the pod IPs, which means you must have a routable pod network and enable connectivity between your clients and the pod network on Kube. The downside is that it only discovers pod IPs at startup and won’t re-join if Kube reschedules server pods.

We currently don’t support exposing servers over host network through the Helm chart, but that could be another option. If you you find this feature useful, you could comment or :+1:.

1 Like
4

Thanks for your reply. I did not see the mentioned documentation, thanks for the link.
Unfortunately it describes what i suspected that one needs to use Nodeports :-/

I think it would be necessary to support exposing consul as a k8s-service to account for the dynamic nature. IPs of the Nodes should be nothing an external consul-agent should be aware of.
Is there already an issue regarding that i can upvote?

Thanks,
Johannes

5

Hi Johannes,
Consul actually keeps the set of node IPs in sync through gossip so if they change it’s not a problem.

6

Well, i have to configure some IP/Hostname in my config.json like, don’t i?
{
“server”: false,
“start_join”: [“10.210.8.101”]
}

How would the agent connect to the cluster on startup when 10.210.8.101 is gone?
Could i use Cloud Auto-join in my szenario?

Thanks,
Johannes