Connect agent running on vm with cluster running on k8s

Hi,

im trying to run a consul cluster in k8s and have agents running on VMs with legay applications connect to it.
Since k8s does not allow mixed protocol Loadbalancers i am not sure on how to acomplish communication between the agents and the cluster. I might use nodeports and have the agents connect against the node diretly but that seems to be a maintenance nightmare with changing nodes.
Can anyone point me towards a solution?

Thanks,
Johannes

Is this an odd szenario? I thought this would be a usual setup :slight_smile: Nobody any hints?

Hey @SpatialJ,

Thanks for this question and apologies for delayed response!

It sounds like you have a Consul server cluster on k8s and Consul client agents running outside of k8s on VMs. Without diving too deep into it, have you already seen these docs on how to connect external clients to your k8s server cluster? It recommends using k8s cloud auto-join provider, however, it only supports joining via the pod IPs, which means you must have a routable pod network and enable connectivity between your clients and the pod network on Kube. The downside is that it only discovers pod IPs at startup and won’t re-join if Kube reschedules server pods.

We currently don’t support exposing servers over host network through the Helm chart, but that could be another option. If you you find this feature useful, you could comment or :+1:.

1 Like

Thanks for your reply. I did not see the mentioned documentation, thanks for the link.
Unfortunately it describes what i suspected that one needs to use Nodeports :-/

I think it would be necessary to support exposing consul as a k8s-service to account for the dynamic nature. IPs of the Nodes should be nothing an external consul-agent should be aware of.
Is there already an issue regarding that i can upvote?

Thanks,
Johannes