Hello together,
I have a small nomad bare metal cluster up and running with exec,docker and qemu.
I am currently looking around to find a good networking solution, especially with regards to VMs started via qemu.
Each nomad client host has 2 network adapters where one is meant for administration tasks (ssh, ansible, connection to nomad servers …) whereas the other interface is meant for the guests (docker, qemu).
Both interfaces are in different subnets and each interface is configured with it’s own DHCP server and therefore with it’s own gateway. The problem at this point is that both gateways (netif1, netif2) can reach overlapping other subnets.
So the question is how can I isolate the netif2 (for guests) in a way that nomad knows the ip/forwarded port of an instance, but also ensure that the guest do not make use of the hosts routing table or in a way that guest traffic never flows through netif1?
It might be that what I am trying does not really make sense, so I am open to any other suggestions on how to deal with that. My main goal was to fully isolate host and guest traffic.
Thanks in advance!