I’ve been looking at migrating our current (see below) Microservices to Docker, and quit Kubernetes 4 times already. Nomad FTW!
I’m having a hard time wrapping my head around a (probably) simple architectural/structural change. I’m hoping that someone with more battle-experience than me might offer some wisdom.
Currently we have 28 separate “services”. Each service is written in Python, and uses Nameko for RPC between the services. There is a “HTTP” service that acts as a bridge into the system for external calls to a REST API.
The services each have their own Gitlab repo, and we use Gitlab’s CI to deploy to 14 servers. The services run via Supervisor, and use VirtualEnv’s for the Python dependencies. Host dependencies are managed separately via Salt.
We exchange files between some of the services (for example, we have a “transcoding” service that is used by a few of the other services). We use NFS mounts inside the cluster of servers to do this, and each service knows “where” it is, and requests that files be saved to the appropriate host’s NFS share.
I’m not afraid to re-engineer the whole setup (if that is what is required). I understand that “Jobs” can share network namespaces - but I’m not clear if Groups in separate Jobs can.
Should I be looking at a single “Job” definition, or stick to separate repo’s and deployments?
I have a demo service working, so the nuances of moving to Docker for the service specifically, isn’t a concern.
It’s really the inter-service exchange of Files. I leverage the ~450 cores to do fan-out transcodes, and it’s working really well.
I’m sure there are lots of details missing here. Thanks for reading!