First let me explain my use-case and that I’m aware that Consul might not work in this scenario. I’m currently looking at a use-case which requires that I use a service endpoint (read: a single instance of the service running somewhere) for a certain data set only, that means that I “acquire” an endpoint, load it with a data set, and when I’m done I need to spin it down. Now comes the crux, this “designated” endpoint must be discoverable to other clients which might want to work on the same data set. Additionally, there can only be one data set on each endpoint. I’ve been looking into having this solely depending on Consul without the need for an external management service, but have so far trouble with coming up with a good solution. I wanted to avoid having an external service which implements this because it would be nice to have this directly available from the infrastructure system.
One idea I had was to use the KV store/Session management of Consul to achieve a locking mechanism for designating a single endpoint for this data set and making it discoverable to other clients. That might work, but I’d have trouble with picking a new endpoint to be used from the service endpoints that are available, as I’ve found no method of either to designate the service endpoints inside the service discovery or removing them from it completely to make sure that no endpoint is picked twice. I could of course fetch the whole list and compare it against the available sessions, but that still leaves me with vulnerable to race conditions between clients needing a new endpoint.
Ideally I guess, I would be able to implement some form of queuing mechanism, which would simply hand me a new endpoint every time which can be used, removing it from the list afterwards either automatically or on my command. But I’m not sure whether such a use-case can be implemented in Consul at all.
If somebody could give me a few hints here, that would be greatly appreciated.