I known Consul provides that we can configure tokens, policies and roles to work across multiple datacenters, and ACL replication can exists in multi-datacenter federation. My question is: Can AcLs not be replicated in a multi-datacenter federation? That is to say, each cluster has its own ACL strategy and does not interfere with each other. If not, please tell me why not support such strategy and what are the considerations inside it.
Currently federation requires that ACLs and tokens are replicated across DCs so that cross-DC service lookups can work.
For example, if client A in DC1 wants to lookup the service web.service.dc2.consul, the servers in DC1 will use an RPC call to forward the query to the servers in DC2. The secondary DC’s servers will use the token provided by client A when resolving the query to determine whether the token is authorized to perform the lookup. So, that token and its linked policies need to exist in DC2, which is why they are currently replicated.
With that said, I would like to understand more about your use case for not replicating ACLs. What challenges are you seeing with the current model? What would an ideal solution look like in your mind to solve for those challenges?
Through your clear and practical case, now I know why that ACL and tokes all need to be replicated across DCs.
In our production usage case, Nomad and Consul are both be federated across multi-DC or multi-region, and all deployment and bootstrap actions of clusters are self-driven in the form of scripts. So ACLs replication across DCs brings a little sophication in this initial bootstraping process. In purticular, we have to do some necessary manual intervention and verification. I know it plays a import part in service discovery and other situations, so I think it should be worth it.
My problem has been solved completely.
By the way, I have one another question:
Can the federated Consul cluster across multi-DC be used with a single Nomad cluster in seperated region? that is to say, can two seperaed Nomad cluster in two regions operate well with one federated Consul cluster across two DCs? (I think it’s ok) and in turn, can the federated Nomad cluster across two regions be used with two seperated Consul cluster in seperated DC? (I think it’s ok too)