I need to use consul only for service discovery, health checking, alerts etc. In which mode should I host consul.
Currently, 2 nodes are enough for me to host consul and as per load, I will increase the count in future.
We are already using a 8 node cluster of hashicorp vault and consul where we’ve hosted a combination of consul server and client. But, we followed this approach because we have used consul server as backend storage for vault.
How to manage in this use case where the sole purpose of using consul is the management of other services. If possible, please share the consul configuration scripts as well.
Thank you. Take care.
You need at least three Consul servers for the Raft protocol to reach a quorum and have high availability. This way, one node can shortly go down without impacting your Consul setup. You don’t gain anything by running only two Consul nodes over a single node.
A Raft cluster of 3 nodes can tolerate a single node failure while a cluster of 5 can tolerate 2 node failures. The recommended configuration is to either run 3 or 5 Consul servers per datacenter. This maximizes availability without greatly sacrificing performance.
Node count is not an issue. I can increase. But here my query is different. Do I need consul as server and client both or server node is enough.
Is it possible to expose ui directly through server like for accessing ui to check the health or setting alerts etc.
In case of vault we have a structure like consul servers nodes connected to vault node via consul client node.
What should be the architecture of consul in such use case.
Running all nodes as server is enough. You can still enable the UI.
@fhemberger , Great. Thanks a lot for the details. One last query. If I host 3 nodes then at random one will become a leader.
Now, for integrating any service do we need to use only the leader node of the consul. Or, can we access the UI or shell prompt from any node and infuse the service details.
Because I remember, while hosting hashicorp vault consul cluster 4 years ago I was stuck while enabling acl’s. At that time I get to know, for any administrative work, we can run commands via leader node only.
@smartaquarius10, sorry, I don’t know off the top of my head if there are any ACL/leader node limitations. If you find something, it would be good to know.
@smartaquarius10, there are few different ways in which you can register services to Consul. You could choose the one that best suits your need. Having said that, one usual pattern is to run Consul Client Agent on the node where your service is running and register the service on that node via the local Consul agent.
By doing this, you are registering both the node and the service to the Consul catalog. If the node health check fails for some reason, Consul will stop advertising the services running on the node to Service Discovery queries. (You should be familiar with this pattern as you are already running a Vault + Consul setup).
Regarding your question on whether you should talk directly to the Leader node, No you don’t have to. In Consul all the Consul Agents will forward the RPC request it receives to the Leader node. You will get to read about it here: Consul Architecture | Consul by HashiCorp
I have custom rest apis hosted in different platforms. Few in azure or kubernetes or virtual machine nodes.
Could you please confirm if this scenario is possible or not.
I’ll take 2 separate nodes and deploy consul as agent on them.
Now I’ll start integrating all the services into consul.
In this case do I need consul server node. As suggested by frederic above I think we need it.
Yes, you definitely need Consul Server agents (unless you are planning to reuse the Consul used with your Vault Infrastructure) and a minimum of 3 nodes for high availability. It’s the servers that store the Service Discovery Catalog (using Raft protocol).
Just to make it clear,
agent is the term used for the Consul process running on a node (irrespective of
server). Ref: Agent | Consul by HashiCorp
Additionally, there is first-class support for services hosted in Kubernetes using Consul connect if you are looking for a full-service mesh solution that spans across infrastructure, deployment types, etc.
Please have a look at the reference architectures linked below
@Ranjandas Yes. I by mistake used the agent for client.
One more query as per this definition on the url
Non-server agents run in client mode. A client is a lightweight process that registers services, runs health checks, and forwards queries to servers. A client must be running on every node in the Consul datacenter that runs services, since clients are the source of truth about service health.
Do I really need consul client. If I have my rest api’s are in azure as app service then can register them directly into consul server and setup alerts. Or, do I need a separate node of consul client for health checking and monitoring. My use case is a small scale so can I avoid using consul client completely.
If you are deploying services on VM’s that you have control of, or on Kubernetes (or other container stacks), having a Consul client is the right way to go.
I don’t know how much work would be involved and would it be worth the effort in the case of Azure App Services. I think the only easy option if at all you want to register the services into Consul will be as External Services Consul and External Services. But then whenever the service moves to a different IP etc, you will have to update the registration.
I am not familiar with Azure App Services. Just a quick search took me to this StackOverflow thread. Using Consul with Azure App Services - Stack Overflow.
@Ranjandas , Its not of IP. I have my url. So I’ll add that url into it rather than using any IP. URL will not change that is confirmed.
Could you please tell me one more thing. In the url you’ve shared this line is written
Consul uses a push-based model where agents send events only upon status changes. As such, even large clusters will have very few requests.
Is it referring to consul client + consul server mechanism where client will be sending updates to server. But, in my case it will always be pull based. Am I correct here.