Gateway Timeout with consul in application logs deployed in user cluster

Consul
1

Hi Team,

We have facing issues with consul, the key-value created in the consul and deleted automatically, also the applications deployed in our user cluster and consul enabled aren’t taking the configurations from consul and are having the below logs in the application pod :

at Consul.GetRequest1.Execute(CancellationToken ct) at Winton.Extensions.Configuration.Consul.ConsulConfigurationProvider.GetKvPairs(Boolean waitForChange, CancellationToken cancellationToken) at Winton.Extensions.Configuration.Consul.ConsulConfigurationProvider.PollingLoop(CancellationToken cancellationToken)Consul.ConsulRequestException: Unexpected response, status code GatewayTimeout: 504 Gateway Time-outThe server didn't respond in time. at Consul.GetRequest1.Execute(CancellationToken ct) at Winton.Extensions.Configuration.Consul.ConsulConfigurationProvider.GetKvPairs(Boolean waitForChange, CancellationToken cancellationToken) at Winton.Extensions.Configuration.Consul.ConsulConfigurationProvider.PollingLoop(CancellationToken cancellationToken)Consul.ConsulRequestException: Unexpected response, status code GatewayTimeout: 504 Gateway Time-outThe server didn’t respond in time.

Please advise and help us resolve the issue.

Regards,
Devika

2

Hi @devika.pujari,

From the logs, it looks like your client is talking to Consul via a Load Balancer, and most probably, due to performance reasons, your Consul servers are not responding in time, resulting in the error shown.

The way forward is to monitor the performance of the servers and add additional capacity if required. The following links will help you to get started:

I hope this helps.

3

Hi Ranjandas,

we have enough resources and is there any other ways to troubleshoot the issue.

Regards,
Devika