How to setup retry-policy for E/W traffic in services (sidecars)

I am using consul and envoy for building service mesh. I am stuck at how to setup the retry-policy for the services.
I have two services A and B behind ingress-gateway. I want to setup the retry policy via some configuration (don’t want to use the headers) so that when service A calls the service B and if service B replies with 5xx. The service A should retry making calls to service B.

Can someone help/guide me in the right direction?

Here is what i have tried so far:

  1. setting up config n the service A config
    {
    “service”: {
    “connect”: {
    “sidecar_service”: {
    “proxy”: {
    “upstreams”: [
    {
    “destination_name”: “serviceB”,
    “local_bind_address”: “127.0.0.1”,
    “local_bind_port”: 8081,
    “config”: {
    “retry_policy”: {
    “retry_on”: “5xx”,
    “num_retries”: 10
    }
    }
    }
    ]
    }
    }
    },
    “name”: “serviceA”,
    “port”: 8080,
    “Meta”: {}
    }
    }

This doesn’t seems to work.

Hi @shweshi I believe what you are looking for a is a service router. You can implement re-try logic with the service router. Take a look at the ServiceRouteDestination configuration block, there you will find the retry parameters.

Hey @karl-cardenas-coding thanks for the quick response… i did try the service router. Which is working fine with the ingress gateway (N/S) so when i make a call from ingress gateway to service a or service b… its performing retry but the same is not working if i make call from service a to service b. Any idea if i am missing something.

  "Destination": [
    {
      "PrefixRewrite": "/",
      "RetryOnConnectFailure": true,
      "Service": "serviceA",
      "NumRetries": 3,
      "RetryOnStatusCodes": [
          500,
          503,
          504
      ]
    }
  ]

Ohh may be i need to configure service route for each of the services @karl-cardenas-coding is it?

@shweshi Do you have a service router configuration for serviceA that specifies a destination of serviceB ? It seems like that piece is also missing.

1 Like

I will give it a try… by the way is there any way to setup default retry-policy so that i don’t need to define a route policy for every service service A is calling.
Else i will need to create routes and destination for each service B and service C in SErvice A service-route.

@shweshi There is no global retry policy that I am aware off. I’m also not sure that’s something I would feel comfortable applying to all services. But I see your point for wanting that.