Found the issue and solution, replying to myself as it may help others.
The problem is that the default value of readinessProbe initialDelaySeconds in chart is too low and the sidecar container keep being restarted.
Solution:
Increase the value of readinessProbe initialDelaySeconds from 2 to 10 or whatever value suits you. You may need to modify the same also for livenessProbe.
Example:
Template: connect-inject-deployment.yaml
readinessProbe:
httpGet:
path: /health/ready
port: 8080
scheme: HTTPS
failureThreshold: 2
initialDelaySeconds: 10 // I increased this one
periodSeconds: 2
successThreshold: 1
timeoutSeconds: 8 // I increased this one