In case of backend failure, it is always better to fail quickly and relieve the back pressure from the actual backend, so it can recover quickly. Choreo Connect supports various types of distributed circuit breakers offered by Envoy. Based on the circuit breaker configuration, the router (i.e. Envoy) will open the circuit and handle the consequent request by itself till the backend recovers.
Enable circuit breakers for API endpoints¶
Circuit breakers are enabled by default and have moderate default values.
The following is the format of the vendor extension that is required to enable circuit breaking for an endpoint cluster in Choreo Connect.
The following is the default values for each circuit breaker which will be enable for each endpoint cluster by default.
The maximum number of connections that Router will establish to all endpoints in the cluster. All connections regardless of the active, idle will be counted. Even that the count of this circuit breaker exceeded for an endpoint cluster, router will always allocate at least one connection for the selected load balanced endpoint.
The maximum number of requests that can be outstanding to all endpoints in a cluster at any given time.
The requests will be added to a pending list when there is no readily available connection to serve the request immediately. This circuit breaker will exercise the maximum number of requests that will be queued while waiting for a ready connection pool connection.
This defines the maximum concurrent initiations of connection pools. When a cluster has exhausted its concurrent connection pools, it will attempt to reclaim an idle one. If it failed to reclaim, then the circuit will be opened.
The maximum number of retries that can be served to any endpoint in a cluster at any given time concurrently. This will be helpful to avoid cascading failures and overloading endpoints due to undesirable retries.
For more information on above circuit breakers, refer the Envoy documentation