Responding very slowly is one of the worst failures you can have.

Sometimes, the problem is not only in your system but in your dependencies.

3 ways to deal with Latency:

Time-outs are easy to overlook, but they are important to get right.

If you wait too long, you can slow the whole system down. Time out too fast, and you'll consider a call that might have worked as failed.

But have no time-outs; a downstream service could hang your whole system.

Look at your system's behavior. If you usually get the response in 1 sec, staying with the default of 30 sec is overkill.

Think about the time-out for the entire operation and adjust your timeouts to fix your budget.

Even if we had the timeouts right, we would have to wait a long time to see the error.

With a circuit breaker, the circuit breaker is blown after a certain number of requests to the resource have failed (due either to an error or to a timeout).

All further requests that go through that circuit breaker fail fast while the breaker.

This pattern protects the consumer from the downstream problem and the downstream service from more calls it can’t handle.

In a bulkhead architecture, elements of an application are isolated into pools so that if one fails, the others will continue to function.

Bulkhead prevents a single component from monopolizing resources and causing a system-wide failure.

In many ways, bulkheads are the most important.

Timeouts and circuit breakers help free up resources when they become a problem, but bulkheads can help avoid the problem in the first place.

Key Takeaways

When it is slow, your system ends up waiting for a long time before giving up; the same applies to your users.

What else are you doing to deal with latency?