Download the PHP package zeusi/circuit-breaker without Composer

Circuit Breaker

A circuit breaker for PHP with bucketed sliding windows and configurable half-open policies, sharing its state through APCu, Redis or Memcached.

When a service your application depends on stops answering, every call to it holds a PHP worker until it times out, and the workers pile up until the application itself stops answering. A circuit breaker notices the failures and, for a while, tells your code not to call that service: your code falls back instead, your workers stay free and the service gets time to recover.

• Failure rate over a sliding time window, with a minimum number of calls before deciding.
• Configurable recovery: after an outage, a half-open policy decides how the service is tested. Besides the classic single trial, a ramp brings the calls back gradually instead of all at once, so a single lucky success is not enough to close the circuit.
• Shared state with atomic operations, safe under concurrent PHP-FPM workers and across servers.
• Storage failures: if the storage fails, calls are permitted and an event tells you.

Installation

Requires 64-bit PHP 8.1 or later, plus the extension of the storage you use: apcu, redis (phpredis) or memcached.

Usage

A circuit breaker protects one service, identified by its name. It holds no state of its own, so creating one per call, or per host discovered at runtime, costs nothing.

Registering outcomes

The circuit measures whether the service is alive. Register:

• a success when the service answered, even with an application error unrelated to its health: not found, invalid data, access denied;
• a failure when it did not answer, or answered with an error of its own: timeout, connection refused, 5xx;
• nothing when the call never left, for example because building the request failed.

A missing outcome is only a sample less: the circuit never waits for it.

Two settings to get right

• The client timeout must be shorter than max_execution_time. A call killed by PHP registers no failure, so the circuit would never open.
• openDuration should be at least as long as the call timeout. Calls started before the circuit opened can still fail afterwards: if they fail while the circuit is testing the service again, they make it look still broken.

How it works

• Closed: every call is permitted. When a failure brings the failure rate of the time window to the threshold or above, with at least minimumRequests outcomes, the circuit opens.
• Open: no call is permitted for openDuration.
• Half-open: the circuit tests the service with part of the calls, as decided by the half-open policy. As soon as the failure rate of the half-open outcomes within the time window reaches the threshold, the circuit opens again; this state does not wait for minimumRequests. Otherwise it closes when the policy says enough has been observed, or at the latest after the policy's maximum duration.

There is no background process: a transition due to time happens on the first call that finds it due, and its event is emitted then. After a long period without calls, the circuit skips the half-open state and closes: no traffic reached the service in the meantime.

getState() returns the state the next call would find, without changing anything.

How outcomes are counted

The time window slides: the failure rate always covers the last timeWindow seconds. Unlike consecutive fixed windows, a burst of failures across the boundary of two intervals is not split in half.

• The window is split into buckets of at least 1 second, at most 60, one storage key per bucket and outcome: a 30 second window has 30 buckets of 1 second, a 10 minute window 60 buckets of 10 seconds. Every outcome is a single atomic increment, so concurrent workers never lose one.
• A count covers the buckets spanning the window, so it is precise within one bucket: 1 second for windows up to a minute, 1/60 of the window beyond. With fewer than 10 buckets the count would approach a fixed window, hence the minimum of 10 seconds.
• Every state counts only its own outcomes: the half-open state is never influenced by the failures that opened the circuit, and the closed state starts afresh after it. Nothing is ever deleted: older keys expire by themselves.
• An active service takes at most about 120 short keys (one per bucket for successes and for failures), a few tens of kilobytes, whatever its traffic.

Configuration

Durations are floats, with millisecond precision. Every parameter is required.

Half-open policies

A ramp permits a growing ratio of calls during its duration, then closes the circuit. It tests the service on real traffic and protects it from getting the full load at once:

The exponential ramp spends more time on low ratios: half way through, 10% of the calls are permitted, against about 50% for the linear one. The initial ratio is relative to your traffic: on a busy service, 1% may already be many calls.

The linear ramp is a power curve with exponent 1. A greater exponent starts more slowly, a lower one faster:

Permitted calls permit the first calls of the half-open state and close the circuit once they all reported back, or at the latest after the maximum duration:

A single successful trial closes the circuit even for a service that succeeds only now and then, and sends it the full traffic: the ramp is usually the safer choice.

A ramp can follow any curve, mapping its progress (0 to 1) to the ratio of calls permitted:

Storages

The shared storages accept a prefix, to separate applications sharing the same backend. Without it, two applications calling the same service name share its circuit.

APCu

APCu memory belongs to a single PHP master process, so every server, every PHP-FPM pool and every CLI command has its own circuits, opened by the failures it sees. When circuits must be shared across servers, use Redis or Memcached.

APCu has no background cleanup: an expired entry is removed when another entry is inserted in the same slot of its hash table. Since buckets are inserted continuously, the number of entries grows for a while, then stays around the live entries plus the number of slots (about four times apc.entries_hint, 4096 by default). This is expected and bounded. With many services, raise apc.entries_hint to keep lookups fast. If the rest of the application fills the cache, APCu before 5.1.25 with apc.ttl=0 clears it entirely, and every circuit restarts closed.

Redis

• The client must use no serializer nor compression: values are stored as plain integers.
• Read from the master: a replica lagging behind would give processes different views of the circuit.
• With Redis Cluster (\RedisCluster), the keys of a service share a hash tag, so they live on the same node. For this reason the prefixes, of the storage and of the client, must contain no braces.

Memcached

The keys of a service are kept on the same server.

Events

Pass an event listener to observe the circuit, for logging, metrics or alerting:

CallRejected is emitted for every rejected call: count it in a metric rather than logging it. Transitions are emitted once, even with many concurrent workers. Exceptions thrown by the listener are caught and ignored, so a failing listener never breaks the call it observes.

When the storage fails

Calls are permitted, outcomes are dropped, and a StorageFailed event is emitted. The circuit does not protect the service until the storage recovers, so watch that event. getState(), meant for diagnostics, throws the storage exception instead.

Testing

Code using the circuit breaker can depend on CircuitBreakerInterface and replace it with a test double. To run a real circuit breaker in tests, use InMemoryStorage and pass a ClockInterface implementation to control time, to both the storage and the circuit breaker.

License

MIT, see LICENSE.