Download the PHP package tawshiqulislam/laravel-llm-failover without Composer

Laravel LLM Failover Gateway

A resilient, provider-agnostic text-generation gateway for Laravel.

Laravel LLM Failover protects chat applications from upstream rate limits, overloads, timeouts, server errors, refusals, and empty responses. It tries each configured provider in order and returns one normalized response shape, regardless of which provider succeeds.

Features

• Ordered failover chains across Gemini, OpenAI, and Anthropic
• Configurable retries, delay, connection timeout, and request timeout
• Graceful, configurable reply after the entire chain is exhausted
• Consistent response and token-usage schema for every provider
• Normalized roles and collapsed consecutive same-role messages
• Eloquent conversation-history builder with deterministic tie-breaking
• Custom drivers through Llm::extend()
• Laravel container binding, facade, package discovery, and publishable config
• Typed exceptions without API keys or raw provider bodies in normal error messages
• PHPUnit, Orchestra Testbench, level-8 PHPStan, dependency audit, and CI coverage
• Runnable Postman examples with credential guards and response assertions

Requirements

• PHP 8.2 or newer
• Laravel 10, 11, 12, or 13

Laravel 13 itself requires PHP 8.3 or newer.

Laravel 10 and 11 are retained for backwards compatibility but no longer receive upstream security fixes. Use a currently supported Laravel release for production; the CI dependency audit runs on Laravel 12 and 13, while the older lines receive compatibility tests only.

Installation

Publish the configuration:

This creates config/llm-failover.php. Laravel package discovery registers the service provider and the Llm facade automatically.

Configuration

Single provider

When failover is disabled, LlmDriverInterface resolves directly to LLM_DEFAULT_DRIVER.

Production failover

Provider names are trimmed, lowercased, and deduplicated. An empty chain uses LLM_DEFAULT_DRIVER.

All environment options

LLM_HTTP_RETRIES is the number of retries after the initial attempt. A value of 2 allows at most three HTTP attempts per provider. Connection failures, 408, 429, every 5xx, and Anthropic 529 responses are retryable.

Gemini and OpenAI default to a temperature of 0.7. Set the relevant value to the literal null if the selected model does not accept sampling parameters:

Each driver config array also accepts timeout, connect_timeout, retries, and retry_delay; driver-specific values override the shared HTTP settings:

After changing .env values in a config-cached application, rebuild the cache:

Usage

Dependency injection

Injecting the interface is the recommended application-facing entry point. It resolves to the failover gateway when failover is enabled and to the default driver otherwise.

Conversation history must be chronological and contain at least one non-empty text message:

Recognized assistant aliases are assistant, model, ai, and bot. Other role values normalize to user. Blank messages are removed, and consecutive messages with the same normalized role are joined with a newline.

Facade

Model selection during failover

Only the first provider receives the model passed to send(). A different provider cannot generally use that model name, so every later provider uses its own configured default.

With LLM_FAILOVER_CHAIN=gemini,openai:

1. Gemini receives the caller's model, or GEMINI_DEFAULT_MODEL when the model is empty.
2. Gemini applies its configured retry policy to retryable failures.
3. If Gemini still fails, OpenAI is called with OPENAI_DEFAULT_MODEL.
4. If every provider fails, the gateway returns LLM_FALLBACK_REPLY.

Retryable responses, network failures, empty replies, safety blocks, and supported refusal responses cause failover. Missing API keys and non-retryable provider rejections are also skipped inside the gateway so another configured provider can answer.

Building Eloquent conversation history

Use HasChronologicalHistory on the model that owns a HasMany message relation:

Fetch the newest 20 messages and return them oldest-first:

The message limit must be at least 1. By default, the trait orders by the related model's id. It reads text from body, then content, and maps either of these common schemas:

When both fields exist, direction takes precedence.

For UUID primary keys, pass a monotonically increasing sequence column whenever possible:

The related primary key is used as a stable secondary sort when the chosen order column contains ties. That makes selection deterministic, but a random UUID cannot reconstruct the true creation order of rows with identical timestamps; use a sequence column if exact ordering matters.

Custom drivers

Register custom drivers from a service provider's boot() method:

Add the driver's config under llm-failover.drivers.mistral, then include mistral in LLM_FAILOVER_CHAIN. A custom driver must implement LlmDriverInterface and return the response shape below. Extending AbstractDriver provides the shared history normalization, HTTP retry configuration, response helpers, and fallback logging.

Response format

Every driver and the gateway implement the same contract:

Fallback usage values are zero because not every failed provider returns comparable usage metadata. See Expected_Response_Schema.md for the focused schema reference.

Error behavior

Direct provider calls and gateway calls intentionally differ:

ProviderRequestException exposes driver(), status(), and responseBody(). Its normal exception message deliberately excludes the raw response body; inspect responseBody() explicitly and avoid sending it to end users or untrusted logs.

MissingApiKeyException and ProviderRequestException extend LlmFailoverException, which extends RuntimeException. UnsupportedDriverException extends InvalidArgumentException for backwards compatibility.

Provider APIs

The bundled drivers currently target:

This v1 package supports text conversations only. Streaming, tool calls, images/audio, structured-output helpers, embeddings, and provider-native server-side fallbacks are outside the current interface.

Postman collection

Import postman/laravel-llm-failover.postman_collection.json into Postman, set one or more API-key collection variables, and run the matching requests. Placeholder-key guards stop a request before it is sent, and each request includes response assertions.

The collection calls providers directly. It validates provider credentials and payload compatibility, but it does not run Laravel or test the package's failover chain. See POSTMAN_COLLECTION.md for detailed instructions and Newman usage.

Never commit real API keys to the collection or an exported Postman environment. Live runs may incur provider charges.

Testing and quality checks

The PHPUnit suite uses Laravel HTTP fakes and SQLite in memory; no API credentials or network access are required. CI covers the compatible Laravel/PHP combinations and runs Composer validation, tests, and static analysis. Dependency auditing runs on the currently security-supported Laravel 12 and 13 jobs.

Security

• API keys are sent in headers and are never placed in provider URLs.
• Raw provider response bodies are available only through the explicit exception accessor.
• Fallback logs contain safe messages and status metadata, not raw response bodies.
• Disable fallback logging with LLM_LOG_FALLBACKS=false if even provider/status metadata is unsuitable for your environment.

Please report security issues privately to the maintainer email listed in composer.json rather than opening a public issue.

Contributing

Contributions are welcome. Please include tests and update the relevant documentation when behavior changes.

Useful contribution areas include additional providers, failover strategies, richer response types, and performance improvements. Run composer check before submitting a pull request.

License

The MIT License. See LICENSE.md.