Download the PHP package cboxdk/laravel-queue-autoscale without Composer

Queue Autoscale for Laravel

Intelligent, predictive autoscaling for Laravel queues with SLA/SLO-based optimization.

Queue Autoscale for Laravel is a smart queue worker manager that automatically scales your queue workers based on workload, predicted demand, and service level objectives. Unlike traditional reactive solutions, it uses a hybrid predictive algorithm combining queueing theory (Little's Law), trend analysis, and backlog-based scaling to maintain your SLA targets.

Features

• 🎯 SLA/SLO-Based Scaling - Define max pickup time instead of arbitrary worker counts
• 📈 Predictive Algorithm - Proactive scaling using trend analysis and forecasting
• 🔬 Queueing Theory Foundation - Little's Law (L = λW) for steady-state calculations
• ⚡ SLA Breach Prevention - Aggressive backlog drain when approaching SLA violations
• 🖥️ Resource-Aware - Respects CPU and memory limits from system metrics
• 🔄 Metrics-Driven - Uses laravel-queue-metrics for queue discovery and all metrics
• 🌐 Cluster-Aware - Multiple autoscale managers auto-join via Redis, elect a leader, and distribute worker targets across hosts
• 🌐 Cluster-Aware - Multiple autoscale managers auto-join via Redis, elect a leader, and distribute worker targets across hosts
• 🎛️ Extensible - Custom scaling strategies and policies via interfaces
• 📊 Event Broadcasting - React to scaling decisions, SLA predictions, worker changes
• 🛡️ Graceful Shutdown - SIGTERM → SIGKILL worker termination
• 🚀 High Performance - Drift-corrected evaluation loop and efficient process management
• 🔒 Secure by Design - Safe process spawning and resource isolation
• 💎 DX First - Clean API following Spatie package conventions

Requirements

• PHP 8.3+
• Laravel 11.0+
• cboxdk/laravel-queue-metrics ^1.0.0
• cboxdk/system-metrics ^1.2

Installation

Install via Composer:

Run the interactive installer to publish config, choose the right topology, and generate the matching .env values:

It guides you through three safe presets:

• single host, low traffic, no Redis infrastructure
• single host with Redis-backed metrics and predictive signals
• multi-host cluster with Redis coordination

If you prefer the manual path, you can still publish the config files yourself:

Publish the configuration file:

Setup Metrics Package

The autoscaler requires laravel-queue-metrics for queue discovery and metrics collection:

Configure storage backend in .env:

Queue Autoscale itself can now run in three safe modes:

auto keeps single-host mode Redis-free and switches to Redis-backed coordination automatically in cluster mode.

The installer can also write the recommended values straight into .env for you.

If using database storage, publish and run migrations:

📚 See Metrics Package Documentation for advanced configuration.

Quick Start

1. Configure SLA Targets (Optional)

Zero Config: By default, the package uses the "Balanced" profile (30s SLA). You can skip configuration if this suits you.

To customize, edit config/queue-autoscale.php:

2. Run the Autoscaler

The autoscaler will:

• Receive all queues and metrics from laravel-queue-metrics
• Apply scaling algorithms to meet SLA targets
• Scale workers up/down based on calculations
• Respect CPU/memory limits from system-metrics
• Log all scaling decisions

3. Monitor with Events

How It Works

Hybrid Predictive Algorithm

The autoscaler uses three complementary approaches and takes the maximum (most conservative):

1. Rate-Based (Little's Law)

Calculates steady-state workers needed for current load.

2. Trend-Based (Predictive)

Uses trend analysis to predict future arrival rates and scale proactively.

3. Backlog-Based (SLA Protection)

Aggressively scales when approaching SLA violations.

Resource Constraints

All calculations are bounded by:

• System capacity - CPU/memory limits from system-metrics
• Config bounds - min/max workers from configuration
• Cooldown periods - Prevents scaling thrash

See Architecture Documentation for detailed algorithm explanation.

Configuration Reference

SLA Defaults

Per-Queue Overrides

Prediction Settings

Resource Limits

Worker Settings

Strategy Configuration

Custom Scaling Strategies

Implement your own scaling logic:

Register in config:

Scaling Policies

Add before/after hooks to scaling operations:

Register in config:

Events

Subscribe to scaling events:

ScalingDecisionMade

WorkersScaled

SlaBreachPredicted

Advanced Usage

Running as Daemon

Use Supervisor to keep the autoscaler running:

Custom Evaluation Interval

Default is 5 seconds between evaluations.

Debugging

Enable detailed logging:

View scaling decisions:

Metrics Integration

This package does NOT discover queues or collect metrics itself. All queue discovery and metrics collection is delegated to laravel-queue-metrics:

Package Responsibilities:

laravel-queue-metrics (dependency)

• ✅ Scans all configured queue connections
• ✅ Discovers active queues
• ✅ Collects queue depth and age metrics
• ✅ Calculates processing rates
• ✅ Analyzes trends and forecasts

laravel-queue-autoscale (this package)

• ✅ Applies scaling algorithms (Little's Law, Trend Prediction, Backlog Drain)
• ✅ Makes SLA-based scaling decisions
• ✅ Manages worker pool lifecycle (spawn/terminate)
• ✅ Enforces resource constraints (CPU/memory limits)
• ✅ Executes scaling policies and broadcasts events

Comparison with Horizon

Testing

Run the test suite:

Run with coverage:

Changelog

Please see CHANGELOG for recent changes.

Contributing

Please see Contributing Guide for details.

Security

If you discover any security related issues, please email [email protected] instead of using the issue tracker.

Credits

• Sylvester Damgaard
• All Contributors

Resources

Documentation

• Architecture - Deep dive into the hybrid predictive algorithm, queueing theory, and system design
• Troubleshooting - Common issues, debugging tips, and solutions
• examples/README.md - Practical examples and templates for custom strategies and policies

Examples

• Custom Strategies

  • TimeBasedStrategy - Scale workers based on time-of-day patterns
  • CostOptimizedStrategy - Prioritize cost efficiency with conservative scaling

• Custom Policies

  • SlackNotificationPolicy - Send Slack alerts on scaling events
  • MetricsLoggingPolicy - Log detailed metrics to dedicated file
• Configuration Patterns
  • config-examples.php - 8 real-world configuration examples for different use cases

License

The MIT License (MIT). Please see License File for more information.