Download the PHP package prahsys/laravel-api-logs without Composer

Prahsys Laravel API Logs

A comprehensive Laravel package for logging API requests and responses with idempotency support, model tracking, and configurable data redaction.

Features

• Request Correlation: Automatic correlation ID handling for API request tracking and audit trails
• Comprehensive Logging: Logs API requests and responses with detailed metadata
• Data Redaction: Configurable pipeline-based redaction for sensitive data (PCI, PII, HIPAA, etc.)
• Model Tracking: Automatic association of created/updated models with API requests
• Async Processing: Event-driven architecture with queue support
• Multiple Channels: Support for raw and redacted logging channels
• Compliance Ready: Built-in redaction support useful for PCI DSS, SOC 2, and other compliance requirements

Architecture Overview

The package follows a clean event-driven architecture with a lightweight database design:

Design Philosophy

ApiLogItem is designed as a lightweight reference to API requests, not a full data store. This approach:

• Keeps database lean: Stores only essential metadata (correlation ID, path, method, timestamps, status)
• Enables long-term retention: Default 365-day database retention for audit trails
• Separates concerns: Heavy request/response data goes to log channels, references stay in database
• Maximizes flexibility: Users can extend the model to store additional data if needed

The actual request/response data is processed through configurable log channels where it can be:

• Stored in log files with native Laravel rotation
• Sent to external services (Axiom, Sentry, etc.)
• Redacted according to compliance requirements
• Retained for different periods per channel

Key Components

1. ApiLogMiddleware: Captures request/response data and manages correlation IDs
2. CompleteApiLogItemEvent: Dispatched after request completion
3. CompleteApiLogItemListener: Processes model associations and log data
4. ApiLogPipelineManager: Registers Monolog processors for automatic redaction
5. ApiLogProcessor: Monolog processor that applies redaction pipelines to log records
6. ApiLogItemTracker: Tracks models during request processing
7. Redaction System: Pipeline-based data redaction with configurable redactors

Installation

Configuration

1. Publish Configuration and Migrations

2. Environment Configuration

3. Logging Channels

Add to your config/logging.php:

4. Middleware Registration

Add to your app/Http/Kernel.php:

Usage

Basic Usage

Once configured, the package automatically logs API requests. Include an Idempotency-Key header for request correlation:

Outbound API Logging with Guzzle

The package includes Guzzle middleware to log outbound HTTP requests your application makes to external APIs.

Basic Setup

Add the middleware to your Guzzle client's handler stack:

Adding to Existing Handler Stack

If you already have a handler stack with other middleware:

Skip Logging for Specific Requests

Configuration

Configure outbound logging in your environment:

Or in config/api-logs.php:

Model Tracking

Add the HasApiLogItems trait to models you want to track:

Models created or updated during API requests are automatically associated with the API log item. This is particularly useful for:

• Audit trails: Understanding which models were affected by a specific API request
• Impact analysis: Tracking the full scope of changes made during a request
• Debugging: Identifying which models were modified when troubleshooting issues
• Compliance: Maintaining detailed records of data modifications for regulatory requirements
• Data lineage: Tracing the history of model changes back to their originating API requests

Accessing Tracked Data

Configuration Options

Channel Configuration

Configure different redaction pipelines for different channels in config/api-logs.php. Each channel can have its own redaction strategy based on the destination's requirements:

Available Redactors

• CommonHeaderFieldsRedactor: Redacts authentication headers (extends DotNotationRedactor)
• CommonBodyFieldsRedactor: Redacts password fields (extends DotNotationRedactor)
• DotNotationRedactor: Base redactor using dot notation (supports * and ** wildcards)

Wildcard Pattern Support

The DotNotationRedactor supports powerful wildcard patterns:

• *Single wildcard (``)**: Matches one level

• Deep wildcard (``)**: Matches any level of nesting

Examples:

Extending ApiLogItem for Custom Data Storage

The default ApiLogItem stores lightweight references. You can extend it to store additional data:

Alternative approaches:

• External correlation: Use correlation IDs to fetch full data from Axiom/Elasticsearch
• Hybrid storage: Store critical fields in database, full payloads in object storage
• Event sourcing: Store lightweight events, reconstruct full state when needed

Creating Custom Redactors

The easiest way to create custom redactors is by extending DotNotationRedactor, just like the built-in CommonHeaderFieldsRedactor and CommonBodyFieldsRedactor:

Example: PCI DSS Redactor

Example: Healthcare (HIPAA) Redactor

Example: General PII Redactor

Advanced: Custom Replacement Logic

You can also provide custom replacement logic using closures:

Using Custom Redactors

Once created, use your custom redactors in your channel configuration:

Event System

Listening to Events

You can listen to CompleteApiLogItemEvent to add custom processing:

Event Data

The event contains:

• requestId: The correlation ID
• apiLogItemId: Database ID of the ApiLogItem
• models: Array of associated models
• apiLogData: Complete API log data object

Compliance Features

This package provides features that are generally useful for compliance requirements:

PCI DSS Support

• Comprehensive audit trails with detailed metadata
• Data isolation through configurable redaction system
• Transaction traceability via correlation IDs
• Protected logging channels with access controls

SOC 2 Support

• Audit-ready logging for security events
• Clear system boundaries through redaction
• Consistent logging format for monitoring
• Data protection through configurable redaction

External Service Integration

Monitoring and Alerting Services

The package integrates seamlessly with external monitoring services:

Sentry Integration:

Axiom Integration:

Other Services:

• Datadog: Use custom handlers for structured logging
• New Relic: Configure with appropriate redaction for APM integration
• Splunk: Set up with compliance-specific redaction pipelines
• Elasticsearch: Use stack channels for search and analytics

Database Pruning and Log Management

Database Pruning

ApiLogItems are designed for long-term retention (365 days by default) but can be pruned using Laravel's built-in model pruning:

Configure retention in your environment:

Log File Management

Separate from database retention, configure log rotation per channel:

Best practices:

• Database: Long retention (365+ days) for audit trails and correlation
• Raw logs: Short retention (7-30 days) for debugging, restricted access
• Redacted logs: Medium retention (30-90 days) for monitoring and analytics
• External services: Per-service retention policies (Sentry 30 days, Axiom 1 year, etc.)

Performance Considerations

• Lightweight database: Only essential metadata stored in database
• Async Processing: Heavy processing is handled by queued event listeners
• Configurable Logging: Exclude paths and request types to reduce overhead
• Efficient Model Tracking: In-memory tracking during request lifecycle

Testing

Run the test suite:

Compliance Configuration Examples

PCI DSS Configuration

For environments handling payment card data, configure appropriate redaction and retention:

SOC 2 Configuration

For SOC 2 environments, configure comprehensive logging with security controls:

Data Retention Policies

Configure retention policies based on your compliance requirements:

Security Notes

• Raw logs should be stored with restricted access permissions
• Consider using separate database connections for different sensitivity levels
• Implement proper log rotation and retention policies
• Review redactor configurations for your specific compliance requirements
• Ensure appropriate access controls for different log sensitivity levels
• Consider encrypting log storage for highly sensitive environments

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Submit a pull request

License

This package is open-sourced software licensed under the MIT license.

Support

For support, please open an issue on the GitHub repository or contact the maintainers.