Download the PHP package marshmallow/laravel-database-sync without Composer

Laravel Database Sync

A powerful Laravel package that enables seamless synchronization of data from a remote database to your local development environment.

Table of Contents

• Requirements
• Installation
• Configuration
  • Environment Variables
• Usage
  • Basic Synchronization
  • Advanced Options
  • Per-Table Sync Tracking
  • Table Configuration
  • Timeout Configuration
  • Synchronization Suites
  • Multi-Tenant Support
• Testing
  • Test Structure
  • Writing Tests
• Security
• Support
• License

Requirements

• PHP ^8.2
• Laravel ^10.0|^11.0|^12.0

Installation

You can install the package via composer:

Configuration

Publish the configuration file:

Environment Variables

Add these variables to your .env file:

Important: When connecting to a Forge-provisioned database server, you must use the main database user that was created during the initial server provisioning. Other database users created afterward may not have the necessary privileges to execute the required database commands for synchronization.

Usage

Basic Synchronization

To sync your remote database to local:

By default, the package uses batch file transfers for optimal performance, transferring all table data in a single file to minimize network overhead.

Advanced Options

The sync command supports several options:

Available options:

• --date: Sync records from a specific date
• --suite: Use a predefined suite of tables
• --table: Sync a specific table
• --tenant: Specify tenant for multi-tenant applications
• --skip-landlord: Skip landlord database in multi-tenant setup
• --full-sync: Sync the full table without a date constraint
• --status: View the sync history and status for all tables
• --individual-transfers: Use individual file transfers for each table (legacy behavior)

Per-Table Sync Tracking

The package now tracks the last sync date for each individual table, preventing data loss when syncing single tables. This means:

• Each table maintains its own sync history
• When syncing a specific table with --table, only that table's sync date is considered
• The package automatically falls back to the global sync date for backward compatibility
• You can view the sync status of all tables using the --status option
• Sync timestamps are captured at the start of the process to prevent missing data created during sync

Viewing Sync Status

To see the last sync date for all tables:

This will display a table showing each table name and its last sync date, helping you track which tables have been synced recently and which might need attention.

How It Works

1. Sync Start Timestamp: The sync timestamp is captured when the sync process begins
2. Individual Table Tracking: Each table's sync date is stored separately in the cache file
3. Atomic Updates: Cache is only updated when the entire sync process completes successfully
4. Automatic Fallback: If no table-specific date exists, the global sync date is used
5. Backward Compatibility: Existing installations continue to work without any changes
6. Debug Information: When running with -vvv (debug mode), you'll see which sync date is being used for each table
7. Error Recovery: If sync fails, the cache remains unchanged with the previous sync dates

File Transfer Optimization

The package uses batch file transfers by default to minimize network overhead:

• Batch Mode (Default): All tables are dumped to a single file and transferred once
• Individual Mode: Each table is transferred separately (legacy behavior)

Configuration

Control the transfer mode in config/database-sync.php:

Or via environment variable:

Benefits of Batch Transfer

• Reduced network overhead: Single file transfer instead of multiple
• Faster sync times: Especially noticeable with many tables
• Better compression: SSH compression works more efficiently on larger files
• Lower resource usage: Fewer process spawns and file operations

When Individual Transfers Are Used

• Single table sync (--table=tablename)
• Explicit override (--individual-transfers)
• Config set to individual mode

Table Configuration

You can exclude specific tables from synchronization in the config/database-sync.php file:

Timeout Configuration

For large databases, you may need to adjust the process timeout to prevent operations from timing out:

You can also set this via environment variable:

This timeout applies to:

• MySQL dump operations (mysqldump)
• MySQL import operations (mysql)
• File transfer operations (scp)

Synchronization Suites

Define custom synchronization suites in the configuration file to group tables for specific sync tasks:

Then use the suite option:

Multi-Tenant Support

The package supports multi-tenant architectures. Enable it in the configuration:

Configure tenant-specific settings in your configuration file and use the --tenant option to sync specific tenant databases:

Testing

This package uses Pest PHP for testing. To run the tests:

To run tests with coverage report:

Test Structure

The test suite includes:

• Unit Tests: Testing individual components

  • Config class tests
  • DatabaseSync class tests
  • Other utility classes
• Feature Tests: Testing the package functionality
  • Command execution tests
  • Multi-tenant functionality
  • Suite configurations
  • Table filtering

Writing Tests

To add new tests, create a new test file in either the tests/Unit or tests/Feature directory. The package uses Pest's expressive syntax:

Security

• Never commit sensitive database credentials to version control
• Always use environment variables for sensitive information
• Ensure proper access controls on both remote and local databases

Support

For support, please email [email protected]

License

The MIT License (MIT). Please see License File for more information.