Download the PHP package elriseio/doctrine-shard-manager-bundle without Composer

Doctrine Shard Manager Bundle for Symfony

Doctrine Shard Manager Bundle is a Symfony bundle for horizontal data sharding on top of Doctrine ORM and DBAL. The bundle provides transparent routing of Connection and EntityManager per shard, pluggable resolution strategies (hash, range, uuidv7), a PSR-6 cache layer, and a CLI for greenfield provisioning and additive migrations.

Key Features

• Transparent sharding. Callers write ShardedRepository calls or ShardContext::withShard() / withEntity() callbacks; the bundle selects the right Connection and EntityManager for the entity being touched.
• Pluggable strategies. Three concrete strategies ship out of the box — hash (SHA-256 modulo), range (binary search over a range_table partition), uuid (UUIDv7 with embedded shard index). Custom strategies register via the app.shard_strategy tag (see ADR-0002).
• Dual configuration surface. PHP #[Sharding] attribute on the entity class, OR YAML under doctrine_shard.sharding_configs.<FQCN>. Resolution precedence is YAML > attribute (see docs/architecture.md::I-RES1).
• Cache-first resolution. Optional PSR-6 CacheItemPoolInterface per strategy and per resolver; cache failures degrade gracefully (logged + bypassed, never propagated).
• Cross-shard reads. ShardedFinder::find(callable $query, array $shardIds) fans out a read query to multiple shards lazily via \Generator (see ADR-0004).
• Schema bootstrap. bin/console shard:add <id> provisions a shard's schema from filtered entity metadata. bin/console shard:migrate <id> runs Doctrine Migrations on a single named shard and is additive (see ADR-0003).
• Doctrine 3.3 + DBAL 4.2 compatibility, PHP 8.3 strict types.

Architecture

Layering rules (see docs/architecture.md for the full invariant list):

• Contract/* interfaces are the only types consumers depend on directly. All concrete implementations are un-tagged internal; consumers MUST type-hint against the interfaces.
• Strategy/* depends only on Contract/ShardStrategyInterface and Config/Dto/*.
• Resolver/* depends only on Contract/*, Attribute/*, and Strategy/* instances injected at construction.
• Context/* orchestrates Resolver, ConnectionManager, EntityManagerProxy — it does not touch Doctrine directly.
• Manager/* is a thin Doctrine binding layer (ShardConnectionManager wraps Connection lookup; EntityManagerProxy wraps EntityManager lookup).
• Repository/* provides a higher-level facade; it composes ShardContext and ShardResolver.

Key interfaces

Strategy contracts

• ShardStrategyInterface — resolveShardId(mixed $key, array $options = []), getTotalShards(), resolveShardIndex(mixed $key), getCacheKey(mixed $key). The single seam for pluggable strategies.
• AbstractShardStrategy — base class that exposes three protected helpers (normalizeKey, defaultCacheKey, resolveShardIndexFromId). Override public methods on the abstract class for custom strategies; concrete strategies (HashShardStrategy, RangeShardStrategy, UuidShardStrategy) are reference implementations and may change between minor versions.

Resolver and context

• ShardResolverInterface — resolveShardId(object $entity), resolveShardIdFromId(mixed $id, string $entityClass), resolveShardIdFromCriteria(array $criteria, string $entityClass), getShardingConfig(string $entityClass). Merges YAML and #[Sharding] attribute.
• ShardContextInterface — withShard(string $shardId, callable $cb), withEntity(object $entity, callable $cb), getCurrentShardId(), getAvailableShards(). The state machine that switches Connection and EntityManager for the duration of the callback.

Manager layer

• ShardConnectionManagerInterface — switchToShard(string $shardId), getCurrentConnection(), getConnectionForShard(string $shardId). Owns the shard-to-Connection registry.
• EntityManagerProxyInterface — forShard(string $shardId), getAvailableShards(). Owns the shard-to-EntityManager registry.

Repository and finder facades

• ShardedRepositoryInterface — findOneById, findBy, findOneBy, persistToShard, removeFromShard, flushShard. Composes ShardContext and ShardResolver; business code stays unchanged.
• ShardedFinder (concrete) — find(callable $query, list<string> $shardIds): \Generator. Bypasses the write-path ShardContext; throws on the first shard failure with the per-shard context framed in the exception message.

Installation

Run the following command to install the bundle:

Register the bundle in config/bundles.php (if you are not using Symfony Flex):

The bundle reads its configuration from config/packages/doctrine_shard.yaml (see Configuration below) and registers all six public services automatically. No additional services.yaml wiring is required for the default strategies.

Configuration

config/packages/doctrine_shard.yaml example for a four-shard hash setup keyed by userId:

doctrine.dbal.default_shard_<N>_connection and doctrine.orm.default_shard_<N>_entity_manager are the standard Doctrine connection / EM service IDs that Symfony's doctrine bundle exposes when you declare dbal: { connections: { default_shard_0: ~, default_shard_1: ~, ... } } and orm: { entity_managers: { default_shard_0: ~, default_shard_1: ~, ... } } in config/packages/doctrine.yaml.

The #[Sharding] attribute (alternative to YAML)

The key parameter may also be string[] for composite keys:

YAML sharding_configs.<FQCN> overrides the attribute when both are present.

Usage

Through ShardContext

withShard($shardId, fn) is the explicit variant when the caller already knows the target shard (e.g. a cron job iterating every shard).

Through ShardedRepository

ShardedRepository::findBy, findOneBy, persistToShard, removeFromShard, flushShard complete the surface; each call resolves the shard ID internally and routes the call to the right EntityManager.

Cross-shard reads with ShardedFinder

ShardedFinder is sequential in v1 (per ADR-0004 §Decision). The generator holds one row at a time across the fan-out, so memory is bounded by the largest single-shard result set. Failures throw with the per-shard context framed in the message.

Extending with a custom strategy

Then reference it as strategy: tenant in sharding_configs.<FQCN> or #[Sharding(strategy: 'tenant', ...)]. The full worked example is in docs/adr/0002-shard-strategy-extensibility.md.

Console commands

The bundle registers two Symfony Console commands on build():

bin/console shard:add <id>

Provisions a new shard's schema. Filters entity metadata to #[Sharding]-annotated classes only and runs SchemaTool::updateSchema($filteredMetadata, saveMode: true).

Safety warning. shard:add is drop-and-recreate, not additive migration. Run it on greenfield shards only — running it on an existing shard deletes the rows. For an existing shard, use bin/console shard:migrate <id> instead (Wave 2 / ADR-0003). See docs/RUNBOOK.md::Failure Mode 10 for the full migration procedure.

bin/console shard:migrate <id>

Runs Doctrine Migrations on a single named shard. Additive; safe to run against an existing shard.

Each command supports the standard --dry-run, --migration-set=<alias-or-version>, and exit-code conventions of the underlying Doctrine Migrations tooling.

Compatibility

• PHP 8.3+
• Symfony 7.2 (7.2.* pin in composer.json)
• Doctrine DBAL 4.2+
• Doctrine ORM 3.3+
• Doctrine Migrations 3.7+
• PSR-6 CacheItemPoolInterface (optional but recommended for hot-path strategies)
• PSR-3 LoggerInterface (optional; graceful degradation when absent)

Known compatibility gaps

• The doctrine/event-subscriber path (modern replacement for the deprecated ShardResolverListener) is planned for Wave 1. Until then, the listener remains @deprecated but is wired in services.yaml for backward compatibility with legacy consumers.
• The three-tier testing infrastructure (unit + bench + itests; ADR-0001) is partially landed. bench/ and itests/ are now populated; the bounded-concurrency CI smoke (DE-011-B) is open and tracked under Issues/open/developer/.
• phpstan and php-cs-fixer are declared in require-dev but the CI-gated enforcement job (Wave 6 / DE-019) is open.

Benchmarks

The bundle ships an in-process benchmark suite under bench/ that exercises the hot path of every public strategy and orchestrator (see ADR-0001). It uses PHPBench and runs without a database by default — subjects that need a Connection opt in via BENCH_DATABASE_URL.

Running

The script invokes phpbench run --report=aggregate. The default runner.path (bench/) is configured in phpbench.json, so the positional bench/ argument is no longer required. For custom runs (extra --revs, --iterations, --filter):

The HTML report and per-subject memory samples are written to bench/.bench/ (gitignored). Subjects honour --filter=<substring> for targeted re-runs.

Latest results (2026-07-21, PHP 8.5.8 NTS, xdebug off, opcache off)

20 subjects, 0 failures, 0 errors. mode is the per-subject median; rstdev is the relative standard deviation across iterations.

Reading the table

• benchResolve* subjects measure strategy hot paths — sub-millisecond per call in a single PHP process.
• benchSwitchToShard* and benchGetConnectionForShard* measure per-shard routing inside ShardConnectionManager; the lookup is amortised to ~1 µs because the manager keeps a WeakMap cache.
• benchWithShard and benchWithEntity measure ShardContext callback wrapping only — no Doctrine round-trip.
• benchFindOneById is the only end-to-end subject that touches a real EntityManager; the value is dominated by Doctrine hydration.
• benchFanOutOver16Shards1kRowsEach exercises ShardedFinder against a stubbed Connection per shard and serves as a regression guard for the fan-out state machine in ADR-0004.

Re-run after any change to src/Strategy/*, src/Resolver/*, src/Manager/*, or src/Context/* and update the table if any subject regresses by more than its published rstdev.

Integration tests

itests/ is the third tier of the testing infrastructure (ADR-0001): end-to-end scenarios that drive the bundle's real classes against real MySQL 8.4 and PostgreSQL 16 instances under controlled concurrency, with a fixed JSON envelope. The scenarios speak to DBAL directly — the bundle ships no Symfony app.

Prerequisites

• Docker Engine with the Compose plugin.
• PHP extensions pdo_mysql and pdo_pgsql enabled (the in-repo composer.json runtime does not load them by default; see Known issues below).

Bootstrap

Default ports are offset (33061 for MySQL, 33062 for MariaDB, 54321 for PostgreSQL) so the stack coexists with the dbal-manager project's stack on the same host. Credentials default to root:itests for MySQL and MariaDB, itests:itests for PostgreSQL.

Running a scenario

Each scenario prints one JSON envelope line on stdout with scenario, db_vendor, rows, chunk, duration_s, ops_per_sec, peak_rss_bytes, and errors. The full envelope (including DB counters and latency percentiles) is produced by the itests/bin/run-scenario.sh wrapper; see itests/README.md for the CI smoke gate contract.

Latest results (2026-07-22, PHP 8.5.8 NTS, rows=200, chunk=50, warmup+reset)

All 12 scenarios pass on MySQL, MariaDB, and PostgreSQL.

Reading the table

• shard_resolution_* subjects measure the strategy hot path under real DB I/O. MySQL is faster on the hash variant; PostgreSQL is consistently faster on the range and custom strategy variants; MariaDB sits between the two with low variance across all four.
• bulk_write_per_shard is a write-heavy workload that exercises the routing-table fan-out (bench_itests_shard_routing). MariaDB is the fastest at this profile (~2881 ops/s, ~11.5× faster than MySQL on the same workload); PostgreSQL is ~3.3× faster than MySQL.
• cross_shard_lookup validates that the shard ID resolved at write time matches the one resolved at read time across every fan-out shard — see itests/README.md for the cross-shard bleed detection contract.

Known issues

• itests/bin/run-scenario.sh wrapper is unusable as-shipped. The wrapper passes --dsn "<value>" (whitespace-separated) to the underlying PHP CLI, but runner_base.php declares getopt('', ['dsn::', …]) which silently drops the value for long options with optional arguments on PHP 8.5. The wrapper prints "scenario did not emit JSON" (rc=2) and exits. Workaround: invoke the scenario script directly with --dsn=<value> (equals-sign form). Fix: either change 'dsn::' to 'dsn:' in itests/runner/runner_base.php:52 or rework the wrapper to pass DSN via env.
• pdo_mysql / pdo_pgsql are not auto-enabled. The runtime image ships the .so files in /usr/lib/php/modules/ but /etc/php/conf.d/ is empty in the default install. Set PHP_INI_SCAN_DIR=~/.php-confd with one extension=<name>.so line per file, or install the distro packages (php-mysql, php-pgsql).
• composer bench is unusable as-shipped. phpbench.json does not declare runner.path, so composer bench aborts with "You must either specify or configure a path". Pass bench/ explicitly, or add "runner.path": "bench" to phpbench.json.

Important notes

• The sharding_configs array is always required even if you rely solely on #[Sharding] — an empty [] is the canonical declaration. The YAML-vs-attribute precedence test (DE-013, closed) lives in tests/Resolver/ShardResolverTest.php.
• connections and entity_managers are passed as Doctrine service IDs (e.g. @doctrine.orm.default_shard_0_entity_manager), but the per-shard connection and EM MUST also be declared in config/packages/doctrine.yaml so that Doctrine itself instantiates them.
• ShardContext::withShard and withEntity route the active Connection and EntityManager for the duration of the callback only. The previous routing is restored in finally. Cross-shard transactions are out of scope by design (see docs/architecture.md::Non-Goals).
• The deprecated ShardResolverListener class is kept for backward compatibility with consumers wiring doctrine.event_listener. New code should rely on ShardedRepository or ShardContext directly. The listener will be removed in 2.0.

License

MIT — see LICENSE for the full text.