Download the PHP package utopia-php/usage without Composer

Utopia Usage

Utopia framework usage library is a simple and lite library for managing application usage statistics. This library is aiming to be as simple and easy to learn and use. This library is maintained by the Appwrite team.

Although this library is part of the Utopia Framework project it is dependency free and can be used as standalone with any other PHP project or framework.

Features

• Two Table Architecture: Separate events and gauges tables optimized for their access patterns
• Events Table: Request-level metrics with dedicated columns for every dimension we filter on (path/method/status, service/resource ids, team ids, country/region, hostname, parsed UA fields)
• Gauges Table: Simple resource snapshots (storage size, user count, etc.)
• Query-Time Aggregation: No write-time period fan-out — aggregate by any interval at query time
• Daily Materialized View: Pre-aggregated daily SummingMergeTree for fast billing queries
• Pluggable Adapters: ClickHouse (production) and Database (development/testing)
• In-Memory Buffering: Collect metrics and flush in batch for high-throughput scenarios
• Rich Queries: Filter, sort, paginate using Utopia\Query\Query objects
• Multi-Tenant: Shared tables with tenant isolation via string tenant IDs
• LowCardinality Columns: Country uses LowCardinality(String) for efficient storage
• Bloom Filter Indexes: Fast filtering on all event columns

Getting Started

Install using composer:

Using ClickHouse Adapter

Using Database Adapter

Multi-tenancy

Usage is stateless: every query/mutation takes the tenant as its first argument, and addBatch carries a tenant on each metric row (so one batch can span tenants). This makes a single Usage instance safe to share across tenants and coroutines.

Callers that only ever touch one tenant can bind it once with the Tenant decorator, which forwards to Usage with the tenant pre-filled (and stamps it onto every addBatch row):

Metric Types

Events (Additive)

Events are request-level metrics like bandwidth, executions, API calls. They are summed when aggregated.

Event-specific columns (see Metric::EVENT_COLUMNS): path, method, status, service, resource, resourceId, resourceInternalId, teamId, teamInternalId, country, region, hostname, osCode, osName, osVersion, clientType, clientCode, clientName, clientVersion, clientEngine, clientEngineVersion, deviceName, deviceBrand, deviceModel.

Strict tag keys

All keys passed in the tags array must map to a known event or gauge column (see Metric::EVENT_COLUMNS and Metric::GAUGE_COLUMNS). Unknown keys throw at write time — there is no JSON catch-all. To add a new dimension, widen the schema and bump the library version.

Gauges (Point-in-Time)

Gauges are resource snapshots like storage size, user count, file count. Last-write-wins semantics.

Gauge-specific columns (see Metric::GAUGE_COLUMNS): teamId, teamInternalId, resourceId, resourceInternalId.

Flushing

The accumulator exposes raw signals — count() (buffered entries) and elapsedSeconds() (seconds since last flush) — and leaves the flush policy to the caller.

Batch Writes

Canonical Samples

Billable inputs that must survive request retries use the separate immutable sample ledger. A sample's identity is derived from its environment, region, project and database internal IDs, member, generation, sequence and metric. Retrying the same identity and payload is safe. Reusing an identity with a different interval, value or event version is returned as a conflict. The event ID is SHA-256 over those identity fields in the documented order, each encoded as its decimal byte length, :, then its UTF-8 value. The payload hash uses the same encoding over event ID, UTC millisecond interval bounds, value and event version.

Each supplied sample row receives an adapter-owned random ingestion ID before its request is sent. getSampleWatermark() performs one bounded ClickHouse snapshot read and captures each visible ingestion ID bound to its canonical ID and payload hash for that stream and range. findSamples() admits only those exact entry fingerprints, so later inserts or changed rows cannot cross the boundary even when their server timestamps would be identical. A transport retry of the same request retains its fingerprint and is counted once; a new logical retry gets a new ingestion ID and is included only when visible to the watermark query.

Both the watermark evidence and findSamples() result are explicitly bounded. A result is complete only when neither bound is truncated and there are no conflicts, sequence gaps or interval-boundary discontinuities. Conflicting physical rows are never combined into a synthetic sample. The sample ledger does not make HTTP delivery or a producer's local spool durable; callers must retain a sample until the write is acknowledged and retry the identical payload.

Querying Metrics

Find with Query Objects

Totals

Time Series

Billing Queries (Daily MV)

The daily materialized view pre-aggregates events by metric + tenant + day for fast billing:

Purge

Architecture

ClickHouse Tables

Events Table Schema

Gauges Table Schema

Daily Table Schema

Creating Custom Adapters

Extend Utopia\Usage\Adapter and implement:

• getName(), setup(), healthCheck()
• addBatch(array $metrics, string $type, int $batchSize): bool (each metric carries its own tenant)
• find(string $tenant, array $queries, ?string $type): array
• count(string $tenant, array $queries, ?string $type, ?int $max): int
• sum(string $tenant, array $queries, string $attribute, string $type): int
• getTimeSeries(string $tenant, array $metrics, string $interval, string $startDate, string $endDate, array $queries, bool $zeroFill, ?string $type): array
• getTotal(string $tenant, string $metric, array $queries, ?string $type): int
• getTotalBatch(string $tenant, array $metrics, array $queries, ?string $type): array
• findDaily(string $tenant, array $queries): array
• sumDaily(string $tenant, array $queries, string $attribute): int
• sumDailyBatch(string $tenant, array $metrics, array $queries): array
• purge(string $tenant, array $queries, ?string $type): bool

All configuration — namespace, database, shared-tables mode, async inserts, query logging — is set once via the adapter constructor (see "Using ClickHouse Adapter" above). Adapters hold no per-request state; the tenant is passed explicitly on every call, so one instance is safe to share across tenants and coroutines.

System Requirements

Utopia Framework requires PHP 8.4 or later. We recommend using the latest PHP version whenever possible.

Copyright and license

The MIT License (MIT) http://www.opensource.org/licenses/mit-license.php