Download the PHP package koco/messenger-kafka without Composer

Symfony Messenger Kafka Transport

This bundle aims to provide a simple Kafka transport for Symfony Messenger. Kafka REST Proxy support coming soon.

Compatibility

Supports Symfony ^5.4 || ^6.4 || ^7.4 || ^8.1, with GitHub Actions coverage for each branch. PHP requirements are 7.4+ for Symfony 5.4, 8.1+ for Symfony 6.4, 8.2+ for Symfony 7.4, and 8.4+ for Symfony 8.1. Symfony 5.4 remains supported here while it receives upstream security fixes.

Applications on older Symfony versions can continue using the published v0.18 release (composer require koco/messenger-kafka:^0.18), provided their other dependency constraints allow it. Those versions are no longer supported by this development branch.

Install the rdkafka PHP extension to use the native Kafka transport. REST Proxy sending uses a PSR-18 HTTP client and PSR-17 factories and supports PSR-7 1.x and 2.x.

Tests

The badge above reports the combined result of all six PHP/Symfony test jobs and the coding-style check. GitHub Actions runs the unit and Kafka integration tests across the PHP/Symfony compatibility matrix, plus a separate coding-style check. Each test job uploads JUnit results and Clover coverage reports as downloadable artifacts, including when tests fail. Runs start on branch pushes, pull requests, or manually from the Actions tab.

To run the style check and unit tests locally, use composer test. To include integration tests, start a Kafka broker at 127.0.0.1:9092 and run vendor/bin/simple-phpunit.

Installation

Applications that use Symfony Flex

Open a command console, enter your project directory and execute:

Applications that don't use Symfony Flex

After adding the composer requirement, enable the bundle by adding it to the list of registered bundles in the config/bundles.php file of your project:

Configuration

DSN

Specify a DSN starting with either kafka:// or kafka+ssl://. Multiple brokers are separated by ,.

• kafka://my-local-kafka:9092
• kafka+ssl://my-staging-kafka:9093
• kafka+ssl://prod-kafka-01:9093,kafka+ssl://prod-kafka-02:9093,kafka+ssl://prod-kafka-03:9093

Example

The configuration options for kafka_conf and topic_conf can be found here. It is highly recommended to set enable.auto.offset.store to false for consumers. Otherwise, every message will be acknowledged, regardless of any error thrown by the message handlers.

Serializer

You will most likely want to implement your own Serializer. Please see: https://symfony.com/doc/current/messenger.html#serializing-messages

The fields key, headers, and body are available in the decode() and encode() methods.

How do I work with Avro?

Same as with the basic example above, you need to build your own serializer. Within the decode() and encode() you can make use of flix-tech/avro-serde-php.

What about the Confluent Schema Registry?

To connect with Schema Registry and control various settings, you can use this bundle:

And configure it to match your setup:

Please see https://github.com/KonstantinCodes/avro-regy for the full documentation.