Download the PHP package oliverde8/php-etl without Composer
On this page you can find all versions of the php package oliverde8/php-etl. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download oliverde8/php-etl
More information about oliverde8/php-etl
Files in oliverde8/php-etl
Package php-etl
Short Description A small etl coded in php, to extract/transform/load Edit with ease.
License MIT
Informations about the package php-etl
PHP ETL Chain
Php etl is a ETL that will allow transformation of data in a simple and efficient way.
It comes in 2 php components :
Rule engine
The rule engine allows to have configuration based transformations to transform a particular data.
This is integrated inside the ETL with the RuleTransformOperation
.
The rule engine can be used in standalone, see docs
The ETL Chain
An ETL chain is described by chain operations and Items. The chain operation holds the logic, this means it can:
- Extract data, (possibly duplicate the item)
- Transform data
- Load the data somewhere.
Data will propagate throught the ETL operations using Items, there can be different type of items (We will detail this later.)
Chain operations consumes one item in order to create a new item or an iterator. The purpose is to always process data individually. For example if we are importing customers we try to never have the data of more than one customer in memory.
Examples of how it works
We will have more detailed real use cases with sample data a bit further in the document.
In the simplest case the chains receive an iterator containing 2 items in input, both items are processed by each chain operation. This could be for example a list of customer. Each operation changes the items.
In the following example the iterator sends a single item. The first operation will then send GroupedItems containing 2 items. The first item could be a customer, and then we fetch each order of the customer in the operation1.
We can also group items, to make aggregations. The chain receives an iterator containg 2 items, the first operation processes both items. It breaks the chain for the first item, and returns an aggregation of item1 & item 2. This can be used to count the number of customers. This kind of grouping can use more memory and should therefore be used with care.
Chains can also be split, this would allow 2 different operations to be executed on the same item.
Creating a chain.
There are 2 ways of writing a chain, either you code it; or you describe the chain in a yaml file.
- Using php code to initiate each operation yourself, this is not recommended!
- Using yaml files to descrive the chain.
Please see the describe chains using yaml configurations, you can check the doc on sub chains and more complex cases as well
Creating you own operations.
Please refer to the Custom Opertions doc
FAQ
Please refer to the FAQ
All versions of php-etl with dependencies
oliverde8/associative-array-simplified Version ^1.0
psr/log Version ^1.0|^2.0|^3.0
symfony/expression-language Version ^4.3|^5.0|^6.0|^7.0
symfony/filesystem Version ^5.4|^6.0|^7.0
symfony/validator Version ^4.3|^5.0|^6.0|^7.0
symfony/yaml Version ^4.3|^5.0|^6.0|^7.0