Download the PHP package mathielen/import-engine without Composer
On this page you can find all versions of the php package mathielen/import-engine. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Informations about the package import-engine
Mathielen Import Engine
Full-blown importer stack for importing almost any data into your application. Can be used for exports, too.
Introduction
This library implements some high-level functionality based on the great Ddeboer Data Import library. As the Data-Import library offers a great toolkit to implement a data import/export process in a quick and clean way, there is still a lot of work to do to have a full blown importer set up for your application. This library helps you with that.
If you are developing a Symfony2 project, you may want to use the comfortable configuration of the ImportEngineBundle.
Features
- A Storage Abstraction-layer that supports nice features like automatic delimiter-discovering or processing compressed files. Currently these storages are supported:
- Structured Files
- CSV, XML, JSON, Excel
- File may be compressed
- Doctrine2 queries
- Service endpoints
- Storage Provisioning. Provide a list of possible storage-containers for your import. I.e. local files, remote files, uploaded files, database connections, service endpoints and more.
- A mapping sub-system, for building various mappings for your import: field-field, field-converter-field, field-converter-object and more.
- Automatic mapping into object tree's using the JMSSerializer
- Source (read) and Target (write) validation using Symfony Validation. Annotations can be used.
- Integrated Eventsystem using Symfony EventDispatcher
- Keeping almost every flexibility that is offered by the Ddeboer Data Import library.
- Well-tested code.
Installation
This library is available on Packagist. The recommended way to install it is through Composer:
Then include Composer’s autoloader:
If you want to make use of excel files, please also make sure to include phpoffice/phpexcel in your project:
Quickstart
Import an arbitrary file into your system
Using the *Provider facilities enables you to let the importer-system figure out what format the file has and what abstraction-classes should be used.
More examples
Have a look at: https://github.com/mathielen/import-engine/tree/master/tests/functional/Mathielen/ImportEngine
Usage
Terminology
- An Importer is the basic definition of the whole import-process. It says what may be imported and where to. It consists of:
- (optional) A StorageProvider, that represents a "virtual file system" for selecting a SourceStorage
- (optional) A SourceStorage that may be a file, a database table, an array, an object-tree, etc
- A TargetStorage that may be a file, a database table, an array, an object-tree, etc
- A Mapping, which may contain converters, field-mappings, etc
- A Validation, that may contain validation-rules for data read from the SourceStorage and/or validation-rules for data that will be written to the TargetStorage.
- An Logging or other interactions within the process.
- An Mapping.
- The ImportRunner is used to process the Import.
- Every run of an Import is represented by an ImportRun
StorageProvider
StorageProviders represents a "virtual file system" for selecting a SourceStorage that can be used as a source or target of the import.
FinderFileStorageProvider
Using the Symfony Finder Component as a collection of possible files that can be imported.
DoctrineQueryStorageProvider
You can use specific Doctrine Queries or only Entity-Classnames (the query will be SELECT * FROM
UploadFileStorageProvider
You can use a Provider to facilitate a File-Upload.
Automatic CSV Delimiter Discovery for FileStorageProviders
FileStorageProviders may use StorageFactories for constructing Storage objects. By default the FormatDiscoverLocalFileStorageFactory is used. This StorageFactory uses a MimeTypeDiscoverStrategy to determine the mime-type of the selected file and use it to create the correct storage-handler. You can change this behavior or extend it. There is a CsvAutoDelimiterTypeFactory that you can use to automaticly guess the correct delimiter of a CSV file.
This way any file that has the text/plain mime-type will be passed to the CsvAutoDelimiterFormatFactory to determine the delimiter.
Storage
A storage is a container of data. Storages provide a reader and writer implementation for itself.
Validation
You can get the source and target validation errors with:
Source data validation
Target data Validation
ClassValidatorFilter
You can use the ClassValidatorFilter to map the data to an object-tree and validate the objects (using annotations, or differently configurated validation rules). Therefore you must provide an ObjectFactory. There is a JmsSerializerObjectFactory you may want to use.
Importer
Import / Source Storage
You can either use a StorageProvider (see above) and set the selection-id or you can use a specific Storage-Handler directly:
Mapping
Also see orginal documentation here.
Converting fields
There are a some field-level build-in converters available:
- upperCase
- lowerCase
- @TODO
Custom fieldlevel-converting
You have to register more complex converters to the importer for selecting them in your import.
Custom rowlevel-converting
Like the fieldlevel converters, you have to register your converters first.
ImportRunner
For running a configured Import you need an ImportRunner. Internally the ImportRunner builds a workflow and runs it. You can change the way how the workflow is built by supplying a different WorkflowFactory.
ImportRun statistics
If you use the DefaultWorkflowFactory with your ImportRunner you get basic statistics from dryRun() and run() invocations.
Eventsystem
You can interact with the running import via the Symfony Eventdispatcher.
License
Import-Engine is released under the MIT license. See the LICENSE file for details.