Download the PHP package inetprocess/neuralyzer without Composer
On this page you can find all versions of the php package inetprocess/neuralyzer. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download inetprocess/neuralyzer
More information about inetprocess/neuralyzer
Files in inetprocess/neuralyzer
Package neuralyzer
Short Description Library and CLI for Data anonymization
License GPL-2.0-or-later
Informations about the package neuralyzer
edyan/neuralyzer
Summary
This project is a library and a command line tool that anonymizes a database by updating data or generating fake data (update vs insert). It uses Faker to generate data from rules defined in a configuration file.
As it can do row per row or use batch mechanisms, you can load tables with dozens of millions of fake records.
It uses Doctrine DBAL to abstract interactions with databases. It's then supposed to be able to work with any database type. Currently it works (tested extensively) with MySQL, PostgreSQL and SQLServer.
~~Neuralyzer has an option to clean tables by injecting a DELETE FROM
with a WHERE
critera
before launching the anonymization (see the config parameters delete
and delete_where
).~~
Neuralyzer had an option to clean tables but it's now managed by pre and post actions :
Installation as a library
Installation as an executable
You can even download the executable directly (example with v3.1):
Usage
The easiest way to use that tool is to start with the command line tool.
After cloning the project and running a composer install
, try:
Generate the configuration automatically
Neuralyzer is able to read a database and generate the configuration for you.
The command config:generate
accepts the following options:
Example
That produces a file which looks like:
You have to modify the file to change its configuration. For example, if you need to remove data while anonymizing and change the language (see Faker's doc for available languages), do :
INFO: You can also use delete in standalone, without anonymizing anything. That will delete everything in books:
If you wanted to delete everything then insert 1000 new books:
Run the anonymizer
To run the anonymizer, the command is simply "run" and expects:
Example
That produces that kind of output:
WARNING: On a huge table, --sql
will produce a HUGE output. Use it for debugging purpose.
Library
The library is made to be integrated with any Tool such as a CLI tool. It contains:
- A Configuration Reader and a Configuration Writer
- A Guesser
- A DB Anonymizer
Guesser
The guesser is the central piece of the config generator. It guesses, according to the field name or field type what type of faker method to apply.
It can be extended very easily as it has to be injected to the Writer.
Configuration Writer
The writer is helpful to generate a yaml file that contains all tables and fields from a DB. A basic usage could be the following:
If you need, you can protect some cols (with regexp) or tables:
Configuration Reader
The configuration Reader is the exact opposite of the Writer. Its main job is to validate that the configuration of the yaml file is correct then to provide methods to access its parameters. Example:
DB Anonymizer
The only anonymizer currently available is the DB one. It expects a PDO and a Configuration Reader objects:
Once initialized, the method that anonymize the table is the following:
Parameters:
Entity
: such as table name (required)Callback
(callable / optional) to use a progress bar for example
A few options can be set by calling :
Full Example:
Pre and Post Actions
You can set an array of pre_actions
and post_actions
that will be
executed before and after neuralyzer starts to anonymize an entity.
These actions are actually symfony expressions (see Symfony Expression Language)
that rely on Services. These Services are loaded from the Service/
directory.
For now there is only one service : Database
that contains a method query
usable like that :
db.query("DELETE FROM table")
.
Configuration Reference
bin/neuralyzer config:example
provides a default configuration with all parameters explained :
Custom application logic
When using custom doctrine types doctrine will produce an error that the type is not know. This can be solved by providing a bootstrap file to register the custom doctrine type.
bootstrap.php
Then provide the bootstrap file to the run command:
Development
Neuralyzer uses Robo to run its tests (via Docker) and build its phar.
Clone the project, run composer install
then...
Run the tests
- Change the
--wait
option if you have a lot of errors because DB is not ready. - Change the
--php
option for7.2
or7.4
- Set
--no-coverage
if you want to disable PHPUnit Code Coverage.
With MySQL
With PostgreSQL 9, 10 and 11 (12 also works)
With SQL Server
Warning : 2 tests fail, because of strange behaviors of SQL Server ... or Doctrine / Dbal. PHPUnit can't compare 2 Datasets because the fields are not in the same order.
Build a release (with a phar and a git tag)
Build the phar only
Improve code quality with phpinsights
Update dependencies to make sure it'll work with PHP 7.2
All versions of neuralyzer with dependencies
ext-pdo Version *
doctrine/dbal Version ^2
fakerphp/faker Version ^1
symfony/config Version ^5
symfony/console Version ^5
symfony/dependency-injection Version ^5
symfony/expression-language Version ^5
symfony/finder Version ^5
symfony/stopwatch Version ^5
symfony/yaml Version ^5