Download the PHP package tanoconsulting/datavalidatorbundle without Composer

Data Validator Bundle

Goals

Allow checking integrity of data in a database, going beyond what the database schema definition enforces.

Allow checking the integrity of a set of files (WIP).

Usecase

There are many scenarios in which usage of the constraints configured in a database schema is not sufficient to enforce data integrity, such as f.e.:

• the db engine in use not supporting advanced/complex data validation constraints
• the db engine in use does support advanced data validation constraints, but those are not being used
• data integrity constraints which are too complex to express easily using the db engine
• db native constraints having been disabled for speed during mass import operations
• constraints having been implemented in application code, with multiple apps writing to the database

In all those cases, a separate tool which can validate that the data stored in the database adheres to a set of rules can come in handy.

Requirements

• php 7.3 or later
• a database supported by Doctrine DBAL (2.11 or 3.0 or later)
• Symfony components: see composer.json

Quick start

1. the set of constraints can be defined in a yaml or json file. This sample shows the supported syntax, using yaml:

2. run the validation command

   php bin/console datavalidator:validate:database --config-file=<my_schema_constraints.yaml>

   This presumes that your application has set up a database connection configuration doctrine named default. If that is not the case, you can run:

   php bin/console datavalidator:validate:database --config-file=<my_schema_constraints.yaml> --database=<mysql://user:pwd@localhost/mydb>

   If you want to list the validations constraints without validating them run:

   php bin/console datavalidator:validate:database --config-file=<my_schema_constraints.yaml> --dry-run

   By default the results show the number of database rows found which violate each constraint. To see the data of those rows instead, use:

   php bin/console datavalidator:validate:database --config-file=<my_schema_constraints.yaml> --display-data

Constraints currently supported

• foreign key definitions
• custom sql queries

See the doc/samples folder for examples constraints of well-known applications' database schemas.

More advanced usage

Defining validation constraints in your code

Instead of using a dedicated configuration file on the command line, you can configure the validation constraints in code, either:

• by setting a value to configuration parameter data_validator.constraints.database, or
• by tagging services with the data_validator.constraint_provider.database tag. Those services will have to implement a public method getConstraintDefinitions() that returns all the relevant constraints definitions

Creating your own constraint types

WIP

Troubleshooting

• Use the -v command line option to see details of execution

• If the execution of the constraint validation is taking a long time, you can use CTRL-C to stop execution halfway: the script will exit gracefully printing any violation found up to that point

• To avoid excessive memory usage from large queries, when running Symfony in "debug mode", such as commonly for "dev" envs, add the --no-debug option to your commands. If possible, use a non-debug Symfony env.

• If you still get an 'allowed memory size' fatal error, run the commands with php -d memory_limit=-1.

FAQ

• Q: can I run the validations in a Controller or Event instead of a cli command? A: technically yes, but it is generally not recommended, as the database queries used for validating the whole data set might take long to execute

Thanks

Code based on the Symfony/Validator component; thanks to all its developers!