Download the PHP package quarks-tech/protoc-gen-validate without Composer

protoc-gen-validate (PGV)

This project is currently in alpha. The API should be considered unstable and likely to change

PGV is a protoc plugin to generate polyglot message validators. While protocol buffers effectively guarantee the types of structured data, they cannot enforce semantic rules for values. This plugin adds support to protoc-generated code to validate such constraints.

Developers import the PGV extension and annotate the messages and fields in their proto files with constraint rules:

Executing protoc with PGV and the target language's default plugin will create Validate methods on the generated types:

Usage

Dependencies

• go toolchain (≥ v1.7)
• protoc compiler in $PATH
• protoc-gen-validate in $PATH
• official language-specific plugin for target language(s)
• Only proto3 syntax is currently supported. proto2 syntax support is planned.

Installation

Installing PGV can currently only be done from source:

Parameters

• lang: specify the target language to generate. Currently, the only supported options are:
  • go
  • cc for c++ (partially implemented)
  • java
• Note: Python works via runtime code generation. There's no compile-time generation. See the Python section for details.

Examples

Go

Go generation should occur into the same output path as the official plugin. For a proto file example.proto, the corresponding validation code is generated into ../generated/example.pb.validate.go:

All messages generated include the following methods:

• Validate() error which returns the first error encountered during validation.
• ValidateAll() error which returns all errors encountered during validation.

PGV requires no additional runtime dependencies from the existing generated code.

Note: by default example.pb.validate.go is nested in a directory structure that matches your option go_package name. You can change this using the protoc parameter paths=source_relative:.. Then --validate_out will output the file where it is expected. See Google's protobuf documentation or packages and input paths or parameters for more information.

There's also support for the module=example.com/foo flag described here.

Java

Java generation is integrated with the existing protobuf toolchain for java projects. For Maven projects, add the following to your pom.xml or build.gradle.

Python

The python implementation works via JIT code generation. In other words, the validate(msg) function is written on-demand and exec-ed. An LRU-cache improves performance by storing generated functions per descriptor.

The python package is available on PyPI.

To run validate(), do the following:

You can view what code has been generated by using the print_validate() function.

Constraint Rules

The provided constraints are modeled largerly after those in JSON Schema. PGV rules can be mixed for the same field; the plugin ensures the rules applied to a field cannot contradict before code generation.

Check the constraint rule comparison matrix for language-specific constraint capabilities.

Numerics

All numeric types (float, double, int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, sfixed32, sfixed64) share the same rules.

• const: the field must be exactly the specified value.

• lt/lte/gt/gte: these inequalities (<, <=, >, >=, respectively) allow for deriving ranges in which the field must reside.

  Inverting the values of lt(e) and gt(e) is valid and creates an exclusive range.

• in/not_in: these two rules permit specifying allow/denylists for the values of a field.

• ignore_empty: this rule specifies that if field is empty or set to the default value, to ignore any validation rules. These are typically useful where being able to unset a field in an update request, or to skip validation for optional fields where switching to WKTs is not feasible.

Bools

• const: the field must be exactly the specified value.

Strings

• const: the field must be exactly the specified value.

• len/min_len/max_len: these rules constrain the number of characters (Unicode code points) in the field. Note that the number of characters may differ from the number of bytes in the string. The string is considered as-is, and does not normalize.

• min_bytes/max_bytes: these rules constrain the number of bytes in the field.

• pattern: the field must match the specified RE2-compliant regular expression. The included expression should elide any delimiters (ie, /\d+/ should just be \d+).

• prefix/suffix/contains/not_contains: the field must contain the specified substring in an optionally explicit location, or not contain the specified substring.

• in/not_in: these two rules permit specifying allow/denylists for the values of a field.

• ignore_empty: this rule specifies that if field is empty or set to the default value, to ignore any validation rules. These are typically useful where being able to unset a field in an update request, or to skip validation for optional fields where switching to WKTs is not feasible.

• well-known formats: these rules provide advanced constraints for common string patterns. These constraints will typically be more permissive and performant than equivalent regular expression patterns, while providing more explanatory failure descriptions.

Bytes

Literal values should be expressed with strings, using escaping where necessary.

• const: the field must be exactly the specified value.

• len/min_len/max_len: these rules constrain the number of bytes in the field.

• pattern: the field must match the specified RE2-compliant regular expression. The included expression should elide any delimiters (ie, /\d+/ should just be \d+).

• prefix/suffix/contains: the field must contain the specified byte sequence in an optionally explicit location.

• in/not_in: these two rules permit specifying allow/denylists for the values of a field.

• ignore_empty: this rule specifies that if field is empty or set to the default value, to ignore any validation rules. These are typically useful where being able to unset a field in an update request, or to skip validation for optional fields where switching to WKTs is not feasible.

• well-known formats: these rules provide advanced constraints for common patterns. These constraints will typically be more permissive and performant than equivalent regular expression patterns, while providing more explanatory failure descriptions.

Enums

All literal values should use the numeric (int32) value as defined in the enum descriptor.

The following examples use this State enum

• const: the field must be exactly the specified value.

• defined_only: the field must be one of the specified values in the enum descriptor.

• in/not_in: these two rules permit specifying allow/denylists for the values of a field.

Messages

If a field contains a message and the message has been generated with PGV, validation will be performed recursively. Message's not generated with PGV are skipped.

• skip: this rule specifies that the validation rules of this field should not be evaluated.

• required: this rule specifies that the field cannot be unset.

Repeated

• min_items/max_items: these rules control how many elements are contained in the field

• unique: this rule requires that all elements in the field must be unique. This rule does not support repeated messages.

• items: this rule specifies constraints that should be applied to each element in the field. Repeated message fields also have their validation rules applied unless skip is specified on this constraint.

• ignore_empty: this rule specifies that if field is empty or set to the default value, to ignore any validation rules. These are typically useful where being able to unset a field in an update request, or to skip validation for optional fields where switching to WKTs is not feasible.

Maps

• min_pairs/max_pairs: these rules control how many KV pairs are contained in this field

• no_sparse: for map fields with message values, setting this rule to true disallows keys with unset values.

• keys: this rule specifies constraints that are applied to the keys in the field.

• values: this rule specifies constraints that are be applied to each value in the field. Repeated message fields also have their validation rules applied unless skip is specified on this constraint.

• ignore_empty: this rule specifies that if field is empty or set to the default value, to ignore any validation rules. These are typically useful where being able to unset a field in an update request, or to skip validation for optional fields where switching to WKTs is not feasible.

Well-Known Types (WKTs)

A set of WKTs are packaged with protoc and common message patterns useful in many domains.

Scalar Value Wrappers

In the proto3 syntax, there is no way of distinguishing between unset and the zero value of a scalar field. The value WKTs permit this differentiation by wrapping them in a message. PGV permits using the same scalar rules that the wrapper encapsulates.

Message Rules can also be used with scalar Well-Known Types (WKTs):

Anys

• required: this rule specifies that the field must be set

• in/not_in: these two rules permit specifying allow/denylists for the type_url value in this field. Consider using a oneof union instead of in if possible.

Durations

• required: this rule specifies that the field must be set

• const: the field must be exactly the specified value.

• lt/lte/gt/gte: these inequalities (<, <=, >, >=, respectively) allow for deriving ranges in which the field must reside.

  Inverting the values of lt(e) and gt(e) is valid and creates an exclusive range.

• in/not_in: these two rules permit specifying allow/denylists for the values of a field.

Timestamps

• required: this rule specifies that the field must be set

• const: the field must be exactly the specified value.

• lt/lte/gt/gte: these inequalities (<, <=, >, >=, respectively) allow for deriving ranges in which the field must reside.

  Inverting the values of lt(e) and gt(e) is valid and creates an exclusive range.

• lt_now/gt_now: these inequalities allow for ranges relative to the current time. These rules cannot be used with the absolute rules above.

• within: this rule specifies that the field's value should be within a duration of the current time. This rule can be used in conjunction with lt_now and gt_now to control those ranges.

Message-Global

• disabled: All validation rules for the fields on a message can be nullified, including any message fields that support validation themselves.

• ignored: Don't generate a validate method or any related validation code for this message.

OneOfs

• required: require that one of the fields in a oneof must be set. By default, none or one of the unioned fields can be set. Enabling this rules disallows having all of them unset.

Development

PGV is written in Go on top of the protoc-gen-star framework and compiles to a standalone binary.

Dependencies

All PGV dependencies are currently checked into the project. To test PGV, protoc must be installed, either from source, the provided releases, or a package manager. The official protoc plugin for the target language(s) should be installed as well.

Make Targets

• make build: generates the constraints proto and compiles PGV into $GOPATH/bin

• make lint: runs static-analysis rules against the PGV codebase, including golint, go vet, and gofmt -s

• make testcases: generates the proto files in /tests/harness/cases. These are used by the test harness to verify the validation rules generated for each language.

• make harness: executes the test-cases against each language's test harness.

Run all tests under Bazel

Ensure that your PATH is setup to include protoc-gen-go and protoc, then:

Docker

PGV comes with a Dockerfile for consistent development tooling and CI. The main entrypoint is make with quick as the default target. This repo should be volumed into /go/src/github.com/envoyproxy/protoc-gen-validate for the proper behavior.