Download the PHP package railt/lexer without Composer
On this page you can find all versions of the php package railt/lexer. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Package lexer
Short Description Fast implementation of the stateful and stateless lexers
License MIT
Homepage http://railt.org
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package lexer
Lexer
Note: All questions and issues please send to https://github.com/railt/railt/issues
Note: Tests can not always pass correctly. This may be due to the inaccessibility of PPA servers for updating gcc and g++. The lexertl build requires the support of a modern compiler inside Travis CI. In this case, a gray badge will be displayed with the message "Build Error".
In order to quickly understand how it works - just write ~4 lines of code:
This example will read the source text and return the set of tokens from which it is composed:
1) T_DIGIT with value "23"
2) T_DIGIT with value "42"
The second argument to the Factory class is the list of token names that are ignored in the lex method result.
That's why we only got two significant tokens T_DIGIT. Although this is not entirely true,
the answer contains a T_EOI (End Of Input) token which can also be removed from the output
by adding an array of the second argument of Factory class.
...and now let's try to understand more!
The lexer contains two types of runtime:
1) Basic - Set of algorithms with one state.
2) Multistate - Set of algorithms with the possibility of state transition between tokens.
In connection with the fact that there were almost no differences in speed between several implementations (Stateful vs Stateless) of the same algorithm, it was decided to abandon the immutable stateful lexers.
In order to tokenize the source text, you must use the method ->lex(...), which returns
iterator of the TokenInterface objects.
A TokenInterface provides a convenient API to obtain information about a token:
Drivers
The factory returns one of the available implementations, however you can create it yourself.
Basic
NativeRegex
NativeRegex implementation is based on the built-in php PCRE functions.
Lexertl
Experimental lexer based on the C++ lexertl library. To use it, you need support for Parle extension.
Be careful: The library is not fully compatible with the PCRE regex syntax. See the official documentation.
Multistate
This functionality is not yet implemented.
All versions of lexer with dependencies
ext-spl Version *
ext-pcre Version *
ext-mbstring Version *
railt/io Version 1.4.*|1.4.x-dev