Download the PHP package rindow/rindow-rl-agents without Composer

On this page you can find all versions of the php package rindow/rindow-rl-agents. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:
If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

  • Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
  • To use Composer is sometimes complicated. Especially for beginners.
  • Composer needs much resources. Sometimes they are not available on a simple webspace.
  • If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
  • Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Please rate this library. Is it a good library?

Informations about the package rindow-rl-agents

Rindow RL Agents

Reinforcement-learning agents and training utilities for Rindow Neural Networks.

The package provides discrete and continuous-control algorithms, replay and rollout buffers, training runners, action-mask support, checkpoint persistence, and CPU/GPU-compatible tensor operations.

Features

Installation

Install the package with Composer:

Composer installs Rindow RL Agents and its declared runtime dependencies. Your application only needs the generated Composer autoloader:

The exact PHP extensions and native libraries required for acceleration depend on the selected Rindow Math Matrix backend. A CPU backend is sufficient to use the package.

Quick start: CartPole with DDQN

The following example assumes that a CartPole environment implementing Interop\Polite\AI\RL\Environment is available. The environment and evaluation environment are separate so evaluation does not disturb the training trajectory.

Reinforcement learning is stochastic. Treat the values above as a complete API example, not as a convergence guarantee for every backend or random seed.

Choosing an algorithm

Algorithm Action space Data model Typical use
A2C Discrete or continuous On-policy rollout Compact synchronous actor-critic
DQN/DDQN Discrete Replay buffer Discrete value-based control
PPO Discrete or continuous On-policy rollout Clipped policy optimization
DDPG Continuous Replay buffer Deterministic continuous control
SAC+gSDE Continuous Replay buffer Entropy-regularized continuous control
REINFORCE Discrete Complete episode Basic Monte Carlo policy gradient
Q-learning Discrete Per transition Linear tile-coded value learning
True Online Sarsa(λ) Discrete Per transition On-policy learning with Dutch traces

CPU and GPU execution

Rindow Neural Networks selects the numerical backend. For example, an OpenCL GPU backend can be selected before starting PHP:

Use the agent backend's NDArray objects for observations, replay storage, and action masks. The bundled device wrappers transfer environment observations to the configured device and actions back to the host environment.

Environment stepping and runner control flow remain on the PHP host. Batched network operations, losses, reductions, masking, replay gathers, and optimizer updates execute on the selected backend.

CPU and GPU runs need not follow identical stochastic trajectories. Compare multiple seeds and task-level statistics rather than expecting bitwise-identical training logs.

Action masks

DQN, A2C, PPO, Q-learning, and True Online Sarsa(λ) support discrete action masks. Configure dictionary fields when creating the agent:

Each mask must contain one value per action and enable at least one action. Disabled actions are excluded during both exploration and greedy selection.

Reward transformation

Several runners accept a rewardFunction callback. It changes the reward stored for training without changing the raw reward reported by evaluation. This is useful for controlled reward-shaping experiments, but arbitrary shaping can change the optimal policy.

Time-limit truncation and true MDP termination are handled separately. The bundled runners normally bootstrap at truncation and use zero bootstrap value at true termination.

Checkpoints

Agents expose a common persistence pattern:

Neural checkpoints include the online models and any target networks needed to resume training. SAC also stores its learned entropy coefficient. Existing model buffers are updated in place so checkpoints and target synchronization remain compatible with compiled Rindow Neural Networks function pipelines.

Samples

The samples/ directory contains complete programs for:

It includes vector and image-observation examples where applicable. Run samples from the project root:

Some samples accept RL_* environment variables for seeds, training length, evaluation intervals, model paths, and demo control. See the constants and environment lookups at the top of each sample.

Documentation

The full reference manual starts at docs/index.md.

Testing

Install development dependencies and run PHPUnit:

On Windows:

The unit suite checks analytical update targets, losses, GAE, action masks, buffer layouts, action bounds, target-network synchronization, Polyak updates, and checkpoint behavior. Learning convergence is intentionally kept outside the unit-test contract; the scripts under tests/ are experiment and smoke-test drivers.

Package

Composer package name: rindow/rindow-rl-agents

Primary namespace: Rindow\RL\Agents


All versions of rindow-rl-agents with dependencies

PHP Build Version
Package Version
Requires php Version ^8.1
rindow/rindow-neuralnetworks Version ^2.2
rindow/rindow-math-matrix-matlibffi Version ^1.1
interop-phpobjects/polite-ai Version ^1.1.1
Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package rindow/rindow-rl-agents contains the following files

Loading the files please wait ...