Download the PHP package flowpack/decoupledcontentstore without Composer

On this page you can find all versions of the php package flowpack/decoupledcontentstore. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:
If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

  • Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
  • To use Composer is sometimes complicated. Especially for beginners.
  • Composer needs much resources. Sometimes they are not available on a simple webspace.
  • If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
  • Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Please rate this library. Is it a good library?

Informations about the package decoupledcontentstore

Decoupled Content Store based on Redis

This is the 2nd generation of a Two-Stack CMS package for Neos.

This Package is used in production in a bigger instance.

The Content Store package is one part of a Two-Stack CMS solution with Neos. A Two-Stack architecture separates editing and publishing from the delivery of content. This is also an architecture that's suitable to+ integrate Neos content in various other systems without adding overhead during delivery.

The first iteration was not open source; developed jointly by Networkteam and Sandstorm and is in use for several large customers. The second iteration (this project) is developed from scratch, in an open-source way, based on the learnings of the first iteration. Especially the robustness has been greatly increased.

What does it do?

The Content Store package publishes content from Neos to a Redis database as immutable content releases. These releases can be atomically switched and a current release points to the active release.

The delivery layer in the Two-Stack architecture uses the current release and looks for matching URLs in the content store and delivers the pre-rendered content. A delivery layer is decoupled from the actual Neos CMS and can be implemented in any language or framework. It is also possible to integrate the delivery layer part in another software (e.g. a shop system) as an extension.

Features

This project is using the go-package prunner and its Flow Package wrapper as the basis for orchestrating and executing a content release.

Requirements

Start up prunner via the following command:

Copy the pipelines_template.yml file into your project and adjust it as needed (see below and the comments in the file for explanation).

Approach to Rendering

The following flow chart shows the rendering pipeline for creating a content release.

The above pipeline is implemented with prunner which is orchestrating the different steps.

Infrastructure

Here, we explain the different infrastructure and setup constraints for using the content store.

It is crucial that Redis is available via lowest latency for Neos AND the Delivery Layer. See the different setup scenarios below for how this can be done.

Minimal Setup

The minimal setup looks as follows:

In this case, the transfer phase does not need to do anything, and you need to configure Neos to use the cloud storage (e.g. via Flownative.Google.CloudStorage or Flownative.Aws.S3) for resources.

This is implemented in the default pipelines_template.yml.

This Setup should be used if:

If you use Cloud Asset Storage, ensure that you never delete assets from there. For Flownative.Aws.S3, you can follow the guide on "Preventing Unpublishing of Resources in the Target".

Manually Sync Assets to the Delivery Layer via RSync

If you can not to use a Cloud Asset Storage, there's a built-in feature to manually sync assets to the delivery layer(s) via RSync.

To enable this, you need to follow the following steps:

  1. Configure in Settings.yaml:

  2. In pipelines.yml, underneath 4) TRANSFER, comment-in the transfer_resources task.

Copy Content Releases to a different Redis instance

This Setup should be used if:

In this case, the content store Redis DB is explicitly synced by Neos to another Delivery layer.

To enable this feature, do the following:

  1. Configure the additional Content Stores in Settings.yaml underneath Flowpack.DecoupledContentStore.redisContentStores. The key is the internal identifier of the content store:

  2. In pipelines.yml, underneath 4) TRANSFER, comment-in and adjust the transfer_content task.

  3. In pipelines.yml, underneath 5) TRANSFER, comment-in the additional contentReleaseSwitch:switchActiveContentRelease commands.

Alternative: Redis Replication

Instead of the explicit synchronization described here, you can also use Redis Replication to synchronize the primary Redis to the other instances.

Using Redis replication is transparent to Neos or the Delivery Layer.

To be able to use Redis replication, the Redis secondary (i.e. the delivery-layer's instance) needs to connect to the primary Redis instance.

For the explicit synchronization described here, the Redis instances do not need to communicate directly with each other; but Neos needs to be able to reach all instances.

Incremental Rendering

As a big improvement for stability (compared to v1), the rendering pipeline does not make a difference whether it is a full or an incremental render. To trigger a full render, the content cache is flushed before the rendering is started.

Options

After changing an Asset (e.g. in the Media Module) an incremental rendering is triggered. You can opt out of this behavior by setting the following configuration: `

What happens if edits happen during a rendering?

If a change by an editor happens during a rendering, the content cache is flushed (by tag) as a result of this content modification. Now, there are two possible cases:

The 2nd case is a bit dangerous, in the sense that we need a re-render to happen soon; otherwise we would not converge to a consistent state.

For use cases like scheduling re-renders, prunner supports a concurrency limit (i.e. how many jobs can run in parallel) - and if this limit is reached, it supports an additional queue which can be also limited.

So the following lines from pipelines.yml are crucial:

So, if a content release is currently running, and we try to start a new content release, then this task is added to the queue (but not yet executed). In case there is already a rendering task queued, this gets replaced by the newer rendering task.

This ensures that we have at most one content release running at any given time; and at most one content-release in the wait-list waiting to be rendered. Additionally, we can be sure that scheduled content releases will be eventually executed, because that's prunner's job.

Extensibility

Custom pipelines.yml

Crafting a custom pipelines.yml is the main extension point for doing additional work (f.e. additional enumeration or rendering).

Custom Document Metadata, integrated with the Content Cache

Sometimes, you need to build additional data structures for every individual document. Ideally, you'll want this structure to be integrated with the content cache; i.e. only refresh it if the page has changed.

Performance-wise, it is clever to do this at the same time as the rendering itself, as the content nodes (which you'll usually need) are already loaded in memory. You can register a Flowpack\DecoupledContentStore\NodeRendering\Extensibility\DocumentMetadataGeneratorInterface in Settings.yaml:

When you implement this class, you can add additional Metadata which is serialized to the Neos content cache for every rendered document.

Often, you'll also want to add another contentReleaseWriter which reads the newly added metadata and adds it to the final content release. Read the next section how this works.

Custom Content Release Writer

You can completely define how a content release is laid out in Redis for consumption by your delivery layer.

By implementing a custom ContentReleaseWriter, you can specify how the rendered content is stored in Redis.

Again, this is registered in Settings.yaml:

Writing Custom Data to the Content Release

In case you write custom data to the content release (using $redisKeyService->getRedisKeyForPostfix($contentReleaseIdentifier, 'foo')), you need to register the custom key also in the settings:

This is needed so that the system knows which keys should be synchronized between the different content stores, and what data to delete if a release is removed.

Rendering additional nodes with arguments (e.g. pagination or filters)

If you render a paginated list or have filters (with a predictable list of values) that can be added to a document via arguments, you can implement a slot for the nodeEnumerated signal to enumerate additional nodes with arguments.

Note: Request arguments must be mapped to URIs via custom routes, since we do not support HTTP query parameters for rendered documents.

Example

Add a slot for the nodeEnumerated signal via Package.php:

Implement the slot and enumerate additional nodes depending on the node type:

The actual logic will depend on your use of the node. Having the actual filtering logic implemented in PHP is beneficial, because it allows you to use it in the rendering process as well as in the additional enumeration.

Extending the backend module

Using different sets of config

In some cases it might be necessary to make fundamental adjustments to some configuration properties that would be really hard to handle (safely, non-breaking) on the consuming site of the content store. Therefore we added the config property configEpoch that can contain a current and previous config version. The current value (that should be used on the consuming site) gets published to the content store.

We decided to save the configEpoch on content store level instead of content release level for simplicity reasons on the consuming site. If you need to switch back to an older release that was rendered with the previous config epoch version and would not match the currently published one, you may manually toggle between current and previous config epoch. There is a button for this in the backend module for each target content store. Obviously this button should be used with extra care as the config epoch needs to fit the current release at all times.

Example:

Development

We use esbuild combined with tailwind.css for building.

Rendering Deep Dive

TODO write

CacheUrlMappingAspect - * NOTE: This aspect is NOT active during interactive page rendering; but only when a content release is built

Debugging

If you need to debug single steps of the pipeline just run the corresponding commands from CLI, e.g. ./flow nodeEnumeration:enumerateAllNodes {{ .contentReleaseId }}.

Testing the Rendering

For executing behavioral tests, install the neos/behat package and run ./flow behat:setup. Then:

Behat also supports running single tests or single files - they need to be specified after the config file, e.g.

In case of exceptions, it might be helpful to run the tests with --stop-on-failure, which stops the test cases at the first error. Then, you can inspect the testing database and manually reproduce the bug.

Additionally, -vvv is a helpful CLI flag (extra-verbose) - this displays the full exception stack trace in case of errors.

TODO

Missing Features from old

data-url-next-page (or so) not supported

License

GPL v3


All versions of decoupledcontentstore with dependencies

PHP Build Version
Package Version
Requires php Version ^7.4 || ^8.0
neos/neos Version ^7.3 || ^8.0
albertofem/rsync-lib Version ~1.0
flowpack/prunner Version *
Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package flowpack/decoupledcontentstore contains the following files

Loading the files please wait ....