Download the PHP package remp/beam-module without Composer

Beam

Beam admin provides a way to display real time usage stats on your website, aggregated article/author/conversion data and allows you to create user segments based on the tracked data.

Technical description

Dashboard

Dashboard provides you with realtime data about your website's traffic. This data include:

• Number of users currently active on your website (based on measured activity in the last 10 minutes)
• Chart of pageviews split by source medium (e.g. email, search engine, internal traffic..) compared with the data from previous period (e.g. today vs this day last week).
• List of currently visited articles with extra properties as number of total visits, conversions and time people spent reading the article.

All of these statistics are provided realtime and updated every couple of seconds automatically.

For dashboard to work correctly, you need to:

• Include tracking JS snippet on your website (this will populate today's chart)
• Push article metadata to Beam via article tracking API
• Enable timespent tracking within this snippet (rempConfig.tracker.timeSpent = { "enabled": true} ).

Note: Timespent tracking is not mandatory. Dashboard can utilize regular pageview data instead to calculate number of concurrents. If you don't plan to track timespent, please see this Telegraf configuration options so you can switch to pageview-based concurrents instead.

Dashboard allows you to display detail of each article with pageview-related chart containing histogram of visits split by traffic source medium (same as in the main dashboard) and also displays article-related events.

Source medium labels

Optionally, you can configure labels for source mediums of your pageviews. Labels will be shown in dashboard traffic (and article detail traffic) instead of real tracked values.

Currently, you can do it directly by editing database table referer_medium_labels.

Accounts

Accounts represent an access point to Beam data. Currently it's not utilized in any way, but in the future you'll be able to configure access to data based on allowed accounts per each user.

As of now, it's OK to create one single account for your organization (e.g. with the name of your newspaper).

Properties

Property is a unit that you want to track (website, subapplication, mobile app). Account can have multiple properties. In the future, Beam will allow to data for selected property.

Each property has a token that needs to be used within JS snippet (rempConfig.token).

Segments

Beam is able to provide behavioral segments based on the events tracked to the system by JS snippet and by backend APIs.

Regular segments

Regular segments can be created via web UI. Segment builder tool allows you to select multiple conditions based on which the users and browsers (implying that also anonymous users are segmentable) will be checked against specific segment.

Segment builder tool always displays only options based on events that were already tracked. If you want to create a segment with some specific condition, there should be at least one event in the system matching the condition.

Author segments

If some of your readers are interested only in specific of authors, Beam can identify such users and generate authors segments. Each author segment contain users, which are returning back and reading mostly the author that the segment belongs to.

As calculation of these segments it's computation intensive operation, author segments are computed on backend by running php artisan segments:compute-author-segments.

Each segment contains users and browsers assigned to it according to criteria, which can be adjusted in Beam admin settings page (/author-segments/configuration).

The command is not run by default (therefore no author segments exist). One has to run it manually when needed or schedule it to recompute segments periodically. We recommend to schedule computation of segment every night and we included Cron snippet within Scheduled events section

Entities

Entities are experimental feature allowing you to define and then track objects into Beam. It's designed to help you eventually create segments based on data in your system - by pushing the necessary data to Beam first.

Let's explain the feature on the example of tracking and generating segments based on delivery of print to your users.

To start using entities, first run EntitySeeder to populate necessary items into the database.

Then, visit /entities and define what's structure of your entities. In our example, this new entity could be called print_delivery and the parent entity user (as we want to track it for each user). Each entity has to belong to some other entity - either to default user entity or to one of your entities.

We could add a couple of parameters for this entity:

• delivery_date (datetime) indicating when the delivery was attempted
• successful (boolean) indicating if the delivery was successful or not
• issue_code (string) indicating reference to issue that you attempted to deliver

Once saved, this entity will be propagated to your Tracker API and you can start tracking your entities. Tracker API will also validate your payload so you don't accidentally push invalid entity.

You can test tracking new print_delivery by running following command (change property_token to yours):

The description of API can be found in Tracker's swagger.json.

As of today, entities are not linked to Segment builder UI and are only stored to Elasticsearch for later use. As they're experimental feature, the APIs might change in the future and it's not recommended for production use.

Newsletters

When Beam is configured to work with Mailer (REMP_MAILER_ADDR is populated), Beam allows you to configure automatic generation of newsletters. Content of this email would be based on articles selected by Beam automatically also with the option to personalize the email for each user separately.

When you want to add newsletter, following is required:

• Name. Name of this newsletter. Used only within newsletter listing in Beam admin.
• Segment. Target segment of users who should receive the newsletter. List of segments is provided by Mailer and may differ from the list of available segments directly in Beam (as Mailer might have registered other segment providers).

• Generator. Mailer generator to be used to generate emails. The generator in Mailer has to be registered with best_performing_articles key. By default Mailer provides such generator combined with very basic template.

• Mail type. Newly created email has to belong to specific mail type (newsletter) so Mailer can check whether user is subscribed to receive this email or not. You should select which newsletter this is.
• Criterion. What should be used as a criteria for selecting (and ordering) articles for newsletter (e.g. number of pageviews, amount of time spent reading the article)
• Timespan. What should be the time range for selected criterion. This should be similar or equal to your recurrence so you can send best articles in the last 24 hours (timespan) every 24 hours (recurrence).
• How many articles. How many articles should be selected into the email.
• Personalized content. Whether all users should receive the same newsletter or whether everyone should receive newsletter with articles they haven't read yet.
• Email subject. What's subject of newsletter (e.g. Dennik N - Daily newsletter).
• Email from. Who's the sender of newsletter (e.g. [email protected])
• Start date. When the email should be sent.
• Recurrence. If the email should be recurring and how often it should be sent.

Once configured, you should run (and schedule) email sending command:

Articles

Beam provides pageview-related and conversion-related stats for articles. To be able to display this data, Beam needs you to push Article and Conversion metadata first. Please see Article tracking and Conversion tracking sections to see definition of API calls you should request from your CMS/CRM.

If you have your Laravel scheduler enabled and you track your pageviews via JS snippet, you can now display statistics for articles:

• Conversion stats. Displaying number of conversions reported for given article, amount earned and averages. All filterable by authors and sections. You can filter data based on publish date of articles and conversion dates.
• Pageview stats. Displating number of pageviews and time spent reading for given article (if being tracked) and related averages. All filterable by authors and sections. You can filter data based on publish date of articles.

Conversions

Conversions section provides information about single conversions that you tracked via Conversion tracking and lets you filter conversions based on authors and sections.

Section also includes experimental User path feature which attempts to extract aggregate of events that occur before the conversion happens. In its current state it only provides counts and share of events that happen right before the payment.

To start using the feature, please run (and schedule) following command to create aggregates first:

Authors

This section provides similar view as Articles section does - with the difference that here you can see aggregated data for each author. When you select a specific author, Beam displays article statistics for given author.

You can filter the data based on publish date of articles.

Visitors

Visitors section provides aggregate view on where your visitors are coming from and from what devices. Section allows you to filter the results based on date range of targeted visit and based on subscription status - so you can differentiate where your subscribers are coming from and where you non-paying users are coming from.

• Devices. Section allows you to see hardware-related statistics of your visitors (with visit counts) based on browsers and specific devices that were extracted from user agents of your visitors.
• Sources. Section allows you to see referer-based statistics and see absolute numbers for given source (host) or medium (e.g. search, internal traffic, etc.)

API Documentation

Beam itself serves as a tool for tracking the data and displaying them in a nice fashion. Beam admin has couple of APIs for providing metadata for Articles and Conversions that need to be stored persistently in MySQL

Note: All Elasticsearch data should be always treated as non-persistent and backed up if necessary.

All examples use http://beam.remp.press as a base domain. Please change the host to the one you use before executing the examples.

All examples use XXX as a default value for authorization token, please replace it with the real token API token that can be acquired in the REMP SSO.

All requests should contain (and be compliant) with the follow HTTP headers.

API responses can contain following HTTP codes:

If possible, the response includes application/json encoded payload with message explaining the error further.

POST /api/articles/upsert (DEPRECATED - click here for v2)

Your CMS should track all article-related changes to Beam so Beam knows about the article, who's the author and to which sections it belongs to. Once the article data is available to Beam, system starts to link various statistics that you've tracked with your JS snippet for given article (e.g. pageviews, time spent, reading progress) and data related to Beam (e.g. A/B testing of titles).

Headers:

Body:

Examples:

Response:

Any create/update matching is based on the article's external_id. You're free to update the article as many times as you want.

POST /api/v2/articles/upsert

Your CMS should track all article-related changes to Beam so Beam knows about the article, who's the author and to which sections it belongs to. Once the article data is available to Beam, system starts to link various statistics that you've tracked with your JS snippet for given article (e.g. pageviews, time spent, reading progress) and data related to Beam (e.g. A/B testing of titles).

Headers:

Body:

Examples:

Response:

Any create/update matching is based on the article's external_id. You're free to update the article as many times as you want.

POST /api/articles/read

List already read articles based on filter.

Headers:

Body:

Examples:

Response:

POST api/conversions/upsert

Beam admin provides statistics about article/author performance. One of the metrics used are conversions. This endpoint stores minimal conversion data. Extended data should be additionally tracked via Tracker API (see /track/commerce definition in Tracker's swagger.json file).

Headers:

Body:

Examples:

Response:

POST api/articles/top

Beam admin provides statistics about article performance. This endpoint return top articles by pageviews. You can filter articles by content type, sections, authors, tags or tag categories.

Headers:

Body:

Examples:

Response:

POST api/v2/articles/top

Beam admin provides statistics about article performance. This endpoint return top articles by pageviews. You can filter articles by content type, sections, authors, tags or tag categories.

You can combine multiple filters for each filter category. Filters in and between categories are joined with AND, values in filter are joined with OR.

Headers:

Body:

Examples:

Response:

POST api/authors/top

Beam admin provides statistics about author performance. This endpoint return top authors by pageviews. You can filter authors by content type, sections, tags or tag categories.

Headers:

Body:

Examples:

Response:

POST api/v2/authors/top

Beam admin provides statistics about author performance. This endpoint return top authors by pageviews. You can filter authors by content type, sections, tags or tag categories.

You can combine multiple filters for each filter category. Filters in and between categories are joined with AND, values in filter are joined with OR.

Headers:

Body:

Examples:

Response:

POST api/tags/top

Beam admin provides statistics about tag performance. This endpoint return top post tags by pageviews. You can filter tags by content type, sections, authors or tag categories.

Headers:

Body:

Examples:

Response:

POST api/v2/tags/top

Beam admin provides statistics about tag performance. This endpoint return top post tags by pageviews. You can filter tags by content type, sections, authors or tag categories.

You can combine multiple filters for each filter category. Filters in and between categories are joined with AND, values in filter are joined with OR.

Headers:

Body:

Examples:

Response:

POST api/pageviews/histogram

Beam admin provides pageviews histogram for articles satisfying the filter for days in date range. You can filter articles by content type, sections, authors, tags or tag categories.

You can combine multiple filters for each filter category. Filters in and between categories are joined with AND, values in filter are joined with OR.

Headers:

Body:

Examples:

Response:

GET /api/articles

Returns list of articles specified by ids or external ids.

Headers:

Parameters:

Examples:

Response:

GET /api/authors

Returns list of authors.

Headers:

Parameters:

Examples:

Response:

GET /api/conversions

Returns list of conversions.

Headers:

Parameters:

Examples:

Response:

GET /api/sections

Returns list of sections.

Headers:

Parameters:

Examples:

Response:

GET /api/tags

Returns list of tags.

Headers:

Parameters:

Examples:

Response: