Download the PHP package alingsas-kommun/data-importer without Composer

Data Importer & Visualizer

Receive JSON data via REST API, store it in custom database tables, and render it anywhere on your site with PHP templates and shortcodes.

Introduction

Data Importer & Visualizer is a WordPress plugin for teams that need to publish structured, frequently-updated data on a website without building a custom integration from scratch. You define one or more data sources, push JSON to a per-source REST endpoint, and display the records using flexible PHP templates rendered through a simple shortcode.

Typical use cases:

• Publishing real-time or scheduled data feeds (events, open data, listings, status boards).
• Bridging an external system (CRM, ERP, open-data API) with a WordPress front-end.
• Displaying tabular or card-based data that is maintained outside WordPress.

Requirements

No third-party services or external API subscriptions are required. All data is stored in the local WordPress database.

What's Possible

Data sources

• Create any number of independent data sources, each with its own slug, one or more API keys, and import settings.
• Choose an import mode per source:
  • replace — deletes all existing records then inserts the new payload (default).
  • append — adds incoming records without touching existing ones.
  • upsert — inserts or updates records matched by one or more key columns.
• Manually import JSON directly from the admin for testing.

REST API import

• Push JSON to a dedicated endpoint per source: POST /wp-json/data-importer/v1/import/{source-slug}
• Authenticate with a per-source API key sent in the X-API-Key header.
• Restrict imports with optional per-key IP or CIDR allowlists.
• Accepts both a single JSON object {} and arrays of objects [{}, {}].
• Override the source's import mode for a single request with the mode query parameter (replace, append, or upsert).

Templates

• Build multiple PHP templates per source.
• Each template has a wrapper before, a per-row template, and a wrapper after section.
• Reference any field in the imported JSON directly as a PHP variable inside the template.
• Attach custom CSS/JS assets to a template (loaded only on pages that use the shortcode).
• Preview a template against live data directly inside the admin.

Shortcode

• Render any source/template combination anywhere in WordPress with [data_importer].
• Filter records from the shortcode without changing the template: where_key, where_op, where_value with support for dot-notation for nested fields.
• Sort records before rendering with template defaults or shortcode attributes, including multi-field sorts and dot-notation keys.
• Control pagination with limit, offset, and database-row order attributes.
• Fetch a single record by database ID with id.

Security & operations

• API key authentication with hashed secrets stored at rest.
• Optional per-key IP/CIDR allowlists for import endpoints.
• Per-source rate limiting with configurable count/window.
• Payload size and max-record guardrails.
• Dedicated Log tab per source with three sections: Import Log (latest 20 imports), Security Log (latest 50 blocked/rejected attempts), and Template Error Log (latest 20 PHP template errors).
• Safe mode — disables all PHP template execution in the frontend with one click while you fix a broken template.
• All security controls are tunable via WordPress filter hooks without patching the plugin.

Installation

1. Upload the plugin folder to /wp-content/plugins/.
2. Activate through Plugins → Installed Plugins in wp-admin.
3. Go to Settings → Data Importer.
4. Click Add new source, give it a name and choose an import mode.
5. Copy the Endpoint URL and the newly created API Key from the API tab immediately, since the plaintext key is only shown once.

Uninstallation

Deactivate and delete the plugin through Plugins → Installed Plugins in wp-admin. The uninstall routine will remove:

• All custom database tables created by the plugin.
• All plugin options (source configuration, import/security/template error logs, safe mode setting).
• All plugin transients (rate-limit counters, API key reveal tokens, manual import drafts).

Note: Content generated by PHP templates — such as files or external records written by template code — is not tracked by the plugin and will not be removed automatically.

Testing

Run the integration suite from the plugin directory with:

For readable per-test output, use:

To list the discovered tests without running them, use:

The PHPUnit bootstrap loads the local WordPress install, then exercises the plugin's real REST, manual-import, shortcode, safe-mode, and logging flows.

Getting Started

1. Create a data source

Go to Settings → Data Importer → Add new source. Fill in:

• Name — human-readable label.
• Slug — used in the REST endpoint URL (auto-generated from name).
• Import mode — replace, append, or upsert.

2. Push data

Send a POST request with a JSON body to the source endpoint.

A successful import returns:

3. Build a template

Open the source, go to the Template tab and create a new template. Example per-row markup:

Add a wrapper in Wrapper before / Wrapper after:

4. Add the shortcode to a page

With a specific template:

Shortcode Reference

Supported where_op values

Shortcode examples

Show all active items:

Filter by a nested field:

Paginate — second page of 10:

Sort by first name ascending, then surname descending:

Sort by a nested numeric value and render the top 10:

Single record:

Template Variables

Inside the per-record template, the following PHP variables are available:

Access fields via $vars:

Conditional rendering:

First, middle, and last row checks:

Optional variable extraction

By default, record fields are only accessible via $vars. You can opt in to also having each top-level key extracted as its own variable ($title, $status, …) using the data_importer_template_extract_vars filter:

Security note: Only enable extraction if you control the shape of all imported records. A record with a key named code, vars, or context can cause subtle or unexpected behaviour in the template scope.

Note: Template code is executed as PHP. Only trusted administrators should be allowed to edit templates. See SECURITY.md.

Import Modes

For upsert, set the Update key field in the source settings to a comma-separated list of JSON field names that uniquely identify a record (e.g. id or event_id,date).

Overriding the import mode per request

You can override the source's configured import mode for a single request by adding a mode query parameter to the endpoint URL:

Valid values for mode are replace, append, and upsert. Any other value is rejected with a 400 response. When the parameter is omitted, the mode configured in the source settings is used.

Security

See SECURITY.md for a full OWASP Top 10 mapping and all filter hooks.

Runtime filter hooks

These WordPress filters let you tune security policy without modifying the plugin:

Rate limiting is best-effort. The built-in limiter uses WordPress transients. Without a persistent object cache (Redis, Memcached) transients are stored in wp_options, which can become a write bottleneck under high traffic. The counter increments are also non-atomic, so concurrent requests may slip past the limit before it is enforced. For production deployments that require reliable rate limiting, enforce limits at the infrastructure layer — an Nginx limit_req directive, a WAF rule, or a reverse-proxy rate limiter will be more accurate and more performant. | data_importer_template_error_details | bool | false | Expose PHP error details in the frontend (enable in development only). | | data_importer_blocked_template_functions | string[] | (see below) | Array of PHP function names that are forbidden inside templates. Add or remove entries to tune the blocklist. | | data_importer_blocked_template_constructs | string[] | (see below) | Array of PHP language constructs that are forbidden inside templates (backticks, include/require family). | | data_importer_template_extract_vars | bool | false | Re-enable legacy extract() behaviour so record keys are available as $name etc. See security note in Template Variables. |

Default blocked functions

The following functions are blocked in templates by default. Use the data_importer_blocked_template_functions filter to add or remove entries:

The callable-accepting functions (call_user_func, array_map, etc.) are blocked because they can invoke arbitrary code when given a user-influenced callable. If a template genuinely needs one of them, remove it via the filter after reviewing how the callable argument is derived.

Default blocked constructs

Language constructs (which are not function calls and cannot be reached by the function blocklist regex) are covered by a separate data_importer_blocked_template_constructs filter:

backticks refers to the backtick shell-exec operator (`command`). The include / require family is blocked because it can execute code from a file path that may be attacker-influenced.

Example — add file_get_contents to the blocklist:

Example — remove a function from the blocklist:

Example — lower the rate limit and disable PHP templates on a staging environment:

Safe Mode

If a broken PHP template causes a fatal error on the front-end:

1. In the Data Importer admin, click Activate safe mode in the notice bar.
2. While safe mode is active, no PHP templates are executed on the front-end.
3. Open the Template tab, fix the template, and save.
4. Click Deactivate safe mode to restore normal rendering.

Safe mode can also be forced programmatically:

Or via filter:

When either the constant or a filter is in effect, the Data Importer admin shows an override in effect notice and hides the Disable Safe Mode button, so administrators understand the UI toggle is currently inert.

Frequently Asked Questions

Can I push a single object instead of an array?

Yes. Both {} and [{}, {}] are accepted. A single object is treated as one record.

Can I restrict which servers are allowed to import?

Yes. In the source API tab, enter a comma-separated list of allowed IP addresses. Leave empty to allow any IP.

How do I retrieve the available field names for a source?

Authenticated admin users can call:

This returns the unique field names found across all stored records.

Can I test an import without an external system?

Yes. Go to the source's Manual tab, paste a JSON payload, and click Import. The result is shown in the Log tab under Import Log.

How do I use multiple templates for the same source?

Create additional templates in the Template tab. Reference them by name in the shortcode:

What happens to old data when I push a new payload in replace mode?

All existing records for that source are deleted before the new records are inserted. Use append or upsert mode if you need to preserve existing data.

Can I use different import modes without changing the source settings?

Yes. Add a mode query parameter to the endpoint URL to override the configured mode for that single request. This is useful when the same source needs different import strategies on different schedules — for example a full replace once a week and an upsert every weekday:

The source settings are not changed; the override only applies to that request.