Download the PHP package danielme85/laravel-log-to-db without Composer

Laravel Log-to-DB

Custom Laravel Log channel handler that can store log events to SQL or MongoDB databases. Uses Laravel native logging functionality through Monolog.

Version compatibility

⚠️ Upgrading from v4 to v5? v4's default datetime format was broken and corrupted the stored datetime column. See Upgrading from v4 to v5: fixing broken datetime values for the fix.

• Installation
• Configuration
• Usage
• Fetching Logs
• Custom Eloquent Model
• Log Cleanup
• Upgrading from v4 to v5: fixing broken datetime values
• Processors
• Lumen Installation
• Local Testing With Docker

Installation

Use the composer require or add to composer.json.

If you are using SQL database server to store log events you can use the migration included. The MongoDB driver does not require the migration. Copy the migration file for log the database table to your app (you can also do any changes you want or manually copy this file your database/migrations folder).

Run the Laravel migration artisan command.

For optional MongoDB support you need to install the mongodb/laravel-mongodb package

Configuration

You will need to add an array under 'channels' in your config/logging.php for Log-to-DB like so:

These are the minimum required logging.php config settings to get started. Please note that the array index 'database' can be whatever string you like as long as it is unique to this logging config. You can also give the logging channel a name that later is referenced in a column in the DB table, this way you can have multiple logging-to-db channels.

• driver = Required to trigger the log driver.
• via = The Log handler class.
• level = The minimum error level to trigger this Log Channel.
• name = The channel name that will be stored with the Log event. Please note that if you use the stack driver the name value in the stack array is used.
• connection = The DB connection from config/database.php to use (default: 'default').
• collection = The DB table or collection name. (Default: log).
• detailed = Store detailed log on Exceptions like stack-trace (default: true).
• processors = Array of additional processors. These will add additional info into the 'extra' field in the logged data. More information about processors

More info about some of these options: https://laravel.com/docs/logging#customizing-monolog-for-channels

There are some default settings and more information about configuring the logger in the 'logtodb.php' config file. This could be copied to your project if you would like edit it with the vendor publish command.

You can also change these settings in your env file.

PLEASE NOTE: Starting with v2.2.0, the datetime column will be saved as a string in the format given in 'datetime_format' in logtodb.php config file, or the LOG_DB_DATETIME_FORMAT value in your .env file. The default format is Y-m-d H:i:s (seconds precision). For sub-second precision use Y-m-d H:i:s.v (milliseconds) or Y-m-d H:i:s.u (microseconds) — both are supported since Monolog captures microsecond timestamps by default.

Config priority order

There are three places you can change different options when using log-to-db:

1. The config file: config/logtodb.php (after doing vendor:publish).
2. Your .env file will override settings in the logtodb.php config file.
3. The Laravel logging config file: config/logging.php. You need to add a custom array here as mentioned above, in this same array you can specify/override config settings specifically for that log channel.

Config values set in point 1 & 2 would work as default for all new log channels you add in the "channels" array for the Laravel logging configuration (config/logging.php).

Log Worker Queue

It might be a good idea to save the log events with a Queue Worker. This way your server does not have to wait for the save process to finish. You would have to configure the Laravel Queue settings and run the Queue listener. https://laravel.com/docs/queues#running-the-queue-worker

The queue can be enabled/disabled in any of the following places:

• LOG_DB_QUEUE = true | in .env
• queue_db_saves => true | in config/logtodb.php
• queue => true | in the log channel config array -> config/logging.php

Usage

Since this is a custom log channel for Laravel, all "standard" ways of generating log events etc should work with the Laravel Log Facade. See https://laravel.com/docs/logging for more information.

You can also log to specific log channels:

Fetching Logs

The logging by this channel is done trough the Eloquent Model builder. LogToDB::model($channel, $connection, $collection); You can skip all function variables and the default settings from the config/logtodb.php will be used.

Some more examples of getting logs

When getting logs for specific channel or DB connection and collection you can either use the channel name matching config/logging.php or connection name from config/databases.php. You can also specify collection/table name if needed as the third function variable when fetching the model.

Custom Eloquent Model

Since Laravel is supposed to use static defined collection/table names, it might be better to use your own model in your app for a more solid approach. You can use your own eloquent model by referencing it in the config, then adding the trait: "LogToDbCreateObject"

SQL

MongoDB

LOG_DB_MODEL='App\Models\CustomLog'

If you want your custom model to follow the channel-level connection/collection config instead of hardcoded $table/$connection properties, also add the BindsDynamically trait — the package will then bind the configured connection and table onto your model at runtime:

WARNING: Fetching the model trough the dynamic Eloquent model (default behavior) have some side-effects as tables and connections are declared dynamically instead of assigned properties in the model class. Certain functions are broken like LogToDB::model->all(), while LogToDB::model->where()->get() will work as normal. Using your own models avoids these problems.

Model Closures and Observers

You can either add closures on your custom application model mentioned above, or add a model observer for the default LogToDb models.
Create a observer:

Then add to your AppServiceProvider (or another provider that calls the app boot function).

Adding tables/expanding collections

The Log handler for SQL expects the following schema:

This is the migration that ships with this plugin. You can add as many tables as you want, and reference them in the 'collection' config value. Collection = table, I used the term collection as it works for both SQL/noSQL. No migrations needed for MongoDB.

No indexes are added per default, so if you fetch a lot of log results based on specific time ranges or types: it might be a good idea to add some indexes.

Log Cleanup

There are config values that you can set to specify the max number of log records to keep, or the max record age in hours.

• logging.php channel array -> (max_records, max_hours).
• .env file -> (LOG_DB_MAX_COUNT, LOG_DB_MAX_HOURS).

These option is set to false per default, these have to be set to desired integers before you can run the "log:delete" artisan command.

This command will delete records based on settings described above. Add this command to your Console/kernel.php, or run manually in cron etc to enable automatic cleanup.

Manual Cleanup

There is a helper function to remove the oldest log events and keep a specified number

Or based on date (most be valid date/datetime supported by strtotime()) http://php.net/manual/en/function.strtotime.php

Upgrading from v4 to v5: fixing broken datetime values

Prior to v5, the default datetime_format was Y-m-d H:i:s:ms. The :ms token is not a valid PHP date() format token, so instead of milliseconds it re-printed the month and seconds again (e.g. 2024-01-01 12:00:00:0100), corrupting the stored datetime column. v5 changes the default to Y-m-d H:i:s; use Y-m-d H:i:s.v for real millisecond precision or Y-m-d H:i:s.u for microseconds.

Existing rows saved under the old format are not automatically corrected. Since every log record also stores a reliable unix_time integer, run the included artisan command to recompute datetime from unix_time using your currently configured datetime_format:

Options:

• --dry-run — report how many records would be fixed without writing anything.
• --channel=database — only fix records for a specific logging channel.
• --chunk=500 — number of records updated per batch (default 500).

Note: recomputing from unix_time only has second precision, so if you were relying on sub-second precision in the old (broken) values, that precision cannot be recovered.

Processors

Monolog ships with a set of processors, these will generate additional data and populate the 'extra' field. I've also added a couple of example processors in this pacage under src/Processors. To enable processors you can add them to the log config array:

You could also create your own custom processor, make sure they implement Monolog\Processor\ProcessorInterface.

Example of custom processor

More logging.php config examples

Lumen Installation

• Create a config folder in your projects root directory (if you don't already have one), then add a logging.php config file there. Use the Laravel logging.php config file as an example/starting point. You can also add all the other "missing" config files from the Laravel config folder to your Lumen project.

• To use these config files you have to load them in your applications bootstrap/app.php file (or add your own service provider file and load it in that same file).

You also need to make sure that all the needed basic config values for logtodb is set by either:

• Copy over logtodb.php from the config folder of this addon,
• or just add all your log-to-db options in your applications config/logging.php file (probably easiest). Just follow the configuration example above under the configuration section.

Since we are using Lumen we need to specify the config and service providers in the "bootstrap/app.php" file.

Next step is to register the service provider, either in bootstrap/app.php or in app/Provider/AppServiceProvider.

After adding the service provider you should be able to run the database migration in Lumen with:

Please note that you need a working db connection in Lumen at this point.

And then maybe it works... ¯_(ツ)_/¯

Using worker queue to write log to db with Lumen

You would need to set up a queue driver in Lumen before you can use the queue (default is: QUEUE_CONNECTION=sync, which is basically no queue). More info about the queues in Lumen doc (they are mostly the same as Laravel). I would recommend the Redis queue driver but database should also work.

How to make Redis work in Lumen (in general).

install per command

OR add to composer.json

Or install other alternatives to predis.

Add service provider and enable eloquent in your bootstrap/app.php file (Eloquent only needed if you use the model/model helper class to fetch new log event);

Now you can set your .env values to use Redis for the queue and cache if you so please:

Local Testing With Docker

There is a helper bash script called 'runLocalTestInDocker.sh', that runs the following docker commands:

To run Docker is required, give execute rights to the script and run:

Development supported by: