Logrotate - A utility to manage log files

Introduction

When working with multiple services running on a system, we saw a large portion of disk space was getting used up by log files. The size of the log files had upsurged in a quite short amount of time. The reason's being, debug logs being added in the log file and many such others.

This brought up a need to manage log files in such a way that the system resources are not exploited.

The key thing to always look at when deploying services is that the system, resources are used with utmost care which in our case had been exploited to a very high extent.

When we started looking out for ways to manage log files, we stumbled upon a utility called Logrotate which did the exact same thing we needed:

1. Take a backup of log files periodically as configured/needed, so that at any given point of time we had a backup we could look at
2. Compress the backed-up log file, so that system resources are not exploited.
3. Delete very old backed-up log files to clear the memory.

Logrotate, as per the definition, is a tool/utility which is used to manage(rotate, compress, remove, and mail) log files. This eases the job of the system administrator by automatically taking care of log files.

Note: By default, when logrotate is installed, logrotate file is copied in /etc/cron.daily folder because of which logrotate is run on a daily basis. Logrotate maintains the log of when logrotate had happen in file /var/lib/logrotate/logrotate.status.

Configuration Files

logrotate.conf

This file is located at /etc/logrotate.conf

It contains default parameters used by logrotate utility to rotate log files.

Following is the default content of the file

# see "man logrotate" for details # rotate log files weekly weekly # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # uncomment this if you want your log files compressed #compress # RPM packages drop log rotation information into this directory include /etc/logrotate.d # no packages own wtmp and btmp -- we'll rotate them here /var/log/wtmp {     monthly     create 0664 root utmp         minsize 1M     rotate 1 } /var/log/btmp {     missingok     monthly     create 0600 root utmp     rotate 1 } # system-specific logs may be also be configured here.

Let's analyze each of the configuration mentioned above

weekly	Files should be rotated weekly
dateext	The backed-up file name should be suffixed with date of rotation
compress	Specifies that the file should be compressed
create	Specifies that a new log file should be created to log incoming logs.
rotate 4	Specifies that the amount of backed-up log files should not be more than 4. Delete the older backed-up log files if the number of backed-up log files is more than 4.

Following line, include /etc/logrotate.d, of the logrotate.conf file is the directory that contains application-specific configuration files.

include /etc/logrotate.d

For example, if we come up with an application that stores log's in /var/log/myapplication/myapplication.log, then the logrotate specific configuration can be put into /etc/logrotate.d/myapplication.conf so that logrotate utility can be used for myapplication.

This helps in enhancing the extensibility of the utility.

logrotate.d

For a package installed, if we wish to have log rotation enabled, we need to place configuration files in /etc/logrotate.d directory.

Consider following example:

cat /etc/logrotate.d/myapplication /var/log/myapplication/*.log {   rotate 12   copytruncate   compress   missingok   notifempty   size 10M }

In the example above, following are the keywords used:

/var/log/myapplication/*.log	Regex to identify the log files to be rotated. In this, it will match each file which ends with .log
rotate 12	The backed-up file name should be suffixed with date of rotation
copytruncate	Specifies that the file should be compressed
compress	Specifies that a new log file should be created to log incoming logs.
missingok	Specifies that the amount of backed-up log files should not be more than 4. Delete the older backed-up log files if the number of backed-up log files is more than 4
notifempty	Do not rotate the file if it is empty
size 10M	Log files are rotated only if the size exceeds 10Mb.

A single configuration file can have multiple entries as follows:

cat /etc/logrotate.d/myapplication /var/log/myapplication/myapplication.log {   rotate 12   copytruncate   compress   missingok   notifempty   size 10M } /var/log/myapplication/audit.log {   rotate 12   copytruncate   compress   missingok   notifempty   size 10M }

Conclusion

Logrotate thus helps manage several log files with simple configurations. By default logrotate is configured to run daily, which can be updated to run hourly by moving the logrotate file from /etc/cron.daily to /etc/cron.hourly.

References

TecMint

Logrotate man page

RedHat