LOG-ROTATE AND Throw backups into AWS S3.

Every restaurant has a base sauce ready for different dishes, the same way I am sharing a base shell script to log-rotate custom logfiles and push archives into S3. You can modify the script as per the taste (^_-)

To begin with, you must install s3cmd on your server first and configure it then create a tmp directory inside your /<absolute_application_log_path>

I have a habit of storing all my customized scripts in “/opt/scripts“, which I call my script home.

• <file_name> – name of your application logfile name Eg:- access.log
• <absolute_application_log_path> – absolute path of your log path location Eg:- /var/log/nginx

I’ll create a logrotate configuration file (logrotate_<file_name>.log) in my script home.

/<absolute_application_log_path>/<file_name>.log {
size 10M
missingok
rotate 10
dateext
dateformat -%d%m%Y
notifempty
copytruncate
}

I’ll create my shell script (<script_name>.sh) in my script home.

!/bin/bash
now=date +"%Y-%m-%d"

rm -rf /<absolute_application_log_path>/tmp/*
logrotate -v /opt/scripts/logrotate_<file_name>.log
mv -f /<absolute_application_log_path>/.log- /<absolute_application_log_path>/tmp/
cd /<absolute_application_log_path>/tmp/ && tar -czvf <file_name>-${now}.tar.gz *
s3cmd put /<absolute_application_log_path>/tmp/*.tar.gz s3://<s3_bucket_name>/

Finally, setup a daily cron

59 23 * * * /bin/bash /opt/scripts/<script_name>.sh

Advertisement