Every restaurant has a base sauce ready for different dishes, the same way I am sharing a base shell script to log-rotate custom logfiles and push archives into S3. You can modify the script as per the taste (^_-)
To begin with, you must install s3cmd on your server first and configure it then create a tmp directory inside your /<absolute_application_log_path>
I have a habit of storing all my customized scripts in “/opt/scripts“, which I call my script home.
- <file_name> – name of your application logfile name Eg:- access.log
- <absolute_application_log_path> – absolute path of your log path location Eg:- /var/log/nginx
I’ll create a logrotate configuration file (logrotate_<file_name>.log) in my script home.
/<absolute_application_log_path>/<file_name>.log { size 10M missingok rotate 10 dateext dateformat -%d%m%Y notifempty copytruncate }
I’ll create my shell script (<script_name>.sh) in my script home.
!/bin/bash
now=date +"%Y-%m-%d"
rm -rf /<absolute_application_log_path>/tmp/*
logrotate -v /opt/scripts/logrotate_<file_name>.log
mv -f /<absolute_application_log_path>/.log- /<absolute_application_log_path>/tmp/
cd /<absolute_application_log_path>/tmp/ && tar -czvf <file_name>-${now}.tar.gz *
s3cmd put /<absolute_application_log_path>/tmp/*.tar.gz s3://<s3_bucket_name>/
Finally, setup a daily cron
59 23 * * * /bin/bash /opt/scripts/<script_name>.sh