Add pruning Nginx cache

This commit is contained in:
Filip Rysavy 2021-12-06 13:35:17 +01:00
parent b2dbe4bdb4
commit 0af14d32db
No known key found for this signature in database
GPG Key ID: EA1F430401C92D99
5 changed files with 38 additions and 0 deletions

View File

@ -0,0 +1 @@
- Added script to prune nginx cache.

View File

@ -29,6 +29,12 @@ the health check.
The `portal-upgrade.sh` script upgrades the docker images for a portal and The `portal-upgrade.sh` script upgrades the docker images for a portal and
clears and leftover images. clears and leftover images.
**nginx-prune.sh**\
The `nginx-prune.sh` script deletes all entries from nginx cache larger than
the given size and smaller entries until nginx cache disk size is smaller than
the given cache size limit. Both values are configured in
`lib/nginx-prune-cache-subscript.sh`. The script doesn't require `sudo`.
## Webportal Upgrade Procedures ## Webportal Upgrade Procedures
TODO... TODO...

View File

@ -0,0 +1,24 @@
#!/usr/local/bin/bash
# This subscript is expected to be run inside docker container using 'bash'
# image. The image is based on Alpine Linux. It's tools (find, stat, awk, sort)
# are non-standard versions from BusyBox.
MAX_CACHE_DIR_SIZE=20000000000
MAX_KEEP_FILE_SIZE=1000000000
total=0
find /home/user/skynet-webportal/docker/data/nginx/cache -type f -exec stat -c "%Y %n %s" {} + | sort -rgk1 | while read line
do
size=$(echo $line | awk '{print $3}')
new_total=$(($total + $size))
if (("$size" <= "$MAX_KEEP_FILE_SIZE" && "$total" < "$new_total"))
then
total=$new_total
continue
fi
filename=$(echo $line | awk '{print $2}')
rm $filename
done

6
scripts/nginx-prune.sh Executable file
View File

@ -0,0 +1,6 @@
#!/bin/bash
# We execute the nginx cache pruning subscript from docker container so that we
# can run the pruning script in user crontab without sudo.
docker run --rm -v /home/user:/home/user bash /home/user/skynet-webportal/scripts/lib/nginx-prune-cache-subscript.sh

View File

@ -4,3 +4,4 @@
30 */4 * * * /home/user/skynet-webportal/setup-scripts/blocklist-airtable.py /home/user/skynet-webportal/.env 30 */4 * * * /home/user/skynet-webportal/setup-scripts/blocklist-airtable.py /home/user/skynet-webportal/.env
0 4 * * * /home/user/skynet-webportal/scripts/db_backup.sh 1 >> /home/user/skynet-webportal/logs/db_backup_`date +"%Y-%m-%d-%H%M"`.log 2 > &1 0 4 * * * /home/user/skynet-webportal/scripts/db_backup.sh 1 >> /home/user/skynet-webportal/logs/db_backup_`date +"%Y-%m-%d-%H%M"`.log 2 > &1
0 5 * * * /home/user/skynet-webportal/scripts/es_cleaner.py 1 http://localhost:9200 0 5 * * * /home/user/skynet-webportal/scripts/es_cleaner.py 1 http://localhost:9200
15 * * * * /home/user/skynet-webportal/scripts/nginx-prune.sh