00fff39bf2 | ||
---|---|---|
.. | ||
support | ||
README.md | ||
blacklist-skylink.sh | ||
bot_utils.py | ||
funds-checker.py | ||
log-checker.py | ||
setup-docker-services.sh | ||
setup-health-check-scripts.sh | ||
setup-server.sh | ||
stats-logger.sh |
README.md
Skynet Portal Setup Scripts
This directory contains a setup guide and scripts that will install and configure some basic requirements for running a Skynet Portal. The assumption is that we are working with a Debian Buster Minimal system or similar.
Initial Setup
You may want to fork this repository and replace ssh keys in
setup-scripts/support/authorized_keys
and optionally edit the setup-scripts/support/tmux.conf
and setup-scripts/support/bashrc
configurations to fit your needs.
Step 0: stack overview
- dockerized services inside
docker-compose.yml
- sia (docker hub): storage provider, heart of the portal setup
- caddy (docker hub): reverse proxy (similar to nginx) that handles ssl out of a box and acts as a transparent entry point
- openresty (docker hub): nginx custom build, acts as a cached proxy to siad and exposes all api endpoints
- health-check: this is a simple service that runs periodically and collects health data about the server (status and response times) and exposes
/health-check
api endpoint that is deliberately delayed based on the response times of the server so potential load balancer could prioritize servers based on that (we use it with cloudflare)
- siad setup: we use "double siad" setup that has one node solely for download and one for upload to improve performance
- we use systemd to manage siad service
- siad is not installed as docker service for improved performance
- discord integration
- funds-checker: script that checks wallet balance and sends status messages to discord periodically
- log-checker: script that scans siad logs for critical errors and reports them to discord periodically
- blacklist-skylink: script that can be run locally from a machine that has access to all your skynet portal servers that blacklists provided skylink and prunes nginx cache to ensure it's not available any more (that is a bit much but that's the best we can do right now without paid nginx version) - if you want to use it, make sure to adjust the server addresses
Step 1: setting up server user
- SSH in a freshly installed Debian machine on a user with sudo access (can be root)
apt-get update && apt-get install sudo
to make suresudo
is availableadduser user
to create user calleduser
(creates/home/user
directory)usermod -a -G sudo user
to add this new user to sudo group- Quit the ssh session with
exit
command
You a can now ssh into your machine as the user user
.
Step 2: setting up environment
- On your local machine:
ssh-copy-id user@ip-addr
to copy over your ssh key to server - On your local machine:
ssh user@ip-addr
to log in to server as useruser
- You are now logged in as
user
Following step will be executed on remote host logged in as a user
:
sudo apt-get install git
to install gitgit clone https://github.com/NebulousLabs/skynet-webportal
- run setup scripts in the exact order and provide sudo password when asked (if one of them fails, you can retry just this one before proceeding further)
/home/user/skynet-webportal/setup-scripts/setup-server.sh
/home/user/skynet-webportal/setup-scripts/setup-docker-services.sh
/home/user/skynet-webportal/setup-scripts/setup-health-check-scripts.sh
(optional)
Step 3: configuring siad
At this point we have almost everything running, we just need to set up your wallet and allowance:
- Create new wallet (remember to save the seeds)
docker exec -it sia siac wallet init
- Unlock wallet (use seed as password)
docker exec -it sia siac wallet unlock
- Generate wallet addresse (save them for later to transfer the funds)
docker exec -it sia siac wallet address
- Set up allowance
docker exec -it sia siac renter setallowance
- 10 KS (keep 25 KS in your wallet)
- default period
- default number of hosts
- 8 week renewal time
- 500 GB expected storage
- 500 GB expected upload
- 5 TB expected download
- default redundancy
- Instruct siad to start making 10 contracts per block with many hosts to potentially view the whole network's files
docker exec -it sia siac renter setallowance --payment-contract-initial-funding 10SC
Step 4: configuring docker services
- generate and copy sia api token
printf ":$(cat /home/user/.sia/apipassword)" | base64
- edit
/home/user/skynet-webportal/.env
and configure following environment variablesDOMAIN_NAME
(optional) is your domain name if you have itEMAIL_ADDRESS
(required) is your email address used for communication regarding SSL certification (required)SIA_WALLET_PASSWORD
(required) is your wallet password (or seed if you did not set a password)HSD_API_KEY
(optional) this is a random security key for an optional handshake integration that gets generated automaticallyCLOUDFLARE_AUTH_TOKEN
(optional) if using cloudflare as dns loadbalancer (need to change it in Caddyfile too)AWS_ACCESS_KEY_ID
(optional) if using route53 as a dns loadbalancerAWS_SECRET_ACCESS_KEY
(optional) if using route53 as a dns loadbalancer
- if you have a custom domain and you configured it in
DOMAIN_NAME
, edit/home/user/skynet-webportal/docker/caddy/Caddyfile
and uncommentimport custom.domain
- only for siasky.net domain instances: edit
/home/user/skynet-webportal/docker/caddy/Caddyfile
, uncommentimport siasky.net
docker-compose up -d
to restart the services so they pick up new env variablesdocker exec caddy caddy reload --config /etc/caddy/Caddyfile
to reload Caddyfile configuration
Useful Commands
- Accessing siac
docker exec -it sia siac
- Checking status of siad service
systemctl --user status siad
- Stopping siad service
systemctl --user stop siad
- Starting siad service
systemctl --user start siad
- Restarting siad service
systemctl --user restart siad
- Restarting caddy gracefully after making changes to Caddyfile
docker exec caddy caddy reload --config /etc/caddy/Caddyfile
- Restarting nginx gracefully after making changes to nginx configs
docker exec nginx openresty -s reload
- Checking siad service logs (follow last 50 lines)
journalctl -f -n 50 --user-unit siad
- Checking caddy logs (for example in case ssl certificate fails)
docker logs caddy -f
- Checking nginx logs (nginx handles all communication to siad instances)
tail -n 50 docker/data/nginx/logs/access.log
to follow last 50 lines of access logtail -n 50 docker/data/nginx/logs/error.log
to follow last 50 lines of error log