a7c57b3c5a | ||
---|---|---|
.. | ||
support | ||
README.md | ||
blacklist-skylink.sh | ||
bot_utils.py | ||
funds-checker.py | ||
log-checker.py | ||
setup-docker-services.sh | ||
setup-health-check-scripts.sh | ||
setup-server.sh | ||
setup-siad.sh | ||
stats-logger.sh |
README.md
Skynet Portal Setup Scripts
This directory contains a setup guide and scripts that will install and configure some basic requirements for running a Skynet Portal. The assumption is that we are working with a Debian Buster Minimal system or similar.
Initial Setup
You may want to fork this repository and replace ssh keys in
setup-scripts/support/authorized_keys
and optionally edit the setup-scripts/support/tmux.conf
and setup-scripts/support/bashrc
configurations to fit your needs.
Step 0: stack overview
- dockerized services inside
docker-compose.yml
- docker-host (docker hub): service that exposes server ip to docker container so we could access siad from within the nginx container
- caddy (docker hub): reverse proxy (similar to nginx) that handles ssl out of a box and acts as an entry point
- openresty (docker hub): nginx custom build, acts as a cached proxy to siad (we only use it because caddy doesn't support proxy caching, otherwise we could drop it)
- health-check: this is a simple service that runs periodically and collects health data about the server (status and response times) and exposes
/health-check
api endpoint that is deliberately delayed based on the response times of the server so potential load balancer could prioritize servers based on that (we use it with cloudflare)
- siad setup: we use "double siad" setup that has one node solely for download and one for upload to improve performance
- we use systemd to manage siad services
- siad is not installed as docker service for improved performance
- discord integration
- funds-checker: script that checks wallet balance and sends status messages to discord periodically
- log-checker: script that scans siad logs for critical errors and reports them to discord periodically
- blacklist-skylink: script that can be run locally from a machine that has access to all your skynet portal servers that blacklists provided skylink and prunes nginx cache to ensure it's not available any more (that is a bit much but that's the best we can do right now without paid nginx version) - if you want to use it, make sure to adjust the server addresses
Step 1: setting up server user
- SSH in a freshly installed Debian machine on a user with sudo access (can be root)
apt-get update && apt-get install sudo
to make suresudo
is availableadduser user
to create user calleduser
(creates/home/user
directory)usermod -a -G sudo user
to add this new user to sudo groupusermod -a -G systemd-journal user
to add this new user to systemd-journal group- Quit the ssh session with
exit
command
You a can now ssh into your machine as the user user
.
Step 2: setting up environment
- On your local machine:
ssh-copy-id user@ip-addr
to copy over your ssh key to server - On your local machine:
ssh user@ip-addr
to log in to server as useruser
- You are now logged in as
user
Following step will be executed on remote host logged in as a user
:
sudo apt-get install git
to install gitgit clone https://github.com/NebulousLabs/skynet-webportal
- run setup scripts in the exact order and provide sudo password when asked (if one of them fails, you can retry just this one before proceeding further)
/home/user/skynet-webportal/setup-scripts/setup-server.sh
/home/user/skynet-webportal/setup-scripts/setup-siad.sh
/home/user/skynet-webportal/setup-scripts/setup-docker-services.sh
/home/user/skynet-webportal/setup-scripts/setup-health-check-scripts.sh
(optional)
Step 3: configuring siad
At this point we have almost everything set up. We have 2 siad instances running as services and we need to set up the wallets and allowance on those.
- Create new wallet for both siad instances (remember to save the seeds)
siac wallet init
to init download node walletsiac-upload wallet init
to init upload node wallet
- Unlock both wallets
siac wallet unlock
to unlock download node wallet (use seed as password)siac-upload wallet unlock
to unlock upload node wallet (use seed as password)
- Generate wallet addresses for both siad instances (save them for later to transfer the funds)
siac wallet address
to generate address for download node walletsiac-upload wallet address
to generate address for upload node wallet
- Set up allowance on both siad instances
siac renter setallowance
to set allowance on download node- 10 KS (keep 25 KS in your wallet)
- default period
- default number of hosts
- 8 week renewal time
- 500 GB expected storage
- 500 GB expected upload
- 5 TB expected download
- default redundancy
siac-upload renter setallowance
to set allowance on upload node- use the same allowance settings as download node
- Run
siac renter setallowance --payment-contract-initial-funding 10SC
so that your download node will start making 10 contracts per block with many hosts to potentially view the whole network's files - Copy over apipassword from
/home/user/.sia/apipassword
and save it for the next step - Edit environment files for both siad instances
/home/user/.sia/sia.env
for the download nodeSIA_API_PASSWORD
to previously copied apipassword (same for both instances)SIA_WALLET_PASSWORD
to be the wallet seedPORTAL_NAME
xxxxed part to some meaningful name likewarsaw.siasky.net
DISCORD_BOT_TOKEN
for discord health check scripts integration
/home/user/.sia/sia-upload.env
for the upload nodeSIA_API_PASSWORD
to previously copied apipassword (same for both instances)SIA_WALLET_PASSWORD
to be the wallet seedPORTAL_NAME
xxxxed part to some meaningful name likewarsaw.siasky.net
DISCORD_BOT_TOKEN
for discord health check scripts integration
Step 4: configuring docker services
- generate and copy sia api token
printf ":$(cat /home/user/.sia/apipassword)" | base64
- edit
/home/user/skynet-webportal/.env
and configure following environment variablesDOMAIN_NAME
(optional) is your domain name if you have itEMAIL_ADDRESS
(required) is your email address used for communication regarding SSL certification (required)SIA_API_AUTHORIZATION
(required) is token you just generated in the previous pointCLOUDFLARE_AUTH_TOKEN
(optional) if using cloudflare as dns loadbalancer (it's just for siasky.net configuration)HSD_API_KEY
(optional) this is a random security key for an optional handshake integration that gets generated automatically
- if you have a custom domain and you configured it in
DOMAIN_NAME
, edit/home/user/skynet-webportal/docker/caddy/Caddyfile
and uncommentimport custom.domain
- only for siasky.net domain instances: edit
/home/user/skynet-webportal/docker/caddy/Caddyfile
, uncommentimport siasky.net
sudo docker-compose up -d
to restart the services so they pick up new env variablessudo docker exec caddy caddy reload --config /etc/caddy/Caddyfile
to reload Caddyfile configuration
Useful Commands
- Accessing siac for both nodes
siac
for download nodesiac-upload
for upload node
- Checking status of siad service
systemctl --user status siad
for download nodesystemctl --user status siad-upload
for upload node
- Stopping siad service
systemctl --user stop siad
for download nodesystemctl --user stop siad-upload
for upload node
- Starting siad service
systemctl --user start siad
for download nodesystemctl --user start siad-upload
for upload node
- Restarting siad service
systemctl --user restart siad
for download nodesystemctl --user restart siad-upload
for upload node
- Restarting caddy gracefully after making changes to Caddyfile
sudo docker exec caddy caddy reload --config /etc/caddy/Caddyfile
- Restarting nginx gracefully after making changes to nginx configs
sudo docker exec nginx openresty -s reload
- Checking siad service logs (follow last 50 lines)
journalctl -f -n 50 --user-unit siad
for download nodejournalctl -f -n 50 --user-unit siad-upload
for upload node
- Checking caddy logs (for example in case ssl certificate fails)
sudo docker logs caddy -f
- Checking nginx logs (nginx handles all communication to siad instances)
tail -n 50 docker/data/nginx/logs/access.log
to follow last 50 lines of access logtail -n 50 docker/data/nginx/logs/error.log
to follow last 50 lines of error log