improve docs

README.md

@@ -18,11 +18,13 @@ List of available parameters:
 ## Contributing
 
 ### Testing you Code
-Before pushing your code you should verify that it will pass our online test
-suite. 
 
-**Cypress Tests**  
+Before pushing your code you should verify that it will pass our online test
+suite.
+
+**Cypress Tests**
 Verify the Cypress test suite by doing the following:
+
 1. In one terminal screen run `GATSBY_API_URL=https://siasky.net yarn workspace webapp start`
 1. In a second terminal screen run `yarn workspace webapp cypress run`
 

setup-scripts/README.md

@@ -15,21 +15,22 @@ You may want to fork this repository and replace ssh keys in
   - [sia](https://sia.tech) ([docker hub](https://hub.docker.com/r/nebulouslabs/sia)): storage provider, heart of the portal setup
   - [caddy](https://caddyserver.com) ([docker hub](https://hub.docker.com/r/caddy/caddy)): reverse proxy (similar to nginx) that handles ssl out of a box and acts as a transparent entry point
   - [openresty](https://openresty.org) ([docker hub](https://hub.docker.com/r/openresty/openresty)): nginx custom build, acts as a cached proxy to siad and exposes all api endpoints
-  - health-check: this is a simple service that runs periodically and collects health data about the server (status and response times) and exposes `/health-check` api endpoint that is deliberately delayed based on the response times of the server so potential load balancer could prioritize servers based on that (we use it with cloudflare)
-- siad setup: we use "double siad" setup that has one node solely for download and one for upload to improve performance
-  - we use systemd to manage siad service
-  - siad is not installed as docker service for improved performance
+  - [health-check](https://github.com/NebulousLabs/skynet-webportal/tree/master/packages/health-check): simple service that runs periodically and collects health data about the server (status and response times) - [read more](https://github.com/NebulousLabs/skynet-webportal/blob/master/packages/health-check/README.md)
+  - [handshake](https://handshake.org) ([github](https://github.com/handshake-org/hsd)): full handshake node
+  - [handshake-api](https://github.com/NebulousLabs/skynet-webportal/tree/master/packages/handshake-api): simple API talking to the handshake node - [read more](https://github.com/NebulousLabs/skynet-webportal/blob/master/packages/handshake-api/README.md)
+  - [webapp](https://github.com/NebulousLabs/skynet-webportal/tree/master/packages/webapp): portal frontend application - [read more](https://github.com/NebulousLabs/skynet-webportal/blob/master/packages/webapp/README.md)
 - discord integration
   - [funds-checker](funds-checker.py): script that checks wallet balance and sends status messages to discord periodically
+  - [health-checker](health-checker.py): script that monitors health-check service for server health issues and reports them to discord periodically
   - [log-checker](log-checker.py): script that scans siad logs for critical errors and reports them to discord periodically
 - [blacklist-skylink](blacklist-skylink.sh): script that can be run locally from a machine that has access to all your skynet portal servers that blacklists provided skylink and prunes nginx cache to ensure it's not available any more (that is a bit much but that's the best we can do right now without paid nginx version) - if you want to use it, make sure to adjust the server addresses
 
 ### Step 1: setting up server user
 
 1. SSH in a freshly installed Debian machine on a user with sudo access (can be root)
-1. `apt-get update && apt-get install sudo` to make sure `sudo` is available
+1. `apt-get update && apt-get install sudo -y` to make sure `sudo` is available
 1. `adduser user` to create user called `user` (creates `/home/user` directory)
-1. `usermod -a -G sudo user` to add this new user to sudo group
+1. `usermod -aG sudo user` to add this new user to sudo group
 1. Quit the ssh session with `exit` command
 
 You a can now ssh into your machine as the user `user`.
@@ -42,7 +43,7 @@ You a can now ssh into your machine as the user `user`.
 
 **Following step will be executed on remote host logged in as a `user`:**
 
-1. `sudo apt-get install git` to install git
+1. `sudo apt-get install git -y` to install git
 1. `git clone https://github.com/NebulousLabs/skynet-webportal`
 1. run setup scripts in the exact order and provide sudo password when asked (if one of them fails, you can retry just this one before proceeding further)
    1. `/home/user/skynet-webportal/setup-scripts/setup-server.sh`
@@ -74,12 +75,11 @@ At this point we have almost everything running, we just need to set up your wal
 
 ### Step 4: configuring docker services
 
-1. generate and copy sia api token `printf ":$(cat /home/user/.sia/apipassword)" | base64`
 1. edit `/home/user/skynet-webportal/.env` and configure following environment variables
    - `DOMAIN_NAME` (optional) is your domain name if you have it
    - `EMAIL_ADDRESS` (required) is your email address used for communication regarding SSL certification (required)
    - `SIA_WALLET_PASSWORD` (required) is your wallet password (or seed if you did not set a password)
-   - `HSD_API_KEY` (optional) this is a random security key for an optional handshake integration that gets generated automatically
+   - `HSD_API_KEY` (optional) this is a random security key for a handshake integration that gets generated automatically
    - `CLOUDFLARE_AUTH_TOKEN` (optional) if using cloudflare as dns loadbalancer (need to change it in Caddyfile too)
    - `AWS_ACCESS_KEY_ID` (optional) if using route53 as a dns loadbalancer
    - `AWS_SECRET_ACCESS_KEY` (optional) if using route53 as a dns loadbalancer
@@ -107,11 +107,11 @@ At this point we have almost everything running, we just need to set up your wal
   - unsafe (force restart)
     > `docker-compose down sia`
     > `docker compose up -d sia`
-- Restarting caddy gracefully after making changes to Caddyfile
+- Restarting caddy gracefully after making changes to Caddyfile (no downtime)
   > `docker exec caddy caddy reload --config /etc/caddy/Caddyfile`
-- Restarting nginx gracefully after making changes to nginx configs
+- Restarting nginx gracefully after making changes to nginx configs (no downtime)
   > `docker exec nginx openresty -s reload`
-- Checking siad service logs (last hour)
+- Checking siad service logs (since last hour)
   > `docker logs --since 1h $(docker ps -q --filter "name=^sia$")`
 - Checking caddy logs (for example in case ssl certificate fails)
   > `docker logs caddy -f`

setup-scripts/setup-docker-services.sh

@@ -12,6 +12,9 @@ sudo apt-get update
 sudo apt-get install -y docker-ce docker-ce-cli containerd.io
 docker --version # sanity check
 
+# add user to docker group to avoid having to use sudo for every docker command
+sudo usermod -aG docker user
+
 # Install docker-compose
 sudo curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
 sudo chmod +x /usr/local/bin/docker-compose