Switch to fully secured cockroachdb.

.gitignore

@@ -89,7 +89,10 @@ __pycache__
 /venv*
 
 # CockroachDB certificates
-.cr_certs/
+docker/cockroach/certs/*.crt
+docker/cockroach/certs/*.key
+docker/kratos/cr_certs/*.crt
+docker/kratos/cr_certs/*.key
 
 # Oathkeeper JWKS signing token
 docker/kratos/oathkeeper/id_token.jwks.json

README.md

@@ -19,20 +19,21 @@ List of available parameters:
 
 Mongo needs a couple of extra steps in order to start a secure cluster.
 
-* Open port 27017 on all nodes that will take part in the cluster. Ideally, you
+* Open port 27017 on all nodes that will take part in the cluster. Ideally, you would only open the port for the other
-would only open the port for the other nodes in the cluster.
+  nodes in the cluster.
-* Manually run an initialisation `docker run` with extra environment variables 
+* Manually run an initialisation `docker run` with extra environment variables that will initialise the admin user with
-that will initialise the admin user with a password (example below).
+  a password (example below).
-* Manually add a `mgkey` file under `./docker/data/mongo` with the respective 
+* Manually add a `mgkey` file under `./docker/data/mongo` with the respective secret (
-secret (see [Mongo's keyfile access control](https://docs.mongodb.com/manual/tutorial/enforce-keyfile-access-control-in-existing-replica-set/) for details).
+  see [Mongo's keyfile access control](https://docs.mongodb.com/manual/tutorial/enforce-keyfile-access-control-in-existing-replica-set/)
-* During the initialisation run mentioned above, we need to make two extra steps 
+  for details).
-within the container:
+* During the initialisation run mentioned above, we need to make two extra steps within the container:
     * Change the ownership of `mgkey` to `mongodb:mongodb`
     * Change its permissions to 400
-* After these steps are done we can open a mongo shell on the master node and 
+* After these steps are done we can open a mongo shell on the master node and run `rs.add()` in order to add the new
-run `rs.add()` in order to add the new node to the cluster.
+  node to the cluster.
 
 Example initialisation docker run command:
+
 ```
 docker run \
 	--rm \
@@ -44,7 +45,9 @@ docker run \
 	-v /home/user/skynet-webportal/docker/data/mongo/mgkey:/data/mgkey \
 	mongo --keyFile=/data/mgkey --replSet=skynet
 ```
+
 Regular docker run command:
+
 ```
 docker run \
 	--rm \
@@ -53,7 +56,9 @@ docker run \
 	-v /home/user/skynet-webportal/docker/data/mongo/db:/data/db \
 	-v /home/user/skynet-webportal/docker/data/mongo
 ```
+
 Cluster initialisation mongo command:
+
 ```
 rs.initiate(
   {
@@ -68,19 +73,19 @@ rs.initiate(
 
 ### Kratos & Oathkeeper Setup
 
-[Kratos](https://www.ory.sh/kratos) is our user management system of choice and 
+[Kratos](https://www.ory.sh/kratos) is our user management system of choice and
 [Oathkeeper](https://www.ory.sh/oathkeeper) is the identity and access proxy.
 
-Most of the needed config is already under `docker/kratos`. The only two things
+Most of the needed config is already under `docker/kratos`. The only two things that need to be changed are the config
-that need to be changed are the config for Kratos that might contain you email 
+for Kratos that might contain you email server password, and the JWKS Oathkeeper uses to sign its JWT tokens.
-server password, and the JWKS Oathkeeper uses to sign its JWT tokens.
 
-To override the default `kratos.yml` you can create ` .kratos.yml` in the root 
+To override the default `kratos.yml` you can create ` .kratos.yml` in the root directory of the project, alongside
-directory of the project, alongside the `.env` file.
+the `.env` file.
+
+To override the JWKS you will need to directly edit
+`docker/kratos/oathkeeper/id_token.jwks.json` and replace it with your generated key set. If you don't know how to
+generate a key set you can use this code:
 
-To override the JWKS you will need to directly edit 
-`docker/kratos/oathkeeper/id_token.jwks.json` and replace it with your generated
-key set. If you don't know how to generate a key set you can use this code:
 ```go
 package main
 
@@ -107,52 +112,56 @@ func main() {
 	os.Stdout.Write(jsonbuf)
 }
 ```
-While you can directly put the output of this programme into the file mentioned 
-above, you can also remove the public key from the set and change the `kid` of 
-the private key to not include the prefix `private:`.
 
+While you can directly put the output of this programme into the file mentioned above, you can also remove the public
+key from the set and change the `kid` of the private key to not include the prefix `private:`.
 
 ### CockroachDB Setup
 
-Kratos uses CockroachDB to store its data. For that data to be shared across all
+Kratos uses CockroachDB to store its data. For that data to be shared across all nodes that comprise your portal cluster
-nodes that comprise your portal cluster setup, we need to set up a CockroachDB 
+setup, we need to set up a CockroachDB cluster, complete with secure communication.
-cluster, complete with secure communication.
 
 #### Generate the certificates for secure communication
 
-For a detailed walk-through, please check [this guide](https://www.cockroachlabs.com/docs/v20.2/secure-a-cluster.html) out.
+For a detailed walk-through, please check [this guide](https://www.cockroachlabs.com/docs/v20.2/secure-a-cluster.html)
+out.
 
 Steps:
-1. Start a local cockroach docker instance: 
+
+1. Start a local cockroach docker instance:
    `docker run -d -v "<local dir>:/cockroach/cockroach-secure" --name=crdb cockroachdb/cockroach start --insecure`
 1. Get a shall into that instance: `docker exec -it crdb /bin/bash`
 1. Go to the directory we which we mapped to a local dir: `cd /cockroach/cockroach-secure`
 1. Create the subdirectories in which to create certificates and keys: `mkdir certs my-safe-directory`
-1. Create the CA (Certificate Authority) certificate and key pair: `cockroach cert create-ca --certs-dir=certs --ca-key=my-safe-directory/ca.key`
+1. Create the CA (Certificate Authority) certificate and key
-1. Create a client certificate and key pair for the root user: `cockroach cert create-client root --certs-dir=certs --ca-key=my-safe-directory/ca.key`
+   pair: `cockroach cert create-ca --certs-dir=certs --ca-key=my-safe-directory/ca.key`
-1. Create the certificate and key pair for your nodes: `cockroach cert create-node mynode.siasky.net --certs-dir=certs --ca-key=my-safe-directory/ca.key`
+1. Create a client certificate and key pair for the root
-    1. If you want to create certificates for more nodes, just delete the `node.*`
+   user: `cockroach cert create-client root --certs-dir=certs --ca-key=my-safe-directory/ca.key`
-       files (after you've finished the next step!) and re-run the above 
+1. Create the certificate and key pair for your
-       command with the new node name.
+   nodes: `cockroach cert create-node cockroach mynode.siasky.net --certs-dir=certs --ca-key=my-safe-directory/ca.key`.
-1. Put the contents of the `certs` folder in `.cr_certs/` under your portal's root 
+   Don't forget the `cockroach` node name - it's needed by our docker-compose setup. If you want to create certificates
-   dir and store the content of `my-safe-directory` somewhere safe.
+   for more nodes, just delete the `node.*` files (after you've finished the next steps for this node!) and re-run the
-   
+   above command with the new node name.
+1. Put the contents of the `certs` folder under `docker/cockroach/certs/*` under your portal's root dir and store the
+   content of `my-safe-directory` somewhere safe.
+1. Put *another copy* of those certificates under `docker/kratos/cr_certs` and change permissions of the `*.key` files,
+   so they can be read by anyone (644).
+
 #### Configure your CockroachDB node
 
 There is some configuration that needs to be added to your `.env`file, namely:
+
 1. CR_NODE - the name of your node
 1. CR_IP - the public IP of your node
-1. CR_CLUSTER_NODES - a list of IPs and ports which make up your cluster, e.g. 
+1. CR_CLUSTER_NODES - a list of IPs and ports which make up your cluster, e.g.
-   `95.216.13.185:26257,147.135.37.21:26257,144.76.136.122:26257`. This will be 
+   `95.216.13.185:26257,147.135.37.21:26257,144.76.136.122:26257`. This will be the list of nodes that will make up your
-   the list of nodes that will make up your cluster, so make sure those are 
+   cluster, so make sure those are accurate.
-   accurate.
 
 ## Contributing
 
 ### Testing Your Code
 
-Before pushing your code you should verify that it will pass our online test
+Before pushing your code, you should verify that it will pass our online test suite.
-suite.
 
 **Cypress Tests**
 Verify the Cypress test suite by doing the following:

docker-compose.yml

@@ -208,11 +208,12 @@ services:
     restart: on-failure
     logging: *default-logging
     environment:
-      - DSN=cockroach://root@cockroach:26257/defaultdb?sslmode=disable&max_conns=20&max_idle_conns=4
+      - DSN=cockroach://root@cockroach:26257/defaultdb?max_conns=20&max_idle_conns=4&sslmode=verify-full&sslcert=/certs/node.crt&sslkey=/certs/node.key&sslrootcert=/certs/ca.crt
       - SQA_OPT_OUT=true
     volumes:
       - ./docker/kratos/config:/etc/config/kratos
       - ./docker/data/cockroach/sqlite:/var/lib/sqlite
+      - ./docker/kratos/cr_certs:/certs
     command: -c /etc/config/kratos/kratos.yml migrate sql -e --yes
     networks:
       shared:
@@ -230,7 +231,7 @@ services:
       - 4433 # public
       - 4434 # admin
     environment:
-      - DSN=cockroach://root@cockroach:26257/defaultdb?sslmode=disable&max_conns=20&max_idle_conns=4
+      - DSN=cockroach://root@cockroach:26257/defaultdb?max_conns=20&max_idle_conns=4&sslmode=verify-full&sslcert=/certs/node.crt&sslkey=/certs/node.key&sslrootcert=/certs/ca.crt
       - LOG_LEVEL=trace
       - SERVE_PUBLIC_BASE_URL=/.ory/kratos/public/
       - SQA_OPT_OUT=true
@@ -239,6 +240,7 @@ services:
       - ./docker/kratos/config:/etc/config/kratos
       - ./docker/data/cockroach/sqlite:/var/lib/sqlite
       - ./.kratos.yml:/etc/config/kratos/kratos.yml
+      - ./docker/kratos/cr_certs:/certs
     networks:
       shared:
         ipv4_address: 10.10.10.81
@@ -293,10 +295,10 @@ services:
   cockroach:
     image: cockroachdb/cockroach:v20.1.10
     container_name: cockroach
-    command: start --advertise-addr=$CR_IP --join=$CR_CLUSTER_NODES --certs-dir=/certs --listen-addr=0.0.0.0:26257 --http-addr=0.0.0.0:8080
+    command: start --advertise-addr=${CR_IP} --join=${CR_CLUSTER_NODES} --certs-dir=/certs --listen-addr=0.0.0.0:26257 --http-addr=0.0.0.0:8080
     volumes:
       - ./docker/data/cockroach/sqlite:/cockroach/cockroach-data
-      - ./.cr_certs:/certs
+      - ./docker/cockroach/certs:/certs
     expose:
       - 8080
       - 26257

docker/cockroach/certs/README.md

@@ -0,0 +1,2 @@
+This directory needs to contain all certificates needed by this cockroachdb node. Those can be generated by the steps
+outlined in the README in the root directory, under "Setting up CockroachDB".

docker/kratos/cr_certs/README.md

@@ -0,0 +1,7 @@
+This directory needs to contain all certificates needed by this cockroachdb node. Those can be generated by the steps
+outlined in the README in the root directory, under "Setting up CockroachDB".
+
+The only difference between the files here and those under
+`docker/cockroach/certs` is that the files here need to be readable by anyone, while the files under `cockroach` need to
+have their original access rights
+(all *.key files should be 600 instead of 644 there).