added docs for storage-node

docs/caching-nodes.md

@@ -10,12 +10,13 @@ Caching nodes are a special kind of node that works as a cache with a local NVMe
 
 Make sure that the Kubernetes worker nodes to be used for cache has access to the simplyblock storage cluster. If you are using terraform to deploy the cluster. Please attach `container-instance-sg` security group to all the instances.
 
-#### Step1: Install nvme cli tools
+#### Step1: Install nvme cli tools and nbd
 
 To attach NVMe device to the host machine, the CSI driver uses [nvme-cli]([url](https://github.com/linux-nvme/nvme-cli)). So lets install that
 ```
 sudo yum install -y nvme-cli
 sudo modprobe nvme-tcp
+sudo modprobe nbd
 ```
 
 #### Step1: Setup hugepages
@@ -58,13 +59,13 @@ lspci
 
 After the nodes are prepared, label the kubernetes nodes
 ```
-kubectl label nodes ip-10-0-4-118.us-east-2.compute.internal ip-10-0-4-176.us-east-2.compute.internal type=cache
+kubectl label nodes ip-10-0-4-118.us-east-2.compute.internal ip-10-0-4-176.us-east-2.compute.internal type=simplyblock-cache
 ```
 Now the nodes are ready to deploy caching nodes.
 
 ### StorageClass
 
-If the user wants to create a PVC that uses NVMe cache, a new storage class can be used with additional volume parameter as `type: simplyblock-cache`.
+If the user wants to create a PVC that uses NVMe cache, a new storage class can be used with additional volume parameter as `type: cache`.
 
 
 ### Usage and Implementation

docs/storage-nodes.md

@@ -0,0 +1,63 @@
+### Storage nodes volume provisioning
+
+We now also have the concept of storage-nodes on kubernetes. These will be Kubernetes nodes which reside within the kubernetes compute cluster as normal worker nodes and can have any PODs deployed. However, they have either NVMe disk locally attached with partitions or ebs volumes without partitions.
+
+
+### Preparing nodes
+
+#### Step 0: Networking & tools
+
+Make sure that the Kubernetes worker nodes used as storage-node has access to the simplyblock cluster. If you are using terraform to deploy the cluster. Please attach `container-instance-sg` security group to all the instances.
+
+#### Step1: Install nvme cli tools and nbd
+
+To attach NVMe device to the host machine, the CSI driver uses [nvme-cli]([url](https://github.com/linux-nvme/nvme-cli)). So lets install that
+```
+sudo yum install -y nvme-cli
+sudo modprobe nvme-tcp
+sudo modprobe nbd
+```
+
+#### Step1: Setup hugepages
+
+Before you prepare the storage nodes, please decide the amount of huge pages that you would like to allocate for simplyblock and set those hugepages accordingly. We suggest allocating at least 8GB of huge pages. 
+
+>[!IMPORTANT]
+>The storage node requires at least 2.2% of the size of the nvme cache + 50 MiB of RAM. This should be the minimum configured as hugepage
+>memory.
+
+```
+sudo sysctl -w vm.nr_hugepages=4096
+```
+
+confirm the hugepage changes by running
+cat /proc/meminfo | grep -i hug
+
+
+and restart kubelet
+```
+sudo systemctl restart kubelet
+```
+
+conform if huge pages are added to the cluster or not.
+```
+kubectl describe node ip-10-0-2-184.us-east-2.compute.internal | grep hugepages-2Mi
+```
+this output should show 8GB. This worker node can allocate 8GB of hugepages to pods which is required in case of SPDK pods.
+
+#### Step2: Mount the SSD or EBS to be used by the storage node
+If the instance comes with a default NVMe disk, it can be used with minimum of 2 partitions and 2 device where one is used for Journal manager and the other storage node. Or 2 additional EBS one for Journal Manager and the other for the Storage. the disks can be viewed by running:
+
+```
+sudo yum install pciutils
+lspci
+```
+
+
+#### Step3: Tag the kubernetes nodes
+
+After the nodes are prepared, label the kubernetes nodes
+```
+kubectl label nodes ip-10-0-4-118.us-east-2.compute.internal ip-10-0-4-176.us-east-2.compute.internal type=simplyblock-storage-plane
+```
+Now the nodes are ready to deploy storage nodes.