From d2d15c860b6346af604820ff9bd34b480f58577b Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 19 Jul 2023 07:32:31 +0200 Subject: [PATCH 01/25] docs: update intro --- docs/docs/intro.md | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/docs/docs/intro.md b/docs/docs/intro.md index 91e2b791..c170c431 100644 --- a/docs/docs/intro.md +++ b/docs/docs/intro.md @@ -4,32 +4,42 @@ slug: / # Welcome to MarbleRun -MarbleRun is a framework for creating distributed confidential computing apps. +MarbleRun is a framework for deploying distributed confidential computing applications. +MarbleRun acts as a confidential operator for your deployment. Think of a trusted party in the control plane. -Build your confidential microservices with [EGo, Gramine, or another runtime](features/runtimes.md), distribute them with Kubernetes on an SGX-enabled cluster, and let MarbleRun take care of the rest. Deploy end-to-end secure and verifiable AI pipelines or crunch on sensitive big data in the cloud. Confidential computing at scale has never been easier. +Build your confidential microservices with tools like [EGo, Gramine, or similar runtime](./features/runtimes.md), orchestrate them with Kubernetes on an SGX-enabled cluster, and let MarbleRun take care of the rest. +Deploy end-to-end secure and verifiable AI pipelines or crunch on sensitive big data in the cloud. Confidential computing at scale has never been easier. -MarbleRun guarantees that the topology of your distributed app adheres to a manifest specified in simple JSON. MarbleRun verifies the integrity of services, bootstraps them, and sets up encrypted connections between them. If a node fails, MarbleRun will seamlessly substitute it with respect to the rules defined in the manifest. +MarbleRun simplifies the process by handling much of the groundwork. +It ensures that your app's topology adheres to your specified manifest. +It verifies the identity and integrity of all your services, bootstraps them, and establishes secure, encrypted communication channels. +As your app needs to scale, MarbleRun manages the addition of new instances, ensuring their secure verification. -To keep things simple, MarbleRun issues one concise remote-attestation statement for your whole distributed app. This can be used by anyone to verify the integrity of your distributed app. +MarbleRun provides comprehensive access and privilege management through its role-based access control (RBAC) feature. This can be used to enforce security policies such as 4-eye principles and other verification procedures in your operations. + +For added simplicity, MarbleRun offers a single remote attestation statement for your entire app. Anyone can use this to verify the integrity of your distributed app, making it easier to assure stakeholders of your app's security. ## Key features 🔒 Authentication and integrity verification of microservices based on the manifest -🔑 Secrets management for microservices +🔑 Secrets management for confidential microservices 📃 Provisioning of certificates, configurations, and parameters -🔬 Remote attestation of the entire cluster +🔬 Remote attestation of the entire deployment + + +đŸ‘„ Role-based access control (RBAC), verifiable updates, and multi-stakeholder support ## Overview Logically, MarbleRun consists of two parts, the control plane called *Coordinator* and the data plane called *Marbles*. -The Coordinator needs to be deployed once in your cluster and the Marble layer needs to be integrated with each service. +The Coordinator needs to be deployed once in your cluster, and the Marble layer needs to be integrated with each service. MarbleRun is configured with a simple JSON document called the *manifest*. -It specifies the topology of the distributed app, the infrastructure properties, and provides configuration parameters for each service. +It specifies the distributed app's topology and infrastructure properties and provides configuration parameters for each service. ![overview](_media/overview.svg) From 54ceb0d39f30d6e534596240ff2e2f1ca9ea0d76 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 19 Jul 2023 11:52:09 +0200 Subject: [PATCH 02/25] docs: refactor getting started --- .../concepts.md | 0 .../coordinator.md | 0 .../marbles.md | 0 docs/docs/features/manifest.md | 2 +- docs/docs/{ => getting-started}/examples.md | 0 docs/docs/getting-started/installation.md | 68 +++++++ docs/docs/getting-started/quickstart-sgx.md | 112 ----------- .../getting-started/quickstart-simulation.md | 99 ---------- docs/docs/getting-started/quickstart.md | 184 +++++++++++++++--- docs/docusaurus.config.js | 4 +- docs/sidebars.js | 56 +++--- samples/helloc++/README.md | 2 +- samples/helloworld/README.md | 2 +- 13 files changed, 257 insertions(+), 272 deletions(-) rename docs/docs/{getting-started => architecture}/concepts.md (100%) rename docs/docs/{getting-started => architecture}/coordinator.md (100%) rename docs/docs/{getting-started => architecture}/marbles.md (100%) rename docs/docs/{ => getting-started}/examples.md (100%) create mode 100644 docs/docs/getting-started/installation.md delete mode 100644 docs/docs/getting-started/quickstart-sgx.md delete mode 100644 docs/docs/getting-started/quickstart-simulation.md diff --git a/docs/docs/getting-started/concepts.md b/docs/docs/architecture/concepts.md similarity index 100% rename from docs/docs/getting-started/concepts.md rename to docs/docs/architecture/concepts.md diff --git a/docs/docs/getting-started/coordinator.md b/docs/docs/architecture/coordinator.md similarity index 100% rename from docs/docs/getting-started/coordinator.md rename to docs/docs/architecture/coordinator.md diff --git a/docs/docs/getting-started/marbles.md b/docs/docs/architecture/marbles.md similarity index 100% rename from docs/docs/getting-started/marbles.md rename to docs/docs/architecture/marbles.md diff --git a/docs/docs/features/manifest.md b/docs/docs/features/manifest.md index 0f4bdb63..82d96389 100644 --- a/docs/docs/features/manifest.md +++ b/docs/docs/features/manifest.md @@ -1,7 +1,7 @@ # Manifest and deployment updates The manifest is a JSON document that defines the essential properties of your deployment: allowed software packages, secrets, access control, update policy, etc. -Particularly, the manifest contains all information required by the [Coordinator](../getting-started/coordinator.md) to verify the confidentiality and integrity of a newly spawned [Marble](../getting-started/marbles.md). +Particularly, the manifest contains all information required by the [Coordinator](../architecture/coordinator.md) to verify the confidentiality and integrity of a newly spawned [Marble](../architecture/marbles.md). On successful verification, the Coordinator provisions the Marble with configuration and secrets. As a cluster owner, you can define in the manifest how rigid you want your deployment to be. diff --git a/docs/docs/examples.md b/docs/docs/getting-started/examples.md similarity index 100% rename from docs/docs/examples.md rename to docs/docs/getting-started/examples.md diff --git a/docs/docs/getting-started/installation.md b/docs/docs/getting-started/installation.md new file mode 100644 index 00000000..9b767f97 --- /dev/null +++ b/docs/docs/getting-started/installation.md @@ -0,0 +1,68 @@ +# Installation and setup + +In this guide, you will install MarbleRun into your Kubernetes cluster and deploy a sample confidential application to demonstrate the capabilities of MarbleRun. + +Installing MarbleRun is easy. First, you will install the CLI (command-line interface) onto your local machine. Using this CLI, you’ll then install the control plane onto your Kubernetes cluster. +Finally, you will add your own services and set up a corresponding manifest. + +:::tip + +A working SGX DCAP environment is required for MarbleRun to work. For the ease of exploring and testing we provide a simulation mode with `--simulation` that runs without SGX hardware. +Depending on your setup you may follow the [first steps for SGX-enabled clusters](../getting-started/quickstart.md). Alternatively, if your setup doesn't support SGX, you can follow the [first steps in simulation mode](../getting-started/quickstart.md). + +::: + +## Prerequisites + +Make sure the following requirements are met: + +* Your machine is running Linux on an x86 (AMD64) CPU +* You have access to a Kubernetes cluster and kubectl installed and configured + +Probably the easiest way to get started is to run Kubernetes on your local machine using [minikube](https://minikube.sigs.k8s.io/docs/start/). Please check our [prerequisites](../deployment/kubernetes.md#prerequisites) if you want to setup an SGX-enabled cluster. Another easy way is to use [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal), which offers SGX-enabled nodes. + +You can validate your setup by running: + +```bash +kubectl version --short +``` + +You should see an output with both a Client Version and Server Version component. +Now your cluster is ready and we’ll install the MarbleRun CLI. + +## Install the MarbleRun CLI + +If this is your first time running MarbleRun, you will need to download the MarbleRun command-line interface (CLI) onto your local machine. The CLI will allow you to interact with your MarbleRun deployment. + +To install the CLI, run: + + + + +```bash +wget -P ~/.local/bin https://github.com/edgelesssys/marblerun/releases/latest/download/marblerun +chmod +x ~/.local/bin/marblerun +``` + + + + +```bash +sudo wget -O /usr/local/bin/marblerun https://github.com/edgelesssys/marblerun/releases/latest/download/marblerun +sudo chmod +x /usr/local/bin/marblerun +``` + + + + +Once installed, verify the CLI is running correctly with: + +```bash +marblerun +``` + +You can use the CLI to check if your cluster is configured to run SGX workloads: + +```bash +marblerun precheck +``` diff --git a/docs/docs/getting-started/quickstart-sgx.md b/docs/docs/getting-started/quickstart-sgx.md deleted file mode 100644 index da29f01d..00000000 --- a/docs/docs/getting-started/quickstart-sgx.md +++ /dev/null @@ -1,112 +0,0 @@ -# Quickstart SGX - -## Step 1: Install the control plane onto your cluster - -Install MarbleRun's *Coordinator* control plane by running: - -```bash -marblerun install -``` - -The `marblerun install` command generates a Kubernetes manifest with all the necessary control plane resources. -This includes a deployment for the Coordinator and for MarbleRun's [admission controller.](../features/kubernetes-integration.md) - -Wait for the control plane to finish installing: - -```bash -marblerun check -``` - -This command will wait until all components of MarbleRun are ready to be used or return an error after a timeout period is reached. - -Port forward the Coordinator's Client API: - -```bash -kubectl -n marblerun port-forward svc/coordinator-client-api 4433:4433 --address localhost >/dev/null & -export MARBLERUN=localhost:4433 -``` - -## Step 2: Verify the Coordinator - -After installing the Coordinator we need to verify its integrity. -For this, we utilize SGX remote attestation and obtain the Coordinator's root certificate. - -Verify the quote and get the Coordinator's root certificate - -```bash -marblerun certificate root $MARBLERUN -o marblerun.crt -``` - -The CLI will obtain the Coordinator's remote attestation quote and verify it against the configuration on our [release page](https://github.com/edgelesssys/marblerun/releases/latest/download/coordinator-era.json). -The SGX quote proves the integrity of the Coordinator pod. -The CLI returns a certificate and stores it as `marblerun.crt` in your current directory. -The certificate is bound to the quote and can be used for future verification. -It can also be used as a root of trust for [authenticating your confidential applications](../features/attestation.md). - -## Step 3: Deploy the demo application - -To get a feel for how MarbleRun would work for one of your services, you can install a demo application. -The emojivoto application is a standalone Kubernetes application that uses a mix of gRPC and HTTP calls to allow the users to vote on their favorite emojis. -Created as a demo application for the popular [Linkerd](https://linkerd.io) service mesh, we've made a confidential variant that uses a confidential service mesh for all gRPC and HTTP connections. -Clone the [demo application's repository](https://github.com/edgelesssys/emojivoto.git) from GitHub by running: - -```bash -git clone https://github.com/edgelesssys/emojivoto.git && cd emojivoto -``` - -### Step 3.1: Configure MarbleRun - -MarbleRun guarantees that the topology of your distributed app adheres to a manifest specified in simple JSON. -MarbleRun verifies the integrity of services, bootstraps them, and sets up encrypted connections between them. -The emojivoto demo already comes with a [manifest](https://github.com/edgelesssys/emojivoto/blob/main/tools/manifest.json), which you can deploy onto MarbleRun by running: - -```bash -marblerun manifest set tools/manifest.json $MARBLERUN -``` - -You can check that the state of MarbleRun changed and is now ready to authenticate your services by running: - -```bash -marblerun status $MARBLERUN -``` - -### Step 3.2: Deploy emojivoto - -Finally, install the demo application onto your cluster. -Please make sure you have [Helm](https://helm.sh/docs/intro/install/) ("the package manager for Kubernetes") installed at least at Version v3.2.0. -Install emojivoto into the emojivoto namespace by running: - -```bash -helm install -f ./kubernetes/sgx_values.yaml emojivoto ./kubernetes --create-namespace -n emojivoto -``` - -## Step 4: Watch it run - -You can now check the MarbleRun log and see the services being authenticated by the Coordinator. - -```bash -kubectl -n marblerun logs -f -l edgeless.systems/control-plane-component=coordinator -``` - -Port forward the front-end web service to access it on your local machine by running: - -```bash -kubectl -n emojivoto port-forward svc/web-svc 8443:443 --address 0.0.0.0 -``` - -Now visit [https://localhost:8443](https://localhost:8443). -You'll be presented with a certificate warning because your browser by default doesn't trust certificates signed by MarbleRun. -You can ignore this error for now and proceed to the website.\ -Voila! Your emoji votes have never been safer! - -## That’s it 👏 - -Congratulations, you’re now a MarbleRun user! Here are some suggested next steps: - -* Explore how [MarbleRun takes care of your secrets](../features/secrets-management.md) -* [Add your own service](../workflows/add-service.md) to MarbleRun -* Learn more about [MarbleRun’s architecture](../getting-started/concepts.md) -* Chat with us on [Discord](https://discord.gg/rH8QTH56JN) -* Try out the full demo on [GitHub](https://github.com/edgelesssys/emojivoto) - -Welcome to the MarbleRun community! diff --git a/docs/docs/getting-started/quickstart-simulation.md b/docs/docs/getting-started/quickstart-simulation.md deleted file mode 100644 index ba8aec50..00000000 --- a/docs/docs/getting-started/quickstart-simulation.md +++ /dev/null @@ -1,99 +0,0 @@ -# Quickstart simulation mode - -## Step 1: Install the control plane onto your cluster - -Install MarbleRun's *Coordinator* control plane by running: - -```bash -marblerun install --simulation -``` - -The `marblerun install` command generates a Kubernetes manifest with all the necessary control plane resources. -This includes a deployment for the Coordinator and for MarbleRun's [admission controller.](../features/kubernetes-integration.md) -The simulation flag tells MarbleRun that real SGX hardware might not be present and the SGX-layer should be emulated. - -Wait for the control plane to finish installing: - -```bash -marblerun check -``` - -This command will wait until all components of MarbleRun are ready to be used or return an error after a timeout period is reached. - -Port forward the Coordinator's Client API: - -```bash -kubectl -n marblerun port-forward svc/coordinator-client-api 4433:4433 --address localhost >/dev/null & -export MARBLERUN=localhost:4433 -``` - -## Step 2: Deploy the demo application - -To get a feel for how MarbleRun would work for one of your services, you can install a demo application. -The emojivoto application is a standalone Kubernetes application that uses a mix of gRPC and HTTP calls to allow the users to vote on their favorite emojis. -Created as a demo application for the popular [Linkerd](https://linkerd.io) service mesh, we've made a confidential variant that uses a confidential service mesh for all gRPC and HTTP connections. -Clone the [demo application's repository]( https://github.com/edgelesssys/emojivoto.git) from GitHub by running: - -```bash -git clone https://github.com/edgelesssys/emojivoto.git && cd emojivoto -``` - -### Step 2.1: Configure MarbleRun - -MarbleRun guarantees that the topology of your distributed app adheres to a manifest specified in simple JSON. -MarbleRun verifies the integrity of services, bootstraps them, and sets up encrypted connections between them. -The emojivoto demo already comes with a [manifest](https://github.com/edgelesssys/emojivoto/blob/main/tools/manifest.json), which you can deploy onto MarbleRun by running: - -```bash -marblerun manifest set tools/manifest.json $MARBLERUN --insecure -``` - -Normally, the CLI will verify the Coordinators SGX quote every time it connects to the Coordinators REST interface. -Since we run MarbleRun in simulation mode, and therefore don't actually generate a quote, we use the `--insecure` flag to skip this verification. - -You can check that the state of MarbleRun changed and is now ready to authenticate your services by running: - -```bash -marblerun status $MARBLERUN --insecure -``` - -### Step 2.2: Deploy emojivoto - -Finally, install the demo application onto your cluster. -Please make sure you have [Helm](https://helm.sh/docs/intro/install/) ("the package manager for Kubernetes") installed at least at Version v3.2.0. -Install emojivoto into the emojivoto namespace by running: - -```bash -helm install -f ./kubernetes/nosgx_values.yaml emojivoto ./kubernetes --create-namespace -n emojivoto -``` - -## Step 3: Watch it run - -You can now check the MarbleRun log and see the services being authenticated by the Coordinator. - -```bash -kubectl -n marblerun logs -f -l edgeless.systems/control-plane-component=coordinator -``` - -Port forward the front-end web service to access it on your local machine by running: - -```bash -kubectl -n emojivoto port-forward svc/web-svc 8443:443 --address 0.0.0.0 -``` - -Now visit [https://localhost:8443](https://localhost:8443). -You'll be presented with a certificate warning because your browser by default doesn't trust certificates signed by MarbleRun. -You can ignore this error for now and proceed to the website.\ -Voila! Your emoji votes have never been safer! - -## That’s it 👏 - -Congratulations, you’re now a MarbleRun user! Here are some suggested next steps: - -* Explore how [MarbleRun takes care of your secrets](../features/secrets-management.md) -* [Add your own service](../workflows/add-service.md) to MarbleRun -* Learn more about [MarbleRun’s architecture](../getting-started/concepts.md) -* Chat with us on [Discord](https://discord.gg/rH8QTH56JN) -* Try out the full demo on [GitHub](https://github.com/edgelesssys/emojivoto) - -Welcome to the MarbleRun community! diff --git a/docs/docs/getting-started/quickstart.md b/docs/docs/getting-started/quickstart.md index a0e3adbd..22aa5c7f 100644 --- a/docs/docs/getting-started/quickstart.md +++ b/docs/docs/getting-started/quickstart.md @@ -1,66 +1,190 @@ -# Quickstart +# First steps with MarbleRun -In this guide, you will install MarbleRun into your Kubernetes cluster and deploy a sample confidential application to demonstrate the capabilities of MarbleRun. +The following steps guide you through the process of deploying MarbleRun in your cluster and deploying a sample app. This example assumes that you have successfully [installed MarbleRun](./installation.md), and have access to a Kubernetes cluster. -Installing MarbleRun is easy. First, you will install the CLI (command-line interface) onto your local machine. Using this CLI, you’ll then install the control plane onto your Kubernetes cluster. -Finally, you will add your own services and set up a corresponding manifest. +A working SGX DCAP environment is required for MarbleRun to work. For the ease of exploring and testing we provide a simulation mode with `--simulation` that runs without SGX hardware. +Depending on your setup you may follow the quickstart for SGX-enabled clusters. Alternatively, if your setup doesn't support SGX, you can follow the quickstart in simulation mode by selecting the respective tabs. -:::tip +## Step 1: Install the control plane onto your cluster -A working SGX DCAP environment is required for MarbleRun to work. For the ease of exploring and testing we provide a simulation mode with `--simulation` that runs without SGX hardware. -Depending on your setup you may follow the [quickstart for SGX-enabled clusters](../getting-started/quickstart-sgx.md). Alternatively, if your setup doesn't support SGX, you can follow the [quickstart in simulation mode](../getting-started/quickstart-simulation.md). +Install MarbleRun's *Coordinator* control plane by running: + + + + +```bash +marblerun install +``` + +The `marblerun install` command generates a Kubernetes manifest with all the necessary control plane resources. +This includes a deployment for the Coordinator and for MarbleRun's [admission controller.](../features/kubernetes-integration.md) + + + + +```bash +marblerun install --simulation +``` + +The `marblerun install` command generates a Kubernetes manifest with all the necessary control plane resources. +This includes a deployment for the Coordinator and for MarbleRun's [admission controller.](../features/kubernetes-integration.md) +The simulation flag tells MarbleRun that real SGX hardware might not be present and the SGX-layer should be emulated. + + + + + + +Wait for the control plane to finish installing: + +```bash +marblerun check +``` + +This command will wait until all components of MarbleRun are ready to be used or return an error after a timeout period is reached. + +Port forward the Coordinator's Client API: + +```bash +kubectl -n marblerun port-forward svc/coordinator-client-api 4433:4433 --address localhost >/dev/null & +export MARBLERUN=localhost:4433 +``` + +## Step 2: Verify the Coordinator + +After installing the Coordinator we need to verify its integrity. +For this, we utilize SGX remote attestation and obtain the Coordinator's root certificate. + +Verify the quote and get the Coordinator's root certificate + + + + +```bash +marblerun certificate root $MARBLERUN -o marblerun.crt +``` + + + -::: +```bash +marblerun certificate root $MARBLERUN -o marblerun.crt --insecure +``` +The insecure flag tells MarbleRun that real SGX hardware might not be present and the quote verification should be omitted. + + + + +The CLI will obtain the Coordinator's remote attestation quote and verify it against the configuration on our [release page](https://github.com/edgelesssys/marblerun/releases/latest/download/coordinator-era.json). +The SGX quote proves the integrity of the Coordinator pod. +The CLI returns a certificate and stores it as `marblerun.crt` in your current directory. +The certificate is bound to the quote and can be used for future verification. +It can also be used as a root of trust for [authenticating your confidential applications](../features/attestation.md). + +## Step 3: Deploy the demo application + +To get a feel for how MarbleRun would work for one of your services, you can install a demo application. +The emojivoto application is a standalone Kubernetes application that uses a mix of gRPC and HTTP calls to allow the users to vote on their favorite emojis. +Created as a demo application for the popular [Linkerd](https://linkerd.io) service mesh, we've made a confidential variant that uses a confidential service mesh for all gRPC and HTTP connections. +Clone the [demo application's repository](https://github.com/edgelesssys/emojivoto.git) from GitHub by running: + +```bash +git clone https://github.com/edgelesssys/emojivoto.git && cd emojivoto +``` + +### Step 3.1: Configure MarbleRun + +MarbleRun guarantees that the topology of your distributed app adheres to a manifest specified in simple JSON. +MarbleRun verifies the integrity of services, bootstraps them, and sets up encrypted connections between them. +The emojivoto demo already comes with a [manifest](https://github.com/edgelesssys/emojivoto/blob/main/tools/manifest.json), which you can deploy onto MarbleRun by running: + + + + +```bash +marblerun manifest set tools/manifest.json $MARBLERUN --insecure +``` + + + -## Step 0: Setup +```bash +marblerun manifest set tools/manifest.json $MARBLERUN --insecure +``` -First, ensure you have access to a Kubernetes cluster and kubectl installed and configured. Probably the easiest way to get started is to run Kubernetes on your local machine using [minikube](https://minikube.sigs.k8s.io/docs/start/). Please check our [prerequisites](../deployment/kubernetes.md#prerequisites) if you want to setup an SGX-enabled cluster. Another easy way is to use [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal), which offers SGX-enabled nodes. + + -You can validate your setup by running: + +You can check that the state of MarbleRun changed and is now ready to authenticate your services by running: + + + ```bash -kubectl version --short +marblerun status $MARBLERUN ``` -You should see an output with both a Client Version and Server Version component. -Now your cluster is ready and we’ll install the MarbleRun CLI. + + + +```bash +marblerun status $MARBLERUN --insecure +``` -## Step 0.5: Install the CLI + + -If this is your first time running MarbleRun, you will need to download the MarbleRun command-line interface (CLI) onto your local machine. The CLI will allow you to interact with your MarbleRun deployment. +### Step 3.2: Deploy emojivoto -To install the CLI, run: +Finally, install the demo application onto your cluster. +Please make sure you have [Helm](https://helm.sh/docs/intro/install/) ("the package manager for Kubernetes") installed at least at Version v3.2.0. +Install emojivoto into the emojivoto namespace by running: - - + + ```bash -wget -P ~/.local/bin https://github.com/edgelesssys/marblerun/releases/latest/download/marblerun -chmod +x ~/.local/bin/marblerun +helm install -f ./kubernetes/sgx_values.yaml emojivoto ./kubernetes --create-namespace -n emojivoto ``` - + ```bash -sudo wget -O /usr/local/bin/marblerun https://github.com/edgelesssys/marblerun/releases/latest/download/marblerun -sudo chmod +x /usr/local/bin/marblerun +helm install -f ./kubernetes/nosgx_values.yaml emojivoto ./kubernetes --create-namespace -n emojivoto ``` -Once installed, verify the CLI is running correctly with: +## Step 4: Watch it run + +You can now check the MarbleRun log and see the services being authenticated by the Coordinator. ```bash -marblerun +kubectl -n marblerun logs -f -l edgeless.systems/control-plane-component=coordinator ``` -You can use the CLI to check if your cluster is configured to run SGX workloads: +Port forward the front-end web service to access it on your local machine by running: ```bash -marblerun precheck +kubectl -n emojivoto port-forward svc/web-svc 8443:443 --address 0.0.0.0 ``` -If your cluster supports SGX, you can follow the [quickstart for clusters with SGX support.](../getting-started/quickstart-sgx.md) -Otherwise, please follow the [quickstart in simulation mode.](../getting-started/quickstart-simulation.md) +Now visit [https://localhost:8443](https://localhost:8443). +You'll be presented with a certificate warning because your browser by default doesn't trust certificates signed by MarbleRun. +You can ignore this error for now and proceed to the website.\ +Voila! Your emoji votes have never been safer! + +## That’s it 👏 + +Congratulations, you’re now a MarbleRun user! Here are some suggested next steps: + +* Explore how [MarbleRun takes care of your secrets](../features/secrets-management.md) +* [Add your own service](../workflows/add-service.md) to MarbleRun +* Learn more about [MarbleRun’s architecture](../architecture/concepts.md) +* Chat with us on [Discord](https://discord.gg/rH8QTH56JN) +* Try out the full demo on [GitHub](https://github.com/edgelesssys/emojivoto) + +Welcome to the MarbleRun community! diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js index 0e08a42e..0e7f11d1 100644 --- a/docs/docusaurus.config.js +++ b/docs/docusaurus.config.js @@ -158,11 +158,11 @@ async function createConfig() { }, { label: 'Concepts', - to: '/getting-started/concepts', + to: '/architecture/concepts', }, { label: 'Examples', - to: '/examples', + to: '/getting-started/examples', }, ], }, diff --git a/docs/sidebars.js b/docs/sidebars.js index ce037341..13b7e271 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -32,33 +32,18 @@ const sidebars = { items: [ { type: 'doc', - label: 'Quickstart', - id: 'getting-started/quickstart', - }, - { - type: 'doc', - label: 'Quickstart SGX', - id: 'getting-started/quickstart-sgx', - }, - { - type: 'doc', - label: 'Quickstart Simulation', - id: 'getting-started/quickstart-simulation', - }, - { - type: 'doc', - label: 'Concepts', - id: 'getting-started/concepts', + label: 'Installation', + id: 'getting-started/installation', }, { type: 'doc', - label: 'Coordinator', - id: 'getting-started/coordinator', + label: 'First steps', + id: 'getting-started/quickstart', }, { type: 'doc', - label: 'Marbles', - id: 'getting-started/marbles', + label: 'Examples', + id: 'getting-started/examples', }, ], }, @@ -188,6 +173,30 @@ const sidebars = { }, ], }, + { + type: 'category', + label: 'Architecture', + link: { + 'type': 'generated-index', + }, + items: [ + { + type: 'doc', + label: 'Concepts', + id: 'architecture/concepts', + }, + { + type: 'doc', + label: 'Coordinator', + id: 'architecture/coordinator', + }, + { + type: 'doc', + label: 'Marbles', + id: 'architecture//marbles', + }, + ] + }, { type: 'category', label: 'Building services', @@ -212,11 +221,6 @@ const sidebars = { }, ], }, - { - type: 'doc', - label: 'Examples', - id: 'examples', - }, { type: 'category', label: 'Reference', diff --git a/samples/helloc++/README.md b/samples/helloc++/README.md index 7961cc99..2465a0ef 100644 --- a/samples/helloc++/README.md +++ b/samples/helloc++/README.md @@ -1,6 +1,6 @@ # How to create a C++ Marble -This example shows how to build a confidential C++ application and run it in MarbleRun. This can serve you as a blueprint for making existing applications MarbleRun-ready or creating new [Marbles](https://docs.edgeless.systems/marblerun/getting-started/marbles). If you haven't already, [setup MarbleRun](../../BUILD.md#build) to get ready. +This example shows how to build a confidential C++ application and run it in MarbleRun. This can serve you as a blueprint for making existing applications MarbleRun-ready or creating new [Marbles](https://docs.edgeless.systems/marblerun/architecture/marbles). If you haven't already, [setup MarbleRun](../../BUILD.md#build) to get ready. **Note:** You can run this example on any hardware by simulating the enclave through setting `OE_SIMULATION=1` as environment variable. This might help you to get started with with the development of confidential apps. However, please notice that this bypasses any security. Detailed information on how to develop secure Marbles can be found in [MarbleRun's documentation](https://docs.edgeless.systems/marblerun/workflows/add-service). diff --git a/samples/helloworld/README.md b/samples/helloworld/README.md index d8d6e6ea..02fe2749 100644 --- a/samples/helloworld/README.md +++ b/samples/helloworld/README.md @@ -1,6 +1,6 @@ # How to build a helloworld Marble -This example shows how to build a confidential Go application with [EGo](https://ego.dev) and run it in MarbleRun. This can serve you as a blueprint for making existing applications MarbleRun-ready or creating new [Marbles](https://docs.edgeless.systems/marblerun/getting-started/marbles). If you haven't already, [setup MarbleRun](../../BUILD.md#build) and EGo to get ready. +This example shows how to build a confidential Go application with [EGo](https://ego.dev) and run it in MarbleRun. This can serve you as a blueprint for making existing applications MarbleRun-ready or creating new [Marbles](https://docs.edgeless.systems/marblerun/architecture/marbles). If you haven't already, [setup MarbleRun](../../BUILD.md#build) and EGo to get ready. **Note:** You can run this example on any hardware by simulating the enclave through setting `OE_SIMULATION=1` as environment variable. This might help you to get started with with the development of confidential apps. However, please notice that this bypasses any security. Detailed information on how to develop secure Marbles can be found in [MarbleRun's documentation](https://docs.edgeless.systems/marblerun/workflows/add-service). From 98c762ebb7cf78ca9845ca16fe08425a4e550446 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 19 Jul 2023 15:06:01 +0200 Subject: [PATCH 03/25] docs: refactor deployments --- .../{cloud.md => platforms/alibaba.md} | 39 +++++-------------- docs/docs/deployment/platforms/azure.md | 31 +++++++++++++++ .../deployment/{ => platforms}/on-prem.md | 0 docs/docs/deployment/platforms/platforms.md | 16 ++++++++ 4 files changed, 57 insertions(+), 29 deletions(-) rename docs/docs/deployment/{cloud.md => platforms/alibaba.md} (53%) create mode 100644 docs/docs/deployment/platforms/azure.md rename docs/docs/deployment/{ => platforms}/on-prem.md (100%) create mode 100644 docs/docs/deployment/platforms/platforms.md diff --git a/docs/docs/deployment/cloud.md b/docs/docs/deployment/platforms/alibaba.md similarity index 53% rename from docs/docs/deployment/cloud.md rename to docs/docs/deployment/platforms/alibaba.md index 03d0f8d4..01bf96b4 100644 --- a/docs/docs/deployment/cloud.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -1,26 +1,22 @@ -# Cloud deployment +## Alibaba Cloud Container Service for Kubernetes (ACK) -This guide walks you through setting up MarbleRun on different CSP offerings individually. - -## Azure confidential computing VMs - -[Azure confidential computing services](https://azure.microsoft.com/en-us/solutions/confidential-compute/) provide access to VMs with Intel SGX enabled in [DCsv2 VM instances](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv2-series). -The description below uses a VM running Ubuntu 18.04. +Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment technique relying on Alibaba's cloud resources. +[ACK hosts Kubernetes pods in SGX-capabale Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing?spm=a2c63.p38356.0.0.31b6925dUJDMZC#task-2490003) and exposes the underlying SGX hardware. ### Prerequisites +* Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX enabled worker nodes. +* [Update and install the Intel DCAP Quote Provider Library](./alibaba.md#prerequisites-1) -* [Update and install EGo](https://github.com/edgelesssys/ego#install) -* [Update and install the Azure DCAP client](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#3-install-the-intel-and-open-enclave-packages-and-dependencies) ### Deploy MarbleRun -You can run MarbleRun standalone on your Azure DCsv2 VM, see our [standalone guide](../deployment/standalone.md). -Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. +See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your ACK cluster. + ## Alibaba Cloud Elastic Compute Service -With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/doc-detail/207734.htm) users can try out and use Intel SGX on Alibaba Cloud. -Currently, security-enhanced instances are only available as part of an invitational preview. +With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/doc-detail/207734.htm) users can use Intel SGX on Alibaba Cloud. +You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances#task-2038128) instance. The description below uses a VM running Ubuntu 18.04. @@ -59,7 +55,7 @@ The description below uses a VM running Ubuntu 18.04. :::note - Currently, the Alibaba Cloud SGX remote attestation service is only supported within [mainland China regions](https://www.alibabacloud.com/help/doc-detail/40654.htm#concept-2459516) + Currently, the Alibaba Cloud SGX remote attestation service is only supported within [mainland China regions, Singapore, and Indonesia](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/build-an-sgx-encrypted-computing-environment) ::: @@ -69,18 +65,3 @@ The description below uses a VM running Ubuntu 18.04. You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../deployment/standalone.md). Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. - -## Azure Kubernetes Services (AKS) - -Azure Kubernetes Service (AKS) offers a popular deployment technique relying on -Azure's cloud resources. AKS hosts Kubernetes pods in Azure confidential compute -VMs and exposes the underlying confidential compute hardware. - -### Prerequisites - -Follow the instructions on the [AKS Confidential Computing Quick Start guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-enclave-nodes-aks-get-started) -to provision an AKS cluster with Intel SGX enabled worker nodes. - -### Deploy MarbleRun - -See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your AKS cluster. diff --git a/docs/docs/deployment/platforms/azure.md b/docs/docs/deployment/platforms/azure.md new file mode 100644 index 00000000..dd243e09 --- /dev/null +++ b/docs/docs/deployment/platforms/azure.md @@ -0,0 +1,31 @@ +## Azure Kubernetes Services (AKS) + +Azure Kubernetes Service (AKS) offers a popular deployment technique relying on +Azure's cloud resources. AKS hosts Kubernetes pods in SGX-capabale Azure VMs and exposes the underlying SGX hardware. + +### Prerequisites + +* Follow the instructions on the [AKS Confidential Computing Quick Start guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-enclave-nodes-aks-get-started) to provision an AKS cluster with Intel SGX enabled worker nodes. +* [Update and install the Azure DCAP client](https://learn.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#install-azure-dcap-client) + + +### Deploy MarbleRun + +See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your AKS cluster. + +## Azure confidential computing VMs + +[Azure confidential computing services](https://learn.microsoft.com/en-us/azure/confidential-computing/virtual-machine-solutions-sgx) provide access to VMs with Intel SGX enabled. +You can follow their [quickstart](https://learn.microsoft.com/en-us/azure/confidential-computing/quick-create-portal) to create a [DCsv2](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv2-series) (Coffee Lake) or [DCsv3](https://learn.microsoft.com/en-us/azure/virtual-machines/dcv3-series) (Ice Lake) VMs with Intel SGX enabled + +The description below uses a VM running Ubuntu 18.04. + +### Prerequisites + +* [Update and install EGo](https://github.com/edgelesssys/ego#install) +* [Update and install the Azure DCAP client](https://learn.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#install-azure-dcap-client) + +### Deploy MarbleRun + +You can run MarbleRun standalone on your Azure DCsv2/3 VM, see our [standalone guide](../deployment/standalone.md). +Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. diff --git a/docs/docs/deployment/on-prem.md b/docs/docs/deployment/platforms/on-prem.md similarity index 100% rename from docs/docs/deployment/on-prem.md rename to docs/docs/deployment/platforms/on-prem.md diff --git a/docs/docs/deployment/platforms/platforms.md b/docs/docs/deployment/platforms/platforms.md new file mode 100644 index 00000000..78a219b3 --- /dev/null +++ b/docs/docs/deployment/platforms/platforms.md @@ -0,0 +1,16 @@ +# Supported Platforms + +MarbleRun supports all platforms where SGX and the [Data Center Attestation Primitives (DCAP)](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf) is available. + +The following gives you an updated list of supported platforms: + + +| Platform | Service | DCAP | +|---------------|------------|-------------------| +| Azure | VMs | Managed | +| | AKS | Managed | +| Alibaba Cloud | VMs | Managed | +| | ACK | Managed | +| Equinix | Bare-Metal | Customer-operated | +| IBM Cloud | Bare-Metal | Customer-operated | +| On-Premises | * | Customer-operated | From d080b06efc410ba67ea67dd74c33656acf7c2563 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Mon, 24 Jul 2023 08:47:01 +0200 Subject: [PATCH 04/25] docs: rename to building enclaves --- docs/docs/{building-services => building-enclaves}/ego.md | 0 .../{building-services => building-enclaves}/gramine.md | 0 .../{building-services => building-enclaves}/occlum.md | 0 docs/docs/deployment/standalone.md | 2 +- docs/sidebars.js | 8 ++++---- 5 files changed, 5 insertions(+), 5 deletions(-) rename docs/docs/{building-services => building-enclaves}/ego.md (100%) rename docs/docs/{building-services => building-enclaves}/gramine.md (100%) rename docs/docs/{building-services => building-enclaves}/occlum.md (100%) diff --git a/docs/docs/building-services/ego.md b/docs/docs/building-enclaves/ego.md similarity index 100% rename from docs/docs/building-services/ego.md rename to docs/docs/building-enclaves/ego.md diff --git a/docs/docs/building-services/gramine.md b/docs/docs/building-enclaves/gramine.md similarity index 100% rename from docs/docs/building-services/gramine.md rename to docs/docs/building-enclaves/gramine.md diff --git a/docs/docs/building-services/occlum.md b/docs/docs/building-enclaves/occlum.md similarity index 100% rename from docs/docs/building-services/occlum.md rename to docs/docs/building-enclaves/occlum.md diff --git a/docs/docs/deployment/standalone.md b/docs/docs/deployment/standalone.md index 09d19b6b..23f15576 100644 --- a/docs/docs/deployment/standalone.md +++ b/docs/docs/deployment/standalone.md @@ -38,7 +38,7 @@ See the [how to add a service](../workflows/add-service.md) documentation for mo ### Run your workloads You first need to build your workloads together with our Marble data plane. -See our guides for building [EGo](../building-services/ego.md), [Gramine](../building-services/gramine.md), and [Occlum](../building-services/occlum.md) workloads. +See our guides for building [EGo](../building-enclaves/ego.md), [Gramine](../building-enclaves/gramine.md), and [Occlum](../building-enclaves/occlum.md) workloads. You can then run your Marble as follows: diff --git a/docs/sidebars.js b/docs/sidebars.js index 13b7e271..dd6866fd 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -199,7 +199,7 @@ const sidebars = { }, { type: 'category', - label: 'Building services', + label: 'Building Enclaves', link: { type: 'generated-index', }, @@ -207,17 +207,17 @@ const sidebars = { { type: 'doc', label: 'EGo', - id: 'building-services/ego', + id: 'building-enclaves/ego', }, { type: 'doc', label: 'Gramine', - id: 'building-services/gramine', + id: 'building-enclaves/gramine', }, { type: 'doc', label: 'Occlum', - id: 'building-services/occlum', + id: 'building-enclaves/occlum', }, ], }, From 4276f948ec03bed001cbcd3116309e10e12b2254 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Tue, 25 Jul 2023 18:24:44 +0200 Subject: [PATCH 05/25] docs: add architecture --- docs/docs/_media/cert-chain.svg | 3 + docs/docs/_media/enc-state-distributed.svg | 3 + docs/docs/_media/enc-state-single.svg | 3 + docs/docs/_media/security_architecture.svg | 3 + docs/docs/architecture/coordinator.md | 8 ++ docs/docs/architecture/marbles.md | 5 +- docs/docs/architecture/security.md | 97 ++++++++++++++++++++++ 7 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 docs/docs/_media/cert-chain.svg create mode 100644 docs/docs/_media/enc-state-distributed.svg create mode 100644 docs/docs/_media/enc-state-single.svg create mode 100644 docs/docs/_media/security_architecture.svg create mode 100644 docs/docs/architecture/security.md diff --git a/docs/docs/_media/cert-chain.svg b/docs/docs/_media/cert-chain.svg new file mode 100644 index 00000000..fb4cbdc1 --- /dev/null +++ b/docs/docs/_media/cert-chain.svg @@ -0,0 +1,3 @@ + + +
Leaf Certificates
Leaf Certificates
Intermediate CA
Intermediate CA
Root CA
Root CA
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certi...
Marble Root Certificate
Marble Root Certif...
Marble Certificate
Marble Certificate
signs
signs
Root Private Key
Root...
signs
signs
signs
signs
Intermediate Private Key
Inte...
Marble Private Key
Marb...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/_media/enc-state-distributed.svg b/docs/docs/_media/enc-state-distributed.svg new file mode 100644 index 00000000..390dd8aa --- /dev/null +++ b/docs/docs/_media/enc-state-distributed.svg @@ -0,0 +1,3 @@ + + +
marble-state Secret
marble-state Secret
Encrypted state
Encrypted state
Encrypted DEK
Encrypted DEK
Coordinator state
Coordinator state
Data Encryption Key
Data Encryption Key
Key Encryption Key
Key Encryption Key
randomly generated by the first Coordinator after receiving the manifest
randomly generated by t...
randomly generated at start up
randomly generated at s...
KEK ConfigMap
KEK ConfigMap
Node-1: enc(KEK)
Node-1: enc(KEK)
Node-2: enc(KEK)
Node-2: enc(KEK)
Node-X: enc(KEK)
Node-X: enc(KEK)
Node-1 SGX 'Product' Sealing Key
Node-1 SGX 'Product' Sealing Key
Node-2 SGX 'Product' Sealing Key
Node-2 SGX 'Product' Sealing Key
Node-X SGX 'Product' Sealing Key
Node-X SGX 'Product' Sealing Key
...
...
...
...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/_media/enc-state-single.svg b/docs/docs/_media/enc-state-single.svg new file mode 100644 index 00000000..8f8acda3 --- /dev/null +++ b/docs/docs/_media/enc-state-single.svg @@ -0,0 +1,3 @@ + + +
Persistent Storage
Persistent Storage
Encrypted state
Encrypted state
Encrypted DEK
Encrypted DEK
Coordinator state
Coordinator state
Data Encryption Key
Data Encryption Key
Key Encryption Key
Key Encryption Key
randomly generated by the Coordinator after receiving the manifest
randomly generated by t...
SGX 'Product' Sealing Key
SGX 'Product' Sealing K...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/_media/security_architecture.svg b/docs/docs/_media/security_architecture.svg new file mode 100644 index 00000000..b00cfa3d --- /dev/null +++ b/docs/docs/_media/security_architecture.svg @@ -0,0 +1,3 @@ + + +
MarbleRun Deployment
MarbleRun Deployment
aTLS
gRPC
aTLS...
Coordinator
Coordinator
Manifest
Manifest
Packages
Packages
Marbles
Marbles
Secrets
Secrets
Users
Users
Certificate Authority
Certificate Authority
mTLS
HTTP, gRPC, TCP
mTLS...
Marble A
Marble A
TLS
HTTP, gRPC, TCP
TLS...
aTLS
gRPC
aTLS...
Marble B
Marble B
Marble
Certificate A
Marble...
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certifi...
Marble
Certificate B
Marble...
aTLS
HTTP
aTLS...
Administrator
Administrat...
aTLS
HTTP
aTLS...
Application user
Application...
Parameters A
Parameters A
 Secrets A
 Secrets A
Parameters B
Parameters B
 Secrets B
 Secrets B
Marble
Private Key A
Marble...
Marble
Private Key B
Marble...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/architecture/coordinator.md b/docs/docs/architecture/coordinator.md index cb4a7e5e..7840411e 100644 --- a/docs/docs/architecture/coordinator.md +++ b/docs/docs/architecture/coordinator.md @@ -1,6 +1,14 @@ # Coordinator The Coordinator is MarbleRun's control plane. +Conceptionally it operators as trusted controller in your confidential deployment. +Therefore, it securely holds the deployment's state in form of the [manifest](../features/manifest.md) and the defined [secrets](../features/secrets-management.md). +Based on the manifest, the Coordinator uses [remote attestation](../features/attestation.md) to authenticate the application's enclaves. + + + +## API and Configuration + It communicates with the Marble's data plane through gRPC and provides an HTTP REST interface on the client-side. The Coordinator can be configured with several environment variables: diff --git a/docs/docs/architecture/marbles.md b/docs/docs/architecture/marbles.md index 85aa7cd5..5db3e264 100644 --- a/docs/docs/architecture/marbles.md +++ b/docs/docs/architecture/marbles.md @@ -1,3 +1,6 @@ # Marbles -Marbles are MarbleRun's data planes and run your application code in secure enclaves within otherwise regular Docker containers. Marbles communicate with the Coordinator via gRPC over TLS. See the [Add a Service](../workflows/add-service.md) section on how to build and configure a Marble. +Marbles are MarbleRun's data planes and run your application code in secure enclaves within otherwise regular Docker containers. +Marbles communicate with the Coordinator via gRPC over TLS. +MarbleRun natively [supports multiple enclave runtimes](../features/runtimes.md) as the data plane. +See the [Add a Service](../workflows/add-service.md) section on how to build and configure an enclave as Marble. diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md new file mode 100644 index 00000000..dfc09b36 --- /dev/null +++ b/docs/docs/architecture/security.md @@ -0,0 +1,97 @@ +# Key management and cryptographic primitives + +MarbleRun protects and isolates your deployments and workloads. To that end, cryptography is the foundation that ensures the confidentiality and integrity of all components. +Evaluating the security and compliance of MarbleRun requires a precise understanding of the cryptographic primitives and keys used. +The following gives an overview of the architecture and explains the technical details. + +## High-level architecture + +The [Coordinator](coordinator.md) and the [Marbles](marbles.md) run inside SGX Enclaves. See [Intel's documentation](https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/overview.html) on the architecture details and cryptographic primitives of SGX. + +MarbleRun uses cryptography for the following tasks. + +* Authentication and authorization +* Establishing a Public Key Infrastructure (PKI) for MarbleRun +* Encrypting network traffic via mutual TLS between enclaves +* Encrypting persistent state + +The following graphic gives an overview of the architecture and the components. + +![Security architecture](../_media/security_architecture.svg) + +## Authentication and authorization + +MarbleRun uses the SGX remote attestation capability to authenticate the Coordinator and the Marble enclaves. See our section on [attestation](../features/attestation.md) for more information on this process. +For authorization, the [manifest](../features/manifest.md) defines the Marble's access to secrets and keys after successful attestation. +Furthermore, MarbleRun's' [RBAC](../workflows/define-manifest.md#roles) attaches a Marble's identity to a role in the manifest. +Each role is associated with a set of operations that Marble can perform in the deployment. +Roles can also be attached to [users](../workflows/define-manifest.md#users), which are authenticated by the Coordinator using an RSA or ECDSA public key defined in the manifest. + +## Public Key Infrastructure and Certificate Authority + +The Coordinator establishes a public key infrastructure (PKI) for MarbleRun and acts as the Certificate Authority (CA). +The goal of the PKI is to make authentication of confidential applications based on remote attestation accessible and usable. +The Coordinator embeds its attestation statement into its root CA certificate, see the [attested TLS](#attested-tls-atls) section for details behind that concept. +All MarbleRun clients and Marbles can then use the attested root CA certificate for authenticating TLS connections. +This is further illustrated conceptionally in the [attestation](../features/attestation.md) section. We now focus on the cryptography. + +During initialization the Coordinator generates a root x509 certificate and corrisponding asymmetric key pair. +The [Elliptic Curve Digital Signature Algorithm (ECDSA)](https://www.secg.org/sec1-v2.pdf#page=49) is used with curve [P256](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf#page=111). +The root certificate has no expiary date set and lives as long as the MarbleRun deployment. + +Alongside the root certificate, the Coordinator generates an intermediate x509 certificate and corresponding asymmetric key pair, again using ECDSA with P256. +The intermediate certificate is signed by the Coordinator's root certificate and rotated with every update of the manifest. +When you push an update to the manifest (for example bump up the *SecurityVersion* of a Mable) the intermediate certificate will change. +Instances with the new version will not authenticate with instances of the old version and vice versa. +Hence, no data flow is happening between different *SecurityVersions* of your application. +However, the root certificate doesn't change. So you can still verify the Coordinator and your application from the outside and make sure it is the same instance you might have interacted with before. + +The Coordinator creates a second intermediate certificate that is called the *Marble root certificate*. +The Marble root certificate is self-signed using the intermediate certificate's private key implementing a [cross-signed certificate chain](https://www.ssltrust.com.au/blog/understanding-certificate-cross-signing). +In that way, the Marbles see a self-signed root certificate, hence, the are dealing with a terminating certificate chain without knowing about the Coordinator's root certificate. +The "outside world" sees an intermediate certificate signed by the Coordinator's root certificate and the corresponding root certificate. +Applications interacting with the MarbleRun deployment can decide wether to use the intermediate or the root certificate as CA depending on if they want to notice and handle manifest updates explicitely or not. + +For every Marble the Coordinator generates a unique leaf "Marble" certificate and corresponding key pair using ECDSA with P256. +The Marble certificate is signed by the Marble root certificate. +The Marble certificate is provisioned to the Marble's enclave via the secure channel established during the [attestation procedure](../features/attestation.md). + +![Security architecture](../_media/cert-chain.svg) + + +## Attested TLS (aTLS) + +In a CC environment, attested TLS (aTLS) can be used to establish secure connections between two parties using the remote attestation features of the CC components. +aTLS modifies the TLS handshake by embedding an attestation statement into the TLS certificate. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. +The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS). + +### Mutual TLS authentication + +Based on MarbleRun's PKI Marbles are provided with a unique leaf certificate and corresponding private key. +As described above these are generated by the Coordinator and provided during the Marble's initialization via the secure aTLS channel. +Depending on the Marble's runtime the certificate can either be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically]((../features/transparent-TLS.md)) for establishing mutually authenticated TLS connetions. + +## Encryption of state + +The Coordinator holds the MarbleRuns state, which consists of the [manifest](../features/manifest.md), the [managed secrets](../features/secrets-management.md), and the [certificates for its CA](../features/attestation.md). +The state is stored encrypted in persistent storage. Therefore Constellation uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a randomly generated 16-byte Data Encryption Key (KEK) to seal the state to persistent storage. +The DEK is also sealed to persistent storage to recover the state in case of a restart autonomously. +[SGX's sealing feature](https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-intel-sgx-sealing.html) is used for that purpose. +The Coordinator encrypts the DEK with a key encryption key (KEK). +The KEK MarbleRun uses the SGX sealing key called `Product key`, which is bound to its `Product ID` and the enclave's author `MRSIGNER` identity. +In other words, a fresh and benign enclave instance of the same identity can recover that key. +Hence, as long as the Coordinator is restarted on the same CPU, it can obtain the same KEK from the CPU based on its identity, decrypt the DEK and recover its state. + + +![Encrypted state single instance](../_media/enc-state-single.svg) + + +### Distributed Coordinator + +The [distributed Coordinator](../features/) works similarly. However, all Coordinators share the same state stored encrypted in the Kubernetes [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) called *marble-state*. +In contrast to the single instance, the KEK is randomly generated at start-up by the first instance. +The existing Coordinators authenticate every new Coordinator instance via remote attestation, and the KEK is subsequently shared via the secure and attested TLS connection. +Every Coordinator instance uses its own SGX Product (Sealing) Key to seal the KEK into a Kubernetes [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) structure called "KEK." + + +![Encrypted state distributed](../_media/enc-state-distributed.svg) From 6a151236f4da4a79f6e15a2e361cdab1182721ab Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 27 Jul 2023 10:17:49 +0200 Subject: [PATCH 06/25] docs: updating sidebar and fixing links --- docs/docs/architecture/security.md | 4 +-- docs/docs/building-enclaves/occlum.md | 2 +- docs/docs/deployment/kubernetes.md | 2 +- docs/docs/deployment/platforms/alibaba.md | 6 ++-- docs/docs/deployment/platforms/azure.md | 6 ++-- docs/docs/deployment/platforms/on-prem.md | 2 +- docs/docs/deployment/platforms/platforms.md | 2 +- docs/docs/features/runtimes.md | 6 ++-- docs/docs/features/transparent-TLS.md | 2 +- docs/docs/getting-started/quickstart.md | 10 +++--- docs/docs/workflows/add-service.md | 6 ++-- docs/docusaurus.config.js | 17 --------- docs/sidebars.js | 39 +++++++++++++++------ 13 files changed, 52 insertions(+), 52 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index dfc09b36..9b2aade7 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -69,7 +69,7 @@ The protocol can be used by clients to verify a server certificate, by a server Based on MarbleRun's PKI Marbles are provided with a unique leaf certificate and corresponding private key. As described above these are generated by the Coordinator and provided during the Marble's initialization via the secure aTLS channel. -Depending on the Marble's runtime the certificate can either be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically]((../features/transparent-TLS.md)) for establishing mutually authenticated TLS connetions. +Depending on the Marble's runtime the certificate can either be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically](../features/transparent-TLS.md) for establishing mutually authenticated TLS connetions. ## Encryption of state @@ -88,7 +88,7 @@ Hence, as long as the Coordinator is restarted on the same CPU, it can obtain th ### Distributed Coordinator -The [distributed Coordinator](../features/) works similarly. However, all Coordinators share the same state stored encrypted in the Kubernetes [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) called *marble-state*. +The [distributed Coordinator](../features/recovery.md#distributed-coordinator) works similarly. However, all Coordinators share the same state stored encrypted in the Kubernetes [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) called *marble-state*. In contrast to the single instance, the KEK is randomly generated at start-up by the first instance. The existing Coordinators authenticate every new Coordinator instance via remote attestation, and the KEK is subsequently shared via the secure and attested TLS connection. Every Coordinator instance uses its own SGX Product (Sealing) Key to seal the KEK into a Kubernetes [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) structure called "KEK." diff --git a/docs/docs/building-enclaves/occlum.md b/docs/docs/building-enclaves/occlum.md index f0c3ec7d..748a5bdc 100644 --- a/docs/docs/building-enclaves/occlum.md +++ b/docs/docs/building-enclaves/occlum.md @@ -2,7 +2,7 @@ Running an Occlum app with MarbleRun requires some changes to its manifest. ## Requirements -Set up an environment to create Occlum images. For an easy start, we recommend that you use either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or use [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our cloud deployment guide](../deployment/cloud.md). +Set up an environment to create Occlum images. For an easy start, we recommend that you use either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or use [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our platform deployment guide](../deployment/platforms/platforms.md). To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running, and then come back here to adapt it for use with MarbleRun. diff --git a/docs/docs/deployment/kubernetes.md b/docs/docs/deployment/kubernetes.md index cf8e867b..84ded68e 100644 --- a/docs/docs/deployment/kubernetes.md +++ b/docs/docs/deployment/kubernetes.md @@ -81,7 +81,7 @@ Intel SGX supports two modes for obtaining remote attestation quotes: * In-process: The software generating the quote is part of the enclave application * Out-of-process: The software generating the quote isn't part of the actual enclave application. This requires the Intel SGX Architectural Enclave Service Manager (AESM) to run on the system -While Marbles built with [Ego](../building-services/ego.md) perform in-process attestation, other frameworks, such as [Gramine](../building-services/gramine.md), use out-of-process attestation. +While Marbles built with [Ego](../building-enclaves/ego.md) perform in-process attestation, other frameworks, such as [Gramine](../building-enclaves/gramine.md), use out-of-process attestation. If your confidential application uses out-of-process attestation, you will need to expose the AESM device to your container. You can follow [the AKS guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-addon) to make your deployments able to use AESM for quote generation. Note, that in this case, your Kubernetes nodes need the AESM service installed. See the [Intel installation guide](https://download.01.org/intel-sgx/sgx-linux/2.12/docs/Intel_SGX_Installation_Guide_Linux_2.12_Open_Source.pdf) for more information. diff --git a/docs/docs/deployment/platforms/alibaba.md b/docs/docs/deployment/platforms/alibaba.md index 01bf96b4..9a3b891d 100644 --- a/docs/docs/deployment/platforms/alibaba.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -10,7 +10,7 @@ Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment ### Deploy MarbleRun -See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your ACK cluster. +See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in your ACK cluster. ## Alibaba Cloud Elastic Compute Service @@ -63,5 +63,5 @@ The description below uses a VM running Ubuntu 18.04. ### Deploy MarbleRun -You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../deployment/standalone.md). -Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. +You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../standalone.md). +Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. diff --git a/docs/docs/deployment/platforms/azure.md b/docs/docs/deployment/platforms/azure.md index dd243e09..98062d85 100644 --- a/docs/docs/deployment/platforms/azure.md +++ b/docs/docs/deployment/platforms/azure.md @@ -11,7 +11,7 @@ Azure's cloud resources. AKS hosts Kubernetes pods in SGX-capabale Azure VMs and ### Deploy MarbleRun -See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your AKS cluster. +See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in your AKS cluster. ## Azure confidential computing VMs @@ -27,5 +27,5 @@ The description below uses a VM running Ubuntu 18.04. ### Deploy MarbleRun -You can run MarbleRun standalone on your Azure DCsv2/3 VM, see our [standalone guide](../deployment/standalone.md). -Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. +You can run MarbleRun standalone on your Azure DCsv2/3 VM, see our [standalone guide](../standalone.md). +Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. diff --git a/docs/docs/deployment/platforms/on-prem.md b/docs/docs/deployment/platforms/on-prem.md index 91026d44..9328f842 100644 --- a/docs/docs/deployment/platforms/on-prem.md +++ b/docs/docs/deployment/platforms/on-prem.md @@ -90,4 +90,4 @@ Similarly, the [EGo image](https://github.com/orgs/edgelesssys/packages?repo_nam You have made sure your hardware supports SGX, updated all firmware, installed the SGX driver, and configured DCAP on all your machines and VMs? Great! Now it's time to install MarbleRun and get going. -You can either [use MarbleRun in standalone mode](../deployment/standalone.md) or [install it in your Kubernetes cluster](../deployment/kubernetes.md). +You can either [use MarbleRun in standalone mode](../standalone.md) or [install it in your Kubernetes cluster](../kubernetes.md). diff --git a/docs/docs/deployment/platforms/platforms.md b/docs/docs/deployment/platforms/platforms.md index 78a219b3..a2027d5e 100644 --- a/docs/docs/deployment/platforms/platforms.md +++ b/docs/docs/deployment/platforms/platforms.md @@ -1,6 +1,6 @@ # Supported Platforms -MarbleRun supports all platforms where SGX and the [Data Center Attestation Primitives (DCAP)](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf) is available. +MarbleRun supports all platforms where Intel SGX and the [Data Center Attestation Primitives (DCAP)](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf) is available. The following gives you an updated list of supported platforms: diff --git a/docs/docs/features/runtimes.md b/docs/docs/features/runtimes.md index 6f06eb92..fe0eeec4 100644 --- a/docs/docs/features/runtimes.md +++ b/docs/docs/features/runtimes.md @@ -3,7 +3,7 @@ MarbleRun strives to be runtime-agnostic. Currently, supported runtimes are described below. More will follow in the future. ## EGo -[EGo](https://github.com/edgelesssys/ego) is the preferred way for writing confidential Go applications from scratch as well as porting existing ones. Usage is similar to conventional Go programming. Start [building your service with EGo](../building-services/ego.md) to use it with MarbleRun. +[EGo](https://github.com/edgelesssys/ego) is the preferred way for writing confidential Go applications from scratch as well as porting existing ones. Usage is similar to conventional Go programming. Start [building your service with EGo](../building-enclaves/ego.md) to use it with MarbleRun. ## Edgeless RT With [Edgeless RT](https://github.com/edgelesssys/edgelessrt) you can create confidential C++ applications with a low TCB. Please follow the build instructions provided [in our C++ sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) to use it with MarbleRun. @@ -12,9 +12,9 @@ With [Edgeless RT](https://github.com/edgelesssys/edgelessrt) you can create con [Gramine](https://gramineproject.io/) is a popular choice for wrapping unmodified applications into enclaves. This approach, commonly known as "lift and shift," facilitates the process of bringing existing applications into the confidential space. Gramine further adds support for dynamically linked libraries and multi-process applications in SGX. -[Running a Gramine app](../building-services/gramine.md) with MarbleRun requires minor changes to its manifest. +[Running a Gramine app](../building-enclaves/gramine.md) with MarbleRun requires minor changes to its manifest. ## Occlum [Occlum](https://github.com/occlum/occlum) is another popular solution which allows wrapping existing applications with minimal to no changes inside an enclave. It requires you to at best recompile existing applications with the provided toolchains with support for common languages such as C, C++, Go, Java, and Rust. Its core is written in the memory-safe programming language Rust and it provides a separated environment under which your application is running. This provides a safe yet powerful way to build your applications. -[Running an Occlum app](../building-services/occlum.md) with MarbleRun requires minor changes to its manifest. +[Running an Occlum app](../building-enclaves/occlum.md) with MarbleRun requires minor changes to its manifest. diff --git a/docs/docs/features/transparent-TLS.md b/docs/docs/features/transparent-TLS.md index 07f2b993..92f6de3f 100644 --- a/docs/docs/features/transparent-TLS.md +++ b/docs/docs/features/transparent-TLS.md @@ -4,7 +4,7 @@ Authenticated and encrypted connections between services are essential for the s Transparent TLS (TTLS) can wrap any connection in TLS on the MarbleRun layer. MarbleRun adds secure communication to your cluster even if your application doesn't support the required TLS features. Just define the desired [connections in the manifest](../workflows/define-manifest.md#tls). -TTLS is currently available with [EGo Marbles](../building-services/ego.md). Other [runtimes](../features/runtimes.md) will be supported in future. +TTLS is currently available with [EGo Marbles](../building-enclaves/ego.md). Other [runtimes](../features/runtimes.md) will be supported in future. ## Authentication and credentials By default the Marble's credentials are automatically configured. Connections between two Marbles are mutually authenticated. diff --git a/docs/docs/getting-started/quickstart.md b/docs/docs/getting-started/quickstart.md index 22aa5c7f..b5774b82 100644 --- a/docs/docs/getting-started/quickstart.md +++ b/docs/docs/getting-started/quickstart.md @@ -10,7 +10,7 @@ Depending on your setup you may follow the quickstart for SGX-enabled clusters. Install MarbleRun's *Coordinator* control plane by running: - + ```bash marblerun install @@ -58,7 +58,7 @@ For this, we utilize SGX remote attestation and obtain the Coordinator's root ce Verify the quote and get the Coordinator's root certificate - + ```bash marblerun certificate root $MARBLERUN -o marblerun.crt @@ -99,7 +99,7 @@ MarbleRun verifies the integrity of services, bootstraps them, and sets up encry The emojivoto demo already comes with a [manifest](https://github.com/edgelesssys/emojivoto/blob/main/tools/manifest.json), which you can deploy onto MarbleRun by running: - + ```bash marblerun manifest set tools/manifest.json $MARBLERUN --insecure @@ -119,7 +119,7 @@ marblerun manifest set tools/manifest.json $MARBLERUN --insecure You can check that the state of MarbleRun changed and is now ready to authenticate your services by running: - + ```bash marblerun status $MARBLERUN @@ -142,7 +142,7 @@ Please make sure you have [Helm](https://helm.sh/docs/intro/install/) ("the pack Install emojivoto into the emojivoto namespace by running: - + ```bash helm install -f ./kubernetes/sgx_values.yaml emojivoto ./kubernetes --create-namespace -n emojivoto diff --git a/docs/docs/workflows/add-service.md b/docs/docs/workflows/add-service.md index 222b6bb8..da555a65 100644 --- a/docs/docs/workflows/add-service.md +++ b/docs/docs/workflows/add-service.md @@ -5,10 +5,10 @@ Adding a service to your application requires three steps, which are described i ## **Step 1:** Get your service ready for MarbleRun To get your service ready for MarbleRun, you need to rebuild it with one of the supported [runtimes](../features/runtimes.md): -* [EGo](../building-services/ego.md) +* [EGo](../building-enclaves/ego.md) * [Edgeless RT](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) -* [Gramine](../building-services/gramine.md) -* [Occlum](../building-services/occlum.md) +* [Gramine](../building-enclaves/gramine.md) +* [Occlum](../building-enclaves/occlum.md) ### Make your service use the provided TLS credentials diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js index 0e7f11d1..ea955e5e 100644 --- a/docs/docusaurus.config.js +++ b/docs/docusaurus.config.js @@ -149,23 +149,6 @@ async function createConfig() { footer: { style: 'dark', links: [ - { - title: 'Learn', - items: [ - { - label: 'Quickstart', - to: '/getting-started/quickstart', - }, - { - label: 'Concepts', - to: '/architecture/concepts', - }, - { - label: 'Examples', - to: '/getting-started/examples', - }, - ], - }, { title: 'Community', items: [ diff --git a/docs/sidebars.js b/docs/sidebars.js index dd6866fd..3f0f41c7 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -98,16 +98,6 @@ const sidebars = { type: 'generated-index', }, items: [ - { - type: 'doc', - label: 'Cloud', - id: 'deployment/cloud', - }, - { - type: 'doc', - label: 'On-premises', - id: 'deployment/on-prem', - }, { type: 'doc', label: 'Kubernetes', @@ -118,6 +108,28 @@ const sidebars = { label: 'Standalone', id: 'deployment/standalone', }, + { + type: 'category', + label: 'Platforms', + link: { type: 'doc', id: 'deployment/platforms/platforms' }, + items: [ + { + type: 'doc', + label: 'Alibaba Cloud', + id: 'deployment/platforms/alibaba', + }, + { + type: 'doc', + label: 'Azure', + id: 'deployment/platforms/azure', + }, + { + type: 'doc', + label: 'On-premises', + id: 'deployment/platforms/on-prem', + }, + ], + }, ], }, { type: 'category', @@ -193,7 +205,12 @@ const sidebars = { { type: 'doc', label: 'Marbles', - id: 'architecture//marbles', + id: 'architecture/marbles', + }, + { + type: 'doc', + label: 'Keys and cryptography', + id: 'architecture/security', }, ] }, From 2c6f8568a602128363009929ea05bbde4efea74c Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 27 Jul 2023 10:28:06 +0200 Subject: [PATCH 07/25] fix platforms --- docs/docs/deployment/platforms/alibaba.md | 2 ++ docs/docs/deployment/platforms/azure.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/docs/docs/deployment/platforms/alibaba.md b/docs/docs/deployment/platforms/alibaba.md index 9a3b891d..d5281a77 100644 --- a/docs/docs/deployment/platforms/alibaba.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -1,3 +1,5 @@ +# Alibaba Cloud MarbleRun deployment + ## Alibaba Cloud Container Service for Kubernetes (ACK) Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment technique relying on Alibaba's cloud resources. diff --git a/docs/docs/deployment/platforms/azure.md b/docs/docs/deployment/platforms/azure.md index 98062d85..8dc8af18 100644 --- a/docs/docs/deployment/platforms/azure.md +++ b/docs/docs/deployment/platforms/azure.md @@ -1,3 +1,5 @@ +# Azure MarbleRun deployment + ## Azure Kubernetes Services (AKS) Azure Kubernetes Service (AKS) offers a popular deployment technique relying on From 422de402dedb27ff14ae85b4290adde7b1d921eb Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Fri, 4 Aug 2023 08:15:42 +0200 Subject: [PATCH 08/25] Apply suggestions --- docs/docs/architecture/security.md | 58 +++++++++++++++++------------- 1 file changed, 34 insertions(+), 24 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 9b2aade7..de4ff1e2 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -23,40 +23,45 @@ The following graphic gives an overview of the architecture and the components. MarbleRun uses the SGX remote attestation capability to authenticate the Coordinator and the Marble enclaves. See our section on [attestation](../features/attestation.md) for more information on this process. For authorization, the [manifest](../features/manifest.md) defines the Marble's access to secrets and keys after successful attestation. -Furthermore, MarbleRun's' [RBAC](../workflows/define-manifest.md#roles) attaches a Marble's identity to a role in the manifest. -Each role is associated with a set of operations that Marble can perform in the deployment. -Roles can also be attached to [users](../workflows/define-manifest.md#users), which are authenticated by the Coordinator using an RSA or ECDSA public key defined in the manifest. +Furthermore, MarbleRun's' [RBAC](../workflows/define-manifest.md#roles) attaches [users](../workflows/define-manifest.md#users) identities to roles in the manifest. +Each role is associated with a set of operations that the user can perform in the deployment. +Users are authenticated by the Coordinator using an RSA or ECDSA public key defined in the manifest. ## Public Key Infrastructure and Certificate Authority The Coordinator establishes a public key infrastructure (PKI) for MarbleRun and acts as the Certificate Authority (CA). The goal of the PKI is to make authentication of confidential applications based on remote attestation accessible and usable. -The Coordinator embeds its attestation statement into its root CA certificate, see the [attested TLS](#attested-tls-atls) section for details behind that concept. -All MarbleRun clients and Marbles can then use the attested root CA certificate for authenticating TLS connections. +The Coordinator provides an [API](../reference/coordinator.md) for retrieving an SGX attestation statement that embeds its *Root CA Certificate* in the user-defined body. +By verifying the statement, clients can verify the certificate's authenticity and, thereby, the MarbleRun CA. +See the [attested TLS](#attested-tls-atls) section for details behind that concept. +All MarbleRun clients and Marbles can then use the attested *Root CA Certificate* for authenticating TLS connections. This is further illustrated conceptionally in the [attestation](../features/attestation.md) section. We now focus on the cryptography. During initialization the Coordinator generates a root x509 certificate and corrisponding asymmetric key pair. The [Elliptic Curve Digital Signature Algorithm (ECDSA)](https://www.secg.org/sec1-v2.pdf#page=49) is used with curve [P256](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf#page=111). -The root certificate has no expiary date set and lives as long as the MarbleRun deployment. +The *Root CA Certificate* has no expiary date set and lives as long as the MarbleRun deployment. -Alongside the root certificate, the Coordinator generates an intermediate x509 certificate and corresponding asymmetric key pair, again using ECDSA with P256. -The intermediate certificate is signed by the Coordinator's root certificate and rotated with every update of the manifest. -When you push an update to the manifest (for example bump up the *SecurityVersion* of a Mable) the intermediate certificate will change. +Alongside the *Root CA Certificate*, the Coordinator generates an x509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. +The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every update of the manifest. +When you push an update to the manifest (for example bump up the *SecurityVersion* of a Mable) the *Intermediate Certificate* will change. Instances with the new version will not authenticate with instances of the old version and vice versa. Hence, no data flow is happening between different *SecurityVersions* of your application. -However, the root certificate doesn't change. So you can still verify the Coordinator and your application from the outside and make sure it is the same instance you might have interacted with before. - -The Coordinator creates a second intermediate certificate that is called the *Marble root certificate*. -The Marble root certificate is self-signed using the intermediate certificate's private key implementing a [cross-signed certificate chain](https://www.ssltrust.com.au/blog/understanding-certificate-cross-signing). -In that way, the Marbles see a self-signed root certificate, hence, the are dealing with a terminating certificate chain without knowing about the Coordinator's root certificate. -The "outside world" sees an intermediate certificate signed by the Coordinator's root certificate and the corresponding root certificate. +However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and make sure it is the same instance you might have interacted with before. Applications interacting with the MarbleRun deployment can decide wether to use the intermediate or the root certificate as CA depending on if they want to notice and handle manifest updates explicitely or not. -For every Marble the Coordinator generates a unique leaf "Marble" certificate and corresponding key pair using ECDSA with P256. -The Marble certificate is signed by the Marble root certificate. -The Marble certificate is provisioned to the Marble's enclave via the secure channel established during the [attestation procedure](../features/attestation.md). +The Coordinator creates a second certificate, with the same key material as the *Intermediate Certificate*, that is called the *Marble Root Certificate*. +In that sense they are siblings containing the same public key. +Howerver, while the *Intermediate Certificate* is signed by the *Root Certificate*, the *Marble Root Certificate* is self-signed using its own private key. +The goal here is to implement a [cross-signed certificate chain](https://www.ssltrust.com.au/blog/understanding-certificate-cross-signing). +In that way, the Marbles see the *Marble Root Certificate* as a self-signed root certificate, hence, they are dealing with a terminating certificate chain without knowing about the Coordinator's *Root CA Certificate*. +The "outside world" sees an intermediate certificate signed by the Coordinator's *Root CA Certificate*. + +For every Marble the Coordinator generates a unique leaf *Marble Certificate* and corresponding key pair using ECDSA with P256. +The *Marble Certificate* is signed by the *Marble Root Certificate*. +The *Marble Certificate* is provisioned to the Marble's enclave via the secure channel established during the [attestation procedure](../features/attestation.md). +Depending on the Marble's runtime the certificate can either be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically](../features/transparent-TLS.md) for establishing mutually authenticated TLS connetions. -![Security architecture](../_media/cert-chain.svg) +![PKI Certificate chain](../_media/cert-chain.svg) ## Attested TLS (aTLS) @@ -65,11 +70,6 @@ In a CC environment, attested TLS (aTLS) can be used to establish secure connect aTLS modifies the TLS handshake by embedding an attestation statement into the TLS certificate. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS). -### Mutual TLS authentication - -Based on MarbleRun's PKI Marbles are provided with a unique leaf certificate and corresponding private key. -As described above these are generated by the Coordinator and provided during the Marble's initialization via the secure aTLS channel. -Depending on the Marble's runtime the certificate can either be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically](../features/transparent-TLS.md) for establishing mutually authenticated TLS connetions. ## Encryption of state @@ -82,6 +82,16 @@ The KEK MarbleRun uses the SGX sealing key called `Product key`, which is bound In other words, a fresh and benign enclave instance of the same identity can recover that key. Hence, as long as the Coordinator is restarted on the same CPU, it can obtain the same KEK from the CPU based on its identity, decrypt the DEK and recover its state. +If the Coordinator is restarted on a different CPU, it won't be able to obtain the same SGX sealing key from the CPU. +Therefore, MarbleRun provides a [recovery feature](../features/recovery.md#recovery). +The manifest allows for specifying a designated Recovery Key. The Recovery Key is a public RSA key. Upon startup, the Coordinator encrypts the DEK with this public key and returns it to the user. +In case of a recovery event, the user decrypts the DEK locally and [uploads it to the Coordinator](../workflows/recover-coordinator.md). +The Coordinator will decrypt the state with the DEK and proceed with operations. + +For [multi-party use cases](../features/recovery.md#multi-party-recovery), MarbleRun allows to split the Recovery Key between parties. +Every recovery party is defined in the manifest with its own public RSA key. +The Coordinator generates a share of the recovery secret for every party and encrypts it with the corresponding RSA key. +During a recovery event, every party will upload their share of the secret, which are all XOR'ed together by the Coordinator to receive the combined key for decrypting the DEK. ![Encrypted state single instance](../_media/enc-state-single.svg) From be36d66d18064156aac71b2672d7c74efee13b0c Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Fri, 4 Aug 2023 09:11:53 +0200 Subject: [PATCH 09/25] Update graphics --- docs/docs/_media/cert-chain.svg | 2 +- docs/docs/_media/enc-state-distributed.svg | 2 +- docs/docs/_media/enc-state-single.svg | 3 ++- docs/docs/_media/security_architecture.svg | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/docs/_media/cert-chain.svg b/docs/docs/_media/cert-chain.svg index fb4cbdc1..bdfbf0b7 100644 --- a/docs/docs/_media/cert-chain.svg +++ b/docs/docs/_media/cert-chain.svg @@ -1,3 +1,3 @@ -
Leaf Certificates
Leaf Certificates
Intermediate CA
Intermediate CA
Root CA
Root CA
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certi...
Marble Root Certificate
Marble Root Certif...
Marble Certificate
Marble Certificate
signs
signs
Root Private Key
Root...
signs
signs
signs
signs
Intermediate Private Key
Inte...
Marble Private Key
Marb...Text is not SVG - cannot display \ No newline at end of file +
Leaf Certificates
Leaf Certificates
Intermediate CA
Intermediate CA
Root CA
Root CA
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Cer...
Marble Root Certificate
Marble Root Cert...
Marble Certificate
Marble Certifica...
signs
signs
Root Private Key
Root...
signs
signs
signs
signs
Intermediate Private Key
Int...
Marble Private Key
Mar...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/_media/enc-state-distributed.svg b/docs/docs/_media/enc-state-distributed.svg index 390dd8aa..646659e4 100644 --- a/docs/docs/_media/enc-state-distributed.svg +++ b/docs/docs/_media/enc-state-distributed.svg @@ -1,3 +1,3 @@ -
marble-state Secret
marble-state Secret
Encrypted state
Encrypted state
Encrypted DEK
Encrypted DEK
Coordinator state
Coordinator state
Data Encryption Key
Data Encryption Key
Key Encryption Key
Key Encryption Key
randomly generated by the first Coordinator after receiving the manifest
randomly generated by t...
randomly generated at start up
randomly generated at s...
KEK ConfigMap
KEK ConfigMap
Node-1: enc(KEK)
Node-1: enc(KEK)
Node-2: enc(KEK)
Node-2: enc(KEK)
Node-X: enc(KEK)
Node-X: enc(KEK)
Node-1 SGX 'Product' Sealing Key
Node-1 SGX 'Product' Sealing Key
Node-2 SGX 'Product' Sealing Key
Node-2 SGX 'Product' Sealing Key
Node-X SGX 'Product' Sealing Key
Node-X SGX 'Product' Sealing Key
...
...
...
...Text is not SVG - cannot display \ No newline at end of file +
Kubernetes Secret
Kubernetes Secret
Encrypted State
Encrypted State
Encrypted DEK
Encrypted DEK
Coordinator State
Coordinator State
Data Encryption Key
Data Encryption K...
Key Encryption Key
Key Encryption Key
Generated by the first Coordinator
Generated by the fir...
Generated by the first Coordinator
Generated by the fir...
Kubernetes ConfigMap
Kubernetes ConfigMap
Node-1: enc(KEK)
Node-1: enc(KEK)
Node-2: enc(KEK)
Node-2: enc(KEK)
Node-X: enc(KEK)
Node-X: enc(KEK)
Node-1 SGX 'Product' Sealing Key
Node-1 SGX 'Product' Sealing Key
Node-2 SGX 'Product' Sealing Key
Node-2 SGX 'Product' Sealing Key
Node-X SGX 'Product' Sealing Key
Node-X SGX 'Product' Sealing Key
...
...
...
...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/_media/enc-state-single.svg b/docs/docs/_media/enc-state-single.svg index 8f8acda3..6dbf8883 100644 --- a/docs/docs/_media/enc-state-single.svg +++ b/docs/docs/_media/enc-state-single.svg @@ -1,3 +1,4 @@ + -
Persistent Storage
Persistent Storage
Encrypted state
Encrypted state
Encrypted DEK
Encrypted DEK
Coordinator state
Coordinator state
Data Encryption Key
Data Encryption Key
Key Encryption Key
Key Encryption Key
randomly generated by the Coordinator after receiving the manifest
randomly generated by t...
SGX 'Product' Sealing Key
SGX 'Product' Sealing K...Text is not SVG - cannot display \ No newline at end of file +
Persistent Storage
Persistent Storage
Encrypted State
Encrypted State
Encrypted DEK
Encrypted DEK
Coordinator State
Coordinator State
Data Encryption Key
Data Encryption K...
Key Encryption Key
Key Encryption Key
Generated by the Coordinator
Generated by the Coordi...
SGX 'Product' Sealing Key
SGX 'Product' Sealing K...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/docs/_media/security_architecture.svg b/docs/docs/_media/security_architecture.svg index b00cfa3d..9221152e 100644 --- a/docs/docs/_media/security_architecture.svg +++ b/docs/docs/_media/security_architecture.svg @@ -1,3 +1,3 @@ -
MarbleRun Deployment
MarbleRun Deployment
aTLS
gRPC
aTLS...
Coordinator
Coordinator
Manifest
Manifest
Packages
Packages
Marbles
Marbles
Secrets
Secrets
Users
Users
Certificate Authority
Certificate Authority
mTLS
HTTP, gRPC, TCP
mTLS...
Marble A
Marble A
TLS
HTTP, gRPC, TCP
TLS...
aTLS
gRPC
aTLS...
Marble B
Marble B
Marble
Certificate A
Marble...
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certifi...
Marble
Certificate B
Marble...
aTLS
HTTP
aTLS...
Administrator
Administrat...
aTLS
HTTP
aTLS...
Application user
Application...
Parameters A
Parameters A
 Secrets A
 Secrets A
Parameters B
Parameters B
 Secrets B
 Secrets B
Marble
Private Key A
Marble...
Marble
Private Key B
Marble...Text is not SVG - cannot display \ No newline at end of file +
MarbleRun Deployment
MarbleRun Deployment
aTLS
gRPC
aTLS...
Coordinator
Coordinator
Manifest
Manifest
Packages
Packages
Marbles
Marbles
Secrets
Secrets
Recovery Keys
Recovery Keys
Certificate Authority
Certificate Authority
mTLS
HTTP, gRPC, TCP
mTLS...
Marble A
Marble A
TLS
HTTP, gRPC, TCP
TLS...
aTLS
gRPC
aTLS...
Marble B
Marble B
Marble
Certificate A
Marble...
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certifi...
Marble
Certificate B
Marble...
aTLS
HTTP
aTLS...
Administrator
Administrat...
aTLS
HTTP
aTLS...
Application user
Application...
Parameters A
Parameters A
 Secrets A
 Secrets A
Parameters B
Parameters B
 Secrets B
 Secrets B
Private Keys
Private Keys
Marble Root Certificate
Marble Root Certific...
Marble Root Certificate
Marble Root Certific...
Marble Root Certificate
Marble Root Certific...Text is not SVG - cannot display \ No newline at end of file From 2fcdd24d22bfe602875ea25ec43d91b3fdae43ec Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Mon, 7 Aug 2023 07:10:57 +0200 Subject: [PATCH 10/25] Apply suggestions from code review Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com> --- docs/docs/architecture/coordinator.md | 4 ++-- docs/docs/architecture/marbles.md | 2 +- docs/docs/deployment/platforms/alibaba.md | 8 +++----- docs/docs/deployment/platforms/azure.md | 6 +----- docs/docs/deployment/platforms/platforms.md | 6 +++--- docs/docs/getting-started/installation.md | 7 ------- docs/docs/intro.md | 6 +++--- docs/sidebars.js | 10 +++++----- 8 files changed, 18 insertions(+), 31 deletions(-) diff --git a/docs/docs/architecture/coordinator.md b/docs/docs/architecture/coordinator.md index 7840411e..39be9edb 100644 --- a/docs/docs/architecture/coordinator.md +++ b/docs/docs/architecture/coordinator.md @@ -1,8 +1,8 @@ # Coordinator The Coordinator is MarbleRun's control plane. -Conceptionally it operators as trusted controller in your confidential deployment. -Therefore, it securely holds the deployment's state in form of the [manifest](../features/manifest.md) and the defined [secrets](../features/secrets-management.md). +Conceptually, it operates as trusted controller in your confidential deployment. +Therefore, it securely holds the deployment's state, including the [manifest](../features/manifest.md) and the defined [secrets](../features/secrets-management.md). Based on the manifest, the Coordinator uses [remote attestation](../features/attestation.md) to authenticate the application's enclaves. diff --git a/docs/docs/architecture/marbles.md b/docs/docs/architecture/marbles.md index 5db3e264..abd4631a 100644 --- a/docs/docs/architecture/marbles.md +++ b/docs/docs/architecture/marbles.md @@ -2,5 +2,5 @@ Marbles are MarbleRun's data planes and run your application code in secure enclaves within otherwise regular Docker containers. Marbles communicate with the Coordinator via gRPC over TLS. -MarbleRun natively [supports multiple enclave runtimes](../features/runtimes.md) as the data plane. +MarbleRun [supports multiple enclave runtimes](../features/runtimes.md) as the data plane. See the [Add a Service](../workflows/add-service.md) section on how to build and configure an enclave as Marble. diff --git a/docs/docs/deployment/platforms/alibaba.md b/docs/docs/deployment/platforms/alibaba.md index d5281a77..2588069d 100644 --- a/docs/docs/deployment/platforms/alibaba.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -3,12 +3,10 @@ ## Alibaba Cloud Container Service for Kubernetes (ACK) Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment technique relying on Alibaba's cloud resources. -[ACK hosts Kubernetes pods in SGX-capabale Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing?spm=a2c63.p38356.0.0.31b6925dUJDMZC#task-2490003) and exposes the underlying SGX hardware. +[ACK hosts Kubernetes pods in SGX-capabale Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing) and exposes the underlying SGX hardware. ### Prerequisites * Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX enabled worker nodes. -* [Update and install the Intel DCAP Quote Provider Library](./alibaba.md#prerequisites-1) - ### Deploy MarbleRun @@ -17,8 +15,8 @@ See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in your ## Alibaba Cloud Elastic Compute Service -With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/doc-detail/207734.htm) users can use Intel SGX on Alibaba Cloud. -You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances#task-2038128) instance. +With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25) users can use Intel SGX on Alibaba Cloud. +You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances) instance. The description below uses a VM running Ubuntu 18.04. diff --git a/docs/docs/deployment/platforms/azure.md b/docs/docs/deployment/platforms/azure.md index 8dc8af18..dd8984ae 100644 --- a/docs/docs/deployment/platforms/azure.md +++ b/docs/docs/deployment/platforms/azure.md @@ -8,8 +8,6 @@ Azure's cloud resources. AKS hosts Kubernetes pods in SGX-capabale Azure VMs and ### Prerequisites * Follow the instructions on the [AKS Confidential Computing Quick Start guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-enclave-nodes-aks-get-started) to provision an AKS cluster with Intel SGX enabled worker nodes. -* [Update and install the Azure DCAP client](https://learn.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#install-azure-dcap-client) - ### Deploy MarbleRun @@ -18,9 +16,7 @@ See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in your ## Azure confidential computing VMs [Azure confidential computing services](https://learn.microsoft.com/en-us/azure/confidential-computing/virtual-machine-solutions-sgx) provide access to VMs with Intel SGX enabled. -You can follow their [quickstart](https://learn.microsoft.com/en-us/azure/confidential-computing/quick-create-portal) to create a [DCsv2](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv2-series) (Coffee Lake) or [DCsv3](https://learn.microsoft.com/en-us/azure/virtual-machines/dcv3-series) (Ice Lake) VMs with Intel SGX enabled - -The description below uses a VM running Ubuntu 18.04. +You can follow their [quickstart](https://learn.microsoft.com/en-us/azure/confidential-computing/quick-create-portal) to create a [DCsv2](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv2-series) (Coffee Lake) or [DCsv3](https://learn.microsoft.com/en-us/azure/virtual-machines/dcv3-series) (Ice Lake) VM. ### Prerequisites diff --git a/docs/docs/deployment/platforms/platforms.md b/docs/docs/deployment/platforms/platforms.md index a2027d5e..4e74677d 100644 --- a/docs/docs/deployment/platforms/platforms.md +++ b/docs/docs/deployment/platforms/platforms.md @@ -1,6 +1,6 @@ -# Supported Platforms +# Supported platforms -MarbleRun supports all platforms where Intel SGX and the [Data Center Attestation Primitives (DCAP)](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf) is available. +MarbleRun supports all platforms where Intel SGX and the [Data Center Attestation Primitives (DCAP)](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/DCAP_ECDSA_Orientation.pdf) are available. The following gives you an updated list of supported platforms: @@ -13,4 +13,4 @@ The following gives you an updated list of supported platforms: | | ACK | Managed | | Equinix | Bare-Metal | Customer-operated | | IBM Cloud | Bare-Metal | Customer-operated | -| On-Premises | * | Customer-operated | +| On-premises | * | Customer-operated | diff --git a/docs/docs/getting-started/installation.md b/docs/docs/getting-started/installation.md index 9b767f97..5368bac3 100644 --- a/docs/docs/getting-started/installation.md +++ b/docs/docs/getting-started/installation.md @@ -5,13 +5,6 @@ In this guide, you will install MarbleRun into your Kubernetes cluster and deplo Installing MarbleRun is easy. First, you will install the CLI (command-line interface) onto your local machine. Using this CLI, you’ll then install the control plane onto your Kubernetes cluster. Finally, you will add your own services and set up a corresponding manifest. -:::tip - -A working SGX DCAP environment is required for MarbleRun to work. For the ease of exploring and testing we provide a simulation mode with `--simulation` that runs without SGX hardware. -Depending on your setup you may follow the [first steps for SGX-enabled clusters](../getting-started/quickstart.md). Alternatively, if your setup doesn't support SGX, you can follow the [first steps in simulation mode](../getting-started/quickstart.md). - -::: - ## Prerequisites Make sure the following requirements are met: diff --git a/docs/docs/intro.md b/docs/docs/intro.md index c170c431..908aa36a 100644 --- a/docs/docs/intro.md +++ b/docs/docs/intro.md @@ -7,7 +7,7 @@ slug: / MarbleRun is a framework for deploying distributed confidential computing applications. MarbleRun acts as a confidential operator for your deployment. Think of a trusted party in the control plane. -Build your confidential microservices with tools like [EGo, Gramine, or similar runtime](./features/runtimes.md), orchestrate them with Kubernetes on an SGX-enabled cluster, and let MarbleRun take care of the rest. +Build your confidential microservices with tools like [EGo, Gramine, or similar runtimes](./features/runtimes.md), orchestrate them with Kubernetes on an SGX-enabled cluster, and let MarbleRun take care of the rest. Deploy end-to-end secure and verifiable AI pipelines or crunch on sensitive big data in the cloud. Confidential computing at scale has never been easier. MarbleRun simplifies the process by handling much of the groundwork. @@ -15,9 +15,9 @@ It ensures that your app's topology adheres to your specified manifest. It verifies the identity and integrity of all your services, bootstraps them, and establishes secure, encrypted communication channels. As your app needs to scale, MarbleRun manages the addition of new instances, ensuring their secure verification. -MarbleRun provides comprehensive access and privilege management through its role-based access control (RBAC) feature. This can be used to enforce security policies such as 4-eye principles and other verification procedures in your operations. +MarbleRun provides comprehensive access and privilege management through its role-based access control (RBAC) feature. You can use this to enforce security policies such as four-eyes principles and other verification procedures in your operations. -For added simplicity, MarbleRun offers a single remote attestation statement for your entire app. Anyone can use this to verify the integrity of your distributed app, making it easier to assure stakeholders of your app's security. +To keep things simple, MarbleRun offers a single remote attestation statement for your entire app. Anyone can use this to verify the integrity of your distributed app, making it easier to assure stakeholders of your app's security. ## Key features diff --git a/docs/sidebars.js b/docs/sidebars.js index 3f0f41c7..29cbc2cd 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -115,13 +115,13 @@ const sidebars = { items: [ { type: 'doc', - label: 'Alibaba Cloud', - id: 'deployment/platforms/alibaba', + label: 'Azure', + id: 'deployment/platforms/azure', }, { type: 'doc', - label: 'Azure', - id: 'deployment/platforms/azure', + label: 'Alibaba Cloud', + id: 'deployment/platforms/alibaba', }, { type: 'doc', @@ -216,7 +216,7 @@ const sidebars = { }, { type: 'category', - label: 'Building Enclaves', + label: 'Building Marbles', link: { type: 'generated-index', }, From abed7f2517dea0002f038fce546004a43761ed95 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Mon, 7 Aug 2023 08:04:52 +0200 Subject: [PATCH 11/25] fix linter --- docs/docs/architecture/security.md | 50 +++++++++++------------ docs/docs/deployment/platforms/alibaba.md | 10 ++--- docs/docs/features/recovery.md | 15 ++++--- docs/docs/features/runtimes.md | 6 +-- 4 files changed, 39 insertions(+), 42 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index de4ff1e2..07517071 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -21,10 +21,10 @@ The following graphic gives an overview of the architecture and the components. ## Authentication and authorization -MarbleRun uses the SGX remote attestation capability to authenticate the Coordinator and the Marble enclaves. See our section on [attestation](../features/attestation.md) for more information on this process. +MarbleRun uses the SGX remote attestation capability to authenticate the Coordinator and the Marble enclaves. See the section on [attestation](../features/attestation.md) for more information on this process. For authorization, the [manifest](../features/manifest.md) defines the Marble's access to secrets and keys after successful attestation. -Furthermore, MarbleRun's' [RBAC](../workflows/define-manifest.md#roles) attaches [users](../workflows/define-manifest.md#users) identities to roles in the manifest. -Each role is associated with a set of operations that the user can perform in the deployment. +Furthermore, MarbleRun's' [RBAC](../workflows/define-manifest.md#roles) attaches [users'](../workflows/define-manifest.md#users) identities to roles in the manifest. +Each role is associated with a set of operations the user can perform in the deployment. Users are authenticated by the Coordinator using an RSA or ECDSA public key defined in the manifest. ## Public Key Infrastructure and Certificate Authority @@ -35,38 +35,36 @@ The Coordinator provides an [API](../reference/coordinator.md) for retrieving an By verifying the statement, clients can verify the certificate's authenticity and, thereby, the MarbleRun CA. See the [attested TLS](#attested-tls-atls) section for details behind that concept. All MarbleRun clients and Marbles can then use the attested *Root CA Certificate* for authenticating TLS connections. -This is further illustrated conceptionally in the [attestation](../features/attestation.md) section. We now focus on the cryptography. - -During initialization the Coordinator generates a root x509 certificate and corrisponding asymmetric key pair. +This is further illustrated conceptionally in the [attestation](../features/attestation.md) section. The following focuses on the cryptographic primitives. +The Coordinator generates a root x509 certificate and corresponding asymmetric key pair during initialization. The [Elliptic Curve Digital Signature Algorithm (ECDSA)](https://www.secg.org/sec1-v2.pdf#page=49) is used with curve [P256](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf#page=111). -The *Root CA Certificate* has no expiary date set and lives as long as the MarbleRun deployment. +The *Root CA Certificate* has no expiry date and lives as long as the MarbleRun deployment. Alongside the *Root CA Certificate*, the Coordinator generates an x509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. -The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every update of the manifest. -When you push an update to the manifest (for example bump up the *SecurityVersion* of a Mable) the *Intermediate Certificate* will change. -Instances with the new version will not authenticate with instances of the old version and vice versa. +The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every manifest update. +When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Mable), the *Intermediate_ Certificate* will change. +Instances with the new version won't authenticate with instances of the old version and vice versa. Hence, no data flow is happening between different *SecurityVersions* of your application. -However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and make sure it is the same instance you might have interacted with before. -Applications interacting with the MarbleRun deployment can decide wether to use the intermediate or the root certificate as CA depending on if they want to notice and handle manifest updates explicitely or not. +However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and ensure it's the same instance you might have interacted with. +Applications interacting with the MarbleRun deployment can decide whether to use the intermediate or the root certificate as CA, depending on whether they want to notice and explicitly handle manifest updates. -The Coordinator creates a second certificate, with the same key material as the *Intermediate Certificate*, that is called the *Marble Root Certificate*. -In that sense they are siblings containing the same public key. -Howerver, while the *Intermediate Certificate* is signed by the *Root Certificate*, the *Marble Root Certificate* is self-signed using its own private key. +The Coordinator creates a second certificate with the same key material as the *Intermediate Certificate* called the *Marble Root Certificate*. +In that sense, they're siblings containing the same public key. +However, while the *Intermediate Certificate* is signed by the *Root Certificate*, the *Marble Root Certificate* is self-signed using its private key. The goal here is to implement a [cross-signed certificate chain](https://www.ssltrust.com.au/blog/understanding-certificate-cross-signing). -In that way, the Marbles see the *Marble Root Certificate* as a self-signed root certificate, hence, they are dealing with a terminating certificate chain without knowing about the Coordinator's *Root CA Certificate*. +In that way, the Marbles see the *Marble Root Certificate* as a self-signed root certificate. Hence, they're dealing with a terminating certificate chain without knowing about the Coordinator's *Root CA Certificate*. The "outside world" sees an intermediate certificate signed by the Coordinator's *Root CA Certificate*. - -For every Marble the Coordinator generates a unique leaf *Marble Certificate* and corresponding key pair using ECDSA with P256. -The *Marble Certificate* is signed by the *Marble Root Certificate*. +The Coordinator generates a unique leaf *Marble Certificate* and corresponding key pair using ECDSA with P256 for every Marble. +The *Marble Root Certificate* signs the *Marble Certificate*. The *Marble Certificate* is provisioned to the Marble's enclave via the secure channel established during the [attestation procedure](../features/attestation.md). -Depending on the Marble's runtime the certificate can either be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically](../features/transparent-TLS.md) for establishing mutually authenticated TLS connetions. +Depending on the Marble's runtime, the certificate can be used [manually](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials) or [automatically](../features/transparent-TLS.md) to establish mutually authenticated TLS connections. ![PKI Certificate chain](../_media/cert-chain.svg) ## Attested TLS (aTLS) -In a CC environment, attested TLS (aTLS) can be used to establish secure connections between two parties using the remote attestation features of the CC components. +In a CC environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components. aTLS modifies the TLS handshake by embedding an attestation statement into the TLS certificate. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS). @@ -74,9 +72,9 @@ The protocol can be used by clients to verify a server certificate, by a server ## Encryption of state The Coordinator holds the MarbleRuns state, which consists of the [manifest](../features/manifest.md), the [managed secrets](../features/secrets-management.md), and the [certificates for its CA](../features/attestation.md). -The state is stored encrypted in persistent storage. Therefore Constellation uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a randomly generated 16-byte Data Encryption Key (KEK) to seal the state to persistent storage. +The state is stored encrypted in persistent storage. Therefore Constellation uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a generated 16-byte Data Encryption Key (KEK) to seal the state to persistent storage. The DEK is also sealed to persistent storage to recover the state in case of a restart autonomously. -[SGX's sealing feature](https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-intel-sgx-sealing.html) is used for that purpose. +[SGX sealing](https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-intel-sgx-sealing.html) is used for that purpose. The Coordinator encrypts the DEK with a key encryption key (KEK). The KEK MarbleRun uses the SGX sealing key called `Product key`, which is bound to its `Product ID` and the enclave's author `MRSIGNER` identity. In other words, a fresh and benign enclave instance of the same identity can recover that key. @@ -88,10 +86,10 @@ The manifest allows for specifying a designated Recovery Key. The Recovery Key i In case of a recovery event, the user decrypts the DEK locally and [uploads it to the Coordinator](../workflows/recover-coordinator.md). The Coordinator will decrypt the state with the DEK and proceed with operations. -For [multi-party use cases](../features/recovery.md#multi-party-recovery), MarbleRun allows to split the Recovery Key between parties. +For [multi-party use cases](../features/recovery.md#multi-party-recovery), MarbleRun allows splitting the Recovery Key between parties. Every recovery party is defined in the manifest with its own public RSA key. The Coordinator generates a share of the recovery secret for every party and encrypts it with the corresponding RSA key. -During a recovery event, every party will upload their share of the secret, which are all XOR'ed together by the Coordinator to receive the combined key for decrypting the DEK. +During a recovery event, every party will upload their share of the secret, which are all XORed together by the Coordinator to receive the combined key for decrypting the DEK. ![Encrypted state single instance](../_media/enc-state-single.svg) @@ -99,7 +97,7 @@ During a recovery event, every party will upload their share of the secret, whic ### Distributed Coordinator The [distributed Coordinator](../features/recovery.md#distributed-coordinator) works similarly. However, all Coordinators share the same state stored encrypted in the Kubernetes [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) called *marble-state*. -In contrast to the single instance, the KEK is randomly generated at start-up by the first instance. +In contrast to the single instance, the KEK is generated at start-up by the first instance. The existing Coordinators authenticate every new Coordinator instance via remote attestation, and the KEK is subsequently shared via the secure and attested TLS connection. Every Coordinator instance uses its own SGX Product (Sealing) Key to seal the KEK into a Kubernetes [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) structure called "KEK." diff --git a/docs/docs/deployment/platforms/alibaba.md b/docs/docs/deployment/platforms/alibaba.md index 2588069d..928bf1a5 100644 --- a/docs/docs/deployment/platforms/alibaba.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -3,19 +3,19 @@ ## Alibaba Cloud Container Service for Kubernetes (ACK) Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment technique relying on Alibaba's cloud resources. -[ACK hosts Kubernetes pods in SGX-capabale Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing) and exposes the underlying SGX hardware. +[ACK hosts Kubernetes pods in SGX-capable Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing) and exposes the underlying SGX hardware. ### Prerequisites * Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX enabled worker nodes. ### Deploy MarbleRun -See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in your ACK cluster. +See our [Kubernetes guide](../kubernetes.md) on installing MarbleRun in your ACK cluster. ## Alibaba Cloud Elastic Compute Service -With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25) users can use Intel SGX on Alibaba Cloud. +With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud. You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances) instance. The description below uses a VM running Ubuntu 18.04. @@ -38,7 +38,7 @@ The description below uses a VM running Ubuntu 18.04. 1. Set configuration for Alibaba Cloud SGX remote attestation service - Alibaba Cloud provides a PCCS for remote attestation, deployed on a per-region basis. For optimal stability it's recommended to access the service in your instance's region. + Alibaba Cloud provides a PCCS for remote attestation, deployed per-region. For optimal stability, accessing the service in your instance's region is recommended. The configuration is set in `/etc/sgx_default_qcnl.conf`. * If your instance is assigned a public IP address, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region: @@ -64,4 +64,4 @@ The description below uses a VM running Ubuntu 18.04. ### Deploy MarbleRun You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../standalone.md). -Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. +Alternatively, you can install a Kubernetes cluster. Probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/). See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. diff --git a/docs/docs/features/recovery.md b/docs/docs/features/recovery.md index b718af6e..6f752019 100644 --- a/docs/docs/features/recovery.md +++ b/docs/docs/features/recovery.md @@ -1,14 +1,14 @@ # State and recovery -Persistent storage for confidential applications in the cloud requires a bit of attention. +Persistent storage for confidential applications in the cloud requires attention. If an application should be able to restart without manual intervention, it needs a way to automatically and securely obtain a secret to decrypt its state. The SGX programming model considers a single, local application running on a specific CPU. -In this case, the application can use the SGX seal key as its root secret. +The application can use the SGX seal key as its root secret. However, this binds the application and its state to the physical machine because seal keys are unique to a single CPU. -In sum, this means that the usual SGX programming model isn't suited for virtual environments or distributed applications. +The usual SGX programming model isn't suited for virtual environments or distributed applications. -With MarbleRun, the Coordinator [manages the Marbles' secrets](../features/secrets-management.md) and Marbles obtain them securely on start. +With MarbleRun, the Coordinator [manages the Marbles[' secrets](../features/secrets-management.md), and Marbles obtain them securely on start. Thus, Marbles can be distributed and rescheduled on arbitrary machines. This narrows the challenge of persistent storage down to the Coordinator itself. @@ -17,7 +17,7 @@ This narrows the challenge of persistent storage down to the Coordinator itself. The straightforward way to run MarbleRun is with a single Coordinator. In this case, the state is encrypted with the SGX seal key and stored on disk. When pinned to a single host, the Coordinator can unseal its state automatically. -However, when the Coordinator is moved to another physical host, a [manual step](#recovery) is required to recover the Coordinator's state. +However, a [manual step](#recovery) is required to recover the Coordinator's state when the Coordinator is moved to another physical host. ### Distributed Coordinator @@ -32,8 +32,7 @@ In this mode of operation, manual recovery is only required when * all Coordinator instances are stopped at the same time, and * all new instances are scheduled on new physical hosts. - -In other words, if at least one instance is scheduled on a host where a previous instance was running, the state can be recovered automatically. +In other words, if at least one instance is scheduled on a host running a previous instance, the state can be recovered automatically. ## Recovery @@ -41,7 +40,7 @@ The manifest allows for specifying a designated *Recovery Key*. The Recovery Key :::caution -The owner of the Recovery Key can access the raw state of the Coordinator. +The Recovery Key's owner can access the Coordinator's raw state. ::: diff --git a/docs/docs/features/runtimes.md b/docs/docs/features/runtimes.md index fe0eeec4..d49dfcf1 100644 --- a/docs/docs/features/runtimes.md +++ b/docs/docs/features/runtimes.md @@ -6,7 +6,7 @@ MarbleRun strives to be runtime-agnostic. Currently, supported runtimes are desc [EGo](https://github.com/edgelesssys/ego) is the preferred way for writing confidential Go applications from scratch as well as porting existing ones. Usage is similar to conventional Go programming. Start [building your service with EGo](../building-enclaves/ego.md) to use it with MarbleRun. ## Edgeless RT -With [Edgeless RT](https://github.com/edgelesssys/edgelessrt) you can create confidential C++ applications with a low TCB. Please follow the build instructions provided [in our C++ sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) to use it with MarbleRun. +With [Edgeless RT](https://github.com/edgelesssys/edgelessrt), you can create confidential C++ applications with a low TCB. Please follow the build instructions provided [in our C++ sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) to use it with MarbleRun. ## Gramine [Gramine](https://gramineproject.io/) is a popular choice for wrapping unmodified applications into enclaves. @@ -15,6 +15,6 @@ Gramine further adds support for dynamically linked libraries and multi-process [Running a Gramine app](../building-enclaves/gramine.md) with MarbleRun requires minor changes to its manifest. ## Occlum -[Occlum](https://github.com/occlum/occlum) is another popular solution which allows wrapping existing applications with minimal to no changes inside an enclave. It requires you to at best recompile existing applications with the provided toolchains with support for common languages such as C, C++, Go, Java, and Rust. -Its core is written in the memory-safe programming language Rust and it provides a separated environment under which your application is running. This provides a safe yet powerful way to build your applications. +[Occlum](https://github.com/occlum/occlum) is another popular solution that allows wrapping existing applications with minimal to no changes inside an enclave. It requires you to, at best, recompile existing applications with the provided toolchains with support for common languages such as C, C++, Go, Java, and Rust. +Its core is written in the memory-safe programming language Rust, and it provides a separated environment under which your application is running. This provides a safe yet powerful way to build your applications. [Running an Occlum app](../building-enclaves/occlum.md) with MarbleRun requires minor changes to its manifest. From 19b3e00d6feb8c9b874f77d4a8cbc5a6b1e67efa Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Mon, 7 Aug 2023 08:21:45 +0200 Subject: [PATCH 12/25] update vocab --- docs/docs/architecture/security.md | 2 +- docs/styles/Vocab/edgeless/accept.txt | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 07517071..3ea1766d 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -42,7 +42,7 @@ The *Root CA Certificate* has no expiry date and lives as long as the MarbleRun Alongside the *Root CA Certificate*, the Coordinator generates an x509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every manifest update. -When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Mable), the *Intermediate_ Certificate* will change. +When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Mable), the *Intermediate Certificate* will change. Instances with the new version won't authenticate with instances of the old version and vice versa. Hence, no data flow is happening between different *SecurityVersions* of your application. However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and ensure it's the same instance you might have interacted with. diff --git a/docs/styles/Vocab/edgeless/accept.txt b/docs/styles/Vocab/edgeless/accept.txt index 301d157b..522831d2 100644 --- a/docs/styles/Vocab/edgeless/accept.txt +++ b/docs/styles/Vocab/edgeless/accept.txt @@ -9,6 +9,7 @@ config Dockerfile emojivoto Env +Equinix [Gg]ramine hardcode helloworld @@ -32,5 +33,6 @@ Tensorflow toleration toolchain updatable +VM walkthrough Xeon From ccc3aa9cd481fca96322b05cb63a792a740b59c7 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Mon, 7 Aug 2023 08:25:21 +0200 Subject: [PATCH 13/25] update vocab --- docs/styles/Vocab/edgeless/accept.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/styles/Vocab/edgeless/accept.txt b/docs/styles/Vocab/edgeless/accept.txt index 522831d2..f00df204 100644 --- a/docs/styles/Vocab/edgeless/accept.txt +++ b/docs/styles/Vocab/edgeless/accept.txt @@ -5,6 +5,7 @@ Alibaba auto-injection [Cc]loud collateral +conceptionally config Dockerfile emojivoto @@ -36,3 +37,4 @@ updatable VM walkthrough Xeon +XORed From 0d6852144bcd2cd07f7a107d6124c14347d3891d Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Mon, 7 Aug 2023 09:00:33 +0200 Subject: [PATCH 14/25] Update cert chain graphic --- docs/docs/_media/cert-chain.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/_media/cert-chain.svg b/docs/docs/_media/cert-chain.svg index bdfbf0b7..3f9325d5 100644 --- a/docs/docs/_media/cert-chain.svg +++ b/docs/docs/_media/cert-chain.svg @@ -1,3 +1,3 @@ -
Leaf Certificates
Leaf Certificates
Intermediate CA
Intermediate CA
Root CA
Root CA
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Cer...
Marble Root Certificate
Marble Root Cert...
Marble Certificate
Marble Certifica...
signs
signs
Root Private Key
Root...
signs
signs
signs
signs
Intermediate Private Key
Int...
Marble Private Key
Mar...Text is not SVG - cannot display \ No newline at end of file +
Leaf Certificates
Leaf Certificates
Intermediate CA
Intermediate CA
Root CA
Root CA
signs
signs
Root Private Key
Root...
signs
signs
signs
signs
Intermediate Private Key
Int...
Marble Private Key
Mar...
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Cer...
Marble Root Certificate
Marble Root Cert...
Marble Certificate
Marble Certifica...Text is not SVG - cannot display \ No newline at end of file From ced1bfafff71377417d428298a32ab324f89dc31 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 9 Aug 2023 07:51:13 +0200 Subject: [PATCH 15/25] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com> Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> --- docs/docs/architecture/security.md | 30 ++++++++++++++----------- docs/docs/features/recovery.md | 2 +- docs/docs/getting-started/quickstart.md | 2 +- docs/styles/Vocab/edgeless/accept.txt | 1 - 4 files changed, 19 insertions(+), 16 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 3ea1766d..0fc12a8a 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -11,7 +11,7 @@ The [Coordinator](coordinator.md) and the [Marbles](marbles.md) run inside SGX E MarbleRun uses cryptography for the following tasks. * Authentication and authorization -* Establishing a Public Key Infrastructure (PKI) for MarbleRun +* Establishing a public key infrastructure (PKI) for MarbleRun * Encrypting network traffic via mutual TLS between enclaves * Encrypting persistent state @@ -21,32 +21,36 @@ The following graphic gives an overview of the architecture and the components. ## Authentication and authorization -MarbleRun uses the SGX remote attestation capability to authenticate the Coordinator and the Marble enclaves. See the section on [attestation](../features/attestation.md) for more information on this process. +MarbleRun uses the SGX [remote attestation](../features/attestation.md) capability to authenticate the Coordinator and the Marble enclaves. For authorization, the [manifest](../features/manifest.md) defines the Marble's access to secrets and keys after successful attestation. -Furthermore, MarbleRun's' [RBAC](../workflows/define-manifest.md#roles) attaches [users'](../workflows/define-manifest.md#users) identities to roles in the manifest. +Furthermore, MarbleRun's [RBAC](../workflows/define-manifest.md#roles) attaches [users'](../workflows/define-manifest.md#users) identities to roles in the manifest. Each role is associated with a set of operations the user can perform in the deployment. Users are authenticated by the Coordinator using an RSA or ECDSA public key defined in the manifest. -## Public Key Infrastructure and Certificate Authority +## Public key infrastructure and certificate authority -The Coordinator establishes a public key infrastructure (PKI) for MarbleRun and acts as the Certificate Authority (CA). +The Coordinator establishes a public key infrastructure (PKI) for MarbleRun and acts as the certificate authority (CA). The goal of the PKI is to make authentication of confidential applications based on remote attestation accessible and usable. The Coordinator provides an [API](../reference/coordinator.md) for retrieving an SGX attestation statement that embeds its *Root CA Certificate* in the user-defined body. By verifying the statement, clients can verify the certificate's authenticity and, thereby, the MarbleRun CA. See the [attested TLS](#attested-tls-atls) section for details behind that concept. All MarbleRun clients and Marbles can then use the attested *Root CA Certificate* for authenticating TLS connections. -This is further illustrated conceptionally in the [attestation](../features/attestation.md) section. The following focuses on the cryptographic primitives. -The Coordinator generates a root x509 certificate and corresponding asymmetric key pair during initialization. +This is further illustrated conceptually in the [attestation](../features/attestation.md) section. The following focuses on the cryptographic primitives. +The Coordinator generates a root X.509 certificate and corresponding asymmetric key pair during initialization. The [Elliptic Curve Digital Signature Algorithm (ECDSA)](https://www.secg.org/sec1-v2.pdf#page=49) is used with curve [P256](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf#page=111). The *Root CA Certificate* has no expiry date and lives as long as the MarbleRun deployment. -Alongside the *Root CA Certificate*, the Coordinator generates an x509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. +Alongside the *Root CA Certificate*, the Coordinator generates an X.509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every manifest update. When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Mable), the *Intermediate Certificate* will change. Instances with the new version won't authenticate with instances of the old version and vice versa. Hence, no data flow is happening between different *SecurityVersions* of your application. However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and ensure it's the same instance you might have interacted with. -Applications interacting with the MarbleRun deployment can decide whether to use the intermediate or the root certificate as CA, depending on whether they want to notice and explicitly handle manifest updates. +Applications interacting with the MarbleRun deployment should use the intermediate certificate as CA to make manifest updates observable: +if the manifest changes, connections to the deployment will fail. +The application owner can then review the changes and install the new intermediate certificate to approve them. +If such observability isn't required, the application can use the root certificate as CA. +In that case, the application will continue to work even if the manifest changes. The Coordinator creates a second certificate with the same key material as the *Intermediate Certificate* called the *Marble Root Certificate*. In that sense, they're siblings containing the same public key. @@ -71,14 +75,14 @@ The protocol can be used by clients to verify a server certificate, by a server ## Encryption of state -The Coordinator holds the MarbleRuns state, which consists of the [manifest](../features/manifest.md), the [managed secrets](../features/secrets-management.md), and the [certificates for its CA](../features/attestation.md). -The state is stored encrypted in persistent storage. Therefore Constellation uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a generated 16-byte Data Encryption Key (KEK) to seal the state to persistent storage. +The Coordinator holds MarbleRun's state, which consists of the [manifest](../features/manifest.md), the [managed secrets](../features/secrets-management.md), and the [certificates for its CA](../features/attestation.md). +The state is stored encrypted in persistent storage. For this, MarbleRun uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a generated 16-byte Data Encryption Key (KEK) to seal the state to persistent storage. The DEK is also sealed to persistent storage to recover the state in case of a restart autonomously. [SGX sealing](https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-intel-sgx-sealing.html) is used for that purpose. The Coordinator encrypts the DEK with a key encryption key (KEK). -The KEK MarbleRun uses the SGX sealing key called `Product key`, which is bound to its `Product ID` and the enclave's author `MRSIGNER` identity. +MarbleRun uses the SGX sealing key called `Product key` as its KEK, which is bound to its `Product ID` and the enclave's author `MRSIGNER` identity. In other words, a fresh and benign enclave instance of the same identity can recover that key. -Hence, as long as the Coordinator is restarted on the same CPU, it can obtain the same KEK from the CPU based on its identity, decrypt the DEK and recover its state. +Hence, if the Coordinator is restarted on the same CPU, it can obtain the same KEK from the CPU, decrypt the DEK, and recover its state. If the Coordinator is restarted on a different CPU, it won't be able to obtain the same SGX sealing key from the CPU. Therefore, MarbleRun provides a [recovery feature](../features/recovery.md#recovery). diff --git a/docs/docs/features/recovery.md b/docs/docs/features/recovery.md index 6f752019..0f97eb54 100644 --- a/docs/docs/features/recovery.md +++ b/docs/docs/features/recovery.md @@ -8,7 +8,7 @@ The application can use the SGX seal key as its root secret. However, this binds the application and its state to the physical machine because seal keys are unique to a single CPU. The usual SGX programming model isn't suited for virtual environments or distributed applications. -With MarbleRun, the Coordinator [manages the Marbles[' secrets](../features/secrets-management.md), and Marbles obtain them securely on start. +With MarbleRun, the Coordinator [manages the Marbles' secrets](../features/secrets-management.md), and Marbles obtain them securely on start. Thus, Marbles can be distributed and rescheduled on arbitrary machines. This narrows the challenge of persistent storage down to the Coordinator itself. diff --git a/docs/docs/getting-started/quickstart.md b/docs/docs/getting-started/quickstart.md index b5774b82..95ad62ad 100644 --- a/docs/docs/getting-started/quickstart.md +++ b/docs/docs/getting-started/quickstart.md @@ -102,7 +102,7 @@ The emojivoto demo already comes with a [manifest](https://github.com/edgelesssy ```bash -marblerun manifest set tools/manifest.json $MARBLERUN --insecure +marblerun manifest set tools/manifest.json $MARBLERUN ``` diff --git a/docs/styles/Vocab/edgeless/accept.txt b/docs/styles/Vocab/edgeless/accept.txt index f00df204..a8048e75 100644 --- a/docs/styles/Vocab/edgeless/accept.txt +++ b/docs/styles/Vocab/edgeless/accept.txt @@ -5,7 +5,6 @@ Alibaba auto-injection [Cc]loud collateral -conceptionally config Dockerfile emojivoto From 3604a46dc747a1b324da06f6f9b797cafb1ec1b8 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 9 Aug 2023 09:04:37 +0200 Subject: [PATCH 16/25] Apply suggestions --- docs/docs/architecture/marbles.md | 2 +- docs/docs/architecture/security.md | 4 +++- docs/docs/features/recovery.md | 2 +- docs/docs/getting-started/installation.md | 2 +- docs/docs/intro.md | 2 +- 5 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/docs/architecture/marbles.md b/docs/docs/architecture/marbles.md index abd4631a..d34f04ec 100644 --- a/docs/docs/architecture/marbles.md +++ b/docs/docs/architecture/marbles.md @@ -1,6 +1,6 @@ # Marbles -Marbles are MarbleRun's data planes and run your application code in secure enclaves within otherwise regular Docker containers. +Marbles are MarbleRun's data planes and run your application code in secure enclaves. Marbles communicate with the Coordinator via gRPC over TLS. MarbleRun [supports multiple enclave runtimes](../features/runtimes.md) as the data plane. See the [Add a Service](../workflows/add-service.md) section on how to build and configure an enclave as Marble. diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 0fc12a8a..9c860fe0 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -69,7 +69,9 @@ Depending on the Marble's runtime, the certificate can be used [manually](../wor ## Attested TLS (aTLS) In a CC environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components. -aTLS modifies the TLS handshake by embedding an attestation statement into the TLS certificate. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. +With aTLS, the party to be authenticated somehow binds its TLS certificate to an attestation statement. +For example, it embeds the certificate's public key into an attestation statement. +Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS). diff --git a/docs/docs/features/recovery.md b/docs/docs/features/recovery.md index 0f97eb54..eae449c1 100644 --- a/docs/docs/features/recovery.md +++ b/docs/docs/features/recovery.md @@ -32,7 +32,7 @@ In this mode of operation, manual recovery is only required when * all Coordinator instances are stopped at the same time, and * all new instances are scheduled on new physical hosts. -In other words, if at least one instance is scheduled on a host running a previous instance, the state can be recovered automatically. +In other words, if at least one instance is scheduled on a host where a previous instance was running, the state can be recovered automatically. ## Recovery diff --git a/docs/docs/getting-started/installation.md b/docs/docs/getting-started/installation.md index 5368bac3..3a56faed 100644 --- a/docs/docs/getting-started/installation.md +++ b/docs/docs/getting-started/installation.md @@ -9,7 +9,7 @@ Finally, you will add your own services and set up a corresponding manifest. Make sure the following requirements are met: -* Your machine is running Linux on an x86 (AMD64) CPU +* Your machine is running Ubuntu 20.04 on an x86 (AMD64) CPU * You have access to a Kubernetes cluster and kubectl installed and configured Probably the easiest way to get started is to run Kubernetes on your local machine using [minikube](https://minikube.sigs.k8s.io/docs/start/). Please check our [prerequisites](../deployment/kubernetes.md#prerequisites) if you want to setup an SGX-enabled cluster. Another easy way is to use [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal), which offers SGX-enabled nodes. diff --git a/docs/docs/intro.md b/docs/docs/intro.md index 908aa36a..85094883 100644 --- a/docs/docs/intro.md +++ b/docs/docs/intro.md @@ -7,7 +7,7 @@ slug: / MarbleRun is a framework for deploying distributed confidential computing applications. MarbleRun acts as a confidential operator for your deployment. Think of a trusted party in the control plane. -Build your confidential microservices with tools like [EGo, Gramine, or similar runtimes](./features/runtimes.md), orchestrate them with Kubernetes on an SGX-enabled cluster, and let MarbleRun take care of the rest. +Build your confidential microservices with [EGo, Gramine, or similar runtimes](./features/runtimes.md), orchestrate them with Kubernetes on an SGX-enabled cluster, and let MarbleRun take care of the rest. Deploy end-to-end secure and verifiable AI pipelines or crunch on sensitive big data in the cloud. Confidential computing at scale has never been easier. MarbleRun simplifies the process by handling much of the groundwork. From c54cfcc45b88cf6b582052f1f52602c747b781fe Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 9 Aug 2023 09:05:08 +0200 Subject: [PATCH 17/25] Fix vale and spelling --- docs/docs/architecture/security.md | 2 +- docs/docs/building-enclaves/occlum.md | 14 ++++++------ docs/docs/deployment/platforms/alibaba.md | 13 +++++------ docs/docs/deployment/platforms/on-prem.md | 15 ++++++------ docs/docs/features/attestation.md | 10 ++++---- docs/docs/features/kubernetes-integration.md | 24 +++++++++----------- docs/docs/features/runtimes.md | 2 +- docs/docs/getting-started/installation.md | 12 +++++----- docs/docs/getting-started/quickstart.md | 18 +++++++-------- docs/docs/workflows/add-service.md | 12 +++++----- docs/docs/workflows/verification.md | 2 +- docs/styles/Vocab/edgeless/accept.txt | 3 +++ 12 files changed, 63 insertions(+), 64 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 9c860fe0..c31191de 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -11,7 +11,7 @@ The [Coordinator](coordinator.md) and the [Marbles](marbles.md) run inside SGX E MarbleRun uses cryptography for the following tasks. * Authentication and authorization -* Establishing a public key infrastructure (PKI) for MarbleRun +* Establishing a public critical infrastructure (PKI) for MarbleRun * Encrypting network traffic via mutual TLS between enclaves * Encrypting persistent state diff --git a/docs/docs/building-enclaves/occlum.md b/docs/docs/building-enclaves/occlum.md index 748a5bdc..7f7251e8 100644 --- a/docs/docs/building-enclaves/occlum.md +++ b/docs/docs/building-enclaves/occlum.md @@ -2,18 +2,18 @@ Running an Occlum app with MarbleRun requires some changes to its manifest. ## Requirements -Set up an environment to create Occlum images. For an easy start, we recommend that you use either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or use [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our platform deployment guide](../deployment/platforms/platforms.md). +Set up an environment to create Occlum images. For an easy start, we recommend using either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our platform deployment guide](../deployment/platforms/platforms.md). -To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running, and then come back here to adapt it for use with MarbleRun. +To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running and then come back here to adapt it for use with MarbleRun. ## Configuration ### Premain executable -Add our pre-built [premain-libos](https://github.com/edgelesssys/marblerun/releases/latest/download/premain-libos) executable to your Occlum image, e.g., by copying it to `image/bin/premain-libos`. By default, Occlum restricts executable files to the `/bin` directory. If you placed the `premain-libos` binary to a different path, you need to adjust this setting accordingly. +Add our pre-built [premain-libos](https://github.com/edgelesssys/marblerun/releases/latest/download/premain-libos) executable to your Occlum image, e.g., by copying it to `image/bin/premain-libos`. By default, Occlum restricts executable files to the `/bin` directory. If you placed the `premain-libos` binary at a different path, you must adjust this setting accordingly. Finally, define the original entry point for your Occlum instance as the first `Argv` parameter for your Marble in MarbleRun's `manifest.json`. See [Defining a manifest](../workflows/define-manifest.md) for more information on how to define the `Argv` parameters. This lets MarbleRun launch your application after it succeeded in authenticating with the Coordinator and provides entrypoint pinning similar to the one offered in `Occlum.json`. ### Environment variables -The Marble needs to retrieve the MarbleRun specific configuration parameters via environment variables, as [described under Step 3 in "Adding a service."](../workflows/add-service.md) +The Marble needs to retrieve the MarbleRun-specific configuration parameters via environment variables, as [described under Step 3 in "Adding a service."](../workflows/add-service.md) To pass environment variables to the enclave, Occlum requires them to be specified in the `env` section in `Occlum.json`. @@ -41,9 +41,9 @@ For example, consider this configuration: This will allow you to embed the expected default values during build time. It also lets the user/host system change them during run time when a non-default Coordinator configuration is used. ### Resource limits -The premain process is written in Go. The enclave needs to have enough resources for the Go runtime, plus additional memory to launch your application. +The premain process is written in Go. The enclave must have enough resources for the Go runtime and additional memory to launch your application. -We recommend starting with the following values which should work fine for light-weight to medium memory demanding applications: +We recommend starting with the following values, which should work fine for light-weight to medium memory-demanding applications: ```javascript "user_space_size": "2048MB", "default_mmap_size": "900MB" @@ -68,7 +68,7 @@ left: `SGX_QL_SUCCESS`, right: `SGX_QL_NETWORK_ERROR`: fail to launch QE', src/util/sgx/dcap/quote_generator.rs:22:13 ``` -You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration needs to be correctly set from **inside the container.** +You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration must be correctly set **inside the container.**** If you use an Azure Confidential Computing machine, you can use our [provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). It patches the official Occlum image to use the Azure DCAP client, which handles the configuration automatically. diff --git a/docs/docs/deployment/platforms/alibaba.md b/docs/docs/deployment/platforms/alibaba.md index 928bf1a5..755ff124 100644 --- a/docs/docs/deployment/platforms/alibaba.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -6,16 +6,15 @@ Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment [ACK hosts Kubernetes pods in SGX-capable Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing) and exposes the underlying SGX hardware. ### Prerequisites -* Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX enabled worker nodes. +* Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX-enabled worker nodes. ### Deploy MarbleRun -See our [Kubernetes guide](../kubernetes.md) on installing MarbleRun in your ACK cluster. +See the [Kubernetes guide](../kubernetes.md) on installing MarbleRun in your ACK cluster. ## Alibaba Cloud Elastic Compute Service - -With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud. +With 7th-generation [security-enhanced ECS](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25) instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud. You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances) instance. The description below uses a VM running Ubuntu 18.04. @@ -38,7 +37,7 @@ The description below uses a VM running Ubuntu 18.04. 1. Set configuration for Alibaba Cloud SGX remote attestation service - Alibaba Cloud provides a PCCS for remote attestation, deployed per-region. For optimal stability, accessing the service in your instance's region is recommended. + Alibaba Cloud provides a PCCS for remote attestation deployed per region. For optimal stability, accessing the service in your instance's region is recommended. The configuration is set in `/etc/sgx_default_qcnl.conf`. * If your instance is assigned a public IP address, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region: @@ -63,5 +62,5 @@ The description below uses a VM running Ubuntu 18.04. ### Deploy MarbleRun -You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../standalone.md). -Alternatively, you can install a Kubernetes cluster. Probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/). See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. +You can run MarbleRun standalone on your Alibaba Cloud ECS VM. See the [standalone](../standalone.md) guide](../standalone.md). +Alternatively, you can install a Kubernetes cluster. Probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/). See the [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. diff --git a/docs/docs/deployment/platforms/on-prem.md b/docs/docs/deployment/platforms/on-prem.md index 9328f842..1adf9f84 100644 --- a/docs/docs/deployment/platforms/on-prem.md +++ b/docs/docs/deployment/platforms/on-prem.md @@ -11,17 +11,16 @@ This guide walks you through setting up MarbleRun for your on-premises deploymen To deploy MarbleRun with Intel SGX, the machine or VM has to support Intel SGX. Particularly, MarbleRun requires support for the SGX Data Center Attestation Primitives (DCAP). You can verify [if your CPU supports DCAP](https://www.intel.com/content/www/us/en/support/articles/000057420/software/intel-security-products.html). - -For more information read this article on [detecting Intel Software Guard Extensions](https://software.intel.com/content/www/us/en/develop/articles/properly-detecting-intel-software-guard-extensions-in-your-applications.html) in your applications. +For more information, read this article on [detecting Intel Software Guard Extensions](https://software.intel.com/content/www/us/en/develop/articles/properly-detecting-intel-software-guard-extensions-in-your-applications.html) in your applications. #### BIOS BIOS support is required for Intel SGX to provide the capability to enable and configure the Intel SGX feature in the system. -Currently, most of the SGX capable systems have SGX disabled by default in the BIOS. This default setting might change but for now, you need to manually enable it if it's not already enabled. +Currently, most of the SGX-capable systems have SGX disabled by default in the BIOS. This default setting might change, but for now, you need to manually enable it if it's not already enabled. #### Updates -As with any modern technology, Intel SGX has been affected by security vulnerabilities. Intel addresses these vulnerabilities by updating the microcode of CPUs, changing the hardware of new CPUs, and updating the system software. Each microcode update that patches an SGX vulnerability requires a BIOS update. During remote attestation, it's checked that the microcode of the CPU which is deployed by the BIOS is up to date. The microcode and platform enclaves are commonly called the platform `Trusted Computing Base (TCB)`. +As with any modern technology, Intel SGX has been affected by security vulnerabilities. Intel addresses these vulnerabilities by updating the microcode of CPUs, changing the hardware of new CPUs, and updating the system software. Each microcode update that patches an SGX vulnerability requires a BIOS update. During remote attestation, it's checked that the microcode of the CPU, which is deployed by the BIOS, is up to date. The microcode and platform enclaves are commonly called the platform `Trusted Computing Base (TCB)`. If your BIOS/firmware is outdated, you will see errors as `Platform TCB (2) is not up-to-date (oe_result_t=OE_TCB_LEVEL_INVALID)` during remote attestation procedures. @@ -39,19 +38,19 @@ Most of the popular hypervisors support SGX: #### Driver You need to install the [DCAP SGX Driver](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf). -Azure provides the instructions on [how to install this driver](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) that you can use for your on-premises machines. +Azure provides instructions on [how](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) to install this driver](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) that you can use for your on-premises machines. ### SGX Data Center Attestation Primitives (DCAP) -DCAP is the new attestation mechanism for SGX [replacing EPID](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html). +DCAP is the new attestation mechanism for SGX, [replacing](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html)[ EPID](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html). You can find an overview of DCAP in the [official Intel docs](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf). MarbleRun only supports DCAP and requires DCAP libraries installed and configured on your system. -From the perspective of MarbleRun and your workloads DCAP is accessed with a [Quote Generation Library (QGL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/README.md) and a [Quote Verification Library (QVL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteVerification/README.md) for generating and verifying quotes respectively. +From the perspective of MarbleRun and your workloads, DCAP is accessed with a [Quote Generation Library (QGL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/README.md) and a [Quote Verification Library (QVL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteVerification/README.md) for generating and verifying quotes respectively. The QGL and QVL libraries need to be configured to talk to a [Provisioning Certificate Caching Service (PCCS)](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf). You currently have two options regarding PCCS for your on-premises machines and clusters: -1. Use a public PCCS service by configuring your QGL and QVL to point to the public endpoints. Currently, Azure and Alibaba Cloud provide such a service, but require using infrastructure by these providers to make full use of the service. +1. Use a public PCCS service by configuring your QGL and QVL to point to the public endpoints. Currently, Azure and Alibaba Cloud provide such a service but require using infrastructure by these providers to make full use of the service. 1. Run your own PCCS and expose it to your machine or cluster. See [Intel's demo reference implementation](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/pccs/README.md) and [design guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/SGX_DCAP_Caching_Service_Design_Guide.pdf) for more information. diff --git a/docs/docs/features/attestation.md b/docs/docs/features/attestation.md index 76191cf2..73f52a6f 100644 --- a/docs/docs/features/attestation.md +++ b/docs/docs/features/attestation.md @@ -8,7 +8,7 @@ However, MarbleRun works with any DCAP service complying with the SGX specificat Initially, the MarbleRun Coordinator is deployed to the cluster. The Coordinator generates an X.509 certificate chain with a root and intermediate certificate authority (CA). -It generates a remote attestation quote which contains the hash of the root certificate (as `report_data`) as well as a hardware measurement of its enclave (`MRENCLAVE` of the report). +It generates a remote attestation quote that contains the hash of the root certificate (as `report_data`) as well as a hardware measurement of its enclave (`MRENCLAVE` of the report). The admin receives the quote and verifies the Coordinator's integrity via the measurement. Next, they verify that the quote contains the root certificate’s hash. This ensures that the certificate was indeed generated by this Coordinator and wasn't manipulated during transport. @@ -22,7 +22,7 @@ The [manifest](../workflows/define-manifest.md) describes which Marbles (service The Coordinator enforces the manifest and only admits the desired Marbles to the cluster. Each Marble registers itself with a quote via gRPC to the Coordinator. This quote is tied to the TLS session of the gRPC call. It contains the Marble's measurements. The Coordinator compares them to the manifest to ensure the Marble's identity and integrity. The [Marble's TLS credentials](../features/secrets-management.md) are generated by the Coordinator and signed with the intermediate certificate. -The Coordinator's certificate chain as well as the freshly generated Marble certificate are then delivered to the Marble. +The Coordinator's certificate chain and the freshly generated Marble certificate are then delivered to the Marble. ![Marble deployment](../_media/marble_deployment.svg) @@ -35,17 +35,17 @@ If a client wants to connect to a Marble, the Coordinator acts as a trusted CA t ![service mesh](../_media/service_mesh.svg) -When the Coordinator receives an update to the manifest, e.g. a Marble is updated to a newer security version, a new intermediate certificate is generated. The communication between Marbles relies on the intermediate certificate as root of trust. This ensures that Marbles which were deployed before the update can't communicate with Marbles that have the newer security version. +When the Coordinator receives an update to the manifest, e.g., a Marble is updated to a newer security version, a new intermediate certificate is generated. The communication between Marbles relies on the intermediate certificate as the root of trust. This ensures that Marbles which were deployed before the update can't communicate with Marbles that have the newer security version. ## Cluster attestation -The Coordinator issues one concise remote-attestation statement for your whole distributed app. +The Coordinator issues one concise remote attestation statement for your whole distributed app. The Coordinator's build-time measurement is distributed to the relying party (this must be done by the admin or operator). The relying party requests the Coordinator's quote and certificate chain. The quote contains the hash of the Coordinator's root CA certificate, which is verified against the received certificate chain. The quote also contains the Coordinator's measurement, which is verified against the build-time measurement. The relying party then requests the manifest from the Coordinator and ensures it contains the expected Marbles and their expected measurements. -The steps required on the client-side are described in our [verification hands-on](../workflows/verification.md). +The steps required on the client side are described in the [verification hands-on](../workflows/verification.md). ![cluster attestation](../_media/verify_cluster.svg) diff --git a/docs/docs/features/kubernetes-integration.md b/docs/docs/features/kubernetes-integration.md index b070957a..677a5c5d 100644 --- a/docs/docs/features/kubernetes-integration.md +++ b/docs/docs/features/kubernetes-integration.md @@ -1,14 +1,13 @@ # Kubernetes integration -MarbleRun provides its data-plane configuration through Kubernetes resource definitions. For this, like regular service meshes, MarbleRun uses Kubernetes' [admission controllers](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook). +MarbleRun provides its data-plane configuration through Kubernetes resource definitions. Like regular service meshes, MarbleRun uses Kubernetes' [admission controllers](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook). MarbleRun optionally injects [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) and [resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for its SGX device plugin. See the [Kubernetes deployment](../deployment/kubernetes.md#sgx-device-plugin-on-kubernetes) section for more information. You can enable auto-injection of the data-plane configuration using Pod labels. ## The `marbletype` label - -In MarbleRun, Marbles (i.e, secure enclaves) are defined in the [manifest](../workflows/define-manifest.md). You need to reference Marbles in your Kubernetes resource description as follows using the `marblerun/marbletype` label: +MarbleRun defines Marbles (i.e., secure enclaves) in the [manifest](../workflows/define-manifest.md). You need to reference Marbles in your Kubernetes resource description as follows using the `marblerun/marbletype` label: ```javascript { @@ -30,21 +29,20 @@ metadata: marblerun/marbletype: voting-svc ``` -We use this label to map Kubernetes Pods to MarbleRun Marbles. +The label is used to map Kubernetes Pods to MarbleRun Marbles. When you deploy your application with the `marblerun/marbletype` label, the Pod's creation is intercepted by MarbleRun. -It will then inject environment variables and SGX resources into the containers of the Pod based on the value of the label. -If the `marblerun/marbletype` label is missing, the Pod's injection is skipped. +It will then inject environment variables and SGX resources into the Pod containers based on the label's value. +The Pod's injection is skipped if the `marblerun/marbletype` label is missing. ## The `marblecontainer` label - -By default MarbleRun will inject environment variables and resource requests into all containers of the Pod. +By default, MarbleRun will inject environment variables and resource requests into all containers of the Pod. You can use the `marblerun/marblecontainer=` label to limit injection to the specified container. -This is useful if your configuration uses multiple containers in the same Pod, e.g. a sidecar proxy, and you wish to prevent non enclave containers taking up resources. +This is useful if your configuration uses multiple containers in the same Pod, e.g., a sidecar proxy, and you wish to prevent non-enclave containers from taking up resources. ## The `resource-injection` label -To prevent MarbleRun from injecting SGX resource request, you can set the label `marblerun/resource-injection=disabled`. -Use this if you want to set your own SGX resource requests, or if you need to start a Marble in simulation mode without any SGX resources. +To prevent MarbleRun from injecting SGX resource requests, you can set the label `marblerun/resource-injection=disabled`. +Use this if you want to set your own SGX resource requests or if you need to start a Marble in simulation mode without any SGX resources. ## Injected environment variables @@ -52,7 +50,7 @@ The webhook will inject the following environment variables into each container * `EDG_MARBLE_TYPE`: The value of the `marblerun/marbletype` label * `EDG_MARBLE_COORDINATOR_ADDR`: The address of the MarbleRun Coordinator running on the cluster -* `EDG_MARBLE_DNS_NAMES`: DNS names of the pod derived from `marbletype` and namespace: `marbletype, marbletype.namespace, marbletype.namespace.svc.cluster.local` +* `EDG_MARBLE_DNS_NAMES`: DNS names of the pod are derived from `marbletype` and namespace: `marbletype, marbletype.namespace, marbletype.namespace.svc.cluster.local` * `EDG_MARBLE_UUID_FILE`: The mounted UUID of the Marble -If an environment variable is already set before the webhook handles the creation request, the variable won't be overwritten and the custom value is used instead. +If an environment variable is already set before the webhook handles the creation request, the variable won't be overwritten, and the custom value is used instead. diff --git a/docs/docs/features/runtimes.md b/docs/docs/features/runtimes.md index d49dfcf1..c30e174d 100644 --- a/docs/docs/features/runtimes.md +++ b/docs/docs/features/runtimes.md @@ -6,7 +6,7 @@ MarbleRun strives to be runtime-agnostic. Currently, supported runtimes are desc [EGo](https://github.com/edgelesssys/ego) is the preferred way for writing confidential Go applications from scratch as well as porting existing ones. Usage is similar to conventional Go programming. Start [building your service with EGo](../building-enclaves/ego.md) to use it with MarbleRun. ## Edgeless RT -With [Edgeless RT](https://github.com/edgelesssys/edgelessrt), you can create confidential C++ applications with a low TCB. Please follow the build instructions provided [in our C++ sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) to use it with MarbleRun. +With [Edgeless RT](https://github.com/edgelesssys/edgelessrt), you can create confidential C++ applications with a low TCB. Please follow the build instructions provided [in the C++ sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) to use it with MarbleRun. ## Gramine [Gramine](https://gramineproject.io/) is a popular choice for wrapping unmodified applications into enclaves. diff --git a/docs/docs/getting-started/installation.md b/docs/docs/getting-started/installation.md index 3a56faed..7640d34a 100644 --- a/docs/docs/getting-started/installation.md +++ b/docs/docs/getting-started/installation.md @@ -3,7 +3,7 @@ In this guide, you will install MarbleRun into your Kubernetes cluster and deploy a sample confidential application to demonstrate the capabilities of MarbleRun. Installing MarbleRun is easy. First, you will install the CLI (command-line interface) onto your local machine. Using this CLI, you’ll then install the control plane onto your Kubernetes cluster. -Finally, you will add your own services and set up a corresponding manifest. +Finally, you will add your services and set up a corresponding manifest. ## Prerequisites @@ -12,22 +12,22 @@ Make sure the following requirements are met: * Your machine is running Ubuntu 20.04 on an x86 (AMD64) CPU * You have access to a Kubernetes cluster and kubectl installed and configured -Probably the easiest way to get started is to run Kubernetes on your local machine using [minikube](https://minikube.sigs.k8s.io/docs/start/). Please check our [prerequisites](../deployment/kubernetes.md#prerequisites) if you want to setup an SGX-enabled cluster. Another easy way is to use [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal), which offers SGX-enabled nodes. +Probably the easiest way to get started is to run Kubernetes on your local machine using [minikube](https://minikube.sigs.k8s.io/docs/start/). Please check our [prerequisites](../deployment/kubernetes.md#prerequisites) if you want to set up an SGX-enabled cluster. Another easy way is to use [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal), which offers SGX-enabled nodes. -You can validate your setup by running: +You can validate your setup by running the following: ```bash kubectl version --short ``` -You should see an output with both a Client Version and Server Version component. +You should see an output with both a Client Version and a Server Version component. Now your cluster is ready and we’ll install the MarbleRun CLI. ## Install the MarbleRun CLI If this is your first time running MarbleRun, you will need to download the MarbleRun command-line interface (CLI) onto your local machine. The CLI will allow you to interact with your MarbleRun deployment. -To install the CLI, run: +To install the CLI, run the following: @@ -48,7 +48,7 @@ sudo chmod +x /usr/local/bin/marblerun -Once installed, verify the CLI is running correctly with: +Once installed, verify the CLI is running correctly with the following: ```bash marblerun diff --git a/docs/docs/getting-started/quickstart.md b/docs/docs/getting-started/quickstart.md index 95ad62ad..ba920889 100644 --- a/docs/docs/getting-started/quickstart.md +++ b/docs/docs/getting-started/quickstart.md @@ -2,8 +2,8 @@ The following steps guide you through the process of deploying MarbleRun in your cluster and deploying a sample app. This example assumes that you have successfully [installed MarbleRun](./installation.md), and have access to a Kubernetes cluster. -A working SGX DCAP environment is required for MarbleRun to work. For the ease of exploring and testing we provide a simulation mode with `--simulation` that runs without SGX hardware. -Depending on your setup you may follow the quickstart for SGX-enabled clusters. Alternatively, if your setup doesn't support SGX, you can follow the quickstart in simulation mode by selecting the respective tabs. +A working SGX DCAP environment is required for MarbleRun to work. For ease of exploring and testing, we provide a simulation mode with `--simulation` that runs without SGX hardware. +Depending on your setup, you may follow the quickstart for SGX-enabled clusters. Alternatively, if your setup doesn't support SGX, you can follow the quickstart in simulation mode by selecting the respective tabs. ## Step 1: Install the control plane onto your cluster @@ -28,7 +28,7 @@ marblerun install --simulation The `marblerun install` command generates a Kubernetes manifest with all the necessary control plane resources. This includes a deployment for the Coordinator and for MarbleRun's [admission controller.](../features/kubernetes-integration.md) -The simulation flag tells MarbleRun that real SGX hardware might not be present and the SGX-layer should be emulated. +The simulation flag tells MarbleRun that real SGX hardware might not be present and the SGX layer should be emulated. @@ -52,7 +52,7 @@ export MARBLERUN=localhost:4433 ## Step 2: Verify the Coordinator -After installing the Coordinator we need to verify its integrity. +After installing the Coordinator, we need to verify its integrity. For this, we utilize SGX remote attestation and obtain the Coordinator's root certificate. Verify the quote and get the Coordinator's root certificate @@ -84,9 +84,9 @@ It can also be used as a root of trust for [authenticating your confidential app ## Step 3: Deploy the demo application To get a feel for how MarbleRun would work for one of your services, you can install a demo application. -The emojivoto application is a standalone Kubernetes application that uses a mix of gRPC and HTTP calls to allow the users to vote on their favorite emojis. +The emojivoto application is a standalone Kubernetes application that uses a mix of gRPC and HTTP calls to allow users to vote on their favorite emojis. Created as a demo application for the popular [Linkerd](https://linkerd.io) service mesh, we've made a confidential variant that uses a confidential service mesh for all gRPC and HTTP connections. -Clone the [demo application's repository](https://github.com/edgelesssys/emojivoto.git) from GitHub by running: +Clone the [demo application's repository](https://github.com/edgelesssys/emojivoto.git) from GitHub by running the following: ```bash git clone https://github.com/edgelesssys/emojivoto.git && cd emojivoto @@ -96,7 +96,7 @@ git clone https://github.com/edgelesssys/emojivoto.git && cd emojivoto MarbleRun guarantees that the topology of your distributed app adheres to a manifest specified in simple JSON. MarbleRun verifies the integrity of services, bootstraps them, and sets up encrypted connections between them. -The emojivoto demo already comes with a [manifest](https://github.com/edgelesssys/emojivoto/blob/main/tools/manifest.json), which you can deploy onto MarbleRun by running: +The emojivoto demo already comes with a [manifest](https://github.com/edgelesssys/emojivoto/blob/main/tools/manifest.json), which you can deploy onto MarbleRun by running the following: @@ -116,7 +116,7 @@ marblerun manifest set tools/manifest.json $MARBLERUN --insecure -You can check that the state of MarbleRun changed and is now ready to authenticate your services by running: +You can check that the state of MarbleRun has changed and is now ready to authenticate your services by running: @@ -173,7 +173,7 @@ kubectl -n emojivoto port-forward svc/web-svc 8443:443 --address 0.0.0.0 ``` Now visit [https://localhost:8443](https://localhost:8443). -You'll be presented with a certificate warning because your browser by default doesn't trust certificates signed by MarbleRun. +You'll be presented with a certificate warning because your browser, by default, doesn't trust certificates signed by MarbleRun. You can ignore this error for now and proceed to the website.\ Voila! Your emoji votes have never been safer! diff --git a/docs/docs/workflows/add-service.md b/docs/docs/workflows/add-service.md index da555a65..8a9dd5fa 100644 --- a/docs/docs/workflows/add-service.md +++ b/docs/docs/workflows/add-service.md @@ -14,9 +14,9 @@ To get your service ready for MarbleRun, you need to rebuild it with one of the Skip this step, when using EGo with [TTLS](../features/transparent-TLS.md). -Quick refresher: MarbleRun's Coordinator issues TLS credentials for each verified Marble (i.e., a service running in a secure enclave) as is described in our [secrets management chapter](../features/secrets-management.md). +Quick refresher: MarbleRun's Coordinator issues TLS credentials for each verified Marble (i.e., a service running in a secure enclave) as is described in the [secrets management chapter](../features/secrets-management.md). -The TLS X.509 certificate and the corresponding private key can be securely passed to a service through files, environment variables, or command line arguments. This is defined in the manifest as is described in our [writing a manifest hands-on](../workflows/define-manifest.md#marbles). +The TLS X.509 certificate and the corresponding private key can be securely passed to a service through files, environment variables, or command line arguments. This is defined in the manifest as is described in the [writing a manifest hands-on](../workflows/define-manifest.md#marbles). Make sure that your service reads the credentials from one of these sources, e.g., the file `/tmp/mycert.cert` or the environment variable `MY_PRIVATE_KEY`, and uses them for internal and external connections. If you're lucky, your service already does this and you don't need to change a thing in the code. @@ -26,7 +26,7 @@ Now that your service is ready, you need to make two types of entries in the man ### **Step 2.1:** Define the enclave software package -As is described in more detail in our [writing a manifest hands-on](../workflows/define-manifest.md#packages), the manifest contains a section `Packages`, in which allowed enclave software packages are defined. +As is described in more detail in the [writing a manifest hands-on](../workflows/define-manifest.md#packages), the manifest contains a section `Packages`, in which allowed enclave software packages are defined. #### EGo / EdgelessRT To add an entry for your EGo / EdgelessRT service, run the `oesign` tool on the enclave file you built in the previous step as follows. (`oesign` is installed with [Edgeless RT](https://github.com/edgelesssys/edgelessrt).) @@ -96,7 +96,7 @@ Use `UniqueID` (i.e., `MRENCLAVE` in Intel SGX speak) or the triplet of `SignerI ### **Step 2.2:** Define the parameters -Now you can define with which parameters (i.e., files, environment variables, and command line arguments) your service is allowed to run. This is done in the `Marbles` section of the manifest as is described in our [writing a manifest hands-on](../workflows/define-manifest.md#marbles). When using EGo, define all TTLS connections as described in the [manifest hands-on](../workflows/define-manifest.md#tls). +Now you can define with which parameters (i.e., files, environment variables, and command line arguments) your service is allowed to run. This is done in the `Marbles` section of the manifest as is described in the [writing a manifest hands-on](../workflows/define-manifest.md#marbles). When using EGo, define all TTLS connections as described in the [manifest hands-on](../workflows/define-manifest.md#tls). Otherwise, as discussed in [Step #1](#make-your-service-use-the-provided-tls-credentials), make sure that the TLS credentials for your service (i.e., `MarbleRun.MarbleCert.Cert` and `MarbleRun.MarbleCert.Private`) are injected such that your service finds them at runtime. @@ -117,7 +117,7 @@ For EGo (`ego marblerun`), Gramine (`gramine-sgx`), and Occlum (`occlum run`) us The environment variables have the following purposes. -* `EDG_MARBLE_COORDINATOR_ADDR` is the network address of the Coordinator's API for Marbles. When you deploy the Coordinator using our Helm repository as is described in our [deploying MarbleRun hands-on](../deployment/kubernetes.md), the default address is `coordinator-mesh-api.marblerun:2001`. +* `EDG_MARBLE_COORDINATOR_ADDR` is the network address of the Coordinator's API for Marbles. When you deploy the Coordinator using the Helm repository as is described in the [deploying MarbleRun hands-on](../deployment/kubernetes.md), the default address is `coordinator-mesh-api.marblerun:2001`. * `EDG_MARBLE_TYPE` needs to reference one entry from your manifest's `Marbles` section. @@ -164,4 +164,4 @@ spec: value: "$PWD/uuid" ``` -Refer to our [emojivoto](https://github.com/edgelesssys/emojivoto) app for complete Helm chart examples. +Refer to the [emojivoto](https://github.com/edgelesssys/emojivoto) app for complete Helm chart examples. diff --git a/docs/docs/workflows/verification.md b/docs/docs/workflows/verification.md index 991b9aa8..103fbb8b 100644 --- a/docs/docs/workflows/verification.md +++ b/docs/docs/workflows/verification.md @@ -57,7 +57,7 @@ To verify the coordinator, `era` requires the Coordinator's UniqueID (or MRENCLA In production, the expected values in `coordinator-era.json` would be generated when building the Coordinator and distributed to your clients. When you build MarbleRun from source, you can find the file in your build directory. For testing with a pre-built release, there's a Coordinator image at `ghcr.io/edgelesssys/marblerun/coordinator`. -You can pull the corresponding `coordinator-era.json` file from our release page: +You can pull the corresponding `coordinator-era.json` file from the release page: ```bash wget https://github.com/edgelesssys/marblerun/releases/latest/download/coordinator-era.json diff --git a/docs/styles/Vocab/edgeless/accept.txt b/docs/styles/Vocab/edgeless/accept.txt index a8048e75..8a18e73d 100644 --- a/docs/styles/Vocab/edgeless/accept.txt +++ b/docs/styles/Vocab/edgeless/accept.txt @@ -6,6 +6,7 @@ auto-injection [Cc]loud collateral config +CPU Dockerfile emojivoto Env @@ -14,6 +15,7 @@ Equinix hardcode helloworld hostname +IP Istio kubectl Linkerd @@ -33,6 +35,7 @@ Tensorflow toleration toolchain updatable +vSphere VM walkthrough Xeon From 4858f640d8a852acd71a9945d67288ff3da229ac Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Wed, 9 Aug 2023 10:59:51 +0200 Subject: [PATCH 18/25] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> --- docs/docs/architecture/security.md | 4 ++-- docs/docs/building-enclaves/occlum.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index c31191de..27046d77 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -11,7 +11,7 @@ The [Coordinator](coordinator.md) and the [Marbles](marbles.md) run inside SGX E MarbleRun uses cryptography for the following tasks. * Authentication and authorization -* Establishing a public critical infrastructure (PKI) for MarbleRun +* Establishing a public key infrastructure (PKI) for MarbleRun * Encrypting network traffic via mutual TLS between enclaves * Encrypting persistent state @@ -69,7 +69,7 @@ Depending on the Marble's runtime, the certificate can be used [manually](../wor ## Attested TLS (aTLS) In a CC environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components. -With aTLS, the party to be authenticated somehow binds its TLS certificate to an attestation statement. +With aTLS, the party to be authenticated binds its TLS certificate to an attestation statement. For example, it embeds the certificate's public key into an attestation statement. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS). diff --git a/docs/docs/building-enclaves/occlum.md b/docs/docs/building-enclaves/occlum.md index 7f7251e8..d35bfe91 100644 --- a/docs/docs/building-enclaves/occlum.md +++ b/docs/docs/building-enclaves/occlum.md @@ -68,7 +68,7 @@ left: `SGX_QL_SUCCESS`, right: `SGX_QL_NETWORK_ERROR`: fail to launch QE', src/util/sgx/dcap/quote_generator.rs:22:13 ``` -You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration must be correctly set **inside the container.**** +You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration must be correctly set **inside the container.** If you use an Azure Confidential Computing machine, you can use our [provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). It patches the official Occlum image to use the Azure DCAP client, which handles the configuration automatically. From 71348a92437a00b96d2c849d66ebdaaaee62ad83 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 10 Aug 2023 09:44:04 +0200 Subject: [PATCH 19/25] update svg with invisible frame --- docs/docs/_media/security_architecture.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/_media/security_architecture.svg b/docs/docs/_media/security_architecture.svg index 9221152e..21970277 100644 --- a/docs/docs/_media/security_architecture.svg +++ b/docs/docs/_media/security_architecture.svg @@ -1,3 +1,3 @@ -
MarbleRun Deployment
MarbleRun Deployment
aTLS
gRPC
aTLS...
Coordinator
Coordinator
Manifest
Manifest
Packages
Packages
Marbles
Marbles
Secrets
Secrets
Recovery Keys
Recovery Keys
Certificate Authority
Certificate Authority
mTLS
HTTP, gRPC, TCP
mTLS...
Marble A
Marble A
TLS
HTTP, gRPC, TCP
TLS...
aTLS
gRPC
aTLS...
Marble B
Marble B
Marble
Certificate A
Marble...
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certifi...
Marble
Certificate B
Marble...
aTLS
HTTP
aTLS...
Administrator
Administrat...
aTLS
HTTP
aTLS...
Application user
Application...
Parameters A
Parameters A
 Secrets A
 Secrets A
Parameters B
Parameters B
 Secrets B
 Secrets B
Private Keys
Private Keys
Marble Root Certificate
Marble Root Certific...
Marble Root Certificate
Marble Root Certific...
Marble Root Certificate
Marble Root Certific...Text is not SVG - cannot display \ No newline at end of file +
MarbleRun Deployment
MarbleRun Deployment
aTLS
gRPC
aTLS...
Coordinator
Coordinator
Manifest
Manifest
Packages
Packages
Marbles
Marbles
Secrets
Secrets
Recovery Keys
Recovery Keys
Certificate Authority
Certificate Authority
mTLS
HTTP, gRPC, TCP
mTLS...
Marble A
Marble A
TLS
HTTP, gRPC, TCP
TLS...
aTLS
gRPC
aTLS...
Marble B
Marble B
Marble
Certificate A
Marble...
Root Certificate
Root Certificate
Intermediate Certificate
Intermediate Certifi...
Marble
Certificate B
Marble...
aTLS
HTTP
aTLS...
aTLS
HTTP
aTLS...
Parameters A
Parameters A
 Secrets A
 Secrets A
Parameters B
Parameters B
 Secrets B
 Secrets B
Private Keys
Private Keys
Marble Root Certificate
Marble Root Certific...
Marble Root Certificate
Marble Root Certific...
Marble Root Certificate
Marble Root Certific...
Administrator
Administrator
Application user
Application u...Text is not SVG - cannot display \ No newline at end of file From 246d88b5317bd67736011cbad5648db7f0a5b99f Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 10 Aug 2023 13:23:32 +0200 Subject: [PATCH 20/25] Apply suggestions from code review Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com> --- docs/docs/architecture/security.md | 6 +++--- docs/docs/deployment/platforms/alibaba.md | 4 ++-- docs/docs/deployment/platforms/on-prem.md | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 27046d77..5ca0255d 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -68,9 +68,9 @@ Depending on the Marble's runtime, the certificate can be used [manually](../wor ## Attested TLS (aTLS) -In a CC environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components. +In a confidential computing (CC) environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components. With aTLS, the party to be authenticated binds its TLS certificate to an attestation statement. -For example, it embeds the certificate's public key into an attestation statement. +For example, it embeds the certificate's public key into the attestation statement. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS). @@ -78,7 +78,7 @@ The protocol can be used by clients to verify a server certificate, by a server ## Encryption of state The Coordinator holds MarbleRun's state, which consists of the [manifest](../features/manifest.md), the [managed secrets](../features/secrets-management.md), and the [certificates for its CA](../features/attestation.md). -The state is stored encrypted in persistent storage. For this, MarbleRun uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a generated 16-byte Data Encryption Key (KEK) to seal the state to persistent storage. +The state is stored encrypted in persistent storage. For this, MarbleRun uses [AES128-GCM](https://www.rfc-editor.org/rfc/rfc5116#section-5.1) and a generated 16-byte data encryption key (DEK). The DEK is also sealed to persistent storage to recover the state in case of a restart autonomously. [SGX sealing](https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-intel-sgx-sealing.html) is used for that purpose. The Coordinator encrypts the DEK with a key encryption key (KEK). diff --git a/docs/docs/deployment/platforms/alibaba.md b/docs/docs/deployment/platforms/alibaba.md index 755ff124..43960785 100644 --- a/docs/docs/deployment/platforms/alibaba.md +++ b/docs/docs/deployment/platforms/alibaba.md @@ -14,7 +14,7 @@ See the [Kubernetes guide](../kubernetes.md) on installing MarbleRun in your ACK ## Alibaba Cloud Elastic Compute Service -With 7th-generation [security-enhanced ECS](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25) instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud. +With 7th-generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud. You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances) instance. The description below uses a VM running Ubuntu 18.04. @@ -62,5 +62,5 @@ The description below uses a VM running Ubuntu 18.04. ### Deploy MarbleRun -You can run MarbleRun standalone on your Alibaba Cloud ECS VM. See the [standalone](../standalone.md) guide](../standalone.md). +You can run MarbleRun standalone on your Alibaba Cloud ECS VM. See the [standalone guide](../standalone.md). Alternatively, you can install a Kubernetes cluster. Probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/). See the [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube. diff --git a/docs/docs/deployment/platforms/on-prem.md b/docs/docs/deployment/platforms/on-prem.md index 1adf9f84..1b3b0f2a 100644 --- a/docs/docs/deployment/platforms/on-prem.md +++ b/docs/docs/deployment/platforms/on-prem.md @@ -38,11 +38,11 @@ Most of the popular hypervisors support SGX: #### Driver You need to install the [DCAP SGX Driver](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf). -Azure provides instructions on [how](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) to install this driver](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) that you can use for your on-premises machines. +Azure provides instructions on [how to install this driver](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) that you can use for your on-premises machines. ### SGX Data Center Attestation Primitives (DCAP) -DCAP is the new attestation mechanism for SGX, [replacing](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html)[ EPID](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html). +DCAP is the new attestation mechanism for SGX, [replacing EPID](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html). You can find an overview of DCAP in the [official Intel docs](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf). MarbleRun only supports DCAP and requires DCAP libraries installed and configured on your system. From 1f19508f823f3155a7fc0b63a99d1cd033b0a160 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 10 Aug 2023 13:42:24 +0200 Subject: [PATCH 21/25] Apply suggestions --- docs/docs/architecture/coordinator.md | 2 +- docs/docs/architecture/security.md | 24 ++++++++++---------- docs/docs/features/kubernetes-integration.md | 2 +- docs/docs/features/recovery.md | 2 +- docs/docs/getting-started/quickstart.md | 2 +- 5 files changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/docs/architecture/coordinator.md b/docs/docs/architecture/coordinator.md index 39be9edb..16a54f0d 100644 --- a/docs/docs/architecture/coordinator.md +++ b/docs/docs/architecture/coordinator.md @@ -2,7 +2,7 @@ The Coordinator is MarbleRun's control plane. Conceptually, it operates as trusted controller in your confidential deployment. -Therefore, it securely holds the deployment's state, including the [manifest](../features/manifest.md) and the defined [secrets](../features/secrets-management.md). +It securely holds the deployment's state, including the [manifest](../features/manifest.md) and the defined [secrets](../features/secrets-management.md). Based on the manifest, the Coordinator uses [remote attestation](../features/attestation.md) to authenticate the application's enclaves. diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 5ca0255d..57fb7422 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -1,7 +1,7 @@ # Key management and cryptographic primitives MarbleRun protects and isolates your deployments and workloads. To that end, cryptography is the foundation that ensures the confidentiality and integrity of all components. -Evaluating the security and compliance of MarbleRun requires a precise understanding of the cryptographic primitives and keys used. +Evaluating the security of MarbleRun requires a precise understanding of the cryptographic primitives and keys used. The following gives an overview of the architecture and explains the technical details. ## High-level architecture @@ -11,11 +11,11 @@ The [Coordinator](coordinator.md) and the [Marbles](marbles.md) run inside SGX E MarbleRun uses cryptography for the following tasks. * Authentication and authorization -* Establishing a public key infrastructure (PKI) for MarbleRun -* Encrypting network traffic via mutual TLS between enclaves +* Establishment of a public key infrastructure (PKI) for MarbleRun +* Encryption of network traffic via mutual TLS between enclaves * Encrypting persistent state -The following graphic gives an overview of the architecture and the components. +The following diagram gives an overview of the architecture and the components. ![Security architecture](../_media/security_architecture.svg) @@ -42,13 +42,13 @@ The *Root CA Certificate* has no expiry date and lives as long as the MarbleRun Alongside the *Root CA Certificate*, the Coordinator generates an X.509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every manifest update. -When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Mable), the *Intermediate Certificate* will change. -Instances with the new version won't authenticate with instances of the old version and vice versa. +When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Marble), the *Intermediate Certificate* will change. +Marbles instances of the new version won't authenticate with instances of the old version and vice versa. Hence, no data flow is happening between different *SecurityVersions* of your application. However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and ensure it's the same instance you might have interacted with. Applications interacting with the MarbleRun deployment should use the intermediate certificate as CA to make manifest updates observable: -if the manifest changes, connections to the deployment will fail. -The application owner can then review the changes and install the new intermediate certificate to approve them. +If the manifest changes, connections to the deployment will fail. +The application user can then review the changes and install the new intermediate certificate to approve them. If such observability isn't required, the application can use the root certificate as CA. In that case, the application will continue to work even if the manifest changes. @@ -86,9 +86,11 @@ MarbleRun uses the SGX sealing key called `Product key` as its KEK, which is bou In other words, a fresh and benign enclave instance of the same identity can recover that key. Hence, if the Coordinator is restarted on the same CPU, it can obtain the same KEK from the CPU, decrypt the DEK, and recover its state. +![Encrypted state single instance](../_media/enc-state-single.svg) + If the Coordinator is restarted on a different CPU, it won't be able to obtain the same SGX sealing key from the CPU. -Therefore, MarbleRun provides a [recovery feature](../features/recovery.md#recovery). -The manifest allows for specifying a designated Recovery Key. The Recovery Key is a public RSA key. Upon startup, the Coordinator encrypts the DEK with this public key and returns it to the user. +To address this, MarbleRun provides a [recovery feature](../features/recovery.md#recovery). +The manifest allows for specifying a designated Recovery Key. The Recovery Key is a RSA public key. Upon startup, the Coordinator encrypts the DEK with this public key and returns it to the user. In case of a recovery event, the user decrypts the DEK locally and [uploads it to the Coordinator](../workflows/recover-coordinator.md). The Coordinator will decrypt the state with the DEK and proceed with operations. @@ -97,8 +99,6 @@ Every recovery party is defined in the manifest with its own public RSA key. The Coordinator generates a share of the recovery secret for every party and encrypts it with the corresponding RSA key. During a recovery event, every party will upload their share of the secret, which are all XORed together by the Coordinator to receive the combined key for decrypting the DEK. -![Encrypted state single instance](../_media/enc-state-single.svg) - ### Distributed Coordinator diff --git a/docs/docs/features/kubernetes-integration.md b/docs/docs/features/kubernetes-integration.md index 677a5c5d..71d94c53 100644 --- a/docs/docs/features/kubernetes-integration.md +++ b/docs/docs/features/kubernetes-integration.md @@ -7,7 +7,7 @@ MarbleRun optionally injects [tolerations](https://kubernetes.io/docs/concepts/s You can enable auto-injection of the data-plane configuration using Pod labels. ## The `marbletype` label -MarbleRun defines Marbles (i.e., secure enclaves) in the [manifest](../workflows/define-manifest.md). You need to reference Marbles in your Kubernetes resource description as follows using the `marblerun/marbletype` label: +In MarbleRun, Marbles (i.e, secure enclaves) are defined in the [manifest](../workflows/define-manifest.md). You need to reference Marbles in your Kubernetes resource description as follows using the `marblerun/marbletype` label: ```javascript { diff --git a/docs/docs/features/recovery.md b/docs/docs/features/recovery.md index eae449c1..9611694c 100644 --- a/docs/docs/features/recovery.md +++ b/docs/docs/features/recovery.md @@ -6,7 +6,7 @@ If an application should be able to restart without manual intervention, it need The SGX programming model considers a single, local application running on a specific CPU. The application can use the SGX seal key as its root secret. However, this binds the application and its state to the physical machine because seal keys are unique to a single CPU. -The usual SGX programming model isn't suited for virtual environments or distributed applications. +In sum, the usual SGX programming model isn't suited for virtual environments or distributed applications. With MarbleRun, the Coordinator [manages the Marbles' secrets](../features/secrets-management.md), and Marbles obtain them securely on start. Thus, Marbles can be distributed and rescheduled on arbitrary machines. diff --git a/docs/docs/getting-started/quickstart.md b/docs/docs/getting-started/quickstart.md index ba920889..9b6c6bb3 100644 --- a/docs/docs/getting-started/quickstart.md +++ b/docs/docs/getting-started/quickstart.md @@ -2,7 +2,7 @@ The following steps guide you through the process of deploying MarbleRun in your cluster and deploying a sample app. This example assumes that you have successfully [installed MarbleRun](./installation.md), and have access to a Kubernetes cluster. -A working SGX DCAP environment is required for MarbleRun to work. For ease of exploring and testing, we provide a simulation mode with `--simulation` that runs without SGX hardware. +A working SGX DCAP environment is required for MarbleRun. For ease of exploring and testing, we provide a simulation mode with `--simulation` that runs without SGX hardware. Depending on your setup, you may follow the quickstart for SGX-enabled clusters. Alternatively, if your setup doesn't support SGX, you can follow the quickstart in simulation mode by selecting the respective tabs. ## Step 1: Install the control plane onto your cluster From 5e3019d9a4dbb505004e707e7d2b12ff3f648eab Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 10 Aug 2023 18:11:37 +0200 Subject: [PATCH 22/25] Rename path for building-marbles --- docs/docs/{building-enclaves => building-marbles}/ego.md | 0 .../docs/{building-enclaves => building-marbles}/gramine.md | 0 docs/docs/{building-enclaves => building-marbles}/occlum.md | 0 docs/docs/deployment/kubernetes.md | 2 +- docs/docs/deployment/standalone.md | 2 +- docs/docs/features/runtimes.md | 6 +++--- docs/docs/features/transparent-TLS.md | 2 +- docs/docs/workflows/add-service.md | 6 +++--- docs/sidebars.js | 6 +++--- 9 files changed, 12 insertions(+), 12 deletions(-) rename docs/docs/{building-enclaves => building-marbles}/ego.md (100%) rename docs/docs/{building-enclaves => building-marbles}/gramine.md (100%) rename docs/docs/{building-enclaves => building-marbles}/occlum.md (100%) diff --git a/docs/docs/building-enclaves/ego.md b/docs/docs/building-marbles/ego.md similarity index 100% rename from docs/docs/building-enclaves/ego.md rename to docs/docs/building-marbles/ego.md diff --git a/docs/docs/building-enclaves/gramine.md b/docs/docs/building-marbles/gramine.md similarity index 100% rename from docs/docs/building-enclaves/gramine.md rename to docs/docs/building-marbles/gramine.md diff --git a/docs/docs/building-enclaves/occlum.md b/docs/docs/building-marbles/occlum.md similarity index 100% rename from docs/docs/building-enclaves/occlum.md rename to docs/docs/building-marbles/occlum.md diff --git a/docs/docs/deployment/kubernetes.md b/docs/docs/deployment/kubernetes.md index 84ded68e..2b82c82f 100644 --- a/docs/docs/deployment/kubernetes.md +++ b/docs/docs/deployment/kubernetes.md @@ -81,7 +81,7 @@ Intel SGX supports two modes for obtaining remote attestation quotes: * In-process: The software generating the quote is part of the enclave application * Out-of-process: The software generating the quote isn't part of the actual enclave application. This requires the Intel SGX Architectural Enclave Service Manager (AESM) to run on the system -While Marbles built with [Ego](../building-enclaves/ego.md) perform in-process attestation, other frameworks, such as [Gramine](../building-enclaves/gramine.md), use out-of-process attestation. +While Marbles built with [Ego](../building-marbles/ego.md) perform in-process attestation, other frameworks, such as [Gramine](../building-marbles/gramine.md), use out-of-process attestation. If your confidential application uses out-of-process attestation, you will need to expose the AESM device to your container. You can follow [the AKS guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-addon) to make your deployments able to use AESM for quote generation. Note, that in this case, your Kubernetes nodes need the AESM service installed. See the [Intel installation guide](https://download.01.org/intel-sgx/sgx-linux/2.12/docs/Intel_SGX_Installation_Guide_Linux_2.12_Open_Source.pdf) for more information. diff --git a/docs/docs/deployment/standalone.md b/docs/docs/deployment/standalone.md index 23f15576..25725b81 100644 --- a/docs/docs/deployment/standalone.md +++ b/docs/docs/deployment/standalone.md @@ -38,7 +38,7 @@ See the [how to add a service](../workflows/add-service.md) documentation for mo ### Run your workloads You first need to build your workloads together with our Marble data plane. -See our guides for building [EGo](../building-enclaves/ego.md), [Gramine](../building-enclaves/gramine.md), and [Occlum](../building-enclaves/occlum.md) workloads. +See our guides for building [EGo](../building-marbles/ego.md), [Gramine](../building-marbles/gramine.md), and [Occlum](../building-marbles/occlum.md) workloads. You can then run your Marble as follows: diff --git a/docs/docs/features/runtimes.md b/docs/docs/features/runtimes.md index c30e174d..1fe7b0ca 100644 --- a/docs/docs/features/runtimes.md +++ b/docs/docs/features/runtimes.md @@ -3,7 +3,7 @@ MarbleRun strives to be runtime-agnostic. Currently, supported runtimes are described below. More will follow in the future. ## EGo -[EGo](https://github.com/edgelesssys/ego) is the preferred way for writing confidential Go applications from scratch as well as porting existing ones. Usage is similar to conventional Go programming. Start [building your service with EGo](../building-enclaves/ego.md) to use it with MarbleRun. +[EGo](https://github.com/edgelesssys/ego) is the preferred way for writing confidential Go applications from scratch as well as porting existing ones. Usage is similar to conventional Go programming. Start [building your service with EGo](../building-marbles/ego.md) to use it with MarbleRun. ## Edgeless RT With [Edgeless RT](https://github.com/edgelesssys/edgelessrt), you can create confidential C++ applications with a low TCB. Please follow the build instructions provided [in the C++ sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) to use it with MarbleRun. @@ -12,9 +12,9 @@ With [Edgeless RT](https://github.com/edgelesssys/edgelessrt), you can create co [Gramine](https://gramineproject.io/) is a popular choice for wrapping unmodified applications into enclaves. This approach, commonly known as "lift and shift," facilitates the process of bringing existing applications into the confidential space. Gramine further adds support for dynamically linked libraries and multi-process applications in SGX. -[Running a Gramine app](../building-enclaves/gramine.md) with MarbleRun requires minor changes to its manifest. +[Running a Gramine app](../building-marbles/gramine.md) with MarbleRun requires minor changes to its manifest. ## Occlum [Occlum](https://github.com/occlum/occlum) is another popular solution that allows wrapping existing applications with minimal to no changes inside an enclave. It requires you to, at best, recompile existing applications with the provided toolchains with support for common languages such as C, C++, Go, Java, and Rust. Its core is written in the memory-safe programming language Rust, and it provides a separated environment under which your application is running. This provides a safe yet powerful way to build your applications. -[Running an Occlum app](../building-enclaves/occlum.md) with MarbleRun requires minor changes to its manifest. +[Running an Occlum app](../building-marbles/occlum.md) with MarbleRun requires minor changes to its manifest. diff --git a/docs/docs/features/transparent-TLS.md b/docs/docs/features/transparent-TLS.md index 92f6de3f..0ed9e71e 100644 --- a/docs/docs/features/transparent-TLS.md +++ b/docs/docs/features/transparent-TLS.md @@ -4,7 +4,7 @@ Authenticated and encrypted connections between services are essential for the s Transparent TLS (TTLS) can wrap any connection in TLS on the MarbleRun layer. MarbleRun adds secure communication to your cluster even if your application doesn't support the required TLS features. Just define the desired [connections in the manifest](../workflows/define-manifest.md#tls). -TTLS is currently available with [EGo Marbles](../building-enclaves/ego.md). Other [runtimes](../features/runtimes.md) will be supported in future. +TTLS is currently available with [EGo Marbles](../building-marbles/ego.md). Other [runtimes](../features/runtimes.md) will be supported in future. ## Authentication and credentials By default the Marble's credentials are automatically configured. Connections between two Marbles are mutually authenticated. diff --git a/docs/docs/workflows/add-service.md b/docs/docs/workflows/add-service.md index 8a9dd5fa..79c0e2d0 100644 --- a/docs/docs/workflows/add-service.md +++ b/docs/docs/workflows/add-service.md @@ -5,10 +5,10 @@ Adding a service to your application requires three steps, which are described i ## **Step 1:** Get your service ready for MarbleRun To get your service ready for MarbleRun, you need to rebuild it with one of the supported [runtimes](../features/runtimes.md): -* [EGo](../building-enclaves/ego.md) +* [EGo](../building-marbles/ego.md) * [Edgeless RT](https://github.com/edgelesssys/marblerun/blob/master/samples/helloc%2B%2B) -* [Gramine](../building-enclaves/gramine.md) -* [Occlum](../building-enclaves/occlum.md) +* [Gramine](../building-marbles/gramine.md) +* [Occlum](../building-marbles/occlum.md) ### Make your service use the provided TLS credentials diff --git a/docs/sidebars.js b/docs/sidebars.js index 29cbc2cd..6d0e53a4 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -224,17 +224,17 @@ const sidebars = { { type: 'doc', label: 'EGo', - id: 'building-enclaves/ego', + id: 'building-marbles/ego', }, { type: 'doc', label: 'Gramine', - id: 'building-enclaves/gramine', + id: 'building-marbles/gramine', }, { type: 'doc', label: 'Occlum', - id: 'building-enclaves/occlum', + id: 'building-marbles/occlum', }, ], }, From db0e36a0889887dfeb8a77e39bcb49e43165afe9 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 10 Aug 2023 18:13:42 +0200 Subject: [PATCH 23/25] Remove acronym CC --- docs/docs/architecture/security.md | 2 +- docs/docs/deployment/kubernetes.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 57fb7422..4c7e438e 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -68,7 +68,7 @@ Depending on the Marble's runtime, the certificate can be used [manually](../wor ## Attested TLS (aTLS) -In a confidential computing (CC) environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components. +In a confidential computing environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the confidential computing components. With aTLS, the party to be authenticated binds its TLS certificate to an attestation statement. For example, it embeds the certificate's public key into the attestation statement. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate. diff --git a/docs/docs/deployment/kubernetes.md b/docs/docs/deployment/kubernetes.md index 2b82c82f..7c9660d0 100644 --- a/docs/docs/deployment/kubernetes.md +++ b/docs/docs/deployment/kubernetes.md @@ -19,7 +19,7 @@ The SGX device plugin can either be deployed manually or as a DaemonSet in the c :::info -If you are using a CC-enlightened, managed Kubernetes cluster, you will usually already have an SGX device plugin installed. +If you are using a confidential computing-enlightened, managed Kubernetes cluster, you will usually already have an SGX device plugin installed. For example, creating a confidential computing cluster on AKS has a pre-configured SGX device plugin. ::: From a9b63ecc5ac99213ca5f53740c389eb07f92e074 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Thu, 10 Aug 2023 18:18:57 +0200 Subject: [PATCH 24/25] fix single instance graphic --- docs/docs/_media/enc-state-single.svg | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/docs/_media/enc-state-single.svg b/docs/docs/_media/enc-state-single.svg index 6dbf8883..50df40aa 100644 --- a/docs/docs/_media/enc-state-single.svg +++ b/docs/docs/_media/enc-state-single.svg @@ -1,4 +1,3 @@ - -
Persistent Storage
Persistent Storage
Encrypted State
Encrypted State
Encrypted DEK
Encrypted DEK
Coordinator State
Coordinator State
Data Encryption Key
Data Encryption K...
Key Encryption Key
Key Encryption Key
Generated by the Coordinator
Generated by the Coordi...
SGX 'Product' Sealing Key
SGX 'Product' Sealing K...Text is not SVG - cannot display \ No newline at end of file +
Persistent Storage
Persistent Storage
Encrypted State
Encrypted State
Encrypted DEK
Encrypted DEK
Coordinator State
Coordinator State
Data Encryption Key
Data Encryption K...
Key Encryption Key
Key Encryption Key
Generated by the Coordinator
Generated by the Coordi...
SGX 'Product' Sealing Key
SGX 'Product' Sealing K...Text is not SVG - cannot display \ No newline at end of file From 5ddd0692be0dde6b592efc5b2fb687e0c0b496e9 Mon Sep 17 00:00:00 2001 From: Moritz Eckert Date: Fri, 11 Aug 2023 09:57:18 +0200 Subject: [PATCH 25/25] Apply suggestions from code review Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com> --- docs/docs/architecture/security.md | 2 +- docs/docs/features/kubernetes-integration.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/architecture/security.md b/docs/docs/architecture/security.md index 4c7e438e..214f6fa6 100644 --- a/docs/docs/architecture/security.md +++ b/docs/docs/architecture/security.md @@ -43,7 +43,7 @@ The *Root CA Certificate* has no expiry date and lives as long as the MarbleRun Alongside the *Root CA Certificate*, the Coordinator generates an X.509 *Intermediate Certificate* and corresponding asymmetric key pair, again using ECDSA with P256. The *Intermediate Certificate* is signed by the Coordinator's *Root CA Certificate* and rotated with every manifest update. When you push an update to the manifest (for example, bump up the *SecurityVersion* of a Marble), the *Intermediate Certificate* will change. -Marbles instances of the new version won't authenticate with instances of the old version and vice versa. +Marble instances of the new version won't authenticate with instances of the old version and vice versa. Hence, no data flow is happening between different *SecurityVersions* of your application. However, the *Root CA Certificate* doesn't change. So you can still verify the Coordinator and your application from the outside and ensure it's the same instance you might have interacted with. Applications interacting with the MarbleRun deployment should use the intermediate certificate as CA to make manifest updates observable: diff --git a/docs/docs/features/kubernetes-integration.md b/docs/docs/features/kubernetes-integration.md index 71d94c53..4a52db4a 100644 --- a/docs/docs/features/kubernetes-integration.md +++ b/docs/docs/features/kubernetes-integration.md @@ -7,7 +7,7 @@ MarbleRun optionally injects [tolerations](https://kubernetes.io/docs/concepts/s You can enable auto-injection of the data-plane configuration using Pod labels. ## The `marbletype` label -In MarbleRun, Marbles (i.e, secure enclaves) are defined in the [manifest](../workflows/define-manifest.md). You need to reference Marbles in your Kubernetes resource description as follows using the `marblerun/marbletype` label: +In MarbleRun, Marbles (i.e., secure enclaves) are defined in the [manifest](../workflows/define-manifest.md). You need to reference Marbles in your Kubernetes resource description as follows using the `marblerun/marbletype` label: ```javascript {