Add projects/backend

packages/backend/README.md

@@ -0,0 +1,58 @@
+# AI Questionaire Backend
+
+The AI questionaire backend is an iExec application responsible for taking the input from the user (currently as a textual prompt in the "email" field of the [protectedData](https://tools.docs.iex.ec/tools/dataProtector/dataProtectorCore/processProtectedData), due to historical reasons, passing it to the AI model (currently, a tiny version of [Qwen2.5](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF) embedded into the image itself), and finally returning it as a result using iExec's result mechanism.
+
+Unfortunatelly, despite all our attempts to get that working with iExec, the only thing we could go working is a mock node.js version, which can be found in [`./node-mocked`](,/node-mocked). Everything else failed in one way or another, and this part of the repository is here to document the various ways in which we couldn't get it to work.
+
+## List of attempts
+
+| # | Attempt name | Description | Status |
+| 1 | [`python-sconify`](./python-sconify) | An attempt to follow iExec's [Build your first application with Scone framework](https://protocol.docs.iex.ec/for-developers/confidential-computing/intel-sgx-technology) documentation. | :x: DCAP validation fail on both Prod and Debug workerpools. |
+| 2 | [`python-transformers`](./python-sconify) | An attempt to use the Huggingface [`transformers`](https://huggingface.co/docs/transformers/index) library for the AI model. | :o: Abandonded as we experiment with simpler Python SGX examples first |
+| 3 | [`python-gramine`](./python-gramine/) | Attempt to use Gramine instead of Scone following the iExec's unlisted [Build a Gramine application](https://github.com/iExecBlockchainComputing/documentation/blob/v8-staging/for-developers/confidential-computing/create-your-first-gramine-app.md) documentation. | :x: iExec does not support Gramine at the moment |
+| 4 | [`node-mocked`](./node-mocked) | The default hello World" app generated by [`idapp-cli`](https://www.npmjs.com/package/idapp-cli) | :check: Deployed as **`0x05F88328fAe2Ac1271C68f2E65864692c3AD9B0A`** |
+| 5 | [`nodejs-transformers.js`](./nodejs-transformers.js/) | An attempt to use the [transformers.js](https://huggingface.co/docs/transformers.js/index) library by Huggingface. | :x: SCONE's Node.js version (14) is too old to support transformers.js. |
+| 6 | [`nodejs-execFile`](./nodejs-execFile/) | An attempt to use the [`llamafile`](https://github.com/Mozilla-Ocho/llamafile) executable with the Node version we have | :x: SCONE limitations: ENOSYS when attempting to spawn the process. |
+| 7 | `nodejs-wllama` | An attempt to use the [`@wllama/wllama`](https://huggingface.co/spaces/ngxson/wllama) library (also based on `llama.cpp`) for inference. | :x: SCONE's Node.js version (14) doesn't support the [WebAssembly exception handling extension](https://webassembly.github.io/exception-handling/js-api/) wllama requires. |
+| 8 | [`python-scone-py`](./python-scone-py/) | An attempt to use the official Scone build image, as suggested by [Scone's documentation](https://sconedocs.github.io/binary_fs/). | :x: DCAP validation fail when running on Prod; maybe mrenclave mismatch on Debug. |
+| 9 | [`python-sconify-debug`](./python-sconify-debug)  | An attempt to pass `--allow-debug-mode` to `sconify_iexec` (after staring at code for way too long), with no observable result. | :x: DCAP validation fail on both prod and debug worker pools. |
+| 10 | [`nodejs-llama.node`](./nodejs-llama.node/) | An attempt to use yet another library for invoking `llama.cpp`, [`@fugood/llama.node`](https://github.com/mybigday/llama.node) | :o: Ran out of time to debug properly, needs an extra library and testing. Could conceivably be made work with `sconify_iexec ... --dlopen=2`, tho that's insecure |
+
+Note that if you want to try all of those for yourself, you would need an account on SCONE's registry whitelisted for use with iExec, as described in [Build your first application with Scone framework](https://protocol.docs.iex.ec/for-developers/confidential-computing/intel-sgx-technology), and might need to deploy/publish/run the app using commands like as described in [1](https://protocol.docs.iex.ec/for-developers/your-first-app), [2](https://protocol.docs.iex.ec/for-developers/confidential-computing/intel-sgx-technology),:
+```
+iexe init && iexec app init && iexec order init # To generate iexec.json
+
+# With any of the non-node examples:
+
+#update iexec.json to include the pushed image in the app section
+iexec app deploy
+
+# With any of the node examples:
+#update deployed.json to include appid
+#update iexec.json to include appid in the order section
+iexec order sign --app && iexec order publish --app
+
+# To finally run:
+iexec app run --tag tee,scone --workerpool debug-v8-bellecour.main.pools.iexec.eth --watch
+```
+
+NOTE: The code in this folder was scraped off a messy attempts folder on a best-effort basis, and as such comes with NO WARRANTY of any kind, including FITNESS for any particular purpose, including reproducing specific bugs.
+
+## General observations
+
+In general, all of the attempts involved trying to run either Python or Node.js.
+
+All the Python attempts ended up hitting the same problem related to DCAP validation, which had something to do with having a SCONE policy not allowing debug-mode SGX enclaves. Since the built/sconified image doesn't _seem_ to contain a policy at all, and the error is the same on both the Prod and Debug pools, and the Node.js image being perfectly happy to run in SGX debug mode, we are not at all sure what's going on.
+
+Meanwhile, all of the Nodejs attempts used the [idapp-cli API server](https://github.com/iExecBlockchainComputing/idapp/tree/main/api) provided and hosted by iExec, since it seemed to be successful in building at least the mock Node.js app. However, as SCONE doesn't have prebuilt images for Node >14, all the newer libraries ended up throwing errors related to newer JavaScript feature, like `??=`, or to missing runtime APIs, like `Blob` or parts of WebAssembly.
+
+Overall, would say that iExec's reliance on SCONE ended up frustrating virtually all of our attempts to run an AI model inside iExec's platform. Unfortunately, Gramine support had been axed a while ago, so we couldn't try using its virtualization model to run our applications. Better yet, actual Intel TDX/AMD SEV-SNP support, such as that afforded by e.g. [Constellation](https://github.com/edgelesssys/constellation/), (WIP) [CoCo](https://github.com/confidential-containers), or (shameless plug, WIP) [Apocryph](https://github.com/comrade-coop/apocryph), would have meant that we would have been able to run any code, without needing finicky conversions to get it inside Intel SGX.
+
+## Future work
+
+A few more things we could have tried:
+* Get `node.llama` working all the way.
+* Run a C++ project that includes `llama.cpp` inside SCONE, as SCONE seems better suited to running compiled languages than interpreted languages.
+* Investigate if there is something else we could have tried for `python-sconified` -- including different versions of the sconification software that might have worked in the past.
+
+

packages/backend/build-common.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+IMG_BASE=bojidarbg/idap-hw
+IEXEC_WALLET_ADDRESS=0x23618e81E3f5cdF7f54C3d65f7FBc0aBf5B21E8f

packages/backend/node-execFile/.nvmrc

@@ -0,0 +1 @@
+14

packages/backend/node-execFile/Dockerfile

@@ -0,0 +1,8 @@
+FROM node:14-alpine3.11
+WORKDIR /app
+RUN curl -L https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-q4_k_m.gguf -o qwen2.5-0.5b-instruct-q4_k_m.gguf
+RUN curl -L -o llamafile https://github.com/Mozilla-Ocho/llamafile/releases/download/0.8.11/llamafile-0.8.11
+COPY package*.json ./
+RUN npm ci
+COPY src .
+ENTRYPOINT ["node", "/app/app.js"]

packages/backend/node-execFile/build.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+
+cd "`dirname $0`"
+. ../build-common.sh
+
+set -ex
+
+VERSION=`basename $(pwd)`-0.1.0
+
+docker build . -t $IMG_BASE:$VERSION
+docker push $IMG_BASE:$VERSION
+curl -X POST http://idapp-poc.westeurope.cloudapp.azure.com:3000/sconify --json '{"dockerhubImageToSconify": "'$IMG_BASE:$VERSION'", "yourWalletPublicAddress": "'$IEXEC_WALLET_ADDRESS'"}' | tee sconify-result.json

packages/backend/node-execFile/package.json

@@ -0,0 +1,15 @@
+{
+  "name": "ethsofia2024-ai",
+  "version": "1.0.0",
+  "description": "",
+  "main": "src/app.js",
+  "scripts": {
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "dependencies": {
+    "@iexec/dataprotector-deserializer": "^0.1.0"
+  }
+}

packages/backend/node-execFile/src/app.js

@@ -0,0 +1,57 @@
+const fsPromises = require('fs').promises;
+const path = require('path');
+const util = require('util');
+const execFile = util.promisify(require('child_process').execFile);
+const {
+  IExecDataProtectorDeserializer,
+} = require('@iexec/dataprotector-deserializer');
+
+const model = path.join(__dirname, 'qwen2.5-0.5b-instruct-q4_k_m.gguf')
+const llamaexec = path.join(__dirname, 'llamafile')
+const llamaparams = [
+  '-m', model,
+  '--cli',
+  '-n', '100',
+  '-p',
+]
+
+const main = async () => {
+  try {
+    const output = process.env.IEXEC_OUT;
+    const message =
+      process.argv.length > 2 && process.argv[2] !== 'undefined'
+        ? process.argv[2]
+        : 'World';
+
+    console.log('Args', process.argv.slice(2));
+
+    let file;
+    let prompt = process.argv.slice(2).join(' ') || 'Hello hello!';
+    try {
+      const deserializer = new IExecDataProtectorDeserializer();
+      file = await deserializer.getValue('email', 'string');
+    } catch (e) {
+      file = 'missing protectedData';
+      console.log('It seems there is an issue with your protectedData :', e);
+    }
+
+    const result = await execFile(llamaexec, llamaparams.concat([prompt]))
+
+    // Append some results in /iexec_out/
+    const combinedContent = `Your ProtectedData content: ${file}\n You AI fortune: ${result.stdout}`;
+    await fsPromises.writeFile(`${output}/result.txt`, combinedContent);
+    // Declare everything is computed
+    const computedJsonObj = {
+      'deterministic-output-path': `${output}/result.txt`,
+    };
+    await fsPromises.writeFile(
+      `${output}/computed.json`,
+      JSON.stringify(computedJsonObj)
+    );
+  } catch (e) {
+    console.log(e);
+    process.exit(1);
+  }
+};
+
+main();

packages/backend/node-llama.node/.nvmrc

@@ -0,0 +1 @@
+14

packages/backend/node-llama.node/Dockerfile

@@ -0,0 +1,7 @@
+FROM node:14-alpine3.11
+WORKDIR /app
+RUN curl -L https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-q4_k_m.gguf -o qwen2.5-0.5b-instruct-q4_k_m.gguf
+COPY package*.json ./
+RUN npm ci
+COPY src .
+ENTRYPOINT ["node", "/app/app.js"]

packages/backend/node-llama.node/build.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+cd "`dirname $0`"
+. ../build-common.sh
+
+set -ex
+
+VERSION=`basename $(pwd)`-0.5.2
+
+docker build . -t $IMG_BASE:$VERSION
+docker push $IMG_BASE:$VERSION
+curl -X POST http://idapp-poc.westeurope.cloudapp.azure.com:3000/sconify --json '{"dockerhubImageToSconify": "'$IMG_BASE:$VERSION'", "yourWalletPublicAddress": "'$IEXEC_WALLET_ADDRESS'"}'
+