Merge pull request #132 from dscc-admin-ch/doc-white-paper

first draft, feel free to improve

.github/workflows/check_security_codeQL.yml

@@ -1,4 +1,4 @@
-name: "CodeQL"
+name: "CodeQL Extended"
 
 on:
   pull_request:

.github/workflows/test_and_coverage_server.yml

@@ -1,4 +1,4 @@
-name: Server Integration Testing
+name: Server Tests
 
 on: 
   pull_request:
@@ -38,4 +38,6 @@ jobs:
       uses: py-cov-action/python-coverage-comment-action@v3
       with:
         GITHUB_TOKEN: ${{ secrets.COVERAGE_REPORT }}
-        COVERAGE_PATH: server/lomas_server/
+        COVERAGE_PATH: server/lomas_server/
+        MINIMUM_GREEN: 95
+        MINIMUM_ORANGE: 75

README.md

@@ -5,7 +5,10 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Python](https://img.shields.io/badge/python-3.11-blue)](https://www.python.org/)
 [![Documentation](https://img.shields.io/badge/docs-Read%20the%20Docs-blue)](https://dscc-admin-ch.github.io/lomas-docs/index.html)
+[![Tests](https://github.com/dscc-admin-ch/lomas/actions/workflows/test_and_coverage_server.yml/badge.svg)](https://github.com/dscc-admin-ch/lomas/actions/workflows/test_and_coverage_server.yml)
 [![Coverage badge](https://raw.githubusercontent.com/dscc-admin-ch/lomas/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://raw.githubusercontent.com/dscc-admin-ch/lomas/python-coverage-comment-action-data/htmlcov/index.html)
+[![CodeQL](https://github.com/dscc-admin-ch/lomas/actions/workflows/check_security_codeQL.yml/badge.svg)](https://github.com/dscc-admin-ch/lomas/actions/workflows/check_security_codeQL.yml)
+[![PyPi version](https://img.shields.io/pypi/v/lomas_client.svg)](https://pypi.org/project/lomas_client/)
 
 
 
@@ -14,19 +17,23 @@
 
 Lomas is a platform for remote data science, enabling sensitive data to be queried remotely while staying protected by a layer of differential privacy.
 
+#### Technical Overview:
+
 The lomas platform follows a classic server/client model.
 On the client side, the user prepares queries for statistical analyses which are sent to the service's REST API via HTTP. The user never has direct access to the sensitive data.
 On the server side, the service is implemented in a micro-service architecture and is thus split into two parts: the administration database and the client-facing HTTP server (which we call server for brevity) that implements the service logic.
 The server is responsible for processing the client requests and updating its own state as well as administrative data (users data, budgets, query archives, etc.) in the administration database.
 
-
 The service is not responsible for storing and managing private datasets, these are usually already stored on the provider's infrastructure.
 
-<!-- See our white paper (TODO link) for detailed explanation of the platform. -->
-## Useful links
+#### Detailed description:
+
+For a detailed description, please see the links below.
+
+* **Lomas Project White Paper**: https://arxiv.org/abs/2406.17087
+* **Swiss Federal Statistical Office Blog**: https://www.bfs.admin.ch/bfs/en/home/dscc/blog/2024-03-lomas.html
+* **Technical Documentation**: https://dscc-admin-ch.github.io/lomas-docs/index.html
 
-* **Documentation**: https://dscc-admin-ch.github.io/lomas-docs/index.html
-* **Bug report**: https://github.com/dscc-admin-ch/lomas/issues
 
 ## Client package `lomas_client`
 
@@ -61,6 +68,12 @@ For extensive informations about how to deploy, please refer to:
 Finally, the service provider is responsible for deploying the service and managing users and private datasets by adding, modifying or deleting information in the administration database.
 It is important to note that the service is not responsible for storing and managing private datasets, these are usually already stored on the provider's infrastructure.
 
+## Disclaimer
+Lomas is a Proof of Concept that is still under development. 
+
+The overall infrastructure security is not our current priority.  While attention has been given to the 'logical' aspects within the server, many security aspects are not handled. For example, user authentication is not implemented. However, Lomas can be integrated into other secure infrastructures.
+
+We welcome any feedback or suggestions for future improvements. External input is valuable as we continue to enhance the security and functionality of Lomas. Please open a bug report or issue here: https://github.com/dscc-admin-ch/lomas/issues.open.
 
 ## History
 The starting point of our platform was the code shared to us by [Oblivious](https://www.oblivious.com/). They originally developed a client/server platform for the [UN PET Lab Hackathon 2022](https://petlab.officialstatistics.org/).

client/README.md

@@ -1,3 +1,10 @@
+[![PyPi version](https://img.shields.io/pypi/v/lomas_client.svg)](https://pypi.org/project/lomas_client/)
+[![PyPi status](https://img.shields.io/pypi/status/lomas_client.svg)](https://pypi.org/project/lomas_client/)
+
+<h1 align="center">
+<img src="https://github.com/dscc-admin-ch/lomas/blob/develop/images/lomas_logo_txt.png?raw=true" width="300">
+</h1><br>
+
 # Lomas Client
 
 The `lomas_client` library is a client to interact with the Lomas server.

client/setup.py

@@ -19,13 +19,26 @@
     author_email="dscc@bfs.admin.ch",
     license="MIT",
     classifiers=[
-        "Development Status :: 3 - Alpha",
+        "Development Status :: 4 - Beta",
         "Intended Audience :: Developers",
+        "Intended Audience :: Science/Research",
         "License :: OSI Approved :: MIT License",
-        "Programming Language :: Python :: 3.8",
-        "Programming Language :: Python :: 3.9",
+        "Natural Language :: English",
+        "Programming Language :: Python :: 3.11",
+        "Topic :: Software Development :: Libraries",
+        "Topic :: Software Development :: Libraries :: Python Modules",
+        "Topic :: Scientific/Engineering",
+        "Topic :: Security",
+    ],
+    keywords=[
+        "differential privacy",
+        "DP",
+        "diffprivlib",
+        "logger",
+        "opendp",
+        "privacy",
+        "smartnoise-sql",
     ],
-    keywords=["opendp", "smartnoise-sql", "logger", "ast"],
     python_requires=">=3.10, <4",
     install_requires=[
         "diffprivlib>=0.6.4",

docs/source/index.rst

@@ -5,26 +5,25 @@
 
 Lomas: The Data Oases Hidden Behind the Mist.
 ========================================
-Lomas is a platform for remote data science, enabling sensitive data to be queried remotely 
-while staying protected by a layer of differential privacy.
+Lomas is a platform for remote data science, enabling sensitive data to be queried remotely while staying protected by a layer of differential privacy.
 
-The lomas platform follows a classic server/client model. On the client side, the 
-user prepares queries for statistical analyses which are sent to the 
-service's REST API via HTTP. The user never has direct access to the sensitive 
-data. On the server side, the service is implemented in a micro-service architecture and is 
-thus split into two parts: the administration database and the client-facing HTTP server 
-(which we call server for brevity) that implements the service logic. The server 
-is responsible for processing the client requests and updating its own state as 
-well as administrative data (users data, budgets, query archives, etc.) in 
-the administration database.
+#### Technical Overview:
 
-The service is not responsible for storing and managing private datasets, 
-these are usually already stored on the provider's infrastructure. 
+The lomas platform follows a classic server/client model.
+On the client side, the user prepares queries for statistical analyses which are sent to the service's REST API via HTTP. The user never has direct access to the sensitive data.
+On the server side, the service is implemented in a micro-service architecture and is thus split into two parts: the administration database and the client-facing HTTP server (which we call server for brevity) that implements the service logic.
+The server is responsible for processing the client requests and updating its own state as well as administrative data (users data, budgets, query archives, etc.) in the administration database.
 
-.. See our white paper (TODO link) for detailed explanation of the platform.
+The service is not responsible for storing and managing private datasets, these are usually already stored on the provider's infrastructure.
+
+#### Detailed description:
+
+For a detailed description, please see the links below.
+
+* **Lomas Project White Paper**: https://arxiv.org/abs/2406.17087
+* **Swiss Federal Statistical Office Blog**: https://www.bfs.admin.ch/bfs/en/home/dscc/blog/2024-03-lomas.html
+* **GitHub repository**: https://github.com/dscc-admin-ch/lomas/tree/master/
 
-You can find our `GitHub repository <https://github.com/dscc-admin-ch/lomas/tree/master/>`_
-following this link.
 
 Client
 ========
@@ -70,6 +69,13 @@ MongoDB database) and a client development environment in a Kubernetes cluster.
 For extensive informations about how to deploy, please refer to :doc:`Deployment <server_deployment>`
 documentation.
 
+## Disclaimer
+Lomas is a Proof of Concept that is still under development. 
+
+The overall infrastructure security is not our current priority.  While attention has been given to the 'logical' aspects within the server, many security aspects are not handled. For example, user authentication is not implemented. However, Lomas can be integrated into other secure infrastructures.
+
+We welcome any feedback or suggestions for future improvements. External input is valuable as we continue to enhance the security and functionality of Lomas. Please open a bug report or issue here: https://github.com/dscc-admin-ch/lomas/issues.open.
+
 History
 ========
 The starting point of our platform was the code shared to us by `Oblivious <https://www.oblivious.com/>`_.

server/README.md

@@ -1 +1,9 @@
+[![Tests](https://github.com/dscc-admin-ch/lomas/actions/workflows/test_and_coverage_server.yml/badge.svg)](https://github.com/dscc-admin-ch/lomas/actions/workflows/test_and_coverage_server.yml)
+[![Coverage badge](https://raw.githubusercontent.com/dscc-admin-ch/lomas/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://raw.githubusercontent.com/dscc-admin-ch/lomas/python-coverage-comment-action-data/htmlcov/index.html)
+
+<h1 align="center">
+<img src="https://github.com/dscc-admin-ch/lomas/blob/develop/images/lomas_logo_txt.png?raw=true" width="300">
+</h1><br>
+
 # Server
+See the technical documentation of the server: https://dscc-admin-ch.github.io/lomas-docs/server_api.html.