Merge pull request #447 from C-Accel-CRIPT/release-v2.3.0

Release v2.3.0

.github/workflows/docs_check.yaml

@@ -15,6 +15,13 @@ on:
  - main
  - develop
  - "*"
+concurrency:
+ # github.workflow: name of the workflow
+ # github.event.pull_request.number || github.ref: pull request number or branch name if not a pull request
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+
+ # Cancel in-progress runs when a new workflow with the same group name is triggered
+ cancel-in-progress: true
 
 jobs:
  build:

.github/workflows/doctest.yaml

@@ -14,6 +14,14 @@ on:
  - main
  - develop
 
+concurrency:
+ # github.workflow: name of the workflow
+ # github.event.pull_request.number || github.ref: pull request number or branch name if not a pull request
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+
+ # Cancel in-progress runs when a new workflow with the same group name is triggered
+ cancel-in-progress: true
+
 jobs:
  doctest:
  strategy:

.github/workflows/mypy.yaml

@@ -14,6 +14,14 @@ on:
  - main
  - develop
 
+concurrency:
+ # github.workflow: name of the workflow
+ # github.event.pull_request.number || github.ref: pull request number or branch name if not a pull request
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+
+ # Cancel in-progress runs when a new workflow with the same group name is triggered
+ cancel-in-progress: true
+
 jobs:
  mypy-test:
  strategy:

.github/workflows/test_coverage.yaml

@@ -15,6 +15,14 @@ on:
  - main
  - develop
 
+concurrency:
+ # github.workflow: name of the workflow
+ # github.event.pull_request.number || github.ref: pull request number or branch name if not a pull request
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+
+ # Cancel in-progress runs when a new workflow with the same group name is triggered
+ cancel-in-progress: true
+
 jobs:
  test-coverage:
  runs-on: ubuntu-latest

.github/workflows/test_examples.yml

@@ -12,6 +12,14 @@ on:
  - main
  - develop
 
+concurrency:
+ # github.workflow: name of the workflow
+ # github.event.pull_request.number || github.ref: pull request number or branch name if not a pull request
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+
+ # Cancel in-progress runs when a new workflow with the same group name is triggered
+ cancel-in-progress: true
+
 jobs:
  test-examples:
  runs-on: ${{ matrix.os }}

.github/workflows/tests.yml

@@ -18,6 +18,14 @@ on:
  - develop
  - "*"
 
+concurrency:
+ # github.workflow: name of the workflow
+ # github.event.pull_request.number || github.ref: pull request number or branch name if not a pull request
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+
+ # Cancel in-progress runs when a new workflow with the same group name is triggered
+ cancel-in-progress: true
+
 jobs:
  install:
  runs-on: ${{ matrix.os }}

.trunk/configs/.cspell.json

@@ -121,6 +121,8 @@
  "doctest",
  "Doctest",
  "Doctests",
- "linenums"
+ "linenums",
+ "XLYOFNOQVPJJNP",
+ "CRIPTUUID"
  ]
 }

CHANGELOG.md

@@ -0,0 +1,105 @@
+# CRIPT Python SDK Changelog
+
+## Version 2.3.0
+
+**New Features:**
+
+- `cript.API` objects now have a `DataSchema` attribute called `schema`, representing the JSON schema for node validation.
+ - This includes the ability to enable and disable node validation.
+- `cript.API` objects now have `logger` attributes from the Python `logging` module to control logging flows.
+- Refactor of Paginator:
+ - New Paginator objects for accessing data from the CRIPT data bank via search.
+ - Paginator are now Python iterators. You can use `for node in paginator: ...` directly.
+ - Paginator now return `cript.Node` objects natively.
+ - JSON response can be requested from Paginator alternatively. This is helpful for internal work and debugging.
+- Additional information is added to the logging message when nodes are validated. For primary nodes, `name` is displayed; for others, the `UUID` is displayed.
+- HTTP requests are now optimized, leaving connections open and grouping requests for better performance.
+- A native iterator for nodes is now offered. It iterates in depth-first order over all child nodes of the root node. Cycles are automatically broken, and every node is visited exactly once.
+- Materials with no identifier issue a warning to users.
+- Debugging messages show full API requests and responses for debugging.
+- Automated UUID caching for nodes can now be explicitly circumvented when using `cript.load_nodes_from_json`. Mostly useful for development.
+
+**Known Issues and Bugs:**
+
+- Saving projects is not supported. Temporarily, you can use `get_expanded_json` to store a JSON representation of projects, which can be uploaded into CRIPT at a later time.
+- BigSMILES search patterns are not supported.
+- Searching for more than 1000 pages (10000 entries) is not supported.
+- Permission settings in CRIPT do not influence the behavior of the SDK objects.
+- Tests that require valid tokens (like saving or searching) are not included in CI/CD tests.
+
+**Bugfixes:**
+
+- `cript.load_nodes_from_json` can now load JSON files that store different nodes in lists or dictionaries.
+- Not all nodes were correctly validated at all times, especially if instantiated from JSON. All nodes are automatically validated now.
+- The documentation has been updated to remove certain mistakes.
+- Users can have only one Python object with the same UUID to avoid mis-updates. This did not work in all cases, but it works in all cases now.
+
+**Breaking Changes:**
+
+- `cript.API()` objects no longer have functions related to JSON schema validation. Please use the new `DataSchema` class instead. The `DataSchema` class can be accessed via the `schema` property of the API class.
+- Indirect logging control via the API is defunct. Please use the direct access to the `logger` attribute of API classes to control logging output.
+- The SDK used to use `identifier` dictionaries for material identifiers. This is updated to use individual attributes of the material node. This is consistent with the JSON schema and front-end but is a breaking change to older SDK versions and the original CRIPT publication.
+- Projects were checked to ensure the presence of material and computation in Experiments. These errors were converted into warnings.
+
+### Health Report
+
+```shell
+=============================================================================================== test session starts ===============================================================================================
+platform linux -- Python 3.11.2, pytest-7.4.3
+plugins: cov-4.1.0
+collected 124 items
+
+tests/test_node_util.py ........... [ 8%]
+tests/api/test_api.py ..... [ 12%]
+tests/api/test_db_schema.py ..... [ 16%]
+tests/api/test_search.py .....F [ 21%]
+tests/nodes/test_utils.py .. [ 23%]
+tests/nodes/primary_nodes/test_collection.py .....F [ 28%]
+tests/nodes/primary_nodes/test_computation.py .....F [ 33%]
+tests/nodes/primary_nodes/test_computational_process.py ....F [ 37%]
+tests/nodes/primary_nodes/test_data.py ....F [ 41%]
+tests/nodes/primary_nodes/test_experiment.py ....F [ 45%]
+tests/nodes/primary_nodes/test_inventory.py ..F [ 47%]
+tests/nodes/primary_nodes/test_material.py ...F [ 50%]
+tests/nodes/primary_nodes/test_process.py .....F [ 55%]
+tests/nodes/primary_nodes/test_project.py ...F [ 58%]
+tests/nodes/primary_nodes/test_reference.py ......F [ 64%]
+tests/nodes/subobjects/test_algorithm.py ..F [ 66%]
+tests/nodes/subobjects/test_citation.py ..F [ 69%]
+tests/nodes/subobjects/test_computational_forcefield.py ..F [ 71%]
+tests/nodes/subobjects/test_condition.py ..F [ 74%]
+tests/nodes/subobjects/test_equipment.py ..F [ 76%]
+tests/nodes/subobjects/test_ingredient.py ..F [ 79%]
+tests/nodes/subobjects/test_parameter.py ..F [ 81%]
+tests/nodes/subobjects/test_property.py ..F [ 83%]
+tests/nodes/subobjects/test_quantity.py ..F [ 86%]
+tests/nodes/subobjects/test_software.py ...F [ 89%]
+tests/nodes/subobjects/test_software_configuration.py ..F [ 91%]
+tests/nodes/supporting_nodes/test_file.py ..s....F [ 98%]
+tests/nodes/supporting_nodes/test_user.py .. [100%]
+============================================================================================= short test summary info =============================================================================================
+FAILED tests/api/test_search.py::test_api_search_bigsmiles - cript.nodes.exceptions.CRIPTJsonDeserializationError: JSON deserialization failed for node type Material with JSON str: Material
+FAILED tests/nodes/primary_nodes/test_collection.py::test_integration_collection - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_computation.py::test_integration_computation - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_computational_process.py::test_integration_computational_process - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_data.py::test_integration_data - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_experiment.py::test_integration_experiment - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_inventory.py::test_integration_inventory - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_material.py::test_integration_material - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_process.py::test_integration_complex_process - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_project.py::test_integration_project - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/primary_nodes/test_reference.py::test_integration_reference - IndexError: list index out of range
+FAILED tests/nodes/subobjects/test_algorithm.py::test_integration_algorithm - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_citation.py::test_integration_citation - IndexError: list index out of range
+FAILED tests/nodes/subobjects/test_computational_forcefield.py::test_integration_computational_forcefield - AttributeError: 'NoneType' object has no attribute 'description'
+FAILED tests/nodes/subobjects/test_condition.py::test_integration_process_condition - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_equipment.py::test_integration_equipment - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_ingredient.py::test_integration_ingredient - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_parameter.py::test_integration_parameter - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_property.py::test_integration_material_property - IndexError: list index out of range
+FAILED tests/nodes/subobjects/test_quantity.py::test_integration_quantity - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_software.py::test_integration_software - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/subobjects/test_software_configuration.py::test_integration_software_configuration - AttributeError: 'list' object has no attribute 'starts with'
+FAILED tests/nodes/supporting_nodes/test_file.py::test_integration_file - AttributeError: 'list' object has no attribute 'starts with'
+======================================================================== 23 failed, 100 passed, 1 skipped, 1 warning in 905.51s (0:15:05) =========================================================================
+```

conftest.py

@@ -8,6 +8,7 @@
 
 The fixtures are all functional fixtures that stay consistent between all tests.
 """
+import logging
 import os
 
 import pytest
@@ -49,14 +50,15 @@ def cript_api():
  """
  storage_token = os.getenv("CRIPT_STORAGE_TOKEN")
 
- with cript.API(host=None, api_token=None, storage_token=storage_token) as api:
+ with cript.API(host=None, api_token=None, storage_token=storage_token, default_log_level=logging.DEBUG) as api:
  # overriding AWS S3 cognito variables to be sure we do not upload test data to production storage
  # staging AWS S3 cognito storage variables
  api._IDENTITY_POOL_ID = "us-east-1:25043452-a922-43af-b8a6-7e938a9e55c1"
  api._COGNITO_LOGIN_PROVIDER = "cognito-idp.us-east-1.amazonaws.com/us-east-1_vyK1N9p22"
  api._BUCKET_NAME = "cript-stage-user-data"
  # using the tests folder name within our cloud storage
  api._BUCKET_DIRECTORY_NAME = "tests"
+ api.extra_api_log_debug_info = True
 
  yield api
 

docs/examples/simulation.md

@@ -361,23 +361,13 @@ bulk.output_data = [final_data]
 
 ## Create a virtual Material
 
-First, we'll create a virtual material and add some
-[`Identifiers`](../../nodes/primary_nodes/material/#cript.nodes.primary_nodes.material.Material.identifier)
-to the material to make it easier to identify and search.
+First, we'll create a virtual material with identifiers to make it easier to search for.
 
 ```python
-# create identifier dictionaries and put it in `identifiers` variable
-identifiers = [{"names": ["poly(styrene)", "poly(vinylbenzene)"]}]
-identifiers += [{"bigsmiles": "[H]{[>][<]C(C[>])c1ccccc1[<]}C(C)CC"}]
-identifiers += [{"chem_repeat": ["C8H8"]}]
-
 # create a material node object with identifiers
-polystyrene = cript.Material(name="virtual polystyrene", identifier=identifiers)
+polystyrene = cript.Material(name="virtual polystyrene", bigsmiles="[H]{[>][<]C(C[>])c1ccccc1[<]}C(C)CC", names = ["poly(styrene)", "poly(vinylbenzene)"], chem_repeat= ["C8H8"])
 ```
 
-!!! note "Identifier keys"
- The allowed [`Identifiers`](../../nodes/primary_nodes/material/#cript.nodes.primary_nodes.material.Material.identifier) keys are listed in the [material identifier keys](https://app.criptapp.org/vocab/material_identifier_key) in the CRIPT controlled vocabulary.
-
 ## Add [`Property`](../../nodes/subobjects/property) sub-objects
 Let's also add some [`Property`](../../nodes/subobjects/property) nodes to the [`Material`](../../nodes/primary_nodes/material), which represent its physical or virtual (in the case of a simulated material) properties.
 

docs/examples/synthesis.md

@@ -116,26 +116,20 @@ They are for example the chemical you buy commercially and use as input into you
 For this we create this inventory by adding the [Material](../../nodes/primary_nodes/material) we need one by one.
 
 ```python
-# create a list of identifiers as dictionaries to
-# identify your material to the community and your team
-my_solution_material_identifiers = [
- {"chemical_id": "598-30-1"}
-]
-
 solution = cript.Material(
  name="SecBuLi solution 1.4M cHex",
- identifier=my_solution_material_identifiers
+ chemical_id = "598-30-1",
 )
 ```
 
 These materials are simple, notice how we use the SMILES notation here as an identifier for the material.
 Similarly, we can create more initial materials.
 
 ```python
-toluene = cript.Material(name="toluene", identifier=[{"smiles": "Cc1ccccc1"}, {"pubchem_id": 1140}])
-styrene = cript.Material(name="styrene", identifier=[{"smiles": "c1ccccc1C=C"}, {"inchi": "InChI=1S/C8H8/c1-2-8-6-4-3-5-7-8/h2-7H,1H2"}])
-butanol = cript.Material(name="1-butanol", identifier=[{"smiles": "OCCCC"}, {"inchi_key": "InChIKey=LRHPLDYGYMQRHN-UHFFFAOYSA-N"}])
-methanol = cript.Material(name="methanol", identifier=[{"smiles": "CO"}, {"names": ["Butan-1-ol", "Butyric alcohol", "Methylolpropane", "n-Butan-1-ol", "methanol"]}])
+toluene = cript.Material(name="toluene", smiles="Cc1ccccc1", pubchem_cid = 1140)
+styrene = cript.Material(name="styrene", smiles = "c1ccccc1C=C", inchi = "InChI=1S/C8H8/c1-2-8-6-4-3-5-7-8/h2-7H,1H2")
+butanol = cript.Material(name="1-butanol", smiles = "OCCCC", inchi_key = "InChIKey=LRHPLDYGYMQRHN-UHFFFAOYSA-N")
+methanol = cript.Material(name="methanol", smiles = "CO", names = ["Butan-1-ol", "Butyric alcohol", "Methylolpropane", "n-Butan-1-ol", "methanol"])
 ```
 
 Now that we defined those materials, we can combine them into an inventory
@@ -250,20 +244,15 @@ that will serve as our product. We give the material a `name` attribute and add
 [Project]((../../nodes/primary_nodes/project).
 
 ```python
-polystyrene = cript.Material(name="polystyrene", identifier=[])
+polystyrene = cript.Material(name="polystyrene", bigsmiles="[H]{[>][<]C(C[>])c1ccccc1[<]}C(C)CC")
 project.material += [polystyrene]
 ```
 
 Let's add some `Identifiers` to the material to make it easier to identify and search.
 
 ```python
-# create a name identifier
-polystyrene.identifier += [{"names": ["poly(styrene)", "poly(vinylbenzene)"]}]
-
-# create a BigSMILES identifier
-polystyrene.identifier += [{"bigsmiles": "[H]{[>][<]C(C[>])c1ccccc1[<]}C(C)CC"}]
 # create a chemical repeat unit identifier
-polystyrene.identifier += [{"chem_repeat": ["C8H8"]}]
+polystyrene.chem_repeat = ["C8H8"]
 ```
 
 Next, we'll add some [Property](../../nodes/subobjects/property) nodes to the