Update improve/V30RC02 with the bugfixes made to main

CONTRIBUTING.md

@@ -0,0 +1,146 @@
+# Contributing Guide
+
+Thank you for investing your time in contributing to the Eclipse-BaSyx-Python SDK!
+
+In this guide you will get an overview of the contribution workflow from opening an
+issue, creating a PR, reviewing, and merging the PR.
+
+Furthermore, it gives some guidelines on how to write commit and pull
+request messages, as well as on codestyle and testing.
+
+## Issues
+
+In order to open an issue, go
+to https://github.com/eclipse-basyx/basyx-python-sdk/issues and click "**New Issue**".
+
+The first step for a good issue is a descriptive title. The title is a brief description
+of the issue, ideally 72 characters or fewer using imperative language.
+Furthermore, if you know which module is the cause of your issue, please mention 
+it at the beginning of the issue title.
+
+Here are some example for a good title:
+
+```
+model.datatypes: Missing type `xs:someThingMadeUp`
+compliance_tool: Fail to check aasx package without thumbnail
+adapter.aasx: `Property.value` `0` are converted into `NoneType`
+```
+
+As you can see, inline code blocks (the backticks) are used to highlight 
+class names or types.
+
+In the issue message, use full text or bullet points to describe your issue in detail.
+Please include a short paragraph on each of
+
+- Expected behavior:  A description of what the expected behavior should be, so that
+  maintainers can understand how the issue differs from the intended functionality.
+- Actual behavior: A description of what is actually happening, which can help pinpoint
+  the cause of the issue.
+- Environment: Information about the operating system, the SDK version used, the version
+  of the specification used, or other relevant technical details that may be
+  contributing to the issue.
+
+Additionally, if you have ideas on how to address the issue, please include them here!
+
+## Contribute Changes
+
+Here's the standard workflow to contribute changes to Eclipse-BaSyx-Python.
+
+Before contributing, please make sure, you fill out
+the [Eclipse Contributor Agreement (ECA)](https://www.eclipse.org/legal/ECA.php). This
+is done by creating an Eclipse account for your git e-mail address and then submitting
+the following
+form: [https://accounts.eclipse.org/user/eca](https://accounts.eclipse.org/user/eca).
+The E-Mail address used to sign the ECA is the same one that needs to be used for
+committing.
+
+After this, the workflow to submit contributions to Eclipse-BaSyx-Python is pretty
+standard, as the picture (based
+on [this blog-post by Tomas Beuzen](https://www.tomasbeuzen.com/post/git-fork-branch-pull/))
+below shows:
+
+![CONTRIBUTING_Workflow](./etc/CONTRIBUTING_Workflow.png)
+
+1. Fork the Eclipse-BaSyx Repository
+2. Clone your fork to your development machine and add Eclipse-BaSyx as `upstream`:
+
+```bash
+git remote add upstream https://github.com/eclipse-basyx/basyx-python-sdk
+```
+
+3. Pull the branch you want to contribute to:
+
+```bash
+git pull upstream <branch_name>
+```
+
+Now, you can create a new local branch in which you can create your changes and actually
+do your changes. When you're done with that, continue with:
+
+4. Push the new branch to your fork:
+
+```bash
+git push origin <your_new_branch>
+```
+
+5. Create a [Pull Request](https://github.com/eclipse-basyx/basyx-python-sdk/pulls) from
+   your fork `<your_new_branch>` to the Eclipse-BaSyx-Python `<branch_name>`
+
+The Eclipse-BaSyx-Python maintainers will then review the pull request and communicate
+the further steps via the comments.
+
+## Commit and Pull Request Messages
+
+In order to effectively communicate, there are some conventions to respect when writing
+commit messages and pull requests.
+
+Similarily to when creating an issue, the commit title, as well as the PR title should 
+be as short as possible, ideally 72 characters or fewer using imperative language.
+If a specific module is affected, please mention it at the beginning of the title.
+
+Here are some examples:
+
+```
+model.datatypes: Add type `xs:someThingMadeUp`
+compliance_tool: Fix fail to check aasx package without thumbnail
+adapter.aasx: Fix `Property.value` `0` converted into `NoneType`
+```
+
+The following guidelines are for the commit or PR message text:
+
+- No imperative, full text, bullet points where necessary
+- Max. 72 characters per line
+- There should be always 2 things in a Commit/PR message:
+    - Currently, the situation is this
+    - Motivate, why is it now different?
+- Don't describe what has been done, as this can be looked up in the code
+- Write as long as necessary, as short as possible
+- Where sensible, reference the specification, ideally
+  via `https://link/to.pdf#Page=123`
+- Optionally, where applicable reference respective issues: `Fixes #123`
+
+## Codestyle and Testing
+
+Our code follows the [PEP 8 -- Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/).
+Additionally, we use [PEP 484 -- Type Hints](https://www.python.org/dev/peps/pep-0484/) throughout the code to enable type checking the code.
+
+Before submitting any changes, make sure to let `mypy` and `pycodestyle` check your code and run the unit tests with
+Python's builtin `unittest`. To install the required tools, use:
+```bash
+pip install mypy pycodestyle
+```
+
+Running all checks:
+```bash
+mypy basyx test
+pycodestyle --max-line-length 120 basyx test
+python -m unittest
+```
+
+We aim to cover our code with test by at least 80%. To check test coverage, you can use `coverage`:
+
+```bash
+pip install coverage
+coverage run --source basyx --branch -m unittest
+coverage report -m
+```

README.md

@@ -158,33 +158,9 @@ correctly) with `aas-compliance-check` on the command line.
 For further usage information consider the `basyx.aas.compliance_tool`-package or invoke with 
 `python -m basyx.aas.compliance_tool.cli --help` respectively `aas-compliance-check --help`.
 
-## Development
+## Contributing
 
-### Codestyle and Testing
-
-Our code follows the [PEP 8 -- Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/).
-Additionally, we use [PEP 484 -- Type Hints](https://www.python.org/dev/peps/pep-0484/) throughout the code to enable type checking the code.
-
-Before submitting any changes, make sure to let `mypy` and `pycodestyle` check your code and run the unit tests with
-Python's builtin `unittest`. To install the required tools, use:
-```bash
-pip install mypy pycodestyle
-```
-
-Running all checks:
-```bash
-mypy basyx test
-pycodestyle --max-line-length 120 basyx test
-python -m unittest
-```
-
-We aim to cover our code with test by at least 80%. To check test coverage, you can use `coverage`:
-
-```bash
-pip install coverage
-coverage run --source basyx --branch -m unittest
-coverage report -m
-```
+For contributing with issues and code, please see our [Contribution Guideline](CONTRIBUTING.md).
 
 ### Eclipse Contributor Agreement
 

basyx/aas/adapter/aasx.py

@@ -111,7 +111,7 @@ def get_thumbnail(self) -> Optional[bytes]:
 
     def read_into(self, object_store: model.AbstractObjectStore,
                   file_store: "AbstractSupplementaryFileContainer",
-                  override_existing: bool = False) -> Set[model.Identifier]:
+                  override_existing: bool = False, **kwargs) -> Set[model.Identifier]:
         """
         Read the contents of the AASX package and add them into a given
         :class:`ObjectStore <aas.model.provider.AbstractObjectStore>`
@@ -147,12 +147,14 @@ def read_into(self, object_store: model.AbstractObjectStore,
         # Iterate AAS files
         for aas_part in self.reader.get_related_parts_by_type(aasx_origin_part)[
                 RELATIONSHIP_TYPE_AAS_SPEC]:
-            self._read_aas_part_into(aas_part, object_store, file_store, read_identifiables, override_existing)
+            self._read_aas_part_into(aas_part, object_store, file_store,
+                                     read_identifiables, override_existing, **kwargs)
 
             # Iterate split parts of AAS file
             for split_part in self.reader.get_related_parts_by_type(aas_part)[
                     RELATIONSHIP_TYPE_AAS_SPEC_SPLIT]:
-                self._read_aas_part_into(split_part, object_store, file_store, read_identifiables, override_existing)
+                self._read_aas_part_into(split_part, object_store, file_store,
+                                         read_identifiables, override_existing, **kwargs)
 
         return read_identifiables
 
@@ -172,7 +174,7 @@ def _read_aas_part_into(self, part_name: str,
                             object_store: model.AbstractObjectStore,
                             file_store: "AbstractSupplementaryFileContainer",
                             read_identifiables: Set[model.Identifier],
-                            override_existing: bool) -> None:
+                            override_existing: bool, **kwargs) -> None:
         """
         Helper function for :meth:`read_into()` to read and process the contents of an AAS-spec part of the AASX file.
 
@@ -188,7 +190,7 @@ def _read_aas_part_into(self, part_name: str,
         :param override_existing: If True, existing objects in the object store are overridden with objects from the
             AASX that have the same Identifer. Default behavior is to skip those objects from the AASX.
         """
-        for obj in self._parse_aas_part(part_name):
+        for obj in self._parse_aas_part(part_name, **kwargs):
             if obj.identification in read_identifiables:
                 continue
             if obj.identification in object_store:
@@ -204,7 +206,7 @@ def _read_aas_part_into(self, part_name: str,
             if isinstance(obj, model.Submodel):
                 self._collect_supplementary_files(part_name, obj, file_store)
 
-    def _parse_aas_part(self, part_name: str) -> model.DictObjectStore:
+    def _parse_aas_part(self, part_name: str, **kwargs) -> model.DictObjectStore:
         """
         Helper function to parse the AAS objects from a single JSON or XML part of the AASX package.
 
@@ -218,12 +220,12 @@ def _parse_aas_part(self, part_name: str) -> model.DictObjectStore:
         if content_type.split(";")[0] in ("text/xml", "application/xml") or content_type == "" and extension == "xml":
             logger.debug("Parsing AAS objects from XML stream in OPC part {} ...".format(part_name))
             with self.reader.open_part(part_name) as p:
-                return read_aas_xml_file(p)
+                return read_aas_xml_file(p, **kwargs)
         elif content_type.split(";")[0] in ("text/json", "application/json") \
                 or content_type == "" and extension == "json":
             logger.debug("Parsing AAS objects from JSON stream in OPC part {} ...".format(part_name))
             with self.reader.open_part(part_name) as p:
-                return read_aas_json_file(io.TextIOWrapper(p, encoding='utf-8-sig'))
+                return read_aas_json_file(io.TextIOWrapper(p, encoding='utf-8-sig'), **kwargs)
         else:
             logger.error("Could not determine part format of AASX part {} (Content Type: {}, extension: {}"
                          .format(part_name, content_type, extension))

basyx/aas/adapter/json/json_deserialization.py

@@ -36,8 +36,8 @@
 from typing import Dict, Callable, TypeVar, Type, List, IO, Optional, Set
 
 from basyx.aas import model
-from .._generic import MODELING_KIND_INVERSE, ASSET_KIND_INVERSE, KEY_ELEMENTS_INVERSE, KEY_TYPES_INVERSE,\
-    IDENTIFIER_TYPES_INVERSE, ENTITY_TYPES_INVERSE, IEC61360_DATA_TYPES_INVERSE, IEC61360_LEVEL_TYPES_INVERSE,\
+from .._generic import MODELING_KIND_INVERSE, ASSET_KIND_INVERSE, KEY_ELEMENTS_INVERSE, KEY_TYPES_INVERSE, \
+    IDENTIFIER_TYPES_INVERSE, ENTITY_TYPES_INVERSE, IEC61360_DATA_TYPES_INVERSE, IEC61360_LEVEL_TYPES_INVERSE, \
     KEY_ELEMENTS_CLASSES_INVERSE
 
 logger = logging.getLogger(__name__)

basyx/aas/adapter/xml/xml_deserialization.py

@@ -85,7 +85,7 @@ def _tag_replace_namespace(tag: str, nsmap: Dict[str, str]) -> str:
     """
     split = tag.split("}")
     for prefix, namespace in nsmap.items():
-        if namespace == split[0][1:]:
+        if prefix and namespace == split[0][1:]:
             return prefix + ":" + split[1]
     return tag
 
@@ -449,9 +449,8 @@ def _amend_abstract_attributes(cls, obj: object, element: etree.Element) -> None
                     continue
                 if len(constraint) > 1:
                     logger.warning(f"{_element_pretty_identifier(constraint)} has more than one constraint, "
-                                   "using the first one...")
-                constructed = _failsafe_construct(constraint[0], cls.construct_constraint, cls.failsafe)
-                if constructed is not None:
+                                   "which is invalid! Deserializing all constraints anyways...")
+                for constructed in _failsafe_construct_multiple(constraint, cls.construct_constraint, cls.failsafe):
                     obj.qualifier.add(constructed)
 
     @classmethod
@@ -729,9 +728,8 @@ def construct_annotated_relationship_element(cls, element: etree.Element,
                     raise KeyError(f"{_element_pretty_identifier(data_element)} has no data element!")
                 if len(data_element) > 1:
                     logger.warning(f"{_element_pretty_identifier(data_element)} has more than one data element, "
-                                   "using the first one...")
-                constructed = _failsafe_construct(data_element[0], cls.construct_data_element, cls.failsafe)
-                if constructed is not None:
+                                   "which is invalid! Deserializing all data elements anyways...")
+                for constructed in _failsafe_construct_multiple(data_element, cls.construct_data_element, cls.failsafe):
                     annotated_relationship_element.annotation.add(constructed)
         return annotated_relationship_element
 
@@ -787,9 +785,9 @@ def construct_entity(cls, element: etree.Element, object_class=model.Entity, **_
                     raise KeyError(f"{_element_pretty_identifier(submodel_element)} has no submodel element!")
                 if len(submodel_element) > 1:
                     logger.warning(f"{_element_pretty_identifier(submodel_element)} has more than one submodel element,"
-                                   " using the first one...")
-                constructed = _failsafe_construct(submodel_element[0], cls.construct_submodel_element, cls.failsafe)
-                if constructed is not None:
+                                   " which is invalid! Deserializing all submodel elements anyways...")
+                for constructed in _failsafe_construct_multiple(submodel_element, cls.construct_submodel_element,
+                                                                cls.failsafe):
                     entity.statement.add(constructed)
         cls._amend_abstract_attributes(entity, element)
         return entity
@@ -912,9 +910,9 @@ def construct_submodel_element_collection(cls, element: etree.Element,
                     raise KeyError(f"{_element_pretty_identifier(submodel_element)} has no submodel element!")
                 if len(submodel_element) > 1:
                     logger.warning(f"{_element_pretty_identifier(submodel_element)} has more than one submodel element,"
-                                   " using the first one...")
-                constructed = _failsafe_construct(submodel_element[0], cls.construct_submodel_element, cls.failsafe)
-                if constructed is not None:
+                                   " which is invalid! Deserializing all submodel elements anyways...")
+                for constructed in _failsafe_construct_multiple(submodel_element, cls.construct_submodel_element,
+                                                                cls.failsafe):
                     collection.value.add(constructed)
         cls._amend_abstract_attributes(collection, element)
         return collection
@@ -985,9 +983,9 @@ def construct_submodel(cls, element: etree.Element, object_class=model.Submodel,
                     raise KeyError(f"{_element_pretty_identifier(submodel_element)} has no submodel element!")
                 if len(submodel_element) > 1:
                     logger.warning(f"{_element_pretty_identifier(submodel_element)} has more than one submodel element,"
-                                   " using the first one...")
-                constructed = _failsafe_construct(submodel_element[0], cls.construct_submodel_element, cls.failsafe)
-                if constructed is not None:
+                                   " which is invalid! Deserializing all submodel elements anyways...")
+                for constructed in _failsafe_construct_multiple(submodel_element, cls.construct_submodel_element,
+                                                                cls.failsafe):
                     submodel.submodel_element.add(constructed)
         cls._amend_abstract_attributes(submodel, element)
         return submodel

basyx/aas/model/__init__.py

@@ -53,7 +53,7 @@
     ConceptDictionary: KeyElements.CONCEPT_DICTIONARY,
     Entity: KeyElements.ENTITY,
     BasicEvent: KeyElements.BASIC_EVENT,
-    Event: KeyElements.EVENT,
+    Event: KeyElements.EVENT,  # type: ignore
     Blob: KeyElements.BLOB,
     File: KeyElements.FILE,
     Operation: KeyElements.OPERATION,
@@ -62,9 +62,9 @@
     MultiLanguageProperty: KeyElements.MULTI_LANGUAGE_PROPERTY,
     Range: KeyElements.RANGE,
     ReferenceElement: KeyElements.REFERENCE_ELEMENT,
-    DataElement: KeyElements.DATA_ELEMENT,
-    SubmodelElementCollection: KeyElements.SUBMODEL_ELEMENT_COLLECTION,
+    DataElement: KeyElements.DATA_ELEMENT,  # type: ignore
+    SubmodelElementCollection: KeyElements.SUBMODEL_ELEMENT_COLLECTION,  # type: ignore
     AnnotatedRelationshipElement: KeyElements.ANNOTATED_RELATIONSHIP_ELEMENT,
     RelationshipElement: KeyElements.RELATIONSHIP_ELEMENT,
-    SubmodelElement: KeyElements.SUBMODEL_ELEMENT,
+    SubmodelElement: KeyElements.SUBMODEL_ELEMENT,  # type: ignore
 }

basyx/aas/model/datatypes.py

@@ -375,10 +375,10 @@ def from_string(cls, value: str) -> "NormalizedString":
 
 
 AnyXSDType = Union[
-    Duration, DayTimeDuration, YearMonthDuration, DateTime, Date, Time, GYearMonth, GYear, GMonthDay, GMonth, GDay,
-    Boolean, Base64Binary, HexBinary, Float, Double, Decimal, Integer, Long, Int, Short, Byte, NonPositiveInteger,
-    NegativeInteger, NonNegativeInteger, PositiveInteger, UnsignedLong, UnsignedInt, UnsignedShort, UnsignedByte,
-    AnyURI, String, NormalizedString]
+    Duration, DayTimeDuration, YearMonthDuration, DateTime, DateTimeStamp, Date, Time, GYearMonth, GYear, GMonthDay,
+    GMonth, GDay, Boolean, Base64Binary, HexBinary, Float, Double, Decimal, Integer, Long, Int, Short, Byte,
+    NonPositiveInteger, NegativeInteger, NonNegativeInteger, PositiveInteger, UnsignedLong, UnsignedInt, UnsignedShort,
+    UnsignedByte, AnyURI, String, NormalizedString]
 
 
 XSD_TYPE_NAMES: Dict[Type[AnyXSDType], str] = {
@@ -548,6 +548,16 @@ def from_xsd(value: str, type_: Type[AnyXSDType]) -> AnyXSDType:  # workaround.
         return _parse_xsd_bool(value)
     elif issubclass(type_, (int, float, str)):
         return type_(value)
+    elif type_ is decimal.Decimal:
+        try:
+            return decimal.Decimal(value)
+        except decimal.InvalidOperation as e:
+            # We cannot use the original exception text here, because the text differs depending on
+            # whether the _decimal or _pydecimal module is used. Furthermore, the _decimal doesn't provide
+            # a real error message suited for end users, but provides a list of conditions that trigger the exception.
+            # See https://github.com/python/cpython/issues/76420
+            # Raising our own error message allows us to verify it in the tests.
+            raise ValueError(f"Cannot convert '{value}' to Decimal!") from e
     elif type_ is Duration:
         return _parse_xsd_duration(value)
     elif type_ is YearMonthDuration: