Merge branch 'main' into filter_update

.github/workflows/linting.yml

@@ -22,7 +22,7 @@ jobs:
         pip install pylint
     - name: Analysing the code with pylint
       run: |
-        pylint  -d C0301,R0913,W1202 $(git ls-files '*.py') --ignored-modules "rdkit"
+        pylint  -d C0301,R0913,W1202 $(git ls-files '*.py') --ignored-modules "rdkit"  --max-positional-arguments 10
   mypy:
     runs-on: ubuntu-latest
     steps:
@@ -150,7 +150,7 @@ jobs:
           pip install isort
       - name: Analysing the code with isort
         run: |
-          isort --profile black .
+          isort --profile black --check-only .
 
   test_basis:
     needs:

.gitignore

@@ -3,4 +3,5 @@ __pycache__
 molpipeline.egg-info/
 lib/
 build/
+lightning_logs/
 

README.md

@@ -1,35 +1,56 @@
 # MolPipeline
-MolPipeline is a Python package providing RDKit functionality in a Scikit-learn like fashion.
+MolPipeline is a Python package for processing molecules with RDKit in scikit-learn.
+
+<p align="center"><img src=".github/molpipeline.png" height="250"/></p>
 
 ## Background
 
-The open-source package [scikit-learn](https://scikit-learn.org/) provides a large variety of machine
+The [scikit-learn](https://scikit-learn.org/) package provides a large variety of machine
 learning algorithms and data processing tools, among which is the `Pipeline` class, allowing users to
 prepend custom data processing steps to the machine learning model.
-`MolPipeline` extends this concept to the field of chemoinformatics by
-wrapping default functionalities of [RDKit](https://www.rdkit.org/), such as reading and writing SMILES strings
+`MolPipeline` extends this concept to the field of cheminformatics by
+wrapping standard [RDKit](https://www.rdkit.org/) functionality, such as reading and writing SMILES strings
 or calculating molecular descriptors from a molecule-object.
 
-A notable difference to the `Pipeline` class of scikit-learn is that the Pipline from `MolPipeline` allows for 
-instances to fail during processing without interrupting the whole pipeline.
-Such behaviour is useful when processing large datasets, where some SMILES strings might not encode valid molecules
-or some descriptors might not be calculable for certain molecules.
+MolPipeline aims to provide:
 
+- Automated end-to-end processing from molecule data sets to deployable machine learning models.
+- Scalable parallel processing and low memory usage through instance-based processing.
+- Standard pipeline building blocks for flexibly building custom pipelines for various
+cheminformatics tasks.
+- Consistent error handling for tracking, logging, and replacing failed instances (e.g., a
+SMILES string that could not be parsed correctly).
+- Integrated and self-contained pipeline serialization for easy deployment and tracking
+in version control.
 
 ## Publications
 
-The publication is freely available [here](https://chemrxiv.org/engage/chemrxiv/article-details/661fec7f418a5379b00ae036).
+[Sieg J, Feldmann CW, Hemmerich J, Stork C, Sandfort F, Eiden P, and Mathea M, MolPipeline: A python package for processing
+molecules with RDKit in scikit-learn, J. Chem. Inf. Model., doi:10.1021/acs.jcim.4c00863, 2024](https://doi.org/10.1021/acs.jcim.4c00863)
+\
+Further links: [arXiv](https://chemrxiv.org/engage/chemrxiv/article-details/661fec7f418a5379b00ae036)
+
+Feldmann CW, Sieg J, and Mathea M, Analysis of uncertainty of neural
+fingerprint-based models, 2024
+\
+Further links: [repository](https://github.com/basf/neural-fingerprint-uncertainty)
 
 ## Installation
 ```commandline
 pip install molpipeline
 ```
 
-## Usage
+## Documentation
+
+The [notebooks](notebooks) folder contains many basic and advanced examples of how to use Molpipeline.
+
+A nice introduction to the basic usage is in the [01_getting_started_with_molpipeline notebook](notebooks/01_getting_started_with_molpipeline.ipynb).
 
-See the [notebooks](notebooks) folder for basic and advanced examples of how to use Molpipeline.
+## Quick Start
 
-A basic example of how to use MolPipeline to create a fingerprint-based model is shown below (see also the [notebook](notebooks/01_getting_started_with_molpipeline.ipynb)): 
+### Model building
+
+Create a fingerprint-based prediction model:
 ```python
 from molpipeline import Pipeline
 from molpipeline.any2mol import AutoToMol
@@ -58,8 +79,42 @@ pipeline.predict(["CCC"])
 # output: array([0.29])
 ```
 
-Molpipeline also provides custom estimators for standard cheminformatics tasks that can be integrated into pipelines,
-like clustering for scaffold splits (see also the [notebook](notebooks/02_scaffold_split_with_custom_estimators.ipynb)):
+### Feature calculation
+
+Calculating molecular descriptors from SMILES strings is straightforward. For example, physicochemical properties can
+be calculated like this:
+```python
+from molpipeline import Pipeline
+from molpipeline.any2mol import AutoToMol
+from molpipeline.mol2any import MolToRDKitPhysChem
+
+pipeline_physchem = Pipeline(
+    [
+        ("auto2mol", AutoToMol()),
+        (
+            "physchem",
+            MolToRDKitPhysChem(
+                standardizer=None,
+                descriptor_list=["HeavyAtomMolWt", "TPSA", "NumHAcceptors"],
+            ),
+        ),
+    ],
+    n_jobs=-1,
+)
+physchem_matrix = pipeline_physchem.transform(["CCCCCC", "c1ccccc1(O)"])
+physchem_matrix
+# output: array([[72.066,  0.   ,  0.   ],
+#                [88.065, 20.23 ,  1.   ]])
+```
+
+MolPipeline provides further features and descriptors from RDKit, 
+for example Morgan (binary/count) fingerprints and MACCS keys.
+See the [04_feature_calculation notebook](notebooks/04_feature_calculation.ipynb) for more examples.
+
+### Clustering
+
+Molpipeline provides several clustering algorithms as sklearn-like estimators. For example, molecules can be
+clustered by their Murcko scaffold. See the [02_scaffold_split_with_custom_estimators notebook](notebooks/02_scaffold_split_with_custom_estimators.ipynb) for scaffolds splits and further examples.
 
 ```python
 from molpipeline.estimators import MurckoScaffoldClustering

molpipeline/abstract_pipeline_elements/any2mol/string2mol.py

@@ -4,8 +4,11 @@
 
 import abc
 
-from molpipeline.abstract_pipeline_elements.core import AnyToMolPipelineElement
-from molpipeline.utils.molpipeline_types import OptionalMol
+from molpipeline.abstract_pipeline_elements.core import (
+    AnyToMolPipelineElement,
+    InvalidInstance,
+)
+from molpipeline.utils.molpipeline_types import OptionalMol, RDKitMol
 
 
 class StringToMolPipelineElement(AnyToMolPipelineElement, abc.ABC):
@@ -43,3 +46,60 @@ def pretransform_single(self, value: str) -> OptionalMol:
         OptionalMol
             RDKit molecule if representation was valid, else InvalidInstance.
         """
+
+
+class SimpleStringToMolElement(StringToMolPipelineElement, abc.ABC):
+    """Transforms string representation to RDKit Mol objects."""
+
+    def pretransform_single(self, value: str) -> OptionalMol:
+        """Transform string to molecule.
+
+        Parameters
+        ----------
+        value: str
+            string representation.
+
+        Returns
+        -------
+        OptionalMol
+            Rdkit molecule if valid string representation, else None.
+        """
+        if value is None:
+            return InvalidInstance(
+                self.uuid,
+                f"Invalid representation: {value}",
+                self.name,
+            )
+
+        if not isinstance(value, str):
+            return InvalidInstance(
+                self.uuid,
+                f"Not a string: {value}",
+                self.name,
+            )
+
+        mol: RDKitMol = self.string_to_mol(value)
+
+        if not mol:
+            return InvalidInstance(
+                self.uuid,
+                f"Invalid representation: {value}",
+                self.name,
+            )
+        mol.SetProp("identifier", value)
+        return mol
+
+    @abc.abstractmethod
+    def string_to_mol(self, value: str) -> RDKitMol:
+        """Transform string representation to molecule.
+
+        Parameters
+        ----------
+        value: str
+            string representation
+
+        Returns
+        -------
+        RDKitMol
+            Rdkit molecule if valid representation, else None.
+        """

molpipeline/any2mol/__init__.py

@@ -2,12 +2,14 @@
 
 from molpipeline.any2mol.auto2mol import AutoToMol
 from molpipeline.any2mol.bin2mol import BinaryToMol
+from molpipeline.any2mol.inchi2mol import InchiToMol
 from molpipeline.any2mol.sdf2mol import SDFToMol
 from molpipeline.any2mol.smiles2mol import SmilesToMol
 
 __all__ = [
     "AutoToMol",
     "BinaryToMol",
     "SmilesToMol",
+    "InchiToMol",
     "SDFToMol",
 ]

molpipeline/any2mol/auto2mol.py

@@ -9,6 +9,7 @@
     InvalidInstance,
 )
 from molpipeline.any2mol.bin2mol import BinaryToMol
+from molpipeline.any2mol.inchi2mol import InchiToMol
 from molpipeline.any2mol.sdf2mol import SDFToMol
 from molpipeline.any2mol.smiles2mol import SmilesToMol
 from molpipeline.utils.molpipeline_types import OptionalMol, RDKitMol
@@ -29,6 +30,7 @@ def __init__(
         uuid: Optional[str] = None,
         elements: tuple[AnyToMolPipelineElement, ...] = (
             SmilesToMol(),
+            InchiToMol(),
             BinaryToMol(),
             SDFToMol(),
         ),
@@ -44,7 +46,7 @@ def __init__(
         uuid: str, optional (default=None)
             Unique identifier of PipelineElement.
         elements: tuple[AnyToMol, ...], optional (default=(SmilesToMol(),
-            BinaryToMol(), SDFToMol()))
+            InchiToMol(), BinaryToMol(), SDFToMol()))
             Elements to try to transform the input to a molecule.
         """
         super().__init__(name=name, n_jobs=n_jobs, uuid=uuid)

molpipeline/any2mol/inchi2mol.py

@@ -0,0 +1,27 @@
+"""Classes ment to transform given inchi to a RDKit molecule."""
+
+from rdkit import Chem
+
+from molpipeline.abstract_pipeline_elements.any2mol.string2mol import (
+    SimpleStringToMolElement,
+)
+from molpipeline.utils.molpipeline_types import RDKitMol
+
+
+class InchiToMol(SimpleStringToMolElement):
+    """Transforms Inchi to RDKit Mol objects."""
+
+    def string_to_mol(self, value: str) -> RDKitMol:
+        """Transform Inchi string to molecule.
+
+        Parameters
+        ----------
+        value: str
+            Inchi string.
+
+        Returns
+        -------
+        RDKitMol
+            Rdkit molecule if valid Inchi, else None.
+        """
+        return Chem.MolFromInchi(value)

molpipeline/any2mol/smiles2mol.py

@@ -5,16 +5,15 @@
 from rdkit import Chem
 
 from molpipeline.abstract_pipeline_elements.any2mol.string2mol import (
-    StringToMolPipelineElement as _StringToMolPipelineElement,
+    SimpleStringToMolElement,
 )
-from molpipeline.abstract_pipeline_elements.core import InvalidInstance
-from molpipeline.utils.molpipeline_types import OptionalMol, RDKitMol
+from molpipeline.utils.molpipeline_types import RDKitMol
 
 
-class SmilesToMol(_StringToMolPipelineElement):
+class SmilesToMol(SimpleStringToMolElement):
     """Transforms Smiles to RDKit Mol objects."""
 
-    def pretransform_single(self, value: str) -> OptionalMol:
+    def string_to_mol(self, value: str) -> RDKitMol:
         """Transform Smiles string to molecule.
 
         Parameters
@@ -24,30 +23,7 @@ def pretransform_single(self, value: str) -> OptionalMol:
 
         Returns
         -------
-        OptionalMol
+        RDKitMol
             Rdkit molecule if valid SMILES, else None.
         """
-        if value is None:
-            return InvalidInstance(
-                self.uuid,
-                f"Invalid SMILES: {value}",
-                self.name,
-            )
-
-        if not isinstance(value, str):
-            return InvalidInstance(
-                self.uuid,
-                f"Not a string: {value}",
-                self.name,
-            )
-
-        mol: RDKitMol = Chem.MolFromSmiles(value)
-
-        if not mol:
-            return InvalidInstance(
-                self.uuid,
-                f"Invalid SMILES: {value}",
-                self.name,
-            )
-        mol.SetProp("identifier", value)
-        return mol
+        return Chem.MolFromSmiles(value)