[UnitaryHack] Highlight executor args/kwargs in the docs (#1680)

* Add exception handle CLI start * Add AWS Lambda Executor auto documentation * Fix slurm executorr auto class reference * Add changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert "Add exception handle CLI start" This reverts commit 560195c. * Fix RTD import * Add install dependencies for docs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix isort * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add step install covalent this branch's source * Fix CHANGELOG * Seperated out autodoc requirements into file * Add executor plugins requirements * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Install covalent first * Fix version conflict 'urllib3<1.27,>=1.25.4' botocore * Move botocore urllib3 dependency to post env create * Fix post create env * Fix post create env * Fix post create env * Revert changes wrt urllib3 version pinning for boto3 * Add urllib3 version pinning for boto3 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use version pinning readthedocs/readthedocs.org#9752 (comment) * Fix "Invalid configuration option: python.version" * Fix version conflict * Add comment on dependency pinning * Remove misplaced EC2 doc --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sankalp Sanand <sankalp@agnostiq.ai>
AgnostiqHQ · Jul 4, 2023 · 691f0eb · 691f0eb
1 parent 9a8e955
commit 691f0eb
Show file tree

Hide file tree

Showing 8 changed files with 56 additions and 10 deletions.
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -31,14 +31,26 @@ sphinx:
 
 # Optionally set the version of Python and requirements required to build your docs
 python:
-  version: 3.8
   install:
     - requirements: requirements.txt
+    - requirements: doc/autodoc_executor_plugins_requirements.txt
     - requirements: doc/requirements.txt
     - method: pip
       path: .
     - method: pip
       path: .
+    - method: pip
+      path: .
 
 build:
-  image: latest
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.8"
+  jobs:
+    # This for dependency pinning because readthedocs.yml automically updates all libraries despite being pinned.
+    # See: https://github.com/readthedocs/readthedocs.org/issues/9752#issuecomment-1324778813
+    post_install:
+      # Botocore from AWS executors requrires urllib3<1.27,>=1.25.4
+      - pip install urllib3==1.26.6
+      # For covalent slurm plugin
+      - pip install aiofiles==0.8.0
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [UNRELEASED]
 
+### Docs
+
+- Fix autodoc for SSH, Slurm, AWS Braket, AWS Lambda, AWS EC2, AWS Batch, Google Batch 
+
 ### Operations
 
 - Bumped versions in pre-commit config

diff --git a/doc/autodoc_executor_plugins_requirements.txt b/doc/autodoc_executor_plugins_requirements.txt
@@ -0,0 +1,7 @@
+covalent-awsbatch-plugin
+covalent-awslambda-plugin
+covalent-braket-plugin
+covalent-ec2-plugin
+covalent-gcpbatch-plugin
+covalent-slurm-plugin
+covalent-ssh-plugin
diff --git a/doc/generate_docs.py b/doc/generate_docs.py
@@ -20,6 +20,29 @@
 
 import os
 import subprocess
+import sys
+
+
+def install_autoclass_doc_dependencies() -> None:
+    """Install the packages required to build the documentation from plugin
+    autoclass."""
+
+    try:
+        # Install covalent from branch source which includes this branch's changes to docs.
+        subprocess.check_call([sys.executable, "-m", "pip", "install", "../"])
+        subprocess.check_call(
+            [
+                sys.executable,
+                "-m",
+                "pip",
+                "install",
+                "-r",
+                "autodoc_executor_plugins_requirements.txt",
+            ]
+        )
+    except subprocess.CalledProcessError as e:
+        print(f"Error handling dependency with error: {e}")
+        sys.exit(1)
 
 
 def run(clean_dir: bool = False) -> None:
@@ -39,6 +62,7 @@ def run(clean_dir: bool = False) -> None:
         print("Removing old build")
         cmd = subprocess.Popen(["make", "clean"])
     else:
+        install_autoclass_doc_dependencies()
         cmd = subprocess.Popen(["make", "html"])
     cmd.communicate()
 

diff --git a/doc/source/api/executors/awsbatch.rst b/doc/source/api/executors/awsbatch.rst
@@ -209,11 +209,6 @@ The following shows an example of how a user might modify their `covalent config
     ...
 
 
-.. autoclass:: covalent.executor.EC2Executor
-    :members:
-    :inherited-members:
-
-
 ===========================================
 4. Required Cloud Resources
 ===========================================
@@ -355,6 +350,6 @@ In order to run your workflows with covalent there are a few notable AWS resourc
 .. 5. Source
 .. ===========================================
 
-.. autoclass:: covalent.executor.AWSBatchExecutor
+.. autoclass:: covalent_awsbatch_plugin.awsbatch.AWSBatchExecutor
     :members:
     :inherited-members:
diff --git a/doc/source/api/executors/awsbraket.rst b/doc/source/api/executors/awsbraket.rst
@@ -419,6 +419,6 @@ Users can use the following `Terraform <https://www.terraform.io/>`_ snippet as
 .. 5. Source
 .. ===========================================
 
-.. autoclass:: covalent.executor.BraketExecutor
+.. autoclass:: covalent_braket_plugin.braket.BraketExecutor
     :members:
     :inherited-members:
diff --git a/doc/source/api/executors/awslambda.rst b/doc/source/api/executors/awslambda.rst
@@ -454,3 +454,7 @@ Once the derived image has been built and pushed to ECR, users need to create a
         --timeout 900
 
 The above CLI command will register a new AWS lambda function that will use the user's custom derived image ``my-custom-lambda-executor:latest`` with a memory size of  ``512 MB`` and a timeout values of ``900`` seconds. The ``role`` argument is used to specify the ARN of the IAM role the AWS Lambda can assume during execution. The necessary permissions for the IAM role have been provided in ``Required AWS resources`` section. More details about creating and updating AWS lambda functions can be found `here <https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-images.html>`_.
+
+.. autoclass:: covalent_awslambda_plugin.AWSLambdaExecutor
+    :members:
+    :inherited-members:
diff --git a/doc/source/api/executors/slurm.rst b/doc/source/api/executors/slurm.rst
@@ -154,6 +154,6 @@ Then, reinstall this plugin using ``pip install covalent-slurm-plugin[sshproxy]`
 
 The ``hosts`` parameter is a list of hostnames for which the ``sshproxy`` service will be used.  If the address provided in the plugin constructor is not present in this list, ``sshproxy`` will not be used.  The ``password`` is the user's password, not including the 6-digit OTP.  The ``secret`` is the 2FA secret provided when a user registers a new device on `Iris <https://iris.nersc.gov>`_.  Rather than scan the QR code into an authenticator app, inspect the Oath Seed URL for a string labeled ``secret=...``, typically consisting of numbers and capital letters.  Users can validate that correct OTP codes are being generated by using the command ``oathtool <secret>`` and using the 6-digit number returned in the "Test" option on the Iris 2FA page.  Note that these values are stored in plaintext in the Covalent configuration file.  If a user suspects credentials have been stolen or compromised, contact your systems administrator immediately to report the incident and request deactivation.
 
-.. autoclass:: covalent.executor.SlurmExecutor
+.. autoclass:: covalent_slurm_plugin.SlurmExecutor
     :members:
     :inherited-members: