Skip to content

Commit

Permalink
MRG: Merge pull request #671 from octue/move-question-retrying-into-a…
Browse files Browse the repository at this point in the history
…sk-method

Move question retrying into ask method
  • Loading branch information
cortadocodes committed Jul 24, 2024
2 parents 84b418a + b143180 commit f0c5eaf
Show file tree
Hide file tree
Showing 7 changed files with 404 additions and 450 deletions.
26 changes: 13 additions & 13 deletions docs/source/asking_questions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,13 @@ You can also set the following options when you call :mod:`Child.ask <octue.reso
- ``asynchronous`` - if ``True``, don't wait for an answer to the question (the result and other events can be :ref:`retrieved from the event store later <retrieving_asynchronous_answers>`)
- ``timeout`` - how long in seconds to wait for an answer (``None`` by default - i.e. don't time out)

If the question fails:
- If ``raise_errors=False``, the unraised error is returned
- If ``raise_errors=False`` and ``max_retries > 0``, the question is retried up to this number of times
- If ``raise_errors=False``, ``max_retries > 0``, and ``prevent_retries_when`` is a list of exception types, the question is retried unless the error type is in the list
- If ``raise_errors=False``, ``log_errors=True``, and the question fails after its final retry, the error is logged


Exceptions raised by a child
----------------------------
If a child raises an exception while processing your question, the exception will always be forwarded and re-raised in
Expand Down Expand Up @@ -185,8 +192,8 @@ access the event store and run:

Asking multiple questions in parallel
=====================================
You can also ask multiple questions to a service in parallel. By default, if any of the questions fail, an error is
raised and no answers are returned.
You can also ask multiple questions to a service in parallel - just provide questions as dictionaries of `Child.ask`
arguments:

.. code-block:: python
Expand All @@ -203,18 +210,11 @@ raised and no answers are returned.
This method uses multithreading, allowing all the questions to be asked at once instead of one after another.

**Options**
.. hint::

- If ``raise_errors=False`` is provided, answers are returned for all successful questions while unraised errors are
logged and returned for unsuccessful ones
- If ``raise_errors=False`` is provided with ``max_retries > 0``, failed questions are retried up to this number of
times
- If ``raise_errors=False`` is provided with ``max_retries > 0`` and ``prevent_retries_when`` is set to a list of
exception types, failed questions are retried except for those whose exception types are in the list
- ``max_workers``: The maximum number of threads that can be used to ask questions in parallel can be set via this
argument. It has no effect on the total number of questions that can be asked via ``Child.ask_multiple``.
- ``log_errors``: If `True` and ``raise_errors=False``, any errors remaining once retries are exhausted are logged in
addition to being returned
The maximum number of threads that can be used to ask questions in parallel can be set via the ``max_workers``
argument. It has no effect on the total number of questions that can be asked, just how many can be in progress at
once.


Asking a question within a service
Expand Down
168 changes: 85 additions & 83 deletions docs/source/inter_service_compatibility.rst

Large diffs are not rendered by default.

7 changes: 0 additions & 7 deletions octue/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
logger = logging.getLogger(__name__)


PYTHONUNBUFFERED = "PYTHONUNBUFFERED"
__all__ = ("Runner",)


Expand All @@ -22,9 +21,3 @@
include_process_name=bool(int(os.environ.get("INCLUDE_PROCESS_NAME_IN_LOGS", 0))),
include_thread_name=bool(int(os.environ.get("INCLUDE_THREAD_NAME_IN_LOGS", 0))),
)

if not os.environ.get(PYTHONUNBUFFERED):
logger.warning(
f"The {PYTHONUNBUFFERED!r} environment variable isn't set - logs may not appear in real time. Set "
f"{PYTHONUNBUFFERED}=1 to fix this."
)
Loading

0 comments on commit f0c5eaf

Please sign in to comment.