Skip to content

Commit

Permalink
DRIVERS-2789 Convert Retryable Reads Spec to Markdown
Browse files Browse the repository at this point in the history
  • Loading branch information
blink1073 committed Apr 30, 2024
1 parent 427e045 commit 6c7e14e
Show file tree
Hide file tree
Showing 10 changed files with 557 additions and 881 deletions.
2 changes: 1 addition & 1 deletion source/change-streams/tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ and sharded clusters unless otherwise specified:
04. `ChangeStream` will not attempt to resume on any error encountered while executing an `aggregate` command. Note that
retryable reads may retry `aggregate` commands. Drivers should be careful to distinguish retries from resume
attempts. Alternatively, drivers may specify `retryReads=false` or avoid using a
[retryable error](../../retryable-reads/retryable-reads.rst#retryable-error) for this test.
[retryable error](../../retryable-reads/retryable-reads.md#retryable-error) for this test.

05. **Removed**

Expand Down
1 change: 1 addition & 0 deletions source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
- [Load Balancer Support](load-balancers/load-balancers.md)
- [Logging](logging/logging.md)
- [MongoDB Driver Performance Benchmarking](benchmarking/benchmarking.md)
- [Retryable Reads](retryable-reads/retryable-reads.md)
- [Server Selection](server-selection/server-selection.md)
- [Server Selection Test Plan](server-selection/server-selection-tests.md)
- [Unified Test Format](unified-test-format/unified-test-format.md)
Expand Down
1,042 changes: 448 additions & 594 deletions source/retryable-reads/retryable-reads.md

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions source/retryable-reads/retryable-reads.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@

.. note::
This specification has been converted to Markdown and renamed to
`retryable-reads.md <retryable-reads.md>`_.
212 changes: 99 additions & 113 deletions source/retryable-reads/tests/README.md
Original file line number Diff line number Diff line change
@@ -1,169 +1,155 @@
=====================
Retryable Reads Tests
=====================
# Retryable Reads Tests

.. contents::
______________________________________________________________________

----
## Introduction

Introduction
============
The YAML and JSON files in this directory are platform-independent tests meant to exercise a driver's implementation of
retryable reads. These tests utilize the \[Unified Test Format\](../../unified-test-format/unified-test-format.md).

The YAML and JSON files in this directory are platform-independent tests meant
to exercise a driver's implementation of retryable reads. These tests utilize
the [Unified Test Format](../../unified-test-format/unified-test-format.md).
Several prose tests, which are not easily expressed in YAML, are also presented in this file. Those tests will need to
be manually implemented by each driver.

Several prose tests, which are not easily expressed in YAML, are also presented
in this file. Those tests will need to be manually implemented by each driver.
## Prose Tests

Prose Tests
===========
### 1. PoolClearedError Retryability Test

1. PoolClearedError Retryability Test
-------------------------------------
This test will be used to ensure drivers properly retry after encountering PoolClearedErrors. It MUST be implemented by
any driver that implements the CMAP specification. This test requires MongoDB 4.2.9+ for `blockConnection` support in
the failpoint.

This test will be used to ensure drivers properly retry after encountering PoolClearedErrors.
It MUST be implemented by any driver that implements the CMAP specification.
This test requires MongoDB 4.2.9+ for ``blockConnection`` support in the failpoint.
1. Create a client with maxPoolSize=1 and retryReads=true. If testing against a sharded deployment, be sure to connect
to only a single mongos.

1. Create a client with maxPoolSize=1 and retryReads=true. If testing against a
sharded deployment, be sure to connect to only a single mongos.
2. Enable the following failpoint:

2. Enable the following failpoint::
```
{
configureFailPoint: "failCommand",
mode: { times: 1 },
data: {
failCommands: ["find"],
errorCode: 91,
blockConnection: true,
blockTimeMS: 1000
}
}
```

{
configureFailPoint: "failCommand",
mode: { times: 1 },
data: {
failCommands: ["find"],
errorCode: 91,
blockConnection: true,
blockTimeMS: 1000
}
}
3. Start two threads and attempt to perform a `findOne` simultaneously on both.

3. Start two threads and attempt to perform a ``findOne`` simultaneously on both.

4. Verify that both ``findOne`` attempts succeed.
4. Verify that both `findOne` attempts succeed.

5. Via CMAP monitoring, assert that the first check out succeeds.

6. Via CMAP monitoring, assert that a PoolClearedEvent is then emitted.

7. Via CMAP monitoring, assert that the second check out then fails due to a
connection error.
7. Via CMAP monitoring, assert that the second check out then fails due to a connection error.

8. Via Command Monitoring, assert that exactly three ``find`` CommandStartedEvents
were observed in total.
8. Via Command Monitoring, assert that exactly three `find` CommandStartedEvents were observed in total.

9. Disable the failpoint.

2. Retrying Reads in a Sharded Cluster
--------------------------------------

These tests will be used to ensure drivers properly retry reads on a different
mongos.
### 2. Retrying Reads in a Sharded Cluster

Note: this test cannot reliably distinguish "retry on a different mongos due to
server deprioritization" (the behavior intended to be tested) from "retry on a
different mongos due to normal SDAM behavior of randomized suitable server
selection". Verify relevant code paths are correctly executed by the tests using
external means such as a logging, debugger, code coverage tool, etc.
These tests will be used to ensure drivers properly retry reads on a different mongos.

2.1 Retryable Reads Are Retried on a Different mongos When One is Available
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Note: this test cannot reliably distinguish "retry on a different mongos due to server deprioritization" (the behavior
intended to be tested) from "retry on a different mongos due to normal SDAM behavior of randomized suitable server
selection". Verify relevant code paths are correctly executed by the tests using external means such as a logging,
debugger, code coverage tool, etc.

This test MUST be executed against a sharded cluster that has at least two
mongos instances, supports ``retryReads=true``, and has enabled the
``configureFailPoint`` command (MongoDB 4.2+).
#### 2.1 Retryable Reads Are Retried on a Different mongos When One is Available

1. Create two clients ``s0`` and ``s1`` that each connect to a single mongos
from the sharded cluster. They must not connect to the same mongos.
This test MUST be executed against a sharded cluster that has at least two mongos instances, supports `retryReads=true`,
and has enabled the `configureFailPoint` command (MongoDB 4.2+).

2. Configure the following fail point for both ``s0`` and ``s1``::
1. Create two clients `s0` and `s1` that each connect to a single mongos from the sharded cluster. They must not connect
to the same mongos.

{
configureFailPoint: "failCommand",
mode: { times: 1 },
data: {
failCommands: ["find"],
errorCode: 6
}
}
2. Configure the following fail point for both `s0` and `s1`:

3. Create a client ``client`` with ``retryReads=true`` that connects to the
cluster using the same two mongoses as ``s0`` and ``s1``.
```
{
configureFailPoint: "failCommand",
mode: { times: 1 },
data: {
failCommands: ["find"],
errorCode: 6
}
}
```

4. Enable failed command event monitoring for ``client``.
3. Create a client `client` with `retryReads=true` that connects to the cluster using the same two mongoses as `s0` and
`s1`.

5. Execute a ``find`` command with ``client``. Assert that the command failed.
4. Enable failed command event monitoring for `client`.

6. Assert that two failed command events occurred. Assert that both events
occurred on different mongoses.
5. Execute a `find` command with `client`. Assert that the command failed.

7. Disable the fail point on both ``s0`` and ``s1``.
6. Assert that two failed command events occurred. Assert that both events occurred on different mongoses.

7. Disable the fail point on both `s0` and `s1`.

2.2 Retryable Reads Are Retried on the Same mongos When No Others are Available
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#### 2.2 Retryable Reads Are Retried on the Same mongos When No Others are Available

This test MUST be executed against a sharded cluster that supports
``retryReads=true`` and has enabled the ``configureFailPoint`` command
(MongoDB 4.2+).
This test MUST be executed against a sharded cluster that supports `retryReads=true` and has enabled the
`configureFailPoint` command (MongoDB 4.2+).

1. Create a client ``s0`` that connects to a single mongos from the cluster.
1. Create a client `s0` that connects to a single mongos from the cluster.

2. Configure the following fail point for ``s0``::
2. Configure the following fail point for `s0`:

{
configureFailPoint: "failCommand",
mode: { times: 1 },
data: {
failCommands: ["find"],
errorCode: 6
}
}
```
{
configureFailPoint: "failCommand",
mode: { times: 1 },
data: {
failCommands: ["find"],
errorCode: 6
}
}
```

3. Create a client ``client`` with ``directConnection=false`` (when not set by
default) and ``retryReads=true`` that connects to the cluster using the same
single mongos as ``s0``.
3. Create a client `client` with `directConnection=false` (when not set by default) and `retryReads=true` that connects
to the cluster using the same single mongos as `s0`.

4. Enable succeeded and failed command event monitoring for ``client``.
4. Enable succeeded and failed command event monitoring for `client`.

5. Execute a ``find`` command with ``client``. Assert that the command
succeeded.
5. Execute a `find` command with `client`. Assert that the command succeeded.

6. Assert that exactly one failed command event and one succeeded command event
occurred. Assert that both events occurred on the same mongos.
6. Assert that exactly one failed command event and one succeeded command event occurred. Assert that both events
occurred on the same mongos.

7. Disable the fail point on ``s0``.
7. Disable the fail point on `s0`.

## Changelog

Changelog
=========
- 2024-04-30: Migrated from reStructuredText to Markdown.

:2024-03-06: Convert legacy retryable reads tests to unified format.
- 2024-03-06: Convert legacy retryable reads tests to unified format.

:2024-02-21: Update mongos redirection prose tests to workaround SDAM behavior
preventing execution of deprioritization code paths.
- 2024-02-21: Update mongos redirection prose tests to workaround SDAM behavior\
preventing execution of
deprioritization code paths.

:2023-08-26: Add prose tests for retrying in a sharded cluster.
- 2023-08-26: Add prose tests for retrying in a sharded cluster.

:2022-04-22: Clarifications to ``serverless`` and ``useMultipleMongoses``.
- 2022-04-22: Clarifications to `serverless` and `useMultipleMongoses`.

:2022-01-10: Create legacy and unified subdirectories for new unified tests
- 2022-01-10: Create legacy and unified subdirectories for new unified tests

:2021-08-27: Clarify behavior of ``useMultipleMongoses`` for ``LoadBalanced`` topologies.
- 2021-08-27: Clarify behavior of `useMultipleMongoses` for `LoadBalanced` topologies.

:2019-03-19: Add top-level ``runOn`` field to denote server version and/or
topology requirements requirements for the test file. Removes the
``minServerVersion`` and ``topology`` top-level fields, which are
now expressed within ``runOn`` elements.
- 2019-03-19: Add top-level `runOn` field to denote server version and/or\
topology requirements requirements for the
test file. Removes the `minServerVersion` and `topology` top-level fields, which are now expressed within `runOn`
elements.

Add test-level ``useMultipleMongoses`` field.
Add test-level `useMultipleMongoses` field.

:2020-09-16: Suggest lowering heartbeatFrequencyMS in addition to minHeartbeatFrequencyMS.
- 2020-09-16: Suggest lowering heartbeatFrequencyMS in addition to minHeartbeatFrequencyMS.

:2021-03-23: Add prose test for retrying PoolClearedErrors
- 2021-03-23: Add prose test for retrying PoolClearedErrors

:2021-04-29: Add ``load-balanced`` to test topology requirements.
- 2021-04-29: Add `load-balanced` to test topology requirements.
Loading

0 comments on commit 6c7e14e

Please sign in to comment.