diff --git a/source/retryable-reads/tests/README.rst b/source/retryable-reads/tests/README.rst index 9e8eb0806b..ec41911497 100644 --- a/source/retryable-reads/tests/README.rst +++ b/source/retryable-reads/tests/README.rst @@ -9,192 +9,18 @@ Retryable Reads Tests Introduction ============ -The YAML and JSON files in the ``legacy`` and ``unified`` sub-directories are platform-independent tests -that drivers can use to prove their conformance to the Retryable Reads spec. Tests in the -``unified`` directory are written using the `Unified Test Format <../../unified-test-format/unified-test-format.md>`_. -Tests in the ``legacy`` directory are written using the format described below. +The YAML and JSON files in this directory are platform-independent tests meant +to exercise a driver's implementation of retryable reads. These tests utilize +the [Unified Test Format](../../unified-test-format/unified-test-format.md). -Prose tests, which are not easily expressed in YAML, are also presented +Several prose tests, which are not easily expressed in YAML, are also presented in this file. Those tests will need to be manually implemented by each driver. -Tests will require a MongoClient created with options defined in the tests. -Integration tests will require a running MongoDB cluster with server versions -4.0 or later. +Prose Tests +=========== -N.B. The spec specifies 3.6 as the minimum server version: however, -``failCommand`` is not supported on 3.6, so for now, testing requires MongoDB -4.0. Once `DRIVERS-560`_ is resolved, we will attempt to adapt its live failure -integration tests to test Retryable Reads on MongoDB 3.6. - -.. _DRIVERS-560: https://jira.mongodb.org/browse/DRIVERS-560 - -Server Fail Point -================= - -See: `Server Fail Point`_ in the Transactions spec test suite. - -.. _Server Fail Point: ../../transactions/tests#server-fail-point - -Disabling Fail Point after Test Execution ------------------------------------------ - -After each test that configures a fail point, drivers should disable the -``failCommand`` fail point to avoid spurious failures in -subsequent tests. The fail point may be disabled like so:: - - db.runCommand({ - configureFailPoint: "failCommand", - mode: "off" - }); - -Network Error Tests -=================== - -Network error tests are expressed in YAML and should be run against a standalone, -shard cluster, or single-node replica set. - - -Test Format ------------ - -Each YAML file has the following keys: - -- ``runOn`` (optional): An array of server version and/or topology requirements - for which the tests can be run. If the test environment satisfies one or more - of these requirements, the tests may be executed; otherwise, this file should - be skipped. If this field is omitted, the tests can be assumed to have no - particular requirements and should be executed. Each element will have some or - all of the following fields: - - - ``minServerVersion`` (optional): The minimum server version (inclusive) - required to successfully run the tests. If this field is omitted, it should - be assumed that there is no lower bound on the required server version. - - - ``maxServerVersion`` (optional): The maximum server version (inclusive) - against which the tests can be run successfully. If this field is omitted, - it should be assumed that there is no upper bound on the required server - version. - - - ``topology`` (optional): An array of server topologies against which the - tests can be run successfully. Valid topologies are "single", - "replicaset", "sharded", and "load-balanced". If this field is omitted, - the default is all topologies (i.e. ``["single", "replicaset", "sharded", - "load-balanced"]``). - - - ``serverless``: (optional): Whether or not the test should be run on Atlas - Serverless instances. Valid values are "require", "forbid", and "allow". If - "require", the test MUST only be run on Atlas Serverless instances. If - "forbid", the test MUST NOT be run on Atlas Serverless instances. If omitted - or "allow", this option has no effect. - - The test runner MUST be informed whether or not Atlas Serverless is being - used in order to determine if this requirement is met (e.g. through an - environment variable or configuration option). - - Note: the Atlas Serverless proxy imitates mongos, so the test runner is not - capable of determining if Atlas Serverless is in use by issuing commands - such as ``buildInfo`` or ``hello``. Furthermore, connections to Atlas - Serverless use a load balancer, so the topology will appear as - "load-balanced". - -- ``database_name`` and ``collection_name``: Optional. The database and - collection to use for testing. - -- ``bucket_name``: Optional. The GridFS bucket name to use for testing. - -- ``data``: The data that should exist in the collection(s) under test before - each test run. This will typically be an array of documents to be inserted - into the collection under test (i.e. ``collection_name``); however, this field - may also be an object mapping collection names to arrays of documents to be - inserted into the specified collection. - -- ``tests``: An array of tests that are to be run independently of each other. - Each test will have some or all of the following fields: - - - ``description``: The name of the test. - - - ``clientOptions``: Optional, parameters to pass to MongoClient(). - - - ``useMultipleMongoses`` (optional): If ``true``, and the topology type is - ``Sharded``, the MongoClient for this test should be initialized with multiple - mongos seed addresses. If ``false`` or omitted, only a single mongos address - should be specified. - - If ``true``, the topology type is ``LoadBalanced``, and Atlas Serverless is - not being used, the MongoClient for this test should be initialized with the - URI of the load balancer fronting multiple servers. If ``false`` or omitted, - the MongoClient for this test should be initialized with the URI of the load - balancer fronting a single server. - - ``useMultipleMongoses`` only affects ``Sharded`` and ``LoadBalanced`` - topologies (excluding Atlas Serverless). - - - ``skipReason``: Optional, string describing why this test should be skipped. - - - ``failPoint``: Optional, a server fail point to enable, expressed as the - configureFailPoint command to run on the admin database. - - - ``operations``: An array of documents describing an operation to be - executed. Each document has the following fields: - - - ``name``: The name of the operation on ``object``. - - - ``object``: The name of the object to perform the operation on. Can be - "database", "collection", "client", or "gridfsbucket." - - - ``arguments``: Optional, the names and values of arguments. - - - ``result``: Optional. The return value from the operation, if any. This - field may be a scalar (e.g. in the case of a count), a single document, or - an array of documents in the case of a multi-document read. - - - ``error``: Optional. If ``true``, the test should expect an error or - exception. - - - ``expectations``: Optional list of command-started events. - -GridFS Tests ------------- - -GridFS tests are denoted by when the YAML file contains ``bucket_name``. -The ``data`` field will also be an object, which maps collection names -(e.g. ``fs.files``) to an array of documents that should be inserted into -the specified collection. - -``fs.files`` and ``fs.chunks`` should be created in the database -specified by ``database_name``. This could be done via inserts or by -creating GridFSBuckets—using the GridFS ``bucketName`` (see -`GridFSBucket spec`_) specified by ``bucket_name`` field in the YAML -file—and calling ``upload_from_stream_with_id`` with the appropriate -data. - -``Download`` tests should be tested against ``GridFS.download_to_stream``. -``DownloadByName`` tests should be tested against -``GridFS.download_to_stream_by_name``. - - -.. _GridFSBucket spec: https://github.com/mongodb/specifications/blob/master/source/gridfs/gridfs-spec.rst#configurable-gridfsbucket-class - - -Speeding Up Tests ------------------ - -Drivers can greatly reduce the execution time of tests by setting `heartbeatFrequencyMS`_ -and `minHeartbeatFrequencyMS`_ (internally) to a small value (e.g. 5ms), below what -is normally permitted in the SDAM spec. If a test specifies an explicit value for -heartbeatFrequencyMS (e.g. client or URI options), drivers MUST use that value. - -.. _minHeartbeatFrequencyMS: ../../server-discovery-and-monitoring/server-discovery-and-monitoring.rst#minheartbeatfrequencyms -.. _heartbeatFrequencyMS: ../../server-discovery-and-monitoring/server-discovery-and-monitoring.rst#heartbeatfrequencyms - -Optional Enumeration Commands -============================= - -A driver only needs to test the optional enumeration commands it has chosen to -implement (e.g. ``Database.listCollectionNames()``). - -PoolClearedError Retryability Test -================================== +1. PoolClearedError Retryability Test +------------------------------------- This test will be used to ensure drivers properly retry after encountering PoolClearedErrors. It MUST be implemented by any driver that implements the CMAP specification. @@ -232,8 +58,8 @@ This test requires MongoDB 4.2.9+ for ``blockConnection`` support in the failpoi 9. Disable the failpoint. -Retrying Reads in a Sharded Cluster -=================================== +2. Retrying Reads in a Sharded Cluster +-------------------------------------- These tests will be used to ensure drivers properly retry reads on a different mongos. @@ -244,8 +70,8 @@ different mongos due to normal SDAM behavior of randomized suitable server selection". Verify relevant code paths are correctly executed by the tests using external means such as a logging, debugger, code coverage tool, etc. -Retryable Reads Are Retried on a Different mongos When One is Available ------------------------------------------------------------------------ +2.1 Retryable Reads Are Retried on a Different mongos When One is Available +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This test MUST be executed against a sharded cluster that has at least two mongos instances, supports ``retryReads=true``, and has enabled the @@ -278,8 +104,8 @@ mongos instances, supports ``retryReads=true``, and has enabled the 7. Disable the fail point on both ``s0`` and ``s1``. -Retryable Reads Are Retried on the Same mongos When No Others are Available ---------------------------------------------------------------------------- +2.2 Retryable Reads Are Retried on the Same mongos When No Others are Available +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This test MUST be executed against a sharded cluster that supports ``retryReads=true`` and has enabled the ``configureFailPoint`` command @@ -316,10 +142,12 @@ This test MUST be executed against a sharded cluster that supports Changelog ========= +:2024-03-06: Convert legacy retryable reads tests to unified format. + :2024-02-21: Update mongos redirection prose tests to workaround SDAM behavior preventing execution of deprioritization code paths. -:2023-08-26 Add prose tests for retrying in a sharded cluster. +:2023-08-26: Add prose tests for retrying in a sharded cluster. :2022-04-22: Clarifications to ``serverless`` and ``useMultipleMongoses``.