diff --git a/scripts/migrate_to_md.py b/scripts/migrate_to_md.py index 0a94f256ce..0bd44dc38b 100644 --- a/scripts/migrate_to_md.py +++ b/scripts/migrate_to_md.py @@ -43,6 +43,13 @@ # Pre-process the file. for (i, line) in enumerate(lines): + # Replace curly quotes with regular quotes. + line = line.replace('”', '"') + line = line.replace('“', '"') + line = line.replace('’', "'") + line = line.replace('‘', "'") + lines[i] = line + # Replace the colon fence blocks with bullets, # e.g. :Status:, :deprecated:, :changed:. # This also includes the changelog entries. @@ -59,12 +66,6 @@ if line.strip() == '.. contents::': lines[i] = '' - # Replace curly quotes with regular quotes. - line = line.replace('”', '"') - line = line.replace('“', '"') - line = line.replace('’', "'") - line = line.replace('‘', "'") - # Run pandoc and capture output. proc = subprocess.Popen(['pandoc', '-f', 'rst', '-t', 'gfm'], stdin=subprocess.PIPE, stdout=subprocess.PIPE) data = ''.join(lines).encode('utf8') @@ -128,15 +129,15 @@ suffix = f'\S*/{target}' rel_pattern = re.compile(f'(\.\.{suffix})') md_pattern = re.compile(f'(\(http{suffix})') -rst_pattern = re.compile(f'(`_. +The connection string parser in the driver is extended with a new protocol `mongodb+srv` as a logical pre-processing +step before it considers the connection string and SDAM specifications. In this protocol, the comma separated list of +host names is replaced with a single host name. The format is: -Specification -============= +``` +mongodb+srv://{hostname}.{domainname}/{options} +``` -Connection String Format ------------------------- +`{options}` refers to the optional elements from the [Connection String](../connection-string/connection-string-spec.md) +specification following the `Host Information`. This includes the `Auth database` and `Connection Options`. -The connection string parser in the driver is extended with a new protocol -``mongodb+srv`` as a logical pre-processing step before it considers the -connection string and SDAM specifications. In this protocol, the comma -separated list of host names is replaced with a single host name. The -format is:: +### MongoClient Configuration - mongodb+srv://{hostname}.{domainname}/{options} +#### srvMaxHosts +This option is used to limit the number of mongos connections that may be created for sharded topologies. This option +limits the number of SRV records used to populate the seedlist during initial discovery, as well as the number of +additional hosts that may be added during +[SRV polling](../polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst). This option +requires a non-negative integer and defaults to zero (i.e. no limit). This option MUST only be configurable at the level +of a `MongoClient`. -``{options}`` refers to the optional elements from the `Connection String`_ -specification following the ``Host Information``. This includes the ``Auth -database`` and ``Connection Options``. +#### srvServiceName +This option specifies a valid SRV service name according to +[RFC 6335](https://datatracker.ietf.org/doc/html/rfc6335#section-5.1), with the exception that it may exceed 15 +characters as long as the 63rd (62nd with prepended underscore) character DNS query limit is not surpassed. This option +requires a string value and defaults to "mongodb". This option MUST only be configurable at the level of a +`MongoClient`. -MongoClient Configuration -------------------------- +#### URI Validation -srvMaxHosts -~~~~~~~~~~~ +The driver MUST report an error if either the `srvServiceName` or `srvMaxHosts` URI options are specified with a non-SRV +URI (i.e. scheme other than `mongodb+srv`). The driver MUST allow specifying the `srvServiceName` and `srvMaxHosts` URI +options with an SRV URI (i.e. `mongodb+srv` scheme). -This option is used to limit the number of mongos connections that may be -created for sharded topologies. This option limits the number of SRV records -used to populate the seedlist during initial discovery, as well as the number of -additional hosts that may be added during -`SRV polling <../polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst>`_. -This option requires a non-negative integer and defaults to zero (i.e. no -limit). This option MUST only be configurable at the level of a ``MongoClient``. +If `srvMaxHosts` is a positive integer, the driver MUST throw an error in the following cases: +- The connection string contains a `replicaSet` option. +- The connection string contains a `loadBalanced` option with a value of `true`. -srvServiceName -~~~~~~~~~~~~~~ +When validating URI options, the driver MUST first do the SRV and TXT lookup and then perform the validation. For +drivers that do SRV lookup asynchronously this may result in a `MongoClient` being instantiated but erroring later +during operation execution. -This option specifies a valid SRV service name according to -`RFC 6335 `_, with -the exception that it may exceed 15 characters as long as the 63rd (62nd with -prepended underscore) character DNS query limit is not surpassed. This option -requires a string value and defaults to "mongodb". This option MUST only be -configurable at the level of a ``MongoClient``. +### Seedlist Discovery +#### Validation Before Querying DNS -URI Validation -~~~~~~~~~~~~~~ +It is an error to specify a port in a connection string with the `mongodb+srv` protocol, and the driver MUST raise a +parse error and MUST NOT do DNS resolution or contact hosts. -The driver MUST report an error if either the ``srvServiceName`` or -``srvMaxHosts`` URI options are specified with a non-SRV URI (i.e. scheme other -than ``mongodb+srv``). The driver MUST allow specifying the ``srvServiceName`` -and ``srvMaxHosts`` URI options with an SRV URI (i.e. ``mongodb+srv`` scheme). +It is an error to specify more than one host name in a connection string with the `mongodb+srv` protocol, and the driver +MUST raise a parse error and MUST NOT do DNS resolution or contact hosts. -If ``srvMaxHosts`` is a positive integer, the driver MUST throw an error in the -following cases: +A driver MUST verify that in addition to the `{hostname}`, the `{domainname}` consists of at least two parts: the domain +name, and a TLD. Drivers MUST raise an error and MUST NOT contact the DNS server to obtain SRV (or TXT records) if the +full URI does not consist of at least three parts. -- The connection string contains a ``replicaSet`` option. -- The connection string contains a ``loadBalanced`` option with a value of - ``true``. +If `mongodb+srv` is used, a driver MUST implicitly also enable TLS. Clients can turn this off by passing `tls=false` in +either the Connection String, or options passed in as parameters in code to the MongoClient constructor (or equivalent +API for each driver), but not through a TXT record (discussed in a later section). -When validating URI options, the driver MUST first do the SRV and TXT lookup and -then perform the validation. For drivers that do SRV lookup asynchronously this -may result in a ``MongoClient`` being instantiated but erroring later during -operation execution. +#### Querying DNS +In this preprocessing step, the driver will query the DNS server for SRV records on `{hostname}.{domainname}`, prefixed +with the SRV service name and protocol. The SRV service name is provided in the `srvServiceName` URI option and defaults +to `mongodb`. The protocol is always `tcp`. After prefixing, the URI should look like: +`_{srvServiceName}._tcp.{hostname}.{domainname}`. This DNS query is expected to respond with one or more SRV records. -Seedlist Discovery ------------------- +The priority and weight fields in returned SRV records MUST be ignored. -Validation Before Querying DNS -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If the DNS result returns no SRV records, or no records at all, or a DNS error happens, an error MUST be raised +indicating that the URI could not be used to find hostnames. The error SHALL include the reason why they could not be +found. -It is an error to specify a port in a connection string with the -``mongodb+srv`` protocol, and the driver MUST raise a parse error and MUST NOT -do DNS resolution or contact hosts. +A driver MUST verify that the host names returned through SRV records have the same parent `{domainname}`. Drivers MUST +raise an error and MUST NOT initiate a connection to any returned host name which does not share the same +`{domainname}`. -It is an error to specify more than one host name in a connection string with -the ``mongodb+srv`` protocol, and the driver MUST raise a parse error and MUST -NOT do DNS resolution or contact hosts. +The driver MUST NOT attempt to connect to any hosts until the DNS query has returned its results. -A driver MUST verify that in addition to the ``{hostname}``, the -``{domainname}`` consists of at least two parts: the domain name, and a TLD. -Drivers MUST raise an error and MUST NOT contact the DNS server to obtain SRV -(or TXT records) if the full URI does not consist of at least three parts. +If `srvMaxHosts` is zero or greater than or equal to the number of hosts in the DNS result, the driver MUST populate the +seedlist with all hosts. -If ``mongodb+srv`` is used, a driver MUST implicitly also enable TLS. Clients -can turn this off by passing ``tls=false`` in either the Connection String, -or options passed in as parameters in code to the MongoClient constructor (or -equivalent API for each driver), but not through a TXT record (discussed in a -later section). +If `srvMaxHosts` is greater than zero and less than the number of hosts in the DNS result, the driver MUST randomly +select that many hosts and use them to populate the seedlist. Drivers SHOULD use the +[Fisher-Yates shuffle](https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle#The_modern_algorithm) for +randomization. +### Default Connection String Options -Querying DNS -~~~~~~~~~~~~ +As a second preprocessing step, a Client MUST also query the DNS server for TXT records on `{hostname}.{domainname}`. If +available, a TXT record provides default connection string options. The maximum length of a TXT record string is 255 +characters, but there can be multiple strings per TXT record. A Client MUST support multiple TXT record strings and +concatenate them as if they were one single string in the order they are defined in each TXT record. The order of +multiple character strings in each TXT record is guaranteed. A Client MUST NOT allow multiple TXT records for the same +host name and MUST raise an error when multiple TXT records are encountered. -In this preprocessing step, the driver will query the DNS server for SRV records -on ``{hostname}.{domainname}``, prefixed with the SRV service name and protocol. -The SRV service name is provided in the ``srvServiceName`` URI option and -defaults to ``mongodb``. The protocol is always ``tcp``. After prefixing, the -URI should look like: ``_{srvServiceName}._tcp.{hostname}.{domainname}``. This -DNS query is expected to respond with one or more SRV records. +Information returned within a TXT record is a simple URI string, just like the `{options}` in a connection string. -The priority and weight fields in returned SRV records MUST be ignored. +A Client MUST only support the `authSource`, `replicaSet`, and `loadBalanced` options through a TXT record, and MUST +raise an error if any other option is encountered. Although using `mongodb+srv://` implicitly enables TLS, a Client MUST +NOT allow the `ssl` option to be set through a TXT record option. -If the DNS result returns no SRV records, or no records at all, or a DNS error -happens, an error MUST be raised indicating that the URI could not be used to -find hostnames. The error SHALL include the reason why they could not be -found. +TXT records MAY be queried either before, in parallel, or after SRV records. Clients MUST query both the SRV and the TXT +records before attempting any connection to MongoDB. -A driver MUST verify that the host names returned through SRV records have the -same parent ``{domainname}``. Drivers MUST raise an error and MUST NOT -initiate a connection to any returned host name which does not share the same -``{domainname}``. +A Client MUST use options specified in the Connection String, and options passed in as parameters in code to the +MongoClient constructor (or equivalent API for each driver), to override options provided through TXT records. -The driver MUST NOT attempt to connect to any hosts until the DNS query has -returned its results. +If any connection string option in a TXT record is incorrectly formatted, a Client MUST throw a parse exception. -If ``srvMaxHosts`` is zero or greater than or equal to the number of hosts in -the DNS result, the driver MUST populate the seedlist with all hosts. +This specification does not change the behaviour of handling unknown keys or incorrect values as is set out in the +[Connection String spec](../connection-string/connection-string-spec.md#defining-connection-options). Unknown keys or +incorrect values in default options specified through TXT records MUST be handled in the same way as unknown keys or +incorrect values directly specified through a Connection String. For example, if a driver that does not support the +`authSource` option finds `authSource=db` in a TXT record, it MUST handle the unknown option according to the rules in +the Connection String spec. -If ``srvMaxHosts`` is greater than zero and less than the number of hosts in the -DNS result, the driver MUST randomly select that many hosts and use them to -populate the seedlist. Drivers SHOULD use the `Fisher-Yates shuffle`_ for -randomization. +### CNAME not supported -.. _`Fisher-Yates shuffle`: https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle#The_modern_algorithm +The use of DNS CNAME records is not supported. Clients MUST NOT check for a CNAME record on `{hostname}.{domainname}`. A +system's DNS resolver could transparently handle CNAME, but because of how clients validate records returned from SRV +queries, use of CNAME could break validation. Seedlist discovery therefore does not recommend or support the use of +CNAME records in concert with SRV or TXT records. +## Example -Default Connection String Options ---------------------------------- +If we provide the following URI: -As a second preprocessing step, a Client MUST also query the DNS server for -TXT records on ``{hostname}.{domainname}``. If available, a TXT record -provides default connection string options. The maximum length of a TXT record -string is 255 -characters, but there can be multiple strings per TXT record. A Client MUST -support multiple TXT record strings and concatenate them as if they were one -single string in the order they are defined in each TXT record. The order of -multiple character strings in each TXT record is guaranteed. -A Client MUST NOT allow multiple TXT records for the same host name and MUST -raise an error when multiple TXT records are encountered. +``` +mongodb+srv://server.mongodb.com/ +``` -Information returned within a TXT record is a simple URI string, just like -the ``{options}`` in a connection string. +The driver needs to request the DNS server for the SRV record `_mongodb._tcp.server.mongodb.com`. This could return: -A Client MUST only support the ``authSource``, ``replicaSet``, and ``loadBalanced`` -options through a TXT record, and MUST raise an error if any other option is -encountered. Although using ``mongodb+srv://`` implicitly enables TLS, a -Client MUST NOT allow the ``ssl`` option to be set through a TXT record -option. +``` +Record TTL Class Priority Weight Port Target +_mongodb._tcp.server.mongodb.com. 86400 IN SRV 0 5 27317 mongodb1.mongodb.com. +_mongodb._tcp.server.mongodb.com. 86400 IN SRV 0 5 27017 mongodb2.mongodb.com. +``` -TXT records MAY be queried either before, in parallel, or after SRV records. -Clients MUST query both the SRV and the TXT records before attempting any -connection to MongoDB. +The returned host names (`mongodb1.mongodb.com` and `mongodb2.mongodb.com`) must share the same parent domain name +(`mongodb.com`) as the provided host name (`server.mongodb.com`). -A Client MUST use options specified in the Connection String, and options -passed in as parameters in code to the MongoClient constructor (or equivalent -API for each driver), to override options provided through TXT records. +The driver also needs to request the DNS server for the TXT records on `server.mongodb.com`. This could return: -.. _`Connection String spec`: ../connection-string/connection-string-spec.md#defining-connection-options +``` +Record TTL Class Text +server.mongodb.com. 86400 IN TXT "replicaSet=replProduction&authSource=authDB" +``` -If any connection string option in a TXT record is incorrectly formatted, a -Client MUST throw a parse exception. +From the DNS results, the driver now MUST treat the host information as if the following URI was used instead: -This specification does not change the behaviour of handling unknown keys or -incorrect values as is set out in the `Connection String spec`_. Unknown keys -or incorrect values in default options specified through TXT records MUST be -handled in the same way as unknown keys or incorrect values directly specified -through a Connection String. For example, if a driver that does not support -the ``authSource`` option finds ``authSource=db`` in a TXT record, it MUST handle -the unknown option according to the rules in the Connection String spec. +``` +mongodb://mongodb1.mongodb.com:27317,mongodb2.mongodb.com:27107/?ssl=true&replicaSet=replProduction&authSource=authDB +``` -CNAME not supported -------------------- +If we provide the following URI with the same DNS (SRV and TXT) records: -The use of DNS CNAME records is not supported. Clients MUST NOT check for a -CNAME record on ``{hostname}.{domainname}``. A system's DNS resolver could -transparently handle CNAME, but because of how clients validate records -returned from SRV queries, use of CNAME could break validation. Seedlist -discovery therefore does not recommend or support the use of CNAME records in -concert with SRV or TXT records. +``` +mongodb+srv://server.mongodb.com/?authSource=otherDB +``` -Example -======= +Then the default in the TXT record for `authSource` is not used as the value in the connection string overrides it. The +Client MUST treat the host information as if the following URI was used instead: -If we provide the following URI:: +``` +mongodb://mongodb1.mongodb.com:27317,mongodb2.mongodb.com:27107/?ssl=true&replicaSet=replProduction&authSource=otherDB +``` - mongodb+srv://server.mongodb.com/ +## Test Plan -The driver needs to request the DNS server for the SRV record -``_mongodb._tcp.server.mongodb.com``. This could return:: +See README.md in the accompanying [test directory](tests). - Record TTL Class Priority Weight Port Target - _mongodb._tcp.server.mongodb.com. 86400 IN SRV 0 5 27317 mongodb1.mongodb.com. - _mongodb._tcp.server.mongodb.com. 86400 IN SRV 0 5 27017 mongodb2.mongodb.com. +Additionally, see the `mongodb+srv` test `invalid-uris.yml` in the +[Connection String Spec tests](../connection-string/tests). -The returned host names (``mongodb1.mongodb.com`` and -``mongodb2.mongodb.com``) must share the same parent domain name -(``mongodb.com``) as the provided host name (``server.mongodb.com``). +## Motivation -The driver also needs to request the DNS server for the TXT records on -``server.mongodb.com``. This could return:: +Several of our users have asked for this through tickets: - Record TTL Class Text - server.mongodb.com. 86400 IN TXT "replicaSet=replProduction&authSource=authDB" +- +- +- -From the DNS results, the driver now MUST treat the host information as if the -following URI was used instead:: +## Design Rationale - mongodb://mongodb1.mongodb.com:27317,mongodb2.mongodb.com:27107/?ssl=true&replicaSet=replProduction&authSource=authDB +The design specifically calls for a pre-processing stage of the processing of connection URLs to minimize the impact on +existing functionality. -If we provide the following URI with the same DNS (SRV and TXT) records:: +## Justifications - mongodb+srv://server.mongodb.com/?authSource=otherDB +### Why Are Multiple Key-Value Pairs Allowed in One TXT Record? -Then the default in the TXT record for ``authSource`` is not used as -the value in the connection string overrides it. The Client MUST treat the host -information as if the following URI was used instead:: +One could imagine an alternative design in which each TXT record would allow only one URI option. No `&` character would +be allowed as a delimiter within TXT records. - mongodb://mongodb1.mongodb.com:27317,mongodb2.mongodb.com:27107/?ssl=true&replicaSet=replProduction&authSource=otherDB +In this spec we allow multiple key-value pairs within one TXT record, delimited by `&`, because it will be common for +all options to fit in a single 255-character TXT record, and it is much more convenient to configure one record in this +case than to configure several. -Test Plan -========= +Secondly, in some cases the order in which options occur is important. For example, readPreferenceTags can appear both +multiple times, and the order in which they appear is significant. Because DNS servers may return TXT records in any +order, it is only possible to guarantee the order in which readPreferenceTags keys appear by having them in the same TXT +record. -See README.rst in the accompanying `test directory`_. +### Why Is There No Mention of UTF-8 Characters? -.. _`test directory`: tests +Although DNS TXT records allow any octet to exist in its value, many DNS providers do not allow non-ASCII characters to +be configured. As it is unlikely that any option names or values in the connection string have non-ASCII characters, we +left the behaviour of supporting UTF-8 characters as unspecified. -Additionally, see the ``mongodb+srv`` test ``invalid-uris.yml`` in the `Connection -String Spec tests`_. +## Reference Implementation -.. _`Connection String Spec tests`: ../connection-string/tests +None yet. -Motivation -========== +## Backwards Compatibility -Several of our users have asked for this through tickets: +There are no backwards compatibility concerns. -* ``_ -* ``_ -* ``_ +## Future Work -Design Rationale -================ +In the future we could consider using the priority and weight fields of the SRV records. -The design specifically calls for a pre-processing stage of the processing of -connection URLs to minimize the impact on existing functionality. +## ChangeLog -Justifications -============== +- 2024-03-06: Migrated from reStructuredText to Markdown. -Why Are Multiple Key-Value Pairs Allowed in One TXT Record? ------------------------------------------------------------ +- 2022-10-05: Revise spec front matter and reformat changelog. -One could imagine an alternative design in which each TXT record would allow -only one URI option. No ``&`` character would be allowed as a delimiter within -TXT records. +- 2021-10-14: Add `srvMaxHosts` MongoClient option and restructure Seedlist\ + Discovery section. Improve documentation + for the `srvServiceName` MongoClient option and add a new URI Validation section. -In this spec we allow multiple key-value pairs within one TXT record, -delimited by ``&``, because it will be common for all options to fit in a -single 255-character TXT record, and it is much more convenient to configure -one record in this case than to configure several. +- 2021-09-15: Clarify that service name only defaults to `mongodb`, and should\ + be defined by the `srvServiceName` URI + option. -Secondly, in some cases the order in which options occur is important. For -example, readPreferenceTags can appear both multiple times, and the order in -which they appear is significant. Because DNS servers may return TXT records -in any order, it is only possible to guarantee the order in which -readPreferenceTags keys appear by having them in the same TXT record. +- 2021-04-15: Adding in behaviour for load balancer mode. -Why Is There No Mention of UTF-8 Characters? --------------------------------------------- +- 2019-03-07: Clarify that CNAME is not supported -Although DNS TXT records allow any octet to exist in its value, many DNS -providers do not allow non-ASCII characters to be configured. As it is -unlikely that any option names or values in the connection string have -non-ASCII characters, we left the behaviour of supporting UTF-8 characters as -unspecified. +- 2018-02-08: Clarify that `{options}}` in the [Specification](#specification) section includes\ + all the optional + elements from the Connection String specification. -Reference Implementation -======================== +- 2017-11-21: Add clause that using `mongodb+srv://` implies enabling TLS. Add\ + restriction that only `authSource` and + `replicaSet` are allows in TXT records. Add restriction that only one TXT record is supported share the same parent + domain name as the given host name. -None yet. +- 2017-11-17: Add new rule that indicates that host names in returned SRV records\ + MUST share the same parent domain + name as the given host name. Remove language and tests for non-ASCII characters. -Backwards Compatibility -======================= +- 2017-11-07: Clarified that all parts of listable options such as\ + readPreferenceTags are ignored if they are also + present in options to the MongoClient constructor. Clarified which host names to use for SRV and TXT DNS queries. -There are no backwards compatibility concerns. +- 2017-11-01: Clarified that individual TXT records can have multiple strings. + +- 2017-10-31: Added a clause that specifying two host names with a\ + `mongodb+srv://` URI is not allowed. Added a few + more test cases. + +- 2017-10-18: Removed prohibition of raising DNS related errors when parsing the URI. + +- 2017-10-04: Removed from [Future Work](#future-work) the line about multiple MongoS\ + discovery. The current + specification already allows for it, as multiple host names which are all MongoS servers is already allowed under + SDAM. And this specification does not modify SDAM. Added support for connection string options through TXT records. + +- 2017-09-19: Clarify that host names in `mongodb+srv://` URLs work like normal\ + host specifications. -Future Work -=========== - -In the future we could consider using the priority and weight fields of the -SRV records. - -ChangeLog -========= - -:2022-10-05: Revise spec front matter and reformat changelog. -:2021-10-14: Add ``srvMaxHosts`` MongoClient option and restructure Seedlist - Discovery section. Improve documentation for the ``srvServiceName`` - MongoClient option and add a new URI Validation section. -:2021-09-15: Clarify that service name only defaults to ``mongodb``, and should - be defined by the ``srvServiceName`` URI option. -:2021-04-15: Adding in behaviour for load balancer mode. -:2019-03-07: Clarify that CNAME is not supported -:2018-02-08: Clarify that ``{options}}`` in the Specification_ section includes - all the optional elements from the Connection String specification. -:2017-11-21: Add clause that using ``mongodb+srv://`` implies enabling TLS. Add - restriction that only ``authSource`` and ``replicaSet`` are allows - in TXT records. Add restriction that only one TXT record is - supported share the same parent domain name as the given host name. -:2017-11-17: Add new rule that indicates that host names in returned SRV records - MUST share the same parent domain name as the given host name. - Remove language and tests for non-ASCII characters. -:2017-11-07: Clarified that all parts of listable options such as - readPreferenceTags are ignored if they are also present in options - to the MongoClient constructor. Clarified which host names to use - for SRV and TXT DNS queries. -:2017-11-01: Clarified that individual TXT records can have multiple strings. -:2017-10-31: Added a clause that specifying two host names with a - ``mongodb+srv://`` URI is not allowed. Added a few more test cases. -:2017-10-18: Removed prohibition of raising DNS related errors when parsing the URI. -:2017-10-04: Removed from `Future Work`_ the line about multiple MongoS - discovery. The current specification already allows for it, as - multiple host names which are all MongoS servers is already allowed - under SDAM. And this specification does not modify SDAM. Added - support for connection string options through TXT records. -:2017-09-19: Clarify that host names in ``mongodb+srv://`` URLs work like normal - host specifications. -:2017-09-01: Updated test plan with YAML tests, and moved prose tests for URI - parsing into invalid-uris.yml in the Connection String Spec tests. +- 2017-09-01: Updated test plan with YAML tests, and moved prose tests for URI\ + parsing into invalid-uris.yml in the + Connection String Spec tests. diff --git a/source/initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst b/source/initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst new file mode 100644 index 0000000000..de1d3facee --- /dev/null +++ b/source/initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst @@ -0,0 +1,378 @@ + +.. note:: + This specification has been converted to Markdown and renamed to + `initial-dns-seedlist-discovery.md `_. + + Use the link above to access the latest version of the specification as the + current reStructuredText file will no longer be updated. + +.. role:: javascript(code) + :language: javascript + +============================== +Initial DNS Seedlist Discovery +============================== + +:Status: Accepted +:Minimum Server Version: N/A + +.. contents:: + +-------- + +Abstract +======== + +Presently, seeding a driver with an initial list of ReplicaSet or MongoS +addresses is somewhat cumbersome, requiring a comma-delimited list of host +names to attempt connections to. A standardized answer to this problem exists +in the form of SRV records, which allow administrators to configure a single +domain to return a list of host names. Supporting this feature would assist +our users by decreasing maintenance load, primarily by removing the need to +maintain seed lists at an application level. + +This specification builds on the `Connection String`_ specification. It adds a +new protocol scheme and modifies how the `Host Information`_ is interpreted. + +.. _`Connection String`: ../connection-string/connection-string-spec.md +.. _`Host Information`: ../connection-string/connection-string-spec.md#host-information + +META +==== + +The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, +“SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be +interpreted as described in `RFC 2119 `_. + +Specification +============= + +Connection String Format +------------------------ + +The connection string parser in the driver is extended with a new protocol +``mongodb+srv`` as a logical pre-processing step before it considers the +connection string and SDAM specifications. In this protocol, the comma +separated list of host names is replaced with a single host name. The +format is:: + + mongodb+srv://{hostname}.{domainname}/{options} + + +``{options}`` refers to the optional elements from the `Connection String`_ +specification following the ``Host Information``. This includes the ``Auth +database`` and ``Connection Options``. + + +MongoClient Configuration +------------------------- + +srvMaxHosts +~~~~~~~~~~~ + +This option is used to limit the number of mongos connections that may be +created for sharded topologies. This option limits the number of SRV records +used to populate the seedlist during initial discovery, as well as the number of +additional hosts that may be added during +`SRV polling <../polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst>`_. +This option requires a non-negative integer and defaults to zero (i.e. no +limit). This option MUST only be configurable at the level of a ``MongoClient``. + + +srvServiceName +~~~~~~~~~~~~~~ + +This option specifies a valid SRV service name according to +`RFC 6335 `_, with +the exception that it may exceed 15 characters as long as the 63rd (62nd with +prepended underscore) character DNS query limit is not surpassed. This option +requires a string value and defaults to "mongodb". This option MUST only be +configurable at the level of a ``MongoClient``. + + +URI Validation +~~~~~~~~~~~~~~ + +The driver MUST report an error if either the ``srvServiceName`` or +``srvMaxHosts`` URI options are specified with a non-SRV URI (i.e. scheme other +than ``mongodb+srv``). The driver MUST allow specifying the ``srvServiceName`` +and ``srvMaxHosts`` URI options with an SRV URI (i.e. ``mongodb+srv`` scheme). + +If ``srvMaxHosts`` is a positive integer, the driver MUST throw an error in the +following cases: + +- The connection string contains a ``replicaSet`` option. +- The connection string contains a ``loadBalanced`` option with a value of + ``true``. + +When validating URI options, the driver MUST first do the SRV and TXT lookup and +then perform the validation. For drivers that do SRV lookup asynchronously this +may result in a ``MongoClient`` being instantiated but erroring later during +operation execution. + + +Seedlist Discovery +------------------ + +Validation Before Querying DNS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It is an error to specify a port in a connection string with the +``mongodb+srv`` protocol, and the driver MUST raise a parse error and MUST NOT +do DNS resolution or contact hosts. + +It is an error to specify more than one host name in a connection string with +the ``mongodb+srv`` protocol, and the driver MUST raise a parse error and MUST +NOT do DNS resolution or contact hosts. + +A driver MUST verify that in addition to the ``{hostname}``, the +``{domainname}`` consists of at least two parts: the domain name, and a TLD. +Drivers MUST raise an error and MUST NOT contact the DNS server to obtain SRV +(or TXT records) if the full URI does not consist of at least three parts. + +If ``mongodb+srv`` is used, a driver MUST implicitly also enable TLS. Clients +can turn this off by passing ``tls=false`` in either the Connection String, +or options passed in as parameters in code to the MongoClient constructor (or +equivalent API for each driver), but not through a TXT record (discussed in a +later section). + + +Querying DNS +~~~~~~~~~~~~ + +In this preprocessing step, the driver will query the DNS server for SRV records +on ``{hostname}.{domainname}``, prefixed with the SRV service name and protocol. +The SRV service name is provided in the ``srvServiceName`` URI option and +defaults to ``mongodb``. The protocol is always ``tcp``. After prefixing, the +URI should look like: ``_{srvServiceName}._tcp.{hostname}.{domainname}``. This +DNS query is expected to respond with one or more SRV records. + +The priority and weight fields in returned SRV records MUST be ignored. + +If the DNS result returns no SRV records, or no records at all, or a DNS error +happens, an error MUST be raised indicating that the URI could not be used to +find hostnames. The error SHALL include the reason why they could not be +found. + +A driver MUST verify that the host names returned through SRV records have the +same parent ``{domainname}``. Drivers MUST raise an error and MUST NOT +initiate a connection to any returned host name which does not share the same +``{domainname}``. + +The driver MUST NOT attempt to connect to any hosts until the DNS query has +returned its results. + +If ``srvMaxHosts`` is zero or greater than or equal to the number of hosts in +the DNS result, the driver MUST populate the seedlist with all hosts. + +If ``srvMaxHosts`` is greater than zero and less than the number of hosts in the +DNS result, the driver MUST randomly select that many hosts and use them to +populate the seedlist. Drivers SHOULD use the `Fisher-Yates shuffle`_ for +randomization. + +.. _`Fisher-Yates shuffle`: https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle#The_modern_algorithm + + +Default Connection String Options +--------------------------------- + +As a second preprocessing step, a Client MUST also query the DNS server for +TXT records on ``{hostname}.{domainname}``. If available, a TXT record +provides default connection string options. The maximum length of a TXT record +string is 255 +characters, but there can be multiple strings per TXT record. A Client MUST +support multiple TXT record strings and concatenate them as if they were one +single string in the order they are defined in each TXT record. The order of +multiple character strings in each TXT record is guaranteed. +A Client MUST NOT allow multiple TXT records for the same host name and MUST +raise an error when multiple TXT records are encountered. + +Information returned within a TXT record is a simple URI string, just like +the ``{options}`` in a connection string. + +A Client MUST only support the ``authSource``, ``replicaSet``, and ``loadBalanced`` +options through a TXT record, and MUST raise an error if any other option is +encountered. Although using ``mongodb+srv://`` implicitly enables TLS, a +Client MUST NOT allow the ``ssl`` option to be set through a TXT record +option. + +TXT records MAY be queried either before, in parallel, or after SRV records. +Clients MUST query both the SRV and the TXT records before attempting any +connection to MongoDB. + +A Client MUST use options specified in the Connection String, and options +passed in as parameters in code to the MongoClient constructor (or equivalent +API for each driver), to override options provided through TXT records. + +.. _`Connection String spec`: ../connection-string/connection-string-spec.md#defining-connection-options + +If any connection string option in a TXT record is incorrectly formatted, a +Client MUST throw a parse exception. + +This specification does not change the behaviour of handling unknown keys or +incorrect values as is set out in the `Connection String spec`_. Unknown keys +or incorrect values in default options specified through TXT records MUST be +handled in the same way as unknown keys or incorrect values directly specified +through a Connection String. For example, if a driver that does not support +the ``authSource`` option finds ``authSource=db`` in a TXT record, it MUST handle +the unknown option according to the rules in the Connection String spec. + +CNAME not supported +------------------- + +The use of DNS CNAME records is not supported. Clients MUST NOT check for a +CNAME record on ``{hostname}.{domainname}``. A system's DNS resolver could +transparently handle CNAME, but because of how clients validate records +returned from SRV queries, use of CNAME could break validation. Seedlist +discovery therefore does not recommend or support the use of CNAME records in +concert with SRV or TXT records. + +Example +======= + +If we provide the following URI:: + + mongodb+srv://server.mongodb.com/ + +The driver needs to request the DNS server for the SRV record +``_mongodb._tcp.server.mongodb.com``. This could return:: + + Record TTL Class Priority Weight Port Target + _mongodb._tcp.server.mongodb.com. 86400 IN SRV 0 5 27317 mongodb1.mongodb.com. + _mongodb._tcp.server.mongodb.com. 86400 IN SRV 0 5 27017 mongodb2.mongodb.com. + +The returned host names (``mongodb1.mongodb.com`` and +``mongodb2.mongodb.com``) must share the same parent domain name +(``mongodb.com``) as the provided host name (``server.mongodb.com``). + +The driver also needs to request the DNS server for the TXT records on +``server.mongodb.com``. This could return:: + + Record TTL Class Text + server.mongodb.com. 86400 IN TXT "replicaSet=replProduction&authSource=authDB" + +From the DNS results, the driver now MUST treat the host information as if the +following URI was used instead:: + + mongodb://mongodb1.mongodb.com:27317,mongodb2.mongodb.com:27107/?ssl=true&replicaSet=replProduction&authSource=authDB + +If we provide the following URI with the same DNS (SRV and TXT) records:: + + mongodb+srv://server.mongodb.com/?authSource=otherDB + +Then the default in the TXT record for ``authSource`` is not used as +the value in the connection string overrides it. The Client MUST treat the host +information as if the following URI was used instead:: + + mongodb://mongodb1.mongodb.com:27317,mongodb2.mongodb.com:27107/?ssl=true&replicaSet=replProduction&authSource=otherDB + +Test Plan +========= + +See README.rst in the accompanying `test directory`_. + +.. _`test directory`: tests + +Additionally, see the ``mongodb+srv`` test ``invalid-uris.yml`` in the `Connection +String Spec tests`_. + +.. _`Connection String Spec tests`: ../connection-string/tests + +Motivation +========== + +Several of our users have asked for this through tickets: + +* ``_ +* ``_ +* ``_ + +Design Rationale +================ + +The design specifically calls for a pre-processing stage of the processing of +connection URLs to minimize the impact on existing functionality. + +Justifications +============== + +Why Are Multiple Key-Value Pairs Allowed in One TXT Record? +----------------------------------------------------------- + +One could imagine an alternative design in which each TXT record would allow +only one URI option. No ``&`` character would be allowed as a delimiter within +TXT records. + +In this spec we allow multiple key-value pairs within one TXT record, +delimited by ``&``, because it will be common for all options to fit in a +single 255-character TXT record, and it is much more convenient to configure +one record in this case than to configure several. + +Secondly, in some cases the order in which options occur is important. For +example, readPreferenceTags can appear both multiple times, and the order in +which they appear is significant. Because DNS servers may return TXT records +in any order, it is only possible to guarantee the order in which +readPreferenceTags keys appear by having them in the same TXT record. + +Why Is There No Mention of UTF-8 Characters? +-------------------------------------------- + +Although DNS TXT records allow any octet to exist in its value, many DNS +providers do not allow non-ASCII characters to be configured. As it is +unlikely that any option names or values in the connection string have +non-ASCII characters, we left the behaviour of supporting UTF-8 characters as +unspecified. + +Reference Implementation +======================== + +None yet. + +Backwards Compatibility +======================= + +There are no backwards compatibility concerns. + +Future Work +=========== + +In the future we could consider using the priority and weight fields of the +SRV records. + +ChangeLog +========= + +:2022-10-05: Revise spec front matter and reformat changelog. +:2021-10-14: Add ``srvMaxHosts`` MongoClient option and restructure Seedlist + Discovery section. Improve documentation for the ``srvServiceName`` + MongoClient option and add a new URI Validation section. +:2021-09-15: Clarify that service name only defaults to ``mongodb``, and should + be defined by the ``srvServiceName`` URI option. +:2021-04-15: Adding in behaviour for load balancer mode. +:2019-03-07: Clarify that CNAME is not supported +:2018-02-08: Clarify that ``{options}}`` in the Specification_ section includes + all the optional elements from the Connection String specification. +:2017-11-21: Add clause that using ``mongodb+srv://`` implies enabling TLS. Add + restriction that only ``authSource`` and ``replicaSet`` are allows + in TXT records. Add restriction that only one TXT record is + supported share the same parent domain name as the given host name. +:2017-11-17: Add new rule that indicates that host names in returned SRV records + MUST share the same parent domain name as the given host name. + Remove language and tests for non-ASCII characters. +:2017-11-07: Clarified that all parts of listable options such as + readPreferenceTags are ignored if they are also present in options + to the MongoClient constructor. Clarified which host names to use + for SRV and TXT DNS queries. +:2017-11-01: Clarified that individual TXT records can have multiple strings. +:2017-10-31: Added a clause that specifying two host names with a + ``mongodb+srv://`` URI is not allowed. Added a few more test cases. +:2017-10-18: Removed prohibition of raising DNS related errors when parsing the URI. +:2017-10-04: Removed from `Future Work`_ the line about multiple MongoS + discovery. The current specification already allows for it, as + multiple host names which are all MongoS servers is already allowed + under SDAM. And this specification does not modify SDAM. Added + support for connection string options through TXT records. +:2017-09-19: Clarify that host names in ``mongodb+srv://`` URLs work like normal + host specifications. +:2017-09-01: Updated test plan with YAML tests, and moved prose tests for URI + parsing into invalid-uris.yml in the Connection String Spec tests. diff --git a/source/initial-dns-seedlist-discovery/tests/README.md b/source/initial-dns-seedlist-discovery/tests/README.md index 96593f3864..5b86b22200 100644 --- a/source/initial-dns-seedlist-discovery/tests/README.md +++ b/source/initial-dns-seedlist-discovery/tests/README.md @@ -1,164 +1,137 @@ -==================================== -Initial DNS Seedlist Discovery tests -==================================== +# Initial DNS Seedlist Discovery tests -This directory contains platform-independent tests that drivers can use -to prove their conformance to the Initial DNS Seedlist Discovery spec. +This directory contains platform-independent tests that drivers can use to prove their conformance to the Initial DNS +Seedlist Discovery spec. -Test Setup ----------- +## Test Setup -The tests in the ``replica-set`` directory MUST be executed against a -three-node replica set on localhost ports 27017, 27018, and 27019 with -replica set name ``repl0``. +The tests in the `replica-set` directory MUST be executed against a three-node replica set on localhost ports 27017, +27018, and 27019 with replica set name `repl0`. -The tests in the ``load-balanced`` directory MUST be executed against a -load-balanced sharded cluster with the mongos servers running on localhost ports -27017 and 27018 and ``--loadBalancerPort`` 27050 and 27051, respectively -(corresponding to the script in `drivers-evergreen-tools`_). The load balancers, -shard servers, and config servers may run on any open ports. +The tests in the `load-balanced` directory MUST be executed against a load-balanced sharded cluster with the mongos +servers running on localhost ports 27017 and 27018 and `--loadBalancerPort` 27050 and 27051, respectively (corresponding +to the script in +[drivers-evergreen-tools](https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/run-load-balancer.sh)). +The load balancers, shard servers, and config servers may run on any open ports. -.. _`drivers-evergreen-tools`: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/run-load-balancer.sh - -The tests in the ``sharded`` directory MUST be executed against a sharded -cluster with the mongos servers running on localhost ports 27017 and 27018. -Shard servers and config servers may run on any open ports. +The tests in the `sharded` directory MUST be executed against a sharded cluster with the mongos servers running on +localhost ports 27017 and 27018. Shard servers and config servers may run on any open ports. In all cases, the clusters MUST be started with SSL enabled. -To run the tests that accompany this spec, you need to configure the SRV and -TXT records with a real name server. The following records are required for -these tests:: - - Record TTL Class Address - localhost.test.build.10gen.cc. 86400 IN A 127.0.0.1 - localhost.sub.test.build.10gen.cc. 86400 IN A 127.0.0.1 - - Record TTL Class Port Target - _mongodb._tcp.test1.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test1.test.build.10gen.cc. 86400 IN SRV 27018 localhost.test.build.10gen.cc. - _mongodb._tcp.test2.test.build.10gen.cc. 86400 IN SRV 27018 localhost.test.build.10gen.cc. - _mongodb._tcp.test2.test.build.10gen.cc. 86400 IN SRV 27019 localhost.test.build.10gen.cc. - _mongodb._tcp.test3.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test5.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test6.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test7.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test8.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test10.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test11.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test12.test.build.10gen.cc. 86400 IN SRV 27017 localhost.build.10gen.cc. - _mongodb._tcp.test13.test.build.10gen.cc. 86400 IN SRV 27017 test.build.10gen.cc. - _mongodb._tcp.test14.test.build.10gen.cc. 86400 IN SRV 27017 localhost.not-test.build.10gen.cc. - _mongodb._tcp.test15.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.not-build.10gen.cc. - _mongodb._tcp.test16.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.not-10gen.cc. - _mongodb._tcp.test17.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.not-cc. - _mongodb._tcp.test18.test.build.10gen.cc. 86400 IN SRV 27017 localhost.sub.test.build.10gen.cc. - _mongodb._tcp.test19.test.build.10gen.cc. 86400 IN SRV 27017 localhost.evil.build.10gen.cc. - _mongodb._tcp.test19.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test20.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test21.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _customname._tcp.test22.test.build.10gen.cc 86400 IN SRV 27017 localhost.test.build.10gen.cc. - _mongodb._tcp.test23.test.build.10gen.cc. 86400 IN SRV 8000 localhost.test.build.10gen.cc. - _mongodb._tcp.test24.test.build.10gen.cc. 86400 IN SRV 8000 localhost.test.build.10gen.cc. - - Record TTL Class Text - test5.test.build.10gen.cc. 86400 IN TXT "replicaSet=repl0&authSource=thisDB" - test6.test.build.10gen.cc. 86400 IN TXT "replicaSet=repl0" - test6.test.build.10gen.cc. 86400 IN TXT "authSource=otherDB" - test7.test.build.10gen.cc. 86400 IN TXT "ssl=false" - test8.test.build.10gen.cc. 86400 IN TXT "authSource" - test10.test.build.10gen.cc. 86400 IN TXT "socketTimeoutMS=500" - test11.test.build.10gen.cc. 86400 IN TXT "replicaS" "et=rep" "l0" - test20.test.build.10gen.cc. 86400 IN TXT "loadBalanced=true" - test21.test.build.10gen.cc. 86400 IN TXT "loadBalanced=false" - test24.test.build.10gen.cc. 86400 IN TXT "loadBalanced=true" +To run the tests that accompany this spec, you need to configure the SRV and TXT records with a real name server. The +following records are required for these tests: + +``` +Record TTL Class Address +localhost.test.build.10gen.cc. 86400 IN A 127.0.0.1 +localhost.sub.test.build.10gen.cc. 86400 IN A 127.0.0.1 + +Record TTL Class Port Target +_mongodb._tcp.test1.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test1.test.build.10gen.cc. 86400 IN SRV 27018 localhost.test.build.10gen.cc. +_mongodb._tcp.test2.test.build.10gen.cc. 86400 IN SRV 27018 localhost.test.build.10gen.cc. +_mongodb._tcp.test2.test.build.10gen.cc. 86400 IN SRV 27019 localhost.test.build.10gen.cc. +_mongodb._tcp.test3.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test5.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test6.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test7.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test8.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test10.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test11.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test12.test.build.10gen.cc. 86400 IN SRV 27017 localhost.build.10gen.cc. +_mongodb._tcp.test13.test.build.10gen.cc. 86400 IN SRV 27017 test.build.10gen.cc. +_mongodb._tcp.test14.test.build.10gen.cc. 86400 IN SRV 27017 localhost.not-test.build.10gen.cc. +_mongodb._tcp.test15.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.not-build.10gen.cc. +_mongodb._tcp.test16.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.not-10gen.cc. +_mongodb._tcp.test17.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.not-cc. +_mongodb._tcp.test18.test.build.10gen.cc. 86400 IN SRV 27017 localhost.sub.test.build.10gen.cc. +_mongodb._tcp.test19.test.build.10gen.cc. 86400 IN SRV 27017 localhost.evil.build.10gen.cc. +_mongodb._tcp.test19.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test20.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test21.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_customname._tcp.test22.test.build.10gen.cc 86400 IN SRV 27017 localhost.test.build.10gen.cc. +_mongodb._tcp.test23.test.build.10gen.cc. 86400 IN SRV 8000 localhost.test.build.10gen.cc. +_mongodb._tcp.test24.test.build.10gen.cc. 86400 IN SRV 8000 localhost.test.build.10gen.cc. + +Record TTL Class Text +test5.test.build.10gen.cc. 86400 IN TXT "replicaSet=repl0&authSource=thisDB" +test6.test.build.10gen.cc. 86400 IN TXT "replicaSet=repl0" +test6.test.build.10gen.cc. 86400 IN TXT "authSource=otherDB" +test7.test.build.10gen.cc. 86400 IN TXT "ssl=false" +test8.test.build.10gen.cc. 86400 IN TXT "authSource" +test10.test.build.10gen.cc. 86400 IN TXT "socketTimeoutMS=500" +test11.test.build.10gen.cc. 86400 IN TXT "replicaS" "et=rep" "l0" +test20.test.build.10gen.cc. 86400 IN TXT "loadBalanced=true" +test21.test.build.10gen.cc. 86400 IN TXT "loadBalanced=false" +test24.test.build.10gen.cc. 86400 IN TXT "loadBalanced=true" +``` Notes: -- ``test4`` is omitted deliberately to test what happens with no SRV record. -- ``test9`` is missing because it was deleted during the development of the - tests. -- The missing ``test.`` sub-domain in the SRV record target for ``test12`` is - deliberate. -- ``test22`` is used to test a custom service name (``customname``). -- ``test23`` and ``test24`` point to port 8000 (HAProxy) and are used for - load-balanced tests. +- `test4` is omitted deliberately to test what happens with no SRV record. +- `test9` is missing because it was deleted during the development of the tests. +- The missing `test.` sub-domain in the SRV record target for `test12` is deliberate. +- `test22` is used to test a custom service name (`customname`). +- `test23` and `test24` point to port 8000 (HAProxy) and are used for load-balanced tests. -In our tests we have used ``localhost.test.build.10gen.cc`` as the domain, and -then configured ``localhost.test.build.10gen.cc`` to resolve to 127.0.0.1. +In our tests we have used `localhost.test.build.10gen.cc` as the domain, and then configured +`localhost.test.build.10gen.cc` to resolve to 127.0.0.1. -You need to adapt the records shown above to replace ``test.build.10gen.cc`` -with your own domain name, and update the "uri" field in the YAML or JSON files -in this directory with the actual domain. +You need to adapt the records shown above to replace `test.build.10gen.cc` with your own domain name, and update the +"uri" field in the YAML or JSON files in this directory with the actual domain. -Test Format and Use -------------------- +## Test Format and Use These YAML and JSON files contain the following fields: -- ``uri``: a ``mongodb+srv`` connection string -- ``seeds``: the expected set of initial seeds discovered from the SRV record -- ``numSeeds``: the expected number of initial seeds discovered from the SRV - record. This is mainly used to test ``srvMaxHosts``, since randomly selected - hosts cannot be deterministically asserted. -- ``hosts``: the discovered topology's list of hosts once SDAM completes a scan -- ``numHosts``: the expected number of hosts discovered once SDAM completes a - scan. This is mainly used to test ``srvMaxHosts``, since randomly selected - hosts cannot be deterministically asserted. -- ``options``: the parsed `URI options`_ as discovered from the - `Connection String`_'s "Connection Options" component and SRV resolution - (e.g. TXT records, implicit ``tls`` default). -- ``parsed_options``: additional, parsed options from other `Connection String`_ - components. This is mainly used for asserting ``UserInfo`` (as ``user`` and - ``password``) and ``Auth database`` (as ``auth_database``). -- ``error``: indicates that the parsing of the URI, or the resolving or - contents of the SRV or TXT records included errors. -- ``comment``: a comment to indicate why a test would fail. -- ``ping``: if false, the test runner should not run a "ping" operation. - -.. _`Connection String`: ../../connection-string/connection-string-spec.md -.. _`URI options`: ../../uri-options/uri-options.rst +- `uri`: a `mongodb+srv` connection string +- `seeds`: the expected set of initial seeds discovered from the SRV record +- `numSeeds`: the expected number of initial seeds discovered from the SRV record. This is mainly used to test + `srvMaxHosts`, since randomly selected hosts cannot be deterministically asserted. +- `hosts`: the discovered topology's list of hosts once SDAM completes a scan +- `numHosts`: the expected number of hosts discovered once SDAM completes a scan. This is mainly used to test + `srvMaxHosts`, since randomly selected hosts cannot be deterministically asserted. +- `options`: the parsed [URI options](../../uri-options/uri-options.rst) as discovered from the + [Connection String](../../connection-string/connection-string-spec.md)'s "Connection Options" component and SRV + resolution (e.g. TXT records, implicit `tls` default). +- `parsed_options`: additional, parsed options from other + [Connection String](../../connection-string/connection-string-spec.md) components. This is mainly used for asserting + `UserInfo` (as `user` and `password`) and `Auth database` (as `auth_database`). +- `error`: indicates that the parsing of the URI, or the resolving or contents of the SRV or TXT records included + errors. +- `comment`: a comment to indicate why a test would fail. +- `ping`: if false, the test runner should not run a "ping" operation. For each YAML file: -- Create a MongoClient initialized with the ``mongodb+srv`` - connection string. -- Run a "ping" operation unless ``ping`` is false or ``error`` is true. +- Create a MongoClient initialized with the `mongodb+srv` connection string. +- Run a "ping" operation unless `ping` is false or `error` is true. Assertions: -- If ``seeds`` is specified, drivers SHOULD verify that the set of hosts in the - client's initial seedlist matches the list in ``seeds``. If ``numSeeds`` is - specified, drivers SHOULD verify that the size of that set matches - ``numSeeds``. - -- If ``hosts`` is specified, drivers MUST verify that the set of - ServerDescriptions in the client's TopologyDescription eventually matches the - list in ``hosts``. If ``numHosts`` is specified, drivers MUST verify that the - size of that set matches ``numHosts``. - -- If ``options`` is specified, drivers MUST verify each of the values under - ``options`` match the MongoClient's parsed value for that option. There may be - other options parsed by the MongoClient as well, which a test does not verify. - -- If ``parsed_options`` is specified, drivers MUST verify that each of the - values under ``parsed_options`` match the MongoClient's parsed value for that - option. Supported values include, but are not limited to, ``user`` and - ``password`` (parsed from ``UserInfo``) and ``auth_database`` (parsed from - ``Auth database``). - -- If ``error`` is specified and ``true``, drivers MUST verify that initializing - the MongoClient throws an error. If ``error`` is not specified or is - ``false``, both initializing the MongoClient and running a ping operation must - succeed without throwing any errors. - -- If ``ping`` is not specified or ``true``, drivers MUST verify that running a - "ping" operation using the initialized MongoClient succeeds. If ``ping`` is - ``false``, drivers MUST NOT run a "ping" operation. - - **Note:** These tests are expected to be run against MongoDB databases with - and without authentication enabled. The "ping" operation does not require - authentication so should succeed with URIs that contain no userinfo (i.e. - no username and password). Tests with URIs that contain userinfo always set - ``ping`` to ``false`` because some drivers will fail handshake on a - connection if userinfo is provided but incorrect. +- If `seeds` is specified, drivers SHOULD verify that the set of hosts in the client's initial seedlist matches the list + in `seeds`. If `numSeeds` is specified, drivers SHOULD verify that the size of that set matches `numSeeds`. + +- If `hosts` is specified, drivers MUST verify that the set of ServerDescriptions in the client's TopologyDescription + eventually matches the list in `hosts`. If `numHosts` is specified, drivers MUST verify that the size of that set + matches `numHosts`. + +- If `options` is specified, drivers MUST verify each of the values under `options` match the MongoClient's parsed value + for that option. There may be other options parsed by the MongoClient as well, which a test does not verify. + +- If `parsed_options` is specified, drivers MUST verify that each of the values under `parsed_options` match the + MongoClient's parsed value for that option. Supported values include, but are not limited to, `user` and `password` + (parsed from `UserInfo`) and `auth_database` (parsed from `Auth database`). + +- If `error` is specified and `true`, drivers MUST verify that initializing the MongoClient throws an error. If `error` + is not specified or is `false`, both initializing the MongoClient and running a ping operation must succeed without + throwing any errors. + +- If `ping` is not specified or `true`, drivers MUST verify that running a "ping" operation using the initialized + MongoClient succeeds. If `ping` is `false`, drivers MUST NOT run a "ping" operation. + + > **Note:** These tests are expected to be run against MongoDB databases with and without authentication enabled. The + > "ping" operation does not require authentication so should succeed with URIs that contain no userinfo (i.e. no + > username and password). Tests with URIs that contain userinfo always set `ping` to `false` because some drivers will + > fail handshake on a connection if userinfo is provided but incorrect. diff --git a/source/polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst b/source/polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst index 7ff7b79346..b255c9ad68 100644 --- a/source/polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst +++ b/source/polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst @@ -25,7 +25,7 @@ Discovery specification, and modifies the `Server Discovery and Monitoring`_ specification's definition of monitoring a set of mongos servers in a Sharded TopologyType. -.. _`Initial DNS Seedlist Discovery`: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst +.. _`Initial DNS Seedlist Discovery`: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md .. _`Server Discovery and Monitoring`: ../server-discovery-and-monitoring/server-discovery-and-monitoring.rst META @@ -127,10 +127,10 @@ every *rescanSRVIntervalMS* value. The *rescanSRVIntervalMS* period SHOULD be calculated from the **end** of the previous rescan (or the **end** of the initial DNS seedlist discovery scan). -.. _seedlist: https://github.com/mongodb/specifications/blob/master/source/initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#seedlist-discovery -.. _srvMaxHosts: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#srvmaxhosts -.. _srvServiceName: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#srvservicename -.. _`initial selection`: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#querying-dns +.. _seedlist: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#seedlist-discovery +.. _srvMaxHosts: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvmaxhosts +.. _srvServiceName: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvservicename +.. _`initial selection`: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#querying-dns Multi-Threaded Drivers ---------------------- diff --git a/source/polling-srv-records-for-mongos-discovery/tests/README.rst b/source/polling-srv-records-for-mongos-discovery/tests/README.rst index 1f1889c049..a32a921f14 100644 --- a/source/polling-srv-records-for-mongos-discovery/tests/README.rst +++ b/source/polling-srv-records-for-mongos-discovery/tests/README.rst @@ -34,8 +34,8 @@ the `test set-up`_ from the `Initial DNS Seedlist Discovery`_ specification:: _mongodb._tcp.test3.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. _customname._tcp.test22.test.build.10gen.cc. 86400 IN SRV 27017 localhost.test.build.10gen.cc. -.. _`test set-up`: https://github.com/mongodb/specifications/blob/master/source/initial-dns-seedlist-discovery/tests/README.rst -.. _`Initial DNS Seedlist Discovery`: ../../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst +.. _`test set-up`: ../../initial-dns-seedlist-discovery/tests/README.md +.. _`Initial DNS Seedlist Discovery`: ../../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md Prose Tests =========== diff --git a/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst b/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst index d888528c5e..76dbd13f10 100644 --- a/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst +++ b/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst @@ -504,7 +504,7 @@ Client construction ''''''''''''''''''' Except for `initial DNS seed list discovery -`_ +<../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md>`_ when given a connection string with ``mongodb+srv`` scheme, the client's constructor MUST NOT do any I/O. This means that the constructor does not throw an exception diff --git a/source/uri-options/uri-options.rst b/source/uri-options/uri-options.rst index 14cc1c9a99..918d25bc08 100644 --- a/source/uri-options/uri-options.rst +++ b/source/uri-options/uri-options.rst @@ -72,7 +72,7 @@ srvServiceName and srvMaxHosts URI options For URI option validation pertaining to ``srvServiceName`` and ``srvMaxHosts``, please see the -`Initial DNS Seedlist Discovery spec <../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#uri-validation>`_ +`Initial DNS Seedlist Discovery spec <../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#uri-validation>`_ for details. @@ -321,7 +321,7 @@ pertaining to URI options apply here. * - srvMaxHosts - non-negative integer; 0 means no maximum - - defined in the `Initial DNS Seedlist Discovery spec <../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#srvmaxhosts>`__ + - defined in the `Initial DNS Seedlist Discovery spec <../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvmaxhosts>`__ - no - The maximum number of SRV results to randomly select when initially populating the seedlist or, during SRV polling, adding new hosts to the @@ -331,7 +331,7 @@ pertaining to URI options apply here. - a valid SRV service name according to `RFC 6335 `_ - "mongodb" - no - - the service name to use for SRV lookup in `initial DNS seedlist discovery <../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst#srvservicename>`__ + - the service name to use for SRV lookup in `initial DNS seedlist discovery <../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvservicename>`__ and `SRV polling <../polling-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst>`_ * - ssl