diff --git a/source/client-side-encryption/client-side-encryption.md b/source/client-side-encryption/client-side-encryption.md index 9cda038837..b81ebadeea 100644 --- a/source/client-side-encryption/client-side-encryption.md +++ b/source/client-side-encryption/client-side-encryption.md @@ -540,35 +540,35 @@ automatic credentials results in a runtime error from [libmongocrypt](#libmongoc > [!NOTE] > Drivers MUST NOT eagerly fill an empty KMS options property. -Once requested, drivers MUST create a new [KMSProviders](#kmsproviders) `$P$` according to the following process: +Once requested, drivers MUST create a new [KMSProviders](#kmsproviders) $P$ according to the following process: -1. Let `$K$` be the [kmsProviders](#kmsproviders) value provided by the user as part of the original +1. Let $K$ be the [kmsProviders](#kmsproviders) value provided by the user as part of the original [ClientEncryptionOpts](#ClientEncryptionOpts) or [AutoEncryptionOpts](#AutoEncryptionOpts). -2. Initialize `$P$` to an empty [KMSProviders](#kmsproviders) object. -3. If `$K$` contains an `aws` property, and that property is an empty map: - 1. Attempt to obtain credentials `$C$` from the environment using similar logic as is detailed in +2. Initialize $P$ to an empty [KMSProviders](#kmsproviders) object. +3. If $K$ contains an `aws` property, and that property is an empty map: + 1. Attempt to obtain credentials $C$ from the environment using similar logic as is detailed in [the obtaining-AWS-credentials section from the Driver Authentication specification](../auth/auth.md#obtaining-credentials), but ignoring the case of loading the credentials from a URI - 2. If credentials `$C$` were successfully loaded, create a new [AWSKMSOptions](#AWSKMSOptions) map from `$C$` and - insert that map onto `$P$` as the `aws` property. -4. If `$K$` contains an `gcp` property, and that property is an empty map: - 1. Attempt to obtain credentials `$C$` from the environment logic as is detailed in + 2. If credentials $C$ were successfully loaded, create a new [AWSKMSOptions](#AWSKMSOptions) map from $C$ and insert + that map onto $P$ as the `aws` property. +4. If $K$ contains an `gcp` property, and that property is an empty map: + 1. Attempt to obtain credentials $C$ from the environment logic as is detailed in [Obtaining GCP Credentials](#obtaining-gcp-credentials). - 2. If credentials `$C$` were successfully loaded, create a new [GCPKMSOptions](#GCPKMSOptions) map from `$C$` and - insert that map onto `$P$` as the `gcp` property. -5. If `$K$` contains an `azure` property, and that property is an empty map: + 2. If credentials $C$ were successfully loaded, create a new [GCPKMSOptions](#GCPKMSOptions) map from $C$ and insert + that map onto $P$ as the `gcp` property. +5. If $K$ contains an `azure` property, and that property is an empty map: 1. If there is a `cachedAzureAccessToken` AND the duration until `azureAccessTokenExpireTime` is greater than one - minute, insert `cachedAzureAccessToken` as the `azure` property on `$P$`. + minute, insert `cachedAzureAccessToken` as the `azure` property on $P$. 2. Otherwise: - 1. Let `$t_0$` be the current time. - 2. Attempt to obtain an Azure VM Managed Identity Access Token `$T$` as detailed in + 1. Let $t_0$ be the current time. + 2. Attempt to obtain an Azure VM Managed Identity Access Token $T$ as detailed in [Obtaining an Access Token for Azure Key Vault](#obtaining-an-access-token-for-azure-key-vault). - 3. If a token `$T$` with expire duration `$d\_{exp}$` were obtained successfully, create a new - [AzureAccessToken](#AzureAccessToken) object with `$T$` as the `accessToken` property. Insert that - [AzureAccessToken](#AzureAccessToken) object into `$P$` as the `azure` property. Record the generated + 3. If a token $T$ with expire duration $d\_{exp}$ were obtained successfully, create a new + [AzureAccessToken](#AzureAccessToken) object with $T$ as the `accessToken` property. Insert that + [AzureAccessToken](#AzureAccessToken) object into $P$ as the `azure` property. Record the generated [AzureAccessToken](#AzureAccessToken) in `cachedAzureAccessToken`. Record the `azureAccessTokenExpireTime` as - `$t_0 + d\_{exp}$`. -6. Return `$P$` as the additional KMS providers to [libmongocrypt](#libmongocrypt). + $t_0 + d\_{exp}$. +6. Return $P$ as the additional KMS providers to [libmongocrypt](#libmongocrypt). @@ -601,37 +601,37 @@ the VM, an identity can be used by obtaining an access token via HTTP from the * The below steps should be taken: -01. Let `$U$` be a new URL, initialized from the URL string `"http://169.254.169.254/metadata/identity/oauth2/token"` +01. Let $U$ be a new URL, initialized from the URL string `"http://169.254.169.254/metadata/identity/oauth2/token"` -02. Add a query parameter `api-version=2018-02-01` to `$U$`. +02. Add a query parameter `api-version=2018-02-01` to $U$. -03. Add a query parameter `resource=https://vault.azure.net/` to `$U$`. +03. Add a query parameter `resource=https://vault.azure.net/` to $U$. -04. Prepare an HTTP GET request `$Req$` based on `$U$`. +04. Prepare an HTTP GET request $Req$ based on $U$. > [!NOTE] - > All query parameters on `$U$` should be appropriately percent-encoded + > All query parameters on $U$ should be appropriately percent-encoded -05. Add HTTP headers `Metadata: true` and `Accept: application/json` to `$Req$`. +05. Add HTTP headers `Metadata: true` and `Accept: application/json` to $Req$. -06. Issue `$Req$` to the Azure IMDS server `169.254.169.254:80`. Let `$Resp$` be the response from the server. If the - HTTP response is not completely received within ten seconds, consider the request to have timed out, and return an - error instead of an access token. +06. Issue $Req$ to the Azure IMDS server `169.254.169.254:80`. Let $Resp$ be the response from the server. If the HTTP + response is not completely received within ten seconds, consider the request to have timed out, and return an error + instead of an access token. -07. If `$Resp_{status} ≠ 200$`, obtaining the access token has failed, and the HTTP response body of `$Resp$` encodes +07. If $Resp\_{status} ≠ 200$, obtaining the access token has failed, and the HTTP response body of $Resp$ encodes information about the error that occurred. Return an error including the HTTP response body instead of an access token. -08. Otherwise, let `$J$` be the JSON document encoded in the HTTP response body of `$Resp$`. +08. Otherwise, let $J$ be the JSON document encoded in the HTTP response body of $Resp$. -09. The result access token `$T$` is given as the `access_token` string property of `$J$`. Return `$T$` as the resulting +09. The result access token $T$ is given as the `access_token` string property of $J$. Return $T$ as the resulting access token. -10. The resulting "expires in" duration `$d_{exp}$` is a count of seconds given as an ASCII-encoded integer string - `expires_in` property of `$J$`. +10. The resulting "expires in" duration $d\_{exp}$ is a count of seconds given as an ASCII-encoded integer string + `expires_in` property of $J$. > [!NOTE] -> If JSON decoding of `$Resp$` fails, or the `access_token` property is absent from `$J$`, this is a protocol error from +> If JSON decoding of $Resp$ fails, or the `access_token` property is absent from $J$, this is a protocol error from > IMDS. Indicate this error to the requester of the access token. > [!NOTE] @@ -809,19 +809,19 @@ A collection supporting Queryable Encryption requires an index and three additio The convenience methods support the following lookup process for finding the `encryptedFields` associated with a collection. -Assume an exposition-only function `$GetEncryptedFields(opts, collName, dbName, askDb)$`, where `$opts$` is a set of -options, `$collName$` is the name of the collection, `$dbName$` is the name of the database associated with that -collection, and `$askDb$` is a boolean value. The resulting `encryptedFields` `$EF$` is found by: +Assume an exposition-only function $GetEncryptedFields(opts, collName, dbName, askDb)$, where $opts$ is a set of +options, $collName$ is the name of the collection, $dbName$ is the name of the database associated with that collection, +and $askDb$ is a boolean value. The resulting `encryptedFields` $EF$ is found by: -1. Let `$QualName$` be the string formed by joining`$dbName$` and `$collName$` with an ASCII dot `"."`. -2. If `$opts$` contains an `"encryptedFields"` property, then `$EF$` is the value of that property. -3. Otherwise, if `AutoEncryptionOptions.encryptedFieldsMap` contains an element named by `$QualName$`, then `$EF$` is - the value of that element. -4. Otherwise, if `$askDb$` is `true`: +1. Let $QualName$ be the string formed by joining$dbName$ and $collName$ with an ASCII dot `"."`. +2. If $opts$ contains an `"encryptedFields"` property, then $EF$ is the value of that property. +3. Otherwise, if `AutoEncryptionOptions.encryptedFieldsMap` contains an element named by $QualName$, then $EF$ is the + value of that element. +4. Otherwise, if $askDb$ is `true`: 1. Issue a `listCollections` command against the database named by $dbName$, filtered by `{name: }`. Let - the result be the document `$L$`. - 2. If `$L$` contains an `options` document element, and that element contains an `encryptedFields` document element, - `$EF$` is `$L$` `["options"]["encryptedFields"]`. + the result be the document $L$. + 2. If $L$ contains an `options` document element, and that element contains an `encryptedFields` document element, + $EF$ is $L$ `["options"]["encryptedFields"]`. 3. Otherwise, $EF$ is *not-found* 5. Otherwise, $EF$ is considered *not-found*. @@ -843,8 +843,8 @@ Drivers MUST support a BSON document option named `encryptedFields` for any > Drivers MUST NOT document the `escCollection` and `ecocCollection` options. For a helper function, `CreateCollection(collectionName, collectionOptions)` with the name of the database associated as -`$dbName$`, look up the encrypted fields `encryptedFields` for the collection as -`$GetEncryptedFields(collectionOptions, collectionName, dbName, false)$` ([See here](#GetEncryptedFields)). +$dbName$, look up the encrypted fields `encryptedFields` for the collection as $GetEncryptedFields(collectionOptions, +collectionName, dbName, false)$ ([See here](#GetEncryptedFields)). If a set of `encryptedFields` was found, then do the following operations. If any of the following operations error, the remaining operations are not attempted: @@ -867,33 +867,33 @@ remaining operations are not attempted: #### Create Encrypted Collection Helper -To support automatic generation of encryption data keys, a helper -`$CreateEncryptedCollection(CE, database, collName, collOpts, kmsProvider, masterKey)$` is defined, where `$CE$` is a -[ClientEncryption](#clientencryption-1) object, `$kmsProvider$` is a [KMSProvider](#KMSProvider) and `$masterKey$` is -equivalent to the `$masterKey$` defined in [DataKeyOpts](#datakeyopts). It has the following behavior: - -- If `$collOpts$` contains an `"encryptedFields"` property, then `$EF$` is the value of that property. Otherwise, report - an error that there are no `encryptedFields` defined for the collection. -- Let `$EF'$` be a copy of `$EF$`. Update `$EF'$` in the following manner: - - Let `$Fields$` be the `"fields"` element within `$EF'$`. - - If `$Fields$` is present and is an array value, then for each element `$F$` of `Fields`: - - If `$F$` is not a document element, skip it. - - Otherwise, if `$F$` has a `"keyId"` named element `$K$` and `$K$` is a `null` value: - - Create a [DataKeyOpts](#datakeyopts) named `$dkOpts$` with the `$masterKey$` argument. - - Let `$D$` be the result of `CE.createDataKey(kmsProvider, dkOpts)`. - - If generating `$D$` resulted in an error `$E$`, the entire `$CreateEncryptedCollection$` must now fail with - error `$E$`. Return the partially-formed `$EF'$` with the error so that the caller may know what datakeys have - already been created by the helper. - - Replace `$K$` in `$F$` with `$D$`. -- Create a new set of options `$collOpts'$` duplicating `$collOpts$`. Set the `"encryptedFields"` named element of - `$collOpts'$` to `$EF'$`. -- Invoke the `CreateCollection` helper as `$CreateCollection(database, collName, collOpts')$`. Return the resulting - collection and the generated `$EF'$`. If an error occurred, return the resulting `$EF$` with the error so that the - caller may know what datakeys have already been created by the helper. - -Drivers MUST document that `$createEncryptedCollection$` does not affect any auto encryption settings on existing +To support automatic generation of encryption data keys, a helper $CreateEncryptedCollection(CE, database, collName, +collOpts, kmsProvider, masterKey)$ is defined, where $CE$ is a [ClientEncryption](#clientencryption-1) object, +$kmsProvider$ is a [KMSProvider](#KMSProvider) and $masterKey$ is equivalent to the $masterKey$ defined in +[DataKeyOpts](#datakeyopts). It has the following behavior: + +- If $collOpts$ contains an `"encryptedFields"` property, then $EF$ is the value of that property. Otherwise, report an + error that there are no `encryptedFields` defined for the collection. +- Let $EF'$ be a copy of $EF$. Update $EF'$ in the following manner: + - Let $Fields$ be the `"fields"` element within $EF'$. + - If $Fields$ is present and is an array value, then for each element $F$ of `Fields`: + - If $F$ is not a document element, skip it. + - Otherwise, if $F$ has a `"keyId"` named element $K$ and $K$ is a `null` value: + - Create a [DataKeyOpts](#datakeyopts) named $dkOpts$ with the $masterKey$ argument. + - Let $D$ be the result of `CE.createDataKey(kmsProvider, dkOpts)`. + - If generating $D$ resulted in an error $E$, the entire $CreateEncryptedCollection$ must now fail with error $E$. + Return the partially-formed $EF'$ with the error so that the caller may know what datakeys have already been + created by the helper. + - Replace $K$ in $F$ with $D$. +- Create a new set of options $collOpts'$ duplicating $collOpts$. Set the `"encryptedFields"` named element of + $collOpts'$ to $EF'$. +- Invoke the `CreateCollection` helper as $CreateCollection(database, collName, collOpts')$. Return the resulting + collection and the generated $EF'$. If an error occurred, return the resulting $EF$ with the error so that the caller + may know what datakeys have already been created by the helper. + +Drivers MUST document that $createEncryptedCollection$ does not affect any auto encryption settings on existing MongoClients that are already configured with auto encryption. Users must configure auto encryption after creating the -encrypted collection with the `$createEncryptedCollection$` helper. +encrypted collection with the $createEncryptedCollection$ helper. #### Drop Collection Helper @@ -912,9 +912,9 @@ Drivers MUST support a BSON document option named `encryptedFields` for any > > Drivers SHOULD NOT document the `escCollection` and `ecocCollection` options. -For a helper function `DropCollection(dropOptions)` with associated collection named `$collName$` and database named -`$dbName$`, look up the encrypted fields `encryptedFields` as -`$GetEncryptedFields(dropOptions, collName, dbname, true)$` ([See here](#GetEncryptedFields)). +For a helper function `DropCollection(dropOptions)` with associated collection named $collName$ and database named +$dbName$, look up the encrypted fields `encryptedFields` as $GetEncryptedFields(dropOptions, collName, dbname, true)$ +([See here](#GetEncryptedFields)). If a set of `encryptedFields` was found, then perform the following operations. If any of the following operations error, the remaining operations are not attempted. A `namespace not found` error returned from the server (server error diff --git a/source/client-side-encryption/tests/README.md b/source/client-side-encryption/tests/README.md index 1b6b1ab2f9..2877fbb964 100644 --- a/source/client-side-encryption/tests/README.md +++ b/source/client-side-encryption/tests/README.md @@ -32,8 +32,8 @@ The spec tests format is an extension of the - An `encrypted_fields` to set on the collection used for operations. - A `key_vault_data` of data that should be inserted in the key vault collection before each test. - Introduction `autoEncryptOpts` to `clientOptions` -- Addition of `$db` to command in `command_started_event` -- Addition of `$$type` to command_started_event and outcome. +- Addition of `$db`to command in`command_started_event` +- Addition of `$$type` to `command_started_event` and outcome. The semantics of `$$type` is that any actual value matching one of the types indicated by either a BSON type string or an array of BSON type strings is considered a match. @@ -2249,7 +2249,7 @@ for the field `cursor.firstBatch.encrypted`. These tests require valid AWS credentials. Refer: [Automatic AWS Credentials](../client-side-encryption.rst#automatic-aws-credentials). -For these cases, create a [ClientEncryption](../client-side-encryption.rst#clientencryption) object `$C$` with the +For these cases, create a [ClientEncryption](../client-side-encryption.rst#clientencryption) object $C$ with the following options: ```typescript @@ -2265,7 +2265,7 @@ class ClientEncryptionOpts { Do not run this test case in an environment where AWS credentials are available (e.g. via environment variables or a metadata URL). (Refer: [Obtaining credentials for AWS](../../auth/auth.rst#obtaining-credentials)) -Attempt to create a datakey with `$C$` using the `"aws"` KMS provider. Expect this to fail due to a lack of KMS provider +Attempt to create a datakey with $C$ using the `"aws"` KMS provider. Expect this to fail due to a lack of KMS provider credentials. #### Case 2: Success @@ -2410,7 +2410,7 @@ impossible by design to omit `RewrapManyDataKeyOpts.provider` when `RewrapManyDa Refer: [Automatic GCP Credentials](../client-side-encryption.rst#automatic-gcp-credentials). -For these cases, create a [ClientEncryption](../client-side-encryption.rst#clientencryption) object `$C$` with the +For these cases, create a [ClientEncryption](../client-side-encryption.rst#clientencryption) object $C$ with the following options: ```typescript @@ -2426,7 +2426,7 @@ class ClientEncryptionOpts { Do not run this test case in an environment with a GCP service account is attached (e.g. any [GCE equivalent runtime](https://google.aip.dev/auth/4115)). This may be run in an AWS EC2 instance. -Attempt to create a datakey with `$C$` using the `"gcp"` KMS provider and following `DataKeyOpts`: +Attempt to create a datakey with $C$ using the `"gcp"` KMS provider and following `DataKeyOpts`: ```typescript class DataKeyOpts { @@ -2448,7 +2448,7 @@ This test case must run in a Google Compute Engine (GCE) Virtual Machine with a for scripts to create a GCE instance for testing. The Evergreen task SHOULD set a `batchtime` of 14 days to reduce how often this test case runs. -Attempt to create a datakey with `$C$` using the `"gcp"` KMS provider and following `DataKeyOpts`: +Attempt to create a datakey with $C$ using the `"gcp"` KMS provider and following `DataKeyOpts`: ```typescript class DataKeyOpts { @@ -2575,7 +2575,7 @@ a timeout. Refer: [Automatic Azure Credentials](../client-side-encryption.rst#obtaining-an-access-token-for-azure-key-vault) -For these cases, create a [ClientEncryption](../client-side-encryption.rst#clientencryption) object `$C$` with the +For these cases, create a [ClientEncryption](../client-side-encryption.rst#clientencryption) object $C$ with the following options: ```typescript @@ -2590,7 +2590,7 @@ class ClientEncryptionOpts { Do not run this test case in an Azure environment with an attached identity. This may be run in an AWS EC2 instance. -Attempt to create a datakey with `$C$` using the `"azure"` KMS provider and following `DataKeyOpts`: +Attempt to create a datakey with $C$ using the `"azure"` KMS provider and following `DataKeyOpts`: ```typescript class DataKeyOpts { @@ -2610,7 +2610,7 @@ This test case must run in an Azure environment with an attached identity. See for scripts to create a Azure instance for testing. The Evergreen task SHOULD set a `batchtime` of 14 days to reduce how often this test case runs. -Attempt to create a datakey with `$C$` using the `"azure"` KMS provider and following `DataKeyOpts`: +Attempt to create a datakey with $C$ using the `"azure"` KMS provider and following `DataKeyOpts`: ```typescript class DataKeyOpts { @@ -2706,7 +2706,7 @@ This test is the most basic to verify that [CreateEncryptedCollection](../client-side-encryption.rst#create-encrypted-collection-helper) created a collection with queryable encryption enabled. It verifies that the server rejects an attempt to insert plaintext in an encrypted fields. -1. Create a new create-collection options `$Opts$` including the following: +1. Create a new create-collection options $Opts$ including the following: ```typescript { @@ -2720,8 +2720,8 @@ queryable encryption enabled. It verifies that the server rejects an attempt to } ``` -2. Invoke `$CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$` to obtain a new collection - `$Coll$`. Expect success. +2. Invoke $CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$ to obtain a new collection + $Coll$. Expect success. 3. Attempt to insert the following document into `Coll`: @@ -2741,8 +2741,8 @@ The [CreateEncryptedCollection](../client-side-encryption.rst#create-encrypted-c create a regular collection if there are no `encryptedFields` for the collection being created. Instead, it should generate an error indicated that the `encryptedFields` option is missing. -1. Create a new empty create-collection options `$Opts$`. (i.e. it must not contain any `encryptedFields` options.) -2. Invoke `$CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$`. +1. Create a new empty create-collection options $Opts$. (i.e. it must not contain any `encryptedFields` options.) +2. Invoke $CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$. 3. Expect the invocation to fail with an error indicating that `encryptedFields` is not defined for the collection, and expect that no collection was created within the database. It would be *incorrect* for [CreateEncryptedCollection](../client-side-encryption.rst#create-encrypted-collection-helper) to create a regular @@ -2760,7 +2760,7 @@ with such invalid settings. > This test is not required if the type system of the driver has a compile-time check that fields' `keyId`s are of the > correct type. -1. Create a new create-collection options `$Opts$` including the following: +1. Create a new create-collection options $Opts$ including the following: ```typescript { @@ -2774,7 +2774,7 @@ with such invalid settings. } ``` -2. Invoke `$CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$`. +2. Invoke $CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$. 3. Expect an error from the server indicating a validation error at `create.encryptedFields.fields.keyId`, which must be a UUID and not a boolean value. @@ -2783,7 +2783,7 @@ with such invalid settings. This test is continuation of the case 1 and provides a way to complete inserting with encrypted value. -1. Create a new create-collection options `$Opts$` including the following: +1. Create a new create-collection options $Opts$ including the following: ```typescript { @@ -2797,11 +2797,11 @@ This test is continuation of the case 1 and provides a way to complete inserting } ``` -2. Invoke `$CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$` to obtain a new collection +2. Invoke $CreateEncryptedCollection(CE, DB, "testing1", Opts, kmsProvider, masterKey)$ to obtain a new collection $Coll$ and data key $key1$. Expect success. -3. Use `$CE$` to explicitly encrypt the string "123-45-6789" using algorithm $Unindexed$ and data key `$key1$`. Refer - result as `$encryptedPayload$`. +3. Use $CE$ to explicitly encrypt the string "123-45-6789" using algorithm $Unindexed$ and data key $key1$. Refer result + as $encryptedPayload$. 4. Attempt to insert the following document into `Coll`: