Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRIVERS-2630: add e2e testing for search index commands and clarifications to search index spec #1442

Merged
merged 16 commits into from
Jul 27, 2023

Conversation

baileympearson
Copy link
Contributor

@baileympearson baileympearson commented Jul 14, 2023

This PR does three things:

  • This PR adds prose tests for the search index management helpers
  • This PR strengthens assertions on the existing unified tests for the search index management helpers.
  • This PR includes some clarifications in the search index management spec.

The third bullet can be separated into a different PR if necessary.

Node POC: mongodb/node-mongodb-native#3736

Notes

Why have unified tests and prose tests?

The initial plan was to enhance the unified test runner to support the new search index management commands. There ended up being complications integrating support for the new commands into the UTR (the largest issue was that the commands are asynchronous, and return from the server before the changes have finished). The unified tests were already written and expected errors when run against non-Atlas deployments, so they effectively serve as unit tests for drivers without writing another specialized test format. The new prose tests add end-to-end functionality for drivers' implementations.

Why setup/teardown a cluster for each test?

  • Drivers already spinup/teardown resources for existing tests (serverless, gcp/azure kms, FAAS testing), so additional CI time was not a concern.
  • We have tooling in drivers-evergreen-tools to setup and teardown Atlas clusters so the setup was easy to reuse.
  • Provisioning a cluster on-demand reduces the changes for collisions and the need to ensure driver use unique databases / collections / commit hashes to avoid CI runs interfering on the same cluster.

CSOT

These commands should work with CSOT as per the CSOT spec, but looking through the specs repo we don't seem to have tests for every driver helper that supports timeoutMS. Consequently, I chose not to add tests for these helpers with timeoutMS.

Please complete the following before merging:

  • Update changelog.
  • Make sure there are generated JSON files from the YAML test files.
  • Test changes in at least one language driver.
  • Test these changes against all server versions and topologies (including standalone, replica set, sharded clusters, and serverless).

@baileympearson baileympearson changed the title improve test assertions DRIVERS-2630: add e2e testing for search index commands and clarifications to search index spec Jul 14, 2023
@baileympearson baileympearson marked this pull request as ready for review July 17, 2023 15:26
@baileympearson baileympearson requested review from a team as code owners July 17, 2023 15:26
@baileympearson baileympearson requested review from dariakp and GromNaN and removed request for a team July 17, 2023 15:26
Copy link
Member

@GromNaN GromNaN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests implemented with success in PHP: tests/Collection/SearchIndexFunctionalTest.php

To improve the coverage, I suggest adding this e2e tests:

  • Call to createSearchIndexes with an empty list of indexes.
  • Call createSearchIndex without name. It fallbacks to the "default" name.

source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Show resolved Hide resolved
source/index-management/tests/README.rst Show resolved Hide resolved
Copy link
Member

@GromNaN GromNaN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I've updated the PHP implementation following our discussions.

source/index-management/index-management.rst Outdated Show resolved Hide resolved
source/index-management/index-management.rst Show resolved Hide resolved
source/index-management/tests/updateSearchIndex.yml Outdated Show resolved Hide resolved
Comment on lines 416 to 420
* or language/implementation equivalent.
*
* If drivers are unable to make the IndexView iterable, they MAY opt to provide the options for
* listing search indexes via the `list` method instead of the `Collection.listSearchIndexes` method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussion with @dariakp on slack, we decided that a more consistent API is to include the name in an options object. There were two reasons for this change:

  • It's odd to have a set of search index management methods that have a name as the first parameter and an optional second parameter in others. It makes more sense to include an optional name in an "options" object.
  • This is more closely aligned with the regular index management API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand this was already allowed, this is how it is already implemented in PHP.

@dariakp dariakp requested a review from kevinAlbs July 25, 2023 20:10
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/tests/README.rst Outdated Show resolved Hide resolved
source/index-management/index-management.rst Outdated Show resolved Hide resolved
source/index-management/index-management.rst Outdated Show resolved Hide resolved
source/index-management/index-management.rst Show resolved Hide resolved
baileympearson and others added 2 commits July 27, 2023 09:47
Co-authored-by: Kevin Albertson <kevin.albertson@mongodb.com>
@dariakp dariakp merged commit 267a54d into master Jul 27, 2023
3 checks passed
@dariakp dariakp deleted the search-indexes-e2e-testing branch July 27, 2023 19:22
kevinAlbs added a commit to kevinAlbs/mongo-c-driver that referenced this pull request Jul 31, 2023
kevinAlbs added a commit to kevinAlbs/mongo-c-driver that referenced this pull request Jul 31, 2023
kevinAlbs added a commit to kevinAlbs/mongo-c-driver that referenced this pull request Jul 31, 2023
Do not intend to merge. This was used to help review PR: mongodb/specifications#1442
kevinAlbs added a commit to mongodb/mongo-c-driver that referenced this pull request Aug 1, 2023
* add index-management tests

From PR mongodb/specifications#1442

* add test search operations and implement without helpers

* document management of search indexes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants