-
Notifications
You must be signed in to change notification settings - Fork 244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DRIVERS-2768 [Vector Search GA] Add support for types in search index creation #1541
Changes from 3 commits
0de1a9a
dc8f737
4336d00
3b8ee0a
850f12f
c8918f9
8a2f326
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -693,6 +693,16 @@ Common API Components | |
*/ | ||
name: String; | ||
|
||
/** | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note: we just switched this spec to Markdown, so you'll have to move these changes to that file. |
||
* Optionally specify a type for the index. Defaults to "search" if not provided. | ||
* Either "search" for regular search indexes or "vectorSearch" for vector search indexes. | ||
* | ||
* Note that to create a vector search index using a helper method, the type "vectorSearch" must be provided. | ||
* | ||
*/ | ||
type: String; | ||
|
||
|
||
/** | ||
* Optionally tells the index to only reference documents with the specified field in | ||
* the index. | ||
|
@@ -1149,3 +1159,4 @@ Changelog | |
:2023-07-27: Add search index management clarifications. | ||
:2023-11-08: Clarify that ``readConcern`` and ``writeConcern`` must not be | ||
applied to search index management commands. | ||
:2024-03-06: Update tests to include search index typing. |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -231,3 +231,97 @@ This test fails if it times out waiting for the deletion to succeed. | |||||
of `true`. | ||||||
|
||||||
6. Assert that `index` has a property `latestDefinition` whose value is `{ 'mappings': { 'dynamic': false } }` | ||||||
|
||||||
#### Case 7: Driver can successfully handle search index types when creating indexes | ||||||
|
||||||
1. Create a collection with the "create" command using a randomly generated name (referred to as ``coll0``). | ||||||
|
||||||
2. Create a new search index on ``coll0`` with the ``createSearchIndex`` helper. Use the following definition: | ||||||
|
||||||
```typescript | ||||||
|
||||||
{ | ||||||
name: 'test-search-index-case7-implicit', | ||||||
definition: { | ||||||
mappings: { dynamic: false } | ||||||
} | ||||||
} | ||||||
``` | ||||||
|
||||||
3. Assert that the command returns the name of the index: ``"test-search-index-case7-implicit"``. | ||||||
|
||||||
4. Run ``coll0.listSearchIndexes('test-search-index-case7-implicit')`` repeatedly every 5 seconds until the following condition is satisfied and store the value in a variable ``index1``: | ||||||
|
||||||
- An index with the ``name`` of ``test-search-index-case7-implicit`` is present and the index has a field ``queryable`` with a value of ``true``. | ||||||
|
||||||
5. Assert that ``index1`` has a property ``type`` whose value is ``search`` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
6. Create a new search index on ``coll0`` with the ``createSearchIndex`` helper. Use the following definition: | ||||||
|
||||||
```typescript | ||||||
|
||||||
{ | ||||||
name: 'test-search-index-case7-explicit', | ||||||
type: 'search', | ||||||
definition: { | ||||||
mappings: { dynamic: false } | ||||||
} | ||||||
} | ||||||
``` | ||||||
|
||||||
7. Assert that the command returns the name of the index: ``"test-search-index-case7-explicit"``. | ||||||
|
||||||
8. Run ``coll0.listSearchIndexes('test-search-index-case7-explicit')`` repeatedly every 5 seconds until the following condition is satisfied and store the value in a variable ``index2``: | ||||||
|
||||||
- An index with the ``name`` of ``test-search-index-case7-explicit`` is present and the index has a field ``queryable`` with a value of ``true``. | ||||||
|
||||||
9. Assert that ``index2`` has a property ``type`` whose value is ``search`` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
10. Create a new vector search index on ``coll0`` with the ``createSearchIndex`` helper. Use the following definition: | ||||||
|
||||||
```typescript | ||||||
|
||||||
{ | ||||||
name: 'test-search-index-case7-vector', | ||||||
type: 'vectorSearch', | ||||||
definition: { | ||||||
"fields": [ | ||||||
{ | ||||||
"type": "vector", | ||||||
"path": "plot_embedding", | ||||||
"numDimensions": 1536, | ||||||
"similarity": "euclidean", | ||||||
}, | ||||||
] | ||||||
} | ||||||
} | ||||||
``` | ||||||
|
||||||
11. Assert that the command returns the name of the index: ``"test-search-index-case7-vector"``. | ||||||
|
||||||
12. Run ``coll0.listSearchIndexes('test-search-index-case7-vector')`` repeatedly every 5 seconds until the following condition is satisfied and store the value in a variable ``index3``: | ||||||
|
||||||
- An index with the ``name`` of ``test-search-index-case7-vector`` is present and the index has a field ``queryable`` with a value of ``true``. | ||||||
|
||||||
13. Assert that ``index3`` has a property ``type`` whose value is ``vectorSearch`` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
14. Create a new vector search index on ``coll0`` with the ``createSearchIndex`` helper. Use the following definition: | ||||||
|
||||||
```typescript | ||||||
|
||||||
{ | ||||||
name: 'test-search-index-case7-error', | ||||||
definition: { | ||||||
"fields": [ | ||||||
{ | ||||||
"type": "vector", | ||||||
"path": "plot_embedding", | ||||||
"numDimensions": 1536, | ||||||
"similarity": "euclidean", | ||||||
}, | ||||||
] | ||||||
} | ||||||
} | ||||||
``` | ||||||
|
||||||
15. Assert that the command throws an exception due to the ``mappings`` field missing. |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Many other driver options are implemented such that we only send a value if the user explicitly provides one. How do you feel about mandating that driver MUST NOT send this field if not provided?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense to me if that's a consistent pattern among drivers. The server should default to
search
implicitly iftype
is not provided, so the behavior is consistent.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this change be made to the
CreateSearchIndexOptions
instead ofIndexOptions
? https://github.com/mongodb/specifications/blob/master/source/index-management/index-management.md#common-interfacesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CreateSearchIndexOptions
is currently unused. If drivers support addingtype
toIndexOptions
, that seems to be a better fit as then the top-level fields oftype
andname
are present in the same interface.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, looking more closely it probably belongs in the
SearchIndexModel
?IndexOptions
is used for regular indexes, not search indexes.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh good catch! Yes,
SearchIndexModel
seems like the correct place fortype
.