Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GEN-1434: Improvement : Show displayName for database and databaseSchema in explore tree #17876

Merged
merged 9 commits into from
Sep 26, 2024
Merged

GEN-1434: Improvement : Show displayName for database and databaseSchema in explore tree #17876

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
71dd2a2
Show displayName for database and databaseSchema in explore tree
sonika-shah Sep 17, 2024
55cc05a
use display name for database and schema
karanh37 Sep 17, 2024
14134c2
add playwright
karanh37 Sep 17, 2024
a36b041
Merge branch 'main' into show-display-name-explore-tree
sonika-shah Sep 21, 2024
e4c6db8
resync indexes of dataAssets based on database & databaseSchema displ…
sonika-shah Sep 21, 2024
89b9ff8
Merge branch 'main' into show-display-name-explore-tree
sonika-shah Sep 22, 2024
68181c2
Merge branch 'main' into show-display-name-explore-tree
sonika-shah Sep 25, 2024
d81327d
Merge branch 'main' into show-display-name-explore-tree
karanh37 Sep 26, 2024
fdcdffb
Merge branch 'main' into show-display-name-explore-tree
karanh37 Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DatabaseRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Objects;
import java.util.UUID;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.csv.CSVPrinter;
Expand Down Expand Up @@ -105,6 +106,18 @@ public EntityInterface getParentEntity(Database entity, String fields) {
return Entity.getEntity(entity.getService(), fields, Include.ALL);
}

@Override
public void entityRelationshipReindex(Database original, Database updated) {
super.entityRelationshipReindex(original, updated);

// Update search indexes of assets and entity on database displayName change
if (!Objects.equals(original.getDisplayName(), updated.getDisplayName())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to do this explicitly, will the updateEntity() in search not take care of this ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateEntity() currents only updates the inherited and propagated fields in children and not other fields of the parent entity

searchRepository
.getSearchClient()
.reindexAcrossIndices("database.fullyQualifiedName", original.getEntityReference());
}
}

@Override
public String exportToCsv(String name, String user) throws IOException {
Database database = getByName(null, name, Fields.EMPTY_FIELDS); // Validate database name
Expand Down
13 changes: 13 additions & 0 deletions ...tadata-service/src/main/java/org/openmetadata/service/jdbi3/DatabaseSchemaRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.Objects;
import java.util.UUID;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.csv.CSVPrinter;
Expand Down Expand Up @@ -179,6 +180,18 @@ private void populateDatabase(DatabaseSchema schema) {
.withServiceType(database.getServiceType());
}

@Override
public void entityRelationshipReindex(DatabaseSchema original, DatabaseSchema updated) {
super.entityRelationshipReindex(original, updated);

// Update search indexes of assets and entity on databaseSchema displayName change
if (!Objects.equals(original.getDisplayName(), updated.getDisplayName())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are reindexing the whole entity on just displayNameChange, should we just update the disaplayName?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this improvement will be handled in a different pr, will link the pr once it is ready

searchRepository
.getSearchClient()
.reindexAcrossIndices("databaseSchema.fullyQualifiedName", original.getEntityReference());
}
}

@Override
public String exportToCsv(String name, String user) throws IOException {
DatabaseSchema schema = getByName(null, name, Fields.EMPTY_FIELDS); // Validate database schema
Expand Down
8 changes: 8 additions & 0 deletions ...vice/src/main/java/org/openmetadata/service/search/elasticsearch/ElasticSearchClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1311,6 +1311,14 @@ private static SearchSourceBuilder buildSearchAcrossIndexesBuilder(
AggregationBuilders.terms("databaseSchema.name.keyword")
.field("databaseSchema.name.keyword")
.size(MAX_AGGREGATE_SIZE));
searchSourceBuilder.aggregation(
AggregationBuilders.terms("database.displayName")
.field("database.displayName")
.size(MAX_AGGREGATE_SIZE));
searchSourceBuilder.aggregation(
AggregationBuilders.terms("databaseSchema.displayName")
.field("databaseSchema.displayName")
.size(MAX_AGGREGATE_SIZE));
return addAggregation(searchSourceBuilder);
}

Expand Down
8 changes: 8 additions & 0 deletions ...ta-service/src/main/java/org/openmetadata/service/search/opensearch/OpenSearchClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1291,6 +1291,14 @@ private static SearchSourceBuilder buildSearchAcrossIndexesBuilder(
AggregationBuilders.terms("databaseSchema.name.keyword")
.field("databaseSchema.name.keyword")
.size(MAX_AGGREGATE_SIZE));
searchSourceBuilder.aggregation(
AggregationBuilders.terms("database.displayName")
.field("database.displayName")
.size(MAX_AGGREGATE_SIZE));
searchSourceBuilder.aggregation(
AggregationBuilders.terms("databaseSchema.displayName")
.field("databaseSchema.displayName")
.size(MAX_AGGREGATE_SIZE));
return addAggregation(searchSourceBuilder);
}

Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/en/database_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/en/database_schema_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "keyword",
"normalizer": "lowercase_normalizer"
Expand Down
2 changes: 2 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/en/stored_procedure_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,7 @@
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
Expand Down Expand Up @@ -235,6 +236,7 @@
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/jp/database_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/jp/database_schema_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "keyword",
"normalizer": "lowercase_normalizer"
Expand Down
2 changes: 2 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/jp/stored_procedure_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
Expand Down Expand Up @@ -231,6 +232,7 @@
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
Expand Down
20 changes: 20 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/jp/table_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down Expand Up @@ -228,6 +238,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/zh/database_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/zh/database_schema_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "keyword",
"normalizer": "lowercase_normalizer"
Expand Down
2 changes: 2 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/zh/stored_procedure_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
Expand Down Expand Up @@ -211,6 +212,7 @@
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
Expand Down
20 changes: 20 additions & 0 deletions openmetadata-service/src/main/resources/elasticsearch/zh/table_index_mapping.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down Expand Up @@ -204,6 +214,16 @@
}
}
},
"displayName": {
"type": "keyword",
"fields": {
"keyword": {
"type": "keyword",
"normalizer": "lowercase_normalizer",
"ignore_above": 256
}
}
},
"fullyQualifiedName": {
"type": "text"
},
Expand Down
10 changes: 2 additions & 8 deletions openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/AdvancedSearch.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,8 @@ test.describe('Advanced Search', { tag: '@advanced-search' }, () => {
tierTag2.responseData.fullyQualifiedName,
],
'service.displayName.keyword': [table1.service.name, table2.service.name],
'database.displayName.keyword': [
table1.database.name,
table2.database.name,
],
'databaseSchema.displayName.keyword': [
table1.schema.name,
table2.schema.name,
],
'database.displayName': [table1.database.name, table2.database.name],
'databaseSchema.displayName': [table1.schema.name, table2.schema.name],
aniketkatkar97 marked this conversation as resolved.
Show resolved Hide resolved
'columns.name.keyword': ['email', 'shop_id'],
'displayName.keyword': [
table1.entity.displayName,
Expand Down
Loading
Loading