Update of metadata entity object record in ES/OS index doesn't remove deleted fields.

[ramzanzan]

Hello dear OpenMetadata developers!

I have found that while update of any metadata entity (such as Table) reindexing of record in ES/OS doesn't remove fields which removed in original entity (such as owner).

I have found that problem is in ES script for update (link to src). This script doesn't count enitity's removed fields cause those fields doesn't present in script params -> there aren't calls ctx._source.put(k, params.get(k)) for them.

Is such behavior a bug?
Why do you use scritptedUpdate and not just upsert?

[mohityadav766]

@ramzanzan if you remove the owner , is it not updating then?

[ramzanzan]

Yes. It will be removed from the relational database but will remain in the index in ES

[ramzanzan]

@mohityadav766 So, can you please clarify what is purpose of use of scripted update (link to src) instead of just update?

[harshach]

@ramzanzan what you mean removed from relational database. How are you removing it from the UI/API or directly changing it in db

[ramzanzan]

From UI/API