From ba8ba8a5e57180f479512b481017c87a0aa8105d Mon Sep 17 00:00:00 2001 From: Mandy Chessell Date: Mon, 9 Sep 2024 10:41:44 +0100 Subject: [PATCH] Remove obsolete services Signed-off-by: Mandy Chessell --- .../database-agents-and-information.svg | 0 .../governance-engine-omas.drawio | 0 ...tor-catalog-target-data-folder-example.svg | 0 ...nector-catalog-target-database-example.svg | 0 .../integration-connector-catalog-target.svg | 0 .../integration-connector-simple.svg | 0 ...n-connector-virtual-connection-example.svg | 0 ...tegration-connector-virtual-connection.svg | 0 .../integration-group-example.svg | 0 .../integration-group-use.svg | 0 .../governance-engine/overview.md | 0 .../asset-lineage/overview.md | 0 .../configuring-a-lineage-warehouse.md | 0 .../lineage-warehouse-janus-connector.md | 0 .../lineage-warehouse-config.svg | 0 .../lineage-warehouse-services-section.md | 0 .../lineage-warehouse-services.md | 0 .../lineage-warehouse-services.svg | 0 saved/lineage-warehouse/lineage-warehouse.md | 18 + .../lineage-warehouse}/lineage-warehouse.svg | 0 site/docs/concepts/asset.md | 1 - site/docs/concepts/engine-host.md | 2 +- .../concepts/governance-engine-definition.md | 6 +- site/docs/concepts/governance-engine.md | 2 +- site/docs/concepts/governance-server.md | 1 - .../concepts/governance-service-definition.md | 4 +- site/docs/concepts/governance-service.md | 2 +- site/docs/concepts/guard.md | 2 +- site/docs/concepts/index.md | 3 +- site/docs/concepts/integration-group.md | 2 +- site/docs/concepts/lineage-warehouse.md | 7 +- site/docs/concepts/omag-server.md | 1 - site/docs/concepts/omag-subsystem.md | 2 - site/docs/concepts/open-lineage-server.md | 2 +- site/docs/concepts/schema.md | 1 - ...ment-watchdog-governance-action-service.md | 2 +- ...-provisioning-governance-action-service.md | 6 +- site/docs/connectors/index.md | 10 - .../kafka-open-metadata-topic-connector.md | 2 +- ...erent-types-of-apis-platform-deployment.md | 2 +- .../egeria-platform-report-mds1-configured.md | 2 +- .../egeria-platform-report-mds1-running.md | 2 +- .../egeria-platform-report-no-servers.md | 2 +- .../egeria-platform-report-no-servers.md | 2 +- .../egeria-platform-report-server-config.md | 2 +- .../egeria-platform-report-server-running.md | 2 +- ...erent-types-of-apis-platform-deployment.md | 2 +- .../features/lineage-management/bad-data.svg | 4 + .../lineage-architecture.svg | 2 +- .../lineage-horizontal-view.svg | 4 + .../lineage-vertical-view.svg | 4 + .../lineage-management/lineage-warehouse.svg | 4 +- .../lineage-management/lineage.drawio | 645 +++++++++++++----- .../features/lineage-management/overview.md | 34 +- site/docs/guides/admin/egeria-admin.drawio | 84 +-- .../configuration-document-structure.svg | 2 +- .../guides/admin/servers/by-section/index.md | 1 - ...tion-document-structure-by-server-type.svg | 2 +- .../admin/servers/by-server-type/index.md | 1 - ...guration-document-structure-simplified.svg | 2 +- .../governance-action-services/overview.md | 6 +- .../java-clients/governance-engine.md | 4 +- .../examples-of-server-start-up-errors.md | 2 +- .../planning/governance-program/overview.md | 4 +- site/docs/guides/planning/runtime/overview.md | 2 +- .../metadata-governance/governance-engines.md | 2 +- .../setting-up-the-engine-host.md | 2 +- site/docs/release-notes/february-2023.md | 2 +- site/docs/services/engine-host-services.md | 4 +- .../services/governance-server-services.md | 1 - site/docs/services/index.md | 1 - .../omas/governance-program/overview.md | 2 +- .../omas/governance-server/overview.md | 6 +- site/docs/services/omas/index.md | 2 - .../TagPropogation/TagPropogation.md | 2 +- .../omes/governance-action/overview.md | 2 +- site/docs/services/omes/index.md | 2 +- .../omes/repository-governance/overview.md | 2 +- .../4/0435-Policy-Management-Capabilities.md | 2 +- .../docs/types/4/0465-Duplicate-Processing.md | 2 +- 80 files changed, 598 insertions(+), 335 deletions(-) rename {site/docs/services/omas => saved}/governance-engine/database-agents-and-information.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/governance-engine-omas.drawio (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-connector-catalog-target-data-folder-example.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-connector-catalog-target-database-example.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-connector-catalog-target.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-connector-simple.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-connector-virtual-connection-example.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-connector-virtual-connection.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-group-example.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/integration-group-use.svg (100%) rename {site/docs/services/omas => saved}/governance-engine/overview.md (100%) rename {site/docs/services/omas => saved/lineage-warehouse}/asset-lineage/overview.md (100%) rename {site/docs/guides/admin/servers/by-server-type => saved/lineage-warehouse}/configuring-a-lineage-warehouse.md (100%) rename {site/docs/connectors => saved/lineage-warehouse}/governance-daemon/lineage-warehouse-janus-connector.md (100%) rename {site/docs/guides/admin/servers/by-server-type => saved/lineage-warehouse}/lineage-warehouse-config.svg (100%) rename {site/docs/guides/admin/servers/by-section => saved/lineage-warehouse}/lineage-warehouse-services-section.md (100%) rename {site/docs/services => saved/lineage-warehouse}/lineage-warehouse-services.md (100%) rename {site/docs/services => saved/lineage-warehouse}/lineage-warehouse-services.svg (100%) create mode 100644 saved/lineage-warehouse/lineage-warehouse.md rename {site/docs/concepts => saved/lineage-warehouse}/lineage-warehouse.svg (100%) create mode 100644 site/docs/features/lineage-management/bad-data.svg create mode 100644 site/docs/features/lineage-management/lineage-horizontal-view.svg create mode 100644 site/docs/features/lineage-management/lineage-vertical-view.svg diff --git a/site/docs/services/omas/governance-engine/database-agents-and-information.svg b/saved/governance-engine/database-agents-and-information.svg similarity index 100% rename from site/docs/services/omas/governance-engine/database-agents-and-information.svg rename to saved/governance-engine/database-agents-and-information.svg diff --git a/site/docs/services/omas/governance-engine/governance-engine-omas.drawio b/saved/governance-engine/governance-engine-omas.drawio similarity index 100% rename from site/docs/services/omas/governance-engine/governance-engine-omas.drawio rename to saved/governance-engine/governance-engine-omas.drawio diff --git a/site/docs/services/omas/governance-engine/integration-connector-catalog-target-data-folder-example.svg b/saved/governance-engine/integration-connector-catalog-target-data-folder-example.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-connector-catalog-target-data-folder-example.svg rename to saved/governance-engine/integration-connector-catalog-target-data-folder-example.svg diff --git a/site/docs/services/omas/governance-engine/integration-connector-catalog-target-database-example.svg b/saved/governance-engine/integration-connector-catalog-target-database-example.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-connector-catalog-target-database-example.svg rename to saved/governance-engine/integration-connector-catalog-target-database-example.svg diff --git a/site/docs/services/omas/governance-engine/integration-connector-catalog-target.svg b/saved/governance-engine/integration-connector-catalog-target.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-connector-catalog-target.svg rename to saved/governance-engine/integration-connector-catalog-target.svg diff --git a/site/docs/services/omas/governance-engine/integration-connector-simple.svg b/saved/governance-engine/integration-connector-simple.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-connector-simple.svg rename to saved/governance-engine/integration-connector-simple.svg diff --git a/site/docs/services/omas/governance-engine/integration-connector-virtual-connection-example.svg b/saved/governance-engine/integration-connector-virtual-connection-example.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-connector-virtual-connection-example.svg rename to saved/governance-engine/integration-connector-virtual-connection-example.svg diff --git a/site/docs/services/omas/governance-engine/integration-connector-virtual-connection.svg b/saved/governance-engine/integration-connector-virtual-connection.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-connector-virtual-connection.svg rename to saved/governance-engine/integration-connector-virtual-connection.svg diff --git a/site/docs/services/omas/governance-engine/integration-group-example.svg b/saved/governance-engine/integration-group-example.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-group-example.svg rename to saved/governance-engine/integration-group-example.svg diff --git a/site/docs/services/omas/governance-engine/integration-group-use.svg b/saved/governance-engine/integration-group-use.svg similarity index 100% rename from site/docs/services/omas/governance-engine/integration-group-use.svg rename to saved/governance-engine/integration-group-use.svg diff --git a/site/docs/services/omas/governance-engine/overview.md b/saved/governance-engine/overview.md similarity index 100% rename from site/docs/services/omas/governance-engine/overview.md rename to saved/governance-engine/overview.md diff --git a/site/docs/services/omas/asset-lineage/overview.md b/saved/lineage-warehouse/asset-lineage/overview.md similarity index 100% rename from site/docs/services/omas/asset-lineage/overview.md rename to saved/lineage-warehouse/asset-lineage/overview.md diff --git a/site/docs/guides/admin/servers/by-server-type/configuring-a-lineage-warehouse.md b/saved/lineage-warehouse/configuring-a-lineage-warehouse.md similarity index 100% rename from site/docs/guides/admin/servers/by-server-type/configuring-a-lineage-warehouse.md rename to saved/lineage-warehouse/configuring-a-lineage-warehouse.md diff --git a/site/docs/connectors/governance-daemon/lineage-warehouse-janus-connector.md b/saved/lineage-warehouse/governance-daemon/lineage-warehouse-janus-connector.md similarity index 100% rename from site/docs/connectors/governance-daemon/lineage-warehouse-janus-connector.md rename to saved/lineage-warehouse/governance-daemon/lineage-warehouse-janus-connector.md diff --git a/site/docs/guides/admin/servers/by-server-type/lineage-warehouse-config.svg b/saved/lineage-warehouse/lineage-warehouse-config.svg similarity index 100% rename from site/docs/guides/admin/servers/by-server-type/lineage-warehouse-config.svg rename to saved/lineage-warehouse/lineage-warehouse-config.svg diff --git a/site/docs/guides/admin/servers/by-section/lineage-warehouse-services-section.md b/saved/lineage-warehouse/lineage-warehouse-services-section.md similarity index 100% rename from site/docs/guides/admin/servers/by-section/lineage-warehouse-services-section.md rename to saved/lineage-warehouse/lineage-warehouse-services-section.md diff --git a/site/docs/services/lineage-warehouse-services.md b/saved/lineage-warehouse/lineage-warehouse-services.md similarity index 100% rename from site/docs/services/lineage-warehouse-services.md rename to saved/lineage-warehouse/lineage-warehouse-services.md diff --git a/site/docs/services/lineage-warehouse-services.svg b/saved/lineage-warehouse/lineage-warehouse-services.svg similarity index 100% rename from site/docs/services/lineage-warehouse-services.svg rename to saved/lineage-warehouse/lineage-warehouse-services.svg diff --git a/saved/lineage-warehouse/lineage-warehouse.md b/saved/lineage-warehouse/lineage-warehouse.md new file mode 100644 index 0000000000..3f56c555dc --- /dev/null +++ b/saved/lineage-warehouse/lineage-warehouse.md @@ -0,0 +1,18 @@ +--- +hide: +- toc +--- + + + + +# Lineage Warehouse + +The lineage warehouse is a [governance server](/concepts/governance-server) +that manages a historical warehouse of lineage information. + +![Lineage Warehouse](lineage-warehouse.svg) + +Its behavior and configuration is described [in the administration guide](/guides/admin/servers/by-server-type/configuring-a-lineage-warehouse). + +--8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/lineage-warehouse.svg b/saved/lineage-warehouse/lineage-warehouse.svg similarity index 100% rename from site/docs/concepts/lineage-warehouse.svg rename to saved/lineage-warehouse/lineage-warehouse.svg diff --git a/site/docs/concepts/asset.md b/site/docs/concepts/asset.md index ad2a8d89e8..0fa1d338f7 100644 --- a/site/docs/concepts/asset.md +++ b/site/docs/concepts/asset.md @@ -56,7 +56,6 @@ Some examples: | [Asset Manager OMAS](/services/omas/asset-manager/overview) | provides a service for exchanging metadata about assets and related information with a third party [asset manager](/concepts/server-capability). This API supports the many-to-many correlation of identifiers used in the third party asset manager and the open metadata ecosystem. | | [Asset Owner OMAS](/services/omas/asset-owner/overview) | provides a service for the owner of an asset to classify and manage the asset, and understand how it is being used by the organization. | | [Data Manager OMAS](/services/omas/data-manager/overview) | enables a data manager (such as a database or file system) to maintain information about the assets it stores. | -| [Governance Engine OMAS](/services/omas/governance-engine/overview) | provides the metadata services for [governance action services](/concepts/governance-action-service) that verify, enhance and correct the properties of assets and their associated elements. | | [IT Infrastructure OMAS](/services/omas/it-infrastructure/overview) | provides a service for maintaining information about the IT assets and supporting infrastructure owned or used by an organization. | | [Data Science OMAS](/services/omas/data-science/overview) | provides a service for maintaining information about analytical models and related assets such as python notebooks. | diff --git a/site/docs/concepts/engine-host.md b/site/docs/concepts/engine-host.md index cd40be125f..b485807726 100644 --- a/site/docs/concepts/engine-host.md +++ b/site/docs/concepts/engine-host.md @@ -10,7 +10,7 @@ hide: An *engine host* is an [OMAG Server](/concepts/omag-server) that hosts one or more [governance engines](/concepts/governance-engine). Governance engines define collections of [governance services](/concepts/gpvernance-service) used to automate governance activity in the digital landscape. The governance services within the governance engines may access third party technology to perform their responsibilities or implement their behavior directly. -The engine host uses a [metadata access server](/concepts/metadata-access-server) to retrieve the [definitions of the governance engines](/governance-engine-definition) and the governance services within them. These definitions are retrieved through the [Governance Engine OMAS](/services/omas/governance-engine/overview). The Governance Engine OMAS also manages the definition of [governance action processes](/concepts/governance-action-process) that choreograph calls to the services in a governance engine in order to implement technical controls in the governance program. +The engine host uses a [metadata access server](/concepts/metadata-access-server) to retrieve the [definitions of the governance engines](/governance-engine-definition) and the governance services within them. These definitions are retrieved through the [Governance Server OMAS](/services/omas/governance-server/overview). The Governance Server OMAS also manages the definition of [governance action processes](/concepts/governance-action-process) that choreograph calls to the services in a governance engine in order to implement technical controls in the governance program. ![Engine host in an OMAG Server ecosystem](engine-host.svg) diff --git a/site/docs/concepts/governance-engine-definition.md b/site/docs/concepts/governance-engine-definition.md index 9b32ae976d..a68681f90b 100644 --- a/site/docs/concepts/governance-engine-definition.md +++ b/site/docs/concepts/governance-engine-definition.md @@ -37,11 +37,11 @@ Each governance engine definition describes a single engine and its specialized !!! education "Further information" - * Governance engine definitions are typically defined in a [governance engine pack](/guides/developer/open-metadata-archives/creating-governance-engine-packs) or through the [Governance Engine OMAS](/services/omas/governance-engine/overview) API. + * Governance engine definitions are typically defined in a [governance engine pack](/guides/developer/open-metadata-archives/creating-governance-engine-packs) or through the [Governance Server OMAS](/services/omas/governance-server/overview) API. - * The Open Metadata Types model [0461 Governance Action Engines](/types/4/0461-Governance-Engines) shows the types for a governance engine definition. + * The Open Metadata Types model [0461 Governance Engines](/types/4/0461-Governance-Engines) shows the types for a governance engine definition. - * Governance engines are run by the [Open Metadata Engine Services (OMES)](/services/omes) in an [Engine Host](/concepts/engine-host) OMAG Server. The [Engine Host Services](/services/engine-host-services) called the [Governance Engine OMAS](/services/omas/governance-engine/overview) + * Governance engines are run by the [Open Metadata Engine Services (OMES)](/services/omes) in an [Engine Host](/concepts/engine-host) OMAG Server. The [Engine Host Services](/services/engine-host-services) called the [Governance Server OMAS](/services/omas/governance-server/overview) --8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/concepts/governance-engine.md b/site/docs/concepts/governance-engine.md index c0900e067a..6d1ac39cde 100644 --- a/site/docs/concepts/governance-engine.md +++ b/site/docs/concepts/governance-engine.md @@ -26,7 +26,7 @@ Calls to the governance engine are made by creating [engine actions](/concepts/e ??? education "Further information" - Each governance engine has a unique name. A *governance engine definition* for this unique name is either created through the [Governance Engine OMAS](/services/omas/governance-engine/overview) API or as part of a [governance engine pack](/concepts/governance-engine-pack). The [governance engine is configured](/guides/admin/servers/by-section/engine-host-services-section) in the [engine host](/concepts/engine-host) using this unique name. + Each governance engine has a unique name. A *governance engine definition* for this unique name is either created through the [Governance Server OMAS](/services/omas/governance-server/overview) API or as part of a [governance engine pack](/concepts/governance-engine-pack). The [governance engine is configured](/guides/admin/servers/by-section/engine-host-services-section) in the [engine host](/concepts/engine-host) using this unique name. The open metadata types for this definition are in model [0461 - Governance Engines](/types/4/0461-Governance-Engines) - specifically notice **Governance Engine** and **GovernanceService** linked by the **SupportedGovernanceService** relationship. diff --git a/site/docs/concepts/governance-server.md b/site/docs/concepts/governance-server.md index b46bca32e4..152bcb3fc8 100644 --- a/site/docs/concepts/governance-server.md +++ b/site/docs/concepts/governance-server.md @@ -20,7 +20,6 @@ The table below shows the different types of governance servers and the OMASs th |:----------------------------------------------------|:-------------------------------------------------------------------------|:---------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Integration Daemon](/concepts/integraition-daemon) | Manages the capture of metadata through configured integration services. | [Integration Daemon Services](/services/integration-daemon-services) | [Governance Server OMAS](/services/omas/governance-server/overview) provides support for retrieving governance engine definitions. The [integration services](../../integration-services) define the OMASs that support each type of integration connector. | | [Engine Host](/concepts/engine-host) | Manages governance engines. | [Engine Host Services](/services/engine-host-services) | [Governance Server OMAS](/services/omas/governance-server/overview) provides support for retrieving governance engine definitions. The [engine services](../../engine-services) define the OMASs that support each type of governance engine. | -| [Lineage Warehouse](/concepts/lineage-warehouse) | Provides a historic reporting warehouse for lineage. | [Lineage Warehouse Services](/services/lineage-warehouse-services) | [Asset Lineage OMAS](/services/omas/asset-lineage/overview) | --8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/governance-service-definition.md b/site/docs/concepts/governance-service-definition.md index 0f7a61c1d1..5650df868c 100644 --- a/site/docs/concepts/governance-service-definition.md +++ b/site/docs/concepts/governance-service-definition.md @@ -74,8 +74,8 @@ A governance service definition is then linked into one or more [Governance Engi !!! education "Further information" - * Governance engine definitions are typically defined in a [governance engine pack](/guides/developer/open-metadata-archives/creating-governance-engine-packs) or through the [Governance Engine OMAS](/services/omas/governance-engine/overview) API. This will typically include the governance service definitions. - * Governance engines are run by the [Open Metadata Engine Services (OMES)](/services/omes) in an [Engine Host](/concepts/engine-host) OMAG Server. The [Engine Host Services](/services/engine-host-services) called the [Governance Engine OMAS](/services/omas/governance-engine/overview). + * Governance engine definitions are typically defined in a [governance engine pack](/guides/developer/open-metadata-archives/creating-governance-engine-packs) or through the [Governance Server OMAS](/services/omas/governance-server/overview) API. This will typically include the governance service definitions. + * Governance engines are run by the [Open Metadata Engine Services (OMES)](/services/omes) in an [Engine Host](/concepts/engine-host) OMAG Server. The [Engine Host Services](/services/engine-host-services) called the [Governance Server OMAS](/services/omas/governance-server/overview). --8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/concepts/governance-service.md b/site/docs/concepts/governance-service.md index c4f96178d4..d81b2045bb 100644 --- a/site/docs/concepts/governance-service.md +++ b/site/docs/concepts/governance-service.md @@ -17,7 +17,7 @@ Instructions for implementing your own governance services are in the developer ## Support for running governance services -The [Governance Engine OMAS](/services/omas/governance-engine/overview) provides: +The [Governance Server OMAS](/services/omas/governance-server/overview) provides: * The API to create [governance engine definitions](/concepts/governance-engine) for the governance services. * The API to link governance services together into [governance action processes](/concepts/governance-action-process). diff --git a/site/docs/concepts/guard.md b/site/docs/concepts/guard.md index 0b6860f77e..b6a288b4fe 100644 --- a/site/docs/concepts/guard.md +++ b/site/docs/concepts/guard.md @@ -5,7 +5,7 @@ # Guards Guards are labels that are created by [governance services](/concepts/governance-service) -and are used by the [Governance Engine OMAS](/services/omas/governance-engine/overview) to +and are used by the [Governance Server OMAS](/services/omas/governance-server/overview) to determine which [engine action](/concepts/engine-action) to run next. diff --git a/site/docs/concepts/index.md b/site/docs/concepts/index.md index 5d4bda1062..5f0ea5fc7e 100644 --- a/site/docs/concepts/index.md +++ b/site/docs/concepts/index.md @@ -157,11 +157,11 @@ - [Governance Domain Identifier](/concepts/governance-domain) - [Governance Engine](/concepts/governance-engine) - [Governance Engine Definition](/concepts/governance-engine-definition) -- [Governance Engine OMAS](/services/omas/governance-engine/overview) - [Governance Program](/services/omas/governance-program/overview) - [Governance Program OMAS](/services/omas/governance-program/overview) - [Governance Request Type](/concepts/governance-request-type) - [Governance Servers](/concepts/governance-server) +- [Governance Server OMAS](/services/omas/governance-server/overview) - [Governance Service](/concepts/governance-service) - [Governance Service Definition](/concepts/governance-service-definition) - [Governance Zones](/concepts/governance-zone) @@ -222,7 +222,6 @@ - [Lineage](/features/lineage-management/overview) - [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) - [Lineage Warehouse](/concepts/lineage-warehouse) -- [Lineage Warehouse Services](/services/lineage-warehouse-services) - [Local Metadata Collection Id](/concepts/metadata-collection-id) - [Local Repository Services (OMRS)](/services/omrs) diff --git a/site/docs/concepts/integration-group.md b/site/docs/concepts/integration-group.md index 0db073dc15..6afeaeee8f 100644 --- a/site/docs/concepts/integration-group.md +++ b/site/docs/concepts/integration-group.md @@ -44,7 +44,7 @@ An [integration daemon is configured](/guides/admin/servers/by-server-type/confi ???+ education "Further information" - Each integration group has a unique name. An *IntegrationGroup* entity with a qualifiedName set to this unique name is created through the [Governance Engine OMAS](/services/omas/governance-engine/overview) API. This API is also used to create and link the integration connector definitions into the integration group. + Each integration group has a unique name. An *IntegrationGroup* entity with a qualifiedName set to this unique name is created through the [Governance Server OMAS](/services/omas/governance-server/overview) API. This API is also used to create and link the integration connector definitions into the integration group. The [integration group's qualifiedName is configured](/guides/admin/servers/by-section/integration-daemon-services-section) in the [integration daemon](/concepts/integration-daemon). diff --git a/site/docs/concepts/lineage-warehouse.md b/site/docs/concepts/lineage-warehouse.md index 3f56c555dc..095c4fb020 100644 --- a/site/docs/concepts/lineage-warehouse.md +++ b/site/docs/concepts/lineage-warehouse.md @@ -8,11 +8,8 @@ hide: # Lineage Warehouse -The lineage warehouse is a [governance server](/concepts/governance-server) -that manages a historical warehouse of lineage information. +The lineage warehouse is a data store +that manages a historical warehouse of lineage information. See [Lineage Management](/features/lineage-managent/overview) for details of how it is populated and used. -![Lineage Warehouse](lineage-warehouse.svg) - -Its behavior and configuration is described [in the administration guide](/guides/admin/servers/by-server-type/configuring-a-lineage-warehouse). --8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/omag-server.md b/site/docs/concepts/omag-server.md index 50ec5985e9..3e1928e224 100644 --- a/site/docs/concepts/omag-server.md +++ b/site/docs/concepts/omag-server.md @@ -48,7 +48,6 @@ The types of OMAG Server are shown in Figure 1. The hierarchy groups similar typ - [Governance Server](/concepts/governance-server) - supports the use of metadata in the broader IT landscape. - [Engine Host](/concepts/engine-host) - provides a runtime for a specific type of [governance engine](/concepts/governance-engine) that runs [governance services](/concepts/governance-service) to monitor, enhance and govern the open metadata ecosystem. - [Integration Daemon](/concepts/integration-daemon) - manages the synchronization of metadata with third party technology that can not call the access services directly. through the [integration services](/services/omis). - - [Lineage Warehouse](/concepts/lineage-warehouse) - Manages the collation of lineage information and maintains it in a format for reporting. This includes the state of the lineage at different points in time. ## Server connectivity diff --git a/site/docs/concepts/omag-subsystem.md b/site/docs/concepts/omag-subsystem.md index 5355f48d84..979548ac73 100644 --- a/site/docs/concepts/omag-subsystem.md +++ b/site/docs/concepts/omag-subsystem.md @@ -31,10 +31,8 @@ Fixed services that are always present in the platform: - Governance server services: - - [Data engine services](/services/data-engine-proxy-services) implements the services for the [Data Engine Proxy](/concepts/date-engine-proxy). - [Engine host services](/services/engine-host-services) implements the services for the [Engine Host](/concepts/engine-host). - [Integration daemon services](/services/integration-daemon-services) implements the services for the [Integration Daemon](/concepts/integration-daemon). - - [Lineage warehouse services](/services/lineage-warehouse-services) implements services for the [Lineage Warehouse](/concepts/lineage-warehouse). - [Connected Asset Services](/services/ocf-metadata-management) for supporting the server-side REST API behind the `ConnectedAssetProperties` interface of an [Open Connector Framework (OCF)](/frameworks/ocf/overview) connector. diff --git a/site/docs/concepts/open-lineage-server.md b/site/docs/concepts/open-lineage-server.md index 79c5a41f77..e781578217 100644 --- a/site/docs/concepts/open-lineage-server.md +++ b/site/docs/concepts/open-lineage-server.md @@ -8,6 +8,6 @@ hide: # Open lineage Server -The Open Lineage Server was renamed to the [Lineage Warehouse](/concepts/lineage-warehouse) in V5.0 of Egeria. +Open Lineage is processed by the [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) running in the [Integration Daemon](/concepts/integration-daemon). Details of how it is used can be found in the [Lineage Management](/features/lineage-management/overview) feature. --8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/schema.md b/site/docs/concepts/schema.md index 347a7bb4e2..3df13ab997 100644 --- a/site/docs/concepts/schema.md +++ b/site/docs/concepts/schema.md @@ -137,7 +137,6 @@ APIs that support the definition of schemas: * [Asset Owner OMAS](/services/omas/asset-owner/overview) * [Asset Manager OMAS](/services/omas/asset-manager/overview) * [Data Manager OMAS](/services/omas/data-manager/overview) -* [Governance Engine OMAS](/services/omas/governance-engine/overview) * [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) * [Database Integrator OMIS](/services/omis/database-integrator/overview) * [Files Integrator OMIS](/services/omis/files-integrator/overview) diff --git a/site/docs/connectors/governance-action/generic-element-watchdog-governance-action-service.md b/site/docs/connectors/governance-action/generic-element-watchdog-governance-action-service.md index c53af0f5bd..28fa433425 100644 --- a/site/docs/connectors/governance-action/generic-element-watchdog-governance-action-service.md +++ b/site/docs/connectors/governance-action/generic-element-watchdog-governance-action-service.md @@ -83,7 +83,7 @@ The rest of the properties are the governance action processes to call for speci } ``` - This is its connection definition to use when creating the definition of the governance action service using the [Governance Engine OMAS](/services/omas/governance-engine). Remove the configuration properties that are not required. Replace `{{typeName}}`, `{{guid}}` and `{{processQualifiedName}}` as required. + This is its connection definition to use when creating the definition of the governance action service using the [Governance Server OMAS](/services/omas/governance-server/overview). Remove the configuration properties that are not required. Replace `{{typeName}}`, `{{guid}}` and `{{processQualifiedName}}` as required. ## Governance action settings diff --git a/site/docs/connectors/governance-action/move-copy-file-provisioning-governance-action-service.md b/site/docs/connectors/governance-action/move-copy-file-provisioning-governance-action-service.md index 0bcf713e8c..e08a13f96b 100644 --- a/site/docs/connectors/governance-action/move-copy-file-provisioning-governance-action-service.md +++ b/site/docs/connectors/governance-action/move-copy-file-provisioning-governance-action-service.md @@ -74,11 +74,9 @@ processing path and the folders represent the data sets. Irrespective of the pattern used, the resulting changes to the metadata can lead to the triggering of additional activity in the open metadata ecosystem. For example: -* The [Asset Lineage OMAS](/services/omas/asset-lineage/overview) -may publish the lineage to the [Lineage Warehouse](/concepts/lineage-warehouse). +* An [integration connector](/services/omas/asset-manager/overview) may publish the lineage to the [Lineage Warehouse](/concepts/lineage-warehouse). -* Additional [Governance Action Services](/concepts/governance-action-service) -may be triggered. +* Additional [Governance Action Services](/concepts/governance-action-service) may be triggered to provide additional enrichment of the lineage graph. ## Configuration diff --git a/site/docs/connectors/index.md b/site/docs/connectors/index.md index d158bbeab1..7b476d06ed 100644 --- a/site/docs/connectors/index.md +++ b/site/docs/connectors/index.md @@ -300,16 +300,6 @@ Egeria's [Open Metadata Repository Services (OMRS)](/services/omrs) provides a d The connection for this connector is configured in the `LocalRepositoryRemoteConnection` property of the cohort member's [Local Repository Configuration](/user/guides/admin/servers/by-section/#configuring-the-local-repository-store). -### Governance Daemon Connectors - ----8<-- "snippets/connectors/governance-daemon-intro.md" - -| Connector | Description | -|------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Lineage Warehouse Janus Connector](/connectors/governance-daemon/lineage-warehouse-janus-connector) | The Lineage Warehouse connectors provide plugins to the [Lineage Warehouse](/concepts/lineage-warehouse) that allow the Lineage Warehouse Services to connect with databases. | - - - --8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md b/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md index 7eb4c69628..4b55a2d44e 100644 --- a/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md +++ b/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md @@ -184,7 +184,7 @@ By default, Egeria generates its topic names using the following naming conventi These need to be done FOR EACH SERVER configured to run one or more OMASs. (For example for Coco Pharmaceuticals this might include `cocoMDS1`, `cocoMDS2`, `cocoMDS3` etc). - Then, FOR EACH OMAS configured (eg Asset Consumer OMAS, Data Platform OMAS, Governance Engine OMAS etc.) that supports topics, you will need to create the topic it is using. + Then, FOR EACH OMAS configured (eg Asset Consumer OMAS, Data Platform OMAS, Governance Server OMAS etc.) that supports topics, you will need to create the topic it is using. This is the default naming convention: diff --git a/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md b/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md index 8cdc96d492..84c7a89924 100644 --- a/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md +++ b/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md @@ -101,7 +101,7 @@ In the `omag-server-platform` assembly, there is a jar file called `egeria-platf DevOps OMAS: Manage a DevOps pipeline Software Developer OMAS: Interact with software development tools Project Management OMAS: Manage governance related projects - Governance Engine OMAS: Set up an operational governance engine + Governance Server OMAS: Set up an operational governance engine Data Privacy OMAS: Manage governance of privacy Security Manager OMAS: Manages exchange of metadata with a security service Governance Action OMES: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. diff --git a/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-configured.md b/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-configured.md index f22786d0e3..f7133ba9f3 100644 --- a/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-configured.md +++ b/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-configured.md @@ -24,7 +24,7 @@ * **DevOps OMAS**: Manage a DevOps pipeline * **Software Developer OMAS**: Interact with software development tools * **Project Management OMAS**: Manage governance related projects -* **Governance Engine OMAS**: Set up an operational governance engine +* **Governance Server OMAS**: Set up an operational governance engine * **Data Privacy OMAS**: Manage governance of privacy * **Security Manager OMAS**: Manages exchange of metadata with a security service * **Repository Governance OMES**: Maintains open metadata archives based on the activity in the connected cohorts. diff --git a/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-running.md b/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-running.md index 3c8cf5f512..ec4fb5f68c 100644 --- a/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-running.md +++ b/site/docs/education/egeria-dojo/developer/egeria-platform-report-mds1-running.md @@ -25,7 +25,7 @@ * **DevOps OMAS**: Manage a DevOps pipeline * **Software Developer OMAS**: Interact with software development tools * **Project Management OMAS**: Manage governance related projects -* **Governance Engine OMAS**: Set up an operational governance engine +* **Governance Server OMAS**: Set up an operational governance engine * **Data Privacy OMAS**: Manage governance of privacy * **Security Manager OMAS**: Manages exchange of metadata with a security service * **Repository Governance OMES**: Maintains open metadata archives based on the activity in the connected cohorts. diff --git a/site/docs/education/egeria-dojo/developer/egeria-platform-report-no-servers.md b/site/docs/education/egeria-dojo/developer/egeria-platform-report-no-servers.md index 104fa59769..9c20b12035 100644 --- a/site/docs/education/egeria-dojo/developer/egeria-platform-report-no-servers.md +++ b/site/docs/education/egeria-dojo/developer/egeria-platform-report-no-servers.md @@ -24,7 +24,7 @@ * **DevOps OMAS**: Manage a DevOps pipeline * **Software Developer OMAS**: Interact with software development tools * **Project Management OMAS**: Manage governance related projects -* **Governance Engine OMAS**: Set up an operational governance engine +* **Governance Server OMAS**: Set up an operational governance engine * **Data Privacy OMAS**: Manage governance of privacy * **Security Manager OMAS**: Manages exchange of metadata with a security service * **Repository Governance OMES**: Maintains open metadata archives based on the activity in the connected cohorts. diff --git a/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-no-servers.md b/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-no-servers.md index e0b709b753..251da1c8ba 100644 --- a/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-no-servers.md +++ b/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-no-servers.md @@ -24,7 +24,7 @@ * **DevOps OMAS**: Manage a DevOps pipeline * **Software Developer OMAS**: Interact with software development tools * **Project Management OMAS**: Manage governance related projects -* **Governance Engine OMAS**: Set up an operational governance engine +* **Governance Server OMAS**: Set up an operational governance engine * **Data Privacy OMAS**: Manage governance of privacy * **Security Manager OMAS**: Manages exchange of metadata with a security service * **Governance Action OMES**: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. diff --git a/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-config.md b/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-config.md index fc6cb0e97d..38dda0c5ef 100644 --- a/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-config.md +++ b/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-config.md @@ -24,7 +24,7 @@ * **DevOps OMAS**: Manage a DevOps pipeline * **Software Developer OMAS**: Interact with software development tools * **Project Management OMAS**: Manage governance related projects -* **Governance Engine OMAS**: Set up an operational governance engine +* **Governance Server OMAS**: Set up an operational governance engine * **Data Privacy OMAS**: Manage governance of privacy * **Security Manager OMAS**: Manages exchange of metadata with a security service * **Governance Action OMES**: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. diff --git a/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-running.md b/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-running.md index 8d9b1ee190..3e55b40b58 100644 --- a/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-running.md +++ b/site/docs/education/tutorials/developer-intellij-tutorial/egeria-platform-report-server-running.md @@ -24,7 +24,7 @@ * **DevOps OMAS**: Manage a DevOps pipeline * **Software Developer OMAS**: Interact with software development tools * **Project Management OMAS**: Manage governance related projects -* **Governance Engine OMAS**: Set up an operational governance engine +* **Governance Server OMAS**: Set up an operational governance engine * **Data Privacy OMAS**: Manage governance of privacy * **Security Manager OMAS**: Manages exchange of metadata with a security service * **Governance Action OMES**: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. diff --git a/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md b/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md index b9d917cd7a..d206481c32 100644 --- a/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md +++ b/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md @@ -101,7 +101,7 @@ In the `omag-server-platform` assembly, there is a jar file called `egeria-platf DevOps OMAS: Manage a DevOps pipeline Software Developer OMAS: Interact with software development tools Project Management OMAS: Manage governance related projects - Governance Engine OMAS: Set up an operational governance engine + Governance Server OMAS: Set up an operational governance engine Data Privacy OMAS: Manage governance of privacy Security Manager OMAS: Manages exchange of metadata with a security service Governance Action OMES: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. diff --git a/site/docs/features/lineage-management/bad-data.svg b/site/docs/features/lineage-management/bad-data.svg new file mode 100644 index 0000000000..5759e4fe2a --- /dev/null +++ b/site/docs/features/lineage-management/bad-data.svg @@ -0,0 +1,4 @@ + + + +
World-Wide Sales Results By Month
Month
Sales Income
Americas Sales Results By Month
Month
Sales Income
Europe/Africa Sales Results By Month
Month
Sales Income
Asia Sales Results By Month
Month
Sales Income
 \ No newline at end of file diff --git a/site/docs/features/lineage-management/lineage-architecture.svg b/site/docs/features/lineage-management/lineage-architecture.svg index ed6d86ae86..0cf8b86a7c 100644 --- a/site/docs/features/lineage-management/lineage-architecture.svg +++ b/site/docs/features/lineage-management/lineage-architecture.svg @@ -1,4 +1,4 @@ -
Integration Daemon
Metadata Access
Server
Cohort
Database
Servers
Document
Managers
File systems
API
Managers
Reporting tools
and Applications
Event
Managers
Data Catalogs
Processing
Engines
Security
Enforcement
Tools
Metadata Access
Server
Lineage Warehouse
Egeria General
User Interface
Egeria Ecosystem
User Interface
Metadata Access
Server
Engine Host
Capture
Stewardship
Preservation and Use
OpenLineage
Log Store
 \ No newline at end of file +
Integration Daemon
Metadata Access
Server
Cohort
Database
Servers
Document
Managers
File systems
API
Managers
Reporting tools
and Applications
Event
Managers
Data Catalogs
Processing
Engines
Security
Enforcement
Tools
Metadata Access
Server
Integration
Daemon
Lineage
Warehouse
Metadata Access
Server
Engine Host
Capture
Stewardship
Preservation and Use
OpenLineage
Log Store
View Server
 \ No newline at end of file diff --git a/site/docs/features/lineage-management/lineage-horizontal-view.svg b/site/docs/features/lineage-management/lineage-horizontal-view.svg new file mode 100644 index 0000000000..9d23f37258 --- /dev/null +++ b/site/docs/features/lineage-management/lineage-horizontal-view.svg @@ -0,0 +1,4 @@ + + + +
Process
Param 1
Param 2
CSV
DataStore
Schema
Col 1
Col 2
DataFlow
Schema
Schema
CSV
DataStore
CSV
DataStore
Process
Param 3
Column 1
Table
LineageMapping
Database
DataFlow
DataFlow
DataFlow
 \ No newline at end of file diff --git a/site/docs/features/lineage-management/lineage-vertical-view.svg b/site/docs/features/lineage-management/lineage-vertical-view.svg new file mode 100644 index 0000000000..f488824682 --- /dev/null +++ b/site/docs/features/lineage-management/lineage-vertical-view.svg @@ -0,0 +1,4 @@ + + + +
Column
Table
Database
Glossary
GlossaryTerm
(First Name)
GlossaryTerm (Employee)
CSV
DataStore
Schema
Column
SemanticAssignment
SemanticAssignment
 \ No newline at end of file diff --git a/site/docs/features/lineage-management/lineage-warehouse.svg b/site/docs/features/lineage-management/lineage-warehouse.svg index 9bd87f854b..579a235bf6 100644 --- a/site/docs/features/lineage-management/lineage-warehouse.svg +++ b/site/docs/features/lineage-management/lineage-warehouse.svg @@ -1,4 +1,4 @@ - + -

Lineage Warehouse
 Lineage Warehouse
Asset Lineage Client
Asset Lineag...
Lineage Graph Connector
Lineage Graph Connector
Lineage Graph
Lineage Graph
Asset Lineage
OMAS
Asset Lineage...
Open Lineage Services
Open Lineage Services
Lineage REST API
Lineage REST API
Lineage GUI
Lineage GUI
Open Lineage Services REST Client
Open Lineage Se...
Lineage Event Listener
Lineage Event Listener
Metadata Access Server
Metadata Access Server
Lineage Topic
Lineage Topic
REST API
REST API
REST API
REST API
1
1
2
2
4
4
3
3
5
5
Lineage
Jobs
Lineage...
Cohort Metadata
Cohort MetadataViewer does not support full SVG 1.1 \ No newline at end of file +
Lineage Warehouse
Asset Lineage
OMAS
Metadata Access Server
REST API
Open Lineage Services
Open Lineage Server
Integration Daemon
Third Party Technology
Asset Manager
OMAS
Metadata Access Server
Asset Manager Topic
REST API
Lineage Integrator
 OMIS
Integration Daemon
Lineage Warehouse Connector
 \ No newline at end of file diff --git a/site/docs/features/lineage-management/lineage.drawio b/site/docs/features/lineage-management/lineage.drawio index f48fb4af4c..7c30b7ef26 100644 --- a/site/docs/features/lineage-management/lineage.drawio +++ b/site/docs/features/lineage-management/lineage.drawio @@ -1,6 +1,160 @@ - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + @@ -132,7 +286,7 @@ - + @@ -140,78 +294,8 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + @@ -244,7 +328,7 @@ - + @@ -261,6 +345,12 @@ + + + + + + @@ -1769,153 +1859,342 @@ - - + + - - - - + + + + - - + + - - - - - + + - - - - - + + - - - - + + - - - - - - + + - - + + - - + + - - + + - - - - - + + - - - - - + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - + + - - + + - - + + - - + + - - - - - + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - + - - + + - - + + + + - - + + - - - - + + + - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - + + - - - - - - + + - - + + - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/features/lineage-management/overview.md b/site/docs/features/lineage-management/overview.md index b7084a6c60..bd29b13f7c 100644 --- a/site/docs/features/lineage-management/overview.md +++ b/site/docs/features/lineage-management/overview.md @@ -11,6 +11,13 @@ Lineage shows how data flows from its origins to its various destinations. This * whether the operational processes that implement the data flows are executing correctly (known as *governance by expectation*). +## An example + +Take a look at this example. The sales are growing in every region, but when the sales are aggregated, they appear to be going down. How is this possible? + +![Bad Data?](bad-data.svg) + +Lineage helps to track down whether there is an error in the aggregation pipeline code, or whether one of the regions failed to deliver their data on time, or the aggregation pipeline is pulling data from the wrong source. ## The lineage graph @@ -48,12 +55,12 @@ Figure 4 shows Egeria's architecture for lineage. There are three parts to it: * *Stewardship* - the lineage information from each of the technologies is linked together. Where the naming of data sources and processes is consistent, this assembling of the lineage graph is automatic. However, experience shows that if it can be different, it will be different. Many technologies make their own choices in naming and so governance action services along with human stewardship is required to match and link the graphs together. The governance action services run in the [Engine Host](/concepts/engine-host) server. They automatically add the relationships between the lineage contributions from each technology that may need to be verified by a human steward. The human steward may also manually add relationships where there is no well known pattern that can be encoded in a governance action services. Stewardship also involves analysis of the lineage to ensure the digital landscape is operating as it should. -* *Preservation and Use* - Once the lineage graphs are assembled, the lineage can be viewed and analysed from a business perspective. Could, for example, the operation of the digital landscape be optimized? Lineage is accessible through standard open metadata queries. However, since the lineage data is large, lineage is automatically captured and stored in the [Lineage Warehouse](/concepts/lineage-warehouse). This optimizes the lineage graphs for quick retrieval and analysis. Its presence allows lineage data to be regularly archived from the operational open metadata ecosystem. This is particularly important in regulated industries where lineage for critical reports may need to be kept for many years. +* *Preservation and Use* - Once the lineage graphs are assembled, the lineage can be viewed and analysed from a business perspective. Could, for example, the operation of the digital landscape be optimized? Lineage is accessible through standard open metadata queries. However, since the lineage data is typically large, lineage can be distributed via the integration daemon to a [Lineage Warehouse](/concepts/lineage-warehouse). This optimizes the lineage graphs for quick retrieval and analysis. Its presence allows lineage data to be regularly archived from the operational open metadata ecosystem. This is particularly important in regulated industries where lineage for critical reports may need to be kept for many years. The three parts of the lineage architecture are summarized in figure 4. ![Figure 4](/features/lineage-management/lineage-architecture.svg) -> **Figure 4:** The lineage architecture showing the three phases of (1) lineage capture typically through Egeria's automated cataloguing capabilities, (2) automated and human stewardship coordinated by the engine host server to stitch the lineage contributions together into full data flows, and finally (3) lineage preservation and use in the Lineage Warehouse. +> **Figure 4:** The lineage architecture showing the three phases of (1) lineage capture typically through Egeria's automated cataloguing capabilities, (2) automated and human stewardship to stitch the lineage contributions together into full data flows, and finally (3) lineage preservation and use in the lineage warehouse. ## Lineage capture @@ -347,46 +354,36 @@ Governing expectations is where the lineage information is used to validate that ## Lineage preservation and use -Design lineage can be consolidated and exported for preservation by the [Asset Lineage OMAS](/services/omas/asset-lineage/overview) and then stored in the [Lineage Warehouse](/concepts/lineage-warehouse). +Design lineage can be consolidated and exported for preservation by the integration daemon's [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) and then stored in the [Lineage Warehouse](/concepts/lineage-warehouse). Figure 35 shows metadata capture using the [Integration Daemon](/concepts/integration-daemon/) to retrieve lineage metadata in automated way and push metadata into the open metadata ecosystem so that is it picked up by the Asset Lineage OMAS and then stored by the Lineage Warehouse. ![Figure 35](/features/lineage-management/lineage-capture-for-lineage-warehouse.svg) -> **Figure 35:** Capturing lineage using the Integration Daemon, Partner OMAS(s) and Asset Lineage OMAS +> **Figure 35:** Capturing lineage using the Integration Daemon, Partner OMAS(s) and Asset Manager OMAS Once the lineage graphs are assembled in the Lineage Warehouse, the lineage can be viewed and analyzed for business cases such as traceability of data, impact analysis or data processes monitoring. ### Building a lineage warehouse -The [Lineage Warehouse](/concepts/lineage-warehouse) is the warehouse for lineage. It is the destination store for all relevant lineage data graphs. +The [Lineage Warehouse](/concepts/lineage-warehouse) is the warehouse for lineage. It is fed by a specialized [integration connector](/concepts/integration-connector) running in the integration daemon. The integration connector receives lineage information though the [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview). ![Figure 36](/features/lineage-management/lineage-warehouse.svg) > **Figure 36:** Lineage Warehouse preservation and use details -1. Metadata instance events from the cohort are distributed to Metadata Access Server running Asset Lineage OMAS. - -2. Asset Lineage OMAS selects instance types suitable for lineage. It retrieves asset related metadata and completes the asset context portion of the graph. Once ready, this lineage context graph structured as vertices and edges is sent on Lineage Output Topic for further processing and preservation. - -3. Lineage context graph events are consumed up by Open Lineage Services Lineage Event listener that in turn uses technology connector to persist the lineage elements building up the large lineage graph. - -4. Lineage jobs run in background to (a) poll for changes and request updates using REST API; and (b) to scan the lineage graph and augment lineage paths for optimized querying. - -5. Lineage REST API is used to query asset lineage for specific business views such as horizontal and vertical. - ### User views #### Horizontal lineage Organizations use horizontal lineage view to understand and visualize how their data flows from origin to various destinations enabling comprehensive data traceability. This view can represent both design or operational lineage aspect with different styles and level of details. -![Figure 37](/features/lineage-management/lineage-warehouse-horizontal-view.svg) +![Figure 37](/features/lineage-management/lineage-horizontal-view.svg) > **Figure 37:** Lineage between data stores and processes on different levels #### Vertical lineage Organizations use vertical lineage view to visualize how business concepts such as glossaries, terms are mapped to data assets and related elements. This allows business users to understand how digital landscape is implemented and perform impact analysis when needed. -![Figure 38](/features/lineage-management/lineage-warehouse-vertical-view.svg) +![Figure 38](/features/lineage-management/lineage-vertical-view.svg) > **Figure 38:** Lineage between business glossaries and data stores !!! summary "Summary" @@ -405,8 +402,7 @@ Organizations use vertical lineage view to visualize how business concepts such APIs for retrieving lineage - - [Lineage Warehouse Services](/services/lineage-warehouse-services) - + - [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) - [Asset Owner](/services/omas/asset-owner/overview) and [Asset Consumer](/services/omas/asset-consumer/overview) through the [Open Connector Framework (OCF)](/frameworks/ocf/overview). diff --git a/site/docs/guides/admin/egeria-admin.drawio b/site/docs/guides/admin/egeria-admin.drawio index 6f74f1fb5e..75baa0abb5 100644 --- a/site/docs/guides/admin/egeria-admin.drawio +++ b/site/docs/guides/admin/egeria-admin.drawio @@ -1,4 +1,4 @@ - + @@ -3758,7 +3758,7 @@ - + @@ -3778,17 +3778,11 @@ - + - - - - - - @@ -3826,17 +3820,11 @@ - + - - - - - - @@ -3850,13 +3838,13 @@ - + - + @@ -3910,13 +3898,13 @@ - + - + @@ -3928,7 +3916,7 @@ - + @@ -3937,7 +3925,7 @@ - + @@ -4168,76 +4156,76 @@ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + @@ -4249,13 +4237,13 @@ - + - + diff --git a/site/docs/guides/admin/servers/by-section/configuration-document-structure.svg b/site/docs/guides/admin/servers/by-section/configuration-document-structure.svg index 1f64b0815a..cde301e54b 100644 --- a/site/docs/guides/admin/servers/by-section/configuration-document-structure.svg +++ b/site/docs/guides/admin/servers/by-section/configuration-document-structure.svg @@ -1,4 +1,4 @@ -
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Conformance Suite Services
Engine Host Services
Lineage Warehouse Services
Data Engine Proxy Services
Audit Trail
Repository Services
Server Security Connection
Integration Daemon Services
View Services
Basic Server Properties
Audit Log Destinations
Metadata Archives
Enterprise Access
Cohort List
Local Repository
Default values used in the
configuration services that
send notifications.
Values used by
all types of server.
Configuration to control
the activation of specialized
services that are specific to
particular types of server.
Values typically
maintained by Egeria
 \ No newline at end of file +
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Conformance Suite Services
Engine Host Services
Audit Trail
Repository Services
Server Security Connection
Integration Daemon Services
View Services
Basic Server Properties
Audit Log Destinations
Metadata Archives
Enterprise Access
Cohort List
Local Repository
Default values used in the
configuration services that
send notifications.
Values used by
all types of server.
Configuration to control
the activation of specialized
services that are specific to
particular types of server.
Values typically
maintained by Egeria
 \ No newline at end of file diff --git a/site/docs/guides/admin/servers/by-section/index.md b/site/docs/guides/admin/servers/by-section/index.md index aabe9d91c3..2627c4eabd 100644 --- a/site/docs/guides/admin/servers/by-section/index.md +++ b/site/docs/guides/admin/servers/by-section/index.md @@ -24,7 +24,6 @@ The configuration document is divided into sections. Some sections contain prop * The [*Engine Host Services*](engine-host-services-section.md) define which governance engines are activated in an [engine host](/concepts/engine-host) server. * The [*Integration Daemon Services*](integration-daemon-services-section.md) define which integration connectors run in an [integration daemon](/concepts/integration-daemon) server. * The [*View Services*](view-services-section.md) define which REST APIs are enabled in a [view server](/concepts/view-server). -* The [*Lineage Warehouse Services*](lineage-warehouse-services-section.md) define the properties of the repository for a [lineage warehouse](/concepts/lineage-warehouse) server. diff --git a/site/docs/guides/admin/servers/by-server-type/configuration-document-structure-by-server-type.svg b/site/docs/guides/admin/servers/by-server-type/configuration-document-structure-by-server-type.svg index 9ec94cb3b1..4db3011793 100644 --- a/site/docs/guides/admin/servers/by-server-type/configuration-document-structure-by-server-type.svg +++ b/site/docs/guides/admin/servers/by-server-type/configuration-document-structure-by-server-type.svg @@ -1,4 +1,4 @@ -
Engine Host
Conformance
Test Server
Repository Proxy
Metadata
Access Store
Lineage
Warehouse
Integration
Daemon
Metadata
Access Point
View
Server
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Enterprise Access
Cohort List
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Metadata Archives
Enterprise Access
Cohort List
Local Repository
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Cohort List
Local Repository
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Audit Trail
Repository Services
Server Security Connection
View Services
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Audit Trail
Repository Services
Server Security Connection
Integration Daemon Services
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Engine Host Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Lineage Warehouse Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Conformance Suite Services
Audit Trail
Server Security Connection
Basic Server Properties
Repository Services
Audit Log Destinations
Cohort List
Local Repository
Default values used in the
configuration services that
send notifications.
Values used by
all types of server.
Configuration to control
the activation of specialized
services that are specific to
particular types of server.
Values typically
maintained by Egeria
 \ No newline at end of file +
Engine Host
Conformance
Test Server
Repository Proxy
Metadata
Access Store
Integration
Daemon
Metadata
Access Point
View
Server
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Enterprise Access
Cohort List
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Metadata Archives
Enterprise Access
Cohort List
Local Repository
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Cohort List
Local Repository
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Audit Trail
Repository Services
Server Security Connection
View Services
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Audit Trail
Repository Services
Server Security Connection
Integration Daemon Services
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Engine Host Services
Audit Trail
Repository Services
Server Security Connection
Basic Server Properties
Audit Log Destinations
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Conformance Suite Services
Audit Trail
Server Security Connection
Basic Server Properties
Repository Services
Audit Log Destinations
Cohort List
Local Repository
Default values used in the
configuration services that
send notifications.
Values used by
all types of server.
Configuration to control
the activation of specialized
services that are specific to
particular types of server.
Values typically
maintained by Egeria
 \ No newline at end of file diff --git a/site/docs/guides/admin/servers/by-server-type/index.md b/site/docs/guides/admin/servers/by-server-type/index.md index b0251449aa..f748cf9d22 100644 --- a/site/docs/guides/admin/servers/by-server-type/index.md +++ b/site/docs/guides/admin/servers/by-server-type/index.md @@ -23,7 +23,6 @@ The configuration document is divided into sections. Some sections contain prop * [View Server](configuring-a-view-server.md) - this server has specialist REST APIs for UIs and other non-Java environments, such as Python. * [Integration Daemon](configuring-an-integration-daemon.md) - this server hosts integration connectors that synchronize metadata between third party technologies and the open metadata ecosystem. * [Engine Host](configuring-an-engine-host.md) - this server hosts governance engines that monitor the open metadata ecosystem and take actions as requested/required. -* [Lineage Warehouse](configuring-a-lineage-warehouse.md) - this server provides optimized access to lineage data. It has its own repository that it maintains by monitoring the open metadata ecosystem. This is how these types of servers work together. diff --git a/site/docs/guides/admin/servers/configuration-document-structure-simplified.svg b/site/docs/guides/admin/servers/configuration-document-structure-simplified.svg index 571ade0652..309ed79ae9 100644 --- a/site/docs/guides/admin/servers/configuration-document-structure-simplified.svg +++ b/site/docs/guides/admin/servers/configuration-document-structure-simplified.svg @@ -1,4 +1,4 @@ -
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Conformance Suite Services
Engine Host Services
Lineage Warehouse Services
Data Engine Proxy Services
Audit Trail
Default values used in the
configuration services that
send notifications.
Values used by
all types of server.
Configuration to control
the activation of specialized
services that are specific to
particular types of server.
Repository Services
Server Security Connection
Integration Daemon Services
View Services
Basic Server Properties
Values typically
maintained by Egeria
 \ No newline at end of file +
Local Server Id
Local Server Name
Local Server Type
Event Bus Defaults
Access Services
Conformance Suite Services
Engine Host Services
Audit Trail
Default values used in the
configuration services that
send notifications.
Values used by
all types of server.
Configuration to control
the activation of specialized
services that are specific to
particular types of server.
Repository Services
Server Security Connection
Integration Daemon Services
View Services
Basic Server Properties
Values typically
maintained by Egeria
 \ No newline at end of file diff --git a/site/docs/guides/developer/governance-action-services/overview.md b/site/docs/guides/developer/governance-action-services/overview.md index adf2136840..f3a28b7010 100644 --- a/site/docs/guides/developer/governance-action-services/overview.md +++ b/site/docs/guides/developer/governance-action-services/overview.md @@ -7,7 +7,7 @@ ## Governance action context -A governance action service is passed a context as it is started. This provides access to the [request type and associated request parameters (name-value pairs)](/concepts/governance-request-type) used to invoke the governance action service, along with its [action targets](/concepts/action-target). There is also a client to access open metadata through the Governance Engine OMAS. +A governance action service is passed a context as it is started. This provides access to the [request type and associated request parameters (name-value pairs)](/concepts/governance-request-type) used to invoke the governance action service, along with its [action targets](/concepts/action-target). There is also a client to access open metadata through the Governance Server OMAS. ![Structure of the governance context](/frameworks/gaf/governance-action-service-with-context.svg) @@ -22,7 +22,7 @@ The methods of the context are colour-coded. The methods shown in white are ava * Performing the action - * getOpenMetadataStore - returns a client to the Governance Engine OMAS that supports the retrieval of metadata elements, their classifications and relationships. + * getOpenMetadataStore - returns a client to the Governance Server OMAS that supports the retrieval of metadata elements, their classifications and relationships. * createIncidentReport - creates an [incident report](/concepts/incident-report) that can provide a focal point for collaboration to resolve a particular issue. Incident reports are often managed by an [incident management tool](/features/incident-reporting/overview). * Recording the outcome of the processing @@ -237,7 +237,7 @@ In addition, it needs to implement the standard [Open Connector Framework (OCF)] ![Structure of a governance engine definition](governance-action-engine-definitions.svg) - These definitions are created as part of a [governance engine pack](/guides/developer/open-metdata-archives/creating-governance-engine-packs) or through the [Governance Engine OMAS](/services/omas/governance-engine/overview) and are stored in the open metadata repositories. + These definitions are created as part of a [governance engine pack](/guides/developer/open-metdata-archives/creating-governance-engine-packs) or through the [Governance Server OMAS](/services/omas/governance-server/overview) and are stored in the open metadata repositories. * Running your governance action service - Governance action engines are hosted by the [Governance Action OMES](/services/omes/governance-action/overview) running on one or more [engine hosts](/concepts/engine-host). The engine services run in dedicated OMAG Server called the [*engine host*](/concepts/engine-host). You can find [instructions for configuring the engine services in the engine host](/guides/admin/servers/by-section/engine-host-services-section) in the administration guide. diff --git a/site/docs/guides/developer/java-clients/governance-engine.md b/site/docs/guides/developer/java-clients/governance-engine.md index 8e90726c91..d7fb30a2d5 100644 --- a/site/docs/guides/developer/java-clients/governance-engine.md +++ b/site/docs/guides/developer/java-clients/governance-engine.md @@ -2,11 +2,11 @@ -# Governance Engine OMAS Java client +# Governance Server OMAS Java client ## Concepts -The concepts introduced by the Governance Engine OMAS are (in alphabetical order): +The concepts introduced by the Governance Server OMAS are (in alphabetical order): * [Engine Action](/concepts/engine-action) * [Governance Action Process](/concepts/governance-action-process) diff --git a/site/docs/guides/diagnostic/examples-of-server-start-up-errors.md b/site/docs/guides/diagnostic/examples-of-server-start-up-errors.md index babf45489d..049102da62 100644 --- a/site/docs/guides/diagnostic/examples-of-server-start-up-errors.md +++ b/site/docs/guides/diagnostic/examples-of-server-start-up-errors.md @@ -49,7 +49,7 @@ At the end of the server start up, the server lists the services it has running. Below are the messages from the five servers show above: ``` -Wed May 12 15:26:13 BST 2021 cocoMDS1 Startup OMAG-ADMIN-0004 The cocoMDS1 server has successfully completed start up. The following services are running: [Open Metadata Repository Services (OMRS), Connected Asset Services, Asset Consumer OMAS, Asset Manager OMAS, Asset Owner OMAS, Community Profile OMAS, Data Manager OMAS, Governance Engine OMAS] +Wed May 12 15:26:13 BST 2021 cocoMDS1 Startup OMAG-ADMIN-0004 The cocoMDS1 server has successfully completed start up. The following services are running: [Open Metadata Repository Services (OMRS), Connected Asset Services, Asset Consumer OMAS, Asset Manager OMAS, Asset Owner OMAS, Community Profile OMAS, Data Manager OMAS, Governance Server OMAS] Wed May 12 15:26:14 BST 2021 cocoMDS4 Startup OMAG-ADMIN-0004 The cocoMDS4 server has successfully completed start up. The following services are running: [Open Metadata Repository Services (OMRS), Connected Asset Services,Asset Consumer OMAS, Community Profile OMAS, Data Science OMAS] diff --git a/site/docs/guides/planning/governance-program/overview.md b/site/docs/guides/planning/governance-program/overview.md index b2eb7e6caf..d3d2d898de 100644 --- a/site/docs/guides/planning/governance-program/overview.md +++ b/site/docs/guides/planning/governance-program/overview.md @@ -312,11 +312,9 @@ These classifications help in the review of the implementation of the governance * The [Digital Service OMAS](/services/omas/digital-service/overview) documents the business capabilities along with their digital services that are supported by the governance program. - * The [Governance Engine OMAS](/services/omas/governance-engine/overview) supports the implementation of technical controls and the choreography of their execution. + * The [Governance Server OMAS](/services/omas/governance-server/overview) supports the implementation of technical controls and the choreography of their execution. * The [Stewardship Action OMAS](/services/omas/stewardship-action/overview) supports the stewards as they manage the exceptions detected to the governance program. - - * The [Data Privacy OMAS](/services/omas/data-privacy/overview) supports the operational aspects of managing privacy as part of the organization's activities. * The [Data Manager OMAS](/services/omas/data-manager/overview) support the automated cataloging of assets and configuration of technology that is managing them. diff --git a/site/docs/guides/planning/runtime/overview.md b/site/docs/guides/planning/runtime/overview.md index 2cc7f9a7c4..b83fe743ed 100644 --- a/site/docs/guides/planning/runtime/overview.md +++ b/site/docs/guides/planning/runtime/overview.md @@ -54,7 +54,7 @@ This architecture means that you can incrementally add function to your deployme 2. Decide on a name for an [open metadata repository cohort](/services/omrs/cohort) and configure your graph metadata repository to join it. 3. If you want to have other third party metadata repositories that you want to share metadata with, configure [repository proxies](/guides/admin/servers/by-server-type/configuring-a-repository-proxy) for each including registering them to the same cohort as the metadata server. 4. If you then want to add in metadata synchronization with other types of technology beyond metadata repositories, work out which [integration daemons](/concepts/integration-daemon) you need and [configure them to connect to the metadata access server](/guides/admin/servers/by-server-type/configuring-an-integration-daemon). Make sure the appropriate [access services](/services/omas) for these integration daemons are enabled in the metadata server. -5. If you want to use the governance services then these run in an [engine host server](/concepts/engine-host) and connect to the metadata server via the [Governance Server OMAS](/services/omas/governance-server/overview) and [Governance Engine OMAS](/services/omas/governance-engine/overview). +5. If you want to use the governance services then these run in an [engine host server](/concepts/engine-host) and connect to the metadata server via the [Governance Server OMAS](/services/omas/governance-server/overview). Working through this exercise gives you an understanding of the Egeria technology that you need for your deployment and how it connects together. The [Solutions Guide](/introduction/patterns) describes different solutions that you can build with Egeria, how they work and the configuration that you will need. diff --git a/site/docs/patterns/metadata-governance/governance-engines.md b/site/docs/patterns/metadata-governance/governance-engines.md index 1d075e3a42..8271c08663 100644 --- a/site/docs/patterns/metadata-governance/governance-engines.md +++ b/site/docs/patterns/metadata-governance/governance-engines.md @@ -47,7 +47,7 @@ The diagram below summarizes how Egeria's governance automation works. Descript ![Governance Action statuses](/patterns/metadata-governance/governance-action-status.svg) ??? info "Governance action processes" - Governance action processes are defined in metadata using a set of linked [governance action process steps](/concepts/governance-action-process-step). They are choreographed in a [Metadata Access Server](/concepts/metadata-access-server) running the [Governance Engine OMAS](/services/omas/governance-engine/overview). When the process is called to run, the Governance Engine OMAS navigates to the first governance action process step in the governance action process flow. It creates a matching engine action entity. This is picked up by the engine host and executed in the governance engine just as if it was called independently. The guards are returned by the engine action as it completes. This change is detected by Governance Engine OMAS which uses the guards to navigate to the next engine action process step(s) found in the governance action process flow. A engine action is created for each of the next governance action process steps and the cycle is repeated until there are no more governance action process steps in the governance action process flow to execute. + Governance action processes are defined in metadata using a set of linked [governance action process steps](/concepts/governance-action-process-step). They are choreographed in a [Metadata Access Server](/concepts/metadata-access-server) running the [Governance Server OMAS](/services/omas/governance-server/overview). When the process is called to run, the Governance Server OMAS navigates to the first governance action process step in the governance action process flow. It creates a matching engine action entity. This is picked up by the engine host and executed in the governance engine just as if it was called independently. The guards are returned by the engine action as it completes. This change is detected by Governance Server OMAS which uses the guards to navigate to the next engine action process step(s) found in the governance action process flow. A engine action is created for each of the next governance action process steps and the cycle is repeated until there are no more governance action process steps in the governance action process flow to execute. A governance action process can be run many times with different parameters. It can be changed, simply by updating the governance action process step metadata entities in the governance action process definition. New processes can be created by creating the appropriate governance process definition. diff --git a/site/docs/patterns/metadata-governance/setting-up-the-engine-host.md b/site/docs/patterns/metadata-governance/setting-up-the-engine-host.md index 9bba18bc4a..730ef5a6a5 100644 --- a/site/docs/patterns/metadata-governance/setting-up-the-engine-host.md +++ b/site/docs/patterns/metadata-governance/setting-up-the-engine-host.md @@ -19,7 +19,7 @@ The *Governance Engine Definition* does not need to be loaded directly into the ![Engine Host Remote Configuring](/patterns/metadata-governance/engine-host-remote-configuring.svg) -**Recap:** calls to a governance engine are made by initiating an [Engine Action](/concepts/engine-action). This can be directly through an API call to the [Governance Engine OMAS](/services/omas/governance-engine/overview) running in a [Metadata Access Store](/concepts/metadata-access-store), or via a [Governance Action Process](/concepts/governance-action-process). +**Recap:** calls to a governance engine are made by initiating an [Engine Action](/concepts/engine-action). This can be directly through an API call to the [Governance Server OMAS](/services/omas/governance-server/overview) running in a [Metadata Access Store](/concepts/metadata-access-store), or via a [Governance Action Process](/concepts/governance-action-process). A Governance Action is an entity in open metadata. When it is created in the Metadata Access Store, it sends an event to all connected Engine Hosts. If it is for a [governance request type](/concepts/governance-request-type) that one of its governance engine's supports, it claims the Governance Action and passes the request to its governance engine to run. diff --git a/site/docs/release-notes/february-2023.md b/site/docs/release-notes/february-2023.md index 662f054c0d..bf358cd91b 100644 --- a/site/docs/release-notes/february-2023.md +++ b/site/docs/release-notes/february-2023.md @@ -116,7 +116,7 @@ Each month we will select one or more new pages that where added/updated in the * [Integration groups](/concepts/integration-group) * [Types for integration groups](/types/4/0464-Dynamic-Integration-Groups) -* [Description of the Governance Engine OMAS](/services/omas/governance-engine/overview) +* [Description of the Governance Server OMAS](/services/omas/governance-server/overview) ## Connecting with the project diff --git a/site/docs/services/engine-host-services.md b/site/docs/services/engine-host-services.md index 329c1d167a..01c7d2e9ea 100644 --- a/site/docs/services/engine-host-services.md +++ b/site/docs/services/engine-host-services.md @@ -10,7 +10,7 @@ hide: # Engine host services -The engine host services support the behavior of the [engine host](/concepts/engine-host) OMAG Server. They manage the start up and shutdown of the configured [Open Metadata Engine Services (OMESs)](/services/omes) as well as retrieve the configuration of their governance engines and governance services through the [Governance Engine OMAS](/services/omas/governance-engine/overview) running in a [Metadata Access Server](/concepts/metadata-access-server). +The engine host services support the behavior of the [engine host](/concepts/engine-host) OMAG Server. They manage the start up and shutdown of the configured [Open Metadata Engine Services (OMESs)](/services/omes) as well as retrieve the configuration of their governance engines and governance services through the [Governance Server OMAS](/services/omas/governance-server/overview) running in a [Metadata Access Server](/concepts/metadata-access-server). The engine host services also provide a REST API to query the status of the governance engines running in the server. @@ -22,7 +22,7 @@ The internals of the Engine Host OMAG Server are shown in Figure 1: The engine host services shown at the top of the diagram are responsible for: * Starting up each of the configured OMESs for the engine host. -* Retrieving the configuration for each governance engine and its governance services configured within the OMESs using the Governance Engine OMAS. +* Retrieving the configuration for each governance engine and its governance services configured within the OMESs using the Governance Server OMAS. * Initializing an appropriate governance engine handler (implemented by the OMES) for each retrieved governance engine configuration. * Listening for [governance action](/concepts/governance-action/overview) requests and passing each one to the appropriate governance engine handler for the governance engine that supports the [request type](/concepts/governance-request-type). * Monitor changes in the governance engine configuration and passing them on to the corresponding governance engine handler so the OMESs have the latest configuration. diff --git a/site/docs/services/governance-server-services.md b/site/docs/services/governance-server-services.md index 1de3d4c892..274b6df5c5 100644 --- a/site/docs/services/governance-server-services.md +++ b/site/docs/services/governance-server-services.md @@ -10,6 +10,5 @@ The governance server services provide the implementation of each type of govern * [Engine Host Services](/services/engine-host-services) for the [Engine Host](/concepts/engine-host). * [Integration Daemon Services](/services/integration-daemon-services) for the [Integration Daemon](/concepts/integration-daemon). -* [Lineage Warehouse Services](/services/lineage-warehouse-services) for the [Lineage Warehouse](/concepts/lineage-warehouse). --8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/services/index.md b/site/docs/services/index.md index 78c8e9fc0a..ca19906885 100644 --- a/site/docs/services/index.md +++ b/site/docs/services/index.md @@ -55,7 +55,6 @@ The governance server services provide the implementation of each type of [gover * [Engine Host Services](/services/engine-host-services) for the [Engine Host](/concepts/engine-host). * [Integration Daemon Services](/services/integration-daemon-services) for the [Integration Daemon](/concepts/integration-daemon). -* [Lineage Warehouse Services](/services/lineage-warehouse-services) for the [Lineage Warehouse](/concepts/lineage-warehouse). --8<-- "snippets/abbr.md" diff --git a/site/docs/services/omas/governance-program/overview.md b/site/docs/services/omas/governance-program/overview.md index 2931de0b5e..24f1ad4f19 100644 --- a/site/docs/services/omas/governance-program/overview.md +++ b/site/docs/services/omas/governance-program/overview.md @@ -114,7 +114,7 @@ As important aspect of the governance program is the ability to [measure its eff * The [Digital Service OMAS](/services/omas/digital-service/overview) documents the business capabilities along with their digital services that are supported by the governance program. -* The [Governance Engine OMAS](/services/omas/governance-engine/overview) supports the implementation of technical controls and the choreography of their execution. +* The [Governance Server OMAS](/services/omas/governance-server/overview) supports the implementation of technical controls and the choreography of their execution. * The [Stewardship Action OMAS](/services/omas/stewardship-action/overview) supports the stewards as they manage the exceptions detected to the governance program. diff --git a/site/docs/services/omas/governance-server/overview.md b/site/docs/services/omas/governance-server/overview.md index 4e3f98c255..9f60f29029 100644 --- a/site/docs/services/omas/governance-server/overview.md +++ b/site/docs/services/omas/governance-server/overview.md @@ -17,7 +17,7 @@ specialist [connectors](/frameworks/ocf/overview) that are defined by: The governance services run in the [Engine Host OMAG Server](/concepts/engine-host) supported by the [Open Metadata Engine Services (OMES)](/services/omes). They are triggered via [governance actions](/concepts/governane-action). -The Governance Engine OMAS has the following capabilities to support governance engines: +The Governance Server OMAS has the following capabilities to support governance engines: | Java Interface | Description | |-------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -31,7 +31,7 @@ The Governance Engine OMAS has the following capabilities to support governance | [GovernanceEngineConfiguration](https://odpi.github.io/egeria/org/odpi/openmetadata/accessservices/governanceengine/api/GovernanceEngineConfiguration.html) | Providing APIs to query the status of the governance capabilities implemented through the governance engines. | | [SpecialGovernanceActionInterface](https://odpi.github.io/egeria/org/odpi/openmetadata/accessservices/governanceengine/api/SpecialGovernanceActionInterface.html) | Providing specialized governance remediation APIs such as linking duplicates. | -Behind the scenes, Governance Engine OMAS handles the choreography of governance actions based on the process definition provided by a [governance action process](/concepts/governance-action-process). +Behind the scenes, Governance Server OMAS handles the choreography of governance actions based on the process definition provided by a [governance action process](/concepts/governance-action-process). !!! info "Further documentation" The documentation for writing governance services is located in the developer guide: @@ -44,7 +44,7 @@ Behind the scenes, Governance Engine OMAS handles the choreography of governance ## Integration Groups -An [integration group](/concepts/integration-group) is a collection of [integration connector](/concepts/integration-connector) definitions intended to run in one or more [integration daemons](/concepts/integration-daemon). The Governance Engine OMAS provides the API to maintain the integration group and integration connector definitions. The integration daemon then loads the definitions from the integration group and runs the requested integration connectors. This is an alternative method to configure the integration daemon from adding the integration connectors directly into the integration daemon's configuration document. The advantage of this approach is that new integration connectors can be added without restarting the integration daemon. +An [integration group](/concepts/integration-group) is a collection of [integration connector](/concepts/integration-connector) definitions intended to run in one or more [integration daemons](/concepts/integration-daemon). The Governance Server OMAS provides the API to maintain the integration group and integration connector definitions. The integration daemon then loads the definitions from the integration group and runs the requested integration connectors. This is an alternative method to configure the integration daemon from adding the integration connectors directly into the integration daemon's configuration document. The advantage of this approach is that new integration connectors can be added without restarting the integration daemon. ![integration group operation](integration-group-use.svg) diff --git a/site/docs/services/omas/index.md b/site/docs/services/omas/index.md index 980034fccd..1d7e1ef26e 100644 --- a/site/docs/services/omas/index.md +++ b/site/docs/services/omas/index.md @@ -15,7 +15,6 @@ The access services are as follows: | OMAS | Description | Supplies APIs and events to | |----------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Asset Consumer](/services/omas/asset-consumer/overview) | The Asset Consumer OMAS is designed for applications that are using [OCF](/frameworks/ocf/overview) connectors to access data stores, APIs and functions such as analytics. The Asset Consumer OMAS provides a factory function for the connectors, the ability to retrieve all of the metadata about the asset and the ability to add feedback on the asset. | [Asset Catalog OMVS](/services/omvs/asset-catalog/overview) | -| [Asset Lineage](/services/omas/asset-lineage/overview) | The Asset Lineage OMAS listens to relevant lineage related events on the enterprise topic level and publishes these on the Asset Lineage OutTopic, combined with relevant context information on the described entities. These events are listened to by the open lineage services governance server. | [Lineage Warehouse Services](/services/lineage-warehouse-services) | | [Asset Manager](/services/omas/asset-manager/overview) | The Asset Manager OMAS manages the exchange of metadata with third party metadata catalogs and asset managers. It is typically called by the [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) to send and receive asset information, including schemas, profiles, policies and lineage information with a third party asset manager. Typical examples of asset managers include data catalogs that are managing metadata for a collection of data assets for a data-serving solution. | [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview), [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview), [Glossary Browser OMVS](/services/omvs/glossary-browser/overview), [Glossary Manager OMVS](/services/omvs/glossary-manager/overview), [Feedback Manager OMVS](/services/omvs/feedback-manager/overview) | | [Asset Owner](/services/omas/asset-owner/overview) | The Asset Owner OMAS provides services for an asset owner to curate metadata about their asset(s) and understand how these assets are being used and governed. | [Survey Action OMES](/services/omes/survey-action/overview), [Automated Curation OMVS](/services/omvs/automated-curation/overview) | | [Community Profile](/services/omas/community-profile/overview) | The Community Profile OMAS supports the administration for a community and related user profiles. These communities are involved in reviewing and crowd-sourcing knowledge about the data assets and their use. | [Organization Integrator OMIS](/services/omis/organization-integrator/overview), [My Profile OMVS](/services/omvs/my-profile/overview) | @@ -24,7 +23,6 @@ The access services are as follows: | [Design Model](/services/omas/design-model/overview) | The Design Model OMAS provides the ability to manage information from all types of design models. These models may come from tools or be part of a packaged standard. This content is useful for governance, system integration and software development. | | | [Digital Architecture](/services/omas/digital-architecture/overview) | The Digital Architecture OMAS provides the ability to define information standards, definitions, solution blueprints and models for an organization. It is designed for architecture tools. It is able to support the definition and management of a digital service through concept to deployment. | | | [Digital Service](/services/omas/digital-service/overview) | The Digital Service OMAS provides services for a managing the lifecycle of an Egeria Digital Service. | [Collection Manager OMVS](/services/omvs/collection-manager/overview) | -| [Governance Engine](/services/omas/governance-engine/overview) | The Governance Engine OMAS provides APIs and events that retrieve and manage metadata for [governance action services](/concepts/governance-action-service). | [Governance Action OMES](/services/omes/governance-action/overview), [Context Event OMES](/services/omes/context-event/overview) | | [Governance Program](/services/omas/governance-program/overview) | The Governance Program OMAS provides the ability to maintain a governance program in the open metadata repositories. It is designed for governance and CDO tools. | [Action Author OMVS](/services/omvs/action-author/overview) | | [Governance Server](/services/omas/governance-server/overview) | The Governance Server OMAS supplies the [governance engine definitions](/concepts/governance-engine-definition) to the [engine hosts](/concepts/engine-host) and the and [integration group definitions](/concepts/integration-group) to the [integration daemons](/concepts/integration-daemon). | [Integration Daemon Services](/services/integration-daemon-services), [Engine Host Services](/services/engine-host-services) | | [IT Infrastructure](/services/omas/it-infrastructure/overview) | The IT Infrastructure OMAS provides support for the design and planning of the information infrastructure that supports the data assets. This includes the development of system blueprints that link down to the metadata about real infrastructure components. This metadata helps in the linkage between information governance metadata and IT infrastructure management (ITIL) metadata typically stored in a Configuration Management Database (CMDB). | [Infrastructure Integrator OMIS](/services/omis/infrastructure-integrator/overview) | diff --git a/site/docs/services/omas/security-manager/TagPropogation/TagPropogation.md b/site/docs/services/omas/security-manager/TagPropogation/TagPropogation.md index ea3e108353..7f82177552 100644 --- a/site/docs/services/omas/security-manager/TagPropogation/TagPropogation.md +++ b/site/docs/services/omas/security-manager/TagPropogation/TagPropogation.md @@ -2,7 +2,7 @@ # Tag Propagation ## Introduction -This document aims to explain how tag propagation will be handled within Governance Engine OMAS. +This document aims to explain how tag propagation will be handled within Security Manager OMAS. The terms tag and classification should be treated interchangeably.... diff --git a/site/docs/services/omes/governance-action/overview.md b/site/docs/services/omes/governance-action/overview.md index c6d71b7005..653bcbc101 100644 --- a/site/docs/services/omes/governance-action/overview.md +++ b/site/docs/services/omes/governance-action/overview.md @@ -20,7 +20,7 @@ The governance Action OMES also provides an API to allow a third party tool to v ## Using the Governance Action OMES -Governance action services are defined and linked to one or more [governance action engines](/concepts/governance-action-engine) using the [Governance Engine OMAS](/services/omas/governance-engine/overview). The definitions for both the governance action engines and their linked services are stored in a [governance engine definition](/concepts/governance-engine-definition). +Governance action services are defined and linked to one or more [governance action engines](/concepts/governance-action-engine) using the [Governance Server OMAS](/services/omas/governance-server/overview). The definitions for both the governance action engines and their linked services are stored in a [governance engine definition](/concepts/governance-engine-definition). When the Governance Action OMES is configured in the engine host, a list of governance action engines is supplied. This determines which governance action engines and hence governance action services that it supports. diff --git a/site/docs/services/omes/index.md b/site/docs/services/omes/index.md index 766452ff86..a047f49d94 100644 --- a/site/docs/services/omes/index.md +++ b/site/docs/services/omes/index.md @@ -13,7 +13,7 @@ The engine services run in the [engine host](/concepts/engine-host). They provi | Name of the Engine Service | Type of Governance Service supported | Partner service in the metadata | |-----------------------------------------------------------------------------|--------------------------------------------------------------------------|-----------------------------------------------------------------------| | [Survey Action OMES](/services/omes/survey-action/overview) | [Survey Action Service](/concepts/survey-action-service) | [Asset Owner OMAS](/services/omas/asset-owner/overview) | -| [Governance Action OMES](/services/omes/governance-action/overview) | [Governance Action Service](/concepts/governance-action-service) | [Governance Engine OMAS](/services/omas/governance-engine/overview) | +| [Governance Action OMES](/services/omes/governance-action/overview) | [Governance Action Service](/concepts/governance-action-service) | [Governance Server OMAS](/services/omas/governance-server/overview) | | [Context Event OMES](/services/omes/context-event/overview) | [Context Event Service](/concepts/context-event-service) | [Stewardship Action OMAS](/services/omas/stewardship-action/overview) | | [Repository Governance OMES](/services/omes/repository-governance/overview) | [Repository Governance Service](/concepts/repository-governance-service) | [Repository Governance OMAS](/services/omrs) | diff --git a/site/docs/services/omes/repository-governance/overview.md b/site/docs/services/omes/repository-governance/overview.md index b47fe919f6..44d76173b5 100644 --- a/site/docs/services/omes/repository-governance/overview.md +++ b/site/docs/services/omes/repository-governance/overview.md @@ -22,6 +22,6 @@ The definitions for both the repository governance engines and their linked serv When the Repository Governance OMES is configured in the engine host, a list of repository governance engine names is supplied. These names are used to retrieve the governance engine definitions for the archive engines in order to configure the Repository Governance OMES. -The Repository Governance OMES is responsible for initializing the repository governance engines and providing the context and runtime environment for the repository services when they are requested through the [engine action](/concepts/engine-action) processing of the [Governance Engine OMAS](/services/omas/governance-engine/overview). +The Repository Governance OMES is responsible for initializing the repository governance engines and providing the context and runtime environment for the repository services when they are requested through the [engine action](/concepts/engine-action) processing of the [Governance Server OMAS](/services/omas/governance-server/overview). --8<-- "snippets/abbr.md" diff --git a/site/docs/types/4/0435-Policy-Management-Capabilities.md b/site/docs/types/4/0435-Policy-Management-Capabilities.md index 510b437c80..5672887a07 100644 --- a/site/docs/types/4/0435-Policy-Management-Capabilities.md +++ b/site/docs/types/4/0435-Policy-Management-Capabilities.md @@ -54,7 +54,7 @@ metadata, we have also the concepts in the design of Egeria itself. In Egeria, the Policy Administration Point is [Governance Program OMAS](/services/omas/governance-program/overview). Services such as -[Governance Engine OMAS](/services/omas/governance-engine/overview) +[Security Manager OMAS](/services/omas/security-manager/overview) act as a Policy Retrieval Points to push policy information to external Policy Enforcement Points such as [Apache Ranger](http://ranger.apache.org/). diff --git a/site/docs/types/4/0465-Duplicate-Processing.md b/site/docs/types/4/0465-Duplicate-Processing.md index 9ccef88f5e..82c7088d0f 100644 --- a/site/docs/types/4/0465-Duplicate-Processing.md +++ b/site/docs/types/4/0465-Duplicate-Processing.md @@ -30,7 +30,7 @@ It may be that this simple set of survivorship rules and consolidation process i The [Stewardship Action OMAS](/services/omas/stewardship-action/overview) provides APIs for setting up peer duplicates and consolidated entities. -The Governance Action Services running in the [Governance Engines](/concepts/governance-engine) can automate the detection of duplicates and the maintenance of consolidated entities. The governance engines are supported by the [Governance Engine OMAS](/services/omas/governance-engine/overview). +The Governance Action Services running in the [Governance Engines](/concepts/governance-engine) can automate the detection of duplicates and the maintenance of consolidated entities. The governance engines are supported by the [Governance Server OMAS](/services/omas/governance-server/overview). ??? deprecated "Deprecated types" - *KnownDuplicateLink* is deprecated in favor of the two specialized relationships: PeerDuplicateLink and ConsolidatedDuplicateLink.