diff --git a/site/docs/concepts/connector.md b/site/docs/concepts/connector.md index 16dadcad26..a2835cae95 100644 --- a/site/docs/concepts/connector.md +++ b/site/docs/concepts/connector.md @@ -42,21 +42,23 @@ The table below lists the different types of connectors supported by Egeria's se -| Type of Connector | Description | Documentation | Connector Catalog link | -| :-----------------------------| :---------- | :------------ | :---------------------- | -| Integration Connector | Implements metadata exchange with third party tools. | [Building Integration Connectors](/guides/developer/integration-connectors/overview) | [Integration Connector Catalog](/connectors/#integration-connectors) | -| Open Discovery Service | Implements automated metadata discovery.| [Building Open Discovery Services](/guides/developer/open-discovery-services/overview) | [Open Discovery Service Catalog](/connectors/#open-discovery-services) | -| Governance Action Service | Implements automated governance. | [Building Governance Action Services](/guides/developer/governance-action-services/overview) | [Governance Action Services Catalog](/connectors/#governance-action-services) | -| Archive Service | Maintains open metadata archives based in the content of the open metadata repositories. | [Building Archive Services](/guides/developer/archive-services/overview) | [Archive Service Catalog](/connectors/#archive-services) | -| Configuration Document Store | Persists the [configuration document](/concepts/configuration-document/#storage) for an OMAG Server. | [Building Configuration Document Store Connectors](/guides/developer/runtime-connectors/configuration-document-store-connector) | [Configuration Document Store Catalog](/connectors/#configuration-document-store-connectors) | -| Platform Security Connector | Manages [service authorization](/features/metadata-security) for the OMAG Server Platform. | [Building Platform Metadata Security Connectors](/guides/developer/runtime-connectors/) | [Platform Metadata Security Connector Catalog](/connectors/#platform-metadata-security-connectors) | -| Server Security Connector | Manages [service and metadata instance authorization](/features/metadata-security) for an OMAG Server. | [Building Server Metadata Security Connectors](/guides/developer/runtime-connectors/server-metadata-security-connector) | [Server Metadata Security Connector Catalog](/connectors/#server-metadata-security-connectors) | -| Open Metadata Archive Store | Reads an open metadata archive from a particular type of store. | [OMRS Open Metadata Archive Store Connector](/concepts/open-metadata-archive-store-connector) | [open-metadata-archive-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/repository-services-connectors/open-metadata-archive-connectors){ target=gh } | -| Audit Log Store | Audit logging destination | [OMRS Audit Log Store Connector](/concepts/audit-log-store-connector) | [audit-log-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/repository-services-connectors/audit-log-connectors){ target=gh } | -| Cohort Registry Store | Local store of membership of an [open metadata repository cohort](/concepts/cohort-member). | [Building Cohort Registry Store Connectors](/guides/developer/runtime-connectors/cohort-registry-store-connector) | [cohort-registry-store-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/repository-services-connectors/cohort-registry-store-connectors){ target=gh } | -| Open Metadata Topic Connector | Sends and received events on a topic hosted by an external [event bus](/concepts/event-bus) such as Apache Kafka. | [Building Open Metadata Topic Connectors](/concepts/open-metadata-topic-connector) | [open-metadata- topic-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/event-bus-connectors/open-metadata-topic-connectors){ target=gh } | -| Metadata Collection (repository) Store | Interfaces with a metadata repository API for retrieving and storing metadata. | [OMRS Repository Connectors](/concepts/repository-connector)| [open-metadata-collection-store-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/repository-services-connectors/open-metadata-collection-store-connectors){ target=gh } | -| Metadata Collection (repository) Event Mapper | Maps events from a third party metadata repository to open metadata events. | [OMRS Event Mappers](/concepts/event-mapper-connector) | none | +| Type of Connector | Description | Documentation | +|:----------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------| +| Digital Resource Connector | Wraps the standard API of a technology into a connector so it can be called in a standard way by other connectors, Egeria's runtimes, and applications. | [Further Information](/concepts/digital-resource-connector) | +| Integration Connector | Implements metadata exchange with third party tools. | [Further Information](/concepts/integration-connector) | +| Survey Action Service | Implements automated metadata discovery. | [Further Information](/concepts/survey-action-service) | +| Governance Action Service | Implements automated governance. | [Further Information](/concepts/governance-action-service) | +| Event Action Service | Implements the maintenance of [context events](/conepts/context-event). | [Further Information](/concepts/event-action-service) | +| Repository Governance Service | Maintains information about the status and content of the open metadata repositories. | [Further Information](/concepts/repository-governance-service) | +| Configuration Document Store | Persists the [configuration document](/concepts/configuration-document/#storage) for an OMAG Server. | [Further Information](/concepts/configuration-document-store-connector) | +| Platform Security Connector | Manages [service authorization](/features/metadata-security) for the OMAG Server Platform. | [Further Information](/concepts/platform-metadata-security-connectors/) | +| Server Security Connector | Manages [service and metadata instance authorization](/features/metadata-security) for an OMAG Server. | [Further Information](/concepts/server-metadata-security-connector) | +| Open Metadata Archive Store | Reads an open metadata archive from a particular type of store. | [Further Information](/concepts/open-metadata-archive-store-connector) | +| Audit Log Store | Audit logging destination | [Further Information](/concepts/audit-log-store-connector) | +| Cohort Registry Store | Local store of membership of an [open metadata repository cohort](/concepts/cohort-member). | [Further Information](/concepts/cohort-registry-store-connector) | +| Open Metadata Topic Connector | Sends and received events on a topic hosted by an external [event bus](/concepts/event-bus) such as Apache Kafka. | [Further Information](/concepts/open-metadata-topic-connector) | +| Metadata Collection (repository) Store | Interfaces with a metadata repository API for retrieving and storing metadata. | [Further Information](/concepts/repository-connector) | +| Metadata Collection (repository) Event Mapper | Maps events from a third party metadata repository to open metadata events. | [Further Information](/concepts/event-mapper-connector) | diff --git a/site/docs/concepts/digital-resource-connector.md b/site/docs/concepts/digital-resource-connector.md index 9fbe5c7a30..0a8b72fd8a 100644 --- a/site/docs/concepts/digital-resource-connector.md +++ b/site/docs/concepts/digital-resource-connector.md @@ -8,7 +8,6 @@ ## !!! education "Further information" - - [Digital resource connector catalog](/connectors/#digital-resource-connectors). - [Open Connector Framework (OCF)](/frameworks/ocf/overview). ---8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/integration-connector.md b/site/docs/concepts/integration-connector.md index 291ae55a66..46df078b3e 100644 --- a/site/docs/concepts/integration-connector.md +++ b/site/docs/concepts/integration-connector.md @@ -7,7 +7,6 @@ !!! education "Further information" - - [Integration connector catalog](/connectors/#integration-connectors). - [Configuring an integration connector](/guides/admin/servers/by-section/integration-daemon-services-section). - [Writing an integration connector](/guides/developer/integration-connectors/overview). - [Open Connector Framework (OCF)](/frameworks/ocf/overview). diff --git a/site/docs/concepts/open-lineage-server.md b/site/docs/concepts/open-lineage-server.md index d743db45ec..79c5a41f77 100644 --- a/site/docs/concepts/open-lineage-server.md +++ b/site/docs/concepts/open-lineage-server.md @@ -6,12 +6,8 @@ hide: -# Lineage Warehouse +# Open lineage Server -The Lineage Warehouse is a [governance server](/concepts/governance-server)that manages a historical warehouse of lineage information. - -![Lineage Warehouse](lineage-warehouse.svg) - -Its behavior and configuration is described [in the administration guide](/guides/admin/servers/by-server-type/configuring-a-lineage-warehouse). +The Open Lineage Server was renamed to the [Lineage Warehouse](/concepts/lineage-warehouse) in V5.0 of Egeria. --8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/open-lineage-server.svg b/site/docs/concepts/open-lineage-server.svg deleted file mode 100644 index 49e4d53a23..0000000000 --- a/site/docs/concepts/open-lineage-server.svg +++ /dev/null @@ -1,3 +0,0 @@ - - -
Metadata Server /
Metadata Access Point
Metadata Server /...
Governance
Server
Governance...
Third Party
Technology
Third Party...
Another Tool
Another Tool
View Server
View Server
View Server
View Server
Lineage Warehouse
Open Lineag...
Metadata Access Server
Metadata Access ServerViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md b/site/docs/connectors/apache-atlas/apache-atlas-catalog-integration-connector.md similarity index 98% rename from site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md rename to site/docs/connectors/apache-atlas/apache-atlas-catalog-integration-connector.md index 54a8f437ab..a3adcdfd88 100644 --- a/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md +++ b/site/docs/connectors/apache-atlas/apache-atlas-catalog-integration-connector.md @@ -9,8 +9,8 @@ - Connector Category: [Integration Connector](/concepts/integration-connector) - Hosting Service: [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) - Hosting Server: [Integration Daemon](/concepts/integration-daemon) - - Source Module: [atlas-integration-connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/integration-connectors/atlas-integration-connector){ target=gh } - - Jar File Name: `atlas-integration-connector.jar` + - Source Module: [apache-atlas-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/system-connectors/atlas-integration-connector){ target=gh } + - Jar File Name: `apache-atlas-connectors.jar` ## Overview diff --git a/site/docs/connectors/apache-atlas/apache-atlas-catalog-integration-connector.svg b/site/docs/connectors/apache-atlas/apache-atlas-catalog-integration-connector.svg new file mode 100644 index 0000000000..7e72e2e1c0 --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-catalog-integration-connector.svg @@ -0,0 +1,4 @@ + + + +
Integration Daemon
Catalog Integrator OMIS
Apache Atlas
 Integration Connector
Apache Atlas
Server
Metadata Access Server
Asset Manager OMAS
Retrieve and update
open metadata
Asset Manager
OMAS OutTopic
Update
Apache Atlas metadata
Metadata changes
Read
Apache Atlas metadata
 \ No newline at end of file diff --git a/site/docs/connectors/apache-atlas/apache-atlas-connectors.drawio b/site/docs/connectors/apache-atlas/apache-atlas-connectors.drawio new file mode 100644 index 0000000000..d14ce8242c --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-connectors.drawio @@ -0,0 +1,1469 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/connectors/apache-atlas/apache-atlas-resource-connector.svg b/site/docs/connectors/apache-atlas/apache-atlas-resource-connector.svg new file mode 100644 index 0000000000..c06f4bfeee --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-resource-connector.svg @@ -0,0 +1,4 @@ + + + +
Apache Atlas
REST Connector
Apache Atlas
Read/Write
Apache Atlas
metadata
 \ No newline at end of file diff --git a/site/docs/connectors/resource/apache-atlas-rest-connector.md b/site/docs/connectors/apache-atlas/apache-atlas-rest-connector.md similarity index 76% rename from site/docs/connectors/resource/apache-atlas-rest-connector.md rename to site/docs/connectors/apache-atlas/apache-atlas-rest-connector.md index abe0938578..1ce4f92e40 100644 --- a/site/docs/connectors/resource/apache-atlas-rest-connector.md +++ b/site/docs/connectors/apache-atlas/apache-atlas-rest-connector.md @@ -5,17 +5,17 @@ ???+ info "Connector summary" - Connector Category: [Digital Resource Connector](/concepts/digital-resource-connector) - - Source Module: [apache-atlas-rest-connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/system-connectors/apache-atlas-rest-connector){ target=gh }. - - Jar File Name: `apache-atlas-rest-connector.jar` + - Source Module: [apache-atlas-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/system-connectors/apache-atlas-connectors){ target=gh }. + - Jar File Name: `apache-atlas-connectors.jar` - Supported Asset Type: [SoftwareServer](/types/0/0040-Software-Servers) + - Deployed Implementation Type: *Apache Atlas Server* Apache Atlas has a REST API that allows external callers to query and create both types and instances. This connector provides a simple Java API to this REST API. It is written without any dependencies on Apache Atlas (or its associated Hadoop components) so it happily runs in the same version of Java as the rest of Egeria. -This connector is used by other connectors from Egeria, and may also be used -by components from outside Egeria. +This connector is used by other connectors from Egeria, and may also be used by components from outside Egeria. The values from the connection used by this connector are: @@ -25,6 +25,4 @@ The values from the connection used by this connector are: * Connection.getConfigurationProperties.get("atlasServerName") for the name of the Apache Atlas server to use in messages. - - --8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/apache-atlas/apache-atlas-rest-connector.svg b/site/docs/connectors/apache-atlas/apache-atlas-rest-connector.svg new file mode 100644 index 0000000000..ac50b49361 --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-rest-connector.svg @@ -0,0 +1,4 @@ + + + +
Apache Atlas
REST Connector
Apache Atlas
Server
Read/Write
Apache Atlas
metadata
 \ No newline at end of file diff --git a/site/docs/connectors/apache-atlas/apache-atlas-schema-extraction.svg b/site/docs/connectors/apache-atlas/apache-atlas-schema-extraction.svg new file mode 100644 index 0000000000..acc79eb31e --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-schema-extraction.svg @@ -0,0 +1,4 @@ + + + +
Entity Data Field
Relationship
Graph Edge
Entity Type
Graph Vertex
Business Metadata
Type
Graph Vertex
Classification Type
Graph Vertex
Classification
Permitted Use
Graph Edge
Graph
Schema Type
SoftwareServer
Apache Atlas
Server
AssetSchemaType
AttributesForSchema
GraphEdgeLink
GraphEdgeLink
GraphEdgeLink
(end 2)
GraphEdgeLink
(end 1)
 \ No newline at end of file diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-analysis.svg b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-analysis.svg similarity index 100% rename from site/docs/connectors/survey-action/apache-atlas-survey-action-service-analysis.svg rename to site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-analysis.svg diff --git a/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-profile.svg b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-profile.svg new file mode 100644 index 0000000000..7205f1fa2e --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-profile.svg @@ -0,0 +1,4 @@ + + + +
Attached
Entity Types
Classification Type
GraphVertex
Relationship Type
GraphEdge
Entity Type
Graph Vertex
Attached ClassificationTypes
End 1 Attached
Entity Types
Data Profile
Annotation
End 1 Attached
Relationship Types
End 2 Attached
Entity Types
Data Profile
Annotation
Data Profile
Annotation
Data Profile
Annotation
Data Profile
Annotation
Attached
Entity Types
Business Metadata Type
GraphVertex
Data Profile
Annotation
End 2 Attached
Relationship Types
Attached Labels
Data Profile
Annotation
Data Profile
Annotation
Attached Business Metadata Types
Data Profile
Annotation
Attached
Entity Type Pairs
Data Profile
Annotation
 \ No newline at end of file diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-reports.svg b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-reports.svg similarity index 100% rename from site/docs/connectors/survey-action/apache-atlas-survey-action-service-reports.svg rename to site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-reports.svg diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-setup.svg b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-setup.svg similarity index 100% rename from site/docs/connectors/survey-action/apache-atlas-survey-action-service-setup.svg rename to site/docs/connectors/apache-atlas/apache-atlas-survey-action-service-setup.svg diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service.md b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service.md similarity index 66% rename from site/docs/connectors/survey-action/apache-atlas-survey-action-service.md rename to site/docs/connectors/apache-atlas/apache-atlas-survey-action-service.md index 6650ebcb5d..405769c021 100644 --- a/site/docs/connectors/survey-action/apache-atlas-survey-action-service.md +++ b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service.md @@ -7,11 +7,11 @@ ???+ info "Connector summary" - Connector Category: [Survey Action Service](/concepts/survey-action-service) - - Hosting Service: [Asset Analysis OMES](/services/omes/asset-analysis/overview) + - Hosting Service: [Survey Action OMES](/services/omes/survey-action/overview) - Hosting Server: [Engine Host](/concepts/engine-host) - - Source Module: [discovery-service-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/discovery-service-connectors){ target=gh } - - Jar File Name: `discovery-service-connectors.jar` - - Connector Provider Class Name: `org.odpi.openmetadata.adapters.connectors.discoveryservices.discoveratlas.DiscoverApacheAtlasProvider` + - Source Module: [apache-atlas-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/system-connectors/apache-atlas-connectors){ target=gh } + - Jar File Name: `apache-atlas-connectors.jar` + - Connector Provider Class Name: `org.odpi.openmetadata.adapters.connectors.appacheatlas.survey.SurveyApacheAtlasProvider` ## Overview @@ -20,22 +20,22 @@ The Apache Atlas Survey Action Service builds a [survey report](/concepts/survey-report) that describes the types defined in the Apache Atlas server and the numbers of instances that are found for these types. ----8<-- "snippets/discovery-services/discovery-service-config-summary.md" +---8<-- "snippets/survey-action-services/survey-action-service-config-summary.md" ![Figure 1](apache-atlas-survey-action-service.svg) > **Figure 1:** Operation of the Apache Atlas Survey Action Service ----8<-- "snippets/discovery-services/discovery-service-operation-summary.md" +---8<-- "snippets/survey-action-services/survey-action-service-operation-summary.md" -## Discovery service function +## Survey action service function -The Apache Atlas Discovery Service provides a summary of the contents of the Apache Atlas repository found at the time it was run. +The Apache Atlas Survey Action Service provides a summary of the contents of the Apache Atlas repository found at the time it was run. It has three analysis steps: -1. STATS - Retrieves the overall metrics from the Apache Atlas server. These are stored in a [*DataSourceMeasurementAnnotation*](/types/0660-Data-Source-Measurement) entity linked to the [*SurveyReport*](/types/6/0605-Open-Discovery-Analysis-Reports) entity generated for each run of the Apache Atlas Discovery Service. -2. SCHEMA - Retrieves the types from Apache Atlas and organizes them in a linked graph of [*DataField*](/types/6/0615-Schema-Extraction) entities. All the data fields are linked to a [*SchemaAnalysisAnnotation*](/types/6/0615-Schema-Extraction) entity which is in turn linked to the [*SurveyReport*](/types/6/0605-Open-Discovery-Analysis-Reports) entity +1. STATS - Retrieves the overall metrics from the Apache Atlas server. These are stored in a [*DataSourceMeasurementAnnotation*](/types/0660-Data-Source-Measurement) entity linked to the [*SurveyReport*](/types/6/0603-Survey-Reports) entity generated for each run of the Apache Atlas Survey Action Service. +2. SCHEMA - Retrieves the types from Apache Atlas and organizes them in a linked graph of [*Schema Attributes*](/types/5/0533-Graph-Schemas) entities. All the graph schema attributes are linked to a [*GraphSchemaType*](/types/5/0533-Graph-Schemas) entity which is in turn linked to the [*SurveyReport*](/types/6/0603-Survey-Reports) entity. 3. PROFILE - Retrieves each entity in the Apache Atlas server and adds the following counts to [*DataProfileAnnotation*](/types/6/0620-Data-Profiling) entities linked from the appropriate data field entities: * The number of instances of each entity type. @@ -44,22 +44,22 @@ It has three analysis steps: * The number of each type of label attached to each type of entity. * The number of business metadata properties of a particular type attached to each type of entity. -Each analysis step builds on the work of its predecessor. The processing requirements increase with each step, so you can choose to stop the processing after any step using the `finalAnalysisStep` property. This can be set as a configuration property in the connection object for this discovery service, or as an analysis parameter passed when the Apache Atlas Discovery Service is run. +Each analysis step builds on the work of its predecessor. The processing requirements increase with each step, so you can choose to stop the processing after any step using the `finalAnalysisStep` property. This can be set as a configuration property in the connection object for this survey action service, or as a request parameter passed when the Apache Atlas Survey Action Service is run. The default value for `finalAnalysisStep` is `PROFILE`. ## Metadata Setup -Prior to running the Apache Atlas Survey Service, an [asset](/concepts/asset) and [connection](/concepts/connection) must be created for the Apache Atlas server that is to be analysed. +Prior to running the Apache Atlas Survey Action Service, an [asset](/concepts/asset) and [connection](/concepts/connection) must be created for the Apache Atlas server that is to be analysed. -Typically, the asset for Apache Atlas is of type [*SoftwareServer*](/types/0/0040-Software-Server). The `networkAddress` in the connection's [endpoint](/concepts/endpoint) is the hostname and port of the Apache Atlas server. For example, `http://localhost:21000`. +Typically, the asset for Apache Atlas is of type [*SoftwareServer*](/types/0/0040-Software-Server) with a *deployedImplementationType* set to *Apache Atlas Server*. The `networkAddress` in the connection's [endpoint](/concepts/endpoint) is the hostname and port of the Apache Atlas server. For example, `http://localhost:21000`. ![Figure 2](apache-atlas-survey-action-service-setup.svg) > **Figure 2:** Metadata added to the open metadata repository ## Survey Reports -Each time the Apache Atlas Survey Service runs, there is a new [survey report](/concepts/discovery-analysis-report) created. +Each time the Apache Atlas Survey Action Service runs, there is a new [survey report](/concepts/survey-report) created. ![Figure 3](apache-atlas-survey-action-service-reports.svg) > **Figure 3:** Survey reports linked from Apache Atlas's asset @@ -84,23 +84,29 @@ This analysis is achieved using two REST API calls and so has minimum impact on ### Schema Analysis Annotation -The schema analysis annotation is created in the SCHEMA analysis step. It is the parent entity for a set of [data fields](#data-fields). These data fields represent the types defined in the Apache Atlas server. +The schema analysis annotation is created in the SCHEMA analysis step. It identifies the name/type of the schema created. -### Data Fields +### Apache Atlas Types as a schema -In the SCHEMA analysis step, a *DataField* is created for each Apache Atlas entity type, relationship type, business metadata type and classification type. +In the SCHEMA analysis step, the apache atlas types extracted from the Apache Atlas server are used to create a schema that describes the graph structure of the metadata found in the Apache Atlas server: -The classification type data fields are linked to the entity type data field to indicate which type of entity it can be attached to. -The relationship type data fields are each attached to two entity type data fields: one for the type of entity that can be attached at end 1 of the relationship; the other for the type of entity that can be attached at end 2. +* A [*GraphVertex*](/types/5/0533-Graph-Schemas) entity is created for each Apache Atlas entity type, business metadata type and classification type. +* A [*GraphEdge*](/types/5/0533-Graph-Schemas) entity is created for each Apache Atlas relationship type, and each permitted use of a classification type by an entity type. -![Figure 5](apache-atlas-survey-action-service-graph-links.svg) -> **Figure 5:** Linkage of data fields based on Apache Atlas type +*GraphVertex* and *GraphEdge* are types of [*SchemaAttribute*](/types/5/0505-Schema-Attributes). The graph schema attributes are connected together using [*GraphEdgeLink*](/types/5/0533-Graph-Schemas) relationships that connect each GraphEdge schema attribute to two GraphVertex schema attributes. + +* The relationship type graph edges are each attached to two entity type graph vertices: one for the type of entity that can be attached at end 1 of the relationship; the other for the type of entity that can be attached at end 2. +* The classification type permitted use graph edges are linked to each of the associated entity type graph vertices. + +All the graph vertices are linked to a [*GraphSchemaType*](/types/5/0533-Graph-Schemas) entity using the [*AttributeForSchema*](/types/5/0505-Schema-Attributes) relationship. The *GraphSchemaType* entity is linked to the asset for the Apache Atlas server using the [AssetSchemaType](/types/5/0503-Asset-Schema/) relationship. + +![Figure 5](apache-atlas-schema-extraction.svg) +> **Figure 5:** Linkage of graph schema elements based on Apache Atlas type. -The data fields are linked together using the [*DiscoveredLinkedDataField*](/types/6/0615-Schema-Extraction) relationship. ### Data Profile Annotation -This discovery service attaches multiple data profile annotations to each data field depending on their category (entity, relationship, classification or business metadata). +This survey action service attaches multiple data profile annotations to each graph schema attribute depending on their category (entity, relationship, classification or business metadata). ![Figure 6](apache-atlas-survey-action-service-profile.svg) > **Figure 6:** Details of the data profile annotations attached to each type of data field @@ -115,7 +121,7 @@ It sets up the following fields in each data profile annotation: The table summarizes the values in each of the data profile annotations depending on the category of the data field it is attached to. -| Data Field Category | Annotation Type | Explanation | Value Count | Instance count in AdditionalProperties | +| Atlas Type Category | Annotation Type | Explanation | Value Count | Instance count in AdditionalProperties | |---------------------|------------------------------------------------|-----------------------------------------------------------------------------------------------------------|--------------------------------------|-------------------------------------------| | Entity | Apache Atlas Attached Classification Types | Count of classification types attached to this type of entity. | Classification Name to Count | Entity instances for this type | | Entity | Apache Atlas End 1 Attached Relationship Types | Count of different types of relationships attached to this type of entity at End 1. | Relationship Name to Count | Entity instances for this type | diff --git a/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service.svg b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service.svg new file mode 100644 index 0000000000..70fc836a97 --- /dev/null +++ b/site/docs/connectors/apache-atlas/apache-atlas-survey-action-service.svg @@ -0,0 +1,4 @@ + + + +
Engine Host
Survey Action OMES
Survey Action Engine
Apache Atlas
 Survey Service
Apache Atlas
Server
Metadata Access Server
Stewardship Action OMAS
Read
Apache Atlas
metadata
Read asset and
connection metadata
Build Survey Report
 \ No newline at end of file diff --git a/site/docs/connectors/connector-catalog.drawio b/site/docs/connectors/connector-catalog.drawio index 0b13b60fb8..f59e4989ee 100644 --- a/site/docs/connectors/connector-catalog.drawio +++ b/site/docs/connectors/connector-catalog.drawio @@ -1,6 +1,6 @@ - + - + @@ -64,7 +64,7 @@ - + @@ -1181,7 +1181,7 @@ - + @@ -3501,88 +3501,8 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + @@ -3764,1852 +3684,532 @@ - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + - + - + - - - - + - - - - + - + - - - - - - - - - - - + + - + - - + + - + - + - - - - + - + - - + + - + - + - - + + - + - + - - + + - + - - + - + + + + - + - - + + - + - + + - - + + - - - - + + + + + + + + + + + + + + - - + + + + + + + + - + - + - - + + + + + + + + - + - - + + + - + - + - - - - + - + - - - - + - + - - - - + - + - - - - - - - - - - - - - + - - + + - - + + - + - - + + - - + + - + - + - - + + + + + + + + - - + + - + - + - - + + - - + + - + - + - + - - + + - - + + - - + + + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - + - + - - - - - + + - - + + - - + + + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - + - + - - + + - - + + - - + + + + - - + + - - + + - - + + - + - + + + + - - - - - + + - + - + - - + + + + + - - + + - - + + - - + + - - + + - - + + - - - - + + - + + + + - + - - + + - - + + - - + + - - + + - - + + - - + + - + - + - - - - - + + - - - - - - - + + - - + + - - + + - - - - + + - - + + - - - - + + - - + + - - + + - + - - - - - + - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service-analysis.svg b/site/docs/connectors/discovery/apache-atlas-discovery-service-analysis.svg deleted file mode 100644 index 0c24bfbaee..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service-analysis.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Business Metadata
Data Field
Business Metadata...
Relationship
Data Field
Relationship...
Business Metadata
Type
Data Field
Business Metadata...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Discovery Analysis
Report
Discovery Analysis...
DataSource
Measurements Annotation
DataSource...
Schema Analysis Annotation
Schema Analysis Anno...
Entity Data Field
Entity Data Field
Relationship Type
Data Field
Relationship Type...
Entity Data Field
Entity Data Field
Classification
Data Field
Classification...
Entity Data Field
Entity Data Field
Classification Type
Data Field
Classification Type...
Relationship
Data Field
Relationship...
Entity Data Field
Entity Data Field
Relationship Type
Data Field
Relationship Type...
Entity Data Field
Entity Data Field
Classification
Data Field
Classification...
Classification Type
Data Field
Classification Type...
Relationship Type
Data Field
Relationship Type...
Entity Type
Data Field
Entity Type...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
STATS
STATS
SCHEMA
SCHEMA
PROFILE
PROFILE
1
1
1
1
2
2
2
2
3
3
3
3
Analysis Steps
Analysis StepsText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service-data-field-links.svg b/site/docs/connectors/discovery/apache-atlas-discovery-service-data-field-links.svg deleted file mode 100644 index fb4c9cd83a..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service-data-field-links.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Entity Data Field
Entity Data Field
Classification Type
Data Field
Classification Type...
Relationship
Data Field
Relationship...
Entity Type
Data Field
Entity Type...
Business Metadata
Type
Data Field
Business Metadata...
end1
end1
end2
end2Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service-profile.svg b/site/docs/connectors/discovery/apache-atlas-discovery-service-profile.svg deleted file mode 100644 index 594589833d..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service-profile.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Attached
Entity Types
Attached...
Classification Type
Data Field
Classification Type...
Relationship Type
Data Field
Relationship Type...
Entity Type
Data Field
Entity Type...
Attached ClassificationTypes
Attached Classificat...
End 1 Attached
Entity Types
End 1 Attached...
Data Profile
Annotation
Data Profile...
End 1 Attached
Relationship Types
End 1 Attached...
End 2 Attached
Entity Types
End 2 Attached...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Attached
Entity Types
Attached...
Business Metadata Type
Data Field
Business Metadata Ty...
Data Profile
Annotation
Data Profile...
End 2 Attached
Relationship Types
End 2 Attached...
Attached Labels
Attached Labels
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Attached Business Metadata Types
Attached Business Me...
Data Profile
Annotation
Data Profile...
Attached
Entity Type Pairs
Attached...
Data Profile
Annotation
Data Profile...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service-reports.svg b/site/docs/connectors/discovery/apache-atlas-discovery-service-reports.svg deleted file mode 100644 index cf486f16da..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service-reports.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Discovery Analysis
Report
Discovery Analysis...
SoftwareServer
SoftwareServer
Apache Atlas
Apache Atlas
Connection
Connection
ApacheAtlas
RESTResource Connection
ApacheAtlas...
ConectorType
ConectorType
Endpoint
Endpoint
AssetConnection
AssetConnection
Discovery Analysis
Report
Discovery Analysis...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service-setup.svg b/site/docs/connectors/discovery/apache-atlas-discovery-service-setup.svg deleted file mode 100644 index 4c5a1a25c2..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service-setup.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
SoftwareServer
SoftwareServer
Apache Atlas
Apache Atlas
Connection
Connection
ApacheAtlas
RESTResource Connection
ApacheAtlas...
ConectorType
ConectorType
Endpoint
Endpoint
AssetConnection
AssetConnectionText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service.md b/site/docs/connectors/discovery/apache-atlas-discovery-service.md deleted file mode 100644 index 1eff4844ad..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service.md +++ /dev/null @@ -1,131 +0,0 @@ - - - ---8<-- "snippets/content-status/tech-preview.md" - -# Apache Atlas Open Discovery Service - -???+ info "Connector summary" - - Connector Category: [Open Discovery Service](/concepts/open-discovery-service) - - Hosting Service: [Asset Analysis OMES](/services/omes/asset-analysis/overview) - - Hosting Server: [Engine Host](/concepts/engine-host) - - Source Module: [discovery-service-connectors :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/discovery-service-connectors){ target=gh } - - Jar File Name: `discovery-service-connectors.jar` - - Connector Provider Class Name: `org.odpi.openmetadata.adapters.connectors.discoveryservices.discoveratlas.DiscoverApacheAtlasProvider` - -## Overview - ----8<-- "snippets/systems/apache-atlas-intro.md" - -The Apache Atlas Open Discovery Service builds a [discovery analysis report](/concepts/discovery-analysis-report) that describes the types defined in the Apache Atlas server and the numbers of instances that are found for these types. - - ----8<-- "snippets/discovery-services/discovery-service-config-summary.md" - - -![Figure 1](apache-atlas-discovery-service.svg) -> **Figure 1:** Operation of the Apache Atlas Open Discovery Service - ----8<-- "snippets/discovery-services/discovery-service-operation-summary.md" - -## Discovery service function - -The Apache Atlas Discovery Service provides a summary of the contents of the Apache Atlas repository found at the time it was run. - -It has three analysis steps: - -1. STATS - Retrieves the overall metrics from the Apache Atlas server. These are stored in a [*DataSourceMeasurementAnnotation*](/types/0660-Data-Source-Measurement) entity linked to the [*DiscoveryAnalysisReport*](/types/6/0605-Open-Discovery-Analysis-Reports) entity generated for each run of the Apache Atlas Discovery Service. -2. SCHEMA - Retrieves the types from Apache Atlas and organizes them in a linked graph of [*DataField*](/types/6/0615-Schema-Extraction) entities. All the data fields are linked to a [*SchemaAnalysisAnnotation*](/types/6/0615-Schema-Extraction) entity which is in turn linked to the [*DiscoveryAnalysisReport*](/types/6/0605-Open-Discovery-Analysis-Reports) entity -3. PROFILE - Retrieves each entity in the Apache Atlas server and adds the following counts to [*DataProfileAnnotation*](/types/6/0620-Data-Profiling) entities linked from the appropriate data field entities: - - * The number of instances of each entity type. - * The number of classifications of a particular type attached to each type of entity. - * The number of relationships of a particular type attached to each type of entity. - * The number of each type of label attached to each type of entity. - * The number of business metadata properties of a particular type attached to each type of entity. - -Each analysis step builds on the work of its predecessor. The processing requirements increase with each step, so you can choose to stop the processing after any step using the `finalAnalysisStep` property. This can be set as a configuration property in the connection object for this discovery service, or as an analysis parameter passed when the Apache Atlas Discovery Service is run. - -The default value for `finalAnalysisStep` is `PROFILE`. - -## Metadata Setup - -Prior to running the Apache Atlas Discovery Service, an [asset](/concepts/asset) and [connection](/concepts/connection) must be created for the Apache Atlas server that is to be analysed. - -Typically, the asset for Apache Atlas is of type [*SoftwareServer*](/types/0/0040-Software-Server). The `networkAddress` in the connection's [endpoint](/concepts/endpoint) is the hostname and port of the Apache Atlas server. For example, `http://localhost:21000`. - -![Figure 2](apache-atlas-discovery-service-setup.svg) -> **Figure 2:** Metadata added to the open metadata repository - -## Discovery Analysis Reports - -Each time the Apache Atlas Discovery Service runs, there is a new [discovery analysis report](/concepts/discovery-analysis-report) created. - -![Figure 3](apache-atlas-discovery-service-reports.svg) -> **Figure 3:** Discovery analysis reports linked from Apache Atlas's asset - -Figure 4 shows the structure of the discovery analysis report. The annotations are labelled with the analysis steps that create them. - -![Figure 4](apache-atlas-discovery-service-analysis.svg) -> **Figure 4:** Analysis stages performed by the discovery service - -### Data Source Measurements Annotation - -The data source measurements annotation is created in the STATS analysis step. It sets up the following properties in the *dataSourceProperties* map: - -* entityInstanceCount - number of active entity instances -* entityInstanceCount:*typeName* - number of active entity instance of this type -* entityWithSubtypesInstanceCount:*typeName* - number of active entity instances of this type and all subtypes. -* classificationCount - number of classifications added to entity instances. -* typeCount - number of defined types (and their versions). -* typeUnusedCount - number of types with no instances. - -This analysis is achieved using two REST API calls and so has minimum impact on the Apache Atlas Server. - -### Schema Analysis Annotation - -The schema analysis annotation is created in the SCHEMA analysis step. It is the parent entity for a set of [data fields](#data-fields). These data fields represent the types defined in the Apache Atlas server. - -### Data Fields - -In the SCHEMA analysis step, a *DataField* is created for each Apache Atlas entity type, relationship type, business metadata type and classification type. - -The classification type data fields are linked to the entity type data field to indicate which type of entity it can be attached to. -The relationship type data fields are each attached to two entity type data fields: one for the type of entity that can be attached at end 1 of the relationship; the other for the type of entity that can be attached at end 2. - -![Figure 5](apache-atlas-discovery-service-data-field-links.svg) -> **Figure 5:** Linkage of data fields based on Apache Atlas type - -The data fields are linked together using the [*DiscoveredLinkedDataField*](/types/6/0615-Schema-Extraction) relationship. - -### Data Profile Annotation - -This discovery service attaches multiple data profile annotations to each data field depending on their category (entity, relationship, classification or business metadata). - -![Figure 6](apache-atlas-discovery-service-profile.svg) -> **Figure 6:** Details of the data profile annotations attached to each type of data field - -It sets up the following fields in each data profile annotation: - -* *analysisStep* - this is always set to "PROFILE". -* *annotationType* - this identifies the type of values that the annotation contains. -* *explanation* - this provides more information about the annotation type. -* *valueCount* - this is a map of typeName to count. For example, if this annotation was counting the classifications attached to the *DataSet* entity type, then the map would include an entry for each type of classification attached to this type of entity and a count of how many times it is used. -* *additionalProperties* - contains the count of instances for the particular type that the data field represents. - -The table summarizes the values in each of the data profile annotations depending on the category of the data field it is attached to. - -| Data Field Category | Annotation Type | Explanation | Value Count | Instance count in AdditionalProperties | -|---------------------|------------------------------------------------|-----------------------------------------------------------------------------------------------------------|--------------------------------------|-------------------------------------------| -| Entity | Apache Atlas Attached Classification Types | Count of classification types attached to this type of entity. | Classification Name to Count | Entity instances for this type | -| Entity | Apache Atlas End 1 Attached Relationship Types | Count of different types of relationships attached to this type of entity at End 1. | Relationship Name to Count | Entity instances for this type | -| Entity | Apache Atlas End 2 Attached Relationship Types | Count of different types of relationships attached to this type of entity at End 2. | Relationship Name to Count | Entity instances for this type | -| Entity | Apache Atlas Attached Labels | Count of the different labels attached to this type of entity. | Label Name to Count | Entity Instances for this type | -| Entity | Apache Atlas Attached Business Metadata Types | Count of the different types of business metadata properties attached to this type of entity. | Business Metadata Type Name to Count | Entity instances for this type | -| Classification | Apache Atlas Attached Entity Types | Count of entities where this classification is attached, organized by entity type. | Entity Type Name to Count | Classification Instances for this type | -| Business Metadata | Apache Atlas Attached Entity Types | Count of entities where this type of business metadata properties are attached, organized by entity type. | Entity Type Name to Count | Business metadata instances for this type | -| Relationship | Apache Atlas Attached End 1 Entity Types | Count of entity types attached at end 1 of this type of relationship. | Entity Type Name to Count | Relationship instances for this type | -| Relationship | Apache Atlas Attached End 2 Entity Types | Count of entity types attached at end 2 of this type of relationship. | Entity Type Name to Count | Relationship instances for this type | -| Relationship | Apache Atlas Attached Entity Type Pairs | Count of entity type pairs for this type of relationship. | Entity Type Name to Count | Relationship instances for this type | - ----8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/discovery/apache-atlas-discovery-service.svg b/site/docs/connectors/discovery/apache-atlas-discovery-service.svg deleted file mode 100644 index a711fa37d0..0000000000 --- a/site/docs/connectors/discovery/apache-atlas-discovery-service.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Engine Host
Engine Host
Asset Analysis OMES
Asset Analysis OMES
Open Discovery Engine
Open Discovery Engine
Apache Atlas
 Discovery Service
Apache Atlas...
Apache Atlas
Apache Atlas
Metadata Access Server
Metadata Access Server
Discovery Engine OMAS
Discovery Engine OMAS
Read
Apache Atlas
metadata
Read...
Read asset and
connection metadata
Read asset and...
Build Discovery
Analysis Report
Build Discovery...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/index.md b/site/docs/connectors/index.md index 79236397c8..4c426db1fa 100644 --- a/site/docs/connectors/index.md +++ b/site/docs/connectors/index.md @@ -9,138 +9,95 @@ A connector is a client to a third party technology. It supports a standard API Connectors enable Egeria to operate in many environments and with many types of third party technologies, just by managing the configuration of the [OMAG servers](/concepts/omag-server). The Connector Catalog list the connector implementations supplied by the Egeria community. There are three broad categories of connectors and the connector catalog is organized accordingly: -* Connectors that support the [exchange and maintenance of metadata](#metadata-exchange-and-maintenance-connectors). This includes the integration connectors, repository connectors, discovery services and governance action services. +* Connectors that support the [exchange and maintenance of metadata](#metadata-exchange-and-maintenance-connectors) with third party technology. This includes the resource connectors, survey action connectors, integration connectors and adapter repository connectors. These connectors are organized by the type of third part technology type work with. -* Connectors that support [Egeria’s runtime](#runtime-connectors). This includes the event bus connectors, cohort registry stores, configuration stores, audit log destination connectors, open metadata archive stores, REST client connectors and the cohort member remote repository connectors +* Connectors that support the [governance of open metadata](#open-metadata-governance-connectors). This includes the event action services and governance action services. These connectors are organized by function. -* Connectors that provide access to [digital resources and their metadata](#digital-resource-connectors) that is stored in the open metadata ecosystem. +* Connectors that support the integration of [Egeria’s runtimes](#runtime-connectors) into the IT infrastructure where it is running. This includes the native repository connectors, event bus connectors, cohort registry stores, configuration stores, audit log destination connectors, open metadata archive stores, REST client connectors and the cohort member remote repository connectors. These connectors are organized by connector type. ## Metadata exchange and maintenance connectors The connectors that support the exchange and maintenance of metadata help to accelerate the rollout of your open metadata ecosystem since they can be used to automate the extraction and distribution of metadata to the third party technologies. -| Type of Connector | Description | -|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Secrets Store connectors](#secrets-store-connectors) | Manage the retrieval of secrets (passwords, certificates, ...) from secured locations at runtime. | -| [Integration connectors](#integration-connectors) | Manage the metadata exchange to a third party technology through an [integration service](/services/omis). | -| [Repository and Event Mapper connectors](#repository-and-event-mapper-connectors) | Integrate metadata repositories into the open metadata ecosystem so that they can interact with one or more [open metadata repository cohorts](/services/omrs/cohort). | -| [Open Discovery Services](#open-discovery-services) | Analyze the content of [resources](/concepts/resource) in the digital landscape and create annotations that are attached to the resource's [asset](/concepts/asset) metadata element in the open metadata repositories in the form of an [discovery analysis report](/concepts/discovery-analysis-report). | -| [Governance Action Services](#governance-action-services) | Perform monitoring of metadata changes, validation of metadata, triage of issues, assessment and/or remediation activities as required. | -| [Governance Daemon Connectors](#governance-daemon-connectors) | Contain specialist connectors for the governance servers that make active use of open metadata. | +* [Secrets Store connectors](#secrets-store-connectors) manage the retrieval of secrets (passwords, certificates, ...) from secured locations at runtime. | +* [File connectors](#files) work with different types of files +* [JDBC Database connectors](#jdbc-databases) make use of the JDBC standards to work with different types of relational databases. +* [Apache Kafka](#apache-kafka) work with the topics and/or events passing through the Apache Kafka event broker. +* [Apache Atlas](#apache-atlas) work with an Apache Atlas server. +* [Strimzi](#strimzi) work with the cloud-based Apache Kafka deployment called Strimzi. +* [Open API Specification](#open-api-specification) extract metadata about APIs through the Open API interfaces provided through the Swagger API. +* [Open Lineage Events](#open-lineage-events) works with the open lineage event standard. ### Secrets Store Connectors ----8<-- "docs/connectors/secrets/secrets-store-connector-intro.md" +* The [Environment Variables Secret Store connector](/connectors/secrets/environment-variable-secrets-store-connector) retrieves secret values from environment variables. +### Files -| Secrets Store Connectors | Description | -|-------------------------------------------------------------------------------------------|-----------------------------------------------------| -| [Environment Variables](/connectors/secrets/environment-variable-secrets-store-connector) | Retrieves secret values from environment variables. | - - -### Integration Connectors - ----8<-- "docs/connectors/integration/integration-connector-intro.md" - -#### Cataloguing Files - -The *files* integration connectors run in the [Files Integrator Open Metadata Integration Service (OMIS)](/services/omis/files-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon). - -| Files Integration Connectors | Description | -|---|---| -| [Data files monitor](/connectors/integration/data-files-monitor-integration-connector) | Maintains a `DataFile` asset for each file in the directory (or any subdirectory). When a new file is created, a new DataFile asset is created. If a file is modified, the lastModified property of the corresponding DataFile asset is updated. When a file is deleted, its corresponding DataFile asset is also deleted (or archived if it is still needed for lineage). | -| [Data folder monitor](/connectors/integration/data-folder-monitor-integration-connector) | Maintains a `DataFolder` asset for the directory. The files and directories underneath it are assumed to be elements/records in the DataFolder asset and so each time there is a change to the files and directories under the monitored directory, it results in an update to the lastModified property of the corresponding DataFolder asset. | - -#### Cataloguing Databases and their Schemas - -The *database* integration connectors run in the [Database Integrator Open Metadata Integration Service (OMIS)](/services/omis/database-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon). +Files provide storage for many types of data. They are organizes into folders (also known as directories on some operating systems). Some connectors work with any type of file. Other connectors are able to understand the content of specific types of file formats and so these connectors are organized by file type. -| Database Integration Connectors | Description | -|--------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [PostgreSQL database connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/egeria-connector-postgres){ target=gh } | Automatically maintains the open metadata instances for the databases hosted on a [PostgreSQL server :material-dock-window:](https://www.postgresql.org){ target=psql } This includes the database schemas, tables, columns, primary keys and foreign keys. | -| [JDBC integration connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/jdbc-integration-connector){ target=gh } | Automatically maintains the open metadata instances on a database server via JDBC. This includes the database schemas, tables, columns, primary keys and foreign keys. | +#### Any type of file -#### Cataloguing event topics and the structure of their events +* The [Basic File Resource Connector](/connectors/resource/basic-file-resource-connector) provides support to read and write to a file using the Java File object. -The *topic* integration connectors run in the [Topic Integrator Open Metadata Integration Service (OMIS)](/services/omis/topic-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon). +#### File Folders (Directories) -| Topic Integration Connector | Description | -|----------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Kafka Monitor topic integration connector](/connectors/integration/kafka-monitor-integration-connector) | Automatically maintains the open metadata instances for the topics hosted on an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka }. | -| [Kafka Audit topic integration connector](/connectors/integration/kafka-topics-audit-integration-connector) | Validates that topics that are active in an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka } are also catalogued in open metadata. Creates an audit log record for each topic that is not catalogued. | -| [Strimzi Monitor topic integration connector](/connectors/integration/strimzi-monitor-integration-connector) | Automatically maintains the open metadata instances for the topics hosted in [Strimzi :material-dock-window:](https://strimzi.io){ target=strimzi }. | +* The [Basic Folder Resource Connector](/connectors/resource/basic-folder-resource-connector) is for accessing the files within a folder (directory). +* The [Data Files Monitor Integration Connector](/connectors/integration/data-files-monitor-integration-connector) maintains a `DataFile` asset for each file in the directory (or any subdirectory). When a new file is created, a new DataFile asset is created. If a file is modified, the lastModified property of the corresponding DataFile asset is updated. When a file is deleted, its corresponding DataFile asset is also deleted (or archived if it is still needed for lineage). -#### Cataloguing APIs +#### Data Folders -The *API* integration connectors run in the [API Integrator Open Metadata Integration Service (OMIS)](/services/omis/api-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon). +* The [Data Folder Resource Connector](/connectors/resource/data-folder-resource-connector) is for accessing data that is stored as a number of files within a folder (directory). +* The [Data folder Monitor Integration Connector](/connectors/integration/data-folder-monitor-integration-connector) maintains a `DataFolder` asset for the directory. The files and directories underneath it are assumed to be elements/records in the DataFolder asset and so each time there is a change to the files and directories under the monitored directory, it results in an update to the lastModified property of the corresponding DataFolder asset. -| API Integration Connectors | Description | -|----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------| -| [Open API Monitor integration connector](/connectors/integration/open-api-monitor-integration-connector) | Automatically maintains the open metadata instances for the APIs extracted from the Open API Specification extracted from an application. | +#### CSV Files -#### Synchronizing with third party catalogs +* The [CSV File Resource Connector](/connectors/resource/csv-file-resource-connector) is able to retrieve data from a Comma Separated Values (CSV) file where the contents are stored in logical columns with a special character delimiter between the columns. -The catalog integration connectors run in the [Catalog Integrator Open Metadata Integration Service (OMIS)](/services/omis/catalog-integrator) hosted in the [integration daemon](/concepts/integration-daemon). +#### Open Metadata Archive Files +* The [File-based Open Metadata Archive Store Connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/repository-services-connectors/open-metadata-archive-connectors/open-metadata-archive-file-connector){ target=gh } stores an open metadata archive as a plain text JSON file. -| Catalog Integration Connectors | Description | -|----------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------| -| [Apache Atlas integration connector](/connectors/integration/apache-atlas-catalog-integration-connector) | Automatically publishes glossary terms from Egeria to Apache Atlas. | +### Relational Databases -#### Populating security enforcement engines +* The [JDBC Resource Connector](/connectors/resource/jdbc-resource-connector) is for accessing a database via the JDBC DataSource interface. +* The [JDBC Integration Connector](/connectors/integration/jdbc-integration-connector) automatically maintains the open metadata instances on a database server via JDBC. This includes the database schemas, tables, columns, primary keys and foreign keys. -The security integration connectors run in the [Security Integrator Open Metadata Integration Service (OMIS)](/services/omis/security-integrator) hosted in the [integration daemon](/concepts/integration-daemon). +### Apache Kafka -#### Capturing and publishing Lineage +* The [Kafka Open Metadata Topic Connector](/connectors/resource-connectors/kafka-open-metadata-topic-connector) implements a [resource connector](/concepts/digital-resource-connector) for a topic that exchanges Java Objects as JSON payloads across an [Apache Kafka](https://kafka.apache.org/) event bus. It is configured in the Egeria [OMAG Servers](/concepts/omag-server) through the [Event Bus Configuration](/concepts/event-bus). This the connector that is used by default in the Egeria runtimes to exchange events (notifications between the [OMAG Servers](/concepts/omag-server). +* The [Kafka Monitor Topic integration connector](/connectors/integration/kafka-monitor-integration-connector) automatically maintains the open metadata instances for the topics hosted on an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka }. +* The [Kafka Audit Topic integration connector](/connectors/integration/kafka-topics-audit-integration-connector) validates that topics that are active in an [Apache Kafka server :material-dock-window:](https://kafka.apache.org/){ target=kafka } are also catalogued in open metadata. Creates an audit log record for each topic that is not catalogued. +* The [Sample Lineage Integration Connector](/connectors/integration/sample-lineage-event-receiver-integration-connector) is a sample connector to listen for lineage events (not in a [open lineage format](#open-lineage-events)) and catalog the associated assets, processes, schema and lineage. | -The lineage integration connectors run in the [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) hosted in the [integration daemon](/concepts/integration-daemon). They support [Lineage Management](/features/lineage-management/overview). +### Apache Atlas +---8<-- "snippets/systems/apache-atlas-intro.md" -| Lineage Integration Connectors | Description | -|--------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Open Lineage Event Receiver integration connector](/connectors/integration/open-lineage-event-receiver-integration-connector) | Connector to receive open lineage events from an event topic and publish them to lineage integration connectors with listeners registered in the same instance of the Lineage Integrator OMIS. | -| [Governance Action to Open Lineage integration connector](/connectors/integration/governance-action-open-lineage-integration-connector) | Connector to listen for governance actions executing in the open metadata ecosystem, generate open lineage events for them and publish them to the integration connectors running in the same instance of Lineage Integrator OMIS that are listening for OpenLineage events. | -| [API-based Open Lineage Log Store integration connector](/connectors/integration/api-based-open-lineage-log-store-integration-connector) | Connector that calls an OpenLineage compliant API to store the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS. | -| [File-based Open Lineage Log Store integration connector](/connectors/integration/file-based-open-lineage-log-store-integration-connector) | Connector that stores the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS. Each OpenLineage event is stored in its own file in JSON format. These files are organized according to the namespace and job name in the event. | -| [Open Lineage Cataloguer integration connector](/connectors/integration/open-lineage-cataloguer-integration-connector) | Connector to register an OpenLineage listener with the Lineage Integrator OMIS and to catalog any processes that are not already known to the open metadata ecosystem. | -| [Sample Lineage integration connector](/connectors/integration/sample-lineage-event-receiver-integration-connector) | Sample Connector to listen for lineage events (not in a open lineage format) and catalog the associated assets, processes, schema and lineage. | +The Apache Atlas connectors provide a suite of function that integrates an Apache Atlas server into the open metadata ecosystem. +* [Apache Atlas REST Connector](/connectors/apache-atlas/apache-atlas-rest-connector) is a [digital resource connector](/concepts/digital-resource-connector) that acts as a Java client to the Apache Atlas Server REST API. It is used by the other Apache Atlas connectors. +* [Apache Atlas Survey Action Service](/connectors/apache-atlas/apache-atlas-survey-action-service) reviews the types and instances stored in an Apache Atlas Server and creates a [survey report](/concepts/survey-report). This connector helps to provide insight into the content of the Apache Atlas server to determine if it contains valuable metadata that should be integrated into the open metadata ecosystem. This connector can also be configured to create a graph schema for the server that describes is supported types and how they link together. +* [Apache Atlas Integration Connector](/connectors/apache-atlas/apache-atlas-catalog-integration-connector) automatically catalogues the content of an Apache Atlas Server into the open metadata ecosystem. It may also be configured to push selected open metadata into the Apache Atlas Server, such as glossary terms, tags and classifications. -??? education "Further information relating to integration connectors" - - - [Configuring an integration connector](/guides/admin/servers/by-section/integration-daemon-services-section/) to understand how to set up an integration connector. - - [Writing an integration connector](/guides/developer/integration-connectors/overview) for more information on writing new integration connectors. +### Strimzi +* [Strimzi Monitor topic integration connector](/connectors/integration/strimzi-monitor-integration-connector) automatically maintains the open metadata instances for the topics hosted in [Strimzi :material-dock-window:](https://strimzi.io){ target=strimzi }. -### Repository and Event Mapper Connectors +### Open API Specification ----8<-- "docs/connectors/repository/repository-connector-intro.md" +The [Open API Monitor integration connector](/connectors/integration/open-api-monitor-integration-connector) automatically maintains the open metadata instances for the APIs extracted from the Open API Specification extracted from an application. -The table below lists the repository connectors supporting the native open metadata repositories. +### Open Lineage Events -| Native Repository Connector | Description | -|-------------------------------------------------------------------------------------|---| -| [JanusGraph OMRS Repository Connector](/connectors/repository/janus-graph/overview) | provides a native repository for a metadata server using [JanusGraph :material-dock-window:](https://janusgraph.org){ target=janus } as the backend. | -| [XTDB OMRS Repository Connector](/connectors/repository/xtdb) | provides a native repository for a metadata server that supports historical queries, using [XTDB :material-dock-window:](https://xtdb.com){ target=xtdb } as the persistent store. | -| [In-memory OMRS Repository Connector](/connectors/repository/in-memory/overview) | provides a simple native repository implementation that "stores" metadata in HashMaps within the JVM; it is used for testing, or for environments where metadata maintained in other repositories needs to be cached locally for performance/scalability reasons. | -| [Read-only OMRS Repository Connector](/connectors/repository/read-only/overview) | provides a native repository implementation that does not support the interfaces for create, update, delete; however, it does support the search interfaces and is able to cache metadata -- this means it can be loaded with open metadata archives to provide standard metadata definitions. | +The open lineage connectors work with the [Open Lineage standard](/features/lineage-management/overview/#the-openlineage-standard) -The table below lists the repository connectors that act as an adapter for third party metadata repositories. - -| Adapter Repository Connectors | Description | -|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Apache Atlas OMRS Repository Connector :material-github:](https://github.com/odpi/egeria-connector-hadoop-ecosystem){ target=gh } | implements read-only connectivity to the [Apache Atlas :material-dock-window:](https://atlas.apache.org){ target=atlas } metadata repository | -| [IBM Information Governance Catalog (IGC) OMRS Repository Connector :material-github:](https://github.com/odpi/egeria-connector-ibm-information-server){ target=gh } | implements read-only connectivity to the metadata repository within the [IBM InfoSphere Information Server :material-dock-window:](https://www.ibm.com/analytics/information-server){ target=ibm } suite | -| [SAS Viya OMRS Repository Connector :material-github:](https://github.com/odpi/egeria-connector-sas-viya){ target=gh } | implements metadata exchange to the metadata repository within the [SAS Viya Platform :material-dock-window:](https://support.sas.com/en/software/sas-viya.html){ target=sas } | -| [Sample Repository proxy (adapter) using polling to access files :material-github:](https://github.com/odpi/egeria-connector-repository-file-sample){ target=gh } | implements metadata exchange to a file system using a polling pattern and an embedded OMRS repository. | -| [HMS Repository proxy (adapter) using polling to access HMS Tables :material-github:](https://github.com/odpi/egeria-connector-hivemetastore){ target=gh } | implements metadata exchange to a Hive metastore using a polling pattern and an embedded OMRS repository. | - -??? education "Further information relating to Repository and Event Mapper connectors" - - - [Configuring a native repository connector](/guides/admin/servers/by-server-type/configuring-a-metadata-access-store/#configure-the-native-repository-connector) to understand how to set up a repository connector in a [Metadata Access Store](/concepts/metadata-access-store). - - [Configuring an adapter repository connector](/guides/admin/servers/by-server-type/configuring-a-repository-proxy/#configure-the-connectors-to-the-third-party-metadata-repository) to understand how to set up a repository connector in a [Repository Proxy](/concepts/repository-proxy). - - [Writing repository and event mapper connectors](/guides/developer/repository-connectors/overview) for more information on writing new repository and event mapper connectors. +* [Open Lineage Event Receiver integration connector](/connectors/integration/open-lineage-event-receiver-integration-connector) receives open lineage events from an event topic and publish them to lineage integration connectors with listeners registered in the same instance of the Lineage Integrator OMIS. +* [Governance Action to Open Lineage integration connector](/connectors/integration/governance-action-open-lineage-integration-connector) listens for governance actions executing in the open metadata ecosystem, generate open lineage events for them and publish them to the integration connectors running in the same instance of Lineage Integrator OMIS that are listening for OpenLineage events. +* [API-based Open Lineage Log Store integration connector](/connectors/integration/api-based-open-lineage-log-store-integration-connector) calls an OpenLineage compliant API to store the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS. +* [File-based Open Lineage Log Store integration connector](/connectors/integration/file-based-open-lineage-log-store-integration-connector) stores the open lineage events that are passed to it through the OpenLineage listener that is registered with the Lineage Integrator OMIS. Each OpenLineage event is stored in its own file in JSON format. These files are organized according to the namespace and job name in the event. +* [Open Lineage Cataloguer integration connector](/connectors/integration/open-lineage-cataloguer-integration-connector) registers an Open Lineage listener with the Lineage Integrator OMIS and to catalog any processes that are not already known to the open metadata ecosystem. ### Open Discovery Services @@ -160,6 +117,8 @@ The table below lists the repository connectors that act as an adapter for third - [Setting up a governance engine content pack](/guides/developer/open-metadata-archive/creating-governance-engine-content-packs) to create an [open discovery engine](/concepts/open-discovery-engine) definition to load into a [Metadata Access Store](/concepts/metadata-access-store). - [Writing an open discovery service](/guides/developer/open-discovery-services/overview) for information on writing new open discovery services. +## Open Metadata Governance Connectors + ### Governance Action Services ---8<-- "docs/connectors/governance-action/governance-action-service-intro.md" @@ -189,16 +148,6 @@ There are currently no event action services supplied by Egeria. - [Setting up a governance engine content pack](/guides/developer/open-metadata-archive/creating-governance-engine-content-packs) to create an [event action engine](/concepts/event-action-engine) definition to load into a [Metadata Access Store](/concepts/metadata-access-store). - [Writing an event action service](/guides/developer/event-action-services/overview) to understand how to write an event-action service. - -### Governance Daemon Connectors - ----8<-- "docs/connectors/governance-daemon/governance-daemon-intro.md" - -| Connector | Description | -|------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Lineage Warehouse Janus Connector](/connectors/governance-daemon/lineage-warehouse-janus-connector) | The Lineage Warehouse connectors provide plugins to the [Lineage Warehouse](/concepts/lineage-warehouse) that allow the Lineage Warehouse Services to connect with databases. | - - ### Repository Governance Services ---8<-- "docs/connectors/repository-governance/repository-governance-service-intro.md" @@ -218,6 +167,7 @@ There are currently no repository governance services supplied by Egeria. | Type | Description | |-------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Repository and Event Mapper connectors](#repository-and-event-mapper-connectors) | Integrate metadata repositories into the open metadata ecosystem so that they can interact with one or more [open metadata repository cohorts](/services/omrs/cohort). | | [Platform Metadata Security Connectors](#platform-metadata-security-connectors) | manage authorization requests for the OMAG Server Platform's services. | | [Server Metadata Security Connectors](#server-metadata-security-connectors) | manage authorization requests for the OMAG Server's services. | | [Configuration Document Store Connectors](#configuration-document-store-connectors) | manage the persistence and retrieval of [configuration documents](/concepts/configuration-document). | @@ -229,6 +179,35 @@ There are currently no repository governance services supplied by Egeria. | [Open Metadata Topic Connectors](#open-metadata-topic-connectors) | send and receive events. | +### Repository and Event Mapper Connectors + +---8<-- "docs/connectors/repository/repository-connector-intro.md" + +The table below lists the repository connectors supporting the native open metadata repositories. + +| Native Repository Connector | Description | +|-------------------------------------------------------------------------------------|---| +| [JanusGraph OMRS Repository Connector](/connectors/repository/janus-graph/overview) | provides a native repository for a metadata server using [JanusGraph :material-dock-window:](https://janusgraph.org){ target=janus } as the backend. | +| [XTDB OMRS Repository Connector](/connectors/repository/xtdb) | provides a native repository for a metadata server that supports historical queries, using [XTDB :material-dock-window:](https://xtdb.com){ target=xtdb } as the persistent store. | +| [In-memory OMRS Repository Connector](/connectors/repository/in-memory/overview) | provides a simple native repository implementation that "stores" metadata in HashMaps within the JVM; it is used for testing, or for environments where metadata maintained in other repositories needs to be cached locally for performance/scalability reasons. | +| [Read-only OMRS Repository Connector](/connectors/repository/read-only/overview) | provides a native repository implementation that does not support the interfaces for create, update, delete; however, it does support the search interfaces and is able to cache metadata -- this means it can be loaded with open metadata archives to provide standard metadata definitions. | + +The table below lists the repository connectors that act as an adapter for third party metadata repositories. + +| Adapter Repository Connectors | Description | +|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [IBM Information Governance Catalog (IGC) OMRS Repository Connector :material-github:](https://github.com/odpi/egeria-connector-ibm-information-server){ target=gh } | implements read-only connectivity to the metadata repository within the [IBM InfoSphere Information Server :material-dock-window:](https://www.ibm.com/analytics/information-server){ target=ibm } suite | +| [SAS Viya OMRS Repository Connector :material-github:](https://github.com/odpi/egeria-connector-sas-viya){ target=gh } | implements metadata exchange to the metadata repository within the [SAS Viya Platform :material-dock-window:](https://support.sas.com/en/software/sas-viya.html){ target=sas } | +| [Sample Repository proxy (adapter) using polling to access files :material-github:](https://github.com/odpi/egeria-connector-repository-file-sample){ target=gh } | implements metadata exchange to a file system using a polling pattern and an embedded OMRS repository. | +| [HMS Repository proxy (adapter) using polling to access HMS Tables :material-github:](https://github.com/odpi/egeria-connector-hivemetastore){ target=gh } | implements metadata exchange to a Hive metastore using a polling pattern and an embedded OMRS repository. | + +??? education "Further information relating to Repository and Event Mapper connectors" + + - [Configuring a native repository connector](/guides/admin/servers/by-server-type/configuring-a-metadata-access-store/#configure-the-native-repository-connector) to understand how to set up a repository connector in a [Metadata Access Store](/concepts/metadata-access-store). + - [Configuring an adapter repository connector](/guides/admin/servers/by-server-type/configuring-a-repository-proxy/#configure-the-connectors-to-the-third-party-metadata-repository) to understand how to set up a repository connector in a [Repository Proxy](/concepts/repository-proxy). + - [Writing repository and event mapper connectors](/guides/developer/repository-connectors/overview) for more information on writing new repository and event mapper connectors. + + ### Platform Metadata Security Connectors ---8<-- "docs/connectors/runtime/platform-metadata-security-connector-intro.md" @@ -243,7 +222,6 @@ There is one implementation of the platform metadata security connector provided - [Metadata Security](/features/metadata-security/overview) to understand the platform metadata security connector in the context of all of the security features. - [Writing a Platform Metadata Security Connector](/guides/developer/runtime-connectors/platform-metadata-security-connector). - ### Server Metadata Security Connectors ---8<-- "docs/connectors/runtime/server-metadata-security-connector-intro.md" @@ -258,7 +236,6 @@ There is one implementation of the server metadata security connector provided b - [Metadata Security](/features/metadata-security/overview) to understand the server metadata security connector in the context of all of the security features. - [Writing a Server Metadata Security Connector](/guides/developer/runtime-connectors/server-metadata-security-connector). - ### Configuration Document Store Connectors ---8<-- "docs/connectors/runtime/configuration-document-store-connector-intro.md" @@ -352,8 +329,6 @@ This is embedded in Egeria's Java clients. See - Egeria's [Platform API clients](/guides/developer/#working-with-the-platform-apis). - Egeria's [OMAS clients](/guides/developer/#working-with-the-open-metadata-and-governance-apis). - - ### Cohort Member Client Connectors ---8<-- "docs/connectors/runtime/cohort-member-client-connector-intro.md" @@ -366,34 +341,15 @@ Egeria's [Open Metadata Repository Services (OMRS)](/services/omrs) provides a d The connection for this connector is configured in the `LocalRepositoryRemoteConnection` property of the cohort member's [Local Repository Configuration](/user/guides/admin/servers/by-section/#configuring-the-local-repository-store). -## Digital resource connectors - ----8<-- "docs/connectors/resource/digital-resource-connector-intro.md" - -### Files - -* The [basic file connector](/connectors/resource/basic-file-resource-connector) provides support to read and write to a file using the Java File object. - -* The [basic folder connector](/connectors/resource/basic-folder-resource-connector) is for accessing the files within a folder (directory). - -* The [CSV file connector](/connectors/resource/csv-file-resource-connector) is able to retrieve data from a Comma Separated Values (CSV) file where the contents are stored in logical columns with a special character delimiter between the columns. - -* The [data folder connector](/connectors/resource/data-folder-resource-connector) is for accessing data that is stored as a number of files within a folder (directory). - -### Databases - -* The [jdbc resource connector](/connectors/resource/jdbc-resource-connector) is for accessing a database. - -### Open Metadata Topic Connectors - ----8<-- "docs/connectors/resource/open-metadata-topic-connector-intro.md" +### Governance Daemon Connectors -Egeria provides a single implementation of an open metadata connector for Apache Kafka that it uses by default. +---8<-- "docs/connectors/governance-daemon/governance-daemon-intro.md" -* The [Kafka Open Metadata Topic Connector](/connectors/resource-connectors/kafka-open-metadata-topic-connector) implements an [Apache Kafka](https://kafka.apache.org/) connector for a topic that exchanges Java Objects as JSON payloads. +| Connector | Description | +|------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Lineage Warehouse Janus Connector](/connectors/governance-daemon/lineage-warehouse-janus-connector) | The Lineage Warehouse connectors provide plugins to the [Lineage Warehouse](/concepts/lineage-warehouse) that allow the Lineage Warehouse Services to connect with databases. | -It is configured in the Egeria [OMAG Servers](/concepts/omag-server) through the [Event Bus Configuration](/concepts/event-bus). diff --git a/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.svg b/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.svg deleted file mode 100644 index b861544494..0000000000 --- a/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Integration Daemon
Integration Daemon
Catalog Integrator OMIS
Catalog Integrator OMIS
Apache Atlas
 Integration Connector
Apache Atlas...
Apache Atlas
Apache Atlas
Metadata Access Server
Metadata Access Server
Asset Manager OMAS
Asset Manager OMAS
Retrieve and update
open metadata
Retrieve and update...
Asset Manager
OMAS OutTopic
Asset Manager...
Update
Apache Atlas metadata
Update...
Metadata changes
Metadata changes
Read
Apache Atlas metadata
Read...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-graph-links.svg b/site/docs/connectors/survey-action/apache-atlas-survey-action-service-graph-links.svg deleted file mode 100644 index fb4c9cd83a..0000000000 --- a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-graph-links.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Entity Data Field
Entity Data Field
Classification Type
Data Field
Classification Type...
Relationship
Data Field
Relationship...
Entity Type
Data Field
Entity Type...
Business Metadata
Type
Data Field
Business Metadata...
end1
end1
end2
end2Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-profile.svg b/site/docs/connectors/survey-action/apache-atlas-survey-action-service-profile.svg deleted file mode 100644 index 594589833d..0000000000 --- a/site/docs/connectors/survey-action/apache-atlas-survey-action-service-profile.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Attached
Entity Types
Attached...
Classification Type
Data Field
Classification Type...
Relationship Type
Data Field
Relationship Type...
Entity Type
Data Field
Entity Type...
Attached ClassificationTypes
Attached Classificat...
End 1 Attached
Entity Types
End 1 Attached...
Data Profile
Annotation
Data Profile...
End 1 Attached
Relationship Types
End 1 Attached...
End 2 Attached
Entity Types
End 2 Attached...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Attached
Entity Types
Attached...
Business Metadata Type
Data Field
Business Metadata Ty...
Data Profile
Annotation
Data Profile...
End 2 Attached
Relationship Types
End 2 Attached...
Attached Labels
Attached Labels
Data Profile
Annotation
Data Profile...
Data Profile
Annotation
Data Profile...
Attached Business Metadata Types
Attached Business Me...
Data Profile
Annotation
Data Profile...
Attached
Entity Type Pairs
Attached...
Data Profile
Annotation
Data Profile...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/survey-action/apache-atlas-survey-action-service.svg b/site/docs/connectors/survey-action/apache-atlas-survey-action-service.svg deleted file mode 100644 index ec70b9e53f..0000000000 --- a/site/docs/connectors/survey-action/apache-atlas-survey-action-service.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
Engine Host
Engine Host
Asset Analysis OMES
Asset Analysis OMES
Survey Action Engine
Survey Action Engine
Apache Atlas
 Survey Service
Apache Atlas...
Apache Atlas
Apache Atlas
Metadata Access Server
Metadata Access Server
Stewardship Actions OMAS
Stewardship Actions OMAS
Read
Apache Atlas
metadata
Read...
Read asset and
connection metadata
Read asset and...
Build Survey
Report
Build Survey...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/features/metadata-security/accessing-config-docs.svg b/site/docs/features/metadata-security/accessing-config-docs.svg new file mode 100644 index 0000000000..e00e992752 --- /dev/null +++ b/site/docs/features/metadata-security/accessing-config-docs.svg @@ -0,0 +1,4 @@ + + + +
Platform
Metadata Security
Connector
Server 1
Configuration
Document

(no server security connection)
Platform
Metadata Security
Connector
Server 1
Configuration
Document
Server Security
Connection
Server 2
Security
Connector
1
2
getStoredConfig
getStoredConfig
 \ No newline at end of file diff --git a/site/docs/features/metadata-security/layers-of-security-checks.svg b/site/docs/features/metadata-security/layers-of-security-checks.svg index 301631a313..a745e41f46 100644 --- a/site/docs/features/metadata-security/layers-of-security-checks.svg +++ b/site/docs/features/metadata-security/layers-of-security-checks.svg @@ -1,4 +1,4 @@ - + -
Server
Server
Service
Service
Operation
Operation
Asset
Asset
Connection
Connection
Repository
Repository
Events
Events
Metadata
Metadata
Glossary
GlossaryText is not SVG - cannot display \ No newline at end of file +
Server
Service
Operation
Asset
Connection
Repository
Events
Metadata
Glossary
 \ No newline at end of file diff --git a/site/docs/features/metadata-security/metadata-security.drawio b/site/docs/features/metadata-security/metadata-security.drawio index f77f418c48..3955db29d4 100644 --- a/site/docs/features/metadata-security/metadata-security.drawio +++ b/site/docs/features/metadata-security/metadata-security.drawio @@ -1 +1,2282 @@ -7Vpde6I4FP41Xk4fSEKAS1s77sX0mU57sTuXEYLmKRoGYtX99RsgfCa2VlF3u2MvmpzEA3nfl3OSgyN4t9xOU5IsHnhI4xGwwu0ITkYA2DYC8l9u2ZUW11GGecpCNakxPLO/qTJayrpmIc06EwXnsWBJ1xjw1YoGomMjaco33WkRj7tXTcicaobngMS69U8WikVp9YDb2P+gbL6ormxjvxyZkeBlnvL1Sl1vBGBUfMrhJal8qYVmCxLyTcsE70fwLuVclK3l9o7GObYVbOX3vu4Zre87pStxyBcULa8kXqulf38YT6XlmaavNJWNx5iIiKdLdb9iV2FUrJLmfqwRvN0smKDPCQny0Y1UhbQtxDKWPVs2MznCVvMnhZmTfyVicXzHY54W/iC1Q4e6+VyR8hfaGvGxCwmWI/rq1ILlrQq6bZnUaqeUL6lId3JKNVohr5QJkOpvGp4dV9kWLY49ZSNKWvPadQOvbCiEzWhDDe0GXRzLC93OJOR4nreeabBOmdjpI3el6HmqD51G0gDwOqgHr6PDW1PQhhcPAC/S4K1VbJ+GDInZfCXbMY1Eo+ZvRW8CfF3MIaFeFJjEjAOPzqKBxOx00bahjnYt8Dba/gBoO3vRPkrKQ9ATSCjlDQyCLehji5zLKRnvVzL4lEpG8IpKdj+3kp0+tpdUMvw2BtFkM8c/tg/h00/6skimXwyJsEJ8QG3nCDG5pxsrs+DJCfLW8Dawsj9Qu+DG6ZCADAJHJoEj52wsmPMlk+henIbIyf+MNBSfYWiQ0bNPAzbQYJtoAGejQU+k3xOaEsH46gpERBEIjM9DiGfYGYoIqBHh60RAz0AEHCLim4nQs+44y6gYOKKbaRgCVM/vgepgQ6S3nEtGej23qkw5mLqvASyGVwfW04DV4MwLEcne9avaCZlV062P4gJBHxfX03HxDMH0bKj476Mi3bAkowcILEvK2lLEtrkodcX1YqdVfA7V3Bu06oi38MQ3yKCzyjg4pNU+5dNBWu2GsabYSyNsf26EIbg6wsCAcHmCC9lrB2n8a53XXm8lCOKLwm4sZxR7qHq0qbeVXuQxelXZnmjCMyZPgvUpUd5ze0LLXFx9iNJd/1TjUC9EuZ2vhInsg7aG2hbQAzM41F4c+rC/WwGGM7/t60KBZ8uqUD+YarxkC5LkzSim23H+rkHiQVehak6CmGQZC96kR70SkOoj2aKg2C6cyCX8pfguOj/zzg1wasRpqL23eBfvdmHbsJ+ubCmN5ZHjteveBLC6wiNn8sLNCdfyuzskC3ZdZHydBlR9q6FJc4SqY2ntqD4qVK4ESedUaK4KxuuFnyAC/Vz8WwTHiQD7fe6OloHu6twy0M/l15eB56PKMNm2hye7du+RpkyuP8/3E+sDwgFWCepbsLj/JoV5aCB9ue6F1aUXG+5fJRnZoPuAA96pnu3U7OBDEjy6bH7XyxEPVJCQCHIa7u8WI5YsDHPPex9/jaGPbczfeb3SU7tjei1r4xtDBQPCs5FxQAnjfAF2y0QdX2W7DK8QuarfRNe8s2t1jo+tZWx6NyhcK7Ziu//AYoCOzd+u3dsK9AP1uePrAZWg82dv+8ay3W4Gx/B/ncBdu5d3He9IgQFNrRBdVGJIr4xNY55lRMXG/0Rl2+tXcG1DZrhoZRsdUA77ffw6ZHPsIjjU8cvg6uinS3abHziW05tfkcL7fwA=7V1tc6M4Ev41rrr7kClASMDHxNncVd3Nzdxm63bvvmGQbWow8gKOM/PrT+LN6IWNMxHyjhVPVQYECHi61epudTcLsNw9/62M99uPJMX5wnPS5wW4X3ie67oO/Y+1fG1bgihqGzZllnYnnRoes2+4a+yu2xyyFFfciTUheZ3t+caEFAVOaq4tLkty5E9bk5y/6z7eYKnhMYlzufXXLK23bWvoBaf2v+Nss+3v7KLu/VZx8mVTkkPR3W/hgXXzaw/v4r6v7kWrbZyS46gJ/LQAy5KQut3aPS9xzrDtYWuve5g4Ojx3iYv6rAsc0F7yFOcH3D9z82T11x4NegkFnu7cHbdZjR/3ccKOHCntadu23uV0z6WbcbVvqbHOnjG9w906y/MlyUnZdNRAEYasnRT1qN1pfrSd0K6zmjFNwHbjPNsUdKdsob6r6pJ8waMLAQIRYDfqXgKXNX6ehMIdAKaMi8kO1+VXekp3ged3NPnaQ+N2fRxPLACC7qTtiPxDY9yx3Wbo/AQ93ejQn6KEL1EiIQlZki0p6/lo8tD8Xk+ThMKKS03Qo0CAHkQy9K6vgL5vfAv0ngT8vw9xGRd1VmAJ+GZwMyAbdF8AX+R/yP6pQJQ4GzU/eoQBm1HJdNtdUpO9HtBDgd17kTSC3I0U3D4I97dALkudZZ4V7D1p6y9lFueVVuRTiMPUPwv50FuBeZGnY+uC0MOXBT6dlfZsc1+SBFfVy2ifpr1Phzpnw0bN/+u1lyRnUSFFKwSRHrx91+Pxht4HKCHuqQDXgHdgH9699O7xRibxDi3EW+DvyCTekX14+wJ/Bybx7vuwCHAQQkGgBCYBd60DXJoxA6OAywr51QPu8BzuOUYBP8MJcG2AewKHh0YBl239awfchb4gw2WjZza47bN53NAVJLhBuJF9cAc8d3uOQbjtszDdSODu0CDc9hmYrif4ZwODU6V99qU0VYbm4Pbssy5dJMAdGYTbPtsS+IIxb9Qb2+NrE+CBYOogIMPtzgS3fZYlQAJ/G/XGevZZliAU+NuoO9bC9TTQq8AXccd69lmX0pRp1B1r4QomADyHm3XHWriECaDA4UbdsUC2MT8VOXu8p6zKVhRzFn/I/mzZJilWJC7TrNiwJhzvFh7KWXzaqqRbG7b1M47Tm2NJKcIeLWlIJJ0kEZXiV/NUK3GVfYtXzQks+mpPsqJu3hTeLeA9o9OhJlUbQOmOyJbjNetKHcNSsZCuYvML27m/8fWQUIgn8hBS0E8VPaeBgL5stf6Hp1xyCi+qWXjRJOVum457kjnxIaVETK+deMCDL1Jv8Nhrp55sk91WFZaDHq9M6Pk+73gIek/uGPQgVMk8HajLptlPz1lVM65fHSj4TkHY32EgvcjvPFgFYXBzUHdNYlivyOe7LE3ZXZTUPUUBOrpc97zgCvpomhEVkCL01NNBA9laO8mtERFk4YPiHUOjWFVtUKIjNlHRsiUllS3pnJJqioQziiq3Z9yeXgpntCp4UoukeoWxt87x8y3LRaCvjYu027xP8riqsmTxh3HCbcoAuEvjajtEvOLnrP6tIxDb/i/b/uDBpn/6Ir/157Gd9iAcIMeplPEgAE5fghzKBPMyuY7LDa7HEkMmzAh4qAC+bytxHtfZE/8YKmp0d/jMmHBEdyEmJQwDvov28burTjSVOqLmFM9APuI7at9Z6qhhjuG1z+MX2Vb9VyNU/1GQY3FlKoQjzmbyuFRlT2iRo68wUfWPy2HwfXAAEAYgCiZHbjCM3M+4zOg7M41jNKtpGbL+JYesqOCEfZTT64cs+uOONA5Z2fhuh+ytkQn1TzBu0XnzqY5xC2XN/57gatA9MVNIz4D7B9U8Bcc1QrLmCWfSPOErFmZmlJi8tGwl4rSaMyEXxwKv56gzqHAZGYiC71RbXMRzS9Av6+mXgVA2S6xRW1BPdwNqC1SZE62tl2ZPHMjo9wPLCb5jWN90aN3SMzpPxXB8ZCE2/VDMir7tPq5jesmnPS4pR5OCydmPuI7Ttv0Rl09NV3nbwcP42lFz82h86xxP+xeW/frx/tH960UfSexd4H1+UtCVDjqVizsxLiaTRzWMj5C3tqkVpvDLe6pZCkENI0S12Nd5QwaPbesrdD4dixP7cr6SnpArse10VorX8SGv/0f1hE5O/X7KAYb38gXVYb8nFNdUfYm3VNyDjrObPP7SHh1c0jd1l++6bGabCsdlsm336pJOmzcJ5XnVI2hlxhfzwOdI/BZtelcVWeiqxK/r6uAuldk4n0QTXXQ/D7T+AcSw9y6GLymGoSfEXCjX11RyGPTO6zeNFNWS9IQcXlLN4rD7flEsS1a5IygKU15uUg2nIgUVrZTUtIvrFJ4QCkHw0FOtHymEZ6hDdqrismfXXf/ZEPwHEJfgXVxeUlz6gmsljGSrTq21usHbxwaSwxEMSsuxbLxSySeqjY6rChcaPCZjAuuo1YRUUftT9KW0yMnmnbzfT94ImKWuyjVqavQWB/Ys10dTydIDigxJFymCXbSQVBUVb2rAvmuvau1VYAhfUfNNrb3qYAjZ88ppmJPofkfpsRiH6/OCy1AS4tV6UmPSgDpw+ExOoCzNNF/xMST78z533E5bHxp+1wn+OkzwmZF9qxD6cFpd1aGUIj7SzFfPa7OBH9qXjAUjnt99g5nLoX2pWDDiORw6BuG2LxELCdLcN5goHtqXhgWFSGFf5VSYC277krBgKAoTg3Dbl4IFHb6qod9nHpiA274iH9Dh7R/omIM7UvknrxxuV+Du0CDc9hX5EP3vfqSoOTEX3J59cEc83NAxCLdsVQq5l7HKpdlkyJI2y/YSCbKXyFwK+VBDoKjJNxj6ulOXItkaVec4E3UYFaMXe4SC/bUrsxmJfhtf4bXx56Lbj+ipFL4ZooEGF3dc2md/+YLKBAzWRUP22V++z+c4+QZ9Z8g++0v8SIUPDH4SxD77y4cCdyuyQ2eD2z77CwgGge+a8531eUkWwS1Olb7ii06zwW3fqp74eQpfkbM1G9z2rer5njhVGoTbvlU9IETGmNS7A/tW9UAgcLcDVaXnZiqQG9hnV4r1iI1q3vbZlSAU+Nuk5m2fXSlW0jOpeYf22ZXiZGlS8w4ttCuFSsQmNe9+SdEmuIWaHyY178g+VdAXSverFnpmg9s+RXAIxh2EicmvqL5CeL+hANKQQzGVWzH9/fG+EJL7wfHAYlwMqT+qp26c20YiyWQyUyAJ9DKtj10QV03PLZA0zEVDiI/Qkb4CSW4fvDXinl+3MXuYtK1flrHtHWZ5v82yulivVVwxX7Hl9Kw5+8hW4b+12UMPf+q19Sme1iAbPIevOwP74Tle7QUKrtSx5u46Kq+TmbIzZyR/qXJChcuaPLL3ejMvJp0BF6nCCHRVnKG7JWEJ+ic5Q3HZfiQpZmf8Hw==7Z1fc+O2FcU/jR+bIXABkHzM7jbpQ9K03XYyfaQl2tasLHokOrubT1/KFmXrwJnsTFYHE59uHmJTfywd3gHu7wL34MLe3n76ftvd3fw4LPv1ha+Wny7s3YX3zrlq+t/+yufHK3XbPl643q6Whyc9XXi/+rU/XDy87vp+tex3J08ch2E9ru5OLy6GzaZfjCfXuu12+Hj6tKthffpX77rrPrvwftGt86s/r5bjzePVxtdP1//Wr65v5r/s0uH7XXaLD9fb4X5z+HsX3q4e/j0+fNvN73X4orubbjl8fHbJ/nphb7fDMD7+dPvpbb/eazvL9vi6737j0ePn3vab8Ute4B9f8Eu3vj989b/3H6cL/x22Hw6fcPw8q/Lwvfr9K92Fvfl4sxr793fdYv/oxykOpms34+368PDVar1+O6yH7cNr7Sru/5uud+vV9Wa6tpg+YT89+GY3bocP/bOnpod/0yO/9NtxNd2Ubw8vGYf93zh84umx/tNvfmt31HKK0X647cft5+kp8wvq8PiSQ3i6+XZ8fLrZrj1cuzm50YeL3SHAro/v/STy9MNB55c1t0zzH4bNcti8Xr2tLal3yPT+9nY3abHsbl+v5MmVlDxmkmdKTyPf3f7Hu+2w6He731f7aWj96X5crzb9b9yFqyu/WHzRXVimyxTT19E7VulU7+i/iZni/iXBv4LeSU9vs1O9E1PvWk9vD/HdMvVu9PQOEN81U+9WTu9QtzCe1ES95/cQEjybMGuq4E5O8NCeRrivqILnpPnaBY8OIryhCp5j5msX3IcGxvCcec4md06Zr17uOsIITpRbjzB9Oo1uXxHl1gNM30B0N0S59fjy+JLjYEKcKvXwMpsqG6LcenTpI8jd8uT2emwZDGCeWoz1gmyZAHWS5XK7M8mtR5YhQnxTi7EvLGC+esFrLFZRBddjyzDXi4qUYwXXL7Mpk1qOFVzADHYa4dxyrOAKZggQ4dRyrOASZh0B6SNVcD3KrAPs+aFGuOlxZp0gwqlZiulxZu0hwql5+DxjKwluOKRQBdcjzZRwX1vFFFyPNPdvdCJ4SxVcjzRTjTvbqILrkWYyLF5FpuB6pJkCbvyhCp6T5j/67W7YdPtnfbda97vsBvyRtodl1zdXXyZyWjT95dUFsbPHv7An5YydPTlzvuvGbrryQ/ehf72qY38PV/WQg+dPd/22G1ePMf9aVccWH7LqevSJW5atYsJQ0KNPbPIxRxVcjz6xy8cCVXA9+sQ2H/NUwfXoE/t8zDFLtkGPPrNJ01MF16NP7PMx6rJb0FvnxD4fM6rgeuucuHnZXA4+55I76q1yYp+PvWAlcTa5/2SUuXu0Mqq+kvIOAj0SldfDTWz5MSPKrQebx5ccxxXerBn1UDObNY0otx5oYsuPBaLcepiJ+5e5tdkoiJnQ8mOO1/IT9SATW364ldmoB5nY8sOtzCY9zMSWH25lNv3JQPMcUya1Mpv0+BJbfriV2aRHmNjyw63MJj3GxJYfoza1JT3KxJYfcoTrcSa2/JCzFD3OxJYfch6uR5rY8mNGFVyPNLHlxxyzIaLWI01s+bFAFVyPNLHlxzxVcD3SxJYf88wOlFqPNLHlx2JgCq5HmhEjnGrxVuuRZqyglkL1Oqj1SDPfi08VXI80Q4II5w4peqRpvvkGdlxxY1yPNa1yKDn14KRGjzbN5VFOlVyPN81lUe6MKbletlK2Lt7oZSt1hDItdeps9LKVuoYIp26XaPRylbILEa1eplJXOKRQBc/zlO9Wm26z+LrGKVfNov9CbS+bGOK+lYdmVxOop/S2PlP8X/2u77aLm9crOXrVkCXPS+M/dpv7q24x3m9Xm+vXqzu61ZB1F6yQQ+N9oJazWsEKObjVBGplpdVjTqyQB+oxKK0ec6JbTeAeSq3HnOhWE6iLbq0ec2aTJvdU6koPOtGuJnKPpa70quPoVxO451JXOXW+dsWx9T4wD6au9HZjoWNNYJ5MXemRDxqnBOZZyZUe+Bxfcoxv5vitxz3Z+M08LbnSwx70TgnE45JdpUc92AnOLRUeC7xKioN7SiAemOycHvOgfQq3VOicXg6Odh7c0pVzegs+2SjOrV05Pe5BQw9y7crpkQ86epBrV06PfdDSI1BNa5zTox/09GDHuB7/4OZlcq7i9fgHXT24K8nO6xEQ7qYN1JN83SywkOLo6xGoR/k6r8ecaOwRqGf5Oq/HnOjsEaiH+Tqvx5xo7RGop/k6r8ec6O0RK6a3h/N6zInmHpFq5u68HgHlW2mpBGR6BIRuE+QYNz0Cyu0m2FGux0C530SknuHrTI+CcsMJ8mqn6XFQ7jgREtNxwpkeCRWu2poeCaHnBHv+1CMhNJ0gr+mbHgmhCQJ57gx6ebl3EOPxhb1wZ9NbLyd3DUT4bHJP0VsvH8/iu2bqrZeLH19yXOlk6q2Xo6Ssg5NaHw/5DqF/3nfbbjPuhULt/4ifyjL2zTJ8kb6Nv7S015dmHRRf6HE7n5+KC3lm+O1iXP3yihVH5yCy4vPZ788V3y5uXrXkaBrEllwvGUcDhNhSl5ejXjqOtkGp4iqul5DjYmcyruJ6KTkaByXHVVyvOI7OQYl6qJiLesVxnDkT9ZArF/XAE62DUuQqrteagtZBybiK6y1HoPVEcjkBnU3vlCPnq9cbrIPSCza159NbjzfROigZU2892sTiePLE8TvpsWY2fhtTbz3SROugFJh6C3ImmE6Q64VJjzPROijN3MmwDkqClAnWQeRqYdKjTLQOIteukh5l4ihOrl3VepyJ1kHk2lWtR5poHUSuXdV6rInWQYlrj1Xr0SZaByXjKq7Hm9iEkqjnnrpajzjROih56gb9Wo840TooUQ/idLUec6J1EJmAaj3mROugFLiK6zEnWgeRmbPRY060DkqeWslq9JgTrYNSpLZGNHrMidZBKXBjXI85s9203Jmz0WNOtA5ix7gec+bWQSlQOb/Ro87cOihRD1R2jR535tZBKXA11yPP3DooOap1UKPHnoWrtq0ee6J1EHn+bPXYE62DkucqrseehevkrR57olkTOVtp9dgTzWxSRTRravXIE82akiOa2bR61JnFt2fqrUecx5cc50ym3nq0mRqoHXLXgFo91kR7LO4akK9y1nyz/3Q/DItuXA2bTP7pm46n+p7Ksxn2Ap+Ie7g067rur/bvgK5At6vlcv9HXrydT5ZF1f7Nh8347HZUD/8O198fPqr7CvcmVCe35mgU8ezGNC/cGP9VbkyOpA835j+77rq/8Gm91/ByO/10PR4j8f/3af+rVfkk8ZIL1Ne5TznIPtynN/e7aaiZxqfplm1W4xfcn22/W/3aXT48Ya/S3bDajA8fLb65iO/2d+Z+HKbnzAPY79yoR5ut3XSDVpvrf+9/efeXUO6uhLnd8Lk5lzvbbclp93FcW131i8+L6Uv66v3Yjbk7mtyNedlt54y3Rg+LM2MMqomXr/TAuKyllK/00Ljs1hpf6cFxjLhFkqu4Hh6HDI+Zm5l8pYfHZS2ljm+ipHgFMU49l/hYYFVSvOgWST+30wspnjX7cvNxp7cUG2qIccezJfFOjzjLbgL2To84w/wVy+SGTo84y1o2eKdHnCFAjJNzQ0HiRHspcm4oSJxZUx515vSCxImVQ6rZ0fEDCyme2f+TY1yPOAuvAHk94iy8AuT1mLPwCpDXY87CK0BekDnLrgB5PeYsvALk9Ziz8AqQF2TOsitApsechVeATJE5C64AzXbKSnqXXQEyPeIsvAJkesRZeAXI9Iiz8AqQCRJn2RUgEyTOsitAJkicZVeATI84C68ABT3izBifux4R9IizcHV8zpSUFC87cwY95ox4FAD1+DkfBJkTDRu4zBn0mLNwdTwIElDZfDzoEVBG+eTsUJCAGjzugrgeEfX4pzBxRj3+yQ4R5WYqUY9/Ctdqo2A2jquc3EwlCmbjeBRa4o7jeitAWVWFu+YW9fgn2+VJjnE9/omzC8FxPxbTqttHPf6JhsRJVTzpEVDEfJxqR++THgFF3B9EPQx6+nNyiof5+LciR+f4JLgChDMn9SAXn/SYM1YQ45GruCBzeohx4yqux5whQIxz8/EkyJw1xLh74eicc61HJD3iDAkinJyN6xFnmP17yuSGtR5xZqM4Nzes9YgzGMQ4Nzec5xAlxSPEODc3rAWJE3uAInXmrAWJM6sccmNcjzhDjYzPjXE94swYn7seUesRZ+HqeK3HnKVnTj3mjLgji3ro9vF4QCHFQ1uUORs95ixcHW8ECahsPt7oEVBG+dzssBEkoAZi3BPXIxo9/ilMnI0e/wQPtVpypqLHP4Vrta1gNp6tcnIVF8zG576n495x6jjeej3Fs6oKN8b1+Cfb5UmOcT3+iRWeZpCoiuvxT5xdkZ/6w6mK6xFQ9BDjkau4HgHFADFuXMUFCQhdsM1RFRdcAcKZMzAVt0qQgByO41zF9QgooF8TNTu0SpCA0HdvPpmHUB23So9/ArpOUHNDq/T4J1S4c4I7iuvxTzaKkzMVwWw886vlKi6YjRvMm4k6js9voqQ4VlUSNcadHv+EhH5N3BjX45+MOKm1WnN6/FO2cmhOj4BKz5x6BBQjxHjgKi5IQOjJzCUgJ7gCVLZW6wQJqGw+7vUIKKN8bnboBQmoxlNpiNXxWV4lvcsSp9fjnzDPXGUyFa/HP4VrtV4wG8+c3qmKm16mEt2p4r5l7mQ208tUYjjNVKziKq6Xq+DZYkbtATLTy1XwHFSj+u6ZCeYqcH6EUbuVzQRrtTBzGtXLxkwvO4ywP8io/cpmgrVajzMnV3HF/UGnMc7Ox/Wq49gfcdyVxqgcBj3ixPPzyNl40CNOXOMkZ+Nzr7SS4vNXLJONBz3ixHmTnI0HQeIEDwRyNh70iBPP+iVn40GQOGEFyKheNtNf01McVjmN6vNhQY84s+o49VQaC4rECTF+zlFl+nU7DOOzx76f5Lr5cVj2+2f8Dw==5Vjfb5swEP5reOwUYyDJY5p020OrScukPTtwgFWDkXGatH/9zmDyA5N17ZJIVZMH7M8+23zfne+ER+fF9ptiVf4gExCeP0q2Hl14vk8IGeHDIM8tMp5OWyBTPLGT9sCSv4AFrV225gnURxO1lELz6hiMZVlCrI8wppTcHE9LpTjetWIZOMAyZsJFf/NE5y068cd7/DvwLO92JpF9vxWLHzMl16Xdz/Np2vza4YJ1a9kXrXOWyM0BRO88OldS6rZVbOcgDLcdba3d1xOju3MrKPW/GPgkaE2emFhDd+bmZPq5YwNNkHjs3G5yrmFZsdiMbFB7xHJdCOwRbKZciLkUUjV2NA3N3+Cy1Af4qPkhzgTPSsQEpHjaWyU101waJDTDEjfi2rhQZLq1VvIRhtaxrwBKw/YkEWRHL7otyAK0esYp1iAKv4StjXVZf2TF3uwdgHb+mR+KP42s41mny3aL74nHhuX+lA7jy+mQMJik8bt0IMG1hSBjv6+EH11VickFlQhhkgQfRIlw3BOCkh1yJSnCC15Okxji9wVF73IKLh8TA1LQqwpBHSHmgpccMyaivxQ3z0gYnlYKW5lpzeoatKMXZrzKNCslY6jr13Xbp9Qfa42bwgk909R/i56OYEm0isLoPIL5pH+JUepGDvEHBItG59Br6ui1YJohcs8ewdEEX1Qfk66g5i9s1Uww5FWSl7o5UXjrhQtD51rLui3ciMuuYc94x8zCWhpVaxMzZfbLdBY3f7sHz6ABjZxEQsnA/TUeEME/hwjdIgci/IQamIrzT6IA7SsQRMFVFSCOAveyTPAS/xT8k8CJgFHgsB9djH3fYX8Wa/70Se4f4np/6LIfXox9N2V3KRppKHtJuh3ARQ/HPHMQ12QBKab+phY6ZdcvBfDJCqNAuaqr3cpngujsw2z4Ikt4gGIFqs551bqzsDfS3HwvwSR9I5oUbbpqly5Mj7XBg94/sHsvopp6CRIbPa9UWKdLof+ssYaj0Bgv7VHJGQItnI6OwoxMpm5tHAzVxsHba2Ps7r/ONGMHn8Do3R8=7Z1bc+M2EoV/jR83RQAESD7OJdl9SGq3dnYr2UdagmzOaEQXRWfG+fVLSoQsNemYHkuHEx15qhIL1oU66Ab66waBK/Pu89e/V/nd7S/l3C+vdDT/emXeX2mtlIqa/7UtD9uWJMu2DTdVMd82RY8NH4o/fPfK0HpfzP26a9s21WW5rIu7w8ZZuVr5WX3QlldV+eXwaYtyOT9ouMtvfK/hwyxf9lt/Leb17bY11clj+z98cXMbPlm57vtd57NPN1V5v+o+70qbxeZn++fPeXiv7ouub/N5+WWvyfx4Zd5VZVlvf/v89Z1fttoeyvbTE3/dXXflV/WYF5hwIb/ny3sfrnlzZfVDUGPzfXz7iujKvP1yW9T+w10+a//6pen/pu22/rxsHqnm13VdlZ/8u3JZVptXG+NMZpoXv10Uy+Ve+2IxmzVm0bSXq7ozAZV0j/eeF21+mvZ8WdysmralXzTf7W133b6q/dcnv73aadrYqi8/+7p6aJ7SvcB1rwhWGtvt4y+PfZ7FXZ/f7vW3iVxna52d3eze+lHr5pdO7iekV9lU0vvNz6TSJ+mh9C7uS69MX3qdRMeQPr1YfZBeRRZp9q6n/M/lal6u/kR/9bz+UmXb/ttTb9Zo46uhjnKbn+YvraZFMwW86V5Sl3fH0dva6FBw5X7oK66yqK/4biJ9jeLJ87beTEN37a93VTnz6/Xzej/Oc/+8r5fFyj/RD4uFns1G9cPcXTvrjmThTh8qHmc9vfWQ3EdQe8TIcm5qp9mh2han9ogp9OzUFrad4tRWI4LFc5M7E8btgHIrPrmNFXL348KTya355JYTZQKU2/DJbQ+tW0dAuWM+uRM5VQLltnRy20yM3cCoW/Wx8tzldlpYt80GsDK879EFJ6RKJewbGXkTYqUR9o2MvPm40rpUyI2bLMcUIc5NbjlZAiNvzceVNj20bmTkHdQlkttFcqoEyt3nyvd5nTctP+effE/515QZ5rlPF+PUdbPUXy+QZYYG7qFlBk0ImCJ9ojNciKL5AFMWGkwElJsQMEWlwRig3IR4KSoNu/UhCLkZ8dIKuYEhCh9eyqnSaJzchg8vZaXBWKDcfHgpKw3GAOXmw0uZPEHG3YawbKmFdUfISoMh5EpRaUBG3oaQK0WlARl5Gz6ulJUGZORt+LhSTpbQyJuPK2WlARp5E3JlJKdKnNwxIVdmYqoErvGJCbnSCesGFuVjPq7sZbxjoNyMXCmsGzmY9Kny337t82p225P9VXeipTM/Utrr1Ma2veUPViKOwXeixYRsKfLeMdLG+dhSlohj4L1oMR9bygkzBi6JjfnYUpaIY+CS2JiQLUWJOAZG35aQLeVUCVwSawnZUpSILXBJrCVkS1EijoFLYi0fW8qsNzLutoQVSy2sG3ozmiXkSlEiRkbelo8rZc0SGgrycWVv9EaGgnxcKWuW0FCQkCsjOXbj5HaEXClqljFwtaAj5EqZE8yA1k3IlaKIBrXuPle+mdXF70e+y9L6dB6PEjbV1wa7maMF32XpCNlS5AVtGM8RFk5IlqKE5iKg3HxkKafL3WJkhNx8ZClLaE4B5eYjS1lCc1GCk5uQLMVU6RRO7oSQLEUJzcVAuQnJUpTQnAbKzUeWMgmLjLsTvoqlLKE1cTewhJYQcqUooSEj74SPK2UJDRkKJnxcKUdvaCjIx5WyhAYNBQm5UpTQoKEgIVeKEpoLN4chDtQh5MpeThBn3SkhV2qJ8UDr5uPKHsYDCwwpIVdOmPFOCalyyqmSjyqTSFi3BsrNR5Uuno4qUz6qnDLjnRJSpdhuw1mg3IRUOSHmZIRUKXMmwLg7I6RKsatgE3cD6zkhCmUSfEKMz/i40iYi4w0MBTNCrpww453xcaXNhHUDQ8GMkCt7tXig3HxcmYTQb2fdwKmSkCt7OUGgdRNypVx0DLRuFRGC5YQFHRURkuWEFR0V+pZKbzl8I/XmI8spazoqIkTLCYs6KuJjyymrOioihMsJyzoq4qPLKes6KiLEywkLOypi5Eth39DKzm4bCSbFJyztKMVHmFPWdlSgWSK9pyzuKMVHmFNWd5QiJMwJyztK8RHmlPUdpQgJc8ICj1KEhDllhUcREqYkemTFQfER5qQZcM3Hl5POl5qPLxMl7Bu49YYK8hLp3duoCsmXmo8vJ82Aa0K+nDL+1nx82eN5ZDyoCfkyFvatoRUHfSFMbETIR5i9/ZOgEQofYU6akTV8hNmvYCL1ZiRMMWOGnZEhevMRZj+DgrRvQsKUazah9k1ImGL3Ugs8ZkcZQsKU25dGSL0JCVPG3wapNx9fJnLFj0LqzceXLlZCb9ypdMoQ8qXc7VsD9Y4J+dIJ+7ZIvfn40skVsgapNx9fuujQvqHxd0zIl1qe1pAhKw4xIWFqMYIjI/CYjzBtOPJmiogwJiRMMYJjI0I+wrSZsG9oREhImEqO30i9CQlT3sMTA+dLS0iYvROOgPZtCQnTiPEEat+EhCmJHllxsHyEOWkG3BLy5aTzJR9fJnKFlUbqTciXdkK+tHx8OWkG3BLy5aTxNyFfSp5HxoOOkC+N3HMGWnFwF8KERoRhvSKR3jYVGVlkhOL4CHPSjKwjJExZwURGKI6QMCO5ixVy/CYkzN6uYUj7JiRMuWYTat+EhBn03UXgBqg3IWGGA4R38SBQ74SQMBNh3xapNyFfZsK+DVJvPr7s7Usd1nBC9CbkSzlfGqTehHzphN4xUm8+vrRyVyVkPJgQ8qXcFU8bZAY8ISRMuUsENCLkI0wrT+KBRih8hNkbwZERSkpImHIXFGSEkhISZipmTAccv8Men0x69zIoSPsmJEwtxhOofV8IE5qRTfkIc9IMYUpImJPOl3x8mUTCvmOk3nx82dsnGco7fHw5aUY2I+TLKePvjJAvJc8j48GQu2HSWwv7NtAMeHYhTGhEmPERpk3kvuvIEZyQMKfMyGaEhNmrqCH15iNMlxzqrTPgmuSMkDCzwwjFREi9CQlTVDAN8B4eHfERpjx51AB3xdMRIWGKcxwM8C5jHWIjJr3FfGmAu87oiJAvxYofA7zHWEd8fClPHjXAXWd0xMiXh/aNjL91RMiX4h6HJv4GZgh1REiY4uQ6bAROSJiihomNwPkI04p9rKARuOIjTDljQiNwxUeYVuxaAI3AQzaYSG95si40Ald8hClPHjXAXWe0IiRMUcE0wF05tCIkzF4GHKk3I2HKjBVwPAny7un9Pq/zVvCy+VbNx63q5h3dsm51rJrfburN15Ytb+Yf79f15+Y7r5sXLxqhdLT0N3n7aR/vq2I9L2Z1Ua5Gvdd67ev2oop1cV0si/YKonw1b/8723S5jmblqumU5Zi3677Qt7z0l7xoTGCVrxrLCpfQWtP9XdvFW/srVjdj3uq/q7mv1nXzHpsXRHN/55um1azw61GX4htb3L60u47Gep/4bOEwjW227ZVfF3/k15vWqHl8VxZtbzVPsW+v7PvW8O/rsnlO8I/gB0u/aF/fWnkxy5dvuua6bP1s3bhdcxn/aR+8/1vculWj756vRJufrv1D9+7JoYcewY12h6Y9hMcDbpQN+JF2x3CksKxmT3Y/v/EfuodlVd+WN+UqX/742Pp2Myb5edcfe3r4r0X9297v/2uf8oNtH63mb6qq/NI0rMp2JGtbfiraC9u8yUdf1w+dym13Nk2Pn/1zuemlgbHvyU56qjObXqkefus+dPPg8Rq3w+Wvne7xrn9bQf68d9th576a+TA6dV1Y59WND921Tcj17aDyy7wufj/8hKE+7V76r9b8H+0nDhsUdfaThmRneIvtRXSvEpaxu4yRo+6IKO7RNtTzM1xvgjLOZGbe7+f4ffvvBf0sxoAjOOpuBUMQOuw0t+eoamjCO46fOpSfPu0f34sPH89P7YCfbo38FX46PogZEaSfrTspgf9ZOIoB404Z17R3PJdxQy6TwFxmBGedrcvIGSgLrgBxmXBY+GUGOqY7JUORooG504gy7dm6U5yJGQjqTnpExfZspU+tgBZsLJ1cRrLjj2TZ0Ej2WuYd704jCvJn604yMEiSfur7hO5kL+50dHcy0VCcDUPT8PGU7iTRFDw7xRd3Or47qWndacRyGhp3ShJkpiecjcGS6YEUOMLi4uNi63CBw2nBCup0BY7wvSj9NBabjKUDx66d0E/7Sl+mvVf7qRma9tKT+GmqDtdPpuEu8/AW2ys7jp+OWL53tn4qD7jFhqdjTsw+dMuXSj9Psuutu8h1Tt5t1jlNKL04O1tFycAYmQ6t1kiOIv6L8rVnJn4qbmkaFt+dTvwXZWzPTPzYRSPEH1qntNtv+3Xij8jvtWsl7578rrNytfKzultIdrVbmzheAy128RuvwXEMcMxh139hA+z6uC//c+pmRxH3RYh+PuI+HGqI0fpFmHV2Wu8CBIzYI2LlU4+c4tZB+MD5onWLfz17c09kXjD29aJVbN8irpul/nrRF3ee+3QxmbiTDJznDV/PaQ0eOE8OW6m+Ns4NGLb16Tw+sdgvjPf1wCHeJ9T+clvC1Yuz9gepv8AqB6m/cFTJCDt49W0JIlOiTpi2H3Mm9tmmA43YJ0DtDiiE5ANDgHbJ23+rpwZPOfDUYNLH9NTxfcpdrj4sg6ld1IXxp77UZz3zfaPPDBW2wg330/gMc0mqNwftpgWMz1yWTL3Wn4buhAsHf03jT8w3x8mlGGh/Yr7LKpU1XnA8fbk54bVjWTo0lp2AfMf3KfO9Pr3YIAoHwGH8CXan77n6U0i6fj98OubE9LP1px6fYuensJrj4k/f7E/6u/MnffGn/fkJme8Jp7Wz5HuOX+lIh3ZbOgW7Dlc6nJa8oE9X6Rhz1vzZemocVpTvlB7YYv2Ennq5z/W1M9/Q5jEJzFPTsIP2LqsSne4WhZQ5h+JSoTQ0RjXhgs90SdIz+Sux1s4M4fapFm6YoDTlohkpvQ0HtWGkf1EF6i+3OOxl0js1MOAcSfrmYVWW9f7c0Gh3+0s59+0z/g8= \ No newline at end of file + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/features/metadata-security/overview.md b/site/docs/features/metadata-security/overview.md index a01ba1c822..349693f298 100644 --- a/site/docs/features/metadata-security/overview.md +++ b/site/docs/features/metadata-security/overview.md @@ -4,37 +4,42 @@ # Open Metadata Security -Open Metadata Security provides fine-grained authorization services for open metadata services, types and instances. Since each organization will have different security requirements, the support is implemented through connectors. Egeria defines the interfaces and when it will call the connector. You define the behavior the connector implements, and Egeria acts on the returned decision. +Open Metadata Security is optional. When it is activated, it provides fine-grained authorization services for open metadata services, types and instances. Since each organization will have different security requirements, the security support is implemented through connectors. Egeria defines the interfaces and when it will call the connector. You define the behavior the connector implements, and Egeria acts on the returned decision. The [`metadata-security`](/services/metadata-security-services) module defines the base classes and interfaces for the open metadata security connectors as well as the server implementation to host and call them. There are two types of connector: -* **[Platform metadata security connector](/concepts/platform-metadata-security-connector)** - secures access to the platform services that are not specific to an OMAG Server. This includes the admin services to create new servers, the ability to start and stop new servers as well as the ability to query whether a server is running, and if it is, what services are active. +* **[Platform metadata security connector](/concepts/platform-metadata-security-connector)** - secures access to the administration and platform services. This includes the services to create new servers, the ability to start and stop servers as well as the ability to query whether a server is running, and if it is, what services are active. -* **[Server metadata security connector](/concepts/server-metadata-security-connector)** - secures access to the specific services of an OMAG server. This includes the server itself, specific services within the server, specific Assets and Connections managed by the server and the types and instances stored in the local repository. +* **[Server metadata security connector](/concepts/server-metadata-security-connector)** - secures access to the specific services of an OMAG server. This includes access to the server's configuration document, starting and stopping the specific server, calling specific services, then within the server, access to specific Assets, Glossaries and Connections managed by the server, and finally the types and instances stored in the local repository. The 2 types of connectors are shown in Figure 1: -![Figure 1](/features/metadata-security/security-connectors.svg) +![Figure 1](security-connectors.svg) > **Figure 1:** positioning of the security connectors -Within an OMAG Server Platform there is one instance of the open metadata platform security connector. This connector is configured once the platform is running using the admin service call: -``` -POST {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/platform/security/connection -``` -where the `{{adminUserId}}` is the administrator's userId. The connection for the connector and the platform URL root are passed in the request body. There are `GET` and `DELETE` services with the same URL to retrieve and remove this connector respectively. +Within an OMAG Server Platform there is one instance of the platform metadata security connector. This connector is configured either through the `application.properties` file before the platform is started, or using a REST API call once the platform is running. -The open metadata server security connector is configured for each OMAG server to allow for each server to have a different implementation. The admin services command to configure a security connector for a server is: -``` -POST {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/security/connection -``` -where the `{{adminUserId}}` is the administrator's userId and `{{serverName}}` is the name of the server where the connector is to run. The connection for the server security connector is passed in the request body. Again, there are `GET` and `DELETE` services with the same URL to retrieve and remove this connector respectively. +Once it is in place, any call to the [administration services](/services/admin-services/overview) or [platform services](/services/platform-services/overview) results in a check to the platform metadata security connector. -The security implementation in a server potentially invokes the server security connector multiple times as the request (shown as dotted white arrow) is handled by the server code. Figure 2 shows the different layers of checks. Each layer is optional and so the server security connector can be implemented to support the most appropriate granularity of security for the situation. Details of the implementation choices are given in the [security connector API](#metadata-security-apis). +The open metadata server security connector is optionally configured for each OMAG server, to allow for each server to have a different implementation. -![Figure 2](/features/metadata-security/layers-of-security-checks.svg) -> **Figure 2:** layers of security checks within the server +The admin services command to configure a security connector for a server adds a connection for the server metadata security connector to the server's [configuration document](/concepts/configuration-document). + +The platform and server metadata security connectors operate independently, depending on which service is called. However there is a hand-off whenever the server's configuration document is read, either as part of a configuration request, or a request to start the server. + +As the request is received, the platform connector checks that the user has access to the platform. Assuming that is ok, the configuration document is read. If the configuration document contains the server metadata security connection, the connector is initiated using the connection. The server connector is then called to check that the calling user has permission to access the server. + +This means that whenever the configuration document is called, the calling user has to have permission to access both the platform and the server. If the request is an administration service request, the user has to have administration access to the platform and server. If the request is to start the server, the user has to have operator access. + +![Figure 2](accessing-config-docs.svg) +> **Figure 2:** Calls to access a server's configuration document. + +During a metadata and governance (OMAG) service call to a server, the security implementation in the server potentially invokes the server security connector multiple times as the request (shown as dotted white arrow) is handled by the server code. Figure 3 shows the different layers of checks. Each layer is optional and so the server security connector can be implemented to support the most appropriate granularity of security for the situation. Details of the implementation choices are given in the [security connector API](#metadata-security-apis). + +![Figure 3](layers-of-security-checks.svg) +> **Figure 3:** layers of security checks within the server The security connectors are optional. If they are not defined then there are no additional authorization checks performed inside the OMAG Server Platform nor the OMAG Servers hosted on the platform. As such, it is important that the open metadata platform security connector is configured as soon as the platform is started, and the server security connector is configured before the server is started for the first time. @@ -50,8 +55,8 @@ The connector that plugs into the platform implements the following interface. * **validateUserForPlatform** - Check that the calling user is authorized to issue a (any) request to the OMAG Server Platform. * **validateUserAsAdminForPlatform** - Check that the calling user is authorized to issue administration requests to the OMAG Server Platform. - * **validateUserAsOperatorForPlatform** - Check that the calling user is authorized to issue operator requests to the OMAG Server Platform. - * **validateUserAsInvestigatorForPlatform** - Check that the calling user is authorized to issue operator requests to the OMAG Server Platform. + * **validateUserAsOperatorForPlatform** - Check that the calling user is authorized to issue operator requests to the OMAG Server Platform such as starting and stopping servers. + * **validateUserAsInvestigatorForPlatform** - Check that the calling user is authorized to issue requests for information to the OMAG Server Platform. For example, to validate a connector or request lists of registered services. ### Open metadata server security connector interface diff --git a/site/docs/features/metadata-security/security-connectors.svg b/site/docs/features/metadata-security/security-connectors.svg index 662dbcdc5a..2834cf624d 100644 --- a/site/docs/features/metadata-security/security-connectors.svg +++ b/site/docs/features/metadata-security/security-connectors.svg @@ -1,3 +1,4 @@ + -
OMAG Server Platform
OMAG Server Platform
Platform
Security
Connector
Platform...
Server 1
Server 1
Server
Security
Connector
Server...
Server 2
Server 2
Server
Security
Connector
Server...Viewer does not support full SVG 1.1 \ No newline at end of file +
OMAG Server Platform
Platform
Metadata Security
Connector
Server 1
Server
Metadata Security
Connector
Server 2
Server
Metadata Security
Connector
 \ No newline at end of file diff --git a/site/docs/release-notes/5-0.md b/site/docs/release-notes/5-0.md index 60032970da..8e096e5a3a 100644 --- a/site/docs/release-notes/5-0.md +++ b/site/docs/release-notes/5-0.md @@ -31,7 +31,7 @@ Release 5.0 is a major functional release focused on usability, both from the po * The [*DataFile*](/types/2/0220-Files-and-Folders) type has a attribute called *fileExtension*. New types that inherit from *DataFile* have been defined for a wide range of file types: [*SpreadsheetFile*](/types/2/0220-Files-and-Folders), [*XMLFile*](/types/2/0220-Files-and-Folders), [*AudioFile*](/types/2/0221-Document-Stores), [*VideoFile*](/types/2/0221-Document-Stores), [*3DImageFile*](/types/2/0221-Document-Stores), [*RasterFile*](/types/2/0221-Document-Stores), [*VectorFile*](/types/2/0221-Document-Stores), [*SourceCodeFile*](/types/2/0280-Software-Development-Assets), [*ExecutableFile*](/types/2/0280-Software-Development-Assets), [*ScriptFile*](/types/2/0280-Software-Development-Assets), [*BuildInstructionFile*](/types/2/0280-Software-Development-Assets), [*YAMLFile*](/types/2/0280-Software-Development-Assets), [*PropertiesFile*](/types/2/0280-Software-Development-Assets), and [*ArchiveFile*](/types/2/0226-Archive-Files). There is also a new relationship called [*ArchiveFileContents*](/types/2/0226-Archive-Files) to identify the contents of an archive file. * There are new attributes called *isCaseSensitive* and *category* for [*ValidValueDefinition*](/types/5/0545-Reference-Data) that provides additional information used to match valid values. In addition, there are two new types of valid value relationship: *ValidValueAssociation* and *ConsistentValidValues*. * The [*ConnectorType*](/types/2/0201-Connectors-and-Connections) entity has a new attribute called *deployedImplementationType* to allow a connector writer to record the [deployed implementation type](/concepts/deployed-implementation-type) that the connector works with. - * The [*Template*](/types/0/0011-Managing-Referenceables) classification has three new properties: *placeholderProperties*, *replacementProperties*, *additionalProperties*. This model also includes a new relationship called *CatalogTemplate* to identify a template for a particular type of technology. + * The [*Template*](/types/0/0011-Managing-Referenceables) classification has three new properties: *versionIdentifier*, *placeholderProperties* and *replacementProperties*. This model also includes a new relationship called *CatalogTemplate* to identify a template for a particular type of technology. ??? warning "XTDB repository connector is now in egeria.git" The XTDB repository connector is now located in the core Egeria git repository which means it is always available in the containers and distributions. This change is in recognition that XTDB is our premier metadata repository. There are three new administration commands to simplify the configuration of an XTDB repository. @@ -73,6 +73,16 @@ Release 5.0 is a major functional release focused on usability, both from the po ??? functional "Placeholder variables supported in the configuration document" It is possible to add placeholders in string properties of an OMAG Server's [configuration document](/concepts/configuration-document). The replacement values are configured either by a REST API call (OMAG Server Platform only), or, more typically in the `application.properties` for either the OMAG Server Platform or OMAG Server Runtime. The placeholders are replaced by these values each time the server starts. This means that the server can be run in different environments without reconfiguring the server. Full details can be found in the [Configuration Document](/concepts/configuration-document) documentation. +??? functional "Placeholder properties supported in metadata templates" + It is possible to add placeholders to properties in metadata elements that . The replacement values are configured either by a REST API call (OMAG Server Platform only), or, more typically in the `application.properties` for either the OMAG Server Platform or OMAG Server Runtime. The placeholders are replaced by these values each time the server starts. This means that the server can be run in different environments without reconfiguring the server. Full details can be found in the [Configuration Document](/concepts/configuration-document) documentation. + +??? danger "Removal of the Anonymous Repository Services" + The anonymous versions to the OMRS Metadata Collection REST APIs have been removed from Egeria's runtimes since they do not include + caller information, and therefore provide opportunities for a denial of service attack. + * getMetadataCollectionId + * getEntity + * getRelationship + ??? tip "*Governance Action* is renamed to *Engine Action*" If you have been using Egeria for a while, you may be familiar with the term *Governance Action* as the mechanism used to control the execution of automated actions in the [Engine Hosts](/concepts/engine-host). In release 5.0, *Governance Action* is renamed to *Engine Action* to create a greater name differentiation between the concepts that are used to define the governance behaviour and those used to control the execution of this behaviour. The term *Governance Action* is now used as a general term for a [Governance Action Process](/concepts/governance-action-process) or a [Governance Action Type](/concepts/governance-action-type). There is also a differentiation between a Governance Action Type that is used to initiate a standalone Engine Action and a specialized Governance Action Type called a [Governance Action Process Step](/concepts/governance-action-process-step) that describes a single step in a Governance Action Process. diff --git a/site/docs/types/0/0011-Managing-Referenceables.md b/site/docs/types/0/0011-Managing-Referenceables.md index 91ecdf2569..f7f832b3e6 100644 --- a/site/docs/types/0/0011-Managing-Referenceables.md +++ b/site/docs/types/0/0011-Managing-Referenceables.md @@ -25,6 +25,7 @@ The properties for the *Template* classification are: * *name* - name of the template, designed to help the selection of the correct template. * *description* - further information on the purpose of the template. +* *versionIdentifier* - allows different versions of a template to be kept in the metadata repository. * *placeholderProperties* - provides the names of the [placeholder properties](/concepts/placeholder) mapped to a description/guidance of the value to supply when the template is used. * *replacementProperties* - provides the names of the attributes in the top-level entity created from the template that should be replaced. Again each attribute name is mapped to a description of the value to supply and its use. replacement properties can be used for attributes of type string, boolean, int and long. * *additionalProperties* - allows additional information about the template to be supplied diff --git a/site/docs/types/0/0011-Managing-Referenceables.svg b/site/docs/types/0/0011-Managing-Referenceables.svg index 64d44098e0..81548135f7 100644 --- a/site/docs/types/0/0011-Managing-Referenceables.svg +++ b/site/docs/types/0/0011-Managing-Referenceables.svg @@ -1,4 +1,4 @@ -
0011 - Managing Referenceables
«classification»
LatestChange
changeTarget : LatestChangeTarget
changeAction : LatestChangeAction
classificationName : string
attachmentGUID : string
attachmentType : string
relationshipType : string
user : string
description : string
«enumeration»
LatestChangeTarget
ENTITY_STATUS = 0
ENTITY_PROPERTY = 1
ENTITY_CLASSIFICATION = 2
ENTITY_RELATIONSHIP = 3
ATTACHMENT = 4
ATTACHMENT_STATUS = 5
ATTACHMENT_PROPERTY = 6
ATTACHMENT_CLASSIFICATION = 7
ATTACHMENT_RELATIONSHIP = 8
OTHER = 99
«enumeration»
LatestChangeAction
CREATED = 0
UPDATED = 1
DELETED = 2
OTHER = 99
«relationship»
SourcedFrom
sourceVersionNumber : long
templateElement
0..1
*
resultingElement
«entity»
Referenceable
«classification»
Template
name : string
description : string
placeholderProperties : map<string, string>
replacementProperties : map<string, string>
additionalProperties : map<string, string>
«classification»
TemplateSubstitute

«entity»
OpenMetadataRoot
*
implementationTypes
templatesForCataloguing
*
«relationship»
CatalogTemplate
«entity»
OpenMetadataRoot
 \ No newline at end of file +
0011 - Managing Referenceables
«classification»
LatestChange
changeTarget : LatestChangeTarget
changeAction : LatestChangeAction
classificationName : string
attachmentGUID : string
attachmentType : string
relationshipType : string
user : string
description : string
«enumeration»
LatestChangeTarget
ENTITY_STATUS = 0
ENTITY_PROPERTY = 1
ENTITY_CLASSIFICATION = 2
ENTITY_RELATIONSHIP = 3
ATTACHMENT = 4
ATTACHMENT_STATUS = 5
ATTACHMENT_PROPERTY = 6
ATTACHMENT_CLASSIFICATION = 7
ATTACHMENT_RELATIONSHIP = 8
OTHER = 99
«enumeration»
LatestChangeAction
CREATED = 0
UPDATED = 1
DELETED = 2
OTHER = 99
«relationship»
SourcedFrom
sourceVersionNumber : long
templateElement
0..1
*
resultingElement
«entity»
Referenceable
«classification»
Template
name : string
description : string
versionIdentifier : string
placeholderProperties : map<string, string>
replacementProperties : map<string, string>
additionalProperties : map<string, string>
«classification»
TemplateSubstitute

«entity»
OpenMetadataRoot
*
implementationTypes
templatesForCataloguing
*
«relationship»
CatalogTemplate
«entity»
OpenMetadataRoot
 \ No newline at end of file diff --git a/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio b/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio index 4d78c3b472..ad7b99c4d0 100644 --- a/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio +++ b/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio @@ -1,6 +1,6 @@ - + - + @@ -443,7 +443,7 @@ - + @@ -510,8 +510,8 @@ - - + + diff --git a/site/mkdocs.yml b/site/mkdocs.yml index b36a6542e6..8b20fe4575 100644 --- a/site/mkdocs.yml +++ b/site/mkdocs.yml @@ -3,7 +3,7 @@ site_name: Egeria site_description: Open metadata and governance for enterprises - automatically capturing, managing and exchanging metadata between tools and platforms, no matter the vendor -copyright: "License: CC BY 4.0, © 2018-2023 contributors to the Egeria project." +copyright: "License: CC BY 4.0, © 2018-2024 contributors to the Egeria project." repo_name: odpi/egeria repo_url: https://github.com/odpi/egeria site_url: https://egeria-project.org @@ -40,66 +40,74 @@ nav: - Catalog Integration: guides/integration/types-of-metadata-repository.md - Connector Catalog: - Connector Overview: connectors/index.md - - Audit Log Destination: - - Console: connectors/runtime/console-audit-log-destination-connector.md - - Event Topic: connectors/runtime/event-topic-audit-log-destination-connector.md - - Cohort Registry Store: - - JSON File: connectors/runtime/file-cohort-registry-store-connector.md - - Digital Resource: - - Open Metadata Topic: - - Kafka Open Metadata Topic Connector: connectors/resource/kafka-open-metadata-topic-connector.md - - In Memory Open Metadata Topic Connector: connectors/resource/in-mem-open-metadata-topic-connector.md - - JDBC Connector: connectors/resource/jdbc-resource-connector.md - - Basic File Connector: connectors/resource/basic-file-resource-connector.md - - CSV File Connector: connectors/resource/csv-file-resource-connector.md - - Basic Folder Connector: connectors/resource/basic-folder-resource-connector.md - - Data Folder Connector: connectors/resource/data-folder-resource-connector.md - - System: - - Apache Atlas REST Connector: connectors/system/apache-atlas-rest-connector.md - - Discovery Services: - - Apache Atlas Discovery Service: connectors/discovery/apache-atlas-discovery-service.md + - Secrets Store: + - Environment Variables: connectors/secrets/environment-variable-secrets-store-connector.md + - Files: + - Any File: + - Read File Contents: connectors/resource/basic-file-resource-connector.md + - File Folders (directories): + - Read Files: connectors/resource/basic-folder-resource-connector.md + - Catalog Files: connectors/integration/data-files-monitor-integration-connector.md + - CSV Files: + - Read File Contents: connectors/resource/csv-file-resource-connector.md + - Data Folders: + - Read files: connectors/resource/data-folder-resource-connector.md + - Monitor files: connectors/integration/data-folder-monitor-integration-connector.md + - Open Metadata Archive Files: + - Read OMArchive Files: connectors/runtime/file-based-open-metadata-archive-store-runtime-connector.md + - Relational Databases: + - JDBC DataSource: connectors/resource/jdbc-resource-connector.md + - Catalog JDBC Database: connectors/integration/jdbc-integration-connector.md + - Apache Kafka: + - JSON Plain Text Events: connectors/resource/kafka-open-metadata-topic-connector.md + - Sample Lineage Event Receiver: connectors/integration/sample-lineage-event-receiver-integration-connector.md + - Catalog Kafka Topics: connectors/integration/kafka-monitor-integration-connector.md + - Audit Kafka Topics: connectors/integration/kafka-topics-audit-integration-connector.md + - Strimzi Topics: + - Catalog Topics: connectors/integration/strimzi-monitor-integration-connector.md + - Apache Atlas Server: + - Call API: connectors/apache-atlas/apache-atlas-rest-connector.md + - Survey: connectors/apache-atlas/apache-atlas-survey-action-service.md + - Catalog: connectors/integration/apache-atlas-catalog-integration-connector.md - Governance Actions: - - Generic Element Watchdog: connectors/governance-action/generic-element-watchdog-governance-action-service.md - - Generic Folder Watchdog: connectors/governance-action/generic-folder-watchdog-governance-action-service.md - - Move/Copy File Provisioning: connectors/governance-action/move-copy-file-provisioning-governance-action-service.md - - Origin Seeker Remediation: connectors/governance-action/origin-seeker-remediation-governance-action-service.md - - Integration (metadata exchange): - - Apache Atlas: connectors/integration/apache-atlas-catalog-integration-connector.md - - Data Files: connectors/integration/data-files-monitor-integration-connector.md - - Data Folders: connectors/integration/data-folder-monitor-integration-connector.md - - Databases (JDBC): connectors/integration/jdbc-integration-connector.md - - Kafka Topics (Cataloguing): connectors/integration/kafka-monitor-integration-connector.md - - Kafka Topics (Auditing): connectors/integration/kafka-topics-audit-integration-connector.md - - Strimzi Topics: connectors/integration/strimzi-monitor-integration-connector.md - - Open APIs via Swagger: connectors/integration/open-api-monitor-integration-connector.md - - Egeria Infrastructure: connectors/integration/egeria-infrastructure-cataloguer-integration-connector.md - - OpenLineage standard: - - OpenLineage Event Receiver: connectors/integration/open-lineage-event-receiver-integration-connector.md - - OpenLineage API Log Store: connectors/integration/api-based-open-lineage-log-store-integration-connector.md - - OpenLineage File Log Store: connectors/integration/file-based-open-lineage-log-store-integration-connector.md - - OpenLineage Cataloguer: connectors/integration/open-lineage-cataloguer-integration-connector.md - - Governance Action OpenLineage: connectors/integration/governance-action-open-lineage-integration-connector.md - - Sample Lineage Event Receiver: connectors/integration/sample-lineage-event-receiver-integration-connector.md - - Open Metadata Archive Store: - - JSON File: connectors/runtime/file-based-open-metadata-archive-store-runtime-connector.md - - Repository: - - Native Repositories: - - XTDB: + - Generic Element Watchdog: connectors/governance-action/generic-element-watchdog-governance-action-service.md + - Generic Folder Watchdog: connectors/governance-action/generic-folder-watchdog-governance-action-service.md + - Move/Copy File Provisioning: connectors/governance-action/move-copy-file-provisioning-governance-action-service.md + - Origin Seeker Remediation: connectors/governance-action/origin-seeker-remediation-governance-action-service.md + - Open APIs: + - Catalog APIs via Swagger: connectors/integration/open-api-monitor-integration-connector.md + - Egeria Infrastructure: + - OMAG Server Platform: + - Catalog Servers: connectors/integration/egeria-infrastructure-cataloguer-integration-connector.md + - Open Lineage standard: + - Open Lineage Event Receiver: connectors/integration/open-lineage-event-receiver-integration-connector.md + - Open Lineage API Log Store: connectors/integration/api-based-open-lineage-log-store-integration-connector.md + - Open Lineage File Log Store: connectors/integration/file-based-open-lineage-log-store-integration-connector.md + - Open Lineage Cataloguer: connectors/integration/open-lineage-cataloguer-integration-connector.md + - Open Lineage Events from Governance Action: connectors/integration/governance-action-open-lineage-integration-connector.md + - Open Metadata Repositories: + - XTDB: - XTDB OMRS Repository Connector: connectors/repository/xtdb/index.md - Upgrading the XTDB Connector: connectors/repository/xtdb/upgrade.md - XTDB Connector Performance: connectors/repository/xtdb/performance.md - - JanusGraph: connectors/repository/janus-graph/overview.md - - In-memory: connectors/repository/in-memory/overview.md - - Read-only: connectors/repository/read-only/overview.md - - Repository Adapters: - - Apache Atlas: connectors/repository/apache-atlas/overview.md - - IBM Information Governance Catalog (IGC): connectors/repository/igc/overview.md - - SAS Viya: connectors/repository/sas-viya/overview.md - - Hive Metastore (HMS): connectors/repository/hms/overview.md - - Sample File OMRS Proxy Repository Connector: connectors/repository/file-sample-omrs-connector/overview.md - - Repository proxy embedded repository: /connectors/repository/repository-proxy-embedded-repository - - Secrets Store: - - Environment Variables: connectors/secrets/environment-variable-secrets-store-connector.md + - JanusGraph: connectors/repository/janus-graph/overview.md + - In-memory: connectors/repository/in-memory/overview.md + - Read-only: connectors/repository/read-only/overview.md + - Open Metadata Archive Store: + - JSON File: connectors/runtime/file-based-open-metadata-archive-store-runtime-connector.md + - Audit Log Destination: + - Console: connectors/runtime/console-audit-log-destination-connector.md + - Event Topic: connectors/runtime/event-topic-audit-log-destination-connector.md + - Cohort Registry Store: + - JSON File: connectors/runtime/file-cohort-registry-store-connector.md + - Open Metadata Topic: + - Kafka Open Metadata Topic Connector: connectors/resource/kafka-open-metadata-topic-connector.md + - In Memory Open Metadata Topic Connector: connectors/resource/in-mem-open-metadata-topic-connector.md + - IBM Information Governance Catalog (IGC): connectors/repository/igc/overview.md + - SAS Viya: connectors/repository/sas-viya/overview.md + - Hive Metastore (HMS): connectors/repository/hms/overview.md + - Sample File OMRS Proxy Repository Connector: connectors/repository/file-sample-omrs-connector/overview.md + - Repository proxy embedded repository: /connectors/repository/repository-proxy-embedded-repository - Lineage Warehouse (JanusGraph): connectors/governance-daemon/lineage-warehouse-janus-connector.md - Developer Guide: - Developer Guide: guides/developer/index.md