diff --git a/.github/dependabot.yml b/.github/dependabot.yml index fbf9ccecae..e77f7a7be2 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -10,4 +10,4 @@ updates: timezone: "Etc/UTC" open-pull-requests-limit: 10 reviewers: - - planetf1 + - mandy-chessell diff --git a/CODEOWNERS b/CODEOWNERS index 6fba7ed277..19cb1a0343 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -24,21 +24,21 @@ site/docs/connectors/repository/ibm-information-server/ @cmgrote site/docs/frameworks/ @mandy-chessell site/docs/education/egeria-dojo @davidradl -site/docs/education/egeria-dojo/running-egeria/ @planetf1 +site/docs/education/egeria-dojo/running-egeria/ @mandy-chessell site/docs/education/egeria-dojo/developer/ @mandy-chessell site/docs/getting-started/ @mandy-chessell -site/docs/getting-started/hands-on-lab/ @mandy-chessell @planetf1 +site/docs/getting-started/hands-on-lab/ @mandy-chessell site/docs/guides/ @mandy-chessell -site/docs/guides/developer/ @mandy-chessell @planetf1 +site/docs/guides/developer/ @mandy-chessell site/docs/guides/cts/ @cmgrote site/docs/introduction/ @mandy-chessell -site/docs/operations/kubernetes/ @planetf1 +site/docs/operations/kubernetes/ @mandy-chessell -site/docs/release-notes/ @mandy-chessell @planetf1 +site/docs/release-notes/ @mandy-chessell site/docs/services/ @mandy-chessell @@ -60,6 +60,6 @@ site/docs/services/omvs/ @davidradl site/docs/types/ @mandy-chessell -site/docs/tutorials/docker-tutorial / @planetf1 -site/docs/tutorials/open-metadata-labs/ @planetf1 -site/docs/tutorials/postman-tutorial / @mandy-chessell @planetf1 +site/docs/tutorials/docker-tutorial/ @mandy-chessell +site/docs/tutorials/open-metadata-labs/ @mandy-chessell +site/docs/tutorials/postman-tutorial/ @mandy-chessell diff --git a/site/docs/community/index.md b/site/docs/community/index.md index adb04b3e97..30977bd4a7 100644 --- a/site/docs/community/index.md +++ b/site/docs/community/index.md @@ -25,8 +25,12 @@ The Egeria project is highly appreciative of the organizations that support and Egeria is pleased to announce integration with the following technologies: +* Apache Atlas * Apache Kafka +* Apache Hive (MetaStore) * PostgreSQL +* Oracle Database +* DB2 * Files * OpenLineage * OpenAPISpecification diff --git a/site/docs/concepts/audit-log-intro.md b/site/docs/concepts/audit-log-intro.md index ce9eb053ed..372ee2b5b9 100644 --- a/site/docs/concepts/audit-log-intro.md +++ b/site/docs/concepts/audit-log-intro.md @@ -28,5 +28,6 @@ The structure is defined by the Audit Log Framework (ALF). The Open Metadata Re | 12 | `Cohort` | The server is exchanging registration information about an open metadata repository cohort that it is connecting to. | | 13 | `Trace` | This is additional information on the operation of the server that may be of assistance in debugging a problem. It is not normally logged to any destination, but can be added when needed. | | 14 | `PerfMon` | This log record contains performance monitoring timing information for specific types of processing. It is not normally logged to any destination, but can be added when needed. | +| 15 | `Activity` | This log record describes the activity of users. It is not normally logged to any destination, but can be added when needed. | | 0 | `` | Uninitialized Severity | diff --git a/site/docs/concepts/audit-log-record-structure.svg b/site/docs/concepts/audit-log-record-structure.svg index ed54c5ebfe..5c83497427 100644 --- a/site/docs/concepts/audit-log-record-structure.svg +++ b/site/docs/concepts/audit-log-record-structure.svg @@ -1,4 +1,4 @@ - + -
guid
guid
timestamp
timestamp
originatorProperties
originatorProperties
serverName
serverName
originatorComponent
originatorComponent
componentId
componentId
actionDescription
actionDescription
threadId
threadId
threadName
threadName
severityCode
severityCode
severity
severity
messageId
messageId
messageText
messageText
messageParameters
messageParameters
addiitonalInformation
addiitonalInformation
systemAction
systemAction
userAction
userAction
exceptionClassName
exceptionClassName
exceptionMessage
exceptionMessage
exceptionStackTrace
exceptionStackTrace
serverType
serverType
localOrganizationName
localOrganizationName
componentName
componentName
componentStatus
componentStatus
componentDescription
componentDescription
componentWikiURL
componentWikiURL
metadataCollectionId
metadataCollectionIdText is not SVG - cannot display \ No newline at end of file +
guid
guid
timestamp
timestamp
originatorProperties
originatorProperties
serverName
serverName
originatorComponent
originatorComponent
componentId
componentId
actionDescription
actionDescription
threadId
threadId
threadName
threadName
severityCode
severityCode
severity
severity
messageId
messageId
messageText
messageText
messageParameters
messageParameters
additionalInformation
additionalInformation
systemAction
systemAction
userAction
userAction
exceptionClassName
exceptionClassName
exceptionMessage
exceptionMessage
exceptionStackTrace
exceptionStackTrace
serverType
serverType
localOrganizationName
localOrganizationName
componentName
componentName
componentStatus
componentStatus
componentDescription
componentDescription
componentWikiURL
componentWikiURL
metadataCollectionId
metadataCollectionIdText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/concepts/omag-server-ecosystem-cts.svg b/site/docs/concepts/omag-server-ecosystem-cts.svg new file mode 100644 index 0000000000..743e6fb25f --- /dev/null +++ b/site/docs/concepts/omag-server-ecosystem-cts.svg @@ -0,0 +1,4 @@ + + + +
Metadata
Repository
Metadata...
Metadata Server /
Metadata Access Point
Metadata Server /...
Cohort
Topic(s)
Cohort...
Another
Cohort
Member
Another...
Metadata Access
Store
Metadata Access...
Conformance Test Server
Conformance Test ServerText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/concepts/omag-server-ecosystem.svg b/site/docs/concepts/omag-server-ecosystem.svg index 27a72756a0..bcc40dacb9 100644 --- a/site/docs/concepts/omag-server-ecosystem.svg +++ b/site/docs/concepts/omag-server-ecosystem.svg @@ -1,3 +1,4 @@ + -
Presentation
Server
Presentation...
Metadata
Repository
Metadata...
Metadata Server /
Metadata Access Point
Metadata Server /...
Governance
Server
Governance...
Third Party Tool
Third Party Tool
Cohort
Topic(s)
Cohort...
Third Party
UI
Third Party...
Third Party
Technology
Third Party...
Another
Cohort
Member
Another...
Third Party
Metadata Server
Third Party...
Third Party
Metadata
Repository
Third Party...
Third Party Tool
Third Party Tool
Conformance
Test Server
Conformance...
Browser
Browser
View Server
View Server
View Server
View Server
Governance
Server
Governance...
Metadata Access
Server
Metadata Access...
Repository
Proxy
Repository...
Conformance
Test Server
Conformance...Viewer does not support full SVG 1.1 \ No newline at end of file +
Presentation
Server
Presentation...
Metadata
Repository
Metadata...
Metadata Server /
Metadata Access Point
Metadata Server /...
Governance
Server
Governance...
Third Party Tool
Third Party Tool
Cohort
Topic(s)
Cohort...
Third Party
UI
Third Party...
Third Party
Technology
Third Party...
Another
Cohort
Member
Another...
Third Party
Metadata Server
Third Party...
Third Party
Metadata
Repository
Third Party...
Third Party Tool
Third Party Tool
Browser
Browser
View Server
View Server
View Server
View Server
Governance
Server
Governance...
Metadata Access
Server
Metadata Access...
Repository
Proxy
Repository...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/concepts/omag-server.md b/site/docs/concepts/omag-server.md index 882ece8d4d..0e53043091 100644 --- a/site/docs/concepts/omag-server.md +++ b/site/docs/concepts/omag-server.md @@ -27,6 +27,7 @@ Most APIs in Egeria require both a [platform URL root](#platform-url-root) and a ## Types of OMAG Server +The function needed to operate the [open metadata ecosystem](/concepts/open-metadata-ecosystem) are organized into different types of [services](/services) that are configured in their own type of OMAG Server. This makes it easier to explain how to set them up and connect them together. The types of OMAG Server are shown in Figure 1. The hierarchy groups similar types of server together. ![Figure 1](types-of-omag-servers.svg) @@ -38,24 +39,34 @@ The types of OMAG Server are shown in Figure 1. The hierarchy groups similar typ Object-oriented software engineers would know of this type of relationship as behavior inheritance. -- [Cohort member](cohort-member) - able to exchange metadata through an open metadata repository cohort - - [Metadata access store](/concepts/metadata-access-store) - supports a metadata repository that can natively store open metadata types as well as specialized metadata APIs for different types of tools (these APIs are called [access services](/services/omas)). - - [Metadata access point](/concepts/metadata-access-point) - supports the access services like the metadata server but does not have a repository. All metadata it serves up and stores belongs to the metadata repositories in other members of the cohort. - - [Repository proxy](/concepts/repository-proxy) - acts as an open metadata translator for a third party metadata repository. It supports open metadata API calls and translates them to the proprietary APIs of the repository. It also translates events from the proprietary repository into open metadata events that flow over the cohort. - - [Conformance test server](/concepts/conformance-test-server) - validates that a member of the cohort is conforming with the open metadata protocols. This server is typically only seen in development and test cohorts rather than production. -- [View server](/concepts/view-server) - manages specialist services for user interfaces. +- [Cohort Member](cohort-member) - All of these types of servers are optionally able to exchange metadata through an open metadata repository cohort. This is a peer-to-peer metadata exchange for metadata repositories. + - [Metadata Access Store](/concepts/metadata-access-store) - supports a metadata repository that can natively store open metadata along with specialized metadata APIs for different types of tools (these APIs are called [Open Metadata Access Services (OMASs)](/services/omas)). + - [Metadata Access Point](/concepts/metadata-access-point) - supports the OMASs like the metadata access store, but but does not have its own repository. All the metadata it serves up and stores belongs to the metadata repositories in other members of the cohort. This type of server needs to be connected to at least one cohort to be useful. + - [Repository Proxy](/concepts/repository-proxy) - acts as an open metadata translator for a third party metadata repository. It supports open metadata API calls and translates them to the proprietary APIs of the repository. It also translates events from the proprietary repository into open metadata events that flow over the cohort. + - [Conformance Test Server](/concepts/conformance-test-server) - validates that a member of the cohort is conforming with the open metadata protocols. This server is typically only seen in development and test cohorts rather than production. +- [View Server](/concepts/view-server) - manages specialist services for user interfaces. - [Governance server](/concepts/governance-server) - supports the use of metadata in the broader IT landscape. - - [Engine host](/concepts/engine-host) - provides a runtime for a specific type of [governance engine](/services/omes). - - [Integration daemon](/concepts/integration-daemon) - manages the synchronization with third party technology that can not call the access services directly through the [integration services](/services/omis). - - [Data engine proxy](/concepts/data-engine-proxy) - supports the capture of metadata from a data engine. This includes details of the processing of data that it is doing which is valuable when piecing together lineage. - - [Open lineage server](/concepts/open-lineage-server) - Manages the collation of lineage information am maintains it in a format for reporting. This includes the state of the lineage at different points in time. + - [Engine Host](/concepts/engine-host) - provides a runtime for a specific type of [governance engine](/concepts/governance-engine) that runs [governance services](/concepts/governance-service) to monitor, enhance and govern the open metadata ecosystem. + - [Integration Daemon](/concepts/integration-daemon) - manages the synchronization of metadata with third party technology that can not call the access services directly. through the [integration services](/services/omis). + - [Data Engine Proxy](/concepts/data-engine-proxy) - supports the capture of metadata from a data engine. This includes details of the processing of data that it is doing which is valuable when piecing together lineage. + - [Open Lineage Server](/concepts/open-lineage-server) - Manages the collation of lineage information and maintains it in a format for reporting. This includes the state of the lineage at different points in time. + +## Server connectivity + +The different types of OMAG Servers connect together as illustrated in Figure 2. Metadata Access Stores and Metadata Access Points (collectively called [Metadata Access Servers](/concepts/metadata-access-server)) provide access to open metadata to the Governance Servers and View Servers via their [Open Metadata Access Services (OMASs)](/services/omas). +The Governance Servers connect to external tools, engines and platforms. The View Servers are called by user interfaces (including Egeria's user interfaces). +Multiple Metadata Access Servers can share metadata with Repository Proxies via one or more cohorts. -## Inter-connectivity -The different types of OMAG Servers connect together as illustrated in Figure 2. There is an inner ring of cohort members communicating via the cohort. Each cohort member is sharing the metadata they receive with the governance servers and view servers that connect to it. The governance servers connect out to external tools, engines and platforms. ![Figure 2](omag-server-ecosystem.svg) -> **Figure 2:** The inter-connectivity between OMAG Servers +> **Figure 2:** The connectivity between OMAG Servers + +The repository for a Metadata Access Store may be connected to a conformance test server via a cohort to test that its implementation is conformant with the open metadata standards. It is shown in figure 3. This only happens during test and evaluation. + +![Figure 3](omag-server-ecosystem-cts.svg) +> **Figure 3:** Conformance testing + !!! education "Further information" diff --git a/site/docs/concepts/open-metadata-ecosystem.md b/site/docs/concepts/open-metadata-ecosystem.md new file mode 100644 index 0000000000..c751621cb2 --- /dev/null +++ b/site/docs/concepts/open-metadata-ecosystem.md @@ -0,0 +1,26 @@ +--- +hide: +- toc +--- + + + + +# The open metadata ecosystem + +The *open metadata ecosystem* is the collection of deployed technology that is connected and exchanging metadata in the open metadata formats defined by Egeria. This ecosystem includes the Egeria technology such as the [OMAG Server Platform](/concepts/omag_server_platform) plus other technologies that have adopted the standards and frameworks defined by Egeria. + +The picture below is an illustration of an open metadata ecosystem. The green clouds represent the different data centres and processing locations used by the organization. They may be cloud services or on premises. + +At each processing location there typically at least one [OMAG Server Platform (blue boxes)](/concepts/omag_server_platform) deployed that hosts [OMAG Servers](/concepts/omag_server) (orange circles) that provide the synchronization services tailored to specific types of tools (grey boxes). The OMAG Servers themselves host [connectors](/concepts/connector) that each interface with a particular third party technology. Egeria's [frameworks](/frameworks) define the interfaces of the connectors and how they plug into the OMAG Servers. + +![Distributed operation](/introduction/egeria-distributed-operation.svg) + + +!!! education "Further information" + + - [Overview of Egeria](/introduction/overview) + - [Open Metadata Frameworks](/frameworks) + - [Open metadata types](/types) + +--8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/concepts/repository-proxy-atlas.svg b/site/docs/concepts/repository-proxy-atlas.svg new file mode 100644 index 0000000000..8e362abf18 --- /dev/null +++ b/site/docs/concepts/repository-proxy-atlas.svg @@ -0,0 +1,4 @@ + + + +
Another
Cohort
Member
Another...
Cohort
Topic(s)
Cohort...
Repository
Proxy
Repository...
Repository
Proxy
Repository...
import
importText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/concepts/repository-proxy.md b/site/docs/concepts/repository-proxy.md index 0b2d65b9a4..ffde099537 100644 --- a/site/docs/concepts/repository-proxy.md +++ b/site/docs/concepts/repository-proxy.md @@ -17,4 +17,9 @@ The repository proxy hosts a repository connector to wrap the third party metada - [Repository connectors for Apache Atlas :material-dock-window:](https://github.com/odpi/egeria-connector-hadoop-ecosystem){ target=atlas } - [Repository connectors for IBM Information Governance Catalog (IGC) :material-dock-window:](https://github.com/odpi/egeria-connector-ibm-information-server){ target=infosvr } +The value of a repository proxy is that it provides access to the metadata that is being collected by the third party metadata repository. For example, in the picture below, Apache Atlas is running in a Hadoop environment and cataloguing the data it contains. Through the repository proxy, this metadata is then visible to all of the other users connected via the [cohort](/features/cohort-operation/overview). + +![Connecting to Apache Atlas](repository-proxy-atlas.svg) + + --8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/repository-proxy.svg b/site/docs/concepts/repository-proxy.svg index 8e362abf18..b248425d8c 100644 --- a/site/docs/concepts/repository-proxy.svg +++ b/site/docs/concepts/repository-proxy.svg @@ -1,4 +1,4 @@ - + -
Another
Cohort
Member
Another...
Cohort
Topic(s)
Cohort...
Repository
Proxy
Repository...
Repository
Proxy
Repository...
import
importText is not SVG - cannot display \ No newline at end of file +
Another
Cohort
Member
Another...
Cohort
Topic(s)
Cohort...
Third Party
UI
Third Party...
Repository
Proxy
Repository...
Third Party
Metadata Server
Third Party...
Third Party
Metadata
Repository
Third Party...
Repository
Proxy
Repository...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/connectors/Romania 2023 .drawio b/site/docs/connectors/Romania 2023 .drawio deleted file mode 100644 index 3d2080d336..0000000000 --- a/site/docs/connectors/Romania 2023 .drawio +++ /dev/null @@ -1,13 +0,0 @@ - - - - - - - - - - - - - diff --git a/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md b/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md index 7785ffc187..c65e63e557 100644 --- a/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md +++ b/site/docs/connectors/integration/apache-atlas-catalog-integration-connector.md @@ -5,7 +5,7 @@ # Apache Atlas Integration Connector -!!! info "Connector details" +???+ info "Connector summary" - Connector Category: [Integration Connector](/concepts/integration-connector) - Hosting Service: [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) - Hosting Server: [Integration Daemon](/concepts/integration-daemon) @@ -14,20 +14,189 @@ ## Overview -The Apache Atlas integration connector publishes ACTIVE glossary terms to Apache Atlas. It can be configured to publish just from a single glossary or all glossaries. The connector makes no attempt to detect changes to these glossary terms in Apache Atlas. However, such changes will be overridden the next time the integration connector refreshes the glossary metadata in Apache Atlas. +[Apache Atlas](https://atlas.apache.org) is a metadata catalog originally designed for the Hadoop ecosystem. It offers integration services called Hooks and Bridges to capture the schemas and data sets of data platforms such as [Apache Hive](https://hive.apache.org/), [Apache HBase](https://hbase.apache.org/) and [Apache Hadoop Distributed File System (HDFS)](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html) along with the processes for creating and maintaining data sets on these platforms. The metadata descriptions of these data sets and processes are linked together using lineage relationships, allowing an understanding of how data is flowing through a Hadoop deployment. Apache Atlas also supports glossaries and a tagging system that can be used both in searches and to control access to data through Apache Ranger (using the TagSync integration). + +In recent years, Apache Atlas has been embedded in popular data catalogs such as [Microsoft Purview](https://azure.microsoft.com/en-gb/products/purview/) and [Atlan](https://atlan.com/) increasing the interest in being able to integrate with this metadata catalog. + +Reasons for integrating Apache Atlas into a wider open metadata ecosystem include: + +* To push glossaries and classification definitions to Apache Atlas so its users get to use the same definitions for the governance of their data as other parts of the organization. This may simply to help start up a new Apache Atlas deployment, or as part of a broader consistency initiative. +* To share glossaries and classification definitions that have been developed by an Apache Atlas user community in order to seed other projects and tools. +* To share details of the data sets that Apache Atlas has catalogued with other catalogs and tools. For example, the Apache Atlas catalog of data sets may be shared with data science or stewardship tools. Tags, classifications and semantic assignments made to these assets by users of these tools could then be pushed back to Apache Atlas to enhance the use and governance of their associated data through Apache Atlas. +* To extract lineage information for data flowing through the data platforms and processes that Apache Atlas is capturing. + +## Integration Connector function + +The Apache Atlas integration connector provides an ongoing exchange of metadata between Apache Atlas and the open metadata ecosystem. In its initial state, it focuses on the two-way exchange of glossaries. Future versions will add entities (such as data sets and process), lineage and classifications. ![Figure 1](apache-atlas-catalog-integration-connector.svg) > **Figure 1:** Operation of the Apache Atlas integration connector +Specifically this connector: + +* Publishes glossaries, glossary categories and ACTIVE glossary terms that originate from the open metadata ecosystem into Apache Atlas. The connector makes no attempt to detect changes to these glossary elements in Apache Atlas. Additionally, such changes will be overridden the next time the integration connector refreshes the glossary metadata in Apache Atlas. +* Publishes glossaries, glossary categories and glossary terms that originate in Apache Atlas into the open metadata ecosystem. These elements will be read-only in the open metadata ecosystem. + + +### Metadata ownership + +The concept of ownership of each metadata instance (such as glossary, glossary catalog and glossary term) is strictly maintained. Simply: + +* If the instance was originally created in the open metadata ecosystem, it is owned by the open metadata ecosystem and any changes or deletions made to the copy in Apache Atlas will be reversed the next time the connector refreshes the Apache Atlas metadata. +* If the instance was originally created in Apache Atlas, it is owned by Apache Atlas. When the instance is copied into the open metadata ecosystem, its [provenance](/features/metadata-provenance/overview) is set to indicate that this Apache Atlas server is its origin, which effectively makes it read-only in the open metadata ecosystem. + +It is possible to create a new glossary term or glossary category in Apache Atlas within a glossary that originated from the open metadata ecosystem. This new entity will be owned by Apache Atlas, even though it belongs to a glossary that is owned by the open metadata ecosystem. + +Similarly, it is possible to create new glossary terms and categories in the open metadata ecosystem that are linked to a glossary that originated from Apache Atlas. These new entities will be owned by the open metadata ecosystem. + + + +### Correlation + +In order to correlate the instances in Apache Atlas with the open metadata ecosystem instance, the connector: + +* Adds two properties to the Apache Atlas glossary instances' *additionalAttributes* attribute: + + * *egeriaGUID* provides the unique identifier of the equivalent instance in the open metadata ecosystem. + * *egeriaOwned* is a boolean flag describing whether the instance is owned by the open metadata ecosystem (true) or Apache Atlas (false). + + ??? example "Example of an Apache Atlas glossary category" + This is an example of an Apache Atlas glossary category that has been copied from open metadata instance `805ba420-6fd5-446f-9d49-e9c7c97c4df4`. + ```json + { + "guid": "50d104bd-af42-40a1-929d-4c4eeaae10d5", + "qualifiedName": "Model Vocabulary: Cloud Information Model (CIM)@Cloud Information Model (CIM)", + "name": "Model Vocabulary: Cloud Information Model (CIM)", + "shortDescription": "The Cloud Information Model (CIM) is an open source glossary and data model that spans the following subject areas:\n\n•\tParty – people, their roles and organizations.\n•\tProduct – product descriptions, structures and packaging.\n•\tSales Order – customer orders for goods and services.\n•\tPayment Method – payment methods including cards, coupons and digital wallets.\n•\tPayment – individual payments for goods and services.\n•\tShipment – shipment of goods and services to the customer to fulfil an order.\n\nThis means it can provide common data structures for new services spanning customer and employee interaction around typical commercial activities such as buying and selling of goods and services.\n\nThe motivation behind the cloud information model is to support organizations who are transforming their digital services to run on a variety of cloud platforms and with their own data centres", + "longDescription": "The Cloud Information Model (CIM) is an open source glossary and data model that spans the following subject areas:\n\n•\tParty – people, their roles and organizations.\n•\tProduct – product descriptions, structures and packaging.\n•\tSales Order – customer orders for goods and services.\n•\tPayment Method – payment methods including cards, coupons and digital wallets.\n•\tPayment – individual payments for goods and services.\n•\tShipment – shipment of goods and services to the customer to fulfil an order.\n\nThis means it can provide common data structures for new services spanning customer and employee interaction around typical commercial activities such as buying and selling of goods and services.\n\nThe motivation behind the cloud information model is to support organizations who are transforming their digital services to run on a variety of cloud platforms and with their own data centres. Often, they are dealing with systems built on many different generations of technology, with data distributed amongst them. The CIM provides a common language to describe the different types of data.\n\nThe CIM has been created to simplify the growing complexity companies experience when integrating data across different systemsin order to deliver highly intelligent and personalized customer engagements. It standardizes data interoperability by creating a set of guidelines to easily connect systems such as a point of sale system, email marketing platform, customer service center, customer relationship management (CRM) system and more. Developers will no longer need to spend months creating custom code in order to deliver innovative customer experiences. The CIM can be easily adopted and extended within days so that developers can focus on creating new innovations faster that deliver a truly connected, personalized and intelligent customer experience. \n", + "additionalAttributes": { + "egeriaGUID": "805ba420-6fd5-446f-9d49-e9c7c97c4df4", + "egeriaOwned": "true" + }, + "anchor": { + "glossaryGuid": "991ea530-fba1-49cf-b58b-24439a3101ad", + "relationGuid": "a0096490-5488-4daa-b23c-d675c56cc37b" + } + } + ``` + +* Uses an [external identifier](/features/external-identifiers/overview) attached to the open metadata instance to hold the unique identifier (GUID) of the equivalent Apache Atlas metadata instance as well as using [metadata provenance](/features/metadata-provenance/overview) to identify the origin of instances being synchronized. The external identifier is returned in the + + * *correlationProperties* returns the external identifier. + * *origin* of the *elementHeader* shows the metadata provenance. + + ??? example "Example of an open metadata glossary" + This is an example of an open metadata glossary that has been copied from Apache Atlas's glossary with unique identifier `d36c7844-1dbc-4f15-bfd2-63a6125e9c4b`. + ```json + { + "elementHeader": { + "class": "ElementHeader", + "headerVersion": 0, + "status": "ACTIVE", + "type": { + "typeId": "36f66863-9726-4b41-97ee-714fd0dc6fe4", + "typeName": "Glossary", + "superTypeNames": [ "Referenceable", "OpenMetadataRoot"], + "typeVersion": 1, + "typeDescription": "A collection of related glossary terms." + }, + "origin": { + "sourceServer": "cocoMDS1", + "originCategory": "EXTERNAL_SOURCE", + "homeMetadataCollectionId": "1cb8fa85-a1ba-480b-9d5e-5e7d3c562603", + "homeMetadataCollectionName": "ApacheAtlasDL01" + }, + "versions": { + "createdBy": "generalnpa", + "createTime": "2023-07-19T17:09:36.767+00:00", + "version": 1 + }, + "guid": "e69cabed-5807-4467-a7e0-6a55acc15938", + "classifications": [ + { + "class": "ElementClassification", + "headerVersion": 0, + "status": "ACTIVE", + "type": { + "typeId": "adce83ac-10f1-4279-8a35-346976e94466", + "typeName": "LatestChange", + "typeVersion": 1, + "typeDescription": "Defines the latest change to an anchor entity and its associated attachments." + }, + "origin": { + "sourceServer": "cocoMDS1", + "originCategory": "LOCAL_COHORT", + "homeMetadataCollectionId": "1cefccc2-082b-4c13-aafd-d1f0a67df3fc", + "homeMetadataCollectionName": "Data Lake Catalog" + }, + "versions": { + "createdBy": "cocoMDS1npa", + "createTime": "2023-07-19T17:09:37.305+00:00", + "version": 1 + }, + "classificationOrigin": "ASSIGNED", + "classificationName": "LatestChange", + "classificationProperties": { + "relationshipType": "ExternalIdLink", + "attachmentType": "ExternalId", + "changeTarget": "EntityRelationship", + "description": "Linking Glossary e69cabed-5807-4467-a7e0-6a55acc15938 to ExternalId 565396f8-4300-4f30-b55b-a6c1f2a6070f", + "attachmentGUID": "565396f8-4300-4f30-b55b-a6c1f2a6070f", + "user": "generalnpa", + "changeAction": "Created" + } + } + ] + }, + "correlationHeaders": [ + { + "synchronizationDirection": "BOTH_DIRECTIONS", + "externalIdentifier": "d36c7844-1dbc-4f15-bfd2-63a6125e9c4b", + "externalIdentifierName": "atlasGUID", + "keyPattern": "LOCAL_KEY", + "assetManagerGUID": "1cb8fa85-a1ba-480b-9d5e-5e7d3c562603", + "assetManagerName": "ApacheAtlasDL01", + "lastSynchronized": "2023-07-19T17:09:37.045+00:00" + } + ], + "glossaryProperties": { + "class": "GlossaryProperties", + "qualifiedName": "AtlasGlossary.Atlas Metadata Glossary", + "displayName": "Atlas Metadata Glossary", + "description": "This glossary has come from Apache Atlas.\n\nThis glossary is to replicated into the open metadata ecosystem by Egeria.", + "language": "English", + "usage": "Just for test" + } + } + ``` ## Configuration -This connector uses the [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) running in the [Integration Daemon](/concepts/integration-daemon). +The Apache Atlas integration connector runs in an [Integration Daemon](/concepts/integration-daemon). There are two parts to the configuration: + +* The specification of the connection object the controls the behaviour of the connector itself. +* The inclusion of the connection object in the configuration of the Integration Daemon. + +### The connection object + +The [connection](/concepts/connection) object provides information about configurations properties and the endpoint where Apache Atlas is running. A connection object has the following structure: + +![Connection structure](/concepts/connection.svg) + +The *Connection* defines the specific use of the connector. The *Connector Type* identifies the implementation of connector to run and the *Endpoint* identifies the host and port where the Apache Atlas server is running. + +Apache Atlas typically runs with *Basic Authorization* - that is a userId and password is passed in the header of REST API calls to Apache Atlas. This userId and password are specified as *userId* and *clearPassword* properties in the connection object. + +??? tip "Using secrets for Apache Atlas's userId and password" + If you are uncomfortable with specifying the userId and password for Apache Atlas in the clear it is possible to embed a connection for a [Secrets Store Connector](/concepts/secrets-store-connector) in the connection for the Apache Atlas integration connector. The secrets store connector will dynamically retrieve the userId and password when the connector starts up. -This is its connection definition to use on the [administration commands that configure the Catalog Integrator OMIS](/guides/admin/configuring-an-integration-daemon/#configure-the-integration-services) if you want all glossaries published to Apache Atlas. +Finally, there is an optional set of configuration properties that controls the behaviour of the Apache Atlas integration connector: -!!! example "Connection configuration for all glossaries" - ```json linenums="1" hl_lines="14" +* *egeriaGlossaryQualifiedName* specifies the qualified name of a single Egeria glossary to publish to Apache Atlas. If this property is omitted, all glossaries found in the open metadata ecosystem will be copied into Apache Atlas. +* *atlasGlossaryName* specifies the name of a single Apache Atlas glossary that is to be synchronized with the open metadata ecosystem. If this property is omitted, all glossaries found in Apache Atlas will be copied into the open metadata ecosystem. + +!!! example "Example connection for the Apache Atlas integration connector" + ```json linenums="1" hl_lines="14 16-17 20-21" { "connection" : { @@ -42,40 +211,99 @@ This is its connection definition to use on the [administration commands that co { "class" : "Endpoint", "address" : "{{atlasServerURL}}" + }, + "userId" : "{{atlasUserId}}", + "clearPassword" : "{{atlasPassword}}", + "configurationProperties" : + { + "egeriaGlossaryQualifiedName" : "{{glossaryQualifiedName}}", + "atlasGlossaryName" : "{{glossaryName}}" } } } ``` - Replace `{{atlasServerURL}}` with the URL of the Apache Atlas server. + - Replace `{{atlasUserId}}` with the userId passed on each REST API call to the Apache Atlas server. + - Replace `{{atlasPassword}}` with the password passed on each REST API call to the Apache Atlas server. + - Replace `{{glossaryQualifiedName}}` with the qualified name of the Egeria glossary to publish to Apache Atlas. + - Replace `{{glossaryName}}` this with the name of a single Apache Atlas glossary that is to be synchronized with the open metadata ecosystem. + +### Adding the connection to Egeria's configuration + +This connector runs in the [Integration Daemon](/concepts/integration-daemon) and uses the [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) interface to access the open metadata ecosystem. + +Once you have the connection specified for the Apache Atlas integration connector, you have two choices on how to add it to Egeria's configuration: + +* *Static configuration* by adding it directly to the [configuration document](/concepts/configuration-document) for the Integration Daemon. This creates a static configuration that requires a restart of the Integration Daemon to change the configuration. +* *Dynamic configuration* by adding it to an open metadata [integration group](concepts/integration-group). The integration group is stored in the open metadata repository of a [Metadata Access Store](/concepts/metadata-access-store). The associate Integration Daemon(s) immediately adapts what they are running as changes are made to the integration group contents. +Whichever method is used, there are four properties that will affect the behaviour of your connector: -!!! example "Connection configuration for a selected glossary" - ```json linenums="1" hl_lines="14 18" +* *metadataSourceQualifiedName* provides a unique name for your Apache Atlas server instance. This value is used to identify the origin of the metadata that comes from your Apache Atlas server instance. If you were running multiple Apache Atlas servers, each one should have its own *metadataSourceQualifiedName*. +* *refreshTimeInterval* defines the number of minutes between each scan the connector makes to find new metadata or changes to metadata. The value to use depends on how often the metadata is likely to change. It could, for instance be set to 1440 for the connector to run a refresh scan once a day. +* *usesBlockingCalls* must be set to *false*. +* *permittedSynchronization* defines which direction metadata is to flow. It may be: + + * *FROM_THIRD_PARTY* meaning that metadata only flows from Apache Atlas to the open metadata ecosystem. + * *TO_THIRD_PARTY* meaning that metadata only flows from the open metadata ecosystem to Apache Atlas. + * *BOTH_DIRECTIONS* meaning the metadata flows between the open metadata ecosystem and Apache Atlas in both directions. + +??? education "Configuring the Integration Daemon" + See [Configuring the integration connectors](/guides/admin/servers/configuring-an-integration-daemon/#configuring-the-integration-connectors) to find out how to add you connection object to the integration daemon. For example, issuing the following command adds the Apache Atlas connection information to the Catalog Integrator OMIS in an Integration Daemon identified by `{integrationDaemonName}}`. + + ``` + POST {{baseURL}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{integrationDaemonName}}/integration-services/catalog-integrator + ``` + with request body: + ```json { - "connection" : - { - "class" : "Connection", - "qualifiedName" : "Egeria:IntegrationConnector:Catalog:Apache Atlas Connection", - "connectorType" : + "class": "IntegrationServiceRequestBody", + "omagserverPlatformRootURL": "http://54798.cocopharmaceuticals.org:9443", + "omagserverName": "cocoMDS1", + "integrationConnectorConfigs": [ { - "class" : "ConnectorType", - "connectorProviderClassName" : "org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasIntegrationProvider" - }, - "endpoint" : - { - "class" : "Endpoint", - "address" : "{{atlasServerURL}}" - }, - "configurationProperties" : - { - "glossaryQualifiedName" : "{{glossaryQualifiedName}}" + "class": "IntegrationConnectorConfig", + "connectorName": "AtlasIntegrator", + "connectorUserId": "generalnpa", + "connection": { + "class": "Connection", + "connectorType": { + "class": "ConnectorType", + "connectorProviderClassName": "org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasIntegrationProvider" + }, + "endpoint": { + "class": "Endpoint", + "address": "http://54798.cocopharmaceuticals.org:21000" + }, + "userId" : "admin", + "clearPassword" : "admin" + }, + "metadataSourceQualifiedName": "ApacheAtlasDL01", + "refreshTimeInterval": "1440", + "usesBlockingCalls": "false", + "permittedSynchronization": "BOTH_DIRECTIONS" } - } + ] } ``` - - Replace `{{atlasServerURL}}` with the URL of the Apache Atlas server. - - Replace `{{glossaryQualifiedName}}` with the qualified name of the Egeria glossary to publish to Apache Atlas. + +## Implementation + +The Apache Atlas integration connector makes use of the [Apache Atlas Glossary REST API](https://atlas.apache.org/api/v2/index.html) to query and update the metadata in Apache Atlas. This REST API is divided into different groups of operations that each follow their own style. + +* *DiscoveryREST* provides the ability to search for entities, relationships and classifications. +* *EntityREST* supports the maintenance of entity instances. +* *GlossaryREST* supports both the querying and maintenance of glossaries, glossary categories and glossary terms. +* *LineageREST* provides for the capture and retrieval of lineage information for an entity. +* *RelationshipREST* supports the maintenance of relationship instances. +* *TypesREST* supports the definition and retrieval of type definitions. + +This difference in style is why the Apache Atlas integration connector provides different capabilities for different types of metadata. For example: + +* Glossary terms can only be updated in their origin server because the Apache Atlas Glossary REST API does not record the version and update time to glossary element instances making it impossible to reconcile the order that updates have been made across the open metadata ecosystem and Apache Atlas. +* Entities/relationships do not have a standard place to store additional attributes, so it is not possible to store the corresponding unique identifier from the open metadata ecosystem. The consequence is that entity and relationship metadata can only flow from Apache Atlas to the open metadata ecosystem. + ---8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/integration/jdbc-integration-connector.md b/site/docs/connectors/integration/jdbc-integration-connector.md index 7e6007a30e..ec4f1194cc 100644 --- a/site/docs/connectors/integration/jdbc-integration-connector.md +++ b/site/docs/connectors/integration/jdbc-integration-connector.md @@ -9,7 +9,7 @@ - Connector Category: [Integration Connector](/concepts/integration-connector) - Hosting Service: [Database Integrator OMIS](/services/omis/database-integrator/overview) - Hosting Server: [Integration Daemon](/concepts/integration-daemon) - - Source Module: [jdbc-integration-connector :material-github:](https://github.com/odpi/egeria-database-connectors/tree/main/jdbc-integration-connector){ target=gh } + - Source Module: [jdbc-integration-connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/integration-connectors/jdbc-integration-connector){ target=gh } - Jar File Name: `jdbc-integration-connector.jar` diff --git a/site/docs/connectors/repository/janus-graph/overview.md b/site/docs/connectors/repository/janus-graph/overview.md index 5bd022d6ee..b1d3400a50 100644 --- a/site/docs/connectors/repository/janus-graph/overview.md +++ b/site/docs/connectors/repository/janus-graph/overview.md @@ -20,7 +20,7 @@ JanusGraph provides a graph library. It uses two pluggable persistence stores: When the JanusGraph connector is started, it builds an index for each primitive property in the type system as well as the standard header properties - such as guid, createTime , ... JanusGraph then maps this call down onto whatever search index technology is configured. At the same time, it is building up the graph representation as entities and relationships are stored. -## Jar files +## Egeria Jar files Egeria's build produces 2 jar files for this connector: @@ -48,7 +48,174 @@ The request body is optional and set to a map of property names to property valu This stores the graph store in the OMAG Server Platform's working directory in file called `./data/servers/{serverName}/repository/graph/berkeley` where the server name is inserted into *{serverName}*. The search index is stored in [Apache Lucene](https://lucene.apache.org/). -Details of the other supported storage backends and how to specify them are found in the [JanusGraph documentation](https://docs.janusgraph.org/storage-backend/). +Details of other supported storage backends and how to specify them are found in the [JanusGraph documentation](https://docs.janusgraph.org/storage-backend/). + +# Deployment Notes + + +Here are some usage notes based on experience configuring and testing JanusGraph using different underlying technologies. + +## Configurations + +As discussed above, there are two steps to establishing a working JanusGraph based Egeria repository. We need to send a JSON snippet to JanusGraph detailing how to communicate with the underlying technologies and we need to ensure the right Jar files are on the Java path. + +In the examples below we show the use of a separate search index - which although not strictly necessary for JanusGraph, does offer improved performance and support for many queries. + + +### Berkeley DB and Lucene + +As mentioned above, the default configuration for JanusGraph is to use BerkeleyDB and Lucene. This will persist the graph and search index files. Note that this simple configuration provides little resilience beyond what your file system provides. + +If you want to change the file locations rather than use the defaults, then you need to pass in a JSON configuration in the body of the REST call. Here is an example: + +```json +{ + "storage.backend" : "berkeleyje", + "storage.directory" : "your data directory", + + "index.search.backend" : "lucene", + "index.search.directory" : "your search directory" + +} +``` + +No additional jar files are required for this default configuration. + +### HBase and ElasticSearch + +Using HBase offers massive scalability and an opportunity to provide storage resilience - but is, however, more effort to configure - particularly if you are unfamiliar with Hadoop technology. If you already have a mature hadoop deployment then this can be a cost-effective option. A couple of additional considerations for HBase: + +- JanusGraph writes its configuration into the first row of the HBase table. So if you want to change configurations by, for example, changing a search engine parameter, then you must delete the row (or the table) in order for your change to be effective. This also means it can be expensive to change configurations once the system is in productive use. +- Some deployments of HBase can be finicky in the hostnames they accept - sometimes numerical IP addresses, sometimes symbolic IP addressed work - and you might need mappings between them in a local /etc/hosts file. +- If "storage.hbase.skip-schema-check" is true then JanusGraph assumes the HBase table already exists; if false, it will create the table automatically if it doesn't exist. +- Additional configuration options can be found at: [JanusGraph - HBase](https://docs.janusgraph.org/storage-backend/hbase/) + + +We also show the use of ElasticSearch which provides a resilient search cluster. ElasticSearch has an extensive set of options that can be configured - especially around security and authentication. Shown below are some basic settings, more robust and sophisticated approaches are available. Here are a couple additional considerations: + +- Many ElasticSearch deployments automatically set up self-signed certificates for SSL. You may need to include this certificate to successfully connect to ElasticSearch. Each node in the ElasticSearch cluster may have its own certificate. There are a couple of different ways the certificates can be injected - one approach is to add it to the Trust Store defined by the Egeria Platform [Application Properties File](https://egeria-project.org/guides/admin/configuring-the-omag-server-platform/#certificates-for-the-omag-server-platform) using utilities like *openssl* and *keytool*. +- Most modern ElasticSearch deployments use the REST interface. +- Additional configuration options can be found at: [JanusGraph - ElasticSearch](https://docs.janusgraph.org/index-backend/elasticsearch/) + + +HBase and ElasticSearch are both commonly deployed as shared services that may be used by many applications - so we must be careful to specify unique table/index names. + + +In the snippet below we also show the use of a local cache to improve performance as we are using remote persistence: + +```json +{ + + "storage.backend": "hbase", + "storage.hostname": "set of hbase zookeeper endpoints", + "storage.port": "2181", + "storage.hbase.skip-schema-check" : "false", + "storage.hbase.table": "table name for my server", + "index.search.backend" : "elasticsearch", + "index.search.hostname" : "set of ElasticSearch endpoints", + "index.search.elasticsearch.connect-timeout": "2000", + "index.search.elasticsearch.ssl.allow-self-signed-certificates": "true", + "index.search.elasticsearch.ssl.enabled": "true", + "index.search.elasticsearch.http.auth.type" : "basic", + "index.search.elasticsearch.http.auth.basic.username" : "myUser", + "index.search.elasticsearch.http.auth.basic.password" : "your password", + "index.search.elasticsearch.interface": "REST_CLIENT", + "index.search.index-name" : "index name for my server", + + "cache.db-cache": "true", + "cache.db-cache-clean-wait": "20", + "cache.db-cache-time": "180000", + "cache.db-cache-size": "0.5" +} +``` + +#### Adding needed Jar files + +The second part of configuring the JanusGraph repository connector is making sure that the necessary Jar files are on the java class path. + +* First we need to decide how we want to make the needed Jar files available on the class path. This will depend on our Egeria deployment approach. +* Second, we need to identify which files are needed - and this can sometimes be a matter of trial and error. +* Finally, we will often want to automate the acquisition and placement of the right Jar files into the right places before starting the Egeria platform so that they will be available to the connector at runtime. + +##### Adding Jar files to the class path + +Injecting needed Jar files onto the class path depends on how you are deploying Egeria. + +* Running as a standard Java application then you can easily download the Jar files and add them to the platform/lib directory under the Egeria OMAG Server working directory. +* If you are deploying with Helm charts in a Kubernetes environment then you may want to create a custom yaml file to download the Jar files during the installation process. +* If your are deploying via a Docker container then you would likely download the files when you build the container. + +To download the files (once they have been identified), you can create a script of curl commands. For example, a command to download the hadoop-hdfs-client-3.3.0 jar file from the *maven* repository would look like: + +```bash +curl -o ./hadoop-hdfs-client-3.3.0.jar https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs-client/3.3.0/hadoop-hdfs-client-3.3.0.jar +``` + +The curl command also specifies where you want to download the file to - so be careful where you run the command and how you formulate the download location. The curl command can also be issued within the docker build file. For kubernetes deployments with Helm charts, a yaml file can contain a similar set of commands of the form: + +```text + - url: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs-client/3.3.0/hadoop-hdfs-client-3.3.0.jar + filename: hadoop-hdfs-client-3.3.0.jar` +``` + +If you create a yaml file called myAddedJars.yaml containing these directives, then it can be executed as part of install command. Here is a simplified example that adds this directive to the command that installs the Egeria Lab helm chart: + +```bash +helm install -f ./myAddedJars.yaml lab egeria/odpi-egeria-lab` +``` + +A more complete discussion can be found at [Egeria Helm Lab Chart](https://egeria-project.org/guides/operations/kubernetes/charts/lab/?h=helm+lab#adding-additional-connectors) + +##### Locate the Jar files + +Discovering the right set of Jar files to support a particular connector can require a bit of trial and error. You add the Jar files you think you need, start up Egeria, configure the connector and then start it - and see if it works by observing the audit log (often the console log). You may see exceptions due to missing classes - which means you then need to find what Jar files contain the missing classes, download the Jar files, and add them to the class path. + +For example, to configure a JanusGraph repository with HBase and ElasticSearch, the following jar files were added to the class path: + + +``` text +woodstox-core-6.2.0.jar +x-pack-transport-7.15.0.jar +zookeeper-3.5.7.jar +zookeeper-jute-3.5.7.jar +stax2-api-4.2.2.jar +tinkergraph-gremlin-3.4.6.jar +transport-nio-client-7.15.0.jar +lucene-suggest-8.6.2.jar +solr-solrj-7.7.2.jar +lucene-analyzers-common-8.6.2.jar +lucene-core-8.6.2.jar +htrace-hbase-4.2.0-incubating.jar +hadoop-shaded-protobuf_3_7-1.1.0.jar +hbase-client-2.3.3.jar +hbase-hadoop2-compat-2.3.3.jar +hbase-hadoop-compat-2.3.3.jar +hadoop-common-3.3.0.jar +gremlin-shaded-3.4.6.jar +hadoop-annotations-3.3.0.jar +hadoop-auth-3.3.0.jar +hadoop-aws-3.3.0.jar +gremlin-core-3.4.6.jar +gremlin-driver-3.4.6.jar +``` + +This list doesn't include JanusGraph ElasticSearch and HBase libraries because these have already been added into the Jar file named **graph-repository-connector-jar-with-dependencies-4.2-SNAPSHOT.jar ** during the Egeria build process - we do need to make sure this key file is on the class path (by default it should be automatically installed during the build process into the *platform/lib* directory). Also note that this list may include more Jar files than strictly needed - it is easier to add more and prune later if desired. + +Finding the Jar files needed can be tedious - often consulting sites like [sonatype](https://oss.sonatype.org/#welcome) is useful. + + + + +### Other Configurations + +JanusGraph supports many other storage and search index technologies including: +- Apache Cassandra +- Google Cloud Bigtable +- ScyllaDB +- Apache Solr + +Please see the JanusGraph documentation for more details. + ## Troubleshooting diff --git a/site/docs/connectors/resource/apache-atlas-rest-connector.md b/site/docs/connectors/resource/apache-atlas-rest-connector.md new file mode 100644 index 0000000000..abe0938578 --- /dev/null +++ b/site/docs/connectors/resource/apache-atlas-rest-connector.md @@ -0,0 +1,30 @@ + + + +# Apache Atlas REST Connector + +???+ info "Connector summary" + - Connector Category: [Digital Resource Connector](/concepts/digital-resource-connector) + - Source Module: [apache-atlas-rest-connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/system-connectors/apache-atlas-rest-connector){ target=gh }. + - Jar File Name: `apache-atlas-rest-connector.jar` + - Supported Asset Type: [SoftwareServer](/types/0/0040-Software-Servers) + +Apache Atlas has a REST API that allows external callers to query and create both +types and instances. This connector provides a simple Java API to this REST API. +It is written without any dependencies on Apache Atlas (or its associated Hadoop components) +so it happily runs in the same version of Java as the rest of Egeria. + +This connector is used by other connectors from Egeria, and may also be used +by components from outside Egeria. + +The values from the connection used by this connector are: + +* Connection.getUserId() and Connection.getClearPassword() for logging in to Apache Atlas. +* Connection.getDisplayName() for the connector name in messages. +* Connection.getEndpoint().getAddress() for the URL root (typically host and port name) of the Apache Atlas server. +* Connection.getConfigurationProperties.get("atlasServerName") for the name of the Apache Atlas server to use in messages. + + + + +--8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/resource/jdbc-resource-connector.md b/site/docs/connectors/resource/jdbc-resource-connector.md index dc11e6e690..3573ebba45 100644 --- a/site/docs/connectors/resource/jdbc-resource-connector.md +++ b/site/docs/connectors/resource/jdbc-resource-connector.md @@ -5,6 +5,13 @@ # JDBC Resource Connector +???+ info "Connector summary" + - Connector Category: [Digital Resource Connector](/concepts/digital-resource-connector) + - Source Module: [jdbc resource connector :material-github:](https://github.com/odpi/egeria/tree/main/open-metadata-implementation/adapters/open-connectors/data-store-connectors/jdbc-resource-connector){ target=gh }. + - Jar File Name: `jdbc-resource-connector.jar` + - Supported Asset Type: [RelationalDatabase](/types/2/0224-Databases) + + The JDBC Resource Connector provides a DataSource, which in turn is used to get a connection to underlying database. It provides access to both the schema metadata and the business data content. ![Figure 1](jdbc-resource-connector.svg) @@ -13,8 +20,39 @@ The JDBC Resource Connector provides a DataSource, which in turn is used to get ## Usage -The connector is initialized using the connection information attached to the [Database](/types/2/0224-Databases) asset in open metadata. +The connector is initialized using the connection information attached to the [RelationalDatabase](/types/2/0224-Databases) asset in the open metadata ecosystem. ![Figure 2](jdbc-resource-connector-use.svg) > **Figure 2:** Connection information used to create an instance of the JDBC resource connector +The optional configuration properties supported by this connector are: + +* `jdbcDriverManagerClassName` - requests the named class to be loaded and registered as a driver. This property only needs to be defined if the connector is experiencing exceptions related to a missing DriverManager class for the database URL. +* `jdbcConnectionTimeout` - sets the maximum time in seconds that this data source will wait while attempting to connect to a database. The default value is 0 which means use the system default timeout, if any; otherwise it means no timeout. +* `jdbcDatabaseName` - provides a name to use in messages about the database. If it is not set then the connection URL string is used. + +Below is an example connection for a PostgreSQL database: + +```json +{ + "class": "Connection", + "connectorType": { + "class": "ConnectorType", + "connectorProviderClassName": "org.odpi.openmetadata.adapters.connectors.resource.jdbc.JDBCResourceConnectorProvider" + }, + "endpoint": { + "class": "Endpoint", + "address": "jdbc:postgresql://localhost:5432/myDatabase" + }, + "userId": "xxxxx", + "clearPassword": "xxxx", + "configurationProperties": { + "jdbcDriverManagerClassName": "org.postgresql.Driver", + "jdbcConnectionTimeout": "10", + "jdbcDatabaseName": "MyDatabase" + } +} +``` + +--8<-- "snippets/abbr.md" + diff --git a/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md b/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md index 375788a69f..d7ae23180d 100644 --- a/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md +++ b/site/docs/connectors/resource/kafka-open-metadata-topic-connector.md @@ -16,6 +16,8 @@ The Kafka Open Metadata Topic Connector implements an [Open Metadata Topic Conne ![Kafka Open Metadata Topic Connector](kafka-open-metadata-topic-connector.svg) +Egeria uses this connector as its default event notification technology (known as the [Event Bus](/concepts/event-bus)). You may also use it in your [integration connectors](/concepts/integration-connector) is they need to send or receive events from a third party technology. + ## Configuration The connection example shows how to configure the connection for this connector. It is passing properties for both the producer and consumer. @@ -54,6 +56,9 @@ The connection example shows how to configure the connection for this connector. ``` Add the name of the topic in {{topicName}}; a unique consumer identifier in {{consumerId}} and the endpoint for Apache Kafka (for example localhost:9092) in {{kafkaEndpoint}}. +???+ info "Configuring Egeria" + Egeria makes extensive use of events, and this connector is its default connector for sending and receiving events. In order to simplify the configuration of Egeria's [OMAG Servers](/concepts/omag-server), it is possible to set up the default configuration properties for this connector in the [Event Bus Configuration](/guides/admin/servers/configuring-event-bus). This configuration is used when configuring the topics for the server's [cohorts](/concepts/cohoer-member) and [Open Metadata Access Services (OMAS)](/services/omas) + ### Default properties for the producer and consumer @@ -93,7 +98,37 @@ The properties for the producer and consumer passed on the connection override t | bring.up.retries | 10 | | bring.up.minSleepTime | 5000 | -#### Security +### Controlling the direction of events + +By default, this connector supports both the receiving and sending of events on a particular topic. It is possible to turn off, either the ability to listen for events through the consumer, or send events through the producer. This is achieved by setting the `eventDirection` configuration property. + +The name of this property is defined in the [OpenMetadataTopicProvider](https://github.com/odpi/egeria/blob/main/open-metadata-implementation/repository-services/repository-services-apis/src/main/java/org/odpi/openmetadata/repositoryservices/connectors/openmetadatatopic/OpenMetadataTopicProvider.java) class. +```text + public static final String EVENT_DIRECTION_PROPERTY_NAME = "eventDirection"; // property name + public static final String EVENT_DIRECTION_INOUT = "inOut"; // default value + public static final String EVENT_DIRECTION_OUT_ONLY = "outOnly"; // do not start the consumer + public static final String EVENT_DIRECTION_IN_ONLY = "inOnly"; // do not start the producer +``` +Below is a code fragment that uses these values: +```text + Map configurationProperties = new HashMap<>(); + + configurationProperties.put(OpenMetadataTopicProvider.EVENT_DIRECTION_PROPERTY_NAME, OpenMetadataTopicProvider.EVENT_DIRECTION_OUT_ONLY); + topicConnection.setConfigurationProperties(configurationProperties); +``` +Using JSON, the property would be set as follows: +```json + + "configurationProperties": + { + "eventDirection" : "outOnly" + } +``` + +!!! danger "Do not use eventDirection for Egeria OMAG Server configuration" + If you are configuring the [event bus configuration](/guides/admin/servers/configuring-event-bus/) in a configuration document using the REST API, do not set the `eventDirection` property because the configuration helper functions use the event bus configuration to set up all types of topics and will assign the appropriate values for `eventDirection`. + +### Security By default, kafka security is not configured. The exact configuration may depend on the specific kafka service being used. Service specific notes are below. They may work for other providers, and feedback is welcome so that this documentation can be updated accordingly. @@ -119,15 +154,14 @@ An example of a use of this configuration can be found in the virtual data conne #### Handling Kafka Cluster Bring Up Issues -It's strongly recommended that you ensure Kafka is started before starting any OMAG servers, for example in Kubernetes you may add an init container into a pod running Egeria's [OMAGServerPlatform](/concepts/omag-server-platform) to validate Kafka is ok, before the main container is launched. The [Helm charts that are provided by the Egeria team](/guides/operations/kubernetes) typically take this approach. You should also ensure that you have a highly available Kafka deployment with multiple brokers. +It's strongly recommended that you ensure Kafka is started before starting any OMAG servers, for example in Kubernetes you may add an init container into a pod running Egeria's [OMAG Server Platform](/concepts/omag-server-platform) to validate Kafka is ok, before the main container is launched. The [Helm charts that are provided by the Egeria team](/guides/operations/kubernetes) typically take this approach. You should also ensure that you have a highly available Kafka deployment with multiple brokers. However, in some environments users have encountered issues where the Kafka Cluster hasn't become fully available. To mitigate this, the Kafka Open Topic Connector provides a mechanism that verifies that the Kafka Cluster brokers are available before attempting to connect. This mechanism is controlled by two producer/consumer properties. * `bring.up.retries` defaults to 10 and specifies the number of times the Egeria KafkaTopicConnector will retry verification before reporting a failure. -* `bring.up.minSleepTime` is set to 5000ms by default and is the minimum amount of time to wait before attempting a verification retry. -If a Kafka verification attempt takes longer than this value the KafkaTopicConnector does not pause before retrying the verification. +* `bring.up.minSleepTime` is set to 5000ms by default and is the minimum amount of time to wait before attempting a verification retry. If a Kafka verification attempt takes longer than this value the KafkaTopicConnector does not pause before retrying the verification. Different Kafka issues will result in differing overall times. For example, if the Kafka brokers are not resolvable (typical if waiting for a Kubernetes service to start), the Kafka library API calls will immediately fail, and so the connector will impose the minimum delay of 5s, each iteration will therefore take 5s by default. If instead the network address is unreachable, the Kafka client library will typically wait for 60s. As this is greater than 5s, no additional wait will be added. @@ -135,36 +169,32 @@ If any capabilities of Egeria require Kafka and the topic connector cannot initi #### Topic Creation -Many enterprise Apache Kafka services do not allow automatic topic creation and so you need to ensure that any topics - -You will need to manually create topics of the following form +Many enterprise Apache Kafka services do not allow automatic topic creation and so you need to ensure that any topics being used by Egeria are defined in Apache Kafka. -BASE_TOPIC_NAME is the value used for topicURLRoot when configuring the egeria event bus. For example, the default -value is `egeria`. +By default, Egeria generates its topic names using the following naming convention where `BASE_TOPIC_NAME` is the value used for topicURLRoot when configuring the egeria event bus. For example, the default value is `egeria`. -##### Cohort topics +??? info "Cohort topics" -For each cohort being used (such as `cocoCohort`): - * BASE_TOPIC_NAME.omag.openmetadata.repositoryservices.cohort.COHORT_NAME.OMRSTopic + For each cohort being used (such as `cocoCohort`): + + * BASE_TOPIC_NAME.omag.openmetadata.repositoryservices.cohort.COHORT_NAME.OMRSTopic -##### OMAS Topics +??? info "OMAS Topics" + [Open Metadata Access Services OMASs](/services/omas) use topics to support event based interaction. -[Open Metadata Access Services OMASs](/services/omas) use topics to support event based interaction. + These need to be done FOR EACH SERVER configured to run one or more OMASs. (For example for Coco Pharmaceuticals this might include `cocoMDS1`, `cocoMDS2`, `cocoMDS3` etc). -These need to be done FOR EACH SERVER configured to run one or more OMASs. -(For example for Coco Pharmaceuticals this might include `cocoMDS1`, `cocoMDS2`, `cocoMDS3` etc). + Then, FOR EACH OMAS configured (eg Asset Consumer OMAS, Data Platform OMAS, Governance Engine OMAS etc.) that supports topics, you will need to create the topic it is using. -FOR EACH OMAS configured (eg Asset Consumer OMAS, Data Platform OMAS, Governance Engine OMAS etc.) + This is the default naming convention: - * BASE_TOPIC_NAME.omag.server.SERVER_NAME.omas.OMAS_NAME.InTopic - * BASE_TOPIC_NAME.omag.server.SERVER_NAME.omas.OMAS_NAME.OutTopic + * BASE_TOPIC_NAME.omag.server.SERVER_NAME.omas.OMAS_NAME.InTopic + * BASE_TOPIC_NAME.omag.server.SERVER_NAME.omas.OMAS_NAME.OutTopic -One way to configure is to initially run against a kafka service which allows auto topic creation, then make note of the kafka -topics that have been created - so that they can be replicated on the restricted setup. +One way to create the list of kafka topics is to initially run against a kafka service which allows auto topic creation, then make note of the kafka topics that have been created - so that they can be replicated on the restricted setup. -In addition, review the Egeria Audit Log for any events beginning OCF-KAFKA-TOPIC-CONNECTOR so that -action may be taken if for example topics are found to be missing. +In addition, review the Egeria Audit Log for any events beginning OCF-KAFKA-TOPIC-CONNECTOR so that action may be taken if for example topics are found to be missing. --8<-- "snippets/abbr.md" diff --git a/site/docs/connectors/romania 2023.drawio b/site/docs/connectors/romania 2023.drawio deleted file mode 100644 index b025fbca61..0000000000 --- a/site/docs/connectors/romania 2023.drawio +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/site/docs/connectors/workshop 2023 Romania.drawio b/site/docs/connectors/workshop 2023 Romania.drawio deleted file mode 100644 index 756a0935cb..0000000000 --- a/site/docs/connectors/workshop 2023 Romania.drawio +++ /dev/null @@ -1,16 +0,0 @@ - - - - - - - - - - - - - - - - diff --git a/site/docs/education/egeria-dojo/developer/asset-create-pom-skeleton.md b/site/docs/education/egeria-dojo/developer/asset-create-pom-skeleton.md index 8209c93cb3..bc5da46d79 100644 --- a/site/docs/education/egeria-dojo/developer/asset-create-pom-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/asset-create-pom-skeleton.md @@ -35,7 +35,7 @@ Paste this structure into your `pom.xml` class. UTF-8 - 3.12 + 4.2 1.2.10 diff --git a/site/docs/education/egeria-dojo/developer/asset-create-skeleton.md b/site/docs/education/egeria-dojo/developer/asset-create-skeleton.md index 5125b4ac84..5ee5a8ac34 100644 --- a/site/docs/education/egeria-dojo/developer/asset-create-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/asset-create-skeleton.md @@ -140,7 +140,7 @@ Paste this code between the curly braces of the `AssetCreate` class. System.out.println("Using userId: " + clientUserId); System.out.println(); - HttpHelper.noStrictSSLIfConfigured(); + HttpHelper.noStrictSSL(); try { diff --git a/site/docs/education/egeria-dojo/developer/asset-listen-pom-skeleton.md b/site/docs/education/egeria-dojo/developer/asset-listen-pom-skeleton.md index ccfed5edc8..dabdbcacd1 100644 --- a/site/docs/education/egeria-dojo/developer/asset-listen-pom-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/asset-listen-pom-skeleton.md @@ -36,7 +36,7 @@ Paste this structure into your `pom.xml` class. UTF-8 - 3.12 + 4.2 1.2.10 @@ -45,12 +45,30 @@ Paste this structure into your `pom.xml` class. + + org.odpi.egeria + audit-log-framework + ${open-metadata.version} + + org.odpi.egeria open-connector-framework ${open-metadata.version} + + org.odpi.egeria + ocf-metadata-client + ${open-metadata.version} + + + + org.odpi.egeria + ocf-metadata-api + ${open-metadata.version} + + org.odpi.egeria asset-consumer-api @@ -81,6 +99,12 @@ Paste this structure into your `pom.xml` class. ${open-metadata.version} + + org.odpi.egeria + basic-file-connector + ${open-metadata.version} + + ch.qos.logback logback-classic diff --git a/site/docs/education/egeria-dojo/developer/asset-listen-skeleton.md b/site/docs/education/egeria-dojo/developer/asset-listen-skeleton.md index 72a52397de..7a86d7ce88 100644 --- a/site/docs/education/egeria-dojo/developer/asset-listen-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/asset-listen-skeleton.md @@ -251,7 +251,7 @@ Paste this code between the curly braces of the `AssetListen` class. System.out.println("Using userId: " + clientUserId); System.out.println(); - HttpHelper.noStrictSSLIfConfigured(); + HttpHelper.noStrictSSL(); try { diff --git a/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md b/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md index 7437aa50b2..dbb0e19341 100644 --- a/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md +++ b/site/docs/education/egeria-dojo/developer/different-types-of-apis-platform-deployment.md @@ -2,9 +2,7 @@ -In `egeria-dev-projects`, there is a module called `egeria-platform-report`. This contains a utility that reports on the status of a running OMAG Server Platform. Navigate down the contents of `egeria-platform-report` until you come across a java class called `EgeriaPlatformReport`. - -![egeria-platform-report](/education/egeria-dojo/developer/egeria-platform-report-module.png) +In the `omag-server-platform` assembly, there is a jar file called `egeria-platform-report.jar` under the `etc/reports` directory. This contains a utility that reports on the status of a running OMAG Server Platform. ??? beginner "Run `EgeriaPlatformReport`" Select `EgeriaPlatformReport` and then `Run 'EgeriaPlatformReport...main()'` from the right-mouse menu. @@ -12,19 +10,18 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th A new *Run* tab appears next to `EgeriaPlatform`. `EgeriaPlatformReport` runs but fails: ```bash - /Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home/bin/java ... org.odpi.openmetadata.devprojects.reports.platform.EgeriaPlatformReport + /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/java -Dfile.encoding=UTF-8 -jar /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/etc/reports/egeria-platform-report.jar =============================== - OMAG Server Platform Report: Sun Feb 13 22:13:33 GMT 2022 + OMAG Server Platform Report: Mon Sep 04 18:00:51 BST 2023 =============================== Running against platform: https://localhost:9443 Using userId: garygeeke Platform report for: https://localhost:9443 - Platform deployment - There was an org.odpi.openmetadata.frameworks.connectors.ffdc.PropertyServerException exception when calling the platform. Error message is: OMAG-COMMON-503-001 A client-side exception was received from API call getPlatformOrigin to OMAG Server EgeriaPlatform at https://localhost:9443. The error message was CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.ResourceAccessException was received by method getPlatformOrigin from API call https://localhost:9443/open-metadata/platform-services/users/{1}/server-platform/origin to server EgeriaPlatform on platform https://localhost:9443. The error message was I/O error on GET request for "https://localhost:9443/open-metadata/platform-services/users/garygeeke/server-platform/origin": PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target - - Process finished with exit code 255 + Platform deployment + There was an org.odpi.openmetadata.frameworks.connectors.ffdc.PropertyServerException exception when calling the platform. Error message is: OMAG-COMMON-503-001 A client-side exception was received from API call getPlatformOrigin to OMAG Server EgeriaPlatform at https://localhost:9443. The error message was CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.ResourceAccessException was received by method getPlatformOrigin from API call https://localhost:9443/open-metadata/platform-services/users/{1}/server-platform/origin to server EgeriaPlatform on platform https://localhost:9443. The error message was I/O error on GET request for "https://localhost:9443/open-metadata/platform-services/users/garygeeke/server-platform/origin": PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target + Process finished with exit code 255 ``` This is because it does not have a valid security certification. You will notice that IntelliJ has added a new configuration for `EgeriaPlatformReport' @@ -47,8 +44,9 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th Re-run `EgeriaPlatformReport` by clicking on the green arrow on the *Run* tab. There is more output this time even though it still seems to fail in the end. It begins with the report header that records the platform it is running against. ```bash + /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/java -Dstrict.ssl=false -Dfile.encoding=UTF-8 -jar /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/etc/reports/egeria-platform-report.jar =============================== - OMAG Server Platform Report: Mon Feb 14 07:57:05 GMT 2022 + OMAG Server Platform Report: Mon Sep 04 19:38:03 BST 2023 =============================== Running against platform: https://localhost:9443 Using userId: garygeeke @@ -59,10 +57,10 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th Then `EgeriaPlatfromReport` starts to call the Egeria APIs that are used for configuring and managing the OMAG Server Platform. ```bash - Platform deployment - Egeria version: Egeria OMAG Server Platform (version 3.12) - Configuration document store connector: - Platform security connector: + Platform deployment + Egeria version: Egeria OMAG Server Platform (version 4.3) + Configuration document store connector: + Platform security connector: ``` The `Egeria version` is retrieved using the `getPlatformOrigin` method of the [Platform Services](/services/platform-services/overview/). @@ -72,14 +70,13 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th String platformOrigin = platformServicesClient.getPlatformOrigin(clientUserId); ``` - The connectors configured for the platform are retrieved using the [Administration Service's `OMAGServerPlatformConfigurationClient`](/services/admin-services/overview/#configuring-the-omag-server-platform){ target=dojo-support } + The connectors configured for the platform are retrieved using the platform services client and [Administration Service's `OMAGServerPlatformConfigurationClient`](/services/admin-services/overview/#configuring-the-omag-server-platform){ target=dojo-support } ```java linenums="1" - - OMAGServerPlatformConfigurationClient platformConfigurationClient = new OMAGServerPlatformConfigurationClient(clientUserId, platformURLRoot); + OMAGServerPlatformConfigurationClient platformConfigurationClient = new OMAGServerPlatformConfigurationClient(clientUserId, platformURLRoot); Connection configurationStoreConnection = platformConfigurationClient.getConfigurationStoreConnection(); - Connection platformSecurityConnection = platformConfigurationClient.getPlatformSecurityConnection(); + Connection platformSecurityConnection = platformServicesClient.getPlatformSecurityConnection(); ``` Next is the list of *Registered services*. These are optional services that can be plugged into the `OMAG Server Platform` and provide the services to retrieve and maintain/govern open metadata. The list that you see is the default set of services installed with the platform. @@ -90,22 +87,21 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th Registered services Data Manager OMAS: Capture changes to the data stores and data set managed by a data manager such as a database server, content manager or file system. Subject Area OMAS: Document knowledge about a subject area - Design Model OMAS: Exchange design model content with tools and standard packages Glossary View OMAS: Support glossary terms visualization - Asset Manager OMAS: Manage metadata from a third party asset manager + Design Model OMAS: Exchange design model content with tools and standard packages Security Officer OMAS: Set up rules and security tags to protect data + Asset Manager OMAS: Manage metadata from a third party asset manager IT Infrastructure OMAS: Manage information about the deployed IT infrastructure Data Science OMAS: Create and manage data science definitions and models Community Profile OMAS: Define personal profile and collaborate - Discovery Engine OMAS: Support for automated metadata discovery engines Data Engine OMAS: Exchange process models and lineage with a data engine + Discovery Engine OMAS: Support for automated metadata discovery engines Digital Architecture OMAS: Design of the digital services for an organization Asset Owner OMAS: Manage an asset Stewardship Action OMAS: Manage exceptions and actions from open governance Governance Program OMAS: Manage the governance program Digital Service OMAS: Manage a digital service through its lifecycle Asset Lineage OMAS: Store asset lineage - Analytics Modeling OMAS: Provides metadata information for Analytics Modeling. Asset Consumer OMAS: Access assets through connectors Asset Catalog OMAS: Search and understand your assets DevOps OMAS: Manage a DevOps pipeline @@ -114,25 +110,29 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th Governance Engine OMAS: Set up an operational governance engine Data Privacy OMAS: Manage governance of privacy Security Manager OMAS: Manages exchange of metadata with a security service - Repository Governance OMES: Maintains open metadata archives based on the activity in the connected cohorts. Governance Action OMES: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. + Repository Governance OMES: Dynamically governance open metadata repositories in the connected cohorts. Asset Analysis OMES: Analyses the content of an asset's real world counterpart, generates annotations in an open discovery report that is attached to the asset in the open metadata repositories. + Infrastructure Integrator OMIS: Exchange information relating to IT infrastructure such as hosts, platforms, servers, server capabilities and services. Files Integrator OMIS: Extract metadata about files stored in a file system or file manager. Topic Integrator OMIS: Exchange metadata with third party event-based brokers. - API Integrator OMIS: Exchange metadata with third party API Gateways. Security Integrator OMIS: Distribute security properties to security enforcement points. + API Integrator OMIS: Exchange metadata with third party API Gateways. Search Integrator OMIS: Store metadata with a third party technology that is focused on search efficiency. - Database Integrator OMIS: Extract metadata such as schema, tables and columns from database managers. Lineage Integrator OMIS: Manage capture of lineage from a third party tool. + Database Integrator OMIS: Extract metadata such as schema, tables and columns from database managers. Catalog Integrator OMIS: Exchange metadata with third party data catalogs. - Analytics Integrator OMIS: Exchange metadata with third party analytics tools. Display Integrator OMIS: Exchange metadata with applications that display data to users. + Analytics Integrator OMIS: Exchange metadata with third party analytics tools. Organization Integrator OMIS: Load information about the teams and people in an organization and return collaboration activity. Glossary Author OMVS: View Service for glossary authoring. Dynamic Infrastructure and Operations OMVS: Explore and operate an open metadata ecosystem. Repository Explorer OMVS: Explore open metadata instances. + Glossary Workflow OMVS: Create glossary terms and organize them into categories as part of a controlled workflow process. It supports the editing glossary and multiple states. + My Profile OMVS: Manage information about the logged on user as well as their preferences. Type Explorer OMVS: Explore the open metadata types in a repository or cohort. Server Author OMVS: Author servers. + Glossary Browser OMVS: View glossary terms and categories within a glossary. ``` The registered are also retrieved through the [Platform Services](/services/platform-services/overview){ target=dojo-support } using the `getXXXServices` method as follows: @@ -142,15 +142,15 @@ In `egeria-dev-projects`, there is a module called `egeria-platform-report`. Th List registeredOMISs = platformServicesClient.getIntegrationServices(clientUserId); List registeredOMVSs = platformServicesClient.getViewServices(clientUserId); ``` -??? beginner "Observe exception from `EgeriaPlatformReport`" +??? beginner "Observe no servers from `EgeriaPlatformReport`" - Finally, `EgeriaPlatformReport` calls the [Administration Services](/services/admin-services/overview){ target=dojo-support } to request the list of configured servers. At this stage, there are none. In fact, the directory where they are kept has not even been created which is why an exception occurs. It is printed out to show which directory (`data/servers`), relative to the working directory set up in the `EgeriaPlatform` configuration (`~/egeria-install/egeria-omag*`). + Finally, `EgeriaPlatformReport` calls the [Administration Services](/services/admin-services/overview){ target=dojo-support } to request the list of configured servers. At this stage, there are none. ```bash Platform servers - There was an org.odpi.openmetadata.adminservices.ffdc.exception.OMAGConfigurationErrorException exception when calling the platform. Error message is: OMAG-ADMIN-500-002 Method retrieveAllServerConfigs returned an unexpected exception of org.odpi.openmetadata.frameworks.connectors.ffdc.OCFRuntimeException with message ENCRYPTED-DOC-STORE-400-014 Unable to retrieve the encrypted configuration files; exception was java.nio.file.NoSuchFileException with message data/servers, while attempting access file data/servers/null/config/null.config + None - Process finished with exit code 255 + Process finished with exit code 0 ``` - The next step will create a server configuration document that will also create the configuration directory and resolve this issue. + The next step will create a server configuration document that will also create the configuration directory and the report will start to show servers. `EgeriaPlatformReport` also creates a [markdown](/guides/contributor/markdown) document called `egeria-platform-report.md` that contains its output. This [version of egeria-platform-report.md](/education/egeria-dojo/developer/egeria-platform-report-no-servers) is included for your reference. \ No newline at end of file diff --git a/site/docs/education/egeria-dojo/developer/different-types-of-apis-server-running.md b/site/docs/education/egeria-dojo/developer/different-types-of-apis-server-running.md index e40a723d4f..9af88c2fbe 100644 --- a/site/docs/education/egeria-dojo/developer/different-types-of-apis-server-running.md +++ b/site/docs/education/egeria-dojo/developer/different-types-of-apis-server-running.md @@ -52,7 +52,7 @@ Now it is time to start `mds1` and observe the changes to `EgeriaPlatformReport` The server is waiting for Apache Kafka to start. If this occurs start Apache Kafka (and Apache Zookeeper) ... ??? tip "Starting Apache Kafka" - ---8<-- "docs/education/tutorials/kafka-tutorial/task-starting-kafka.md" + ---8<-- "snippets/tasks/task-starting-kafka.md" Once Apache Kafka is running, `mds1` completes start-up and `ServerOps` outputs the success message in its *Run* tab as shown in outcome 1. It is possible, however, that you do not manage to get Kafka running in time and `mds1` fails to start. If that happens just re-run the `start mds1` command of `ServerOps` once you have Kafka running. diff --git a/site/docs/education/egeria-dojo/developer/event-logging-connector-pom-skeleton.md b/site/docs/education/egeria-dojo/developer/event-logging-connector-pom-skeleton.md index 38071a45cb..093c7af763 100644 --- a/site/docs/education/egeria-dojo/developer/event-logging-connector-pom-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/event-logging-connector-pom-skeleton.md @@ -35,7 +35,7 @@ Paste this structure into your `pom.xml` class. UTF-8 - 3.12 + 4.2 diff --git a/site/docs/education/egeria-dojo/developer/event-logging-provider-skeleton.md b/site/docs/education/egeria-dojo/developer/event-logging-provider-skeleton.md index 0919cd6dcc..d6264c2f2a 100644 --- a/site/docs/education/egeria-dojo/developer/event-logging-provider-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/event-logging-provider-skeleton.md @@ -21,7 +21,7 @@ Paste this code between the curly braces of the `EventLoggingProvider` class. /* * Class of the connector. */ - private static final Class connectorClass = EventLoggingConnector.class; + private static final String connectorClass = "egeria.dojo.connector.eventlogging.EventLoggingConnector"; /** * Constructor used to initialize the ConnectorProviderBase with the Java class name of the specific diff --git a/site/docs/education/egeria-dojo/developer/intellij-run-omag-server-platform.md b/site/docs/education/egeria-dojo/developer/intellij-run-omag-server-platform.md index eea1e3b330..138c6f7cc7 100644 --- a/site/docs/education/egeria-dojo/developer/intellij-run-omag-server-platform.md +++ b/site/docs/education/egeria-dojo/developer/intellij-run-omag-server-platform.md @@ -3,9 +3,11 @@ ### Creating a configuration for OMAG Server Platform -From your `egeria-dev-projects` IntelliJ window, look for a box called `Add Configuration...` +From your IntelliJ window, look for a box called `Add Configuration...` ![Add Configuration](/education/egeria-dojo/developer/intellij-add-configuration.png) -Click on it and a wizard window opens. Open the list of *Templates* on the left hand column. +Or you may see a dropdown, in which case select "Edit Configurations ..." + +Click on it and a wizard window opens. Open the list of *Templates* on the left hand column (you may need to click on the `+` sign in some versions of IntelliJ). ![Configuration Templates](/education/egeria-dojo/developer/intellij-configuration-templates.png) Scroll down to *Jar Application* template and select it. A form appears on the right-hand side. ![Jar Application Template](/education/egeria-dojo/developer/intellij-jar-application-template.png) @@ -16,7 +18,7 @@ Enter: * `VM Options` is set to `-Dstrict.ssl=false -Dloader.path=server/lib -Dserver.port=9443`. This turns off SSL checking, points to the `lib` directory where you will be adding your new connector jar files and requests that the OMAG Server listens on port `9443`. * `Working Directory` is set to the parent directory above the directory with the OMAG Server Platform jar. This directory includes the `truststore.p12` certificate file needed for the platform to start successfully. -The example below is from a machine where the directory used to install Egeria was `/Users/mandy-chessell/egeria-install`. Notice the directory name needs to be specified in full, - the `~` notation does not work here. In this example, the Egeria release used was `3.12`. +The example below is from a machine where the directory used to install Egeria was `/Users/mandy-chessell/egeria-install`. Notice the directory name needs to be specified in full, - the `~` notation does not work here. In this example, the Egeria release used was `4.3`. ![OMAG Server Platform Configuration](/education/egeria-dojo/developer/intellij-omag-server-platform-configuration.png) Once you have entered these values, click `OK`. The `Add Configurations ...` box has been replaced by a drop-down menu that includes your new configuration called `EgeriaPlatform`. @@ -30,7 +32,7 @@ If you click on the down arrow next to `EgeriaPlatform` you will see `Add Config With `EgeriaPlatform` showing in the configurations box, click the green triangle and a *Run* tab opens showing the platform is starting up. The first line of the output is the call used to start the platform. You should be able to see the options that you added to the `EgeriaPlatform` configuration. After the command is the console output of the OMAG Server Platform. When you see the `OMAG server platform ready for more configuration` message, it is ready for work. ```bash -/Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home/bin/java -Dstrict.ssl=false -Dloader.path=server/lib -Dserver.port=9443 -Dfile.encoding=UTF-8 -jar /Users/mandy-chessell/egeria-install/egeria-omag-3.12/server/server-chassis-spring-3.12.jar +/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/java -DString.ssl=false -Dloader.path=lib,extra -Dserver.port=9443 -Dfile.encoding=UTF-8 -jar /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/platform/omag-server-platform-4.3.jar Project Egeria - Open Metadata and Governance ____ __ ___ ___ ______ _____ ____ _ _ ___ / __ \ / |/ // | / ____/ / ___/ ___ ____ _ __ ___ ____ / _ \ / / __ / / / _ /__ ____ _ _ @@ -38,16 +40,17 @@ With `EgeriaPlatform` showing in the configurations box, click the green triangl / /_/ // / / // ___ |/ /_/ / ___/ // __// / | |/ // __// / / __ // // / \ / /_ / _// / // / / / / / \____//_/ /_//_/ |_|\____/ /____/ \___//_/ |___/ \___//_/ /_/ /_/ \__/\//___//_/ \__//_/ /_/ /_/ - :: Powered by Spring Boot (v2.6.3) :: - -2022-02-14 07:52:43.693 INFO 85347 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : Starting OMAGServerPlatform using Java 11.0.12 on Amandas-MacBook-Pro.local with PID 85347 (/Users/mandy-chessell/egeria-install/egeria-omag-3.12/server/server-chassis-spring-3.12.jar started by mandy-chessell in /Users/mandy-chessell/egeria-install/egeria-omag-3.12) -2022-02-14 07:52:43.696 INFO 85347 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : No active profile set, falling back to default profiles: default -2022-02-14 07:52:46.583 INFO 85347 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9443 (https) -2022-02-14 07:52:47.741 WARN 85347 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : strict.ssl is set to false! Invalid certificates will be accepted for connection! -2022-02-14 07:52:50.421 INFO 85347 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9443 (https) with context path '' -2022-02-14 07:52:50.436 INFO 85347 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : Started OMAGServerPlatform in 7.476 seconds (JVM running for 8.348) -Mon Feb 14 07:52:50 GMT 2022 No OMAG servers listed in startup configuration -Mon Feb 14 07:52:50 GMT 2022 OMAG server platform ready for more configuration + :: Powered by Spring Boot (v3.1.1) :: + +2023-09-04T15:46:39.390+01:00 INFO 76773 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : Starting OMAGServerPlatform using Java 17.0.6 with PID 76773 (/Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/platform/omag-server-platform-4.3.jar started by mandy-chessell in /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/platform) +2023-09-04T15:46:39.392+01:00 INFO 76773 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : No active profile set, falling back to 1 default profile: "default" +2023-09-04T15:46:43.116+01:00 INFO 76773 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9443 (https) +2023-09-04T15:46:44.443+01:00 INFO 76773 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : Working directory is: /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/platform +2023-09-04T15:46:44.443+01:00 WARN 76773 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : Java trust store 'javax.net.ssl.trustStore' is null - this is needed by Tomcat - using 'server.ssl.trust-store' +2023-09-04T15:46:46.591+01:00 INFO 76773 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9443 (https) with context path '' +2023-09-04T15:46:46.608+01:00 INFO 76773 --- [ main] o.o.o.s.springboot.OMAGServerPlatform : Started OMAGServerPlatform in 7.925 seconds (process running for 8.73) +Mon Sep 04 15:46:46 BST 2023 No OMAG servers listed in startup configuration +Mon Sep 04 15:46:46 BST 2023 OMAG server platform ready for more configuration ``` The *Run* tab where the OMAG Server Platform is running includes a control panel that allows you to start, restart, stop and control the scrolling of output of the program that is running. If you hover your mouse over the controls, the hover-text indicates what it does. diff --git a/site/docs/education/egeria-dojo/developer/overview-old.md b/site/docs/education/egeria-dojo/developer/overview-old.md index 9f495e54dc..0161caf6a7 100644 --- a/site/docs/education/egeria-dojo/developer/overview-old.md +++ b/site/docs/education/egeria-dojo/developer/overview-old.md @@ -22,17 +22,17 @@ After completing developer day of the egeria dojo you should feel comfortable wi Egeria's interfaces are written in Java. It is expected that you have basic Java skills and have the Java 17 SDK installed on your machine. ??? tip "Installing Java ..." - --8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-java.md" + --8<-- "snippets/tasks/task-installing-java.md" You will need to build Egeria's source which needs Apache Maven installed. ??? tip "Installing Apache Maven ..." - --8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-maven.md" + --8<-- "snippets/tasks/task-installing-maven.md" During the dojo you will running Apache Kafka to experiment with event notifications. ??? tip "Installing Apache Kafka ..." - --8<-- "docs/education/tutorials/kafka-tutorial/task-installing-kafka.md" + --8<-- "snippets/tasks/task-installing-kafka.md" The dojo makes use of egeria libraries and code samples in the following git repositories and you will need a clone of them on your machine. @@ -41,7 +41,7 @@ After completing developer day of the egeria dojo you should feel comfortable wi - https://github.com/odpi/egeria-dev-projects ??? tip "Downloading (cloning) Egeria's git repositories ..." - --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md" + --8<-- "snippets/tasks/task-cloning-egeria-source.md" Many of the contributors to the Egeria project use the *[IntelliJ IDEA](https://www.jetbrains.com/idea/)* IDE. Jetbrains offers a [free community edition of IntelliJ](https://www.jetbrains.com/idea/download/) which is sufficient to work with Egeria. You are free to use any IDE during the dojo, however the instructions will focus on IntelliJ IDEA. @@ -75,7 +75,7 @@ After completing developer day of the egeria dojo you should feel comfortable wi ??? beginner "Installing Egeria's core libraries (15 mins)" ### Installing Egeria --8<-- "docs/education/tutorials/intellij-tutorial/intellij-open-terminal.md" - --8<-- "docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md" + --8<-- "snippets/tasks/terminal-installing-egeria.md" ??? beginner "Setting up your Test environment (30 mins)" diff --git a/site/docs/education/egeria-dojo/developer/overview.md b/site/docs/education/egeria-dojo/developer/overview.md index f3f89bdceb..34740440ee 100644 --- a/site/docs/education/egeria-dojo/developer/overview.md +++ b/site/docs/education/egeria-dojo/developer/overview.md @@ -34,17 +34,17 @@ This dojo was last tested using Egeria release 3.14. Egeria's interfaces are written in Java. It is expected that you have basic Java skills and have the Java 17 SDK installed on your machine. ??? tip "Installing Java ..." - --8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-java.md" + --8<-- "snippets/tasks/task-installing-java.md" - You will need to build Egeria's source which needs Apache Maven installed. + You will need to build your code which needs Apache Maven installed. ??? tip "Installing Apache Maven ..." - --8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-maven.md" + --8<-- "snippets/tasks/task-installing-maven.md" During the dojo you will running Apache Kafka to experiment with event notifications. ??? tip "Installing Apache Kafka ..." - --8<-- "docs/education/tutorials/kafka-tutorial/task-installing-kafka.md" + --8<-- "snippets/tasks/task-installing-kafka.md" Egeria's source is on Github ... @@ -58,7 +58,7 @@ This dojo was last tested using Egeria release 3.14. - https://github.com/odpi/egeria-dev-projects ??? tip "Downloading (cloning) Egeria's git repositories ..." - --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md" + --8<-- "snippets/tasks/task-cloning-egeria-source.md" These next steps build the Egeria platform and then install it ready for the dojo. @@ -94,11 +94,10 @@ This dojo was last tested using Egeria release 3.14. ```bash ./gradlew build ``` - After about 15 minutes you will see the **BUILD SUCCESSFUL** message. ??? tip "Installing Egeria's core libraries (15 mins)" - --8<-- "docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md" + --8<-- "snippets/tasks/terminal-installing-egeria.md" There is a template git repository that contains a set of projects to support you as you work through the developer dojo exercises. @@ -123,22 +122,11 @@ This dojo was last tested using Egeria release 3.14. In this dojo, you will have an opportunity to build and run Java utilities and connectors. However before you get to the coding, you will spend some time setting up your development and test environment. ??? beginner "Setting up your Test environment (30 mins)" - - The `egeria-dev-projects.git` repository contains the utilities to support your Egeria test environment. This was one of the git repositories that you cloned in the prerequisite tasks. - - Open the `egeria-dev-projects.git` repository in a **new window** in IntelliJ. - - ??? beginner "Get the latest code for `egeria-dev-projects.git` ..." - --8<-- "docs/education/tutorials/intellij-tutorial/intellij-open-terminal.md" - Use the `git pull` command to get the latest code onto your machine. - Now build the `egeria-dev-projects` code ... - - ??? beginner "Building egeria-dev-projects.git in IntelliJ IDEA ..." - --8<-- "docs/education/tutorials/intellij-tutorial/intellij-building-egeria-dev-projects-git.md" + Egeria's services run in the [OMAG Server Platform](/concepts/omag-server-platform){ target=dojo-support }. The jar file for the OMAG Server Platform was built by the `egeria.git` build. It is located in the `platform` directory of your Egeria install and is called `omag-server-platform-{{release}}.jar`. You will need an easy way to start and stop the OMAG Server Platform. This is done with an IntelliJ *Configuration*. + + Open your copy of the `egeria-dojo-developer.git` repository in a **new window** in IntelliJ. - Egeria's services run in the [OMAG Server Platform](/concepts/omag-server-platform){ target=dojo-support }. The jar file for the OMAG Server Platform was built by the `egeria.git` build. It is located in the `server` directory of your Egeria install and is called `omag-server-platform-{{release}}.jar`. You will need an easy way to start and stop the OMAG Server Platform. This is done with an IntelliJ *Configuration*. - ??? beginner "Set up IntelliJ to run the OMAG Server Platform ..." ---8<-- "docs/education/egeria-dojo/developer/intellij-run-omag-server-platform.md" @@ -298,7 +286,7 @@ This dojo was last tested using Egeria release 3.14. The maven build has created a jar file in the `target` directory of your IntelliJ project called `event-logging-connector-3.12.jar` - The release number used in the name is controlled by the `3.12` entry in your `pom.xml` file. + The release number used in the name is controlled by the `4.2` entry in your `pom.xml` file. From your `egeria-dojo3` IntelliJ project's terminal window enter the following to copy the connector's jar file into your egeria-install server lib directory, making adjustments for your connector's release as appropriate. ```bash diff --git a/site/docs/education/egeria-dojo/developer/topic-cataloguing-connector-pom-skeleton.md b/site/docs/education/egeria-dojo/developer/topic-cataloguing-connector-pom-skeleton.md index 139a14c869..6689043a31 100644 --- a/site/docs/education/egeria-dojo/developer/topic-cataloguing-connector-pom-skeleton.md +++ b/site/docs/education/egeria-dojo/developer/topic-cataloguing-connector-pom-skeleton.md @@ -36,7 +36,7 @@ Paste this structure into your `pom.xml` class. UTF-8 - 3.12 + 4.2 3.1.0 diff --git a/site/docs/education/open-metadata-labs/running-natively.md b/site/docs/education/open-metadata-labs/running-natively.md index e0ed4600f3..7ef9e9f3ea 100644 --- a/site/docs/education/open-metadata-labs/running-natively.md +++ b/site/docs/education/open-metadata-labs/running-natively.md @@ -3,9 +3,7 @@ # Running Egeria natively -The technologies describes on this page are required to operation Egeria. They are included when using the -self-contained environments, but they can also be installed and run natively (directly) -on your system. +The technologies described on this page are required to operation Egeria. They are included when using the self-contained environments, but they can also be installed and run natively (directly) on your system. Just be aware that running them natively on your system will require the additional effort of downloading, installing, configuring and operating each one -- which the self-contained environments largely take care of for you. To make use @@ -14,14 +12,14 @@ order to use it natively. ## Installing and configuring prerequisite software ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-java.md" ---8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-installing-git.md" ---8<-- "docs/education/tutorials/kafka-tutorial/task-installing-kafka.md" ---8<-- "docs/education/tutorials/kafka-tutorial/task-starting-kafka.md" ---8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md" ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-building-egeria-source.md" ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md" ---8<-- "docs/education/tutorials/jupyter-tutorial/task-installing-jupyter.md" +--8<-- "snippets/tasks/task-installing-java.md" +--8<-- "snippets/tasks/task-installing-git.md" +--8<-- "snippets/tasks/task-installing-kafka.md" +--8<-- "snippets/tasks/task-starting-kafka.md" +--8<-- "snippets/tasks/task-downloading-egeria-source.md" +--8<-- "snippets/tasks/task-building-egeria-source.md" +--8<-- "snippets/tasks/task-installing-egeria.md" +--8<-- "snippets/tasks/task-installing-jupyter.md" ## Starting up your environment @@ -37,6 +35,6 @@ You need to start four OMAG Server Platforms at the following URLs - remembering ## Help -For additional help refer to our slack channels at http://slack.lfaidata.foundation +For additional help refer to our Slack channels at http://slack.lfaidata.foundation --8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/building-egeria-tutorial/omag-server-platform-assembly.svg b/site/docs/education/tutorials/building-egeria-tutorial/omag-server-platform-assembly.svg new file mode 100644 index 0000000000..29207a274a --- /dev/null +++ b/site/docs/education/tutorials/building-egeria-tutorial/omag-server-platform-assembly.svg @@ -0,0 +1,4 @@ + + + +
assembly
assembly
content-packs
content-packs
data
data
platform
platform
lib
lib
extra
extra
omag-server-platform-{release}.jar
omag-server-platform-{release}.j...
platform
platform
servers
servers
server1
server1
server1
server1
serverN
serverN
keys
keys
application.properties
application.properties
sample clients
sample clients
platform
platform
repository
repository
config
config
cohort registry
cohort registry
sample data
sample data
opt
opt
debug logs
debug logs
serverN
serverN
serverN
serverN
serverN
serverN
debug logs
debug logs
access logs
access logs
trust store
trust store
key store
key store
etc
etc
dist
dist
 utilities
 utilities
logs
logs
sample configs
sample configs
audit logs
audit logs
logback.xml
logback.xml
reports
reports
Docker
Docker
postman collections
postman collectionsText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/education/tutorials/building-egeria-tutorial/overview.md b/site/docs/education/tutorials/building-egeria-tutorial/overview.md index e034cfce86..b327f3c5a5 100644 --- a/site/docs/education/tutorials/building-egeria-tutorial/overview.md +++ b/site/docs/education/tutorials/building-egeria-tutorial/overview.md @@ -12,11 +12,9 @@ Alternatively you can also use [Kubernetes](/guides/operations/kubernetes) to ru ## Prerequisite technology for building Egeria ??? tip "Installing Java" - --8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-java.md" -??? tip "Installing Maven" - --8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-maven.md" -??? tip "Installing Git" - --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-installing-git.md" + --8<-- "snippets/tasks/task-installing-java.md" +??? tip "Installing Maven (rarely needed)" + --8<-- "snippets/tasks/task-installing-maven.md" ## Tutorial tasks @@ -24,13 +22,13 @@ Alternatively you can also use [Kubernetes](/guides/operations/kubernetes) to ru 2. [Building the Egeria source with Apache Maven](#building-the-egeria-source) 3. [Installing Egeria](#installing-egeria) ---8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md" +--8<-- "snippets/tasks/task-cloning-egeria-source.md" You are now ready to [build the egeria source](#building-the-egeria-source). ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-building-egeria-source.md" +--8<-- "snippets/tasks/task-building-egeria-source.md" ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md" +--8<-- "snippets/tasks/task-installing-egeria.md" ## What next? diff --git a/site/docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md b/site/docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md deleted file mode 100644 index ab3b14edc4..0000000000 --- a/site/docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md +++ /dev/null @@ -1,135 +0,0 @@ - - - - -The [egeria build process](/education/tutorials/building-egeria-tutorial/overview) creates the distribution files for Egeria in the `open-metadata-distribution/open-metadata-assemblies` module. To see its contents, after a full gradle build completes, use the following `cd` command to change to its `build/distributions` directory: - -```bash -cd open-metadata-distribution/open-metadata-assemblies/build/distributions -``` -List the files: -```bash -ls -``` -The distribution tar file is `{release}-distribution.tar.gz` or `egeria-4.2-distribution.tar.gz` in this example. - -```bash -egeria-4.2-distribution.tar.gz -``` - -Create a directory for the install and copy the tar file into it. The two commands shown below create an install directory in your home directory and then copies the egeria distribution file into it. - -```bash -mkdir ~/egeria-install -``` -```bash -cp egeria*-distribution.tar.gz ~/egeria-install -``` - -These next commands change to the new directory and lists its contents. - -```bash -cd ~/egeria-install -``` - -It is now possible to unpack the tar file with the following steps. - -```bash -gunzip egeria*-distribution.tar.gz -``` -```bash -tar -xf egeria*-distribution.tar -``` -A new directory is created called `{release}-distribution.tar.gz` or `egeria-4.2-distribution.tar.gz` in this example. Change to this new directory and list its contents as shown below. - -```bash -cd egeria*gz -``` -```bash -ls -``` -```bash -LICENSE content-packs samples utilities -NOTICE keystore.p12 server -application.properties platform truststore.p12 -conformance-suite sample-data user-interface -``` - -As before, you may notice different files as Egeria evolves. - -Under `platform` is a directory for the [OMAG Server Platform](/concepts/omag-server-platform) that is used to run open metadata and governance services. This is the `omag-server-platform-{{release}}.jar`. - -```bash -ls platform -``` -```bash -lib omag-server-platform-4.2.jar -``` -The `lib` directory is where the jar files for connectors, samples and new registered services are installed. The initial list includes the connectors that are located in the `egeria.git` repository. -```bash -ls platform/lib -``` -```bash -audit-log-console-connector-4.2.jar -audit-log-event-topic-connector-4.2.jar -audit-log-file-connector-4.2.jar -audit-log-slf4j-connector-4.2.jar -avro-file-connector-4.2.jar -basic-file-connector-4.2.jar -cohort-registry-file-store-connector-4.2.jar -configuration-encrypted-file-store-connector-4.2.jar -configuration-file-store-connector-4.2.jar -csv-file-connector-4.2.jar -data-folder-connector-4.2.jar -discovery-service-connectors-4.2.jar -dynamic-archiver-connectors-4.2.jar -elasticsearch-integration-connector-4.2.jar -files-integration-connectors-4.2.jar -governance-action-connectors-4.2.jar -governance-services-sample-4.2.jar -graph-repository-connector-jar-with-dependencies-4.2.jar -inmemory-open-metadata-topic-connector-4.2.jar -inmemory-repository-connector-4.2.jar -kafka-integration-connector-4.2.jar -kafka-open-metadata-topic-connector-4.2.jar -omrs-rest-repository-connector-4.2.jar -open-lineage-janus-connector-4.2.jar -open-metadata-archive-directory-connector-4.2.jar -open-metadata-archive-file-connector-4.2.jar -open-metadata-security-samples-4.2.jar -openapi-integration-connector-4.2.jar -openlineage-integration-connectors-4.2.jar -spring-rest-client-connector-4.2.jar -``` - -Copy the jar files for any additional connectors you want to use into the `lib` directory. The connectors available for Egeria are listed in the [Connector Catalog](/connectors). - -The `content-packs` directory contains [Open Metadata Archives](/concepts/open-metadata-archive) that provide sample open metadata content. The `README.md` describes their content. -```bash -ls content-packs -``` -```bash -CloudInformationModel.json DataStoreConnectorTypes.json -CocoBusinessSystemsArchive.json OpenConnectorsArchive.json -CocoClinicalTrialsTemplatesArchive.json OpenMetadataTypes.json -CocoComboArchive.json README.md -CocoGovernanceEngineDefinitionsArchive.json SimpleAPICatalog.json -CocoGovernanceProgramArchive.json SimpleDataCatalog.json -CocoOrganizationArchive.json SimpleEventCatalog.json -CocoSustainabilityArchive.json SimpleGovernanceCatalog.json -CocoTypesArchive.json -``` -The `sample-data` directory contains sample data that is used in various labs and samples. -```bash -ls sample-data/* -``` -```bash -sample-data/oak-dene-drop-foot-weekly-measurements: -week1.csv week3.csv week5.csv week7.csv week9.csv -week2.csv week4.csv week6.csv week8.csv - -sample-data/old-market-drop-foot-weekly-measurements: -week1.csv week3.csv week5.csv week7.csv week9.csv -week2.csv week4.csv week6.csv week8.csv -``` ---8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/docker-tutorial/overview.md b/site/docs/education/tutorials/docker-tutorial/overview.md index 250cd36fa1..6ec832b272 100644 --- a/site/docs/education/tutorials/docker-tutorial/overview.md +++ b/site/docs/education/tutorials/docker-tutorial/overview.md @@ -4,7 +4,69 @@ # Docker tutorial **Docker** is a [simple container runtime and standard](https://www.docker.com/why-docker). -Every day, the egeria build processing creates a docker image of egeria and pushes it to the docker catalog on the **Docker website**. + +Every time the egeria Pull Request (PR) build runs, it creates a **latest** docker image of egeria based on the latest content of the `main` branch of [egeria.git](https://github.com/odpi/egeria) and pushes it to the docker catalog on the [Docker website](https://hub.docker.com/r/odpi/egeria). The site also contains docker images for each official release of egeria. + +Alternatively, you can build a customized docker image of egeria, that includes or excludes the connectors and services that you need. + +This page takes you through the different options for building and running egeria using docker containers. + +## Building your own docker image + +When you [build the main egeria repository](/docs/education/tutorials/building-egeria-tutorial/overview) a new `build` directory is created under `open-metadata-distribution/omag-server-platform`. + +Open up this directory and you will see two directories of interest: + +* *distributions* contains the packaged "assembly" of the [OMAG Server Platform](/concepts/omag-server-platform). +* *unpacked* contains the same content as the "assembly" but it is not packaged into a tar file. + +Take a copy of the "assembly" from either directory: + +* Copy and unpack the tar file or +* Copy all the files under `unpacked` + +Change into the assembly's top-level directory. When you list the contents + +```bash +$ ls +Dockerfile LICENSE NOTICE README.md assembly dist +``` + +Run the docker command: + +```bash +docker build -t egeria-platform:{myversion} -f Dockerfile . +``` +replacing `{myversion}` with a tag name for this docker image. The example below used `latest` as the tag name. + +```bash +? docker build -t egeria-platform:latest -f Dockerfile . +[+] Building 44.9s (5/5) FINISHED docker:desktop-linux + => [internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 3.28kB 0.0s + => [internal] load .dockerignore 0.0s + => => transferring context: 2B 0.0s + => [internal] load metadata for registry.access.redhat.com/ubi9/openjdk-17:latest 1.5s + => [1/1] FROM registry.access.redhat.com/ubi9/openjdk-17@sha256:3eded7b50a5ff8a55895c7b70c9dfc6e320363e5812a68747f281f8f4bb323ac 43.3s + => => resolve registry.access.redhat.com/ubi9/openjdk-17@sha256:3eded7b50a5ff8a55895c7b70c9dfc6e320363e5812a68747f281f8f4bb323ac 0.0s + => => sha256:5cbda490fcb7ab72e85cfba7098858cd3ed6bb15f95687e86979f6d4ac7b2f15 596B / 596B 0.0s + => => sha256:5b5deb1288720666a590472d9a506500df9ebe3c817e9710327162ccd24c4e22 24.19kB / 24.19kB 0.0s + => => sha256:62742f27dce5ebff467a57ad6bfa680820f3bc534cc313627f8113246276bf0f 37.83MB / 37.83MB 17.9s + => => sha256:f008a4f4b21c818e8bbd4e2521eb30ab0f8a43dc259e9e51c0d134641e343acd 110.80MB / 110.80MB 37.5s + => => sha256:3eded7b50a5ff8a55895c7b70c9dfc6e320363e5812a68747f281f8f4bb323ac 1.47kB / 1.47kB 0.0s + => => extracting sha256:62742f27dce5ebff467a57ad6bfa680820f3bc534cc313627f8113246276bf0f 2.0s + => => extracting sha256:f008a4f4b21c818e8bbd4e2521eb30ab0f8a43dc259e9e51c0d134641e343acd 5.5s + => exporting to image 0.0s + => => exporting layers 0.0s + => => writing image sha256:5a2c784ed7558dc625d109c04c8b5e78534b270a29d3011eb771fd01724c097f 0.0s + => => naming to docker.io/library/egeria-platform:latest 0.0s + +What's Next? + View summary of image vulnerabilities and recommendations → docker scout quickview + +``` + +## Using the standard docker container on docker hub This docker image provides a simple way to bring a runnable version of Egeria onto your machine. It also provides the basis for a [Kubernetes](/guides/operations/kubernetes) deployment of Egeria. diff --git a/site/docs/education/tutorials/downloading-egeria/git-hub-download-zip.png b/site/docs/education/tutorials/downloading-egeria/git-hub-download-zip.png new file mode 100644 index 0000000000..67c79bbd59 Binary files /dev/null and b/site/docs/education/tutorials/downloading-egeria/git-hub-download-zip.png differ diff --git a/site/docs/education/tutorials/downloading-egeria/git-hub-select-release.png b/site/docs/education/tutorials/downloading-egeria/git-hub-select-release.png new file mode 100644 index 0000000000..cc3a8516f9 Binary files /dev/null and b/site/docs/education/tutorials/downloading-egeria/git-hub-select-release.png differ diff --git a/site/docs/education/tutorials/downloading-egeria/overview.md b/site/docs/education/tutorials/downloading-egeria/overview.md new file mode 100644 index 0000000000..e0c76c1ddf --- /dev/null +++ b/site/docs/education/tutorials/downloading-egeria/overview.md @@ -0,0 +1,31 @@ + + + +# Downloading Egeria + +Egeria is an open source project that can be: + +* Downloaded from [GitHub](https://github.com/odpi/egeria) and built to create your own customized version of Egeria. +* Used directly from a prebuilt docker image from either [DockerHub](https://hub.docker.com/r/odpi/egeria) or [Quay.io](https://quay.io/repository/odpi/egeria) + +!!! tip "For developers wishing to contribute to Egeria" + Contributions to Egeria need to be made via a Pull Request (PR) from you own fork of the project. There are detailed instructions on how to do this in the [Git and GitHub Tutorial](/education/tutorials/git-and-git-hub-tutorial) + +## Downloading Egeria from GitHub + +There are two ways to download Egeria from GitHub: + +* Directly from the website +* Using git commands from your laptop + +The second approach is faster, but requires [git to be installed on your machine](/education/tutorials/git-and-git-hub-tutorial/overview/#installing-git-on-your-local-machine) which is worthwhile doing if you are anticipating downloading Egeria frequently. + +??? education "Downloading Egeria from GitHub website" + --8<-- "snippets/tasks/task-downloading-egeria-source.md" + +??? education "Downloading the Egeria Source using git commands" + --8<-- "snippets/tasks/task-cloning-egeria-source.md" + +Once you have downloaded the Egeria source, it is [ready to build](/education/tutorials/building-egeria-tutorial/overview). + +--8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/education/tutorials/git-and-git-hub-tutorial/egeria-git-repositories.md b/site/docs/education/tutorials/git-and-git-hub-tutorial/egeria-git-repositories.md index fb27474cd0..7fcb5aefad 100644 --- a/site/docs/education/tutorials/git-and-git-hub-tutorial/egeria-git-repositories.md +++ b/site/docs/education/tutorials/git-and-git-hub-tutorial/egeria-git-repositories.md @@ -13,40 +13,34 @@ Egeria has the following repositories: | Repository | Purpose | | --- | --- | | Repository | Purpose | -| [`data-governance` :material-github:](https://github.com/odpi/data-governance){ target=gh } | Egeria's Guidance on Governance as well as large media files such as presentations and movies | -| [`egeria-api-mocks` :material-github:](https://github.com/odpi/egeria-api-mocks){ target=gh } | Support for mocking Egeria API for UI dev test | +| [`egeria` :material-github:](https://github.com/odpi/egeria){ target=gh } | Egeria core | +| [`egeria-docs` :material-github:](https://github.com/odpi/egeria-docs){ target=gh } | Documentation repository for the Egeria project. | | [`egeria-charts` :material-github:](https://github.com/odpi/egeria-charts){ target=gh } | Helm chart repository | -| [`egeria-connector-hadoop-ecosystem` :material-github:](https://github.com/odpi/egeria-connector-hadoop-ecosystem){ target=gh } | Hadoop ecosystem connectors for Egeria: repository proxy connector for Apache Atlas. | +| [`egeria-jupyter-notebooks` :material-github:](https://github.com/odpi/egeria-jupyter-notebooks){ target=gh } | Egeria Jupiter notebooks used in the Open Metadata Labs | +| [`egeria-template-newrepo` :material-github:](https://github.com/odpi/egeria-template-newrepo){ target=gh } | Template for new Egeria repository | | [`egeria-connector-hivemetastore` :material-github:](https://github.com/odpi/egeria-connector-hivemetastore){ target=gh } | Egeria repository connector for Hive metastore. Initially targeted at spark metadata | | [`egeria-connector-ibm-information-server` :material-github:](https://github.com/odpi/egeria-connector-ibm-information-server){ target=gh } | IBM Information Server connectors for Egeria: repository proxy connector for IGC, data engine proxy connector for DataStage. | | [`egeria-connector-integration-event-schema` :material-github:](https://github.com/odpi/egeria-connector-integration-event-schema){ target=gh } | Provides an integration connector that extract event schemata from a schema registry (including Confluent schema registry). The connector will be a polling connector and will look in Egeria for new topics that if present in the confluent registry, the associate schema elements will be brought into Egeria. | | [`egeria-connector-integration-lineage-event-driven-sample` :material-github:](https://github.com/odpi/egeria-connector-integration-lineage-event-driven-sample){ target=gh } | Sample showing how to bring lineage into Egeria | | [`egeria-connector-integration-topic-strimzi` :material-github:](https://github.com/odpi/egeria-connector-integration-topic-strimzi){ target=gh } | Strimzi Egeria integration connector for Kafka Topics | -| [`egeria-connector-jdbc` :material-github:](https://github.com/odpi/egeria-connector-jdbc){ target=gh } | JDBC connectors for Egeria (Integration and Resource) | | [`egeria-connector-omrs-caching` :material-github:](https://github.com/odpi/egeria-connector-omrs-caching){ target=gh } | Provides an OMRS repository proxy connector that has an embedded repository that can be used to cache entities and relationships. | | [`egeria-connector-repository-file-sample` :material-github:](https://github.com/odpi/egeria-connector-repository-file-sample){ target=gh } | Provides a repository proxy sample that uses polling. The target is a file folder | | [`egeria-connector-sas-viya` :material-github:](https://github.com/odpi/egeria-connector-sas-viya){ target=gh } | Egeria connector for SAS Information Catalog | | [`egeria-connector-xtdb` :material-github:](https://github.com/odpi/egeria-connector-xtdb){ target=gh } | Pluggable repository for Egeria, using XTDB (formerly "Crux") as the back-end to natively support historical metadata. | -| [`egeria-dev-projects` :material-github:](https://github.com/odpi/egeria-dev-projects){ target=gh } | Fun projects for developers to help them learn about the Egeria technology | -| [`egeria-docs` :material-github:](https://github.com/odpi/egeria-docs){ target=gh } | Documentation repository for the Egeria project. | | [`egeria-dojo-developer` :material-github:](https://github.com/odpi/egeria-dojo-developer){ target=gh } | Provides the starting project structure for the developer dojo. The participant creates a clone of it and works from there - they do not merge their code back into the repository. There is also a branch which includes the sample answers that they can refer to if they get stuck. | -| [`egeria-js-commons` :material-github:](https://github.com/odpi/egeria-js-commons){ target=gh } | Common module for storing static data such as key value objects, SVGs, icon mappings, API urls. | -| [`egeria-jupyter-notebooks` :material-github:](https://github.com/odpi/egeria-jupyter-notebooks){ target=gh } | Egeria Jupiter notebooks used in the Open Metadata Labs | | [`egeria-k8s-operator` :material-github:](https://github.com/odpi/egeria-k8s-operator){ target=gh } | Egeria Kubernetes operator | | [`egeria-python` :material-github:](https://github.com/odpi/egeria-python){ target=gh } | Python library for Egeria | | [`egeria-react-ui` :material-github:](https://github.com/odpi/egeria-react-ui){ target=gh } | A multi-tenant, role based Egeria React UI for Open Metadata, that has eco-system and solutions capabilities. | | [`egeria-samples-api` :material-github:](https://github.com/odpi/egeria-samples-api){ target=gh } | A collection of samples illustrating the different APIs of Egeria. | | [`egeria-samples` :material-github:](https://github.com/odpi/egeria-samples){ target=gh } | various samples that can be useful either for learning or as initial starting points for working with Egeria | -| [`egeria-template-newrepo` :material-github:](https://github.com/odpi/egeria-template-newrepo){ target=gh } | Template for new Egeria repository | | [`egeria-test-cts` :material-github:](https://github.com/odpi/egeria-test-cts){ target=gh } | Automated CTS (conformance test) execution for Egeria | +| [`egeria-js-commons` :material-github:](https://github.com/odpi/egeria-js-commons){ target=gh } | Common module for storing static data such as key value objects, SVGs, icon mappings, API urls. | | [`egeria-ui-components` :material-github:](https://github.com/odpi/egeria-ui-components){ target=gh } | Encapsulated reactjs components with business logic inside. | | [`egeria-ui-core` :material-github:](https://github.com/odpi/egeria-ui-core){ target=gh } | Core library that includes generic reactjs components used across the user interface. | | [`egeria-ui` :material-github:](https://github.com/odpi/egeria-ui){ target=gh } | User interface instance using main Egeria functionalities. | -| [`egeria` :material-github:](https://github.com/odpi/egeria){ target=gh } | Egeria core | | [`happi-graph` :material-github:](https://github.com/odpi/happi-graph){ target=gh } | Generic graph rendering solution. | +| [`egeria-api-mocks` :material-github:](https://github.com/odpi/egeria-api-mocks){ target=gh } | Support for mocking Egeria API for UI dev test | All of these repositories are publicly visible. However, if you want to contribute new content then you need to create a GitHub account. This can be done from [the top of the GitHub home page :material-dock-window:](https://github.com){ target=gh }. - ----8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/git-and-git-hub-tutorial/git-introduction.md b/site/docs/education/tutorials/git-and-git-hub-tutorial/git-introduction.md index cc61708738..bd82c68a31 100644 --- a/site/docs/education/tutorials/git-and-git-hub-tutorial/git-introduction.md +++ b/site/docs/education/tutorials/git-and-git-hub-tutorial/git-introduction.md @@ -11,4 +11,3 @@ Git is an open source version control system. The Egeria project uses git to: As a result, it gives us a definitive source for the latest and greatest source code for Egeria itself, its history, and the rationale behind various decisions that are made over time. ----8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/git-and-git-hub-tutorial/overview.md b/site/docs/education/tutorials/git-and-git-hub-tutorial/overview.md index d44e1729e9..124b065985 100644 --- a/site/docs/education/tutorials/git-and-git-hub-tutorial/overview.md +++ b/site/docs/education/tutorials/git-and-git-hub-tutorial/overview.md @@ -4,6 +4,7 @@ # Git and GitHub Tutorial --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/git-introduction.md" + --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/egeria-git-repositories.md" ## General tasks for all community members @@ -17,8 +18,8 @@ This tutorial covers common tasks associated with using Git and GitHub to work w --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-getting-git-hub-id.md" --8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-creating-an-issue-on-git-hub.md" ---8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-installing-git.md" ---8<-- "docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md" +--8<-- "snippets/tasks/task-installing-git.md" +--8<-- "snippets/tasks/task-cloning-egeria-source.md" ## Using Git and GitHub when making a contribution diff --git a/site/docs/education/tutorials/index.md b/site/docs/education/tutorials/index.md new file mode 100644 index 0000000000..cd3ad21fc1 --- /dev/null +++ b/site/docs/education/tutorials/index.md @@ -0,0 +1,55 @@ + + + +# Tutorials + +The tutorials provide step-by-step guides that help you work with Egeria and its associated technology. They provide help to: + +* **Those that just want to run and use Egeria**. This includes how to download and build Egeria, install it in a convenient directory, customise it and use containers for cloud deployment. +* **Those that want to develop code with Egeria** (such as building new connectors or calling Egeria's APIs). This includes setting Egeria up in IntelliJ ready for your development work. +* **Those that wish to contribute to Egeria's code base**. This includes fixing bugs and adding new features to Egeria. + +--8<-- "snippets/supported-platforms.md" + +## Tutorials for running Egeria + +Egeria is an open source project that can be: + +* Started up ready to run the open metadata labs. These labs show Egeria in an operational mode serving different users in the Coco Pharmaceuticals business. Egeria is integrated with a variety of technologies in this environment, and it is set up to specifically support the scenarios within the labs. The implementation of this approach is using Kubernetes. Follow [this approach if you want to run the labs](/education/open-metadata-labs/overview). + +* Used directly from a prebuilt docker image from either [DockerHub](https://hub.docker.com/r/odpi/egeria) or [Quay.io](https://quay.io/repository/odpi/egeria). This docker image includes all the Egeria runtimes and can be used to experiment with different features of Egeria. If this option is for you, [go to the docker tutorial](/education/tutorials/docker-tutorial/overview). + +* Downloaded and built to create your own customized version of Egeria. Egeria is highly customizable, use this method if you wish to develop and deploy a customized Egeria for your organization. Go to [Starting with the code](#starting-with-the-code) if this is for you. + +## Tutorials for using Egeria + +There are two choices in learning about Egeria. + +The open metadata labs show how Egeria can be used in multiple scenarios. They are organized by user role, allowing you to focus on the aspects of Egeria that interests you. + +## Tutorials for developing with Egeria + +The Egeria community typically uses IntelliJ for its development. Information on how to set up your copy of IntelliJ with Egeria and build new components that work with Egeria are found in the [Using IntelliJ tutorial](/education/tutorials/intellij-tutorial/overview). + +## Tutorials for contributing to Egeria + +Egeria uses [GitHub's fork and pull model :material-dock-window:](https://help.github.com/articles/about-collaborative-development-models/){ target=gh } to create a contribution. This process is described in detail in the [Git and GitHub tutorial](/education/tutorials/git-and-git-hub-tutorial/overview#using-git-and-github-when-making-a-contribution). + +Each change should have a [GitHub issue](https://github.com/odpi/egeria/issues) explaining why the change is being made. The new or updated content should follow the Egeria [developer guidelines](/guides/contributor/guidelines). + +Every contribution is signed to say that the contributor has the rights to make the contribution and agrees with the [Developer Certificate of Origin (DCO) :material-github:](https://github.com/odpi/egeria/blob/main/developer-resources/why-the-dco.md){ target=gh } + +## Starting with the code + +??? education "Downloading Egeria's source code" + --8<-- "snippets/tasks/task-downloading-egeria-source.md" + +??? education "Installing Java" + --8<-- "snippets/tasks/task-installing-java.md" + +??? education "Building Egeria" + --8<-- "snippets/tasks/task-building-egeria-source.md" + +??? education "Installing Egeria" + --8<-- "snippets/tasks/task-installing-egeria.md" + diff --git a/site/docs/education/tutorials/installing-egeria-tutorial.md b/site/docs/education/tutorials/installing-egeria-tutorial.md index 1270479996..baa4677d5c 100644 --- a/site/docs/education/tutorials/installing-egeria-tutorial.md +++ b/site/docs/education/tutorials/installing-egeria-tutorial.md @@ -1,7 +1,7 @@ ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md" +--8<-- "snippets/tasks/task-installing-egeria.md" ---8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/intellij-tutorial/intellij-installing-egeria.md b/site/docs/education/tutorials/intellij-tutorial/intellij-installing-egeria.md index 327298b394..2e4a45aa9e 100644 --- a/site/docs/education/tutorials/intellij-tutorial/intellij-installing-egeria.md +++ b/site/docs/education/tutorials/intellij-tutorial/intellij-installing-egeria.md @@ -19,5 +19,5 @@ Process finished with exit code 0 Click on the `Terminal` tab - typically at the bottom of the IntelliJ window to open the terminal window. ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md" +--8<-- "snippets/tasks/task-installing-egeria.md" diff --git a/site/docs/education/tutorials/intellij-tutorial/task-installing-egeria-via-intellij.md b/site/docs/education/tutorials/intellij-tutorial/task-installing-egeria-via-intellij.md index 54c5d90b1f..2a4e4ddb65 100644 --- a/site/docs/education/tutorials/intellij-tutorial/task-installing-egeria-via-intellij.md +++ b/site/docs/education/tutorials/intellij-tutorial/task-installing-egeria-via-intellij.md @@ -6,6 +6,6 @@ If your development work is using the main Egeria libraries rather than developing its contents, you may wish to install Egeria into a more convenient location on your local disk. --8<-- "docs/education/tutorials/intellij-tutorial/intellij-open-terminal.md" ---8<-- "docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md" +--8<-- "snippets/tasks/terminal-installing-egeria.md" diff --git a/site/docs/education/tutorials/jupyter-tutorial/overview.md b/site/docs/education/tutorials/jupyter-tutorial/overview.md index 2070899e87..002565d9a4 100644 --- a/site/docs/education/tutorials/jupyter-tutorial/overview.md +++ b/site/docs/education/tutorials/jupyter-tutorial/overview.md @@ -8,7 +8,7 @@ The [Jupyter project :material-dock-window:](https://jupyter.org){ target=jupyte ![Example Jupyter notebook from the hands-on labs](jupyter-notebook-browser-window.png) ---8<-- "docs/education/tutorials/jupyter-tutorial/task-installing-jupyter.md" +--8<-- "snippets/tasks/task-installing-jupyter.md" --8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/kafka-tutorial/overview.md b/site/docs/education/tutorials/kafka-tutorial/overview.md index 20ae8e9300..361363af0c 100644 --- a/site/docs/education/tutorials/kafka-tutorial/overview.md +++ b/site/docs/education/tutorials/kafka-tutorial/overview.md @@ -5,9 +5,9 @@ [Apache Kafka :material-dock-window:](https://kafka.apache.org){ target=kafka } is an event bus that may be used to pass events between different Egeria servers. ---8<-- "docs/education/tutorials/kafka-tutorial/task-installing-kafka.md" ---8<-- "docs/education/tutorials/kafka-tutorial/task-starting-kafka.md" ---8<-- "docs/education/tutorials/kafka-tutorial/task-stopping-kafka.md" +--8<-- "snippets/tasks/task-installing-kafka.md" +--8<-- "snippets/tasks/task-starting-kafka.md" +--8<-- "snippets/tasks/task-stopping-kafka.md" --8<-- "snippets/abbr.md" diff --git a/site/docs/education/tutorials/omag-client-tutorial.md b/site/docs/education/tutorials/omag-client-tutorial.md index 0d7f8553ce..33a01392ef 100644 --- a/site/docs/education/tutorials/omag-client-tutorial.md +++ b/site/docs/education/tutorials/omag-client-tutorial.md @@ -1,7 +1,7 @@ -# OMAG Client Libraries Tutorial +# Running Reports Tutorial diff --git a/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md b/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md new file mode 100644 index 0000000000..dbb0e19341 --- /dev/null +++ b/site/docs/education/tutorials/running-from-assembly-tutorial/different-types-of-apis-platform-deployment.md @@ -0,0 +1,156 @@ + + + + +In the `omag-server-platform` assembly, there is a jar file called `egeria-platform-report.jar` under the `etc/reports` directory. This contains a utility that reports on the status of a running OMAG Server Platform. + +??? beginner "Run `EgeriaPlatformReport`" + Select `EgeriaPlatformReport` and then `Run 'EgeriaPlatformReport...main()'` from the right-mouse menu. + + A new *Run* tab appears next to `EgeriaPlatform`. `EgeriaPlatformReport` runs but fails: + + ```bash + /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/java -Dfile.encoding=UTF-8 -jar /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/etc/reports/egeria-platform-report.jar + =============================== + OMAG Server Platform Report: Mon Sep 04 18:00:51 BST 2023 + =============================== + Running against platform: https://localhost:9443 + Using userId: garygeeke + + Platform report for: https://localhost:9443 + Platform deployment + There was an org.odpi.openmetadata.frameworks.connectors.ffdc.PropertyServerException exception when calling the platform. Error message is: OMAG-COMMON-503-001 A client-side exception was received from API call getPlatformOrigin to OMAG Server EgeriaPlatform at https://localhost:9443. The error message was CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.ResourceAccessException was received by method getPlatformOrigin from API call https://localhost:9443/open-metadata/platform-services/users/{1}/server-platform/origin to server EgeriaPlatform on platform https://localhost:9443. The error message was I/O error on GET request for "https://localhost:9443/open-metadata/platform-services/users/garygeeke/server-platform/origin": PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target + + Process finished with exit code 255 + ``` + This is because it does not have a valid security certification. You will notice that IntelliJ has added a new configuration for `EgeriaPlatformReport' + +??? beginner "Set up VM Options" + + ![egeria-platform-report configuration](/education/egeria-dojo/developer/egeria-platform-report-configuration.png) + + Click on the white down arrow of the configurations box and select `Edit Configurations...`. Select the configuration for `EgeriaPlatformReport` to show its configuration form. + + Add `-Dstrict.ssl=false` to the `VM Options` and click `OK` to save the configuration. This turns off the need for a certificate. + + ??? tip "Don't see VM Options in the configuration form?" + You may need to use the `Modify Options` dropdown to `Add VM Options` to your configuration form. + ![Add VM Options](/education/egeria-dojo/developer/intellij-vm-options.png) + + ![egeria-platform-report configuration fixed](/education/egeria-dojo/developer/egeria-platform-report-configuration-fixed.png) + +??? beginner "Rerun `EgeriaPlatformReport`" + + Re-run `EgeriaPlatformReport` by clicking on the green arrow on the *Run* tab. There is more output this time even though it still seems to fail in the end. It begins with the report header that records the platform it is running against. + + ```bash + /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/java -Dstrict.ssl=false -Dfile.encoding=UTF-8 -jar /Users/mandy-chessell/egeria-install/egeria-platform-4.3-distribution.tar.gz/assembly/etc/reports/egeria-platform-report.jar + =============================== + OMAG Server Platform Report: Mon Sep 04 19:38:03 BST 2023 + =============================== + Running against platform: https://localhost:9443 + Using userId: garygeeke + + Platform report for: https://localhost:9443 + ``` + + Then `EgeriaPlatfromReport` starts to call the Egeria APIs that are used for configuring and managing the OMAG Server Platform. + + ```bash + Platform deployment + Egeria version: Egeria OMAG Server Platform (version 4.3) + Configuration document store connector: + Platform security connector: + ``` + + The `Egeria version` is retrieved using the `getPlatformOrigin` method of the [Platform Services](/services/platform-services/overview/). + + ```java linenums="1" + PlatformServicesClient platformServicesClient = new PlatformServicesClient("MyPlatform", platformURLRoot); + + String platformOrigin = platformServicesClient.getPlatformOrigin(clientUserId); + ``` + The connectors configured for the platform are retrieved using the platform services client and [Administration Service's `OMAGServerPlatformConfigurationClient`](/services/admin-services/overview/#configuring-the-omag-server-platform){ target=dojo-support } + + ```java linenums="1" + OMAGServerPlatformConfigurationClient platformConfigurationClient = new OMAGServerPlatformConfigurationClient(clientUserId, platformURLRoot); + + Connection configurationStoreConnection = platformConfigurationClient.getConfigurationStoreConnection(); + Connection platformSecurityConnection = platformServicesClient.getPlatformSecurityConnection(); + ``` + + Next is the list of *Registered services*. These are optional services that can be plugged into the `OMAG Server Platform` and provide the services to retrieve and maintain/govern open metadata. The list that you see is the default set of services installed with the platform. + + ??? beginner "List of registered services from `EgeriaPlatformReport`" + + ```bash + Registered services + Data Manager OMAS: Capture changes to the data stores and data set managed by a data manager such as a database server, content manager or file system. + Subject Area OMAS: Document knowledge about a subject area + Glossary View OMAS: Support glossary terms visualization + Design Model OMAS: Exchange design model content with tools and standard packages + Security Officer OMAS: Set up rules and security tags to protect data + Asset Manager OMAS: Manage metadata from a third party asset manager + IT Infrastructure OMAS: Manage information about the deployed IT infrastructure + Data Science OMAS: Create and manage data science definitions and models + Community Profile OMAS: Define personal profile and collaborate + Data Engine OMAS: Exchange process models and lineage with a data engine + Discovery Engine OMAS: Support for automated metadata discovery engines + Digital Architecture OMAS: Design of the digital services for an organization + Asset Owner OMAS: Manage an asset + Stewardship Action OMAS: Manage exceptions and actions from open governance + Governance Program OMAS: Manage the governance program + Digital Service OMAS: Manage a digital service through its lifecycle + Asset Lineage OMAS: Store asset lineage + Asset Consumer OMAS: Access assets through connectors + Asset Catalog OMAS: Search and understand your assets + DevOps OMAS: Manage a DevOps pipeline + Software Developer OMAS: Interact with software development tools + Project Management OMAS: Manage governance related projects + Governance Engine OMAS: Set up an operational governance engine + Data Privacy OMAS: Manage governance of privacy + Security Manager OMAS: Manages exchange of metadata with a security service + Governance Action OMES: Executes requested governance action services to monitor, assess and maintain metadata and its real-world counterparts. + Repository Governance OMES: Dynamically governance open metadata repositories in the connected cohorts. + Asset Analysis OMES: Analyses the content of an asset's real world counterpart, generates annotations in an open discovery report that is attached to the asset in the open metadata repositories. + Infrastructure Integrator OMIS: Exchange information relating to IT infrastructure such as hosts, platforms, servers, server capabilities and services. + Files Integrator OMIS: Extract metadata about files stored in a file system or file manager. + Topic Integrator OMIS: Exchange metadata with third party event-based brokers. + Security Integrator OMIS: Distribute security properties to security enforcement points. + API Integrator OMIS: Exchange metadata with third party API Gateways. + Search Integrator OMIS: Store metadata with a third party technology that is focused on search efficiency. + Lineage Integrator OMIS: Manage capture of lineage from a third party tool. + Database Integrator OMIS: Extract metadata such as schema, tables and columns from database managers. + Catalog Integrator OMIS: Exchange metadata with third party data catalogs. + Display Integrator OMIS: Exchange metadata with applications that display data to users. + Analytics Integrator OMIS: Exchange metadata with third party analytics tools. + Organization Integrator OMIS: Load information about the teams and people in an organization and return collaboration activity. + Glossary Author OMVS: View Service for glossary authoring. + Dynamic Infrastructure and Operations OMVS: Explore and operate an open metadata ecosystem. + Repository Explorer OMVS: Explore open metadata instances. + Glossary Workflow OMVS: Create glossary terms and organize them into categories as part of a controlled workflow process. It supports the editing glossary and multiple states. + My Profile OMVS: Manage information about the logged on user as well as their preferences. + Type Explorer OMVS: Explore the open metadata types in a repository or cohort. + Server Author OMVS: Author servers. + Glossary Browser OMVS: View glossary terms and categories within a glossary. + ``` + The registered are also retrieved through the [Platform Services](/services/platform-services/overview){ target=dojo-support } using the `getXXXServices` method as follows: + + ```java linenums="1" + List registeredOMASs = platformServicesClient.getAccessServices(clientUserId); + List registeredOMESs = platformServicesClient.getEngineServices(clientUserId); + List registeredOMISs = platformServicesClient.getIntegrationServices(clientUserId); + List registeredOMVSs = platformServicesClient.getViewServices(clientUserId); + ``` +??? beginner "Observe no servers from `EgeriaPlatformReport`" + + Finally, `EgeriaPlatformReport` calls the [Administration Services](/services/admin-services/overview){ target=dojo-support } to request the list of configured servers. At this stage, there are none. + ```bash + Platform servers + None + + Process finished with exit code 0 + ``` + The next step will create a server configuration document that will also create the configuration directory and the report will start to show servers. + + `EgeriaPlatformReport` also creates a [markdown](/guides/contributor/markdown) document called `egeria-platform-report.md` that contains its output. This [version of egeria-platform-report.md](/education/egeria-dojo/developer/egeria-platform-report-no-servers) is included for your reference. \ No newline at end of file diff --git a/site/docs/frameworks/alf/audit-log-framework.drawio b/site/docs/frameworks/alf/audit-log-framework.drawio index ad7d3748a3..56de2e83ba 100644 --- a/site/docs/frameworks/alf/audit-log-framework.drawio +++ b/site/docs/frameworks/alf/audit-log-framework.drawio @@ -1 +1,231 @@ -7Vxbc+I2FP41zLQPm7ElX+CRhGR32uwks7TTbt+ELUCNsVwhAuyvr4TlqwwBgi9Jdl9iHUuyfb5zP2J78Gax+cxQNP9KfRz0gOFvenDUA8AcQCj+SMo2priuIswY8dWkjDAmP7AiGoq6Ij5eFiZySgNOoiLRo2GIPV6gIcboujhtSoPiUyM0wxph7KFAp/5FfD6PqX3gZvQvmMzmyZNNZxDfWaBksvqS5Rz5dJ0jwdsevGGU8vhqsbnBgWRewpdnw0V/jn//9uA8Bv/943Lbc3/7FG92d8qS9BMYDvllt7bU3s8oWCmGqY/l24SDjK5CH8tdjB68Xs8Jx+MIefLuWsiMoM35IhAjU1wuOaNPKaclZUqC4IYGlO12g76N+76Vzszd6YMJdBxxx0fL+e55cjkVjyJcCp8tH38kK5LPwozjTU4QFGs+Y7rAnG3FFHX3kw0V7krOTVvBvs6kxkqEep6TGGgoIlKSOks3z9AQFwqQE8DRocC+kG01pIzP6YyGKLjNqNdFsLI595RGiqP/Ys63SlHRitMigHhD+N9y+ZWtRt9zd0YbtfNusE0Gofjc3CI5/J6/ly3bjZJ1JdFw/L7rVIoGHNoWNE+Gf0lXzMMH5ilTxhGb4UP7KWMh2X9QmBgOECfPRdNzcbmAms4OVz7hgnRPZ69TX6kwRBjPYUBmoaBxKTVnqHBqK08H7QSdhWZJZwe6zpqgQmdNpy6dtVrR2VT/CtqXKeMe/duNHjEj4tsxy2l2rMvgeAtQkhBk2AD4VRLijMw743RD/rImu0eqMrAurctq6SMlIc8JJ+gXhdPq28U94m9SyzKxGzKGtrlpkZywPOFBtmGXxDjeMhPq9CvPl3P7g/mmLku00ZRE2+dJ9KVkzmlX5tyfQpfb0GlK6EBTZhQcNKP6AtNxmre77uHYzwkEaNcTJq5m8mqEl5yEAgAavi4wPD0ErA4l2wkM08TtpcCwtriwr+Em8PqGPcp8DRjxlbzIfaR46AlGyUBNY+6C+H5s7/CS/ECT3VYSViX2Yl/7umePJNPFdsLKLWODdx62zaAIrCKKMCk95VB0K0AEdYE4aNMBJde50P4Y/wMKDijNFrqakZvgWAd08ajnVaKRvE5Owb/i5VIWJ4Exxrxt6zuhnNNFY6qrVdNaN8CmqUHQYmYOXkrN344iwm7pIdD08JFRT6giCSvio6EnXkmWly+pntPpFHheFVi+M3Hs5hyom1Tzt4ljrNBCu1Et1GuXykyO8JSEpDJQ/RkPxXBaJaMKDEuHs6pFUVtAZFr7vV5Z1e4oWyAu4XqVrtXGXWiUuJu0CPPcNSq4a9XGXVvj7i7JM5K8QWMx8jhlrzRmzTG4qlrfLIMdjcFjwUCsMVDodyQvvW1ABCfZy1ycxCy/n6QE5D3NdkA8rLjYZY+x6YzvMAel6oaro+U0CpbbSgCXlP8K2Rc4Pv0qxn1954W4r7Il03pZMHEg3emumG6pWleWu/hVa6tFm63WAppogNQkQo13KeyB3ahkgA7ZKbeZMlGHDVKSD3YkPQV6mWgoYN7+2B9zRHH6ekTI8cYiDK1v7dpHRRiwtvNW+mG4hwgzJELCveisFsFQBt25HPQeTXDwKBLKXTKbK70F8sZ1ilIRE/GvIo+NmyY5kGkM7U16WvI8dZzSkFc9P0W+1rN21sAqIG9VIA+bLEskdZI3UNkXA+3Ezj4Pf94BoPaNu3VstGl2y7j3W5GiDH23U068Bd+8JyS0B6VGYvnI7p6QUN/JSjBOe/6lnS51GkF7kGscfjPHKHvTwoJ6TiMAvf43wh7xK6IZ7Vx4/tC4cqaLzUz+COFqGtC1N0eMX/lkGQXyrSuDn2lAoi8dj3IsWPJ1fd3XVTWxa6ujJMdcylXFlovutQFgu24BAGDodV2ryaI51HOAbyKs03Xm3UAAyym5oetAVRuqPgj0SP8P9CRb9bIh2IEWVG1QOP2iOYKurg39RqEAHxUKN0lnOgOF3pz9aIbJMVs2THpA9c4h0Mq1oG3foHdd3zkEZskQtQ+B3pfVuN/UAZ1mf3R67DG52rIDqB8wb431zR4Rr/i9byXra5N6S88Lxh6NZChEpzIeOnTQ/46hBV5T9iQm/DK8v/v1/Zoryy2GsKAio65E7ozGgRhmP/CPqybZf5MAb/8H5ZvdV9sgFMD/mj6600CSto8zfsxN57Z2x80XDya3CZOEjFDb+NePNCRtxXl0c8NzeBIuhJb7g/uVOsBRvjoWpMzOeAJsgIbJaoAPBgh53jBUfxpJ3UpGI9wKUkETPWkjmNI70MKhli5oAtXORMk5k7TcFca8KCCWOzIiBF/uTptztvupJUnBEExjwkzpBU1k1krHaLSRvwOaZt0ne+GkHclJN1nvpMpIwpdbInw4wJHgXLatfBUBa5TX6QUdL9LJ97cfo5/stDgLr06pvNtrFzt6ziP9FgQU8o+Xnu5dlO+PRv4hvzqfzqKYYX6iHxneErbQ+mp46f3KulOi4IsigWah4QDvLzMqYVqSuBldqmOjZJnMmep5qkkYTQvVZjBX33Z/ThmLOONivRSez+cojpW8koLfwNZIEl6HQdiMdIpuVnvi3rWObkFIWG2R17o4Bp6DFLWa0o0Geu/6XHsj3V9uTkl/iLOtE4K1jOiDmfZLb5SvGlr/z2DhGSwkzaGSJC9dBIKG1oFMDCIGCEiUtdFdLmTGU14QdriR7u+i2sw55bzUKv0BUtbadJKF5Lv4lJ5F/U0/v+58bzpvgq57sNoePKh17y84D/tpnc3EzyZf8YWI4RH1amVKIlKQj2EYtxMbVT96kAQwIuntruF/8VOBjEPBBU1pQSQXnwQv1YGnyuG5eGOx9Rs7NuBUINSOPpIcXEDi+/e8WmgbCQpehRF9cWP4p1b5hY0ofqIRRf6rMqL4ESMa8bzkRaMeBy6sYUPtX1jfYBN3RE6cSA3uG1E8nFhmYiIhsaS8OIAqFrRsmi6AMS7LxPZlMZ2bzASQxI2Lcp8HRrZ5hL/h4Ur8ZxAJbBMZGUQqUPuhso548k+ZJAGME/8hJmN0jUN7TMa2mTyUJbVMXLwjvmebh1lnyqGqSApuupE+/rJXRzDr4prIDFZO5iW+/fK4WR/XTD4Rody7BOFk1S2wXyc3S6IkSSiVTYXmpJhzkRNXU5TAfk3UrLVUdSUhfxs7C8V6kcUzM/pFBcJhJNZTec/M5WEVw7q4EjFSVa7mkKH1rB6Z4ViP5qyNAZwEYz25R2ZM1oOZShLfzESjchfZWE/ykRmVte9CZ3XpBBLjXah1F4PMUIzxmLBzkZKC3q2DZFe8zH06yL6XMYvH/YsvV6lgbPvVFzILyD0V5WLkwomk3+AS/kcul/Pz2w/vz6azy9HR54tpeBeIbw/8crTH4thbSYPNxDYbMyrr2VzQG/r1y6mLXHxkm4sZkal1SUIkUUpisE7+3Sj5G97/H6Yyqrv5kf56bOtfHfDhLw== \ No newline at end of file + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/frameworks/ocf/overview.md b/site/docs/frameworks/ocf/overview.md index 3908ff49e4..5fdcf4258a 100644 --- a/site/docs/frameworks/ocf/overview.md +++ b/site/docs/frameworks/ocf/overview.md @@ -225,9 +225,9 @@ The [`ConnectorProviderBase` :material-github:](https://github.com/odpi/egeria/b */ public BasicFileStoreProvider() { - Class connectorClass = XXXStoreConnector.class; + String connectorClassName = "packagename.XXXStoreConnector"; - super.setConnectorClassName(connectorClass.getName()); + super.setConnectorClassName(connectorClassName); ConnectorType connectorType = new ConnectorType(); connectorType.setType(ConnectorType.getConnectorTypeType()); @@ -267,9 +267,9 @@ The [`ConnectorProviderBase` :material-github:](https://github.com/odpi/egeria/b */ public BasicFileStoreProvider() { - Class connectorClass = BasicFileStoreConnector.class; + String connectorClassName = "org.odpi.openmetadata.adapters.connectors.datastore.basicfile.BasicFileStoreConnector"; - super.setConnectorClassName(connectorClass.getName()); + super.setConnectorClassName(connectorClassName); ConnectorType connectorType = new ConnectorType(); connectorType.setType(ConnectorType.getConnectorTypeType()); diff --git a/site/docs/guides/admin/egeria-admin.drawio b/site/docs/guides/admin/egeria-admin.drawio index adb9bba372..dbafadd948 100644 --- a/site/docs/guides/admin/egeria-admin.drawio +++ b/site/docs/guides/admin/egeria-admin.drawio @@ -1,4 +1,4 @@ - + @@ -542,10 +542,245 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + diff --git a/site/docs/guides/admin/servers/configuring-the-workbenches.md b/site/docs/guides/admin/servers/configuring-the-workbenches.md index f933d63d48..9d7e04e320 100644 --- a/site/docs/guides/admin/servers/configuring-the-workbenches.md +++ b/site/docs/guides/admin/servers/configuring-the-workbenches.md @@ -18,15 +18,22 @@ To run a metadata repository through the Repository Workbench, first configure a Send the repository workbench configuration as the request body, similar to the following: - ```json hl_lines="3" + ```json hl_lines="3-5" { "class": "RepositoryConformanceWorkbenchConfig", "tutRepositoryServerName": "myserver", + "testEntityTypes" : "Asset", "maxSearchResults": 5 } ``` - The required `tutRepositoryServerName` parameter defines the name of the repository server you wish to test, while the optional `maxSearchResults` parameter controls the sizing of the tests: both the number of instances the tests will attempt to create to carry out its tests and how extensive the search-based tests are. + The required `tutRepositoryServerName` parameter defines the name of the repository server you wish to test, while the optional `maxSearchResults` parameter controls the sizing of the tests: both the number of instances the tests will attempt to create to carry out its tests and how extensive the search-based tests are. The `testEntityTypes` is also optional and is used to restrict the types of the instances tested to: + + * These listed entities and their super types. + * Any relationships that have the selected entity types at both ends. + * Any classifications that can be attached to the selected entity types. + + If `testEntityTypes` is either an empty list or not set, then all known types from the technology under test is tested. !!! attention "Start the technology under test **after** the CTS server" This repository server to test (`myserver` in the example above) should be [configured and started](/guides/admin/configuring-a-metadata-access-store) **after** starting the CTS repository workbench instance. Once the CTS server instance is started it will wait for the technology under test (the server named by the `tutRepositoryServerName` parameter) to be up and running before then starting its suite of tests. diff --git a/site/docs/guides/community.md b/site/docs/guides/community.md index 825cd66a42..4f9acecfc9 100644 --- a/site/docs/guides/community.md +++ b/site/docs/guides/community.md @@ -9,17 +9,10 @@ This project welcomes contributors from any organization or background, provided ### Live discussions -We have a variety of weekly and bi-weekly meetings to which all are welcome: +We have weekly community meeting to which all are welcome. We cover demos, releases, meetups and other activities going on in the community ([agenda and minutes :material-dock-window:](https://wiki.lfaidata.foundation/display/EG/Community+Meetings){ target=wiki }). These calls run January to June, September to November. July, August and December are popular vacation times, and so we stop the calls while the attendance is low. -| Call | Purpose | -|---|---| -| Developers call | Discussion on code development (not minuted) | -| Community call | Demos, meetups and other activities going on in the community ([agenda and minutes :material-dock-window:](https://github.com/odpi/data-governance/wiki){ target=wiki }) | -| TSC call | Strategy, planning and decision-making with the [Technical Steering Committee (TSC)](https://wiki.lfaidata.foundation/display/EG/Egeria+Technical+Steering+Committee+%28TSC%29+Home) | - - ### Asynchronous dialog Egeria uses the [LF AI & Data Slack community :material-slack:](https://slack.lfaidata.foundation){ target=slack } to provide an ongoing dialogue between members. This creates a recorded discussion of design decisions and discussions that complement the project meetings. @@ -27,7 +20,7 @@ Egeria uses the [LF AI & Data Slack community :material-slack:](https://slack.lf Follow the link above and register with the LF AI & Data Slack service using your email address. Once signed in you can see all the active Slack channels. !!! tip "The main Slack channel for the Egeria project is called `#egeria-discussion`" - Additional channels are added from time to time as new workgroups and discussion topics are established. For Egeria these channel names will begin with `#egeria`. You can also view Slack channels from other LF AI & Data projects. + We use `#egeria-announce` for meetings and release announcements. Additional channels are added from time to time as new workgroups and discussion topics are established. For Egeria these channel names will begin with `#egeria`. You can also view Slack channels from other LF AI & Data projects. ??? question "More information about Slack?" Slack is an instant messaging tool that allows multiple conversations to occur amongst the community members at any one time. Each conversation is called a *channel*. Channels can be set up for a specific event or have a long-term existence. @@ -38,9 +31,7 @@ The community tends to use the [mailing list :material-dock-window:](https://lis ### Other websites and resources -- [Webinars :material-github:](../../education/webinar-program/overview){ target=gh } - virtual events covering topics of interest. -- [Workshops :material-github:](https://github.com/odpi/data-governance/tree/main/workshops){ target=gh } - face-to-face workshops promoting discussion and education on Egeria -- [Presentations :material-github:](https://github.com/odpi/data-governance/tree/main/presentations){ target=gh } - presentations given at conferences and private gatherings +From time to time we screen webinars that are open to all. The aim of these webinars is to create in depth videos of a particular topic of interest. Past webinars are described on the [webinars page](/education/webinar-program/overview). ## Git repositories @@ -61,7 +52,7 @@ Every contribution is signed to say that the contributor has the rights to make ## Creating a Linux Foundation account -The Linux Foundation provide build and distribution facilities. You need an account to access some of the reports from the build. +The Linux Foundation provide build and distribution facilities. You need an account to access some build reports. This is the link to create a [Linux Foundation account :material-dock-window:](https://identity.linuxfoundation.org){ target=lf }. Note the username and password you selected. diff --git a/site/docs/guides/contributor/development.md b/site/docs/guides/contributor/development.md index 9aa373ce33..5d07ffdece 100644 --- a/site/docs/guides/contributor/development.md +++ b/site/docs/guides/contributor/development.md @@ -20,21 +20,37 @@ We provide [our own tutorial for IntelliJ](/education/tutorials/intellij-tutorial/overview). -## Apache Maven and Gradle +## Build tools -[Apache Maven :material-dock-window:](https://maven.apache.org/){ target=maven } is the tool that supports our project build. This includes the code compilation, running unit tests, validating dependencies and Javadoc as well as build our distribution archive. +The build process takes the source files from the git repository and creates executable libraries needed to run Egeria. When you download (clone) the contents of a git repository from GitHub, a new directory is created that is named after the repository that you cloned. For example, the directory created when the main `egeria.git` repository is cloned is called `egeria`. This directory contains all the source and the build scripts. -Maven 3.5 or higher is required to build Egeria. 3.6.x or above is recommended. +The project uses three main build technologies: -[Gradle :material-dock-window:](https://gradle.org/){ target=gradle } is an alternative build tool to Maven and offers: +* [Gradle :material-dock-window:](https://gradle.org/){ target=gradle } is the primary build tool for the Egeria repositories. +* [Apache Maven :material-dock-window:](https://maven.apache.org/){ target=maven } is an alternative build tool to Gradle and is being phased out. +* [npm :material-dock-window:](https://www.npmjs.com){ target=npm } is used for Javascript repositories associated with the User Interfaces. -- better support for parallel builds -- more flexibility for build tasks -- breaking the link between directory structure and maven artifacts -- extremely fast incremental builds +The build scripts that use these technologies ensure the software is built in the correct order. -Our direction is for a Gradle build to replace Maven; however, [that work is still underway :material-dock-window:](https://github.com/odpi/egeria/issues/3370){ target=gh }. As such, our supported build environment remains [Maven](#apache-maven) +##### Building with Gradle + +--8<-- "snippets/tasks/task-building-with-gradle.md" + +??? tip "Gradle development" + For egeria [Gradle :material-dock-window:](https://gradle.org/){ target=gradle } is a replacement build tool to Maven and offers: + + - better support for parallel builds + - more flexibility for build tasks + - breaking the link between directory structure and maven artifacts + - extremely fast incremental builds + + As of version 4, Egeria can only be built using gradle. + +##### Building with Maven + +If building a version of Egeria prior to version 4, the maven instructions can be found below: + +--8<-- "snippets/tasks/task-building-with-maven.md" -The [building Egeria tutorial](/education/tutorials/building-egeria-tutorial/overview) covers more details on the build process. --8<-- "snippets/abbr.md" diff --git a/site/docs/guides/contributor/release-process/index.md b/site/docs/guides/contributor/release-process/index.md index f3da4af3a1..46a45e5ceb 100644 --- a/site/docs/guides/contributor/release-process/index.md +++ b/site/docs/guides/contributor/release-process/index.md @@ -12,8 +12,7 @@ Releases are published to: ### Overall release policy -- Aim to release the core Egeria approximately every month -- Typically, we target end of month for external availability +- Aim to release the core Egeria approximately every six weeks [see schedule](/release-notes/overview#core-egeria-function). - Will only release an update between releases in exceptional circumstances - Preserves backwards compatibility as much as possible - Try and maintain a regular heartbeat: even if completion of some features continues in a subsequent release diff --git a/site/docs/guides/developer/implementing-a-connector-provider.md b/site/docs/guides/developer/implementing-a-connector-provider.md index 17b516e2c8..d58968d537 100644 --- a/site/docs/guides/developer/implementing-a-connector-provider.md +++ b/site/docs/guides/developer/implementing-a-connector-provider.md @@ -56,7 +56,7 @@ public class XXXStoreProvider extends ConnectorProviderBase /* * Define the name of the connector implementation. */ - private static final Class connectorClass = XXXStoreConnector.class; + private static final String connectorClassName = "packagename.XXXStoreConnector"; /* * Define the name of configuration properties. @@ -73,7 +73,7 @@ public class XXXStoreProvider extends ConnectorProviderBase /* * Set up the class name of the connector that this provider creates. */ - super.setConnectorClassName(connectorClass.getName()); + super.setConnectorClassName(connectorClassName); /* * Set up the connector type that should be included in a connection used to configure this connector. diff --git a/site/docs/guides/developer/integration-connectors/implementing-an-integration-connector-provider.md b/site/docs/guides/developer/integration-connectors/implementing-an-integration-connector-provider.md index 22e6512314..c0d9ad0229 100644 --- a/site/docs/guides/developer/integration-connectors/implementing-an-integration-connector-provider.md +++ b/site/docs/guides/developer/integration-connectors/implementing-an-integration-connector-provider.md @@ -43,7 +43,7 @@ public class XXXStoreProvider extends IntegrationConnectorProviderBase /* * Define the name of the connector implementation. */ - private static final Class connectorClass = XXXStoreConnector.class; + private static final String connectorClassName = "packagename.XXXStoreConnector"; /* * Define the name of configuration properties (optional). @@ -60,7 +60,7 @@ public class XXXStoreProvider extends IntegrationConnectorProviderBase /* * Set up the class name of the connector that this provider creates. */ - super.setConnectorClassName(connectorClass.getName()); + super.setConnectorClassName(connectorClassName); /* * Set up the connector type that should be included in a connection used to configure this connector. diff --git a/site/docs/guides/developer/integration-connectors/overview.md b/site/docs/guides/developer/integration-connectors/overview.md index 7dfb948069..7962448ca9 100644 --- a/site/docs/guides/developer/integration-connectors/overview.md +++ b/site/docs/guides/developer/integration-connectors/overview.md @@ -206,7 +206,7 @@ An integration connector that is listening for events from the open metadata eco | [Security Integrator OMIS](/services/omis/security-integrator/overview) | [Security Manager OMAS](/services/omas/security-manager/overview) | [SecurityManagerEventListener](https://odpi.github.io/egeria/org/odpi/openmetadata/accessservices/securitymanager/api/SecurityManagerEventListener.html) | | [Topic Integrator OMIS](/services/omis/topic-integrator/overview) | [Data Manager OMAS](/services/omas/data-manager/overview) | [DataManagerEventListener](https://odpi.github.io/egeria/org/odpi/openmetadata/accessservices/datamanager/api/DataManagerEventListener.html) | -Your integration connector registers itself as a listener in the `start()` method, and the `processEvent()` method is called each time an event occurs. +Your integration connector registers itself as a listener in the `start()` method, and the `processEvent()` method is called each time an event occurs. The event type passed on `processEvent()` depends on the OMIS that the connector is using. In the example, the event type comes from [Asset Manager OMAS](/services/omas/asset-manager/overview) so the connector is either using the [Catalog Integrator OMAS](/servifes/omis/catalog-integrator/overview) or [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview). ```java /** @@ -230,7 +230,28 @@ Your integration connector registers itself as a listener in the `start()` metho } } + /** + * Process an event that was published by the Asset Manager OMAS. This connector is only interested in + * glossaries, glossary categories and glossary terms. The listener is only registered if metadata is flowing + * from the open metadata ecosystem to Apache Atlas. + * + * @param event event object + */ + @Override + public void processEvent(AssetManagerOutTopicEvent event) + { + /* + * Only process events if refresh() is not running because the refresh() process creates lots of events and + * proceeding with event processing at this time causes elements to be processed multiple times. + */ + if (! myContext.isRefreshInProgress()) + { + ... + } + } + ``` +The `isRefreshInProgress()` call is used to ensure this connector ignores events while its `refresh()` is being called. For many connectors, many of the events created during this time are caused by the connector's own activity. Therefore, ignoring events at this time can avoid processing elements multiple times. ### Working with the third party technology diff --git a/site/docs/guides/developer/open-discovery-services/overview.md b/site/docs/guides/developer/open-discovery-services/overview.md index f667010bda..6abb567475 100644 --- a/site/docs/guides/developer/open-discovery-services/overview.md +++ b/site/docs/guides/developer/open-discovery-services/overview.md @@ -113,7 +113,7 @@ public class XXXDiscoveryProvider extends IntegrationConnectorProviderBase /* * Define the name of the connector implementation. */ - private static final Class connectorClass = XXXDiscoveryService.class; + private static final String connectorClassName = "packagename.XXXDiscoveryService"; /* * Define the name of configuration properties (optional). @@ -130,7 +130,7 @@ public class XXXDiscoveryProvider extends IntegrationConnectorProviderBase /* * Set up the class name of the open discovery service that this provider creates. */ - super.setConnectorClassName(connectorClass.getName()); + super.setConnectorClassName(connectorClassName); /* * Set up the connector type that should be included in a connection used to configure this open discovery service. diff --git a/site/docs/guides/developer/runtime-connectors/audit-log-destination-connector.md b/site/docs/guides/developer/runtime-connectors/audit-log-destination-connector.md index bd5df4c98a..dc48edf41c 100644 --- a/site/docs/guides/developer/runtime-connectors/audit-log-destination-connector.md +++ b/site/docs/guides/developer/runtime-connectors/audit-log-destination-connector.md @@ -125,8 +125,8 @@ public class XXXAuditLogDestinationProvider extends OMRSAuditLogStoreProvider /* * Set up the class name of the connector that this provider creates. */ - Class connectorClass = XXXStoreConnector.class; - super.setConnectorClassName(connectorClass.getName()); + String connectorClassName = "packagename.XXXStoreConnector"; + super.setConnectorClassName(connectorClassName); /* * Set up the connector type that should be included in a connection used to configure this connector. diff --git a/site/docs/guides/diagnostic/ffdc.md b/site/docs/guides/diagnostic/ffdc.md index 1dfb8398b1..94ea9c70d3 100644 --- a/site/docs/guides/diagnostic/ffdc.md +++ b/site/docs/guides/diagnostic/ffdc.md @@ -63,6 +63,80 @@ Details of the supported audit log store connectors and how to set them up are d - Exception objects containing stack traces never leave the OMAG Server Platform. The full exception is added to the Audit Log so the stack trace can be analysed by the platform team. Important diagnostic information - such as the exception type, message, system action and user action is captured in the REST response. If the calling program is an Egeria client, it recreates the exception (minus the stack trace of course) and throws it to its caller. That way, all information about the exception is preserved without compromising the security of the server platform. +## Audit log record examples + +Below is an example of an audit log record formatted in JSON. This is the style of output from Egeria's file-based and topic-based audit log destinations. Notice it includes details of the originating server, component and thread. The message description includes information about how to resolve an issue (if there is one to resolve). The message parameters are the values from the runtime that are inserted into the message. They are provided to allow translated messages to be produced for non-English speakers. + +```json +{ + "guid": "f150879d-f807-47ae-a7e6-5d48118044a5", + "timeStamp": 1689528452722, + "originatorProperties": { + "organizationName": null, + "serverType": "Open Metadata and Governance Server", + "serverName": "atlasexchange" + }, + "originatorComponent": { + "componentId": 659, + "componentDevelopmentStatus": "IN_DEVELOPMENT", + "componentName": "Egeria:IntegrationConnector:Catalog:ApacheAtlas:3ab99700-7750-4eca-8670-55b6f21bda14", + "componentDescription": "Connector publishes active glossary terms to Apache Atlas.", + "componentWikiURL": "https://egeria-project.org/connectors/integration/apache-atlas-integration-connector/" + }, + "actionDescription": "processAtlasGlossaryTerm", + "threadId": 185, + "threadName": "atlasexchange::IntegrationDaemonThread", + "severityCode": 5, + "severity": "Error", + "messageId": "APACHE-ATLAS-INTEGRATION-CONNECTOR-0015", + "messageText": "The open metadata glossary term 52ca009f-0fa6-495e-b760-2a5b33f7281f for equivalent Apache Atlas glossary term custName has been unilaterally deleted; connector AtlasIntegrator is putting it back", + "messageParameters": [ + "52ca009f-0fa6-495e-b760-2a5b33f7281f", + "custName", + "AtlasIntegrator" + ], + "systemAction": "The open metadata glossary term can not be retrieved. This glossary term is owned by Apache Atlas. The connector is creating a new copy of the Apache Atlas glossary term in the open metadata ecosystem.", + "userAction": "Open metadata glossary terms that are copies from Apache Atlas should not be unilaterally removed. Investigate why this element is missing from the open metadata ecosystem and make changes so it can not happen again." +} +``` + +This next example shows an *Exception* message. Typically, these are produced (rather than an "Error" audit log record) when the cause and resolution is unknown, and so some expert diagnotics work may be needed to resolve the issue. It includes information of the exception and the stack trace. +```json +{ + "guid": "fa564cc3-699d-4c52-afdf-db4f578baf9e", + "timeStamp": 1689539792943, + "originatorProperties": { + "organizationName": null, + "serverType": "Open Metadata and Governance Server", + "serverName": "atlasexchange" + }, + "originatorComponent": { + "componentId": 659, + "componentDevelopmentStatus": "IN_DEVELOPMENT", + "componentName": "Egeria:IntegrationConnector:Catalog:ApacheAtlas:3ab99700-7750-4eca-8670-55b6f21bda14", + "componentDescription": "Connector publishes active glossary terms to Apache Atlas.", + "componentWikiURL": "https://egeria-project.org/connectors/integration/apache-atlas-integration-connector/" + }, + "actionDescription": "saveAtlasGlossaryCategory(glossaryGUID)", + "threadId": 297, + "threadName": "OpenMetadataTopicListener: egeria.omag.server.cocoMDS1.omas.assetmanager.outTopic", + "severityCode": 6, + "severity": "Exception", + "messageId": "APACHE-ATLAS-INTEGRATION-CONNECTOR-0031", + "messageText": "A client-side exception was received from API call saveAtlasGlossaryCategory(glossaryGUID) to server Apache Atlas at https://cocopharmaceuticals.com:21000. The error message was CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.HttpClientErrorException$Conflict was received by method saveAtlasGlossaryCategory(glossaryGUID) from API call https://cocopharmaceuticals.com:21000/api/atlas/v2/glossary/category/{0} to server Apache Atlas on platform https://cocopharmaceuticals.com:21000. The error message was 409 Conflict: \"{\"errorCode\":\"ATLAS-409-00-00A\",\"errorMessage\":\"Glossary category with qualifiedName Payment Method.Payment Method@Cloud Information Model (CIM) already exists\"}\"", + "messageParameters": [ + "saveAtlasGlossaryCategory(glossaryGUID)", + "Apache Atlas", + "https://cocopharmaceuticals.com:21000", + "CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.HttpClientErrorException$Conflict was received by method saveAtlasGlossaryCategory(glossaryGUID) from API call https://cocopharmaceuticals.com:21000/api/atlas/v2/glossary/category/{0} to server Apache Atlas on platform https://cocopharmaceuticals.com:21000. The error message was 409 Conflict: \"{\"errorCode\":\"ATLAS-409-00-00A\",\"errorMessage\":\"Glossary category with qualifiedName Payment Method.Payment Method@Cloud Information Model (CIM) already exists\"}\"" + ], + "systemAction": "The server has issued a call to the open metadata access service REST API in a remote server and has received an exception from the local client libraries.", + "userAction": "Look for errors in the local server's console to understand and correct the source of the error.", + "exceptionClassName": "org.odpi.openmetadata.adapters.connectors.restclients.ffdc.exceptions.RESTServerException", + "exceptionMessage": "CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.HttpClientErrorException$Conflict was received by method saveAtlasGlossaryCategory(glossaryGUID) from API call https://cocopharmaceuticals.com:21000/api/atlas/v2/glossary/category/{0} to server Apache Atlas on platform https://cocopharmaceuticals.com:21000. The error message was 409 Conflict: \"{\"errorCode\":\"ATLAS-409-00-00A\",\"errorMessage\":\"Glossary category with qualifiedName Payment Method.Payment Method@Cloud Information Model (CIM) already exists\"}\"", + "exceptionStackTrace": "RESTServerException{reportedHTTPCode=503, reportingClassName='org.odpi.openmetadata.adapters.connectors.restclients.spring.SpringRESTClientConnector', reportingActionDescription='saveAtlasGlossaryCategory(glossaryGUID)', errorMessage='CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.HttpClientErrorException$Conflict was received by method saveAtlasGlossaryCategory(glossaryGUID) from API call https://cocopharmaceuticals.com:21000/api/atlas/v2/glossary/category/{0} to server Apache Atlas on platform https://cocopharmaceuticals.com:21000. The error message was 409 Conflict: \"{\"errorCode\":\"ATLAS-409-00-00A\",\"errorMessage\":\"Glossary category with qualifiedName Payment Method.Payment Method@Cloud Information Model (CIM) already exists\"}\"', reportedSystemAction='The client has issued a call to the open metadata access service REST API in a remote server and has received an exception from the local client libraries.', reportedUserAction='Review the error message to determine the cause of the error. Check that the server is running an the URL is correct. Look for errors in the local server's console to understand and correct the cause of the error. Then rerun the request', reportedCaughtException=org.springframework.web.client.HttpClientErrorException$Conflict: 409 Conflict: \"{\"errorCode\":\"ATLAS-409-00-00A\",\"errorMessage\":\"Glossary category with qualifiedName Payment Method.Payment Method@Cloud Information Model (CIM) already exists\"}\"}\n\tat org.odpi.openmetadata.adapters.connectors.restclients.spring.SpringRESTClientConnector.callPutRESTCall(SpringRESTClientConnector.java:568)\n\tat org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasRESTClient.callPutRESTCall(ApacheAtlasRESTClient.java:610)\n\tat org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasRESTClient.saveAtlasGlossaryCategory(ApacheAtlasRESTClient.java:343)\n\tat org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasIntegrationConnector.refreshEgeriaGlossaryCategoryInAtlas(ApacheAtlasIntegrationConnector.java:1585)\n\tat org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasIntegrationConnector.processEgeriaGlossaryCategory(ApacheAtlasIntegrationConnector.java:946)\n\tat org.odpi.openmetadata.adapters.connectors.integration.apacheatlas.ApacheAtlasIntegrationConnector.processEvent(ApacheAtlasIntegrationConnector.java:407)\n\tat org.odpi.openmetadata.accessservices.assetmanager.connectors.outtopic.AssetManagerOutTopicClientConnector.processEvent(AssetManagerOutTopicClientConnector.java:75)\n\tat org.odpi.openmetadata.repositoryservices.connectors.openmetadatatopic.OpenMetadataTopicConnector.distributeEvent(OpenMetadataTopicConnector.java:163)\n\tat org.odpi.openmetadata.repositoryservices.connectors.openmetadatatopic.OpenMetadataTopicConnector.run(OpenMetadataTopicConnector.java:121)\n\tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: org.springframework.web.client.HttpClientErrorException$Conflict: 409 Conflict: \"{\"errorCode\":\"ATLAS-409-00-00A\",\"errorMessage\":\"Glossary category with qualifiedName Payment Method.Payment Method@Cloud Information Model (CIM) already exists\"}\"\n\tat org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:121)\n\tat org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:183)\n\tat org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:137)\n\tat org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)\n\tat org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:915)\n\tat org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:864)\n\tat org.springframework.web.client.RestTemplate.execute(RestTemplate.java:764)\n\tat org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:646)\n\tat org.odpi.openmetadata.adapters.connectors.restclients.spring.SpringRESTClientConnector.callPutRESTCall(SpringRESTClientConnector.java:533)\n\t... 9 more\n" +} +``` ## Exceptions diff --git a/site/docs/guides/diagnostic/overview.md b/site/docs/guides/diagnostic/overview.md index 709da45f6e..d7fc630979 100644 --- a/site/docs/guides/diagnostic/overview.md +++ b/site/docs/guides/diagnostic/overview.md @@ -10,8 +10,7 @@ If you are having problems building Egeria, please see the [Building Egeria](/ed ## The diagnosis process -The OMAG Server Platform has many functions itself, and is typically be embedded in complex deployment environments. This means that diagnosing (and most importantly fixing them) needs to be done in a -systematic way. +The OMAG Server Platform has many functions itself, and is typically be embedded in complex deployment environments. This means that diagnosing problems (and most importantly fixing them) needs to be done in a systematic way. Figure 1 shows a view of the diagnostic process. @@ -24,8 +23,8 @@ So now consider each phase of the process. * **Information Gathering** - The first phase is to gather the information needed to understand the nature of the problem. - * **Clarify the problem** - Create a clear picture in your mind of the symptoms of your problem. Be as precise as possible since that will simplify your work at a later stage. - Ask yourself: + * **Clarify the problem** - Create a clear picture in your mind of the symptoms of your problem. Be as precise as possible since that will simplify your work at a later stage. Ask yourself: + * What is not working? Did something fail? Or is it unresponsive, or creating the wrong results? * Has it ever worked or has it just stopped working? * If it has just stopped working, what has changed recently? @@ -33,8 +32,7 @@ So now consider each phase of the process. * What is the context of the problem? For example, what is the deployment topology in terms of the machines, containers, OMAG Server Platforms, OMAG Services and connected services (such as Apache Kafka). - * **Classifying the problem** - Identifying the type of problem guides you to which diagnostics will be helpful. - For example, consider the following options, and also identify whether this is a consistent or intermittent error. + * **Classifying the problem** - Identifying the type of problem guides you to which diagnostics will be helpful. For example, consider the following options, and also identify whether this is a consistent or intermittent error. * The platform failed to start * The platform failed while it was running @@ -49,58 +47,37 @@ So now consider each phase of the process. * Metadata is not being shared across the cohort * Requests are running slowly - * **Gather diagnostics** - This is where you are looking to capture the evidence that may identify - both the source of the error and the nature of the failure. - The different types of diagnostics that are available are described [in Diagnostic Source](/guides/diagnostic/diagnostic-sources). - If the problem is in the Egeria code, - rather than its runtime environment, then the - [Egeria diagnostics may also defined how to fix the problem](/guides/diagnostic/ffdc). + * **Gather diagnostics** - This is where you are looking to capture the evidence that may identify both the source of the error and the nature of the failure. The different types of diagnostics that are available are described in [Diagnostic Sources](/guides/diagnostic/diagnostic-sources). If the problem is in the Egeria code, rather than its runtime environment, then the [Egeria diagnostics may also define how to fix the problem](/guides/diagnostic/ffdc). -* **Information Integration and Interpretation** - The second phase uses the evidence gathered in the - to make a diagnosis to determine which component is in error and what it is doing that is incorrect. - To do this, you need to: +* **Information Integration and Interpretation** - The second phase uses the evidence gathered in the to make a diagnosis to determine which component is in error and what it is doing that is incorrect. To do this, you need to: + + * **Understand the correct behavior** - how should it be working? If the desired capability is in the Egeria code, what is the status of the module and is the function that you are using complete and expected to work. - * **Understand the correct behavior** - how should it be working? If the desired capability - is in the Egeria code, what is the status of the module and is the function that you are using complete - and expected to work. + * **Understand the actual behavior** - compare and contrast how it is actually working with how it should be working. - * **Understand the actual behavior** - compare and contrast how it is actually working with - how it should be working. + * **Classify the behavior** - identify the nature of the behavior you are seeing. For example: - * **Classify the behavior** - identify the nature of the behavior you are seeing. - For example: * **Not Implemented** - The desired behavior is not yet implemented - or in development. * **Bad Environment** - The OMAG Server Platform's runtime (container) environment is incorrect or failing or lacking resources. * **Bad Platform Configuration** - The OMAG Server Platform is incorrectly configured. - * **Bad Server Configuration** - The OMAG Server is incorrectly configured. This may be its subsystems, or tis connectors. + * **Bad Server Configuration** - The OMAG Server is incorrectly configured. This may be its subsystems, or its connectors. * **Failing dependent service** - A service being called by the platform or a server (for example, Kafka, LDAP) is not running correctly. * **Insufficient security access** - A platform, server or user does not have security authorization to complete a request. * **Caller error** - An error in the caller to Egeria's request * **Bad connector implementation** - An error in the implementation of a connector. * **Egeria platform bug** - one of the subsystems of Egeria is not operating correctly. - With this information, it may be possible to match what is occurring in your environment with - descriptions of common errors, or errors found by other users of Egeria. These will hopefully - also include a description of how to fix the problem. + With this information, it may be possible to match what is occurring in your environment with descriptions of common errors, or errors found by other users of Egeria. These will hopefully also include a description of how to fix the problem. * **Taking Action** - Once the cause of the problem is understood, you next need to fix it. There are three parts to this: - * **Agree necessary changes** - Sometimes the diagnostics describe how to fix the problem, sometimes that are - choices to be agreed with the people responsible for the components/configuration that needs fixing. + * **Agree necessary changes** - Sometimes the diagnostics describe how to fix the problem, sometimes that are choices to be agreed with the people responsible for the components/configuration that needs fixing. * **Make necessary changes** - Once the approach has been agreed, the changes need to be made. - * **Monitor results** - Finally the changes need to be tested to ensure the problem is fixed and - there are no unwanted side effects. If the solution did not work, then begin again with the - information gathering process. It may also be a time to request help from the - [Egeria community](/guides/community). If the situation is worse, you may wish to back out the changes - you have made before trying the diagnosis process again. - -The description above is an overview of what to do to diagnose and fix a problem. -This may be enough to help you resolve the issues you are seeing. -However, whilst some types of issues are frequently encountered, others are unique to your situation. -What follows are some descriptions of the diagnostics for issues that are frequently seen. -These can help you to eliminate the obvious causes. + * **Monitor results** - Finally the changes need to be tested to ensure the problem is fixed and there are no unwanted side effects. If the solution did not work, then begin again with the information gathering process. It may also be a time to request help from the [Egeria community](/guides/community). If the situation is worse, you may wish to back out the changes you have made before trying the diagnosis process again. + +The description above is an overview of what to do to diagnose and fix a problem. This may be enough to help you resolve the issues you are seeing. However, whilst some types of issues are frequently encountered, others are unique to your situation. What follows are some descriptions of the diagnostics for issues that are frequently seen. These can help you to eliminate the obvious causes. * [Examples of OMAG Server Platform startup errors](/guides/diagnostic/examples-of-platform-start-up-errors) * [Examples of OMAG Server startup errors](/guides/diagnostic/examples-of-server-start-up-errors) diff --git a/site/docs/guides/index.md b/site/docs/guides/index.md index cf55fac3e9..95bab2b8a4 100644 --- a/site/docs/guides/index.md +++ b/site/docs/guides/index.md @@ -3,7 +3,7 @@ # Using Egeria -Egeria's guides provide practical guidance on different aspects of using Egeria in your organization. As a technology, Egeria has a lot of features and these are explained along with information about each feature's business value and how to incorporate it in your organization. Usiing Egeria should enable your organization to change the way it works, by improving the collaboration and shared understanding of your data and digital resources. +Egeria's guides provide practical guidance on different aspects of using Egeria in your organization. As a technology, Egeria has a lot of features and these are explained along with information about each feature's business value and how to incorporate it in your organization. Using Egeria should enable your organization to change the way it works, by improving the collaboration and shared understanding of your data and digital resources. --8<-- "snippets/getting-started.md" diff --git a/site/docs/guides/operations/starting-and-stopping-omag-server.md b/site/docs/guides/operations/starting-and-stopping-omag-server.md index 83c691cb51..f09b22e680 100644 --- a/site/docs/guides/operations/starting-and-stopping-omag-server.md +++ b/site/docs/guides/operations/starting-and-stopping-omag-server.md @@ -1,7 +1,7 @@ -# Starting and Stopping the OMAG server +# Starting and Stopping an OMAG server Once a [configuration document](/concepts/configuration-document) has been completed for an [OMAG Server](/concepts/omag-server), it can be started using the following REST call: @@ -18,13 +18,13 @@ DELETE {platformURLRoot}/open-metadata/platform-services/users/{adminUserId}/ser The configuration document is not changed by these calls. It is possible to query the running server's configuration using the following REST API: ``` -GET {platformURLRoot}/open-metadata/admin-services/users/{adminUserId}/servers/{serverName}/instance/configuration +GET {platformURLRoot}/open-metadata/platform-services/users/{adminUserId}/server-platform/servers/{serverName}/instance/configuration ``` If you want to delete the server's configuration document then issue: ``` -DELETE {platformURLRoot}/open-metadata/admin-services/users/{adminUserId}/servers/{serverName} +DELETE {platformURLRoot}/open-metadata/platform-services/users/{adminUserId}/server-platform/servers/{serverName} ``` If the OMAG server is running, this command also unregisters the named server from the cohorts it is connected to. Only use this command if the server is being permanently removed. diff --git a/site/docs/introduction/challenge.md b/site/docs/introduction/challenge.md index 358aed0f7a..0004416b09 100644 --- a/site/docs/introduction/challenge.md +++ b/site/docs/introduction/challenge.md @@ -27,7 +27,7 @@ Our guiding beliefs: --8<-- "docs/introduction/open-metadata-ecosystem.md" -!!! tip "Code talks" - Egeria provides an Apache 2.0 licensed platform to support vendors that sign up to the open metadata and governance manifesto. +!!! tip "What is included in Egeria?" + Egeria provides [Apache 2.0 licensed standards and technology](/introduction/overview) to support the deployment of the open metadata ecosystem. --8<-- "snippets/abbr.md" diff --git a/site/docs/introduction/key-concepts.md b/site/docs/introduction/key-concepts.md index 3a7cfbd3ca..48f5b842b4 100644 --- a/site/docs/introduction/key-concepts.md +++ b/site/docs/introduction/key-concepts.md @@ -1,9 +1,11 @@ -# Key concepts +# Key concepts of the Egeria technology -To further explain some key concepts of Egeria, let us delve deeper into the technology. +The functions needed to support the [open metadata ecosystem](/introduction/challenge/#the-open-metadata-ecosystem) are complex and need to be organized so they can be understood and adapted over time as the needs of your organization changes and grows. There is also an belief that it should be possible to dynamically activate and deactivate capability in the open metadata ecosystem through APIs allowing a self-service and local ownership approach to be adopted. This way, an organization does not need a large IT team to manage the deployment. + +The organization of the components requires some new terminology to be introduced to lable the different types of components, their purpose and how they connect together. The [overview](/introduction/overview) introduced the concept of the [OMAG Server Platform](/concepts/omag-server-platform) deployed multiple types in different processing centres and running [OMAG Servers](/concepts/omag-server). These OMAG Servers are specialized to perform specific functions. Figure 1 shows the different types of OMAG Servers and how they work together. ![Figure 1](egeria-solution-components.svg) > **Figure 1:** This picture shows the different types of Egeria's OMAG servers and how they are connected together in a solution. They are not all required. You choose which ones you need, and how many of them to run, to match the needs of your organization. The servers are organized into three rings. In the inner-ring (labeled *Integrated Metadata*), a Metadata Access Server, Repository Proxy and Conformance Test Server are members of an *Open Metadata Repository Cohort*, or "cohort" for short, communicating via Egeria's peer-to-peer protocols. In the next ring out, called *Integrated Governance*, are the Governance Servers connected to the Metadata Access Server in order to access metadata in the open metadata ecosystem. In the outer ring, called *Governance Solution*, are the View Server and Presentation Server also connected to the Metadata Access Server. diff --git a/site/docs/introduction/open-metadata-ecosystem.md b/site/docs/introduction/open-metadata-ecosystem.md index ace4d15425..4e8beb02ea 100644 --- a/site/docs/introduction/open-metadata-ecosystem.md +++ b/site/docs/introduction/open-metadata-ecosystem.md @@ -4,7 +4,7 @@ ## The open metadata ecosystem -The content of the data/metadata shared between teams needs to follow standards that ensure clarity both in meaning and how it should be used and managed. Its completeness and quality need to be appropriate for the organization's uses. These uses will change over time. +The content of the data/metadata shared between teams needs to follow standards that ensure clarity, both in meaning and how it should be used and managed. Its completeness and quality need to be appropriate for the organization's uses. These uses will change over time. The ecosystem that supplies and uses this data/metadata must evolve and adapt to the changing and growing needs of the organization because trust is required not just for today's operation but also into the future. @@ -12,7 +12,7 @@ You can make your own choices on how to build trust in your data/metadata. Eger * Egeria defines a standard format for storing and distributing metadata. This includes an extendable type system so that any type of metadata that you need can be supported. -* Egeria provides technology to manage, store, distribute this standardized metadata. This technology is inherently distributed, enabling you to work across multiple cloud platforms, data centres and other distributed environments. Collectively, a deployment of this technology is referred to as the *open metadata ecosystem*. +* Egeria provides technology to manage, store, distribute this standardized metadata. This technology is inherently distributed, enabling you to work across multiple cloud platforms, data centres and other distributed environments. Collectively, a deployment of this technology is referred to as the **open metadata ecosystem**. * Egeria provides connector interfaces to allow third party technology to plug into the open metadata ecosystem. These connectors translate metadata from the third party technology's native format to the open metadata format. This allows: diff --git a/site/docs/introduction/overview.md b/site/docs/introduction/overview.md index fdd9e3f1d1..a5833b1021 100644 --- a/site/docs/introduction/overview.md +++ b/site/docs/introduction/overview.md @@ -23,7 +23,7 @@ Although Egeria is shown above as a centralized solution, it is implemented as a Today's organizations have their tools and technologies distributed across multiple data centres and cloud providers (green clouds). Each of these locations may be managed by different groups and use different technologies. This makes a centralized metadata solution impractical from both a funding and operational perspective. -Using Egeria, each location runs a platform (blue boxes) that hosts the integration services tailored to specific types of tools (orange circles). +Using Egeria, each location runs a [platform (blue boxes)](/concepts/omag_server_platform) that hosts [servers](/concepts/omag_server) (orange circles) that provide the synchronization services tailored to specific types of tools (grey boxes). ![Distributed operation](egeria-distributed-operation.svg) diff --git a/site/docs/practices/common-data-definitions/open-metadata-for-common-definitions.md b/site/docs/practices/common-data-definitions/open-metadata-for-common-definitions.md index 900626135a..95fb07be53 100644 --- a/site/docs/practices/common-data-definitions/open-metadata-for-common-definitions.md +++ b/site/docs/practices/common-data-definitions/open-metadata-for-common-definitions.md @@ -1,6 +1,14 @@ +--- +hide: +- toc +--- + + + + # Building Subject Area materials Egeria provides a comprehensive set of [open metadata types](/types) for managing [common data definitions](/practices/common-data-definitions/overview). These types provide a common language and format for exchanging these definitions between tools and metadata repositories. Each tool/repository provides a mapping to the Egeria types and Egeria manages the exchange of metadata between these parties. diff --git a/site/docs/practices/common-data-definitions/overview.md b/site/docs/practices/common-data-definitions/overview.md index 2ddbf18774..572c9a4ee9 100644 --- a/site/docs/practices/common-data-definitions/overview.md +++ b/site/docs/practices/common-data-definitions/overview.md @@ -1,3 +1,9 @@ +--- +hide: +- toc +--- + + @@ -13,7 +19,7 @@ Typically, the common data definitions consist of: * Definitions of the valid data values for specific concepts and associated governance rules. * Preferred logical and physical data formats for storing data about these concepts. -The common data definitions in open metadata are part of the *Subject Area Materials* and are grouped into [subject areas](/concepts/subject-area). The core materials that cover the meaning of data are described in one or more [glossaries](/practices/common-data-definitions/anatomy-of-a-glossary).These are augmented by valid values, reference data and quality rules. +The common data definitions in open metadata are part of the *Subject Area Materials* and are grouped into [subject areas](/concepts/subject-area). The core materials that cover the meaning of data are described in one or more [glossaries](/practices/common-data-definitions/anatomy-of-a-glossary). These are augmented by [valid values](/practices/common-data-definitions/open-metadata-for-common-definitions), [reference data](/practices/common-data-definitions/open-metadata-for-common-definitions) and [quality rules](/practices/common-data-definitions/open-metadata-for-common-definitions). [Governance action classifications](/types/4/0422-Governance-Action-Classifications) may be added to these definitions which in turn link to governance requirements. This determines how data that is linked to the subject area material should be governed. diff --git a/site/docs/release-notes/4-1.md b/site/docs/release-notes/4-1.md index ad022882da..9c7386c680 100644 --- a/site/docs/release-notes/4-1.md +++ b/site/docs/release-notes/4-1.md @@ -29,7 +29,7 @@ ??? functional "HMS listener" The [Hive Metastore (HMS) listener](/connectors/repository/hms/overview/#using-with-the-hive-metastore-listener) runs in the HMS process, - and issues Egerie Events when HMS tables and added, dropped or altered. The git repository for the listener is [https://github.com/odpi/egeria-listener-hms](https://github.com/odpi/egeria-listener-hms). + and issues Egeria Events when HMS tables and added, dropped or altered. The git repository for the listener is [https://github.com/odpi/egeria-listener-hms](https://github.com/odpi/egeria-listener-hms). ??? functional "Other functional fixes and enhancements" diff --git a/site/docs/release-notes/4-2.md b/site/docs/release-notes/4-2.md index 67bce888dc..b99e1985fb 100644 --- a/site/docs/release-notes/4-2.md +++ b/site/docs/release-notes/4-2.md @@ -2,18 +2,41 @@ -## Release 4.2 (expected June 2023) +## Release 4.2 (July 2023) -_**These are DRAFT release notes, initially copied from the previous release -The notes will be updated until & when 4.2 is released**_ +Release 4.2 has a focus on runtime optimization and the start of the work to optimize the process of getting Egeria up and running in a new environment. This theme is continued in release 4.3. -??? functional "Open Metadata Types" - * The [*CollectionMembership*](/types/0/0021-Collections) relationship has new properties *stewardTypeName*, *stewardPropertyName*, *userDefinedStatus* and *notes* to improve ability to manage collection membership. +??? functional "OMAG Server Topic Configuration" + Except when the `no-topics` option is specified, the configuration document helper methods add a connection for both the [InTopic](/concepts/in-topic) and the [OutTopic](/concepts/out-topic) to each Open Metadata Access Service (OMAS) configuration irrespective of whether the particular OMAS supports these topics. This has no impact at runtime because the OMASs simply do not start topics they do not use. However, from a set-up point of view, it is misleading - and can result in an organization thinking they need to set up (and may be pay for) topics that are not use. + + This release adds knowledge of whether an OMAS supports the inTopic and/or OutTopic to the [AccessServiceDescroption](https://github.com/odpi/egeria/blob/main/open-metadata-implementation/admin-services/admin-services-api/src/main/java/org/odpi/openmetadata/adminservices/configuration/registration/AccessServiceDescription.java) class. This information is used by the configuration helper methods when setting up the access service configuration. For example, this is the list of topics for `cocoMDS1` at release 4.2: + + ``` + List of Topics used by cocoMDS1 + egeria.omag.server.cocoMDS1.omas.assetcatalog.outTopic + egeria.omag.server.cocoMDS1.omas.assetconsumer.outTopic + egeria.omag.server.cocoMDS1.omas.assetmanager.outTopic + egeria.omag.server.cocoMDS1.omas.assetowner.outTopic + egeria.omag.server.cocoMDS1.omas.communityprofile.outTopic + egeria.omag.server.cocoMDS1.omas.dataengine.inTopic + egeria.omag.server.cocoMDS1.omas.datamanager.outTopic + egeria.omag.server.cocoMDS1.omas.digitalarchitecture.outTopic + egeria.omag.server.cocoMDS1.omas.governanceengine.outTopic + egeria.omag.server.cocoMDS1.omas.assetlineage.outTopic + ``` + + Notice Data Engine OMAS supports an InTopic only. All of the others support just an OutTopic. Prior to this change, each OMAS had both an InTopic and an OutTopic configured. ??? functional "OMAG Server Platform" * Fixed bug with http headers propagation feature. * The name of the JAR file to start the platform is now `omag-server-platform.jar` in the `platform` directory of the distribution/install image. The previous `server-chassis-spring.jar` is still present in the `server` directory, to allow time for teams to migrate their operational scripts. However, this old jar file name is deprecated and will be removed in a future release. * The REST API calls to start and stop OMAG Servers have been moved from the [Administration Services](/services/admin-services/overview) to the [Platform Services](/services/platform-services/overview). The original Java client and REST API calls are still available, but they have been deprecated and will be removed at a later date. This change is to separate the operational responsibilities from the configuration responsibilities. Prior to this change, the operational responsibilities where split between the Platform Services and the Administration Services. +??? functional "New My Profile Open Metadata View Services (OMVS)" + In this release we have added a new view service called [My Profile OMVS](/services/omvs/my-profile/overview) to aid organizations writing user interfaces (UIs). My Profile OMVS returns information about the supplied user's profile. It is intended to supply information about the logged-on user. + +??? functional "Open Metadata Types" + * The [*CollectionMembership*](/types/0/0021-Collections) relationship has new properties *stewardTypeName*, *stewardPropertyName*, *userDefinedStatus* and *notes* to improve ability to manage collection membership. + --8<-- "snippets/abbr.md" diff --git a/site/docs/release-notes/4-3.md b/site/docs/release-notes/4-3.md new file mode 100644 index 0000000000..d97f5b417a --- /dev/null +++ b/site/docs/release-notes/4-3.md @@ -0,0 +1,77 @@ + + + +## Release 4.3 (expected September 2023) + +_**These are DRAFT release notes. The notes will be updated until & when 4.3 is released**_ + +Release 4.3 continues the process of improving Egeria's usability. These changes have some effect on backwards compatibility but will lead to a smoother experience in the long term. + +??? warning "Change to the original Open Metadata Assembly" + When the gradle build runs in the core egeria git repository (egeria.git), it creates an assembly of the Egeria runtimes. This is located in the `build` directory under `open-metadata-distributions/open-metadata-assemblies`. In release 4.2, this assembly was updated to create a new directory called `platform` which held the runtime and libraries for the OMAG Server Platform. The docker image for Egeria (and hence the helm charts) was updated so that it runs the platform from this new directory. The platform runtime was originally located in a directory called `server` which caused confusion. The content of the `server` directory was not changed at that time. However, in 4.3, the `server` directory now contains a new cloud native runtime for a single OMAG Server. This is the structure you will see in 4.3. + + ![New assembly structure](structure-of-open-metadata-assemblies.png) + +??? functional "New assemblies for the OMAG Server Platform and Egeria UI Platform" + The single assembly described above suffered from conflicts of runtime settings between the different runtimes. The result was a complex and less than obvious organization of the files. There are new assemblies created during the core egeria build that are focused on a single runtime. Their structure is simpler, making it more obvious which files/properties are controlling each runtime. These assemblies are located under `open-metadata-distribution`. Each assembly is created by its own module. The `egeria-ui-application` module builds the an assembly for the Egeria UI Application. This includes the certificates and property files used by this application. The OMAG Server Platform assembly is created by a module called `omag-server-platform`. It includes three main directories: + + * `platform` - The platform runtime. + * `etc` - Utilities and reports. + * `opt` - Optional content including sample metadata (loadable as an Open Metadata Archive), sample data, sample applications that call the Egeria APIs and postman collections + + The top level directory also includes a docker build configuration that packages files under the `platform` directory into a docker container and publishes it to your docker daemon. Once egeria's grable build is complete, you can add and remove libraries and other content from the `platform` directory and then build your own customized docker container. + + Each directory in the assembly includes a README.md file to explain its contents. + +??? warning "Database connectors are now in egeria.git" + The JDBC database connectors are now located in the core Egeria git repository. The [JDBC Resource Connector](/connectors/resource/jdbc-resource-connector) now includes helper functions when issuing simple SQL commands. + +??? warning "Developer utilities are now in egeria.git" + The utilities located in `egeria-dev-projects.git` are now located in `egeria.git` under the `open-metadata-resources` module. They are now included automatically in the `omag-server-platform` assembly. These utilities are used in the [Developer Dojo](/education/egeria-dojo/developer/overview). This will be updated to use the new assembly once release 4.3 is shipped. + +??? functional "Extended integration connector function" + [Integration connectors](/concepts/integration-connector) can now check to see if they are in the middle of refresh() processing. This is valuable in multi-threaded operation, for example when processing events. Typically many of the events that are generated while the connector is being called to `refresh()` are caused by the connector's activity. There is a new method on the integration context called [`isRefreshInProgress()`](https://odpi.github.io/egeria/org/odpi/openmetadata/frameworks/integration/context/IntegrationContext.html). It returns true if the connector is currently running its `refresh()` method. An example of how to use this method is found in the [developer guide](/guides/developer/integration-connectors/overview/#registering-a-listener-with-open-metadata). + +??? functional "New configuration command for when a cohort member moves platform" + Servers that are [members of a cohort](/concepts/cohort-member) regularly broadcast their network address to other members to allow them to dynamically configure their federated query support. This network address is configured in the OMAG Server's [configuration document](/concepts/configuration-document) as the `localRepositoryRemoteConnection`'s endpoint address. When an OMAG Server is relocated to a new platform, this address must change. There is a new administration command called [`resetRemoteCohortURL`]() with URL `POST {platformURLRoot}/open-metadata/admin-services/users/{userId}/servers/{serverName}/local-repository/configuration/remote-repository-connector-url`. The new URL is passed in the request body: + + ```json + { + "class": "URLRequestBody", + "urlRoot": "{platformURLRoot}/servers/{serverName}" + } + ``` + +??? functional "Open Metadata Integration Services (OMISs) updates" + * [Category Integrator OMIS](/services/omis/catalog-integrator/overview) supports the stewardship services from [Asset Manager OMAS](/services/omas/asset-manager/overview) through the [`StewardshipExchangeService`](https://odpi.github.io/egeria/org/odpi/openmetadata/integrationservices/catalog/connector/StewardshipExchangeService.html). + * A number of fixes have been made to the context methods of the OMISs that were incorrectly exposing the `userId`, `externalSourceGUID` and `externalSourceName` parameters. These have been corrected so these values come from the connector's configuration. The affected OMISs are: + + * [Catalog Integrator OMIS](/services/omis/catalog-integrator/overview) + * [Display Integrator OMIS](/services/omis/display-integrator/overview) + * [Infrastructure Integrator OMIS](/services/omis/infrastructure-integrator/overview) + * [Lineage Integrator OMIS](/services/omis/lineage-integrator/overview) + * [Security Integrator OMIS](/services/omis/security-integrator/overview) + +??? functional "Apache Atlas Integration Connector" + The [Apache Atlas Integration Connector](/connectors/integration/apache-atlas-catalog-integration-connector) has been enhanced to catalog Assets and Lineage from Apache Atlas into the open metadata ecosystem. + + A future version will maintain [semantic assignment](/types/3/0370-Semantic-Assignment) relationships between the glossary terms, exchanged in the first release of the connector, with the Apache Atlas assets it finds. + + Release 4.3 also extracted the java client from this connector and created a new [resource connector](/concepts/digital-resource-connector) for Apache Atlas. The resulting [Apache Atlas REST Connector](/connectors/resource/apache-atlas-rest-connector) provides a Java API that calls the Apache Atlas REST API with no dependencies on Apache Atlas nor the Hadoop libraries it depends on. This resource connector is used in the Apache Atlas Integration Connector and is intended for new connectors for Apache Atlas planned to be written for a future release. + +??? functional "Conformance Test Suite" + There is a new option on the repository workbench to allow you to restrict the number of different types of instances that are tested. By default, all combinations of the types supported by the technology under test, are used when testing the instance methods. For native repository connectors, that can add up to a lot of tests, taking many hours to run. In 4.3, we have introduced a new option called *testEntityTypes* that is [added to the workbench configuration](/guides/admin/servers/configuring-a-conformance-test-server). It is set to a list of entity types such as `["Asset", "InformalTag"]` and the tests are restricted to these types, their super types and any relationships and classifications that can connector to these entity types. + +??? functional "Metadata Observability" + There is a new [Audit log severity called ACTIVITY](/concepts/audit_log) that is used to report activity in the open metadata ecosystem. This severity is used when logging messages about: + + * The REST API calls made by each user. + * The requests to retrieve, create and maintain assets. + +??? functional "Open Metadata Types" + * The [*HostClusterMember*](/types/0/0035-Complex-Hosts) relationship has new properties *memberRole* and *additionalProperties* to improve ability to describe a host cluster operation. + * The [*DataSet*](/types/0/0010-Base-Model), [*DataStore*](/types/2/0210-Data-Stores), [*DeployedAPI*](/types/2/0212-Deployed-APIs), [*DeployedSoftwareComponent*](/types/2/0215-Software-Components) and [*DataFeed*](/types/2/0223-Events-and-Logs) entity types each have a new attribute called *deployedImplementationType* to describe the technology supporting the associated component. This addition is to enable integration connectors to be more precise in describing the technology they are cataloguing. This attribute is also used in [*SoftwareServerPlatform*](/types/0037-Software-Server-Platforms), [*SoftwareServer*](/types/0/0040-Software-Servers) and [*SoftwareCapability*](/types/0/0042-Software-Capabilities) entity types; along with the [ServerPurpose](/types/0/0041-Server-Purpose) and [*CloudPlatform*](/types/0/0090-Cloud-Platforms-and-Services) classification. There are subtypes of *DataStore* (*LogFile*, *GraphStore*, *MetadataRepository*) have been modified because they already include the property and duplicate properties in the type hierarchy cause indexing problems in some repositories. Similarly, there were duplicated *deployedImplementationType* properties in the [*ServerPurpose*](/types/0/0041-Server-Purpose) classification affecting *RepositoryProxy* and *MetadataServer* classifications. + * A new entity type called [*RelationalDatabase*](/types/2/0224-Databases) has been added to distinguish databases that follow the relational schema and can be accessed through JDBC. + * The [*Person*](/types/1/0112-People) entity type has a new attribute called *residentCountry* to describe the primary country of residence. In addition, there is a new attribute called *timeZone* to describe the time zone of the location where this person is located. + +--8<-- "snippets/abbr.md" diff --git a/site/docs/release-notes/april-2023.md b/site/docs/release-notes/april-2023.md index ef52728603..9b29ddf441 100644 --- a/site/docs/release-notes/april-2023.md +++ b/site/docs/release-notes/april-2023.md @@ -61,7 +61,7 @@ The NEW Egeria UI has been shipped as part of the v4 release package. ![EgeriaUIv4](new-ui-look.png) ->We are working on updating the documnetation and lab notebooks where UI is used. These updates and some additional usability improvements will go into the next Egeria release. +>We are working on updating the documentation and lab notebooks where UI is used. These updates and some additional usability improvements will go into the next Egeria release. ## Cloud-native workgroup diff --git a/site/docs/release-notes/august-2023.md b/site/docs/release-notes/august-2023.md new file mode 100644 index 0000000000..82d15c3e6f --- /dev/null +++ b/site/docs/release-notes/august-2023.md @@ -0,0 +1,155 @@ + + + +# August 2023 + +This is August's report from the Egeria community. Since our [last newsletter](/release-notes/april-2023), our focus has been on improving Egeria's usability around: + +* The steps to get Egeria up and running. +* Setting up different flavours of metadata repository, and their persistence stores. +* Writing integration connectors. + +There have also been some important functional improvements including: + +* Support for controlled glossary development (glossary workflows). +* An Apache Atlas integration connector. +* A specialized connector for managing secrets. +* A new runtime for cloud-native operation. + +Finally, there have been improvements in runtime efficiency. + +## New Egeria Releases + +Since the last newsletter, there have been two releases of core Egeria (4.1 and 4.2) and the function for release 4.3 is present in the `main` branch. The release notes detail the specific changes to the code. The sections that follow put those changes in context. + +* [Release notes for 4.1 (May 2023)](/release-notes/4_1) +* [Release notes for 4.2 (July 2023)](/release-notes/4_2) +* [Candidate release notes for 4.3 (expected September 2023)](/release-notes/4_3) + +## Updates to the assemblies + +When the gradle build runs in the core egeria git repository (egeria.git), it creates an assembly of the Egeria runtimes. This is located in the `build` directory under `open-metadata-distributions/open-metadata-assemblies`. + +In release 4.2, this assembly was updated to create a new directory called `platform` which held the runtime and libraries for the OMAG Server Platform. The name of the JAR file to start the platform in the `platform` directory is `omag-server-platform.jar`. + +The docker image for Egeria (and hence the helm charts) was updated so that it runs the platform from this new directory. + +The platform runtime was originally located in a directory called `server` which caused confusion. The content of the `server` directory was not changed at that time, so `server-chassis-spring.jar` was still present in the `server` directory, to allow time for teams to migrate their operational scripts. + +However, in 4.3, the `server` directory now contains a new cloud native runtime for a single OMAG Server. This is the structure you will see in 4.3. + +![New assembly structure](structure-of-open-metadata-assemblies.png) + +The purpose of renaming and reorganizing this assembly is to ensure the naming of the jar files is consistent with the runtimes they represent. However, despite these improvement, there are more fundamental problems with the single assembly. It suffered from conflicts of settings (mainly related to `application.properties`) between the different runtimes. The result was a complex and less than obvious organization of the files in order to remove the conflicts. + +In 4.3 we are introducing two new assemblies that are each focused on a single runtime. Their structure is simpler, making it more obvious which files/properties are controlling each runtime. These assemblies are located under `open-metadata-distribution`. Each assembly is created by its own module. + +The `egeria-ui-application` module builds the an assembly for the Egeria UI Application. This includes the certificates and property files used by this application. + +![Egeria UI Assembly](egeria-ui-assembly.png) + +The OMAG Server Platform assembly is created by a module called `omag-server-platform`. It includes three main directories: + +* `platform` - The platform runtime. +* `etc` - Utilities and reports. +* `opt` - Optional content including sample metadata (loadable as an Open Metadata Archive), sample data, sample applications that call the Egeria APIs and postman collections + +The top level directory also includes a docker build configuration that packages files under the `platform` directory into a docker container and publishes it to your docker daemon. Once egeria's grable build is complete, you can add and remove libraries and other content from the `platform` directory and then build your own customized docker container. + +Each directory in the assembly includes a README.md file to explain its contents. + +![OMAG Server Platform Assembly](omag-server-platform-assembly.png) + +## Controlled glossary development (glossary workflows) + +[Glossary terms](/practices/common-data-definitions/anatomy-of-a-glossary) provide important definitions within an organization's governance programs. They help to clarify the terminology being used and identify where data with a certain meaning is located. As a result, organizations often want to operate a collaborative process for updating their glossary terms. Anticipating this, Egeria's original type system included the [ControlledGlossaryTerm](/types/3/0385-Controlled-Glossary-Development#controlledglossaryterm-entity) that had additional statuses beyond ACTIVE to support a complex development lifecycle. In release 4.1 this was extended with [new types](/types/3/0385-Controlled-Glossary-Development) and [services](/services/omvs/glossary-workflow/overview) to control the visibility of glossary terms as they are developed. + +## Extended integration connector function (4.3) + +[Integration connectors](/concepts/integration-connector) can now check to see if they are in the middle of refresh() processing. This is valuable in multi-threaded operation, for example when processing events. Typically many of the events that are generated while the connector is being called to `refresh()` are caused by the connector's activity. There is a new method on the integration context called [`isRefreshInProgress()`](https://odpi.github.io/egeria/org/odpi/openmetadata/frameworks/integration/context/IntegrationContext.html). It returns true if the connector is currently running its `refresh()` method. An example of how to use this method is found in the [developer guide](/guides/developer/integration-connectors/overview/#registering-a-listener-with-open-metadata). + + +## Improved use of Apache Kafka (4.2) + +Except when the `no-topics` option is specified, the configuration document helper methods add a connection for both the [InTopic](/concepts/in-topic) and the [OutTopic](/concepts/out-topic) to each Open Metadata Access Service (OMAS) configuration irrespective of whether the particular OMAS supports these topics. This has no impact at runtime because the OMASs simply do not start topics they do not use. However, from a set-up point of view, it is misleading - and can result in an organization thinking they need to set up (and may be pay for) topics that are not used. + +This release adds knowledge of whether an OMAS supports the inTopic and/or OutTopic to the [AccessServiceDescroption](https://github.com/odpi/egeria/blob/main/open-metadata-implementation/admin-services/admin-services-api/src/main/java/org/odpi/openmetadata/adminservices/configuration/registration/AccessServiceDescription.java) class. This information is used by the configuration helper methods when setting up the access service configuration. For example, this is the list of topics for `cocoMDS1` at release 4.2: + + ``` + List of Topics used by cocoMDS1 + egeria.omag.server.cocoMDS1.omas.assetcatalog.outTopic + egeria.omag.server.cocoMDS1.omas.assetconsumer.outTopic + egeria.omag.server.cocoMDS1.omas.assetmanager.outTopic + egeria.omag.server.cocoMDS1.omas.assetowner.outTopic + egeria.omag.server.cocoMDS1.omas.communityprofile.outTopic + egeria.omag.server.cocoMDS1.omas.dataengine.inTopic + egeria.omag.server.cocoMDS1.omas.datamanager.outTopic + egeria.omag.server.cocoMDS1.omas.digitalarchitecture.outTopic + egeria.omag.server.cocoMDS1.omas.governanceengine.outTopic + egeria.omag.server.cocoMDS1.omas.assetlineage.outTopic + ``` + +Notice Data Engine OMAS supports an InTopic only. All of the others support just an OutTopic. Prior to this change, each OMAS had both an InTopic and an OutTopic configured. + +## New and enhanced connectors + +The following connectors are now available: + +### Secrets Store Connectors (4.1) + +The [Secrets Store Connector](/concepts/secrets-store-connector){target=blank} is a new type of connector that enables a connector to access its secrets (passwords, certificates, ...) from a secure location at runtime. There is an implementation of this new type of connector that works with environment variables. See the [connector catalog](/connectors/secrets/environment-variable-secrets-store-connector){target=blank} for more details. + +### Apache Atlas Connectors (4.1+) + +Release 4.1 added an [Apache Atlas](/connectors/integration/apache-atlas-catalog-integration-connector){target=blank} integration connector that is able to publish active glossary terms to one or more glossaries in Apache Atlas. This has been enhanced in release 4.3 to catalog the assets and lineage from Apache Atlas. Release 4.3 also extracted the java client from this connector and created a new [resource connector](/concepts/digital-resource-connector) for Apache Atlas. The resulting [Apache Atlas REST Connector](/connectors/resource/apache-atlas-rest-connector) provides a Java API that calls the Apache Atlas REST API with no dependencies on Apache Atlas nor the Hadoop libraries it depends on. This resource connector is used in the Apache Atlas Integration Connector and is intended for new connectors for Apache Atlas planned to be written for a future release. + +### Server Metadata Security Connector (4.1) + +The [Server Metadata Security Connector](/concepts/server-metadata-security-connector){target=blank} has been extended with a new optional interface that allows the connector to control authorization requests to glossaries. See the [Metadata Security](/features/metadata-security/overview){target=blank} feature for more details. + +## View services + +View services are designed to provide REST APIs to UIs. In release 4.1 we added the following view services to aid organizations writing user interfaces (UIs): + +* [Glossary Browser OMVS](#glossary-browser-omvs) +* [Glossary Workflow OMVS](#glossary-workflow-omvs) + +These view services are supported by enhancements to the [Asset Manager OMAS](/services/omas/asset-manager/overview){target=blank}. + +In release 4.2 we added another view service called [My Profile OMVS](#my-profile-omvs). + +### Glossary Workflow OMVS + +Glossary Workflow OMVS supports the development of glossaries using a controlled workflow process +[... more information](/services/omvs/glossary-workflow/overview){target=blank}. + +### Glossary Browser OMVS + +Glossary Browser OMVS supports the searching and navigation through a published glossary plus the ability to add feedback (comments, likes and reviews) to its content. +[... more information](/services/omvs/glossary-browser/overview){target=blank}. + +### My Profile OMVS + +My Profile OMVS returns information about the supplied user's profile. It is intended to supply information about the logged-on user. In future, it will be enhanced with support for the user's roles and resources. This can be used to link to other UI functions that are relevant to the user's roles. My Profile OMVS is built on the [Community Profile OMAS](/services/omas/community-profile/overview){target=blank}. +[... more information](/services/omvs/my-profile/overview) + +## Cloud-native workgroup + +The cloud-native workgroup has created a new single server runtime for Kubernetes. This will be shipped in release 4.3. + +You can find the workgroup updates on the [wiki page](https://wiki.lfaidata.foundation/display/EG/2023+2Q+update){target=blank} + +## New documentation pages + +Each month we will select one or more new pages that where added/updated in the last month to [https://egeria-project.org](https://egeria-project.org){target=blank} so you can keep up-to-date with how it is evolving. + +- [How to use Egeria's new UI](/user-interfaces/general/overview){target=blank} +- [Configuring the OMAG Server Platform](/guides/admin/configuring-the-omag-server-platform/){target=blank} + + +## Connecting with the project + +!!! info "Connecting with the project" + Go to our [community guide](/guides/community){target=blank} to find out how to find out more about the activities of the Egeria project. + +--8<-- "snippets/abbr.md" diff --git a/site/docs/release-notes/egeria-ui-assembly.png b/site/docs/release-notes/egeria-ui-assembly.png new file mode 100644 index 0000000000..412271e271 Binary files /dev/null and b/site/docs/release-notes/egeria-ui-assembly.png differ diff --git a/site/docs/release-notes/latest.md b/site/docs/release-notes/latest.md index 900d4b314f..f18a689f8c 100644 --- a/site/docs/release-notes/latest.md +++ b/site/docs/release-notes/latest.md @@ -1 +1 @@ ---8<-- "docs/release-notes/4-1.md" \ No newline at end of file +--8<-- "docs/release-notes/4-2.md" \ No newline at end of file diff --git a/site/docs/release-notes/next.md b/site/docs/release-notes/next.md index 1822b2e19b..2e6b72ee61 100644 --- a/site/docs/release-notes/next.md +++ b/site/docs/release-notes/next.md @@ -1 +1 @@ ---8<-- "docs/release-notes/4-2.md" +--8<-- "docs/release-notes/4-3.md" diff --git a/site/docs/release-notes/omag-server-platform-assembly.png b/site/docs/release-notes/omag-server-platform-assembly.png new file mode 100644 index 0000000000..7ca785b59c Binary files /dev/null and b/site/docs/release-notes/omag-server-platform-assembly.png differ diff --git a/site/docs/release-notes/overview.md b/site/docs/release-notes/overview.md index f7bb6defe1..92a0646c96 100644 --- a/site/docs/release-notes/overview.md +++ b/site/docs/release-notes/overview.md @@ -5,7 +5,7 @@ Egeria is a large and active project with teams advancing its function on multiple fronts. Our work is split across [multiple git repositories](https://egeria-project.org/education/tutorials/git-and-git-hub-tutorial/egeria-git-repositories/) that supply components on different release schedules. Therefore, we aim to provide an update on our activities and releases in our monthly newsletter. -The [roadmap for Egeria](/release-notes/roadmap) describes the end vision for Egeria and our current status. +The [roadmap for Egeria](/release-notes/roadmap) describes the end vision for Egeria and our current status. !!! tip "Feedback" One way you can help us is to feedback on your experiences, both good and bad. We would love to hear from you! @@ -18,7 +18,21 @@ The [roadmap for Egeria](/release-notes/roadmap) describes the end vision for Eg The *Core Egeria* technology is located in the [egeria.git](https://github.com/odpi/egeria) repository. It provides the [OMAG Server Platform](/concepts/omag-server-platform), java clients for the [OMAG Servers](/concepts/omag-server) that run on the OMAG Server Platform along with the [Conformance Test Suite (CTS)](/guides/cts/overview). -The project aims to produce a new release of Core Egeria about once a month. Each release includes new features and fixes to existing function. +The project aims to produce a new release of Core Egeria about once every 6 weeks. Each release includes new features and fixes to existing function. This is also an opportunity to update the levels of our dependencies to reduce vulnerabilities. + +The table below shows the planned dates for future releases. The branch date is the date that a copy of the main branch is taken. This copy is the candidate code for the release. We then run tests on this copy and release to maven central once any issues have been resolved. Any fixes that are necessary are made to the main branch and then back-ported to the release branch. It typically takes a week to go from making the release branch to the new release going to maven central. + +| Release | Branch date | Release Date | Core theme | +|---------|--------------------|----------------|---------------------------| +| 4.2 | 17th July 2023 | 20th July 2023 | Runtime optimization | + | 4.3 | 4th September 2023 | | New assemblies | +| 4.4 | 23rd October 2023 | | Apache Atlas connector(s) | +| 4.5 | 4th December 2023 | | OpenLineage update | +| 4.6 | 22nd January 2024 | | | +| 4.7 | 11th March 2024 | | | +| 4.8 | 20th May 2024 | | | +| 4.9 | 1st July 2024 | | | +| 4.10 | 2nd September 2024 | | | !!! attention "Backwards compatibility" The team aims to provide complete backward compatibility for all components that are officially released[^1]. diff --git a/site/docs/release-notes/previous.md b/site/docs/release-notes/previous.md index c279cc75a8..a63e8e1eeb 100644 --- a/site/docs/release-notes/previous.md +++ b/site/docs/release-notes/previous.md @@ -4,56 +4,61 @@ # Earlier releases +???+ info "Release 4.1 (May 2023)" + --8<-- "docs/release-notes/4-1.md" + ???+ info "Release 4.0 (March 2023)" --8<-- "docs/release-notes/4-0.md" -???+ info "Release 3.15 (January 2023)" - --8<-- "docs/release-notes/3-15.md" - -???+ info "Release 3.14 (December 2022)" - --8<-- "docs/release-notes/3-14.md" - -???+ info "Release 3.13 (November 2022)" - --8<-- "docs/release-notes/3-13.md" - -??? info "Release 3.12 (October 2022)" - --8<-- "docs/release-notes/3-12.md" - -??? info "Release 3.11 (September 2022)" - --8<-- "docs/release-notes/3-11.md" - -??? info "Release 3.10 (July 2022)" - --8<-- "docs/release-notes/3-10.md" - -??? info "Release 3.9 (June 2022)" - --8<-- "docs/release-notes/3-9.md" - -??? info "Release 3.8 (May 2022)" - --8<-- "docs/release-notes/3-8.md" - -??? info "Release 3.7 (April 2022)" - --8<-- "docs/release-notes/3-7.md" - -??? info "Release 3.6 (March 2022)" - --8<-- "docs/release-notes/3-6.md" - -??? info "Release Release 3.5 (February 2022)" - --8<-- "docs/release-notes/3-5.md" - -??? info "Release 3.4 (December 2021)" - --8<-- "docs/release-notes/3-4.md" - -??? info "Release 3.3 (November 2021)" - --8<-- "docs/release-notes/3-3.md" - -??? info "Release 3.2 (October 2021)" - --8<-- "docs/release-notes/3-2.md" - -??? info "Release 3.1 (September 2021)" - --8<-- "docs/release-notes/3-1.md" - -??? info "Release 3.0 (August 2021)" - --8<-- "docs/release-notes/3-0.md" +??? info "v3.x Releases" + + ???+ info "Release 3.15 (January 2023)" + --8<-- "docs/release-notes/3-15.md" + + ???+ info "Release 3.14 (December 2022)" + --8<-- "docs/release-notes/3-14.md" + + ???+ info "Release 3.13 (November 2022)" + --8<-- "docs/release-notes/3-13.md" + + ??? info "Release 3.12 (October 2022)" + --8<-- "docs/release-notes/3-12.md" + + ??? info "Release 3.11 (September 2022)" + --8<-- "docs/release-notes/3-11.md" + + ??? info "Release 3.10 (July 2022)" + --8<-- "docs/release-notes/3-10.md" + + ??? info "Release 3.9 (June 2022)" + --8<-- "docs/release-notes/3-9.md" + + ??? info "Release 3.8 (May 2022)" + --8<-- "docs/release-notes/3-8.md" + + ??? info "Release 3.7 (April 2022)" + --8<-- "docs/release-notes/3-7.md" + + ??? info "Release 3.6 (March 2022)" + --8<-- "docs/release-notes/3-6.md" + + ??? info "Release Release 3.5 (February 2022)" + --8<-- "docs/release-notes/3-5.md" + + ??? info "Release 3.4 (December 2021)" + --8<-- "docs/release-notes/3-4.md" + + ??? info "Release 3.3 (November 2021)" + --8<-- "docs/release-notes/3-3.md" + + ??? info "Release 3.2 (October 2021)" + --8<-- "docs/release-notes/3-2.md" + + ??? info "Release 3.1 (September 2021)" + --8<-- "docs/release-notes/3-1.md" + + ??? info "Release 3.0 (August 2021)" + --8<-- "docs/release-notes/3-0.md" ??? info "v2.x Releases" diff --git a/site/docs/release-notes/structure-of-open-metadata-assemblies.png b/site/docs/release-notes/structure-of-open-metadata-assemblies.png new file mode 100644 index 0000000000..53de38549e Binary files /dev/null and b/site/docs/release-notes/structure-of-open-metadata-assemblies.png differ diff --git a/site/docs/services/gaf-metadata-management.md b/site/docs/services/gaf-metadata-management.md index d39d2063f2..c068ac8bdc 100644 --- a/site/docs/services/gaf-metadata-management.md +++ b/site/docs/services/gaf-metadata-management.md @@ -10,13 +10,7 @@ hide: # GAF Metadata Management -GAF metadata management provides the *Metadata Store Services* that are -defined by the [Governance Action Framework (GAF)](/frameworks/gaf/overview) and are available through all -[Open Metadata Access Services (OMASs)](/services/omas). - -GAF metadata management also provides a comprehensive interface for working with all types of metadata, subject to the user's (and the associated OMAS's) security permissions. -the interface supports search, maintenance of metadata elements, classifications and relationships plus the ability to raise incident reports -and todos along with the ability to work with metadata valid values and translations. - +GAF metadata management provides the context services +defined by the [Governance Action Framework (GAF)](/frameworks/gaf/overview). --8<-- "snippets/abbr.md" diff --git a/site/docs/services/oms-metadata-management.md b/site/docs/services/oms-metadata-management.md new file mode 100644 index 0000000000..39ca4190ac --- /dev/null +++ b/site/docs/services/oms-metadata-management.md @@ -0,0 +1,22 @@ +--- +hide: +- toc +--- + + + + +--8<-- "snippets/content-status/stable.md" + +# Open Metadata Store (OMS) Metadata Management + +OMS metadata management provides the *Metadata Store Services* that are +consumed by the [Open Connector Framework (OCF)](/frameworks/ocf/overview), [Open Integration Framework (OIF)](/frameworks/oif/overview), [Open Discovery Framework (ODF)](/frameworks/odf/overview), [Governance Action Framework (GAF)](/frameworks/gaf/overview) and are available through all +[Open Metadata Access Services (OMASs)](/services/omas). + +OMS metadata management also provides a comprehensive interface for working with all types of metadata, subject to the user's (and the associated OMAS's) security permissions. +the interface supports search, maintenance of metadata elements, classifications and relationships plus the ability to raise incident reports +and todos along with the ability to work with metadata valid values and translations. + + +--8<-- "snippets/abbr.md" diff --git a/site/docs/services/omvs/index.md b/site/docs/services/omvs/index.md index f0d4d587d2..d402023506 100644 --- a/site/docs/services/omvs/index.md +++ b/site/docs/services/omvs/index.md @@ -7,15 +7,16 @@ The Open Metadata View Services (OMVS) provide task oriented, domain-specific se The view services are as follows: -| OMVS | Summary | Description | -|-----------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Glossary Author](/services/omvs/glossary-author/overview) | **Develop new glossary terms and categories.** | The Glossary Author OMVS is for user interfaces supporting the creating and editing of glossary content, such as glossary terms and categories. The changes made are immediately visible to all users. | -| [Glossary Browser](/services/omvs/glossary-browser/overview) | **Search and view the contents of specific glossaries and their links to governance definitions and assets.** | The Glossary Browser OMVS is for user interfaces that wish to provide search facilities within a single glossary (or glossaries) along with the ability to browse the structure of the glossary and view an element within a glossary along with connected governance classifications, governance definitions, user feedback and attached assets. | -| [Glossary Workflow](/services/omvs/glossary-workflow/overview) | **Develop new glossary terms and categories in a controlled workflow process.** | The Glossary Workflow OMVS is for user interfaces supporting the creation and editing of glossary content, such as glossary terms and categories, in a controlled workflow process. This means that as terms and categories are created, updated and deleted, these changes are invisible to the general user until they are approved. | -| [Repository Explorer](/services/omvs/rex/overview) | **Explorer interface to inspect instances across a cohort of repositories.** | The Repository Explorer (Rex) OMVS is for user interfaces that support enterprise architects who need to inspect, navigate or explore the instance data stored in an open metadata repository or a cohort of repositories. It enables the retrieval of instance data (entities and relationships) and exploration of the graph of instances connected to those entities. This enables the user to construct a graph (as a diagram) to visualize the details and connectivity of a group of instances of interest to the user. | -| [Type Explorer](/services/omvs/tex/overview) | **Explorer interface to inspect types across a cohort of repositories.** | The Type Explorer (Tex) OMVS is for user interfaces that support enterprise architects who need to inspect, navigate or explore the open metadata types supported by a repository or a cohort of repositories. It enables the retrieval of type data (relating to entities, relationships and classifications) and exploration of the graphs of entity type inheritance and the supported combinations of entity and relationship types. | -| [Dynamic Infrastructure and Operations](/services/omvs/dino/overview) | **Admin interface to inspect servers, services, cohorts and platforms.** | The Dino OMVS is for user interfaces that support Egeria operators who need to inspect, navigate or explore the open metadata servers, services, cohorts and platforms that are configured or actively running. It is intended for operations and problem-determination. | -| [Server Author](/services/omvs/server-author/overview) | **An authoring interface for servers.** | The Server Author OMVS is for user interfaces that support Egeria Server authors. This user interface allows the author to create, delete or update a server configuration using an intuitive UI, rather than needing to use the low level admin rest calls and associated json payloads. | +| OMVS | Summary | Description | +|-----------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [My Profile](/services/omvs/my-profile/overview) | **Locate, retrieve and update information relating to the calling user's profile.** | The My Profile OMVS is for user interfaces supporting a personalized experience. A registered user is able to maintain information about themselves and their network as well as set up and use personalized collections of metadata elements. | +| [Glossary Author](/services/omvs/glossary-author/overview) | **Develop new glossary terms and categories.** | The Glossary Author OMVS is for user interfaces supporting the creating and editing of glossary content, such as glossary terms and categories. The changes made are immediately visible to all users. | +| [Glossary Browser](/services/omvs/glossary-browser/overview) | **Search and view the contents of specific glossaries and their links to governance definitions and assets.** | The Glossary Browser OMVS is for user interfaces that wish to provide search facilities within a single glossary (or glossaries) along with the ability to browse the structure of the glossary and view an element within a glossary along with connected governance classifications, governance definitions, user feedback and attached assets. | +| [Glossary Workflow](/services/omvs/glossary-workflow/overview) | **Develop new glossary terms and categories in a controlled workflow process.** | The Glossary Workflow OMVS is for user interfaces supporting the creation and editing of glossary content, such as glossary terms and categories, in a controlled workflow process. This means that as terms and categories are created, updated and deleted, these changes are invisible to the general user until they are approved. | +| [Repository Explorer](/services/omvs/rex/overview) | **Explorer interface to inspect instances across a cohort of repositories.** | The Repository Explorer (Rex) OMVS is for user interfaces that support enterprise architects who need to inspect, navigate or explore the instance data stored in an open metadata repository or a cohort of repositories. It enables the retrieval of instance data (entities and relationships) and exploration of the graph of instances connected to those entities. This enables the user to construct a graph (as a diagram) to visualize the details and connectivity of a group of instances of interest to the user. | +| [Type Explorer](/services/omvs/tex/overview) | **Explorer interface to inspect types across a cohort of repositories.** | The Type Explorer (Tex) OMVS is for user interfaces that support enterprise architects who need to inspect, navigate or explore the open metadata types supported by a repository or a cohort of repositories. It enables the retrieval of type data (relating to entities, relationships and classifications) and exploration of the graphs of entity type inheritance and the supported combinations of entity and relationship types. | +| [Dynamic Infrastructure and Operations](/services/omvs/dino/overview) | **Admin interface to inspect servers, services, cohorts and platforms.** | The Dino OMVS is for user interfaces that support Egeria operators who need to inspect, navigate or explore the open metadata servers, services, cohorts and platforms that are configured or actively running. It is intended for operations and problem-determination. | +| [Server Author](/services/omvs/server-author/overview) | **An authoring interface for servers.** | The Server Author OMVS is for user interfaces that support Egeria Server authors. This user interface allows the author to create, delete or update a server configuration using an intuitive UI, rather than needing to use the low level admin rest calls and associated json payloads. | diff --git a/site/docs/services/omvs/my-profile/overview.md b/site/docs/services/omvs/my-profile/overview.md new file mode 100644 index 0000000000..1aad9aad25 --- /dev/null +++ b/site/docs/services/omvs/my-profile/overview.md @@ -0,0 +1,27 @@ +--- +hide: +- toc +--- + + + + +--8<-- "snippets/content-status/in-development.md" + +# My Profile OMVS + +The My Profile Open Metadata View Services (OMVS) is a REST API designed to back user interfaces (UIs) that support personalization. + +At the heart of personalization is the personal profile that describes the individual with details such as their name, job role and contact details. The personal profile then becomes the parent node where the individual can attach and maintain collections of favourite elements such as assets, projects and the profiles of other people in their network. + + + + + +---8<-- "snippets/abbr.md" + + + + + + diff --git a/site/docs/services/server-chassis.md b/site/docs/services/server-chassis.md index 46ca0c5940..0f12c47a79 100644 --- a/site/docs/services/server-chassis.md +++ b/site/docs/services/server-chassis.md @@ -1,4 +1,15 @@ +--- +hide: +- toc +--- + ---8<-- "docs/services/platform-chassis.md" +--8<-- "snippets/content-status/in-development.md" + +## Server Chassis + +The server chassis provides the base for a Spring application that runs a single [OMAG Server](/concepts/omag-server) in a cloud native style. + +--8<-- "snippets/abbr.md" diff --git a/site/docs/services/server-operations.md b/site/docs/services/server-operations.md new file mode 100644 index 0000000000..5bf504bff4 --- /dev/null +++ b/site/docs/services/server-operations.md @@ -0,0 +1,15 @@ +--- +hide: +- toc +--- + + + + +--8<-- "snippets/content-status/stable.md" + +# Server Operations + +Server operations provides support to start up and shutdown [OMAG Servers](/concepts/omag-server) within either the [Platform Chassis](/services/platform-chassis) or [Server Chassis](/services/server-chassis). + +--8<-- "snippets/abbr.md" diff --git a/site/docs/types/0/0010-Base-Model.md b/site/docs/types/0/0010-Base-Model.md index 874f7adf82..753fd9fb7d 100644 --- a/site/docs/types/0/0010-Base-Model.md +++ b/site/docs/types/0/0010-Base-Model.md @@ -7,11 +7,11 @@ The base model is the starting point for the open metadata type model. ![UML](0010-Base-Model.svg) -## OpenMetadataRoot +## OpenMetadataRoot entity *OpenMetadataRoot* is the root entity for all open metadata entity types. -## Referenceable +## Referenceable entity *Referenceable* is the super type for many of the open metadata entity types. A *Referenceable* entity is something that is important enough to be assigned a unique (qualified) name within its type. This unique name is called the *qualifiedName* and may be set to the unique identifier value used outside the open metadata ecosystem. Alternatively, it is often set to a concatenation of an element's type name along with a number of its properties to create a unique string. @@ -20,7 +20,7 @@ Referenceable also has provision for storing additional properties. This is a se * [Further information on the use of Referenceable.](/concepts/referenceable) * [Further information on external identifiers](/features/external-identifiers/overview) -## Asset +## Asset entity An [Asset](/concepts/asset) is a metadata entity that describes a [resource](/concepts/resource) (either physical or digital) that is of value and so needs to be managed and governed. [*Infrastructure*](#infrastructure), [*Process*](#process), [*DataStore*](/types/2/0210-Data-Stores), [*DataFeed*](/types/2/0223-Events-and-Logs), [*DeployedAPI*](/types/2/0212-Deployed-APIs), [*DataSet*](#dataset) and [*RunnableSoftwareComponent*](/types/2/0282-Released-Software-Components) are subtypes of *Assets*. @@ -44,7 +44,7 @@ The values set in an *Asset* entity tend to be focused around the implementation More information on assets can be found in the [Metadata Manager](/patterns/metadata-manager/overview) overview. -### Infrastructure +### Infrastructure entity *Infrastructure* represents both the physical and digital assets that the organization runs its business on. [*ITInfrastructure*](/types/0/0030-Hosts-and-Platforms) is a subtype of *Infrastructure* describing Information Technology (IT) infrastructure that runs IT services. There is more information on the different types of *ITInfrastructure* in: @@ -54,7 +54,7 @@ More information on assets can be found in the [Metadata Manager](/patterns/meta - [0040 Software servers](/types/0/0040-Software-Servers) - [0042 Software capabilities](/types/0/0042-Software-Capabilities) -### Process +### Process entity *Process* describes a well-defined set of processing steps and decisions that drive a particular aspect of the organization's business. Most processes are automated with software (see [*DeployedSoftwareComponent*](/types/2/0215-Software-Components/#deployedsoftwarecomponent)) but they may also be a manual procedure. An automated process can be invoked from a remote server through a [*DeployedAPI*](/types/2/0212-deployed-apis/#deployedapi). @@ -68,24 +68,26 @@ Processes have an advanced lifecycle. They can have the following [instance sta The *formula* attribute can describe its behaviour, *formulaType* describes the notation language used to describe the formula. -### DataSet +### DataSet entity -*DataSet* represents a collection of related data. This data does not need to be stored together. The *formula* property describes the logic used to populate the DataSet, *formulaType* describes the notation language used to describe the formula. See [*DataStore*](/types/2/0210-Data-Stores) for the `Asset` that represents a physical store and the [*DataContentForDataSet*](/types/2/0210-Data-Stores/#datacontentfordataset) relationship links the DataSet to the asset(s) that describe the data sources. +*DataSet* represents a collection of related data. This data does not need to be stored together. The *formula* property describes the logic used to populate the DataSet, *formulaType* describes the notation language used to describe the formula, and *deployedImplementationType* describes the technology supporting the implementation of the technology. -## SampleData +See [*DataStore*](/types/2/0210-Data-Stores) for the `Asset` that represents a physical store and the [*DataContentForDataSet*](/types/2/0210-Data-Stores/#datacontentfordataset) relationship links the DataSet to the asset(s) that describe the data sources. + +## SampleData relationship The *SampleData* relationship links an *Asset* entity describing a collection of sample data that originates from the resource represented by the *Referenceable* entity. -## Anchors +## Anchors classification The *Anchors* classification is used internally by the open metadata ecosystem to optimize the lookup of the entity at the root of a cluster of elements that represents a larger object. Currently, there is support for objects uniquely "owned" by an entity to store the GUID of that entity. !!! info "Further information on the use of Anchors" * [Anchor Management](/concepts/anchor). -## Memento +## Memento classification -Finally, the *Memento* classification identifies that the Referenceable refers to a real-world asset/artifact that has either been deleted or archived offline. The metadata element has been retained to show its role in the [lineage of other assets/artifacts](/features/lineage-management/overview). The properties in this classification identifies the archive processing and any information that helps to locate the asset/artifact in the archive (if applicable). +Finally, the *Memento* classification identifies that the Referenceable entity it is attached to, refers to a real-world asset/artifact that has either been deleted or archived offline. The entity has been retained to show its role in the [lineage of other assets/artifacts](/features/lineage-management/overview). The properties in this classification identifies the archive processing and any information that helps to locate the asset/artifact in the archive (if applicable). diff --git a/site/docs/types/0/0010-Base-Model.svg b/site/docs/types/0/0010-Base-Model.svg index b064054dc9..5058398948 100644 --- a/site/docs/types/0/0010-Base-Model.svg +++ b/site/docs/types/0/0010-Base-Model.svg @@ -1,4 +1,4 @@ - + -
0010 - Base Model
0010 - Base Model
«entity»
Referenceable
«entity»...
«entity»
Infrastructure
«entity»...
qualifiedName : string
additionalProperties : map<string, string>
qualifiedName : string...
«entity»
Asset
«entity»...
name : string
versionIdentifier : string
description : string
name : string...
«entity»
Process
«entity»...
formula : string
formulaType : string
formula : string...
«entity»
DataSet
«entity»...
formula : string
formulaType : string
formula : string...
«entity»
OpenMetadataRoot
«entity»...
«classification»
Anchors
«classification»...
anchorGUID : string
anchorGUID : string
«classification»
Memento
«classification»...
archiveDate : Date
archiveUser : string
archiveProcess : string
archiveService : string
archiveMethod : string
archiveProperties : map<string, string>
archiveDate : Date...
«relationship»
SampleData
«relationship»...
samplingMethod : string
samplingMethod : string
*
*
*
*
sampleData
sampleData
sourceOfSample
sourceOfSampleText is not SVG - cannot display \ No newline at end of file +
0010 - Base Model
0010 - Base Model
«entity»
Referenceable
«entity»...
«entity»
Infrastructure
«entity»...
qualifiedName : string
additionalProperties : map<string, string>
qualifiedName : string...
«entity»
Asset
«entity»...
name : string
versionIdentifier : string
description : string
name : string...
«entity»
Process
«entity»...
formula : string
formulaType : string
formula : string...
«entity»
DataSet
«entity»...
formula : string
formulaType : string
deployedImplementationType : string
formula : string...
«entity»
OpenMetadataRoot
«entity»...
«classification»
Anchors
«classification»...
anchorGUID : string
anchorGUID : string
«classification»
Memento
«classification»...
archiveDate : Date
archiveUser : string
archiveProcess : string
archiveService : string
archiveMethod : string
archiveProperties : map<string, string>
archiveDate : Date...
«relationship»
SampleData
«relationship»...
samplingMethod : string
samplingMethod : string
*
*
*
*
sampleData
sampleData
sourceOfSample
sourceOfSampleText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/0/0017-External-Identifiers.md b/site/docs/types/0/0017-External-Identifiers.md index c6286cae1b..e7dd89a8a4 100644 --- a/site/docs/types/0/0017-External-Identifiers.md +++ b/site/docs/types/0/0017-External-Identifiers.md @@ -3,18 +3,23 @@ # 0017 External Identifiers -External identifiers are used to correlate the identifiers used in third party metadata catalogs with open metadata elements. +External identifiers are used to correlate the identifiers used in third party systems with open metadata elements. ![UML](0017-External-Identifiers.svg) -## ExternalId +## ExternalId entity -The *`ExternalId`* entity describes an external identifier from a specific third party metadata repository. It includes: +The *ExternalId* entity describes an external identifier from a specific third party system. It includes: -- The identifier value itself in `identifier`. -- The pattern used for the identifier (how it is generated and managed) is stored in `keyPattern`. +* *identifier* - the unique identifier of the element in the third party metadata repository. +* *keyPattern* - The pattern used for the identifier (how it is generated and managed) is stored in the *KeyPattern* enumeration. +* *externalInstanceCreatedBy* - the username of the person or process that created the instance in the external system. +* *externalInstanceCreationTime* - the date/time when the instance in the external system was created. +* *externalInstanceLastUpdatedBy* - the username of the person or process that last updated the instance in the external system. +* *externalInstanceLastUpdateTime* - the date/time that the instance in the external system was last updated. +* *externalInstanceVersion* - the latest version of the element in the external system. -## KeyPattern +## KeyPattern enumeration *KeyPattern* describes the pattern used for the identifier (how it is generated and managed). These are the values it can take, with the default (and most used) being `LOCAL_KEY`: @@ -30,17 +35,17 @@ The *`ExternalId`* entity describes an external identifier from a specific third | OTHER | 99 | "Other" | Another key pattern. | -## ExternalIdLink +## ExternalIdLink relationship -The mapping of identifiers can be many-to-many, which is why you see that the *`ExternalIdLink`* relationship between the [`OpenMetadataRoot`](/types/0/0010-Base-Model/#openmetadataroot) (open metadata resources) and the `ExternalId` is also many-to-many. +The mapping of identifiers can be many-to-many, which is why you see that the *ExternalIdLink* relationship between the [*OpenMetadataRoot*](/types/0/0010-Base-Model/#openmetadataroot) (open metadata resources) and the *ExternalId* entity is also many-to-many. -This relationship includes properties to help to map the `OpenMetadataRoot` to the external identifier. +This relationship includes properties to help to map the *OpenMetadataRoot* entity to the external identifier. -## ExternalIdScope +## ExternalIdScope relationship -There is no guarantee that external identifiers from a third party metadata catalog are globally unique and so the *`ExternalIdScope`* relationship links the external identifier to the [`Referenceable`](/types/0/0010-Base-Model/#referenceable) that represents the third party metadata catalog. Typically, this is a type of [`SoftwareCapability`](/types/0/0042-Software-Capabilities/#softwarecapability), for example, [`AssetManager`](/types/0/0056-Resource-Managers/#assetmanager). +There is no guarantee that external identifiers from a third party metadata catalog are globally unique and so the *ExternalIdScope* relationship links the external identifier to the [Referenceable](/types/0/0010-Base-Model/#referenceable) that represents the third party system. Typically, this is a type of [SoftwareCapability](/types/0/0042-Software-Capabilities/#softwarecapability), for example, [AssetManager](/types/0/0056-Resource-Managers/#assetmanager). -## PermittedSynchronization +## PermittedSynchronization enumeration *PermittedSynchronization* defines the direction of flow of metadata. @@ -51,6 +56,22 @@ There is no guarantee that external identifiers from a third party metadata cata | FROM_THIRD_PARTY | 2 | "From Third Party" | The third party technology is logically upstream (the originator and owner of the metadata). Any updates made in open metadata are not passed to the third party technology and the third party technology is requested to refresh the open metadata version. | | OTHER | 99 | "Other" | Another type of synchronization rule - see description property. | +## Example + +The picture below shows the value of the external identifiers in providing traceability between metadata elements in different systems. + +Starting on the left-hand side of the diagram is a [SoftwareCapability](/types/0/0042-Software-Capabilies) entity representing an Apache Atlas server. + +Following the *ExternalIdScope* relationships takes you to the identifiers of all the Apache Atlas entities that have been synchronized into the open metadata ecosystem. + +From one of the *ExternalId* entities it is possible to navigate to the open metadata entity/entities that store the equivalent metadata by following the *ExternalIdLink* relationships. + +An open metadata ecosystem entity may have been synchronized with multiple third party systems. Following all the *ExternalIdLink* relationships from such an open metadata entity creates a list of external identifiers for each of the third party systems it has been synchronized with. In this example, the glossary term has been synchronized with Apache Atlas and DataHub servers. The *lastSynchronized* property in the *externalIdLink* documents when Egeria last checked for updates from the associated third party system. The other properties in the *ExternalId* entity provide more detail about the element in the third party system. + +![Example](external-identifier-example.svg) + + + ??? education "Further information" There is an article on [managing external identifiers](/features/external-identifiers/overview) to correlate metadata elements from different types of technologies. diff --git a/site/docs/types/0/0017-External-Identifiers.svg b/site/docs/types/0/0017-External-Identifiers.svg index 476f3533f8..242f25c32e 100644 --- a/site/docs/types/0/0017-External-Identifiers.svg +++ b/site/docs/types/0/0017-External-Identifiers.svg @@ -1,4 +1,4 @@ - + -
0017 - External Identifiers
0017 - External Identifiers
«entity»
Referenceable
«entity»...
«relationship»
ExternalIdLink
«relationship»...
description : string
usage : string
source : string
lastSynchronized : date
mappingProperties : map<string, string>
description : string...
«entity»
ExternalId
«entity»...
identifier : string
keyPattern : KeyPattern
identifier : string...
«relationship»
ExternalIdScope
«relationship»...
permittedSynchronization : PermittedSynchronization
description : string
permittedSynchronization : PermittedSynchronization...
«enumeration»
KeyPattern
«enumeration»...
LOCAL_KEY = 0
RECYCLED_KEY = 1
NATURAL_KEY = 2
MIRROR_KEY = 3
AGGREGATE_KEY = 4
CALLERS_KEY = 5
STABLE_KEY = 6
OTHER = 99
LOCAL_KEY = 0...
resources
resources
*
*
*
*
*
*
*
*
alsoKnownAs
alsoKnownAs
scopedTo
scopedTo
managedResources
managedResources
«enumeration»
PermittedSynchronization
«enumeration»...
BOTH_DIRECTIONS = 0
TO_THIRD_PARTY = 1
FROM_THIRD_PARTY = 2
OTHER = 99
BOTH_DIRECTIONS = 0...
«entity»
OpenMetadataRoot
«entity»...Text is not SVG - cannot display \ No newline at end of file +
0017 - External Identifiers
0017 - External Identifiers
«entity»
Referenceable
«entity»...
«relationship»
ExternalIdLink
«relationship»...
description : string
usage : string
source : string
lastSynchronized : date
mappingProperties : map<string, string>
description : string...
«entity»
ExternalId
«entity»...
identifier : string
keyPattern : KeyPattern
externalInstanceCreatedBy : string
externalInstanceCreationTime : date
externalInstanceLastUpdatedBy : string
externalInstanceLastUpdateTime : date
externalInstanceVersion : long
identifier : string...
«relationship»
ExternalIdScope
«relationship»...
permittedSynchronization : PermittedSynchronization
description : string
permittedSynchronization : PermittedSynchronization...
«enumeration»
KeyPattern
«enumeration»...
LOCAL_KEY = 0
RECYCLED_KEY = 1
NATURAL_KEY = 2
MIRROR_KEY = 3
AGGREGATE_KEY = 4
CALLERS_KEY = 5
STABLE_KEY = 6
OTHER = 99
LOCAL_KEY = 0...
resources
resources
*
*
*
*
*
*
*
*
alsoKnownAs
alsoKnownAs
scopedTo
scopedTo
managedResources
managedResources
«enumeration»
PermittedSynchronization
«enumeration»...
BOTH_DIRECTIONS = 0
TO_THIRD_PARTY = 1
FROM_THIRD_PARTY = 2
OTHER = 99
BOTH_DIRECTIONS = 0...
«entity»
OpenMetadataRoot
«entity»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/0/0035-Complex-Hosts.md b/site/docs/types/0/0035-Complex-Hosts.md index ef7efcac09..9f4f94a66e 100644 --- a/site/docs/types/0/0035-Complex-Hosts.md +++ b/site/docs/types/0/0035-Complex-Hosts.md @@ -8,56 +8,56 @@ In today's systems, hardware is managed to get the maximum use out of it. Theref ![UML](0035-Complex-Hosts.svg) -## Host +## Host entity The concept of a *Host* is abstracted to describe a deployment environment that has access to hardware and has a basic software stack, typically including the operating systems. The host can be linked to its location through the [AssetLocation](/types/0/0025-Locations/#assetlocation) relationship. -## DeployedOn +## DeployedOn relationship The *DeployedOn* relationship shows where IT Infrastructure is deployed to. -## BareMetalComputer +## BareMetalComputer entity A *BareMetalComputer* describes a connected set of physical hardware. The open metadata types today do not attempt to model hardware in detail but this could be easily added if a contributor with the appropriate expertise was willing to work on it. -## VirtualMachine +## VirtualMachine entity A *VirtualMachine* provides virtualized hardware through a hypervisor that allows a single physical bare metal computer to run multiple virtual machines. -## VirtualContainer +## VirtualContainer entity A *VirtualContainer* provides the services of a host to the [software servers](/types/0/0040-Software-Servers) deployed on it. When the server makes requests for storage, network access, and other resources, the *VirtualContainer* delegates the requests to the equivalent services of the actual host it is deployed on. *VirtualContainer*s can be hosted on other *VirtualContainer*'s, but to actually run they need to ultimately be deployed onto a real physical [Host](/types/0/0030-Hosts-and-Platforms/#host). -### DockerContainer +### DockerContainer entity *DockerContainer* provides a specific type for the popular container type called [docker :material-dock-window:](https://www.docker.com/){ target=docker }. -## HostCluster +## HostCluster entity A *HostCluster* describes a collection of hosts that together are providing a service. Clusters are often used to provide horizontal scaling of services. -There are two specific types of host clusters defined: in both, the hosts that they manage are often referred to as *nodes*. +Within the host cluster there may be a special host (node) that is controlling the execution of the other members. This host is modelled with a [SoftwareServerPlatform](/types/0/0037-Software-Server-Platforms/#softwareserverplatform) that describes the cluster management platform, and optional [SoftwareServer](/types/0/0040-Software-Servers/#softwareserver) assets. [SoftwareCapabilities](/types/0/0042-Software-Capabilities/#softwarecapability) needed to manage the cluster are linked to these [ITInfrastructure](/types/0/0030-Hosts-and-Platforms/#itinfrastructure) using the [ServerAssetUse](/types/0/0045-Servers-and-Assets/#serverassetuse) relationship. -Within the host cluster is typically a special host (node) that is controlling the execution of the other members. This host is modelled with a [`SoftwareServerPlatform`](/types/0/0037-Software-Server-Platforms/#softwareserverplatform) that describes the cluster management platform, and optional [`SoftwareServer`](/types/0/0040-Software-Servers/#softwareserver) assets. [`SoftwareCapabilities`](/types/0/0042-Software-Capabilities/#softwarecapability) needed to manage the cluster are linked to these [`ITInfrastructure`](/types/0/0030-Hosts-and-Platforms/#itinfrastructure) using the [`ServerAssetUse`](/types/0/0045-Servers-and-Assets/#serverassetuse) relationship. - -### HadoopCluster +### HadoopCluster entity *HadoopCluster* describes a [Hadoop cluster :material-dock-window:](https://hadoop.apache.org/){ target=apache } that uses multiple bare metal computers/virtual machines to manage big data workloads. -### KubernetesCluster +### KubernetesCluster entity *KubernetesCluster* describes a [Kubernetes cluster :material-dock-window:](https://kubernetes.io/){ target=k8s } that manages containerized applications across multiple bare metal computers/virtual machines. The containerized applications managed by Kubernetes are represented as *VirtualContainer*'s. +## HostClusterMember relationship -## HostClusterMember +The host cluster is linked to the hosts it is managing using the *HostClusterMember* relationship. The properties help to clarify how the member in the cluster is used. -The host cluster is linked to the hosts it is managing using the *HostClusterMember* relationship. +* *memberRole* describes the role of the member in a cluster. For example, in a kubernetes cluster there are two membership roles: the *pods* and the *hosts*. The *membershipRole* can be set up to show which hosts are the containers running as pods and which are providing the processing location as *hosts*. The pods and the hosts are then linked together using the *deployedOn* relationship to show where the various pods have been deployed. +* *additionalProperties* provides a place to add additional information about the membership of the cluster such as configuration properties. ??? deprecated "Deprecated types" - *DeployedVirtualContainer* - use *DeployedOn*, which is more general. diff --git a/site/docs/types/0/0035-Complex-Hosts.svg b/site/docs/types/0/0035-Complex-Hosts.svg index d5a1681fd7..7ada9a71ed 100644 --- a/site/docs/types/0/0035-Complex-Hosts.svg +++ b/site/docs/types/0/0035-Complex-Hosts.svg @@ -1,4 +1,4 @@ - + -
0035 - Hosts
0035 - Hosts
«entity»
VirtualContainer
«entity»...
0..1
0..1
hostCluster
hostCluster
«entity»
HostCluster
«entity»...
«relationship»
HostClusterMember
«relationship»...
«entity»
VirtualMachine
«entity»...
«entity»
BareMetalComputer
«entity»...
«entity»
DockerContainer
«entity»...
«entity»
HadoopCluster
«entity»...
«entity»
KubernetesCluster
«entity»...
«entity»
Host
«entity»...
*
*
deployedElement
deployedElement
«entity»
ITInfrastructure
«entity»...
«entity»
Infrastructure
«entity»...
«entity»
Asset
«entity»...
«relationship»
DeployedOn
«relationship»...
deploymentTime : date
deployer : string
deployerTypeName : string
deployerPropertyName ; string
deploymentStatus : OperationalStatus
deploymentTime : date...
*
*
deployedTo
deployedTo
*
*
managedHosts
managedHosts
«enumeration»
OperationalStatus
«enumeration»...
DISABLED = 0
ENABLED = 1
DISABLED = 0...Text is not SVG - cannot display \ No newline at end of file +
0035 - Hosts
0035 - Hosts
«entity»
VirtualContainer
«entity»...
0..1
0..1
hostCluster
hostCluster
«entity»
HostCluster
«entity»...
«relationship»
HostClusterMember
«relationship»...
memberRole : string
additionalProperties : map<string, string>
memberRole : string...
«entity»
VirtualMachine
«entity»...
«entity»
BareMetalComputer
«entity»...
«entity»
DockerContainer
«entity»...
«entity»
HadoopCluster
«entity»...
«entity»
KubernetesCluster
«entity»...
«entity»
Host
«entity»...
*
*
deployedElement
deployedElement
«entity»
ITInfrastructure
«entity»...
«entity»
Infrastructure
«entity»...
«entity»
Asset
«entity»...
«relationship»
DeployedOn
«relationship»...
deploymentTime : date
deployer : string
deployerTypeName : string
deployerPropertyName ; string
deploymentStatus : OperationalStatus
deploymentTime : date...
*
*
deployedTo
deployedTo
*
*
managedHosts
managedHosts
«enumeration»
OperationalStatus
«enumeration»...
DISABLED = 0
ENABLED = 1
DISABLED = 0...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/0/0036-Storage.md b/site/docs/types/0/0036-Storage.md index 1058e2a832..c5fb074c61 100644 --- a/site/docs/types/0/0036-Storage.md +++ b/site/docs/types/0/0036-Storage.md @@ -3,16 +3,16 @@ # 0036 Storage -It is common for the processing running on a [`Host`](/types/0/0030-Hosts-and-Platforms/#host) to need to persist data to storage. +It is common for the processing running on a [*Host*](/types/0/0030-Hosts-and-Platforms/#host) to need to persist data to storage. ![UML](0036-Storage.svg) -## StorageVolume +## StorageVolume entity -*`StorageVolume`* describes a persistent storage volume. +*StorageVolume* describes a persistent storage volume. -## AttachedStorage +## AttachedStorage relationship -*`AttachedStorage`* identifies the host(s) that the `StorageVolume` is connected to. +*AttachedStorage* identifies the host(s) that the StorageVolume is connected to. --8<-- "snippets/abbr.md" diff --git a/site/docs/types/0/0041-Server-Purpose.md b/site/docs/types/0/0041-Server-Purpose.md index 323ba0c455..5e51dd8b64 100644 --- a/site/docs/types/0/0041-Server-Purpose.md +++ b/site/docs/types/0/0041-Server-Purpose.md @@ -7,23 +7,63 @@ The server purpose classifications allow more detail about the type of middlewar ![UML](0041-Server-Purpose.svg) -## ServerPurpose +## ServerPurpose classification -The *deployedImplementationType* attribute in [SoftwareServerPlatform](/types/0/0037-Software-Server-Platforms) and [Software Server](/types/0/0040-Software-Servers) identifies the software is running in the platform - it could be a name from a product or open source project. For example, if the open metadata types were describing an Egeria deployment, the [OMAG Server Platform](/concepts/omag-server-platform) would be represented as a *SoftwareServerPlatform* entity - -The *ServerPurpose* classification provides a common base type for the server purposes - so it is possible to, for example, query a list of database servers, even through they are implemented using various technologies. +The *ServerPurpose* classification provides a common base type for the server purposes - so it is possible to, for example, query a list of database servers, even through they are implemented using various technologies and hence represented by entities with different open metadata types. The *deployedImplementationType* attribute allows a specific software package supporting the specific purpose. The subtypes of the *ServerPurpose* classification are: -- *ApplicationServer* - hosting application logic. -- *Webserver* - hosting endpoint for HTTP requests. -- *DatabaseServer* - hosting a database management system (DBMS). -- *IntegrationServer* - hosting data movement processes. -- *MetadataServer* - manages and stores a collection of metadata. -- *RepositoryProxy* - Acts as an adapter for a metadata server, translating requests between different APIs, formats and protocols. -- *StewardshipServer* - hosting stewardship processes. -- *GovernanceDaemon* - hosting automated governance processing. +- [*ApplicationServer*](#applicationserver-classification) - hosting application logic. +- [*Webserver*](#webserver-classification) - hosting endpoint for HTTP requests. +- [*DatabaseServer*](#databaseserver-classification) - hosting a database management system (DBMS). +- [*IntegrationServer*](#integrationserver-classification) - hosting data movement processes. +- [*MetadataServer*](#metadataserver-classification) - manages and stores a collection of metadata. +- [*RepositoryProxy*](#repositoryproxy-classification) - Acts as an adapter for a metadata server, translating requests between different APIs, formats and protocols. +- [*StewardshipServer*](#stewardshipserver-classification) - hosting stewardship processes. +- [*GovernanceDaemon*](#governancedaemon-classification) - hosting automated governance processing. + +## ApplicationServer classification + +An application server hosts what is often called "business logic". It is the software that supports business operations such as serving customer, building goods and services, invoicing and collecting money, etc. There are many ways to implement business logic and the popular approaches change over time. For example, microservices are popular at the time of writing this, before that it was web services, before that it was Enterprise Java Beans. For an organization with event a modest amount of longevity, it is common to find a mix of implementation styles supporting their business. Each implementation style varies the type of frameworks and underlying middleware, leading to different system structures. Hence the usefulness of being able to identify where the business logic is located. + +## Webserver classification + +A webserver supports HTTP requests. This is used to support browser-based user interfaces (including websites and many applications) as well as REST API calls. As a consequence, many platforms/servers include a webserver, so you will find them in unexpected places. For example, Egeria's [OMAG Server Platform](/concepts/omag-server-platform) includes the [Tomcat Webserver](https://tomcat.apache.org/). Webservers provide external access to application function and so are a potential access point for cyber attacks. Using this classification on the appropriate IT Infrastructure entities enables a simple query to retrieve all of your HTTP access points. If these entities are linked to an [Endpoint](/types/0/0026-Endpoints) via a *ServerEndpoint* relationship, the combination enables a report of the HTTP endpoints and the servers behind it. + +## DatabaseServer classification + +A database server supports data query services such as SQL. These services used to be the preserve of specialist relational database software. However, standards such as SQL have become ubiquitous and we see many types of services supporting data query services. Classifying the IT infrastructure entities that provide data query services helps to identify where collections of data can be accessed directly. + +## IntegrationServer classification + +An integration server manages the flow of data between servers (typically application servers and database servers). This copying of data can be done in batch or by monitoring changes to the source and sending the appropriate updates to its downstream destinations. Examples of integration servers are: + +* Extract, transformation, load (ETL) engines +* Replication engines +* Data virtualization engines/services +* Event brokers + +Integration servers often include mapping and transformation logic needed to mold the data to the downstream systems needs. So these types of servers are: + +* The means by which data (and metadata) flows between the organization systems and so are very important in understanding [lineage](/features/lineage-management/overview). +* A potential place where errors in data can be introduced. + +## MetadataServer classification + +A metadata server is a store of metadata that describes external elements - typically people, processes and technology. It is a means to understand and manage a digital landscape. Egeria's [metadata access store](/concepts/metadata-access-store) is an example of a metadata server; [Apache Atlas](https://atlas.apache.org) is another. + +As IT landscapes have got more complex, and the desire to extract data from this landscape for reporting, analytics and AI, there has been explosion in the number of metadata servers, often called data catalogs or configuration management databases, and the number of data tools that include a metadata server. Understanding the location of metadata servers can help to kickstart a new initiative that needs to find existing resources. + +## RepositoryProxy classification + +This is a concept from Egeria. A [repository proxy](/concepts/repository-proxy) is a metadata server that is acting as a proxy (adapter) to another metadata server. Although it provides metadata services, calls to these services are delegated to another metadata server. + +## StewardshipServer classification + +A stewardship server is one where corrections are made to data outside of their origin system. +## GovernanceDaemon classification +A governance daemon is a server that is supporting governance processing. It is typically a background server. It often feeds work to a stewardship server. Egeria's [governance servers](/concepts/governance-server). --8<-- "snippets/abbr.md" diff --git a/site/docs/types/0/0042-Software-Capabilities.md b/site/docs/types/0/0042-Software-Capabilities.md index 3a03cbd1d3..d47a034508 100644 --- a/site/docs/types/0/0042-Software-Capabilities.md +++ b/site/docs/types/0/0042-Software-Capabilities.md @@ -3,9 +3,11 @@ # 0042 Software Capabilities +Software capabilities are the capabilities implemented in software and supported by [ITInfrastructure](/types/0/0030-Hosts-and-Platforms) such as hosts, software server platforms and software servers. They are composed of, support and/or consume various [digital resources](/concepts/resources) represented as [Asset entities](/types/0/0010-Base-Model). They are linked together using the [ServerAssetUse relationship](/types/0045-Servers-and-Assets). + ![UML](0042-Software-Capabilities.svg) -## SoftwareCapability +## SoftwareCapability entity [*IT Infrastructure*](/types/0/0030-Hosts-and-Platforms) contains many capabilities. Each capability can be catalogued using the *SoftwareCapability* entity and linked to the hosting *ITInfrastructure* entity using the *SupportedSoftwareCapability* relationship. @@ -49,20 +51,15 @@ In addition, it is possible to augment software capabilities with the following - [*FileManager*](/types/0/0056-Resource-Managers/#filemanager) - A manager of a collection of files and folders. - [*NotificationManager*](/types/0/0056-Resource-Managers/#notificationmanager) - A software capability that is distributing events from a topic to its subscriber list. -## SupportedSoftwareCapability - -Defines the relationship between a *SoftwareCapability* and the *ITInfrastructure* asset that hosts it. - -## ProcessingState - -Defines a classification for a *SoftwareCapability*. The *ProcessingState* can be used to describe additional runtime processing information used by various SoftwareCapabilities. +## SupportedSoftwareCapability relationship +Defines the relationship between a *SoftwareCapability* and the *ITInfrastructure* entity that hosts it. This relationship, plus the attached *SoftwareCapability* entity, help to build out a picture of the capabilities of a particular deployed host, software server platform or software server. Together they help to describe where and how various IT capabilities are delivered to the owning organization. This model can form the basis of traceability from business function to IT Infrastructure, particularly when combined with [location information](/types/0/0025-Locations) and [links to the data assets and APIs](/types/0/0045-Servers-and-Assets). -## ProcessingState +## ProcessingState classification -Defines a classification for a `SoftwareCapability`. It stores the last sync timestamp for critical elements. +Defines a classification for a *SoftwareCapability*. The *ProcessingState* can be used to describe additional runtime processing information used by the component represented by the *SoftwareCapability* entity. ??? deprecated "Deprecated types" - - *SoftwareServerSupportedCapability* is deprecated in favor of *SupportedSoftwareCapability*. + - *SoftwareServerSupportedCapability* is deprecated in favour of the *SupportedSoftwareCapability* relationship. --8<-- "snippets/abbr.md" diff --git a/site/docs/types/0/0045-Servers-and-Assets.md b/site/docs/types/0/0045-Servers-and-Assets.md index 6857fbcb46..cb424b3b0a 100644 --- a/site/docs/types/0/0045-Servers-and-Assets.md +++ b/site/docs/types/0/0045-Servers-and-Assets.md @@ -3,10 +3,27 @@ # 0045 Servers and Assets +Many data catalogs capture information about data stores, data sets, APIs and topics. These [digital resources](/concepts/resource) provide access to data and reusable function which is why they are of interest. These resources run on IT infrastructure. + ![UML](0045-Servers-and-Assets.svg) -## ServerAssetUse +## ServerAssetUseType enumeration + +The *ServerAssetUseType* describes the different types of dependencies that a software capability has on an asset. + +* *OWNS* means that the asset is hosted and controlled by the software capability. For example, a database server OWNS the databases deployed to it. +* *GOVERNS* means the the asset is monitored and action is taken if the asset moves out of certain boundaries or thresholds. For example, an archiving service GOVERNS a database by removing records that are past a certain date. +* *MAINTAINS* means that although the software capability does not host the asset, it is responsible for maintaining its contents. For example, an application may store its data in a database running on a remote database server. The application MAINTAINS the database. +* *USES* means that the software capability is a consumer of the asset. For example, an analytical service may read the data in a database to feed an algorithm. The analytical service USES the database. +* *OTHER* means there is another type of dependency between the software capability and the asset. + +## ServerAssetUse relationship + +[Assets](/types/0/0010-Base-Model/#asset) are managed or consumed by [SoftwareCapabilities](/types/0/0042-Software-Capabilities/#softwarecapability), which is captured by the *ServerAssetUse* relationship. The properties characterize the relationship: -[`Asset`s](/types/0/0010-Base-Model/#asset) are managed or consumed by [`SoftwareCapabilities`](/types/0/0042-Software-Capabilities/#softwarecapability), which is captured by the *`ServerAssetUse`* relationship. +* *useType* defines how the software capability depends on the asset as defined by the *SoftwareAssetUseType* enumeration. +* *description* provides a description of the use of the asset by the software capability. +* *minimumInstances* defines the minimum number of instances of the asset connected to the software capability. +* *maximumInstances* defines the maximum number of instances of the asset connected to the software capability. --8<-- "snippets/abbr.md" diff --git a/site/docs/types/0/0057-Software-Services.md b/site/docs/types/0/0057-Software-Services.md index a1e7104923..40b2ac75a7 100644 --- a/site/docs/types/0/0057-Software-Services.md +++ b/site/docs/types/0/0057-Software-Services.md @@ -7,37 +7,47 @@ ## SoftwareService -A *`SoftwareService`* provides a well-defined software component that can be called by remote clients across the network. They may offer a request-response or an event-driven interface or both. +A *SoftwareService* provides a well-defined software component that can be called by remote clients across the network. They may offer a request-response or an event-driven interface or both. -### ApplicationService +## ApplicationService -Typically, software services implement specific business functions such as on-boarding a new customer, taking an order or sending an invoice. These are called *`ApplicationService`s* +Typically, software services implement specific business functions such as on-boarding a new customer, taking an order or sending an invoice. These are called *ApplicationService`s* + +## SecurityService + +A *SecurityService* service describes a server that provides security services. The [Policy Management Classifications](/types/4/0435-Policy-Management-Capabilities) define the role that the service is playing in the security architecture. + +## Open Metadata and Governance Services Egeria offers specialized software services related to the capture and management of open metadata. These are shown as specialist types: ### MetadataIntegrationService -A *`MetadataIntegrationService`* describes an [Open Metadata Integration Service (OMIS)](/services/omis) that runs in an [integration daemon](/concepts/integration-daemon). +A *MetadataIntegrationService* describes an [Open Metadata Integration Service (OMIS)](/services/omis) that runs in an [integration daemon](/concepts/integration-daemon). ### MetadataAccessService -A *`MetadataAccessService`* describes an [Open Metadata Access Service (OMAS)](/services/omas) that runs in a [metadata access point](/concepts/metadata-access-point). +A *MetadataAccessService* describes an [Open Metadata Access Service (OMAS)](/services/omas) that runs in a [metadata access point](/concepts/metadata-access-point). ### EngineHostingService -An *`EngineHostingService`* describes an [Open Metadata Engine Service (OMES)](/services/omes) that runs in an [engine host](/concepts/engine-host). +An *EngineHostingService* describes an [Open Metadata Engine Service (OMES)](/services/omes) that runs in an [engine host](/concepts/engine-host). ### UserViewService -A *`UserViewService`* describes an [Open Metadata View Service (OMVS)](/services/omvs) that runs in a [view server](/concepts/view-server). +A *UserViewService* describes an [Open Metadata View Service (OMVS)](/services/omvs) that runs in a [view server](/concepts/view-server). ### EnterpriseAccessLayer -A *`EnterpriseAccessLayer`* describes the enterprise repository services provided by the [Open Metadata Repository Services (OMRS)](/services/omrs) that runs in a [metadata access server](/concepts/metadata-access-server). +A *EnterpriseAccessLayer* describes the enterprise repository services provided by the [Open Metadata Repository Services (OMRS)](/services/omrs) that runs in a [Metadata Access Server](/concepts/metadata-access-server). + +### MetadataRepositoryService + +A *MetadataRepositoryService* service describes the services provided by the [Open Metadata Repository Services (OMRS)](/services/omrs) that provides access to a metadata repository, either local or remote. It could be a [Metadata Access Server](/concepts/metadata-access-server), [Repository Proxy](/concepts/repository-proxy) or a third party metadata service. ### CohortMember -A *`CohortMember`* service describes the cohort services provided by the [Open Metadata Repository Services (OMRS)](/services/omrs) that runs in a [cohort member server](/concepts/cohort-member). +A *CohortMember* service describes the cohort services provided by the [Open Metadata Repository Services (OMRS)](/services/omrs) that runs in a [cohort member server](/concepts/cohort-member). --8<-- "snippets/abbr.md" diff --git a/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio b/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio index fe2cdf3e60..c05df6a732 100644 --- a/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio +++ b/site/docs/types/0/area-0-basic-types-and-infrastructure.drawio @@ -1,6 +1,6 @@ - + - + @@ -320,7 +320,7 @@ - + @@ -376,8 +376,8 @@ - - + + @@ -1023,7 +1023,7 @@ - + @@ -1056,8 +1056,8 @@ - - + + @@ -1095,13 +1095,13 @@ - + - + @@ -1118,6 +1118,108 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2079,7 +2181,7 @@ - + @@ -2093,94 +2195,94 @@ - + - + - + - + - + - + - - + + - + - - + + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + @@ -2309,7 +2411,7 @@ - + @@ -2338,7 +2440,7 @@ - + @@ -2364,7 +2466,7 @@ - + @@ -2522,7 +2624,7 @@ - + @@ -2590,7 +2692,7 @@ - + @@ -2644,7 +2746,7 @@ - + @@ -2734,7 +2836,7 @@ - + @@ -2820,7 +2922,7 @@ - + @@ -2976,7 +3078,7 @@ - + @@ -3242,7 +3344,7 @@ - + diff --git a/site/docs/types/0/external-identifier-example.svg b/site/docs/types/0/external-identifier-example.svg new file mode 100644 index 0000000000..e5007bad16 --- /dev/null +++ b/site/docs/types/0/external-identifier-example.svg @@ -0,0 +1,4 @@ + + + +
Apache Atlas
Asset Manager
Apache Atlas...
SoftwareCapability
SoftwareCapability
Apache Atlas GUID for
Synchronized Glossary Term
Apache Atlas GUID for...
ExternalId
ExternalId
Synchronized
Glossary Term
Synchronized...
GlossaryTerm
GlossaryTerm
ExternalIdScope
ExternalIdScope
ExternalIdLink
ExternalIdLink
Scope where the external identifier is known
Scope where the exte...
Equivalent resource in the open metadata
ecosystem
Equivalent resource...
Apache Atlas GUID for
Synchronized Glossary Term
Apache Atlas GUID for...
ExternalId
ExternalId
Apache Atlas GUID for
Synchronized Glossary Term
Apache Atlas GUID for...
ExternalId
ExternalId
 DataHub identifier for
Synchronized Glossary Term
DataHub identifier for...
ExternalId
ExternalId
ExternalIdLink
ExternalIdLink
DataHub
Asset Manager
DataHub...
SoftwareCapability
SoftwareCapability
ExternalIdScope
ExternalIdScope
Identifier of the equivalent
glossary term in the
third party systems  
Identifier of the equivale...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/1/0112-People.svg b/site/docs/types/1/0112-People.svg index 79d9b76f0e..cffd1814f3 100644 --- a/site/docs/types/1/0112-People.svg +++ b/site/docs/types/1/0112-People.svg @@ -1,4 +1,4 @@ - + -
0112 - People
0112 - People
«entity»
Person
«entity»...
title : string
initials : string
givenNames : string
surname : string
fullName : string
pronouns : string
jobTitle : string
employeeNumber : string
employeeType : string
preferredLanguage : string
isPublic : boolean
title : string...
«entity»
PersonRole
«entity»...
name : string
identifier : string
description : string
scope : string
headCount : int
name : string...
«relationship»
PersonalContribution
«relationship»...
*
*
*
*
rolePerformers
rolePerformers
performsRoles
performsRoles
«entity»
ActorProfile
«entity»...
«relationship»
Peer
«relationship»...
*
*
*
*
peers
peers
peers
peers
«entity»
ContributionRecord
«entity»...
isPublic : boolean
karmaPoints : long
isPublic : boolean...
contributionRecord
contributionRecord
contributor
contributor
0..1
0..1
0..1
0..1
«relationship»
PersonRoleAppointment
«relationship»...
isPublic : boolean
isPublic : boolean
«entity»
Referenceable
«entity»...
«entity»
Actor
«entity»...Text is not SVG - cannot display \ No newline at end of file +
0112 - People
0112 - People
«entity»
Person
«entity»...
title : string
initials : string
givenNames : string
surname : string
fullName : string
pronouns : string
jobTitle : string
employeeNumber : string
employeeType : string
preferredLanguage : string
residentCountry : string
timeZone :  string
isPublic : boolean
title : string...
«entity»
PersonRole
«entity»...
name : string
identifier : string
description : string
scope : string
headCount : int
name : string...
«relationship»
PersonalContribution
«relationship»...
*
*
*
*
rolePerformers
rolePerformers
performsRoles
performsRoles
«entity»
ActorProfile
«entity»...
«relationship»
Peer
«relationship»...
*
*
*
*
peers
peers
peers
peers
«entity»
ContributionRecord
«entity»...
isPublic : boolean
karmaPoints : long
isPublic : boolean...
contributionRecord
contributionRecord
contributor
contributor
0..1
0..1
0..1
0..1
«relationship»
PersonRoleAppointment
«relationship»...
isPublic : boolean
isPublic : boolean
«entity»
Referenceable
«entity»...
«entity»
Actor
«entity»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/1/area-1-collaboration.drawio b/site/docs/types/1/area-1-collaboration.drawio index 004275be04..30ff4c3de5 100644 --- a/site/docs/types/1/area-1-collaboration.drawio +++ b/site/docs/types/1/area-1-collaboration.drawio @@ -1 +1,1527 @@ -3ZrBctowEEC/hmNmZMkY50goJDmkZUpm2h6FLYwa2UuFCKRf33UQYCN1pj0QYx8yEbuLjZ7fKCuFHhvlu3vNV8snSIXqUZLueuxTj9IgiCP8VUbe9pHBgO0DmZapLToFZvK3sEFioxuZinWt0AAoI1f1YAJFIRJTi3GtYVsvW4Cq33XFM+EEZglXbvSbTM3yMK/o9pR4EDJb2lvHdLBP5PxQbGeyXvIUtpUQG/fYSAOY/SjfjYQq4R247N83+Uv2+MG0KMy/vGG8/fJ1Nkxfvj8+rza/Ivojnqobur/KK1cbO+GhFhwjeHlygz8jUIrPQXMjobATMW8HOho2RSrKG5Aeu9supRGzFU/K7BZ9wNjS5ApfBTjkSmYFjpVY4Ae+exXaSAQ9tGEDZb07KzvRslzsKiE7y3sBuTD6DUsOWWuYNS60c9xWHh+J7ENZVh7dYGCD3DqTHS99wooDS/Y/KIcOZYdkhdPaaHgRCB70e47N437YJxWCCcIRmLxbSKUqhYs4EUmC8UzzVGJRJTeZEBKTixCmLmEfX3opvJGDlwQMbxap0rO5xlFWjqYafuIasXYz7RA7ojXsrB+6Zoce8PGlwN92zOtzwAeYFcA+vkF4Ib6HP4J1s/uuv09CGFlkrTX7uEhfjdpB4KBvt9sO4sbldnsP8v6MzxUeQZ5vCmmkaK3f0VlLErvsP1hv1jG9zwlHUcN2uz0fCfoeuydCpHOevLRVbRoG1+Z2v2NuO4gbl9vXcUceuT+D6cyiHQWN9ySDjol9Tpg03ZJ4tA48Wg8TA7q9XtcbwaB5r+OueX1GuHGv3Y06ik09JyQCVki8pWIfoV6N2dTdwLfbbAdx02pT75rtOSJ5Fjxv7ZId0qsz2929t9tsB3HjZrtbdNxBesweadimWDiDjU5kkbVX8ms7JaFd+8eNg7jpnaS7oyEBG3g7bglFZ9bv5s+3u9ZyO4SbPt6m7ikUNiYetcdFJgvsuclUA8Jr8XkJubomxT2narflDuIPbFJMsGPZfbGeSMbgUYKcPky9JybUd2KyXiPBvJw5NioJrErNWyF1fSuP68rHOe0l3rW++xwwvZjR+PL0bav3XOU7a2z8Bw==5VzbcuI4EP0aXraKKct3HoGQSbYyk0wuu5N92XJsAdoxFmOLEPL1K9uSsS0BTrAdAlSlYrd1sfqoW91Hgo42nL18DZ359Bv2oN9RFe+lo511VBUA26T/YskqlViWlgomIfJYobXgDr1CJlSYdIE8GBUKEox9guZFoYuDALqkIHPCEC+LxcbYL/Y6dyZQENy5ji9K/0YemfJxmb31gwuIJlPWta1a6YOZwwuzkURTx8PLnEgbdbRhiDFJr2YvQ+jHyuN6GS2vb+/63q+fl/fzxW9TfbRv/G7a2PlbqmRDCGFA6m1aTZt+dvwF05cCYuSULv3ruwSHERs8WXGNhngReDBuVelog+UUEXg3d9z46ZLOISqbkplP7wC9dHw0Cei1D8f01QfPMCSIgtNnYoLj8hXHx/QQtwFfcuiy8X6FeAZJuKJF+FM2VdnU1dlol7l5oJgM3WluDlgWEzps8k2yptcKphdMx3J935Dlj5//wh+vM3v1fHeNgPHnVVcXtAk9Ol3ZLQ7JFE9w4PijtXRQ1Pe6zBWOlZdo+T9IyIrZnrMguIgBfEHkZ1z9i8HuHllj8fXZS/5mxW8Crx9bH7198rH7KxWdo3i4vADrEFhvhjDCi9CFW8ox4IgTTiDZok/WXqzErRMihL5D0HPRJ+yD7ra3zllTZ6h2+lRXBJFVfD2gijL92BKeQno1IYnqmIQLEru7CfEYUfzZQ/o+T+sKpTmUwzoiIf4Fh9jHYfJMe7IN3VBylujSt4H04YA27+cKjm0Xui6VT0LHQ7RQ7tn5uaLYSoOWqvV0Ojnzxqr1bMFaVU1irEYNtiodhC6ieXyKL7lIVXSRMgepNqVzQ9B54MyoEVAV9BPVhyiYiCZEIww3RHOCcLCzbK3rWXv2oOsfbQ92Pd7tFo4hVZcLnaca3JtnPpmGWcXKxmN1k5WNx17PNBuE01SsAphAUcRQRGZojYFpbgRzMYOhk1jSDkSjuRNsRHmIA+K45Buk0Yp3v5pvQDqTSRvbbLNH4Hj5FGBTwgQS+wZtTgneWWHBS5Hw0HNB/+bvRZz1DOhYSZepOfa5iVfMnnL0Rt/6l1eJX6ZV42FnqCftFkHfr6ubi+vvo6wr0GRXw4v+fdaT2uigbq/PL6/Ww9Ka7Kw/HF4/fF+PTG+ys+v7i9Ft1lWvt6mvfVZswT8oyWfTShB/2lvnBS/AaYldXiCrWL8bAKIbeM86z1aAM0gc5Ecnm8dI/XqrcRsQSZ5KQbWbApgu3tXK5pd7VmNjILCh8h0Mn5H7xh7/SoZWrvIp43zTrDhfempT82V/kkpCPvHrhHpKiag1xRTgAAoMk4TUkpJfnhNNk55B/TwUYODURzCxqjcY0RfM5oGh9ErzwNBVLuHNpGwYq1kCOXuVPXAXk+/U8adjw0E0RfO3uv/7KZ0Zk+mJuH8VCDACi0s+LrCXJHvHG1JJMJC5VBkCjRHDwBIQ+EOAIA6Zi3oOYYReE5okhWUem37ybsagY5zFOqV+MWI0vMw45PstNNl26Qp5n7jQrt4gGLpdBkOX8bptUoxA5LFOFQvjw7HoCViwuE7cdjxGSMRlX2YerTLwqsgCuaVM7hSAAYpgK5YATK8hYKRbnYag+IPdOqbDzVWKbx95e/HNulpydyBbztnplM+156zWRNc8RDC89Fil04jWLZ5CH8yWMw/Mc2AuElx20iEeigh9sEBxLvxdRu98AjakDEj1Pc+muBB+GKxeLkTpfEYuRG2JC7FKm6W62i4PotbKg7BzPKfmWu0ihoYNPpwGUUUaJMQ+jDlpqcMsQxsX/vpwebbbF285jXK0tEsZcVNTq+Ftak3hfaqJvmWVllFFE6FoNacU03zlyxdwEmCYqnCSSwKH3iYcmpjiL9bRP4KnkeJbil0KM3QBlqYSfDksYiY354eAm8Nj68H8DzUTVTQTo1Wv1do+KOdL8myJkBdgOi8KecEG9kaaF7wNtvccz99KThwIV8IPO9eczRW4MXC4qBkVUQMfhhq2yAMgLh794/uPl4oDuiPclRxLfA/BdYXd9FjraRw55q5yC7fV2IljKYwitbV/Vl0bpp8jqwamWVoks8CxjaxaCusBfaONP9n5jTYpxVYTdSZVkla3Q5UzZ2o58eOUR/3M2bZh5kzciSLsosRGpecIj5YOkZhqT6lKgWm98tGv2sxV5DY5RNDj3rRK/rcznZshz0uN/a2ZSRFXZqt5N8xETUGXQZJZkWTpbIo/kYImEpdr0NhKeOqY2RXCHemRr6Ywe9+Jr2OGCOyGqFWreh89fMQIaVbZ8bXHG0sREhniFmNLCVFT4ciLjMKpdORFGpNGNGAjJSohkeXIhJpDVYblTpKg9pB2r8NR+xM7R/e7GlI9VUXXPih0a2KAkt/VOBGqwCilgtnq28bZpm3H5mr7EkJLiZygSLOiIrWmFCkSaaexrW0YRSRAT5Ml1PUEKPR2/cteKT+y/n00bfQ/5Vxtc5s4EP41/nIzyfD+8tF2kvbmeq0vcefafulgkG01GHFCru3++pNA2IDkQBIgachMJrCsZbSPdvdZvWSkTzf7d9iL13+jAIQjTQn2I/1qpGmq6lj0D5McMolt65lghWHAlU6CO/gLcKHCpVsYgKSkSBAKCYzLQh9FEfBJSeZhjHZltSUKy98aeysgCO58LxSl/8KArPN+We7pwXsAV2v+1Y5mZw82Xq7Me5KsvQDtCiL9eqRPMUIku9rspyBkxsvtcr37dHs3Du6//DmPt/9Z2ldnFl5kjd085iPHLmAQkXab1rKmf3rhlttLUVUmuaC/M4Biasas8+SQWxSjbRQA1qoy0ie7NSTgLvZ89nRHxxCVrckmpHcqvfRCuIrodQiW9NUnPwEmkIIz5mKCmH7D/nE7sDbAvoAu7+87gDaA4ANVyZ/yocqHrsF7uyuMA8Xi6K4LY8C2udDjg291bPpkYHrBbfwIe+uCvUdTbTSe0I5DcmDXE2oNK2S2WmB6tSKpfbgkF8wATlCUi+mbLE6qFbQKWCQEo3swRSHC6TN94ZiGqRRQ8ul7APpwsoRhWFBcOj7wfSpfYS+AVKnw7OZGURylQxR1RynBqBm6gKOmS2A0u0LREFF8e2avOI8mOo/MdbSubG4JNqcew6ITtcE4tT2G0Ur0HRhBAr0wqVVcwZ8g+uhtQL1qssURVazVW27D8GMTxRijiEbV+m/+gRbzRt0GmzhEBwA+bjcLOrSaqs8PcZO3BUuAMQg+eNFqy/JvLQjJbLsIoX9UXFAKALyo1dzSW/xxG8Yf1bY7cgZbMBwIKOvhtwiTNVqhyAuvT9JJ2bQFM4I9JF+4mF1/ZdeXJruL6Mt+Kd9+5Zo/ACEHTve8LUFUdPreD4jl9az1KBgzGkdvIxSBTHIDWX+vHh++ErTFPnhAz+EU08Mr8FB7HEJmtgfHAAahR2hkKPPStvF06hMK9SzqbSmSNW4h5Bsl/ZGnkWX6I8tFGBHac8RkF26XicZwLDq8ih6m5h5XZGq2JNm0gEf87YOP9dvFdO7Mx593V/Pw3fw42p7uX1JfkPiMzA8zZ8s9kV9f7Ys3h/zm5F2LEPn3gnvRO/6F1ICte5uqie4mNajyqtwtf+12aPgtYvl4EFRcU8qp0DJER+2XiqtiRdWImsGAYb2EDbhRABIfwziNhbXM0EcNCNQaeMGUhgty1ITs+jfgQsIAcBsOAMfoagAYrXKhGrbSlPUEXrJO21fLNMpQnCKRUi/VI7GiwQRSkzCf74IYqaYYqv+KtU/f98HnK231/Z/N529k837RPjPiH52hdJAfs75bzvm6Y+csIG8ke1X+ucowOb7IM0aOeSYLZD1DUbKG8SNzgUdDNQUTLrZZuBhEVrCMCpSWlkNZDAtqr3lBnC94XoB9HqvuOCxXETDUxgjonZEsW0DgDwEC2j9StjMGCfzlLVIFZt2YuX/6buZkZF4xm9K4m3BOK3MN+RxzQpGlyXiehugLo8t6xrbKYNiWDIw+J89UscAcCBaGWiGszgsj4QpIYFo/0OyxRHhDc8gwYHGdqosIsLh9wpI3XIAlziBJWHk3EFSqzmKL5L5fVFQBlSfV6WOfIDzDiJKmoVTq1UlrVX/xSl0RzPzaCrXHgVFfcnHFrksp3bRLWB8XKPoqpM656RMLKTYnMwgnNSsLS5aEnPRaNeVF0jCqpor5dbWh+TsrmTRxY8EwaLpuVTzBfGGarolzQ8NEwlCMF0ZCnMiJwVDqJF0rM3JK6wQ0zF7RECd1BoSGWkZDPsvWFR7SyXqxbH1SgVSctL4FPsKBnIGJDb05Tma7lVTU525DKcYiuz6/naoK0L2HN94s87RcOURsTfI5rK4nfiZA0XTjVWdQiIvNvsRzBhAKLbuyO9QQA6HaFU+QQiNS5yM0NHYMAhO14i626C52n5CIHFq5vBS3Vr1FLGzTrMXC6BMLkUUPBgsr3+71QEbvFYvedu++qilSqSl4OdP3ZhNT7WyG9KFutrnThC1LjePUGzcMi2HMmAobHZwep0yl0Irrud0cdnjVM6lVWEzJPEVnU6nyoqnd1abf9IiE3DT5Mdy6nYCtb9qWB2fXtSv7/48ZuiY8C005rlZtSnHLTWXm6SzSy9bCBJL1Zo9yOG5565PqymbIujrKIQdEO5N7HzlFdsvOvIHIBykXfm62DayFZVpNsu1yqZ3Ltstl4FpWl3BqRvVkjiEFVHYStLNFSrOTyF46dFOJyogOnlJUPnOY50x05xnieFPJHqeTPeldfrTn6VH/ofOztQfjcn95JUd13O7TuFbO42WUbLPFodAypHpDSK2XQlQakMV82AKi6qhKzJ5AvqTYdo9ZP9zrWechW1pmSvfhDaRSrZ6UMjVHtgu8peULenv6RzQZlz39Ox/9+n8=7Vxtc6o4FP41ftmZOpDw5kfty92d7Z12ru5s+xElKi0SN4Sq99dvgCCEpGpXwN6ynekUDiEk58l58pwE2oPXq+034q6X37GHgh7QvG0P3vQA0HXHYn8Syy6z2DbMDAvie7xQYRj7PxE3atwa+x6KhIIU44D6a9E4w2GIZlSwuYTgjVhsjgPxqWt3gSTDeOYGsvVv36PLvF/WoLjwO/IXS/5oB9jZhZWbF+Y9iZauhzclE7ztwWuCMc2OVttrFCTOy/1yu3n4MR56r09/TNbxPxZ4dh6Dq6yyu4/csu8CQSGtt2qQVf3mBjH3l6brJrNcsd8JclcR7zvd5Q4lOA49lFSq9eBos/QpGq/dWXJ1w4YQsy3pKmBnOjt0A38RsuMAzVnLR2+IUJ9hM+RmipPyJ3aPuyGpA21L4PLufkN4hSjZsSL5VT5S+cg1eGc3pWGgWRzcZWkI2DY3unzsLfZVF/5lB9zFH3C3Ibm7dw16wxHruE93yfGIecMKEl9NCTta0NQ/3JIbEmByI2vHtChYwaqEREQJfkXXOMAkvQanjmmYWgmjGWsFYhdHcz8ISgXnzgzNZsy+IK7ns0Kla3d3muZoDWJo2LYAIoBQQhFABYhmUxiaMoZfz+2V0AFy6KgCBzTlc1vyOWUhMNmtEbP24DB1P/HDhRw8bPZhwTX3mYurRWsktogVZHXep2c3oM2AcE4MCMNuCBxH8iTy2BTMTzGhS7zAoRvcFtaR6OuSX9HWp0/cnBw/J8d9MzkLWWOfxNNnXvIFUbrj2sONKWam4rn3OJllstpDb5hoCnYa4hBlljs/6e/Nx8MpwjGZoQPluMSgLlmgQ/XxaSFx28EhQFDgUv9NVDa14zk4TnAsgNxwkSJ5JE4k/tPSHzWtzdMfFTcSTFnPcWK7GjRJfKYD+6YQYHquBsq6wVaQX1N46LL/Gwqww8FxapB5brRMn6zXH1H5vFRfqPBbH7HPGrgfBpapi9oROmIVWUzzuyoA75txBubwHaWY9QuH0dJff0gv3iPXQyS9rRvK0YYVCHXFRKm3qRx1Wf6vceSnvFanNjmPc7miaQ0W60RYYGP8Kiv63+QZD22p6GWCIv+nO00LJL5dJ1yQts0c9cybxKOMJKOML3VVtKizYS4lJymfMmnQHBSWLs50EA5kKNqU+brVVSSAIyBhQllztIuEnHEFbAKJspWHLiBiHo8Nu1VEHAkRWkzr3cDEOB4lg1YxUaRK/2U9bzijmDwSzOQT6og6qy5j6EDBeK2u6+2X+etJsy6fTDWdIxlmdSXKFKt4J0difnF3pWKciN59DhwAcahYdrm6o+XzdhWDI2vBO3db6qftm5vlqI1lfjkO9WV+Y0riGY1JV6hFNyoLOIaCWlRqqjlqARKmHgrQIsXzNmJTano0jBkBkGTiyLPBKcZMdoUSSF8nHZTAGpwIVmPpIJCXXrR+X5cw+IoKy9ArlJ4rrhIaRpsKC8hLJt3ICA1Y2YEbWPm69KVyQiCvk0TxGpHO5ISGVokOIEdHqzkhkNdLonjaGTzgQMQD6kARI61uVOcZy7kZ4SPL6XH4A3coH7QELE1TlQ/qLYq2/HWherfd8uNi203I8OpKGz8G09Ftt3z3q+mU0oSiGjRAu9tuUF5kOzP5+o5W005tuwFHEyHUVKSsD8wWQxnK63Td23iTgDFPBqaxXMuQcy1aBE2TIubgC6ktSspK+nvxJW2j5YTrk+BgGtXXfi6+BWe0vC39aZG4+La0IW+GrlKGepg3nWt9ElAMeBwSXWsIE//h/uXnZILiMX25iv9cGY/bv/JXJs8Q50rdrNDX6k2daYBnr5I8zwRA/p0Jv4VXpztn6XKlF7h3j75gmnNZ+2+YHmp2HZ9BZFKhI9oaAqdvi5FoqdbLm9o3VYIp71c1oq1bUskKF9uq/SOVi2FTLjYvQ3bq9+331262vPLsbLc/O0KSFUqUObNmkgS/JkmC+kgye0moIyRpAbMPPhtJynnur0ySChe3SpLq8D2bJI9+JQHE75DKrKgls4ZAfpiBLXDfO5TbxvqtcSIF2pdiQDWi5897Kkg5jBzVylQHpG/L6lqSPzhnssYUl5KT/ErEMKOVIZXaSoOq5sFif/r5Utkcq5nBYsuDpcbxUH6GOOj+HyxHBot6B8muiHeYf/Vz9ut77LT4Xw9Z8eI/ZsDbfwE=5Vltc9o4EP41/tIZOn7BL3wEkjR0em2mMHfJfRO2bOtiLE6WA+TX38qWwUYuIQHTmYOZBGm1lrXPs1rtCs0aL9ZfGFrGf9AAJ5qpB2vNutFM0zA8B76EZFNKXNcqBREjgVTaCabkFUuhLqU5CXDWUOSUJpwsm0Kfpin2eUOGGKOrplpIk+ZblyjCimDqo0SV/kUCHld2OYPdwD0mUSxf7ZluObBAlbK0JItRQFc1kXWrWWNGKS9bi/UYJwK8Cpfb1Y+f02Hw/DiZLfN/HfPJe0h65WR373lkawLDKT/v1GY59QtKcomXbhgAgN6Dv8kM/j0wGpIEGCwx4JsKWEbzNMBicl2zRquYcDxdIl+MrsCVQBbzRQI9A5ooIVEK7QSHYMHoBTNOgKOhFHMq9I80U8Ih5sDrGsnS7C+YLjBnG1CpRqXHSg/uS6NXNXfQHUlyXHMF15VCJH0w2k69wxkaEup3wN5XYNfGpjYcgeGEb0R7BGg4icBqzqAV8QIfKakEk5lkpxqBxcx32nuE1ejIOKPPeEwTyooxa+7ZfVuvEeXDUjAMjmD6pKYYej72fZBHDAUElGpjd3e67ukdEml5ZoNJ09MVKk2rhUm7KyJtlcj/H+x7+8dU90/b7jG7wtx7G/NTYtOlnNeyjnReqysgBwpuOICzU3Yp4zGNaIqS25101ES2hiJeE/4oxaL9JNqfbdFLYbGPze6T1PwHc76RSQPKOQXR7r3fqDgWytnTYCiSAeimNMWl5I4Ie2/e7/oZzZmPD+lJyDliET40oQzkAreDPsBwgjh5aeYkZye0WvahrcEZQWlUcPnGvlCilV582oNQWHzaIhmjHEynQtYbdBmm+p732W5GKsNVj3q3JVR1RYjZQshHDvqhzym77rPecFQuOzvrPRxOv9mz+eKReP7X53gwX3zv6QrKp4TLNwLa8YHx18G16NysG72N7GUQ2ni1Agr+WcnkGozzB1VPjanD7HWBX6mxiu+9dPLnd+/+716F8/liqnz0gRKwY+tgtq43z2Nnz3FKg+RT9cLqjYns/bKhtFiZqHDCrT0f90tLiTGf1KiP17zpgAxn5BXNCwXhEEuxvGLB9kizb0R4ALfLSg802qJFexGXwXlC0mhWeCicj13G+z0C3ZaEqqPEtJUItaq7TiJs+zcToVZlAc58RuY4GG22x+gVMKOEphZmjItS4yjU5JlgZZhlmF8JJ/ZemtoWuAaXJMU9a2ZzoBA8T84ToCwu3nxaktIKhfd7ko++Y180Z2i5QynqktIsmmYxWb7jGnKShgxBoZH7PGf4ugoVp793vdxvOf2MSxYqA5Xcky7ITrsI6PhaTYF/cCT81hmK/kP1y8k1f3kinriHAmfu2M4xeygMzV/toTAMBo7TZQZZ3crsl1D1PKXtkvkDewi6u9/syni6++XTuv0P7Vpbb+MoFP41eVmpFb5gO4+5tDNaTXdnktmdaV9W1CYxHcekmDZJf/2CTWJs3FyaSzVtI1U1BzgO5/vgHD6l5fQm808MTeMrGuGkZYNo3nL6Ldu2rMAT/6RlUVh83ykMY0YiNag0DMkTVkagrA8kwlllIKc04WRaNYY0TXHIKzbEGJ1Vh41oUn3rFI2xYRiGKDGtP0jE4+W6vHbZ8RmTcaxeHdh+0TFBy8FqJVmMIjrTTM5Fy+kxSnnxNJn3cCKDt4zL33+d/ZmlBGazf8J+u+Nai4F7Vji73GXKagkMp/ywru3C9SNKHlS8gCUXB87EXyfLyDidyJfaYBjSqcCxiARfLMPL6EMaYfkK0HK6s5hwPJyiUPbOBKGELeaTRLQs8YgS4U88J3gk1tF9xIwTgVRHmTmV47dcrAqK9IHnGtRq8Z8wnWDOFmLIslfxVvHYVUufaaQAnoI61gjh+8qIFBPHK9dltMWDCvgOwXeN4Bvh1YKXcUZ/4R5NKMv7nNsAuhBoYQ1FxLDo7I5IkmgDR0GIw1DYxwxFRAzS+i4vAQjA6cJum2FvCrp9gJjPBje9f7uAhfO7/+7SwdW9dfND7ZFKzHt2q9MVqyZ8IZ+7IhReIvl5y8TTmOfBUZalYYBHWEQqxOhWQKV6xRe6LWfsgGTk3XrQ2wbJ0ch+DsnRKGp73hGR9FxQgdJqA3MLNYEJjwWmZUQZR+LwV03KeEzHNEXJRWntVg8sDRU8J/yn9nwth5xD0brDnC9UZkMPnApT6fsLladWMSuNOjJjyXcLghWWS5Lob+jP1ZvzxkI1doMsow8sxGviolIbR2yM+ebNIIO2lgAMJ4iTx2pGbUJTTf1KSZ4zFHFcq0oc24JVF8WC1Cw9hdUcwfYGR8WKDUc5uVbreTnfzPN5H76VbElpiitsATuwLkJZnPvXSQzOXT+oEBl4axkoG18xIyJE8uDZk5X3o45zh8D8+v4b/3YPPW4l7qnYBu06SfytSGI6cmHVke9vRdtDsc0sD9CqIPu+mIq0I0Dq5HxkJB2bWUvU3SEjU05oaozdp4wz8hfIP8+lJfkxi79jZan6YeMs63ktS4lvfMIsBZ/BEUd5TW1gIRbKqwFnOCNPeaVR4DOVlMu/Juy2YF8GVxwQWXFW6IX2qo5oLrUzAbFgw/f8LBF8Ox4q0K2WgY21AzhlIeg9i0on5JSZN523CIvj185K+7Vh8Q1Y/ngXSHhWsBEJ+5RABO8UCMevZX3wykC0DSA+bqxbYmlbr3hjXVcDG2AWNS9Ns5hMt4a01OdUIbEnqL+HoGTcJv0TFnjOMO5lg5svPXhx0yd9+DS3rxpE1I8d+sId6non3KGNYDaIsh9gvkwgbATTPyWY0IjyroJNo+TSIM1sJSSukV9SsdyfekObJZvltLy1nJdxxHhNRsptmpC0SZbcjQ66zLPuNNTFx3Ub7XBy0F5M8fZmSgMDStSvTSwOK/odWlduDJJ/aMieEd4c+7xahLsBPN9O6jWdef65XfPmBHVvh5Py1gXuMGXfkKNfOKZJhNl7KflcqwIgDBoSS+CcMLGYt+OsRGVAE1OcfbuCaw0cz7G2A8c51llu3phDOpkQcVsSR25kQGGqGBtFiQmJoiIF7Kp3VKFUOUDfa8p0LLTq8obbcCVuH0neaATLMq/EOlrdxfvGCzq1n6001dTWSQGzDMC2UQbfMEZOsPnecyzJsBkiU5F45xDV5fWmY+9AEIlm+ZvAooAsf1npXPwP7Vxbc5s4FP41fulMMiBx82Pu3dnttBNnp+2+KSDbNBi5IMdxf/0KkMxFso0bUNKQzHQKR0JG5zvn6OiT0AheLJ5uErScfyIBjkbACJ5G8HIEgGl6Dvsvk2wKievCQjBLwoBXKgWT8BfmQoNLV2GA01pFSkhEw2Vd6JM4xj6tyVCSkHW92pRE9V9dohmWBBMfRbL0axjQueiXMy4LPuJwNuc/7QG3KFggUZn3JJ2jgKwrIng1ghcJIbS4Wjxd4ChTntDL1frz7eQsePj2191y9dMB370v0UnR2PUxj2y7kOCYdts0KJp+RNGK68swIWvHOGH/viTkB4Mk5d2nG6HThKziAGftGiN4vp6HFE+WyM9K18yKmGxOFxG7M9klisJZzK4jPGUvf/6IExoyeM64mJKsfsseck1kbeCnCr68xzeYLDBNNqyKKOXGyo3X4v1dVyzBcDi+84oVuC4XIm5+s23TpYrZBdeyWuMXq79vY/Nf8w7Or5Llw5T8egAnpimpEwfMYvktSeiczEiMoqtSel5XeFnnH5JpL1fzD0zphrsfWlFSBwE/hfRb9vipze++88ay68un6s2G36QUJfQsc0EmiEmMhew6zLpcPBAHosZ9RPyHQlSvwF/KdI/GOSWrxMd76nF02UvN8N72uE9nmt5rNgmOEA0f67HjOTaw77UrXje6AKMzpiwa0k12fc405USZv9wn7GpGc91xiRBw/xRy9ir3Zd2GjVVsIaUJecAXJCJJXgbvPduyjYqr+uxFMCs8nzIgKxWnno99n8lnCQpCVqlSdn1tGJ7RoytbllHzZRaqJWcGUOHLdgeurOyCJcP49tTeiKBAjqCq+An60rnz7OipjIqgGhbzopi9bSViZrfbkFkGPR4W6zFPEYqVIbvjYOi1DYbmqwqGbi+IVke5EsFvVXSrZd0AunsszX+wLMrvRFnHZiDGuoN2AEHXdpA/yjSJNpUKSxLGRQ7JW/6SCcoAA9wMhGqMMS0hKe2qaLW0su3r/b7heVL4ZjMVNgJPQxaEWQCDZ7lZJmE8kwfjGC3wwUps4uMn4ZKGJD5YN8+sLhEtWw3ym2a9ZYTYRCm4ioM2lYsMYUIRXaXSK3SY1Wsb9F1PGMuhYd80+4pYomEpf/MjlKbMfnyUQ942j7tD6cNAkjho1vG0rLGEpmlDjUmcKU+B/wjPkDQ5bqlJ2Jsm3R1uceS05hZPMdOWj9F9hJ/tF4Fz79hOG7+YTsEuv5hOg7HjaIxzptkyyvXmFwD2n5eBWmK2zYyKQreRmRFmV7XMbAfroSPVbss7wFeRYVmgMYg286vDTzi2jowM2FIEWaAYzXCwk4pk/kXrppbgNPyVh47CpLh2WG37fGRfZpGAGUrKSSlVYFBTlCmL+Cxtusttis39ewztRkP7IoxXQ7upc9YNHAmYD4OAwmoBBdCKhDzI5i7C1LGlA/vDZS+Hr9FBzCYV6ChmBZpdRJ5OitwlCB9riDg/V9ma0XkGzAlXbTYv41a/LW9mSEzBsZAZp6dmJTeqFlXE+S/vSJoG4a6mzBFbWq1i3EcWJa67Z7AClM7zX+4jaTJ6yYYOJy+2KaKDaKRI3PhzDYg7yGNET6WZUNEzEqfzcHnsMs/HECco8eebgVAFY7NBC+6I8p5OtgAqqJ9nsQUSOEb+t2samv3p4xhkCFy3LQTA7gsCBWHzHPbtAi2WKFPnQNwKNNl2MWZWEXU9nU6lWA//Eyg4WZV2S1X2xsFBq9N8Ywc102XGUdtlwjme6i6T4xORg6yMpykPsZoLW9AaNzORImuSMhF5lcxpGtuWOj+Q1DyXLSpfui1bBIGpgy2CMlsU4CWzQmYlg+aLoFigqoYhrSwFHCxf1HQFW7leqRULmTEq3CT9HOtwk9fCGTWdxFIksnqB0csYfXiniw56qyEHTr0m8U4XlbsrX4gugmXapokusvqgiy55KuQPhi+Ssm3FGrpetsiS2aJgC8tktVigvAP7N4ZJUL0hfqnhedtV5oOQiaSme8h2sUtHbmI58ylJBuJ5zc0rrjFuu0mvP9frlgjZs634xUfSvgdI27Zq8DqG1Wp4lBqCNmw21JIO6WyklSmD54+0dxgtBuLpEDZZKNtV+brmYVZmGyjD5JZE8hbtNzuWytA40Gg9nPa1VdqSyYdhEEE2rAdNOFatnGmdXFoy3zAQLOyGY3jK/EQrFmMJC/6txjXxV8OgrW0HNmds5YpaBRhPJzC2PBdPV8sly9mYUvKkfhjgWFYbn+ltix25GV/9F8++rsGtP7mZfb0ktwvF97bcZ9JPxS7hQSDjuE2+Cr44MnJaPYyRxXEbAQzoI5CVQMiZMHeRwkP6DV6vZKXHs93mUvlYtQq33fiiBRnFx1Hviz1azQJIOyiUZqHVXz1J9UNY71GqgmfEfbNZnl0fPi27x+WefR3tlIMqwvsiQ2IYTJRjNCcPFmi9etAFE6WE1lSs5L1dykmBgdt6i3YX+4PVGOz6Ov/3TlfiidNA3Mp16tHRAcppX1/bg9WAgo6dSpN7SLr0lHysSpfi4+vOdQl2bTU41jnYVILEBcM+SMewgYqq0usYvXyhX08eO/0If/+ZWtoOSFKck7XXV174632v3DS+ZXyE7e1aaZWfKe1VuSdbasE0xU4IMbSbTsvF2hZtWbq/yLP6Oc3iVD427lWeV9H2aDiBk/6j4dRnpcoD/ftZqb87vxZ58UvFPXlfidP4uFBsiD42vjQbMt2uggu7Lc93LqqXp2TDq/8B5VlRb6M4EP41eTmpFbYDIY9N2u6ddKtdbStd++iAAe8SnDNOk+yvvzGYADFJk0vhYTdSFDMejGe++cYzZETmy+0nSVfJZxGydISdcDsi9yOMEfI9+NGSXSmZTEgpiCUPjVIteOI/mRE6RrrmIctbikqIVPFVWxiILGOBasmolGLTVotE2n7qisbMEjwFNLWl//BQJZVd3rSe+JPxODGP9vGknFjSStlYkic0FJuGiDyMyFwKocrRcjtnqXZe5ZeHzZdvT3fhj5e/nlfrfz386n9Nb8rFHi+5ZW+CZJn62KVxufQbTdfGXw4iLkhu4PuZMcWzODfmq13lUynWWcj0us6IzDYJV+xpRQM9u4EoAlmililcIRjSlMcZjFMWweZnb0wqDvDcGbESWv9MC40n9Bps28DXWPyJiSVTcgcq1awJVhO8Y2PvphEJjmfwTRpRMJkYITXhF++Xrl0MA+PlCzxOLI+P5nh0NwPDudrp8Qy84aXaVwsJo1gV/jGSSmCwqeSwlUWtewBXA4xcSfGDzUUqZDFHFr47dp0GTAFshMHkLOJp2lCM/IAFAchjSUMOSo25x0fH8Z0eYcQVHAZHTJAFJCYdOLp9wTi2Yfz13H7AHmyzp4s7uC+f+5bPgTLgbXgguSt8LxuUqMmTKyrVM1/WmiFVzNZjWXiGllh8h3OKv73/3CXP1grOvkoPjjMKXvDoUqe8tNyaubWUtbkujzL6mgQ8GEd9dOuex1J/3FPETC3PsRCqA3MppEpELDKaPtTSWdu3DT+yLVcvRqzHr3oMJurAgc2+tC9fjeZ3ptTOlEV0rYSOoP1z/xb69CtXz8I7Xe7AZSYyVkoeubb3/nKO52ItA3ZKz8AA1IjZqQVNCtB+OxkFkqW0YEWrgPtoQKttn8q7QCmaxQWW7zDDSstO8enOtlHx6UrZUigwXWjZzbTPfEwmXkUpQzLk+XZFM+nIyb0BQoai2Gl6nEuzkOZJ8WTUA6fGH80Vc+tXwWGD+zBw8UGmdQ7yZ0lqc9cBwPttXIF5R/FT1LClXSLLE766tJLNf5NS1nOcQw5P/I6DEvlkwHIWuRakf9iJlW1Vm6+S5fwnXRQK2mkrHXLF3tzZyL3XWAAX85KWqAua7nYwh5QNMfFc0BZq7f7gcJ02l4jT0VkMWeQiz0LCub1FvyUYCBMLjPGgYEyOZLoLu/VvLGLgrIAV8Fyb6EJv4bneOYkuivCxRBdF4dTzBuwHELJLlWF7dmQ3kMvy8Hnerex27pqe67rKsudOzT6CMJ6eewSR3irJ6TF0vmwyiO4eE+DJV5MDpj/stxkz9ixMpkOmP2x3W0WJx8Kjr4Z/QVScNioE2RUCcoaExbW8/gEtVzUuWy580HMJCIFWz4UuerVhXo/sL+q+7jLU3m3BDFTvvtWoDvbeWzXvSPBUS5QW/Y9WDS7rv55K9foPPPLwHw==7Vxbc5s4FP41ftmZZJC4P/qWbHbbJFN7ts2+7BAj22wwcgRO4v76lUBgQKqNG0Qz9namLRyEhM6nc9WRe/pw9XZNvPXyM/ZR2IOa/9bTRz0IAXAs+h+jbDOKbesZYUECnzfaESbBd8SJGqduAh/FlYYJxmESrKvEGY4iNEsqNI8Q/FptNsdhddS1t0ACYTLzQpH6NfCTZT4vy909+B0FiyUf2oF29mDl5Y35TOKl5+PXEkkf9/QhwTjJrlZvQxQy5uV8Gb/efZn0/advN9P15tmCD859eJF1dnXMK8UUCIqSdruGWdcvXrjh/NKAThmgXdC//VkS4ChOuU7ov/cIrylbM2Yk25zDBG8iH7FRtJ4+eF0GCZqsvRl7+krXFKUtk1VI7wC99MJgEdHrEM3pVAYviCQBBavPyQlm7RvOl/OF9YHeSmjz+V8jvEIJ2dIm+VO+dPlSNvjsX0vrQrM42svSmrBtTvT4YlwUXe8YTi84z4/gvy7wvzeEvf6ATjxItux6QLlhhYxXjxQDa5Gk/OGUnDDFI5wT6Xc87hrWsCohEScEP6EhDim27Jn+6JiGqZUwmtGvQPThYB6EYanh3Jmh2YzSF8TzA9qo9OzqStMcTSGGBrAuzQqMEFoCjlCXwGiqQtEQUTw9xteEB4rCIxMdqIrnlsDzyFtR5URZ0E9ZT4JoIYoOtUUzEqyZYjvYNqFSNd2uD3c6I8hjPU6D0hf4XoLEpmsSYMJEO28WUJTEr9ygBn3N8Irq44YDx4mXbOKiEdMYE05qUZ+rk3q7uvpyjXxI5oGuagHaAuOQTz0PfotJssQLHHnheEcdVFlbYiN6C5JvnMyuH9g11XM6swRk+616+8Bb/ouSZMtdLm+TYErajfsJM1ua9R75feZK0dsIRyijXAVsvqPjVUaMN2SG9rRzuJvnkQXa1x9XKIxte9cAQSEVr5eqQ9c6ns5hJU7F34sWKZIHxELQ8Vr6R6665+kfmf4nOEkVC6VduCqVuwmr2h1Yjugb2RL1rgoNtyvp2i8ZTSXM9+JlOjJoX5zySKo9OeGv3uPU+ORrwLIEz0ozqp1kIs3fqyFcfMjPg57PVHCHs5nRWGQZrBs7xUX4chZ+sW1ULSSwnRzNsgw7eod+MQACnr+JShW9JVVxJSgOvnuPaQPGsjVbb+m3mYOeOWJIUFGMM6kEMmDkYWVM1TX136ap1FKnXR0YllEXJT13n8sOS5f+MhBD/XPBQq9hAaCIhcx3bAMLuHq+vRr88bf7h3Pz/fnpr/XSdi80gfHHGjepEZIYq2ZGkN6N3kqPRtv8pnA+K67nJSxc0d176d3uxcKuYrriKnZVhYnUG7qc1odyOUFL6Z8vaI4oJ2coFdT32jvferRMq4m9m8/hj+zdfO67ltVpHggA0WftNg8ExETQCtEAneo5WRrhPXH3+wIMxdF63ReB0G3qi+jKZM0UsGH5naG3icWUdot2cG+iu0MraBt1cTHEtKnTqUcipvBStYv8wnM/Q1gKDEqwuJ3CIkmE/IxRoiCyXaOzCL4MYNdgtKyGCUp1xsgVcXyXyeku1VvjpQ0a8lJZrhe+32M/mXRU7g2rTkeZOqitA8MqIrmOElJQTGC8OyHVj2MqRyu02/c5ceVoOQKQuiw3VZSMdKIec23Ymnr80B65BAOnMQbKfHIoxr/nkZMygQCHBkUwunT7oBi7nisWliHxNzrFQh6rfgri5DwgEU2/BiTaqtOCEyiGq15qytN4FROl4eoHgkaryoouZhGAzIiow8X+gY+Gos0KkWz//ICLRvkX7S2uy0tlZA5bQZP2Qk7Ys7PriwFKFGe3Xp2siiNDwg9eKvy3njc4rfKiUnrB2cyys6knVjzN0bu7H9/2WAKXvkmduh3oabdVzN830s3tP/df7q6/jCeTYkCgcsCv/Zvpze11MRhUOdjw7vP9p/F0XIymqxxNJt/9Qf92dHc7HhWf4LpHCLH8Y082cBBE3AbNRNzV3y/i0s1TMSTXLi+BAIFofA/a0lXg+1nm5lgzXYWQp25UQaJbVUhk29mGIgssRUSMpL000zHJUklnhIxWQ8YQdyRV5fKlyIjxtdd4a+VkQDGqoMg2WMwuQfm5QPtk4HDF0qjuUh9SQCyB/S0k9/Nrxcn9H9fuHAdgOe0vZRKPslRn/XW3VjoAlJWg7ptmOwn/NHLkRvAsUv3A1OpVBo4rCLeyIlQpog3q+k/HXRcBgJJkprLKG+wbz084frUeNO3pZvPny5Bs8mrH/4vcjofTrQVfEtdFWVGBFEtdhan88PvgUlYY3RjEIrguNsFr2LZnEPdNs80d8GlWpXseJrGQxWLv2+hy71uKqbi95JWAuW1yjnbPcVJJU5KM0kOpDY+zHtX4M4pj9hsQZ1NdK6wop/GKcs16AU1ra0rcHTuP7WNJPGtKrLSieFYKhRjDnAcUdVNpmcavBUJynjHwmds7D5DPVGbzQudTSf9Ap17dqdti+hqoOo4lh0lMYKMQsQq9eIq/YvJ0F50RQo5REyJxwwd0Wg1jCsxvPT0H2ww6ikNzxc0usEl9odrJuJSm7mwcN8wHj8aB1n+OQR7Q2KB+5EEHtU6yOSkLacTNwl92/NI+dP6y02OUkqWyL6PzUU5RqvklluJo7EMhyu3oh4+sA5yOkKW3ux/Oy+R69/OD+vg/5VrZcuI4FP0aXqYqKVvyAo9s6U5Vp0kBM5M8GluAum2LlkWA/vqRbNl4UVgCdqYTqlKRr2Qt5+hukluwH2y/UGe1fCAe8ltA87YtOGgBoOtti/8Tkl0isW2YCBYUe7LRXjDBv5EUalK6xh6KCg0ZIT7Dq6LQJWGIXFaQOZSSTbHZnPjFUVfOAlUEE9fxq9J/sceW6bqszr7iK8KLpRy6DeykInDSxnIl0dLxyCYngsMW7FNCWFIKtn3kC/BSXIab0XjS9X4+3U9X618WeG4/+jdJZ3fnvJItgaKQXbdrkHT94vhriZemG7wf7Yb/9UkQrEPMMOcvQYDtUlgpWYceEl1rLdjbLDFDk5XjitoN30hctmSBz590XnR8vAh52UdzPv/eC6IMc4a6UsyIaH/iIiUYog+0zVEsF/0FkQAxuuNN0lq5X+X+NeSSN7nNoFmS4mVuI9i2FDpyBy6yrvco84IE+gzQYQX0Vh+0uj2+cMx2otzjaFi+wGpGeWnBYnykJBWk9OzSGj6Z2b51ibAcHRGj5CfqE5/QuA7O2qZhajmiXD4VxCt7c+z7uYbztotcl8sX1PEwb5Sru7vTtLZWI5FAt2/NApfANlNJjk4AFWyadZFpVMn8eNCXdAhUdUilQaAuzK0K5qETIC5pwW4MPcXhoqpB3A+5FK8YJuHRtgGOolPaXdUwNqc5ELy/5tgV7JDHXbd8JJQtyYKEjj/cS3tFdHNIoi1mT1Isys+izJcIhVWlu6fi47Ns+QMxtpMxi7NmhIv2434jwi8lvYdeV8Qi/DEkIUokd1isd3C+3kVkTV10oF1bxkkOXaBD/UmtFLAd3AYU+Q7DL8WI6Op8to9bQq5ETriImTyiGRVDqcU/tf2bxz+VEaWEObG+w8FNp04LCYFRNJE6UCiYbiusZF18dK4cZjygYIboJwk2DE0r8Gml0eC7mcs0oTqkX/9H51NG0gYnIglrQ1KvAFeT5znsNU71Pp4TLeOR9eu7Gh1c24fIVx8J5hPMdoGpm6UQxCiHh4m7k++VKM4mcgHr1Vw3sYfJykgYLfHqrVYxfvdzWEazU2YSdgxVMKlKCeqzjtWkWru9rSo6XyQr6i9FEf7tzOIGAreV2IDx9MxeyxwIOrhuRoma6ip21OcZETe8PGWYxmrM08QaLaxRCe8NTcGI0WSOplcT478+BRsmgCU2TF3h8RrlwrxOKDhGc8ThclFM0KXmzrNmlmmdYu7mc/CauZvPvY5lNZo765YqtG84GLSaCmFeT56vFd7EnQ62Wa/iadeqJbFO86+jmXXn2lHRZWzbVf39PKm1qVllg9q23ju51hWnHW8xqY88diThmFzBnv4Z4aOhF9NBE6q4zK4Pm7Gm1aOSIAvrp7vV/mhZEfYn9Zck4pcpZM3pezXcN4wTg31Yl/KB6lGIWyKmzlDz4DVmg4GmUWZGFWjqjUaaqb2taNJonruu/PDUGEannJHpqgsXXRU01kfOaycgKFwHiCbe/Ijj4hiG5xyKJNZR5dkymbJL+oFdoA1LLtBS5IeNOkCg+CRBMuHhlwL+1q+1+OClJ5T2RsIsPGOseVlttiNG36fj+97f09E49qG8A3EcnHEf916k/rIBu4OH++/3k+m4mx9Sr3PIb8PuYLgfC9Q51qg3GY7/4aPFQ8E6h1Jp/Gj6NbfSTucMLVZP9MOGTWUdt9JLh2M63gE16bhZQfsKJwZpOTkxAKVbD8LtSeFYQD/jWOA8Zo6m/vLA5GjmnyZ2zV+IQDOTpN0kq3rDlQh/3H+YmDTff94Jh/8B7V1bc6JIFP41vmxVpoBubo9qSCZVM5pVZ2dmX6YQWmUHwUWMcX/9NtDcuttbBExCUjU1cGyBPt+592nsgP7y+T4wV4uvvo3cjiTYzx1w25EkUdQU/F9E2SUUVQUJYR44dkIScsLY+Q+Rb6bUjWOjNaElpND33dBZlYmW73nICks0Mwj8bXnYzHftEmFlzhFDGFumy1K/O3a4SOel6PkHn5EzX5Bba5KafLA008FkJuuFafvbAgkYHdAPfD9MjpbPfeRGzEv5YmyHo3HX/v3jYbLa/KtIP7VH9ya52N05X8mmECAvrPbSUnLpJ9PdEH4JooyvI9zgf3cI2VPT+k2mH+5Sngb+xrNRdF2hA3rbhROi8cq0ok+3WIowbREuXXwm4kPTdeYePnbRDD987wkFoYPh6RJy6EfjT5wh4UR0DfRcwJfM+B75SxQGOzwk/ZQIKxFeSOa7LUiCoBB8FwUpUFVCNIn4zbNL5yzGB4TLZ3AcMBzv9KVOt4cn7oS76LiHuaG4Ea+mAT6ahzF/CCUl9P3lMuIVoeNHmeZjKbgKYKzDwP+N+r7rB/FnYKrJUBYKMFn4qgh/2Js5rlsYONMsZFmYPg9M28GDCp/d3QmCJtQIoywpn+QSkrKYUQpgSoCDpVwXlJCF8v2xntIgidUgnv5IdfFcYXiO54HVALOgG7M+cLw5qz9Woi6T3QplY/tFWoUWrj4lUMsqoFxfBVSGc8jG7pec+kG48Oe+Z7pGTu2VeVvgI3p2wh+EHB3/jI7xFEFkHIPdj/LpTzLyHxSGOxJ3mJvQx6T8vl/8yL0kV/fsbhRP4FPP91BCuXOi+d6er0BrfxNY6MA4jcQ6ZjBHh65H1Cti20EhCJBrhs5TOaqpHE/tuEnDGmZ68xjJI3rBWDwh/uMbsln8x7OGgR/imfsR7Uav09QpgIoWFJGjYKLKMXcV4PHgrr3BUvpH7W3dv7u+9Si50xuW/efqF1cXODrD08NE2VJNJMe3z8WTXXqyR0PF9CT/WnyWfy/Tyqnr41iTVkt8Rh5UVKrXUp3VUi4O4qvSUr2aGPKL8xu1JICUoF5SbVGFrGLroEHPmebHh0zta4xCaE5iwmmcBLVxUmQYV1MQcjiAODUQsc31Ir6zWL09E6WqDRX56qPvRNlmKgWqyiZkUvkiiU0l36Mgzh7kAtTZ0kViBJOZ+d564axONoXdMDQtjEq70mpNUMoBj6ayuqzJTVpFtjzyBxuBRhlfSWkDtHb+M6fxgIhhq0jq4meTex35NsIBK+SaxBE8WPhlqTW2tjifnMS6i/P9+qBQZUahNJ2X3jWZbItshaMdaMiqRKEhaeKVsZCrCfpGaIYwuywUA3SpnbOVqSIrp9i52UzaZ+dmM1tXzg/uLykfQpGX2jVbOxHZMN5ZP26mrmNlNaqp77vI9C6LDC/Lw2uOJ2kfJAOeD4IcTZNrwkViI/M1DmZGOKrw5utW2D+oM/ZP1RlYtCbtXxqjFlAhRd1aITm4atVgeKBJVHCgcuyX2igg+wJw5G2WKEhKdke8Emagd2yJK6nP8/xURuNeJnjHgbsIYUkcFJEbK4pNujOJs7JJsLCdpxICyr+baNG8F+npDWF05O1iZcs+TfG7NwbGqPvlV3/49asxmMS+EV8EO7BcAuI7lAXgspv++c0YTx6Gg+xuYp136w7G341Rdi+pznuNv93fU3MDdd6Pp+vGj0dj9GAM+kb2DPCIPjf1aMPJ5wISun6GmeE/6LuN3BgjpAqnGiFdqssIcdbkK3VK4ywS/PBJVIgC6JhRlNnScMM+iZM0V2I5BsPJr5GReKRb47YZnzQcGL/Gk25unWr1SZPvw/LdavVKk88jg5pd417pbvhtVH6E1+KU7h7+opgjf/il0w1RzK7r+iW2b0n49IlduHqHmasilrCA3MY92Gjmqu4rJXQ9a4HFtgWoUPUECLmtLkKjsGgM41u7jiuRalvt67iCQNlKkDc9NbSSK7F18ApWcg8F6e8v9tbKRhYAnpFtdikX8Bpc2NBm/5LHnvjm/cYtNIgaH8Qm10JAPb01pcbCU5pzX2ST93crVmytQeVNvHxrDcVyxQXoeqOWGuwr+V9kqVvUhgggk5rogKfk6UalZiz1/sr9h6XmwwjoqAnyYeTFz7XZali/raZatt+irZabsdWSTG0mAEqztnpfv9BFtnpitiWkFum6hSZdP6Rmq0gfhvowiOUyhwz4INYUUnO3koCLrfRLtvS8ze05aS/YG9ufA9gC48t6NdtUw4BquRHw+lt0ALsbMmoEXGd2lV0MzsEM0JODttnQdGPyJaa2IaNJA4EpPKPJhQLUBAVku/+CmPH1V+xfTVOmIpT3fwOdV7NvtCkzTVMLsLg4m28RKJAu2nGsVqNtmbCtO2hEsYyEpAsMEo3u2IBsL1JotqSBnO4Nkzg7y+pa8+UGiJfXRj420h+K1CHxRG8sUof7qiRnRuoP3swPlqbbnvqIXA5GuNF6WkJp5m1MbHHkWCEkBxQb5oG5zN8NtO89QnjcLVpbgbOKO0mPDWdwf43xvqyVTTVvQz/vHU+yXheSbPocRZXt8Jx03Rgr2rX7pSCbB7cjpAQCtViXGrRrhZQy21OBLdIc2YaLkvcWtAAVSL0HDaZ1mqt1rslsdaJWBXklTYRQY7dVc19J1ygWbEmiJZ22skI3FwD16r22MluOaAkcqiDScHALds3CUVOPgMQ2dGVprMrJY7Nc1ceCUUpV92TV3Oz7POiO5rAkgTj6ykYy7pWksDKbwrYjOlMFmVIwRZKu7HzYHNS0LLQKkd311lsUtOJdBWnTaIYKW4gVebtv6oOFTSjboSKyVg6V97wnoFEs2GTSjFUD2X9u0DruTGqBkkCFjp25atIsNHqlwcEb2GaTlrXqbgZUIIV1vj+joXZAha0cXNIOWHJqbal4y3R+pXCWGJvtCFTYysNHR+B5IKqQB2KTHYEcDF+yBjVcIe8rCk3bDM1R9Ks07Xi5H12kVVk4uTl2fQtR1brRA3um0hS7mF9Xu8d1/zp11AkXUll8TCvk8RV7a3hqgi5W3uN/mUAI9QjEoaKLxPxOxrsUiTSNvIJIxF/FszV3hQEkDdkbCEo63cOnFMWLGS9r9I4/hRLH5AkqjRVB4waMqhFWKa5XEEpw6m+/vBKhlNXykjMge4f2CaWSbjvlj69HKOU6ZFIsSGRmMysyk6lwix3GO19BJq/nOvkypOVN5WkyJUA6J05m9YKcGJ/mv8uYDM9/3RIY/wM=5Vptb+I4EP41fDmJKi/EgY9A6XZPW8EBt20/hsQkvg0x6zgF9tfvOHEgqd0WtoSTWiSEM36ZeB4/4xmblj1cbb8wbx3d0QDHLcsIti37umVZptlF8CMku0LiunYhCBkJZKODYEZ+YSk0pDQjAU5rDTmlMSfrutCnSYJ9XpN5jNFNvdmSxnWtay/EimDme7EqvScBj8p5od6h4haTMJKqu5ZbVKy8srGcSRp5Ad1URPaoZQ8ZpbworbZDHAvjlXYZbcbTWT/48fB1vs5+IuuxO4nbxWA3p3TZT4HhhJ93aKsY+smLM2kvw3QckLThC3PbBPA7oxnzSRJKM/BdaVtGsyTAYnyjZQ82EeF4tvZ8UbuB1QSyiK9ieDKh6MUkTKAc4yVMYvCEGScAU1+KORXtj5yptIgYA28rOMuZf8F0hTnbQZOyVi5auYg7ct6byoowkMQ5qqwG15VCTy7DcD/0wdRQkNY+wfK2YvnW0Gr1BzBxwneiPABroFjYasGgFPLcPlJSCgRGSVAiNKTQe5FxysqG8G6LQ+dn+FXQSTmjP/CQxtBV1NmLrtNxjApuPrwZhsrBksRxpeGy62PfB3nIvIBAo0rdzY1hdI0GcS0xk7gCfa8cBVrLclRknaaA7ajAfjy7P+OTpfJJxyarKZsjxcQ4gF1APlLGIxrSxItHB+mg7rwqkOAt4Q9SLMqPogzLyhbcZLuH+uOjbPkf5nwntz8PGAiig95vVHi3YvQk6IttDR4TmuBCckPEfK9PRywVxMevtJObGfdYiF8bT+IpzPYq/gzHHidP9c317Hi6b3OIM+IlYY7kGxuPQjEj/+iZs8w/OvoxymHmVMjavUa5hTo1cplOT92tXA27mkKje56taoqXGGzlY28BsL13fwrQAjnoGD+5XFov+cnlMugh1CCWrm1euU499ugYKpw6b9nYDmWaiqUbcpevu7pjXWbgpVGu2Ty/fyzTlPM5Ptl1Qgm84H4hOK66EJBTBivlOIWbll2fobx/l3cAb7zA5GJyNEkjsj4p9KxFnkyEnsJDfpLYEymQOq6jiz/Ni7JbzekYjfF8twanC/bol5ldCd2U5u7YW4l9M1mkax1cJ6V679txZYJ4QdiQdTRsdlObrPlyQpitMCtCjzeoma695Di6Ssw1NN3LtIOxD8xns9e5slQ+u/87oXUZZYFHQJ5qKKCfmTiNGsCMeVsaW/A9p9S+tsRwMh1PxrPRNHcL0BsYeoA/H7qO/vu0TUffv47uK9rMJrXN/p1MxtN5RZ3VpLr+BIz5vaLNblLbeH5bUdXrvaTrw/pwDVWRhfait6jabYyqjkLVvxQYBMZ1WzOckl95VlRAsxYBYP5uzqDlXAu7QnicFpFy9Sx170f1p6ngwoWzn+eRdLvTZJ5jqOGtq+Y51iUPhUz0ScHomK6adGqocVEw1CMdv5IypJ8CGORoTgNUYLoXBUY93dkDQ9nnwEVHGHQ5XKa33bu/27t+8nDHs+zu7mlsDdqaVP3j7ufPr3Hco69xzpGPae1vqvb/k0PPCWYpTV5Otz5cFoVQDUrTUA+vG7uPwzfYiAbtSfCPP3OMb2Z2H4Q6Iv0JkH3/89yqasJr3an1Ze9Ve4qhz3BoXZZ1d3yt6g3fWQ+082Gvt1Ul1zv5lHKP8VILhVVbyqSe8o0P3cVD2fu0FVE9HX+NO2/eHsoYovHbQ3g8/N+mOBU//GvJHv0G7Vxdc5s4FP01ftmZdJDElx/tOMlu222zTWfa7ksHg2xrQ5BXlmu7v74ChA1I8UeMSDZsZjKFiyyje3Suzr1S00OXD+sbFsxnf9IIxz1oReseGvUgBMB3xT+pZZNbPA/lhikjkWy0M9yRn1gaLWldkggvKg05pTEn86oxpEmCQ16xBYzRVbXZhMbVb50HU6wY7sIgVq1fSMRnxbjc/u7B75hMZ/KrfejlDx6CorEcyWIWRHRVMqGrHrpklPL86mF9iePUeYVfrlYfP90Novuvf3yeL/914Tf/Nr7IO7s+5SPbITCc8Ga7hnnXP4J4Kf1lAVf0Y12I3w+UC+TysfNN4VBGl0mE006tHhquZoTju3kQpk9XYgoJ24w/xOIOiMsgJtNEXMd4It58+AMzTgQ2A2nmNG1/5PCkG9I+8LoErhzuDaYPmLONaFI8lTNVzlxbDnZVmgaWK8GdlaaA50ljIOfedNv1zr/iQrr4BHcjxd29S9gbDMXACd+k10PhDTdOfTVm4mrKM/9IS2FIgXlPp4VdvMp417YGVwmMBWf0Hl/SmLLsGRr7ju1YJZhC8SJYPBxOSByXGk78EIehsE9ZEBHRqPTs+tqyfMsgjLbnvXEqSELPLiwlMCHSYOmYgtJWoXx9rq8xCKoM0vEHmvK5q/g8CR6wsPTQIHM9I8lU5Y9Yf0JG5pzQ5GDbRsOdOUb4FVwQPJINtmMIGU/xG47EMixvKeMzOqVJEF/trMOqZ0texGvCv0pzev0tvRaER2mcZJuv1dtvsuU/mPON1B/BklNh2n3ve5quNHnvSTRIdYW4TWiCc8s1Scc7Op1LC7pkId7TTiLFAzbF+/qTTEvdtncKMBwHnPyoqpvG8fQPRzdBoCCZZkgeYIUS/KzsRx/TJtmPLjAyyoOMw2h00TcZ9RzfroY9ADTLDfA0kc8UHgAo/m+CYG88p8yxI9hxLMuiYDHLvrr4vtG6xOjRxgzZAGyaRfKjt5SIF9zOEOS61QBs1Za8nO7yUzXst69xxnRQdXsuJPNx0WQxI/Oj5eSA8yAUaElZeZVkY++EtoQIVIC0oaeler+wtqItgZon/KbGX7zmVVozvCA/g3HWIHXaPJ1+2bs5w54zSrEQjF3k5AU6aPT52UJEdiGWPmfkFsLXHByeVZf6CPV1Ur9N1QlUqd8VNKoyE/jus2PhPBL4TsygP+EJFu4KcQbQubEucseu4x4T6yYT+Fism0yivuu2mkcDVyds2s2jQf+w1DwpBztPbRrO3MR6U8NAoKJbbzScMpW7FfKnnFbnSkAt/jUY5vaWBFsMcqC24BQz/bmKHFBTWXpqkbBLUq4A6QXVCKG6WgkI48MVq4xmr6JUVUPl+EqVbQoTV/Hb/6Wq00pVRYQynmU7tbI/qnWRj8hYlg3VKlheSh5rtsVe48oIa2pRJ/+9VtfGflvkNVAFa5itRdQ0zUIPKttvyKt2YrjaVYzUSLWrI+rIr+XysCB3Oe3wYYviCKlpB1ncLscxCbfSZ0xpjIPkPKXzohNCBRf3SFyMpYOoq8VH21HPGTx78RF1tfgI+9XKPOw/PxZqOhcbLpK8GDhspJBDC4jdKiDqgZA0zyJm5fmLwQT26yUPHSKOIUT+Wt+O4i8f+N2Nfz2H3985H7+/vbAUx5+jzxtS4dtUfHtTS9N3e9PZXbE5vRC6lhdvQMUkK2zyHQyIejmhyym41s+gJfHv9J36hlxxcPZAEi78FmxKzSS99nxT9ayZK+uyu9mY9/jUzELvxvOPVWhnnGZmao9fKKcvHj0oUeLCOKbhvUKGXOUWR5wlfeQrAP+smap33bPN1LMClJp0nHMCd7AU4LOOpJDIrioyt6/ulEAETZ2U0MLZ1VzF8ZS4XOwZtqCOtVB0NVFxPFiDwtERo00o1DwlyALVtujVDX3sWocpAkCbwKj5Sg5MN/BQxaSOKaY227WAtHZu/UXtJ2hd4Tet047NKBzgVjtpbjth30Cb2U2oCMHsk90Qg55dh9EuCnblVU8XXY2JwU6dJFMBcOwjAahvozeXXD22UXdidnWL2YImn2gDRzP/G2xCbr26qlsZgd/m4SWkssfsQRltdS6q1d926+YjdRbtuvn0pXLf/9V6hoMyZyHqmAAUVACFTaqdfTPjZdRm955RN34wQ60BONszj80fkUrY/N37uwm5GL6Nbu4Rs67/Xl1oAv7rXXLrBTDP1hbA1Bj9hL16cbv7yxI5Wru/z4GufgE= \ No newline at end of file + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/types/2/0210-Data-Stores.md b/site/docs/types/2/0210-Data-Stores.md index 3fdc9b089e..d3dd511891 100644 --- a/site/docs/types/2/0210-Data-Stores.md +++ b/site/docs/types/2/0210-Data-Stores.md @@ -9,7 +9,7 @@ Both [*DataSets*](/types/0/0010-Base-Model#dataset) and [*DataStores*](#datastor ## DataStore entity -The *DataStore* entity describes a physical [digital resource](/concepts/resource) that supplies data. +The *DataStore* entity describes a physical [digital resource](/concepts/resource) that supplies data. The *deployedImplementationType* attribute describes the technology that is used in its implementation. ## DataContentForDataSet relationship @@ -19,8 +19,5 @@ The *DataContentForDataSet* relationship defines how data is supplied to a [Data The *DataStoreEncoding* classification provides the ability to store details of the data stores physical characteristics. -## DataFieldValues classification - -The *DataFieldValues* describe the characteristics of a collection fo data values. It is typically attached to an [*Asset*](/types/0/0010-Base-Model), a [*GlossaryTerm*](/types/3/03303-Terms) entity or a [*SchemaAttribute*](/types/5/0505-Schema-Attributes) entity. --8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/types/2/0210-Data-Stores.svg b/site/docs/types/2/0210-Data-Stores.svg index fbc627c63b..a4d1ebd95e 100644 --- a/site/docs/types/2/0210-Data-Stores.svg +++ b/site/docs/types/2/0210-Data-Stores.svg @@ -1,4 +1,4 @@ - + -
0210 - Data Stores
0210 - Data Stores
«entity»
DataStore
«entity»...
«entity»
Asset
«entity»...
«entity»
DataSet
«entity»...
*
*
dataContent
dataContent
supportedDataSets
supportedDataSets
*
*
queryId : string
query : string
queryId : string...
«relationship»
DataContentForDataSet
«relationship»...
encoding : string
language : string
description : string
properties : map<string, string>
encoding : string...
«classification»
DataStoreEncoding
«classification»...
storeCreateTime : date
storeUpdateTime : date
pathName : string
storeCreateTime : date...
«entity»
Referenceable
«entity»...
defaultValue : string
sampleValues : array<string>
dataPattern : array<string>
namePattern : array<string>
defaultValue : string...
«classification»
DataFieldValues
«classification»...Text is not SVG - cannot display \ No newline at end of file +
0210 - Data Stores
0210 - Data Stores
«entity»
DataStore
«entity»...
«entity»
Asset
«entity»...
«entity»
DataSet
«entity»...
*
*
dataContent
dataContent
supportedDataSets
supportedDataSets
*
*
queryId : string
query : string
queryId : string...
«relationship»
DataContentForDataSet
«relationship»...
encoding : string
language : string
description : string
properties : map<string, string>
encoding : string...
«classification»
DataStoreEncoding
«classification»...
storeCreateTime : date
storeUpdateTime : date
pathName : string
deployedImplementationType : string
storeCreateTime : date...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/0212-Deployed-APIs.md b/site/docs/types/2/0212-Deployed-APIs.md index d202a3c7db..53ceeb90d5 100644 --- a/site/docs/types/2/0212-Deployed-APIs.md +++ b/site/docs/types/2/0212-Deployed-APIs.md @@ -11,7 +11,7 @@ APIs exchange data structures and commands. Each API is an access point for data ## DeployedAPI -The *DeployedAPI* is the entity that represents the API as an [Asset](/types/0/0010-Base-Model). The definition of the API's operations, their parameters and responses are described via the [APISchemaType](/types/5/0536-API-Schemas) entity, which is connected to the *DeployedAPI* entity with the [AssetSchemaType](/types/5/0503-Asset-Schema) relationship. +The *DeployedAPI* is the entity that represents the API as an [Asset](/types/0/0010-Base-Model). The *deployedImplementationType* attribute describes the technology that is used in its implementation. The definition of the API's operations, their parameters and responses are described via the [APISchemaType](/types/5/0536-API-Schemas) entity, which is connected to the *DeployedAPI* entity with the [AssetSchemaType](/types/5/0503-Asset-Schema) relationship. The classifications [RequestResponseInterface](#requestresponseinterface), [ListenerInterface](#listenerinterface) and [PublisherInterface](#publisherinterface) can be used to describe the style of the API. They can appear in combination on a single *DeployedAPI*. diff --git a/site/docs/types/2/0212-Deployed-APIs.svg b/site/docs/types/2/0212-Deployed-APIs.svg index 49c0e03d3c..cf4671dafb 100644 --- a/site/docs/types/2/0212-Deployed-APIs.svg +++ b/site/docs/types/2/0212-Deployed-APIs.svg @@ -1,4 +1,4 @@ - + -
0212 - Deployed APIs
0212 - Deployed APIs
«entity»
DeployedAPI
«entity»...
«entity»
Asset
«entity»...
«entity»
Endpoint
«entity»...
*
*
supportedAPIs
supportedAPIs
accessEndpoints
accessEndpoints
*
*
«relationship»
APIEndpoint
«relationship»...
«classification»
RequestResponseInterface
«classification»...
«classification»
ListenerInterface
«classification»...
«classification»
PublisherInterface
«classification»...Text is not SVG - cannot display \ No newline at end of file +
0212 - Deployed APIs
0212 - Deployed APIs
«entity»
DeployedAPI
«entity»...
«entity»
Asset
«entity»...
«entity»
Endpoint
«entity»...
*
*
supportedAPIs
supportedAPIs
accessEndpoints
accessEndpoints
*
*
«relationship»
APIEndpoint
«relationship»...
«classification»
RequestResponseInterface
«classification»...
deployedImplementationType : string
deployedImplementationType : string
«classification»
ListenerInterface
«classification»...
«classification»
PublisherInterface
«classification»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/0215-Software-Components.md b/site/docs/types/2/0215-Software-Components.md index 74be1bcf05..293b0de08c 100644 --- a/site/docs/types/2/0215-Software-Components.md +++ b/site/docs/types/2/0215-Software-Components.md @@ -1,25 +1,35 @@ ---- -hide: -- toc ---- - # 0215 Software Components -`DeployedSoftwareComponent` describes a code [asset](/0/0010-Base-Model) that is deployed to implement a [software capability](/types/0/0042-Software-Capabilities). Each software component has a well defined interface describe by an [APISchema](/types/5/0536-API-Schemas) entity that is linked to the `DeployedSoftwareComponent` by the [AssetSchemaType](/types/5/0503-Asset-Schema) relationship. +Many processes running in an organization's IT landscape are implemented as pluggable software components. Some of these are simple, others are more complex. The types shown on this page describe such components, their internal structure and running instances (if the process is short-lived and each instance has different [lineage](/features/lineage-management/overview)). -`DeployedConnector` represents specialist software component called a [connector](/concepts/connector) that provides pluggable access to third party technologies. These connectors implement the [Open Connector Framework (OCF)](/frameworks/ocf/overview) interfaces. The `DeployedConnector` entity is typically linked to a [`Connection`](/types/2/0201-Connectors-and-Connections) entity via a [`ConnectionToAsset`](/types/2/0205-Connection-Linkage) relationship. +![UML](0215-Software-Components.svg) -`EmbeddedProcess` describes a processing element nested within a `DeployedSoftwareComponent`. -The `TransientEmbeddedProcess` describes an `EmbeddedProcess` that runs only for a short period of time. -These variations are used to provide more information for [lineage graphs](/features/lineage-management/overview). +## DeployedSoftwareComponent entity -`ProcessHierarchy` defines a parent-child relationship between processes, which can be used to define more abstract processes that are comprised of lower-level processes; helping to support navigating the process hierarchy. +The *DeployedSoftwareComponent* entity describes a code [asset](/0/0010-Base-Model) that is deployed to implement a [software capability](/types/0/0042-Software-Capabilities). Each software component has a well defined interface describe by an [APISchema](/types/5/0536-API-Schemas) entity that is linked to the *DeployedSoftwareComponent* by the [AssetSchemaType](/types/5/0503-Asset-Schema) relationship. Its optional properties are: -![UML](0215-Software-Components.svg) +* *deployedImplementationType* describes the technology used to implement the component (for example "Apache Spark"). +* *implementationLanguage* describes the language used to implement the component. + +## DeployedConnector entity + +The *DeployedConnector* entity represents specialist software component called a [connector](/concepts/connector) that provides pluggable access to third party technologies. These connectors implement the [Open Connector Framework (OCF)](/frameworks/ocf/overview) interfaces. The *DeployedConnector* entity is typically linked to a [Connection](/types/2/0201-Connectors-and-Connections) entity via a [ConnectionToAsset](/types/2/0205-Connection-Linkage) relationship. + +## EmbeddedProcess entity + +The *EmbeddedProcess* entity describes a processing element nested within a *DeployedSoftwareComponent* entity. Because it is transient it is possible to add the *processStartTime* and *processEndTime* properties to this entity. + +## TransientEmbeddedProcess + +The *TransientEmbeddedProcess* entity describes an *EmbeddedProcess* entity that runs only for a short period of time. + +## ProcessHierarchy relationship + +The *ProcessHierarchy* relationship defines a parent-child relationship between processes, which can be used to define more abstract processes that are comprised of lower-level processes; helping to support navigating the process hierarchy. Typically the top if the process hierarchy inherits from *DeployedSoftwareComponent* and the nested processes inherit from *EmbeddedProcess*. ??? education "Further information" diff --git a/site/docs/types/2/0215-Software-Components.svg b/site/docs/types/2/0215-Software-Components.svg index 9393284c85..60b1540dee 100644 --- a/site/docs/types/2/0215-Software-Components.svg +++ b/site/docs/types/2/0215-Software-Components.svg @@ -1,4 +1,4 @@ - + -
0215 - Software Components
0215 - Software Components
«entity»
Process
«entity»...
«entity»
DeployedSoftwareComponent
«entity»...
implementationLanguage : string
implementationLanguage : string
«entity»
DeployedConnector
«entity»...
«entity»
EmbeddedProcess
«entity»...
«entity»
TransientEmbeddedProcess
«entity»...
processStartTime : date
processEndTime : date
processStartTime : date...
«entity»
Process
«entity»...
«relationship»
ProcessHierarchy
«relationship»...
containmentType : ProcessContainmentType
containmentType : ProcessContainmentType
parentProcess
*
parentProcess...
childProcess
*
childProcess...
«enumeration»
ProcessContainmentType
«enumeration»...
OWNED = 0
USED = 1
OTHER = 99
OWNED = 0...Text is not SVG - cannot display \ No newline at end of file +
0215 - Software Components
0215 - Software Components
«entity»
Process
«entity»...
«entity»
DeployedSoftwareComponent
«entity»...
deployedImplementationType : string
implementationLanguage : string
deployedImplementationType : string...
«entity»
DeployedConnector
«entity»...
«entity»
EmbeddedProcess
«entity»...
«entity»
TransientEmbeddedProcess
«entity»...
processStartTime : date
processEndTime : date
processStartTime : date...
«entity»
Process
«entity»...
«relationship»
ProcessHierarchy
«relationship»...
containmentType : ProcessContainmentType
containmentType : ProcessContainmentType
parentProcess
*
parentProcess...
childProcess
*
childProcess...
«enumeration»
ProcessContainmentType
«enumeration»...
OWNED = 0
USED = 1
OTHER = 99
OWNED = 0...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/0222-Graph-Stores.svg b/site/docs/types/2/0222-Graph-Stores.svg index fb71ec20dc..336969b61a 100644 --- a/site/docs/types/2/0222-Graph-Stores.svg +++ b/site/docs/types/2/0222-Graph-Stores.svg @@ -1,3 +1,4 @@ + -
0222 - Graph Stores
0222 - Graph Stores
«entity»
DataStore
«entity»...
«entity»
GraphStore
«entity»...
deployedImplementationType : string
deployedImplementationType : stringViewer does not support full SVG 1.1 \ No newline at end of file +
0222 - Graph Stores
0222 - Graph Stores
«entity»
DataStore
«entity»...
«entity»
GraphStore
«entity»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/0223-Events-and-Logs.md b/site/docs/types/2/0223-Events-and-Logs.md index 60c0e17571..763cd16993 100644 --- a/site/docs/types/2/0223-Events-and-Logs.md +++ b/site/docs/types/2/0223-Events-and-Logs.md @@ -7,27 +7,27 @@ Processing is often triggered when a specific event or situation occurs. There ![UML](0223-Events-and-Logs.svg) -## DataFeed +## DataFeed entity -*DataFeed* identifies an asset that provides a continuous feed of data. +*DataFeed* identifies an asset that provides a continuous feed of data. The *deployedImplementationType* attribute describes the technology that is used in its implementation. -## Topic +## Topic entity A *Topic* is a type of [*DataSet*](/types/0/0010-Base-Model/#dataset) that is accessed using a publish-subscribe event interface. There are processes that add events to the topic, and other processes that subscribe to the topic and receive a copy of each event that is added. It is shown as a data set because some topics are in memory and some make use of an underlying [*DataStore*](/types/2/0210-Data-Stores) to provide a persistent/reliable delivery service. -## KafkaTopic +## KafkaTopic entity [Apache Kafka](https://kafka.apache.org/) is a popular open source technology that provides support for *Topics*. There is a specific type for an Apache Kafka topic because there is a [resource connector](/contectors/resource/kafka-open-metadata-topic-connector) for Apache Kafka and this type identifies when it is appropriate to use this connector to access the topic contents. -## SubscriberList and TopicSubscribers +## SubscriberList entity and TopicSubscribers relationship Some services manage a specific subscriber list. This is represented by the *SubscriberList* entity type. The links to the topics that the subscription refers to is represented by the *TopicSubscribers* relationship type. -## LogFile +## LogFile entity -A *LogFile* entity is a subtype of [*DataFile*](/types/2/0220-Files-and-Folders) that contains events that have been captured and stored. +A *LogFile* entity is a subtype of [*DataFile*](/types/2/0220-Files-and-Folders) that contains events that have been captured and stored. The *purpose* attribute describes the use of log file - for example, it is a debug log, audit log, .... -## AssociatedLog +## AssociatedLog relationship The *AssociatedLog* relationship shows the [Asset](/types/0/0010-Base-Model) (typically a *LogFile*) where logging information for the linked [Referenceable](/types/0/0010-Base-Model) is located. diff --git a/site/docs/types/2/0223-Events-and-Logs.svg b/site/docs/types/2/0223-Events-and-Logs.svg index 7aa6ecac6a..a1c9d8f72c 100644 --- a/site/docs/types/2/0223-Events-and-Logs.svg +++ b/site/docs/types/2/0223-Events-and-Logs.svg @@ -1,4 +1,4 @@ - + -
0223 - Events and Logs
0223 - Events and Logs
«entity»
DataFile
«entity»...
«entity»
LogFile
«entity»...
deployedImplementationType : string
deployedImplementationType : string
«entity»
Topic
«entity»...
topicType : string
topicType : string
«entity»
DataSet
«entity»...
«entity»
SubscriberList
«entity»...
«relationship»
TopicSubscribers
«relationship»...
subscribers
subscribers
*
*
topics
topics
*
*
«entity»
KafkaTopic
«entity»...
partitions : int
replicas : int
partitions : int...
«entity»
Asset
«entity»...
*
*
logSubjects
logSubjects
*
*
associatedLogs
associatedLogs
«relationship»
AssociatedLog
«relationship»...
«entity»
Referenceable
«entity»...
«entity»
DataFeed
«entity»...
«entity»
Asset
«entity»...Text is not SVG - cannot display \ No newline at end of file +
0223 - Events and Logs
0223 - Events and Logs
«entity»
DataFile
«entity»...
«entity»
LogFile
«entity»...
purpose : string
purpose : string
«entity»
Topic
«entity»...
topicType : string
topicType : string
«entity»
DataSet
«entity»...
«entity»
SubscriberList
«entity»...
«relationship»
TopicSubscribers
«relationship»...
subscribers
subscribers
*
*
topics
topics
*
*
«entity»
KafkaTopic
«entity»...
partitions : int
replicas : int
partitions : int...
«entity»
Asset
«entity»...
*
*
logSubjects
logSubjects
*
*
associatedLogs
associatedLogs
«relationship»
AssociatedLog
«relationship»...
«entity»
Referenceable
«entity»...
«entity»
DataFeed
«entity»...
«entity»
Asset
«entity»...
deployedImplementationType : string
deployedImplementationType : stringText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/0224-Databases.md b/site/docs/types/2/0224-Databases.md index 5d00b9a345..7432968b0d 100644 --- a/site/docs/types/2/0224-Databases.md +++ b/site/docs/types/2/0224-Databases.md @@ -8,16 +8,24 @@ A database is a structured store of data. It is typically managed as a shared d ![UML](0224-Databases.svg) -## Database +## Database entity The *Database* entity represents the database store. It inherits from [*DataStore*](/types/2/0210-Data-Stores) to indicate that the physical data is stored under this asset. -## DeployedDatabaseSchema +## RelationalDatabase entity + +The *RelationalDatabase* entity defines a database that supports the relational schema that can be accessed via the Java Database Connectivity (JDBC) interface. + +## DeployedDatabaseSchema entity Often the data within the database is organized into distinct collections of data for use by the different applications or processes. The structure of the data in each collection is called a database schema. Such database schemas are used to limit the scope of data that a consuming application/process is exposed to. Where this is the case, each database schema is represented by the *DeployedDatabaseSchema* entity. *DeployedDatabaseSchema* inherits from [*DataSet*](/types/0/0010-Base-Model) to indicate that it is a logical collection of data. The physical data is stored in the database. *DeployedDatabaseSchema* is linked to its *Database* using the [*DataContentForDataSet*](/types/2/0210-Data-Stores) relationship. +## TableDatabase entity + +The *TableDataSet* can be used to identify an important table in a database that can be considered an asset in its own right. + !!! info "Defining the structure of data within a database" The structure of the data stored in a database is represented by a subclass of [*RootSchemaType*](0530-Tabular-Schemas). For example, if the database has a relational database structure (ie tables and columns) the root schema type used is [*RelationalDBSchemaType*](/types/5/0534-Relational-Schemas). diff --git a/site/docs/types/2/0224-Databases.svg b/site/docs/types/2/0224-Databases.svg index beed9250a7..33863c83c7 100644 --- a/site/docs/types/2/0224-Databases.svg +++ b/site/docs/types/2/0224-Databases.svg @@ -1,4 +1,4 @@ - + -
0224 - Databases
0224 - Databases
«entity»
DeployedDatabaseSchema
«entity»...
«entity»
DataSet
«entity»...
deployedImplementationType : string
databaseVersion : string
instance : string
importedFrom : string
deployedImplementationType : strin...
«entity»
Database
«entity»...
«entity»
DataStore
«entity»...
«entity»
TableDataSet
«entity»...Text is not SVG - cannot display \ No newline at end of file +
0224 - Databases
0224 - Databases
«entity»
DeployedDatabaseSchema
«entity»...
«entity»
DataSet
«entity»...
databaseVersion : string
instance : string
importedFrom : string
databaseVersion : string...
«entity»
Database
«entity»...
«entity»
DataStore
«entity»...
«entity»
TableDataSet
«entity»...
«entity»
RelationalDatabase
«entity»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/0225-Metadata-Repositories.svg b/site/docs/types/2/0225-Metadata-Repositories.svg index c340bdc002..dd31f1f5ca 100644 --- a/site/docs/types/2/0225-Metadata-Repositories.svg +++ b/site/docs/types/2/0225-Metadata-Repositories.svg @@ -1,4 +1,4 @@ - + -
0225 - Metadata Repositories
0225 - Metadata Repositories
«entity»
CohortMember
«entity»...
«relationship»
CohortMemberMetadataCollection
«relationship»...
cohortMember
cohortMember
0..1
0..1
localMetadataCollection
localMetadataCollection
0..1
0..1
deployedImplementationType : string
deployedImplementationType : string
«entity»
MetadataRepository
«entity»...
«entity»
DataStore
«entity»...
«entity»
CohortRegistryStore
«entity»...
«entity»
DataSet
«entity»...
«entity»
MetadataCollection
«entity»...
managedMetadataCollectionId : string
managedMetadataCollectionId : stringText is not SVG - cannot display \ No newline at end of file +
0225 - Metadata Repositories
0225 - Metadata Repositories
«entity»
CohortMember
«entity»...
«relationship»
CohortMemberMetadataCollection
«relationship»...
cohortMember
cohortMember
0..1
0..1
localMetadataCollection
localMetadataCollection
0..1
0..1
«entity»
MetadataRepository
«entity»...
«entity»
DataStore
«entity»...
«entity»
CohortRegistryStore
«entity»...
«entity»
DataSet
«entity»...
«entity»
MetadataCollection
«entity»...
managedMetadataCollectionId : string
managedMetadataCollectionId : stringText is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/area-2-assets.drawio b/site/docs/types/2/area-2-assets.drawio index 77e4bb02ea..300b1dfcc9 100644 --- a/site/docs/types/2/area-2-assets.drawio +++ b/site/docs/types/2/area-2-assets.drawio @@ -1,6 +1,6 @@ - + - + @@ -538,7 +538,7 @@ - + @@ -555,77 +555,58 @@ - + - + - + - - + + - + - + - + - + - - + + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - + + - + @@ -691,8 +672,8 @@ - - + + @@ -720,69 +701,69 @@ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + @@ -806,13 +787,13 @@ - + - - + + @@ -913,7 +894,7 @@ - + @@ -1035,7 +1016,7 @@ - + @@ -1156,7 +1137,7 @@ - + @@ -1200,23 +1181,23 @@ - + - + - + - + - + @@ -1330,7 +1311,7 @@ - + @@ -1346,7 +1327,7 @@ - + @@ -1359,7 +1340,7 @@ - + @@ -1378,7 +1359,7 @@ - + @@ -1486,11 +1467,14 @@ + + + - + @@ -1500,7 +1484,7 @@ - + @@ -1512,10 +1496,10 @@ - - + + - + @@ -1524,20 +1508,31 @@ - + - + - + + + + + + + + + + + + - + @@ -1577,8 +1572,8 @@ - - + + @@ -1590,7 +1585,7 @@ - + diff --git a/site/docs/types/3/0340-Dictionary.md b/site/docs/types/3/0340-Dictionary.md index d33cac859c..80cd0acef2 100644 --- a/site/docs/types/3/0340-Dictionary.md +++ b/site/docs/types/3/0340-Dictionary.md @@ -24,13 +24,27 @@ ActivityDescription has a single property called `type` which is an enumeration * PROJECT - describes a type of project. * OTHER - describes some other type of activity -## AbstractConcept +## AbstractConcept classification *AbstractConcept* is a classification that means that the term describes an abstract concept. -## DataValue +## DataValue classification *DataValue* is a classification that means that the glossary term describes a valid value for a data item. +## DataFieldValues classification + +The *DataFieldValues* describe the characteristics of a collection of data values located in a particular data field (such as a [RelationalColumn](/types/5/0534-Relational-Schemas)). It is typically attached to a [*GlossaryTerm*](/types/3/03303-Terms) entity or a [*SchemaAttribute*](/types/5/0505-Schema-Attributes) entity. For example: + +* It can be attached to a glossary term to indicate that the glossary term describes a data field and the characteristics of this type of data. This can be used to guide a steward, or an automated process (such as an [open discovery service](/concepts/open-discovery-service)) to create a candidate mappings from a [SchemaAttribute](/types/5/0505-Schema-Attributes) or [DataField](/types/6/0615-Schema-Extraction) to the glossary term. +* It can be attached to a schema attribute to describe the properties of the data stored in the location that the schema attribute represents. + +Its attributes are: + +* *defaultValue* - Value that is used when an instance of the data field is created. +* *sampleValues* - List of sample values for the data field. +* *dataPattern* - A regular expression that characterizes the values in the data field. +* *namePattern* - A regular expression that characterizes the name of the data field. + --8<-- "snippets/abbr.md" \ No newline at end of file diff --git a/site/docs/types/3/0340-Dictionary.svg b/site/docs/types/3/0340-Dictionary.svg index bb471e92d6..4b6ade4293 100644 --- a/site/docs/types/3/0340-Dictionary.svg +++ b/site/docs/types/3/0340-Dictionary.svg @@ -1,3 +1,4 @@ + -
0340 - Dictionary
0340 - Dictionary
type : ActivityType
type : ActivityType
«classification»
ActivityDescription
«classification»...
«entity»
GlossaryTerm
«entity»...
OPERATION = 0
ACTION = 1
TASK = 2
PROCESS = 3
PROJECT = 4
OTHER = 99
OPERATION = 0...
«enumeration»
ActivityType
«enumeration»...
«classification»
DataValue
«classification»...
«classification»
AbstractConcept
«classification»...Viewer does not support full SVG 1.1 \ No newline at end of file +
0340 - Dictionary
0340 - Dictionary
type : ActivityType
type : ActivityType
«classification»
ActivityDescription
«classification»...
«entity»
GlossaryTerm
«entity»...
OPERATION = 0
ACTION = 1
TASK = 2
PROCESS = 3
PROJECT = 4
OTHER = 99
OPERATION = 0...
«enumeration»
ActivityType
«enumeration»...
«classification»
DataValue
«classification»...
«classification»
AbstractConcept
«classification»...
«entity»
Referenceable
«entity»...
defaultValue : string
sampleValues : array<string>
dataPattern : array<string>
namePattern : array<string>
defaultValue : string...
«classification»
DataFieldValues
«classification»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/3/area-3-glossary.drawio b/site/docs/types/3/area-3-glossary.drawio index 67106147ee..a9af95a315 100644 --- a/site/docs/types/3/area-3-glossary.drawio +++ b/site/docs/types/3/area-3-glossary.drawio @@ -1,4 +1,4 @@ - + @@ -474,7 +474,7 @@ - + @@ -493,7 +493,7 @@ - + @@ -506,19 +506,39 @@ - + - + - + - + - + + + + + + + + + + + + + + + + + + + + + diff --git a/site/docs/types/5/0534-Relational-Schemas.md b/site/docs/types/5/0534-Relational-Schemas.md index bf5e81ed2a..454b98926c 100644 --- a/site/docs/types/5/0534-Relational-Schemas.md +++ b/site/docs/types/5/0534-Relational-Schemas.md @@ -60,6 +60,12 @@ It indicates that the values stored in the first relational column are values fr Foreign keys are used to represent relationships between relational tables. +## Example + +The diagram below shows the typical structure of a relational database. There are two types of asset: the [Database](/types/2/0224-Databases) and optional nested [DeployedDatabaseSchema](/types/2/0224-Databases) entities. Linked to the asset via the [AssetSchema](/types/5/0503-Asset-Schema) relationship is a *RelationalDBSchemaType* entity that is in turn linked to the *RelationalTable* entities and the *RelationalColumn* entities beneath them. This example shows the use of the [TypeEmbeddedAttribute](/types/5/0505-Schema-Attributes) classification used to hold the schema type. + +![Database example](database-example.svg) + ??? deprecated "Deprecated types" - The supertype of *RelationalTableType* has be changed to *ComplexSchemaType* rather than *TabularColumnType* since [TabularColumnType](/types/5/0530-Tabular-Schemas) is now deprecated. - *DerivedRelationalColumn* has been replaced by [CalculatedValue](/types/5/0512-Derived-Schema-Elements)and [DerivedSchemaTypeQueryTarget](/types/5/0512-Derived-Schema-Elements). diff --git a/site/docs/user-interfaces/general/egeria-ui-about-icon.png b/site/docs/user-interfaces/general/egeria-ui-about-icon.png new file mode 100644 index 0000000000..bb46d69cf2 Binary files /dev/null and b/site/docs/user-interfaces/general/egeria-ui-about-icon.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-about.png b/site/docs/user-interfaces/general/egeria-ui-about.png new file mode 100644 index 0000000000..f1900df431 Binary files /dev/null and b/site/docs/user-interfaces/general/egeria-ui-about.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-category.png b/site/docs/user-interfaces/general/egeria-ui-category.png deleted file mode 100644 index b381e1c35c..0000000000 Binary files a/site/docs/user-interfaces/general/egeria-ui-category.png and /dev/null differ diff --git a/site/docs/user-interfaces/general/egeria-ui-glossary-categories.png b/site/docs/user-interfaces/general/egeria-ui-glossary-categories.png new file mode 100644 index 0000000000..8c2c093d7e Binary files /dev/null and b/site/docs/user-interfaces/general/egeria-ui-glossary-categories.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-glossary-terms.png b/site/docs/user-interfaces/general/egeria-ui-glossary-terms.png new file mode 100644 index 0000000000..4e6dfad1e0 Binary files /dev/null and b/site/docs/user-interfaces/general/egeria-ui-glossary-terms.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-glossary.png b/site/docs/user-interfaces/general/egeria-ui-glossary.png index 0b302f11d2..3587b45494 100644 Binary files a/site/docs/user-interfaces/general/egeria-ui-glossary.png and b/site/docs/user-interfaces/general/egeria-ui-glossary.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-profile-icon.png b/site/docs/user-interfaces/general/egeria-ui-profile-icon.png new file mode 100644 index 0000000000..588bae15c4 Binary files /dev/null and b/site/docs/user-interfaces/general/egeria-ui-profile-icon.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-profile.png b/site/docs/user-interfaces/general/egeria-ui-profile.png new file mode 100644 index 0000000000..b86b02bc4d Binary files /dev/null and b/site/docs/user-interfaces/general/egeria-ui-profile.png differ diff --git a/site/docs/user-interfaces/general/egeria-ui-terms.png b/site/docs/user-interfaces/general/egeria-ui-terms.png deleted file mode 100644 index 9413165bf0..0000000000 Binary files a/site/docs/user-interfaces/general/egeria-ui-terms.png and /dev/null differ diff --git a/site/docs/user-interfaces/general/overview.md b/site/docs/user-interfaces/general/overview.md index 04c8308df3..1f4fa019a0 100644 --- a/site/docs/user-interfaces/general/overview.md +++ b/site/docs/user-interfaces/general/overview.md @@ -59,14 +59,39 @@ On first load, the glossary is only able to show a list of available glossaries ![glossary](egeria-ui-glossary.png) -When clicking the `Details` button in third column of the glossary table, the categories connected to this respective glossary will be loaded. -The properties of these categories are displayed in the same way as the glossaries; their name and status are visible and another `Details` button is available. +When clicking anywhere on the row of the glossary, the categories connected to this respective glossary will be loaded in the next table. +The properties of these categories are displayed in the same way as the glossaries; their name and status are visible. -![category](egeria-ui-category.png) +![category](egeria-ui-glossary-categories.png) -Clicking this `Details` button will show the terms associated with its respective category. Each term in this case describes a concept or phrase used in the data model, and its `Details` button will redirect to the same `asset details` page as from the asset catalog. +Clicking a cateogry row in this table will show the terms associated with its respective category. Each term in this case describes a concept or phrase used in the data model, and clicking on one of these terms rows will redirect to the same `asset details` page as from the asset catalog. -![terms](egeria-ui-terms.png) +![terms](egeria-ui-glossary-terms.png) + +## Profile + +The profile page is accessible by clicking its icon on the bottom of the left navigation bar. + +![profile](egeria-ui-profile-icon.png) + +On the profile page it's possible to see information about the *scopes* your user account has and the *components* of the ui that are visible to your account. Above all that is the name of your account. + +![profile](egeria-ui-profile.png) + +## About + +Right below the profile page icon, in the left navigation bar, is the button that redirects to the about page. + +![about](egeria-ui-about-icon.png) + +The about page shows some useful information about the Egeria UI: + +- **Name**: The name of the Egeria UI package. +- **Version**: The version of the Egeria UI. +- **CommitId**: The commit id. +- **BuildTime**: The date and time that this version of the Egeria UI has been built. + +![about](egeria-ui-about.png) ## Extended features diff --git a/site/mkdocs.yml b/site/mkdocs.yml index b783acbd25..9f0b06c7da 100644 --- a/site/mkdocs.yml +++ b/site/mkdocs.yml @@ -40,66 +40,61 @@ nav: - Catalog Integration: guides/integration/types-of-metadata-repository.md - Connector Catalog: - Connector Overview: connectors/index.md + - Audit Log Destination: + - Console: connectors/runtime/console-audit-log-destination-connector.md + - Event Topic: connectors/runtime/event-topic-audit-log-destination-connector.md + - Cohort Registry Store: + - JSON File: connectors/runtime/file-cohort-registry-store-connector.md + - Digital Resource: + - Open Metadata Topic: + - Kafka Open Metadata Topic Connector: connectors/resource/kafka-open-metadata-topic-connector.md + - In Memory Open Metadata Topic Connector: connectors/resource/in-mem-open-metadata-topic-connector.md + - JDBC Connector: connectors/resource/jdbc-resource-connector.md + - Avro File Connector: connectors/resource/avro-file-resource-connector.md + - Basic File Connector: connectors/resource/basic-file-resource-connector.md + - CSV File Connector: connectors/resource/csv-file-resource-connector.md + - Data Folder Connector: connectors/resource/data-folder-resource-connector.md - Governance Actions: - Generic Element Watchdog: connectors/governance-action/generic-element-watchdog-governance-action-service.md - Generic Folder Watchdog: connectors/governance-action/generic-folder-watchdog-governance-action-service.md - Move/Copy File Provisioning: connectors/governance-action/move-copy-file-provisioning-governance-action-service.md - Origin Seeker Remediation: connectors/governance-action/origin-seeker-remediation-governance-action-service.md - - Governance Daemon: - - Open Lineage Janus Connector: connectors/governance-daemon/open-lineage-janus-connector.md - - Integration: - - File Connectors: - - Data Files Monitor: connectors/integration/data-files-monitor-integration-connector.md - - Data Folder Monitor: connectors/integration/data-folder-monitor-integration-connector.md - - Topic Connectors: - - Kafka Topics Monitor Connector: connectors/integration/kafka-monitor-integration-connector.md - - Kafka Topics Audit Connector: connectors/integration/kafka-topics-audit-integration-connector.md - - Strimzi Topics Monitor Connector: connectors/integration/strimzi-monitor-integration-connector.md - - API Connectors: - - Open API Monitor Connector: connectors/integration/open-api-monitor-integration-connector.md - - Infrastructure Connectors: - - Egeria Infrastructure Cataloguer Connector: connectors/integration/egeria-infrastructure-cataloguer-integration-connector.md - - Lineage Connectors: + - Lineage Warehouse (JanusGraph): connectors/governance-daemon/open-lineage-janus-connector.md + - Integration (metadata exchange): + - Apache Atlas Connector : connectors/integration/apache-atlas-catalog-integration-connector.md + - Data Files: connectors/integration/data-files-monitor-integration-connector.md + - Data Folders: connectors/integration/data-folder-monitor-integration-connector.md + - Databases (JDBC): connectors/integration/jdbc-integration-connector.md + - Kafka Topics (Cataloguing): connectors/integration/kafka-monitor-integration-connector.md + - Kafka Topics (Auditing): connectors/integration/kafka-topics-audit-integration-connector.md + - Strimzi Topics: connectors/integration/strimzi-monitor-integration-connector.md + - Open APIs via Swagger: connectors/integration/open-api-monitor-integration-connector.md + - Egeria Infrastructure: connectors/integration/egeria-infrastructure-cataloguer-integration-connector.md + - OpenLineage standard: - OpenLineage Event Receiver: connectors/integration/open-lineage-event-receiver-integration-connector.md - OpenLineage API Log Store: connectors/integration/api-based-open-lineage-log-store-integration-connector.md - OpenLineage File Log Store: connectors/integration/file-based-open-lineage-log-store-integration-connector.md - OpenLineage Cataloguer: connectors/integration/open-lineage-cataloguer-integration-connector.md - - Governance Action OpenLineage Integrator: connectors/integration/governance-action-open-lineage-integration-connector.md - - Sample Lineage Event Receiver: connectors/integration/sample-lineage-event-receiver-integration-connector.md - - Database: - - JDBC Integration Connector: connectors/integration/jdbc-integration-connector.md - - Catalog: - - Apache Atlas Connector : connectors/integration/apache-atlas-catalog-integration-connector.md - - Runtime: - - Repository: - - Native Repositories: - - XTDB: - - XTDB OMRS Repository Connector: connectors/repository/xtdb/index.md - - Upgrading the XTDB Connector: connectors/repository/xtdb/upgrade.md - - XTDB Connector Performance: connectors/repository/xtdb/performance.md - - JanusGraph OMRS Repository Connector: connectors/repository/janus-graph/overview.md - - In-memory OMRS Repository Connector: connectors/repository/in-memory/overview.md - - Read-only OMRS Repository Connector: connectors/repository/read-only/overview.md - - Repository Adapters: - - Apache Atlas OMRS Repository Connector: connectors/repository/apache-atlas/overview.md - - IBM Information Governance Catalog (IGC) OMRS Repository Connector: connectors/repository/igc/overview.md - - SAS Viya OMRS Repository Connector: connectors/repository/sas-viya/overview.md - - Hive Metastore (HMS) OMRS Proxy Repository Connector: connectors/repository/hms/overview.md - - Sample File OMRS Proxy Repository Connector: connectors/repository/file-sample-omrs-connector/overview.md - - Repository proxy embedded repository: /connectors/repository/repository-proxy-embedded-repository - - Audit Log Destination: - - Console: connectors/runtime/console-audit-log-destination-connector.md - - Event Topic: connectors/runtime/event-topic-audit-log-destination-connector.md - - Open Metadata Archive Store: - - JSON File: connectors/runtime/file-based-open-metadata-archive-store-runtime-connector.md - - Cohort Registry Store: - - JSON File: connectors/runtime/file-cohort-registry-store-connector.md - - Digital Resource: - - Open Metadata Topic: - - Kafka Open Metadata Topic Connector: connectors/resource/kafka-open-metadata-topic-connector.md - - In Memory Open Metadata Topic Connector: connectors/resource/in-mem-open-metadata-topic-connector.md - - Database: - - JDBC Connector: connectors/resource/jdbc-resource-connector.md + - Governance Action OpenLineage: connectors/integration/governance-action-open-lineage-integration-connector.md + - Sample Lineage Event Receiver: connectors/integration/sample-lineage-event-receiver-integration-connector.md + - Open Metadata Archive Store: + - JSON File: connectors/runtime/file-based-open-metadata-archive-store-runtime-connector.md + - Repository: + - Native Repositories: + - XTDB: + - XTDB OMRS Repository Connector: connectors/repository/xtdb/index.md + - Upgrading the XTDB Connector: connectors/repository/xtdb/upgrade.md + - XTDB Connector Performance: connectors/repository/xtdb/performance.md + - JanusGraph: connectors/repository/janus-graph/overview.md + - In-memory: connectors/repository/in-memory/overview.md + - Read-only: connectors/repository/read-only/overview.md + - Repository Adapters: + - Apache Atlas: connectors/repository/apache-atlas/overview.md + - IBM Information Governance Catalog (IGC): connectors/repository/igc/overview.md + - SAS Viya: connectors/repository/sas-viya/overview.md + - Hive Metastore (HMS): connectors/repository/hms/overview.md + - Sample File OMRS Proxy Repository Connector: connectors/repository/file-sample-omrs-connector/overview.md + - Repository proxy embedded repository: /connectors/repository/repository-proxy-embedded-repository - Secrets Store: - Environment Variables: connectors/secrets/environment-variable-secrets-store-connector.md - Developer Guide: @@ -202,6 +197,32 @@ nav: - User Feedback: features/user-feedback/overview.md - Learning: - Learning Overview: education/index.md + - Tutorials: + - Tutorials: education/tutorials/index.md + - Running Egeria: + - Downloading Egeria: education/tutorials/downloading-egeria-tutorial/overview.md + - Building Egeria: education/tutorials/building-egeria-tutorial/overview.md + - Installing Egeria: education/tutorials/installing-egeria-tutorial.md + - Using Egeria: + - Open Metadata Labs: education/open-metadata-labs/overview.md + - Platform and Servers: education/tutorials/omag-server-tutorial/overview.md + - Reports and utilities: education/tutorials/omag-client-tutorial.md + - Sample Clients: education/tutorials/running-samples-tutorial.md + - Developing with Egeria: + - Testing Egeria: education/tutorials/testing-egeria-tutorial.md + - Tools: + - Git and GitHub: education/tutorials/git-and-git-hub-tutorial/overview.md + - Using Kubernetes: guides/operations/kubernetes/index.md + - Using Docker: education/tutorials/docker-tutorial/overview.md + - Using IntelliJ: education/tutorials/intellij-tutorial/overview.md + - Using Postman: education/tutorials/postman-tutorial/overview.md + - Using Kafka: education/tutorials/kafka-tutorial/overview.md + - Webinars: education/webinar-program/overview.md + - Comparisons: + - Apache Atlas: education/comparisons/apache-atlas.md + - Scenarios: + - Data Manager Integration: scenarios/data-manager-integration/overview.md + - File Lineage: scenarios/file-lineage/overview.md - Dojos: - Dojo Overview: education/egeria-dojo/index.md - Running Egeria Day : @@ -220,26 +241,6 @@ nav: - Metadata Governance Day : education/egeria-dojo/metadata-governance/overview.md - Egeria Contributor Day : education/egeria-dojo/egeria-contributor.md - Egeria in Production Day : education/egeria-dojo/egeria-in-production.md - - Open Metadata Labs: education/open-metadata-labs/overview.md - - Tutorials: - - Git and GitHub: education/tutorials/git-and-git-hub-tutorial/overview.md - - Using Kubernetes: guides/operations/kubernetes/index.md - - Using Docker: education/tutorials/docker-tutorial/overview.md - - Using IntelliJ: education/tutorials/intellij-tutorial/overview.md - - Using Postman: education/tutorials/postman-tutorial/overview.md - - Using Kafka: education/tutorials/kafka-tutorial/overview.md - - Building Egeria: education/tutorials/building-egeria-tutorial/overview.md - - Installing Egeria: education/tutorials/installing-egeria-tutorial.md - - Running OMAG Clients: education/tutorials/omag-client-tutorial.md - - Running OMAG Servers: education/tutorials/omag-server-tutorial/overview.md - - Running Samples: education/tutorials/running-samples-tutorial.md - - Testing Egeria: education/tutorials/testing-egeria-tutorial.md - - Comparisons: - - Apache Atlas: education/comparisons/apache-atlas.md - - Scenarios: - - Data Manager Integration: scenarios/data-manager-integration/overview.md - - File Lineage: scenarios/file-lineage/overview.md - - Webinars: education/webinar-program/overview.md - Old Dojo (deprecated): - Dojo Introduction: getting-started/dojo/index.md - Day 1: @@ -313,6 +314,7 @@ nav: - Releases: - Overview: release-notes/overview.md - Monthly News: + - August 2023: release-notes/august-2023.md - April 2023: release-notes/april-2023.md - February 2023: release-notes/february-2023.md - January 2023: release-notes/january-2023.md @@ -570,12 +572,13 @@ nav: - Topic Integrator: services/omis/topic-integrator/overview.md - View Services (OMVS): - Open Metadata View Services: services/omvs/index.md + - My Profile: services/omvs/my-profile/overview.md - Glossary Author: services/omvs/glossary-author/overview.md - Glossary Browser: services/omvs/glossary-browser/overview.md - Glossary Workflow: services/omvs/glossary-workflow/overview.md - - Dino View: services/omvs/dino/overview.md - - Rex View: services/omvs/rex/overview.md - - Tex View: services/omvs/tex/overview.md + - Dynamic Infrastructure and Ops: services/omvs/dino/overview.md + - Repository Explorer: services/omvs/rex/overview.md + - Type Explorer: services/omvs/tex/overview.md - Server Author: services/omvs/server-author/overview.md - Frameworks: - Frameworks Overview : frameworks/index.md @@ -734,6 +737,7 @@ nav: - Open Metadata and Governance: concepts/open-metadata-and-governance.md - Open Metadata Archive: concepts/open-metadata-archive.md - Open Metadata Archive Store Connector: concepts/open-metadata-archive-store-connector.md + - Open Metadata Ecosystem: concepts/open-metadata-ecosystem.md - Open Metadata Exchange Rule: concepts/open-metadata-exchange-rule.md - Open Metadata Instance: concepts/open-metadata-instances.md - Open Metadata Type Definitions: concepts/open-metadata-type-definitions.md @@ -800,7 +804,7 @@ nav: - Gary Geeke: practices/coco-pharmaceuticals/personas/gary-geeke.md - George Pie: practices/coco-pharmaceuticals/personas/george-pie.md - Grant Able: practices/coco-pharmaceuticals/personas/grant-able.md - - harry Hopeful: practices/coco-pharmaceuticals/personas/harry-hopeful.md + - Harry Hopeful: practices/coco-pharmaceuticals/personas/harry-hopeful.md - Ivor Padlock: practices/coco-pharmaceuticals/personas/ivor-padlock.md - Jules Keeper: practices/coco-pharmaceuticals/personas/jules-keeper.md - Julie Stitched: practices/coco-pharmaceuticals/personas/julie-stitched.md @@ -830,7 +834,7 @@ nav: - Governance Team: practices/coco-pharmaceuticals/scenarios/building-the-governance-team/overview.md - Information Supply Chains: practices/coco-pharmaceuticals/scenarios/defining-information-supply-chains/overview.md - Multi-Faceted Governance: practices/coco-pharmaceuticals/scenarios/defining-multi-faceted-governance/overview.md - - New Systems Archiecture: practices/coco-pharmaceuticals/scenarios/defining-new-systems-architecture-overview/overview.md + - New Systems Architecture: practices/coco-pharmaceuticals/scenarios/defining-new-systems-architecture-overview/overview.md - New Digital Service: practices/coco-pharmaceuticals/scenarios/new-clinical-trials-digital-service/overview.md - Receiving Patient Data: practices/coco-pharmaceuticals/scenarios/receiving-patient-data-from-a-hospital/overview.md - Subject Areas: practices/coco-pharmaceuticals/scenarios/defining-subject-areas/overview.md diff --git a/site/docs/education/tutorials/building-egeria-tutorial/task-building-egeria-source.md b/site/snippets/supported-platforms.md similarity index 66% rename from site/docs/education/tutorials/building-egeria-tutorial/task-building-egeria-source.md rename to site/snippets/supported-platforms.md index 7105e85112..220a651ae9 100644 --- a/site/docs/education/tutorials/building-egeria-tutorial/task-building-egeria-source.md +++ b/site/snippets/supported-platforms.md @@ -1,9 +1,5 @@ - - -### Building the Egeria Source - -The build process takes the source files from the git repository and creates executable libraries needed to run Egeria. + === "Supported Platforms - linux, macOS" @@ -42,38 +38,3 @@ The build process takes the source files from the git repository and creates exe Yet another option to use IntelliJ is to make use of [Remote Development](https://www.jetbrains.com/remote-development/){ target=jb }. With this configuration you would use a seperate linux system, and connect remotely. This is beyond the scope of these docs. Feedback on Windows, offers to help, clarification on the steps can be directed to [odpi/egeria-docs#335](https://github.com/odpi/egeria-docs/issues/335){ target=gh } - - -#### Running the build - -When you download (clone) the contents of a git repository from GitHub, a new directory is created that is named after the repository that you cloned. For example, the directory created when the main `egeria.git` repository is cloned is called `egeria`. This directory contains all the source and the build scripts. - -The project uses three main build technologies: - -* [Gradle :material-dock-window:](https://gradle.org/){ target=gradle } is the primary build tool for the Egeria repositories. -* [Apache Maven :material-dock-window:](https://maven.apache.org/){ target=maven } is an alternative build tool to Gradle and is being phased out. -* [npm :material-dock-window:](https://www.npmjs.com){ target=npm } is used for Javascript repositories associated with the User Interfaces. - -The build scripts that use these technologies ensure the software is built in the correct order. - -##### Building with Gradle - ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-building-with-gradle.md" - -??? tip "Gradle development" - For egeria [Gradle :material-dock-window:](https://gradle.org/){ target=gradle } is a replacement build tool to Maven and offers: - - - better support for parallel builds - - more flexibility for build tasks - - breaking the link between directory structure and maven artifacts - - extremely fast incremental builds - - As of version 4, Egeria can only be built using gradle. - -##### Building with Maven - -If building a version of Egeria prior to version 4, the maven instructions can be found below: - ---8<-- "docs/education/tutorials/building-egeria-tutorial/task-building-with-maven.md" - - diff --git a/site/snippets/tasks/task-building-egeria-source.md b/site/snippets/tasks/task-building-egeria-source.md new file mode 100644 index 0000000000..725a501810 --- /dev/null +++ b/site/snippets/tasks/task-building-egeria-source.md @@ -0,0 +1,37 @@ + + + +### Building the Egeria Source + +When you download (clone) the contents of the egeria.git repository from GitHub, a new directory is created that is named after the repository that you cloned. So the directory created when the main `egeria.git` repository is cloned is called `egeria`. This directory contains all the source and the build scripts. + +Change to the egeria directory: +```bash +cd egeria +``` +The build scripts creates executable libraries needed to run Egeria using the source files from the git repository. They ensure the software is built consistently, in the correct order. + +???+ education "Building with Gradle (V4.0 and beyond)" + --8<-- "snippets/tasks/task-building-with-gradle.md" + +??? education "Building with Maven (Before V4.0)" + If building a version of Egeria prior to version 4, the maven instructions can be found below: + + --8<-- "snippets/tasks/task-building-with-maven.md" + +Once the build is complete, the executable files you need can be found under `open-metadata-distribution`. Change to this directory and list the files. + +```bash +cd open-metadata-distribution +``` +```bash +ls +``` + +The directories that you see will depend on the release of Egeria. Look in the `README.md` for a description of the different distributions that have been built. + +```bash +more README.md +``` + + diff --git a/site/docs/education/tutorials/building-egeria-tutorial/task-building-with-gradle.md b/site/snippets/tasks/task-building-with-gradle.md similarity index 100% rename from site/docs/education/tutorials/building-egeria-tutorial/task-building-with-gradle.md rename to site/snippets/tasks/task-building-with-gradle.md diff --git a/site/docs/education/tutorials/building-egeria-tutorial/task-building-with-maven.md b/site/snippets/tasks/task-building-with-maven.md similarity index 100% rename from site/docs/education/tutorials/building-egeria-tutorial/task-building-with-maven.md rename to site/snippets/tasks/task-building-with-maven.md diff --git a/site/docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md b/site/snippets/tasks/task-cloning-egeria-source.md similarity index 88% rename from site/docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md rename to site/snippets/tasks/task-cloning-egeria-source.md index 1d15530a79..8be4636d6d 100644 --- a/site/docs/education/tutorials/git-and-git-hub-tutorial/task-downloading-egeria-source.md +++ b/site/snippets/tasks/task-cloning-egeria-source.md @@ -1,9 +1,12 @@ -### Downloading the Egeria Source from GitHub +### Cloning the Egeria Source using git commands -The code for Egeria is downloaded from each git repository one at a time. The commands shown in each tab below create a clone (copy) of the egeria git repositories for your own use. If you want to make a contribution to Egeria, you need to [clone your own fork](/education/tutorials/git-and-git-hub-tutorial/#creating-a-fork-and-a-clone) of a repository rather than the main repository itself. +The code for Egeria is downloaded from each git repository one at a time. The commands shown in each tab below create a clone (copy) of the egeria git repositories for your own use. + +??? tip "Want to make a contribution to Egeria?" + If you want to make a contribution to Egeria, you need to [clone your own fork](/education/tutorials/git-and-git-hub-tutorial/#creating-a-fork-and-a-clone) of a repository rather than the main repository itself. === "egeria.git" diff --git a/site/snippets/tasks/task-downloading-egeria-source.md b/site/snippets/tasks/task-downloading-egeria-source.md new file mode 100644 index 0000000000..07a7c27956 --- /dev/null +++ b/site/snippets/tasks/task-downloading-egeria-source.md @@ -0,0 +1,19 @@ + + + +### Downloading Egeria from GitHub website + +Open Egeria's git repository on GitHub by clicking on this link: [https://github.com/odpi/egeria](https://github.com/odpi/egeria). The code that is displayed is the latest version of the Egeria code and it is in a branch called `main`. This is the default code that will be downloaded. If you want a specific release, click on the `main` button to reveal a scrollable menu. Scroll down to the branches named `egeria-release-{version-number}` and select the release you desire. + +![Select release](/education/tutorials/downloading-egeria/git-hub-select-release.png) + +Click on the green `<> Code` button to reveal a menu and select `Download ZIP` + +![Download ZIP](/education/tutorials/downloading-egeria/git-hub-download-zip.png) + +The egeria source is quite big do it may take a few minutes to download. Once it is downloaded, you can unpack the zip file into a suitable directory (folder) on your machine. The file structure will be the same as the structure shown on GitHub. + + + + + diff --git a/site/docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md b/site/snippets/tasks/task-installing-egeria.md similarity index 72% rename from site/docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md rename to site/snippets/tasks/task-installing-egeria.md index 0c9a4b5d3e..8458b9c584 100644 --- a/site/docs/education/tutorials/building-egeria-tutorial/task-installing-egeria.md +++ b/site/snippets/tasks/task-installing-egeria.md @@ -5,6 +5,6 @@ Change to the top level `egeria` directory where your local copy of `egeria.git` is downloaded to. ---8<-- "docs/education/tutorials/building-egeria-tutorial/terminal-installing-egeria.md" +--8<-- "snippets/tasks/terminal-installing-egeria.md" diff --git a/site/docs/education/tutorials/git-and-git-hub-tutorial/task-installing-git.md b/site/snippets/tasks/task-installing-git.md similarity index 100% rename from site/docs/education/tutorials/git-and-git-hub-tutorial/task-installing-git.md rename to site/snippets/tasks/task-installing-git.md diff --git a/site/docs/education/tutorials/building-egeria-tutorial/task-installing-java.md b/site/snippets/tasks/task-installing-java.md similarity index 93% rename from site/docs/education/tutorials/building-egeria-tutorial/task-installing-java.md rename to site/snippets/tasks/task-installing-java.md index bdc7412450..cfd51de77f 100644 --- a/site/docs/education/tutorials/building-egeria-tutorial/task-installing-java.md +++ b/site/snippets/tasks/task-installing-java.md @@ -16,9 +16,9 @@ if java is already installed by running the command `java -version` from the com Java can be installed by: 1. Downloading the *OpenJDK 17 (LTS) HotSpot* JVM from [Adoptium](https://adoptium.net/). -1. Running the installer that is downloaded. +2. Running the installer that is downloaded. -Alternatively you may wish to install from your package manager such as `homebrew` on MacOS. +Alternatively you may wish to install from your package manager such as `homebrew` on macOS. diff --git a/site/docs/education/tutorials/jupyter-tutorial/task-installing-jupyter.md b/site/snippets/tasks/task-installing-jupyter.md similarity index 83% rename from site/docs/education/tutorials/jupyter-tutorial/task-installing-jupyter.md rename to site/snippets/tasks/task-installing-jupyter.md index ac25429384..8dd125569d 100644 --- a/site/docs/education/tutorials/jupyter-tutorial/task-installing-jupyter.md +++ b/site/snippets/tasks/task-installing-jupyter.md @@ -13,9 +13,9 @@ Full instructions for downloading and installing either of these options can be `https://jupyter.org/` -Alternatively, if you are running MacOS you can use HomeBrew. +Alternatively, if you are running macOS you can use HomeBrew. -???+ cli "Installing JupyterLab on MacOS using HomeBrew" +???+ cli "Installing JupyterLab on macOS using HomeBrew" Using [HomeBrew :material-dock-window:](https://brew.sh){ target=homebrew } you can simply run: ```shell diff --git a/site/docs/education/tutorials/kafka-tutorial/task-installing-kafka.md b/site/snippets/tasks/task-installing-kafka.md similarity index 100% rename from site/docs/education/tutorials/kafka-tutorial/task-installing-kafka.md rename to site/snippets/tasks/task-installing-kafka.md diff --git a/site/docs/education/tutorials/building-egeria-tutorial/task-installing-maven.md b/site/snippets/tasks/task-installing-maven.md similarity index 80% rename from site/docs/education/tutorials/building-egeria-tutorial/task-installing-maven.md rename to site/snippets/tasks/task-installing-maven.md index eb7ea7d178..9cf3e36d44 100644 --- a/site/docs/education/tutorials/building-egeria-tutorial/task-installing-maven.md +++ b/site/snippets/tasks/task-installing-maven.md @@ -3,11 +3,9 @@ ### Installing Maven -[Apache Maven :material-dock-window:](https://maven.apache.org/){ target=maven } is a build tool at is being phased out in the Egeria project, but is still required by some repositories. It is capable of code compilation, running unit tests, validating dependencies and Javadoc as well as build our distribution archive. +[Apache Maven :material-dock-window:](https://maven.apache.org/){ target=maven } is a build tool at is being phased out in the Egeria project, but is still required by some repositories and the Egeria developer dojo. It is capable of code compilation, running unit tests, validating dependencies and Javadoc as well as build our distribution archive. -Egeria 4.0 and above cannot be built using maven. - -where it is used, Egeria requires Maven 3.5 or higher. 3.6.x or above is recommended. +Where it is used, Egeria requires Maven 3.5 or higher. 3.6.x or above is recommended. !!! cli "Check if Maven is installed" ```shell diff --git a/site/docs/education/tutorials/kafka-tutorial/task-starting-kafka.md b/site/snippets/tasks/task-starting-kafka.md similarity index 100% rename from site/docs/education/tutorials/kafka-tutorial/task-starting-kafka.md rename to site/snippets/tasks/task-starting-kafka.md diff --git a/site/docs/education/tutorials/kafka-tutorial/task-stopping-kafka.md b/site/snippets/tasks/task-stopping-kafka.md similarity index 100% rename from site/docs/education/tutorials/kafka-tutorial/task-stopping-kafka.md rename to site/snippets/tasks/task-stopping-kafka.md diff --git a/site/snippets/tasks/terminal-installing-egeria.md b/site/snippets/tasks/terminal-installing-egeria.md new file mode 100644 index 0000000000..736164990e --- /dev/null +++ b/site/snippets/tasks/terminal-installing-egeria.md @@ -0,0 +1,138 @@ + + + + +The [egeria build process](/education/tutorials/building-egeria-tutorial/overview) creates the distribution files for Egeria in the `open-metadata-distribution/open-metadata-assemblies` module. To see its contents, after a full gradle build completes, use the following `cd` command to change to its `build/distributions` directory: + +```bash +cd open-metadata-distribution/omag-server-platform/build/distributions +``` +List the files: +```bash +ls +``` +The distribution tar file is `{release}-distribution.tar.gz` or `egeria-platform-4.3-distribution.tar.gz` in this example. + +```bash +egeria-platform-4.3-distribution.tar.gz +``` + +Create a directory for the install and copy the tar file into it. The two commands shown below create an install directory in your home directory and then copies the egeria distribution file into it. + +```bash +mkdir ~/egeria-install +``` +```bash +cp egeria*-distribution.tar.gz ~/egeria-install +``` + +These next commands change to the new directory and lists its contents. + +```bash +cd ~/egeria-install +``` + +It is now possible to unpack the tar file with the following steps. + +```bash +gunzip egeria*-distribution.tar.gz +``` +```bash +tar -xf egeria*-distribution.tar +``` +A new directory is created called `{release}-distribution.tar.gz` or `egeria-4.2-distribution.tar.gz` in this example. Change to this new directory and list its contents as shown below. + +```bash +cd egeria*gz +``` +```bash +ls +``` +```bash +Dockerfile LICENSE NOTICE README.md assembly dist +``` +As before, you may notice different files as Egeria evolves. + +The `Dockerfile` contains the directives to build a docker image. Instructions for doing this are in the `README.md` file. Change to the `assembly` directory. + +```bash +cd assembly +``` +```bash +ls +``` +```bash +README.md etc opt platform +``` +Under `platform` is a directory for the [OMAG Server Platform](/concepts/omag-server-platform) that is used to run open metadata and governance services. This is the `omag-server-platform-{{release}}.jar`. + +```bash +ls platform +``` +```bash +README.md data keystore.p12 logs truststore.p12 +application.properties extra lib omag-server-platform-4.3.jar +``` +The `platform/lib` directory is where the jar files for connectors, samples and new registered services are installed. It includes the connectors that are located in the `egeria.git` repository. +```bash +ls platform/lib +``` +```bash +README.md csv-file-connector-4.3.jar jdbc-integration-connector-4.3.jar +apache-atlas-rest-connector-4.3.jar data-folder-connector-4.3.jar jdbc-resource-connector-4.3.jar +atlas-integration-connector-4.3.jar discovery-service-connectors-4.3.jar kafka-integration-connector-4.3.jar +audit-log-console-connector-4.3.jar dynamic-archiver-connectors-4.3.jar kafka-open-metadata-topic-connector-4.3.jar +audit-log-event-topic-connector-4.3.jar elasticsearch-integration-connector-4.3.jar omrs-rest-repository-connector-4.3.jar +audit-log-file-connector-4.3.jar env-variable-secrets-store-connector-4.3.jar open-lineage-janus-connector-4.3.jar +audit-log-slf4j-connector-4.3.jar files-integration-connectors-4.3.jar open-metadata-archive-directory-connector-4.3.jar +avro-file-connector-4.3.jar governance-action-connectors-4.3.jar open-metadata-archive-file-connector-4.3.jar +basic-file-connector-4.3.jar governance-services-sample-4.3.jar open-metadata-security-samples-4.3.jar +cohort-registry-file-store-connector-4.3.jar graph-repository-connector-jar-with-dependencies-4.3.jar openapi-integration-connector-4.3.jar +configuration-encrypted-file-store-connector-4.3.jar inmemory-open-metadata-topic-connector-4.3.jar openlineage-integration-connectors-4.3.jar +configuration-file-store-connector-4.3.jar inmemory-repository-connector-4.3.jar spring-rest-client-connector-4.3.jar + +``` +The `platform/extra` directory is where connectors from other repositories are added (including those that you write yourself). Copy the jar files for any additional connectors you want to use into the `extra` directory. The connectors available for Egeria are listed in the [Connector Catalog](/connectors). + +The `opt` and `etc` directories contain additional content that can used with the platform. For example, the `etc/reports` directory contains java programs that describe different aspects of the platform. +```bash +ls etc/reports +``` +```bash +README.md component-id-report.jar database-report.jar egeria-platform-report.jar +``` +The `opt/content-packs` directory contains [Open Metadata Archives](/concepts/open-metadata-archive) that provide sample open metadata content. The `README.md` describes their content. +```bash +ls content-packs +``` +```bash +CloudInformationModel.json CocoGovernanceProgramArchive.json OpenConnectorsArchive.json SimpleDataCatalog.json +CocoBusinessSystemsArchive.json CocoOrganizationArchive.json OpenMetadataTypes.json SimpleEventCatalog.json +CocoClinicalTrialsTemplatesArchive.json CocoSustainabilityArchive.json README.md SimpleGovernanceCatalog.json +CocoComboArchive.json CocoTypesArchive.json SimpleAPICatalog.json +CocoGovernanceEngineDefinitionsArchive.json DataStoreConnectorTypes.json SimpleCatalog.json +``` +The `/opt/sample-data` directory contains sample data that is used in various labs and samples. +```bash +ls /opt/sample-data/* +``` +```bash +README.md database old-market-drop-foot-weekly-measurements +data-files oak-dene-drop-foot-weekly-measurements +mandy-chessell@Amandas-MacBook-Pro assembly % ls opt/sample-data/* +opt/sample-data/README.md + +opt/sample-data/data-files: +CompDir-ContactEmail.csv CompDir-ContactPhone.csv Employee-Dept.csv Location-WorkLocation.csv +CompDir-ContactList.csv EmplSAnl-EmpSalaryAnalysis.csv Employee-Employee.csv Patient-Patient.csv + +opt/sample-data/database: +ibm-db2 mariadb postgresql + +opt/sample-data/oak-dene-drop-foot-weekly-measurements: +week1.csv week2.csv week3.csv week4.csv week5.csv week6.csv week7.csv week8.csv week9.csv +``` + +Notice that each directory contains a `README.md` file that explains the content of the directory. + +--8<-- "snippets/abbr.md"