From 6d33ada917fbe73c81b96e29f5424a66dcf7cf6d Mon Sep 17 00:00:00 2001 From: Mandy Chessell Date: Wed, 20 Dec 2023 17:31:49 +0000 Subject: [PATCH] Add file-types and deployed-implementation-types Signed-off-by: Mandy Chessell --- .../concepts/deployed-implementation-type.md | 8 +- site/docs/concepts/file-type.md | 19 ++ site/docs/concepts/index.md | 5 +- site/docs/concepts/omag-server-runtime.md | 8 + site/docs/concepts/service-url-marker.md | 11 + .../reference-data-management.drawio | 70 +++--- .../valid-value-definition.svg | 2 +- .../valid-value-set.svg | 2 +- ...onfiguring-omag-server-basic-properties.md | 92 +++++-- site/docs/release-notes/4-4.md | 4 +- .../2/0280-Software-Development-Assets.svg | 2 +- site/docs/types/2/area-2-assets.drawio | 227 +++++++++--------- site/docs/types/5/0545-Reference-Data.svg | 2 +- .../types/5/area-5-models-and-schema.drawio | 26 +- site/mkdocs.yml | 3 +- .../deployed-implementation-types.md | 20 ++ site/snippets/content-packs/file-types.md | 134 +++++++++++ 17 files changed, 446 insertions(+), 189 deletions(-) create mode 100644 site/docs/concepts/file-type.md create mode 100644 site/docs/concepts/omag-server-runtime.md create mode 100644 site/docs/concepts/service-url-marker.md create mode 100644 site/snippets/content-packs/deployed-implementation-types.md create mode 100644 site/snippets/content-packs/file-types.md diff --git a/site/docs/concepts/deployed-implementation-type.md b/site/docs/concepts/deployed-implementation-type.md index 0f569a2aa0..19f9484c02 100644 --- a/site/docs/concepts/deployed-implementation-type.md +++ b/site/docs/concepts/deployed-implementation-type.md @@ -24,12 +24,8 @@ A *deployed implementation type* is a type of reference data that describes a cl * [*CloudPlatform*](/types/0/0090-Cloud-Platforms-and-Services) * [*ServerPurpose*](/types/0/0041-Server-Purpose) +## Examples of Deployed Implementation Types - -## File Type - -A *file type* is similar to *deployed implementation type*. It is a reference data value that that describes a type of file. It includes the file extension for the type of file to allow matching of a file's extension to a specific asset type. Again the file type links to templates and resources for processing files of this type. - -The file type's *preferredValue* can be used to consistently populate the *fileType* attribute of [*DataFile*](/types/2/0220-Files-and-Folders) assets. This +--8<-- "snippets/content-packs/deployed-implementation-types.md" --8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/file-type.md b/site/docs/concepts/file-type.md new file mode 100644 index 0000000000..261647fc62 --- /dev/null +++ b/site/docs/concepts/file-type.md @@ -0,0 +1,19 @@ +--- +hide: +- toc +--- + + + + +# File Type + +A *file type* is similar to [*deployed implementation type*](/concepts/deployed-implementation-type). It is a reference data value that that describes a type of file. It links to reference data about the file extensions for the type of file to allow matching of a file's extension to a specific asset type. Again the file type may link to templates and resources for processing files of this type. + +The file type's *preferredValue* can be used to consistently populate the *fileType* attribute of [*DataFile*](/types/2/0220-Files-and-Folders) assets. This acts as a logical file type. + +## Examples of File Types + +--8<-- "snippets/content-packs/file-types.md" + +--8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/index.md b/site/docs/concepts/index.md index 813395fbc7..f88d8b2dfa 100644 --- a/site/docs/concepts/index.md +++ b/site/docs/concepts/index.md @@ -115,7 +115,8 @@ - [Event](/concepts/basic-concepts/#event) - [Cohort Events](/concepts/cohort-events) - + +- [Event Action Framework](/frameworks/eaf/overview) - [Event Bus](/concepts/event-bus) - [Event Broker](/concepts/basic-concepts/#event-broker) - [Event Management Services Subsystem (OMRS)](/services/omrs) @@ -132,6 +133,7 @@ - [omag.server.servername.config](/concepts/configuration-document) - [servername.cohortname.registrystore](/connectors/runtime/file-cohort-registry-store-connector) + - [*.omarchive](/connectors/runtime/file-based-open-metadata-archive-store-runtime-connector) - [Files Integrator OMIS](/services/omis/files-integrator/overview) - [Framework Services](/services/#framework-services) @@ -142,6 +144,7 @@ - [Open Integration Framework (OIF)](/frameworks/oif/overview) - [Open Discovery Framework (ODF)](/frameworks/odf/overview) - [Governance Action Framework (GAF)](/frameworks/gaf/overview) + - [Event Action Framework (EAF)](/frameworks/eaf/overview) ## G diff --git a/site/docs/concepts/omag-server-runtime.md b/site/docs/concepts/omag-server-runtime.md new file mode 100644 index 0000000000..ef3b6a306d --- /dev/null +++ b/site/docs/concepts/omag-server-runtime.md @@ -0,0 +1,8 @@ + + + +# OMAG Server Runtime + +The *OMAG Server Runtime* provides a runtime process for a single [OMAG Server](/concepts/omag-server). + +--8<-- "snippets/abbr.md" diff --git a/site/docs/concepts/service-url-marker.md b/site/docs/concepts/service-url-marker.md new file mode 100644 index 0000000000..08f95aaf90 --- /dev/null +++ b/site/docs/concepts/service-url-marker.md @@ -0,0 +1,11 @@ + + + +# Service URL Marker + +The *Service URL Marker* identifies a specific Egeria service in a URL. This URL may be for a REST API call, or a documentation page. + +The value used for each service can be retrieved via the [Platform Services](/services/platform-services/overview/#registered-services). + + +--8<-- "snippets/abbr.md" diff --git a/site/docs/features/reference-data-management/reference-data-management.drawio b/site/docs/features/reference-data-management/reference-data-management.drawio index 48067a3aee..4c0e8356ed 100644 --- a/site/docs/features/reference-data-management/reference-data-management.drawio +++ b/site/docs/features/reference-data-management/reference-data-management.drawio @@ -1,4 +1,4 @@ - + @@ -115,8 +115,8 @@ - - + + @@ -223,19 +223,19 @@ - + - + - + - - + + @@ -1022,10 +1022,10 @@ - + - + @@ -1036,16 +1036,16 @@ - + - + - + - + @@ -1054,27 +1054,27 @@ - + - + - + - + - + - + - + @@ -1082,38 +1082,38 @@ - + - + - + - + - + - + - + - + - + - + @@ -1123,7 +1123,7 @@ - + @@ -1133,13 +1133,13 @@ - + - + - + diff --git a/site/docs/features/reference-data-management/valid-value-definition.svg b/site/docs/features/reference-data-management/valid-value-definition.svg index 510ad27c75..e3b7fdf0e6 100644 --- a/site/docs/features/reference-data-management/valid-value-definition.svg +++ b/site/docs/features/reference-data-management/valid-value-definition.svg @@ -1,4 +1,4 @@ -
«entity»
Referenceable
«entity»...
qualifiedName : string
additionalProperties : map<string, string>
qualifiedName : string...
«entity»
ValidValueDefinition
«entity»...
name : string
description : string
category : string
usage : string
scope : string
preferredValue : string
name : string...Text is not SVG - cannot display \ No newline at end of file +
«entity»
Referenceable
«entity»...
qualifiedName : string
additionalProperties : map<string, string>
qualifiedName : string...
«entity»
ValidValueDefinition
«entity»...
name : string
description : string
category : string
usage : string
scope : string
preferredValue : string
isDeprecated : boolean
isCaseSensitive : boolean
name : string...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/features/reference-data-management/valid-value-set.svg b/site/docs/features/reference-data-management/valid-value-set.svg index f9468d5f60..3456fc1ab4 100644 --- a/site/docs/features/reference-data-management/valid-value-set.svg +++ b/site/docs/features/reference-data-management/valid-value-set.svg @@ -1,4 +1,4 @@ -
«entity»
Referenceable
«entity»...
qualifiedName : string
additionalProperties : map<string, string>
qualifiedName : string...
«entity»
ValidValueDefinition
«entity»...
name : string
description : string
category : string
usage : string
scope : string
preferredValue : string
name : string...
«entity»
ValidValuesSet
«entity»...
memberOfValidValuesSet
memberOfValidVa...
validValuesSet
validValuesSet
*
*
*
*
«relationship»
ValidValueMember
«relationship»...Text is not SVG - cannot display \ No newline at end of file +
«entity»
Referenceable
«entity»...
qualifiedName : string
additionalProperties : map<string, string>
qualifiedName : string...
«entity»
ValidValueDefinition
«entity»...
name : string
description : string
category : string
usage : string
scope : string
preferredValue : string
isDeprecated : boolean
isCaseSensitive : boolean
name : string...
«entity»
ValidValuesSet
«entity»...
memberOfValidValuesSet
memberOfValidVa...
validValuesSet
validValuesSet
*
*
*
*
«relationship»
ValidValueMember
«relationship»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/guides/admin/servers/configuring-omag-server-basic-properties.md b/site/docs/guides/admin/servers/configuring-omag-server-basic-properties.md index 6148dc8c5b..3096180ad4 100644 --- a/site/docs/guides/admin/servers/configuring-omag-server-basic-properties.md +++ b/site/docs/guides/admin/servers/configuring-omag-server-basic-properties.md @@ -7,39 +7,73 @@ The basic properties are used in logging and events originating from the server. The basic properties include two unique identifiers that are set up when you first create the configuration document: -| Property | Description | -|---|---| -| `localServerId` | Unique identifier for this server. By default, this is initialized to a randomly generated Universal Unique identifier (UUID). | -| `localServerName` | Meaningful name for the server for use in messages and UIs. Ideally this value is unique to aid administrators in understanding the source of messages and events from the server. This value is set to the server name assigned when the configuration is created. | +| Property | Description | +|-------------------|---| +| *localServerId* | Unique identifier for this server. By default, this is initialized to a randomly generated Universal Unique identifier (UUID). | +| *localServerName* | Meaningful name for the server for use in messages and UIs. Ideally this value is unique to aid administrators in understanding the source of messages and events from the server. This value is set to the server name assigned when the configuration is created. | The other basic properties have values that can be changed through the admin services API: -| Property | Description | -|---|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `localServerType` | Descriptive type name for the server. Again this is useful information for the administrator to understand the role of the server. The default value is `Open Metadata and Governance Server`. | -| `organizationName` | Descriptive name for the organization that owns the local server/repository. This is useful when the open metadata repository cluster consists of metadata servers from different organizations, or different departments of an enterprise. The default value is `null`. | -| `localServerUserId` | UserId to use for server-initiated REST calls. The default is `OMAGServer`. | -| `localServerPassword` | Password to use for server-initiated REST calls. The default is `null`. This means that only the userId is sent in the HTTP header. | -| `maxPageSize` | The maximum page size that can be set on requests to the server. The default value is `1000`. A value of zero means unlimited page size. Although supported, the zero value is not recommended because it provides no protection from a large request denial of service attack. | +| Property | Description | +|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| *localServerDescription* | Description for the server. This is useful information for the administrator to understand the role of the server. The default value is `null`. | +| *organizationName* | Descriptive name for the organization that owns the local server/repository. This is useful when the open metadata repository cluster consists of metadata servers from different organizations, or different departments of an enterprise. The default value is `null`. | +| *localServerUserId* | UserId to use for server-initiated REST calls. The default is `OMAGServer`. | +| *localServerPassword* | Password to use for server-initiated REST calls. The default is `null`. This means that only the userId is sent in the HTTP header. | +| *localServerType* | Descriptive type name for the server. Again this is useful information for the administrator to understand the role of the server. The default value is `null` which means that the [server operation services](/services/server-operations) will populate it automatically based on the services that are configured for the server. | +| *maxPageSize* | The maximum page size that can be set on requests to the server. The default value is `1000`. A value of zero means unlimited page size. Although supported, the zero value is not recommended because it provides no protection from a large request denial of service attack. | The sections that follow cover how to set up these values. -### Set server type name +### Set basic server properties The server type name should be set to something that describes the OMAG Server's role. It may be the name of a specific product that it is enabling, or a role in the metadata and governance landscape. The default value is `Open Metadata and Governance Server`. If you have no specific value to set the server type name to, we recommend that you set the server type name to null. This will cause the server start up process will derive a standard server type name based on the rest of the configuration for the server. -!!! post "POST - set server type" +!!! post "POST - setBasicServerProperties" +``` +{{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-properties" +``` +The request body contains the properties to set. + +| Property | Description | +|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| *localServerDescription* | Description for the server. This is useful information for the administrator to understand the role of the server. The default value is `null`. | +| *organizationName* | Descriptive name for the organization that owns the local server/repository. This is useful when the open metadata repository cluster consists of metadata servers from different organizations, or different departments of an enterprise. The default value is `null`. | +| *localServerURL* | The *platformURLRoot* for the platform where this server is to run. For example `https://localhost:9443`. It is used if the server connects to a [cohort](/concepts/cohort-member). | +| *localServerUserId* | UserId to use for server-initiated REST calls. The default is `OMAGServer`. | +| *localServerPassword* | Password to use for server-initiated REST calls. The default is `null`. This means that only the userId is sent in the HTTP header. | +| *maxPageSize* | The maximum page size that can be set on requests to the server. The default value is `1000`. A value of zero means unlimited page size. Although supported, the zero value is not recommended because it provides no protection from a large request denial of service attack. | + +For example: + +```json +{ + "localServerDescription" : "This server supports the governance teams", + "organizationName" : "Coco Pharmaceuticals", + "localServerURL" : "https://localhost:9443", + "localServerUserId" : "cocomds2npa", + "localServerPassword" : "secret", + "maxPageSize" : 600 +} +``` + +### Set server description + +The server description should be set to something that describes the OMAG Server's role. It may be the name of a specific product that it is enabling, or a role in the metadata and governance landscape. Its purpose is to help administrators identify which server configuration they need to work with. + +!!! post "POST - setServerDescription" ``` - {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-type?typeName="{{serverTypeName}}" + {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-description" ``` +The description is passed in the request body as a text string. ### Set organization name -The organization name may be the owning organization or department or team supported by the server. +The organization name may be the owning organization or you may use it to identify the department or team that is supported by this server. -!!! post "POST - set organization name" +!!! post "POST - setOrganizationName" ``` {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/organization-name?name="{{organizationName}}" ``` @@ -50,12 +84,12 @@ The server's userId is used when processing requests that do not have an end use If the password is specified as well, the userId and password combination are used to provide authentication information on each REST call made by the server. -!!! post "POST - set server's userId" +!!! post "POST - setServerUserId" ``` {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-user-id?id="{{serverUserId}}" ``` -!!! post "POST - set server's password" +!!! post "POST - setServerPassword" ``` {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-user-password?password="{{serverUserPassword}}" ``` @@ -64,7 +98,27 @@ If the password is specified as well, the userId and password combination are us The maximum page size value sets an upper limit on the number of results that a caller can request on any paging REST API to this server. Setting maximum page size helps to prevent a denial of service attack that uses very large requests to overwhelm the server. A value of `0` means no limit, and leaves the server open to such attacks. -!!! post "POST - set maximum page size" +!!! post "POST - setMaxPageSize" ``` {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/max-page-size?limit={{maxPageSize}} ``` + +### Set server type name + +Typically, the server type name should be left blank to allow the [server operations service](/services/server-operations) to classify the server from its configuration, since it gives the servers standard type values. + +If, however, you want to use your own values, then this is the command to set it up. + +!!! post "POST - setDescriptiveServerType" + ``` + {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-type?typeName="{{serverTypeName}}" + ``` + +### Get server type classification + +If you are curious to see the automated server type that has been assigned to a server, it can be retrieved with the following command. + +!!! get "GET - getServerClassification" + ``` + {{platformURLRoot}}/open-metadata/admin-services/users/{{adminUserId}}/servers/{{serverName}}/server-type-classification" + ``` \ No newline at end of file diff --git a/site/docs/release-notes/4-4.md b/site/docs/release-notes/4-4.md index 7a392b2cbb..e9a35a418e 100644 --- a/site/docs/release-notes/4-4.md +++ b/site/docs/release-notes/4-4.md @@ -18,8 +18,8 @@ _**These are DRAFT release notes. The notes will be updated until & when 4.4 is * There is a new schema extraction relationship called [*DiscoveredLinkedDataField*](/types/6/0615-Schema-Extraction) that can be used to describe associations between data fields. The [*DataField*](/types/6/0615-Schema-Extraction) entity has new attributes to align it more closely with [SchemaAttribute](/types/5/0505-Schema-Attributes). * There are new lineage relationships called [*UltimateSource*](/types/7/0755-Ultimate-Source-Destination) and [*UltimateDestination*](/types/7/0755-Ultimate-Source-Destination) can be used to capture the size and edges of an element's lineage graph have been added. * The *GroupedMedia* relationship is deprecated in favour of [*DataContentForDataSet*](/types/2/0210-Data-Stores). - * The [*DataFile*](/types/2/0220-Files-and-Folders) type has a attribute called *fileExtension*. New types that inherit from *DataFile* have been defined for a wide range of file types: [*SpreadsheetFile*](/types/2/0220-Files-and-Folders), [*XMLFile*](/types/2/0220-Files-and-Folders), [*AudioFile*](/types/2/0221-Document-Stores), [*VideoFile*](/types/2/0221-Document-Stores), [*3DImageFile*](/types/2/0221-Document-Stores), [*RasterFile*](/types/2/0221-Document-Stores), [*VectorFile*](/types/2/0221-Document-Stores), [*SourceCodeFile*](/types/2/0280-Software-Development-Assets), [*ExecutableFile*](/types/2/0280-Software-Development-Assets), [*ScriptFile*](/types/2/0280-Software-Development-Assets), [*BuildInstructionFile*](/types/2/0280-Software-Development-Assets), [*YAMLFile*](/types/2/0280-Software-Development-Assets), and [*ArchiveFile*](/types/2/0226-Archive-Files). There is also a new relationship called [*ArchiveFileContents*](/types/2/0226-Archive-Files) to identify the contents of an archive file. - * There is a new attribute called *category* for [*ValidValueDefinition*](/types/5/0545-Reference-Data) that provides a descriptive name of the concept that this valid value describes a possible value for. + * The [*DataFile*](/types/2/0220-Files-and-Folders) type has a attribute called *fileExtension*. New types that inherit from *DataFile* have been defined for a wide range of file types: [*SpreadsheetFile*](/types/2/0220-Files-and-Folders), [*XMLFile*](/types/2/0220-Files-and-Folders), [*AudioFile*](/types/2/0221-Document-Stores), [*VideoFile*](/types/2/0221-Document-Stores), [*3DImageFile*](/types/2/0221-Document-Stores), [*RasterFile*](/types/2/0221-Document-Stores), [*VectorFile*](/types/2/0221-Document-Stores), [*SourceCodeFile*](/types/2/0280-Software-Development-Assets), [*ExecutableFile*](/types/2/0280-Software-Development-Assets), [*ScriptFile*](/types/2/0280-Software-Development-Assets), [*BuildInstructionFile*](/types/2/0280-Software-Development-Assets), [*YAMLFile*](/types/2/0280-Software-Development-Assets), [*PropertiesFile*](/types/2/0280-Software-Development-Assets), and [*ArchiveFile*](/types/2/0226-Archive-Files). There is also a new relationship called [*ArchiveFileContents*](/types/2/0226-Archive-Files) to identify the contents of an archive file. + * There are new attribute called *isCaseSensitive* and *category* for [*ValidValueDefinition*](/types/5/0545-Reference-Data) that provides additional information used to match valid values. * There are new types to help automate the curation of assets. They are in a new model called [0207](/types/2/0207-Deployed-Implementation-Types) and are called *DeployedImplementationType*, *FileType* and *CatalogTarget*. ??? warning "XTDB repository connector are now in egeria.git" diff --git a/site/docs/types/2/0280-Software-Development-Assets.svg b/site/docs/types/2/0280-Software-Development-Assets.svg index 8e8a999602..13083916a2 100644 --- a/site/docs/types/2/0280-Software-Development-Assets.svg +++ b/site/docs/types/2/0280-Software-Development-Assets.svg @@ -1,4 +1,4 @@ -
0280 - Software Development Assets
0280 - Software Development Assets
«entity»
DataStore
«entity»...
«entity»
DataFile
«entity»...
«entity»
SourceCodeFile
«entity»...
language : string
language : string
«entity»
Asset
«entity»...
purpose : string
buildId : string
purpose : string...
«classification»
BuildTarget
«classification»...
«entity»
ExternalReference
«entity»...
«entity»
ImplementationLocation
«entity»...
«classification»
ReusableTechnique
«classification»...
«relationship»
ReusableTechniqueUse
«relationship»...

 
 
«entity»
Asset
«entity»...
reusedTechnique
reusedTechnique
techniqueInUseBy
techniqueInUseBy
*
*
*
*
«entity»
YAMLFile
«entity»...
«entity»
BuildInstructionFile
«entity»...
language : string
language : string
«entity»
ScriptFile
«entity»...
language : string
language : string
«entity»
ExecutableFile
«entity»...
language : string
language : stringText is not SVG - cannot display \ No newline at end of file +
0280 - Software Development Assets
0280 - Software Development Assets
«entity»
DataStore
«entity»...
«entity»
DataFile
«entity»...
«entity»
SourceCodeFile
«entity»...
language : string
language : string
«entity»
Asset
«entity»...
purpose : string
buildId : string
purpose : string...
«classification»
BuildTarget
«classification»...
«entity»
ExternalReference
«entity»...
«entity»
ImplementationLocation
«entity»...
«classification»
ReusableTechnique
«classification»...
«relationship»
ReusableTechniqueUse
«relationship»...

 
 
«entity»
Asset
«entity»...
reusedTechnique
reusedTechnique
techniqueInUseBy
techniqueInUseBy
*
*
*
*
«entity»
YAMLFile
«entity»...
«entity»
BuildInstructionFile
«entity»...
language : string
language : string
«entity»
ScriptFile
«entity»...
language : string
language : string
«entity»
ExecutableFile
«entity»...
language : string
language : string
«entity»
PropertiesFile
«entity»...Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/2/area-2-assets.drawio b/site/docs/types/2/area-2-assets.drawio index a8a175c8a6..2c41d25aed 100644 --- a/site/docs/types/2/area-2-assets.drawio +++ b/site/docs/types/2/area-2-assets.drawio @@ -1,6 +1,6 @@ - + - + @@ -148,10 +148,10 @@ - + - + @@ -1081,7 +1081,7 @@ - + @@ -1257,39 +1257,39 @@ - + - + - + - + - + - + - + - + - + @@ -1373,71 +1373,71 @@ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + @@ -1466,7 +1466,7 @@ - + @@ -1658,7 +1658,7 @@ - + @@ -1744,65 +1744,65 @@ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + @@ -1848,7 +1848,7 @@ - + @@ -1995,7 +1995,7 @@ - + @@ -2009,154 +2009,165 @@ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - - + + - + - + - - + + - - + + - + - + - - + + - - + + - - + + - + - + - - + + - - + + - - + + - - + + - - + + - + - - + + - - + + + + + + + + + + + + + diff --git a/site/docs/types/5/0545-Reference-Data.svg b/site/docs/types/5/0545-Reference-Data.svg index 6e18d0353b..49702f793c 100644 --- a/site/docs/types/5/0545-Reference-Data.svg +++ b/site/docs/types/5/0545-Reference-Data.svg @@ -1,4 +1,4 @@ -
0545 - Reference Data
0545 - Reference Data
«entity»
ValidValueDefinition
«entity»...
name : string
description : string
category : string
usage : string
scope : string
preferredValue : string
isDeprecated : boolean
name : string...
«entity»
Referenceable
«entity»...
«entity»
Asset
«entity»...
«relationship»
ValidValuesImplementation
«relationship»...
symbolicName : string
implementationValue : string
additionalValues : map<string, string>
symbolicName : string...
*
*
*
*
validValues
validValues
validValuesImplementation
validValuesImplementation
«classification»
ReferenceData
«classification»...
«entity»
ValidValuesSet
«entity»...
«relationship»
ValidValuesAssignment
«relationship»...
strictRequirement : boolean
strictRequirement : boolean
memberOfValidValuesSet
memberOfValidVa...
validValuesSet
validValuesSet
validValuesConsumer
validValuesCons...
validValues
validValues
*
*
*
*
*
*
0..1
0..1
«relationship»
ValidValueMember
«relationship»...
isDefaultValue : boolean
isDefaultValue : boolean
matchingValue
matchingValue
matchingValue
matchingValue
«relationship»
ValidValuesMapping
«relationship»...
associationDescription : string
confidence : int
steward : string
stewardTypeName : string
stewardPropertyName : string
notes : string
associationDescription : string...
*
*
*
*
*
*
*
*
«relationship»
ReferenceValueAssignment
«relationship»...
attributeName : string
confidence : int
steward : string
stewardTypeName : string
stewardPropertyName : string
notes : string
attributeName : string...
assignedItem
assignedItem
referenceValue
referenceVal...
consistentValue
consistentVa...
consistentValue
consistentValue
«relationship»
ConsistentValidValues
«relationship»...
*
*
*
*Text is not SVG - cannot display \ No newline at end of file +
0545 - Reference Data
0545 - Reference Data
«entity»
ValidValueDefinition
«entity»...
name : string
description : string
category : string
usage : string
scope : string
preferredValue : string
isDeprecated : boolean
isCaseSensitive : boolean
name : string...
«entity»
Referenceable
«entity»...
«entity»
Asset
«entity»...
«relationship»
ValidValuesImplementation
«relationship»...
symbolicName : string
implementationValue : string
additionalValues : map<string, string>
symbolicName : string...
*
*
*
*
validValues
validValues
validValuesImplementation
validValuesImplementation
«classification»
ReferenceData
«classification»...
«entity»
ValidValuesSet
«entity»...
«relationship»
ValidValuesAssignment
«relationship»...
strictRequirement : boolean
strictRequirement : boolean
memberOfValidValuesSet
memberOfValidVa...
validValuesSet
validValuesSet
validValuesConsumer
validValuesCons...
validValues
validValues
*
*
*
*
*
*
0..1
0..1
«relationship»
ValidValueMember
«relationship»...
isDefaultValue : boolean
isDefaultValue : boolean
matchingValue
matchingValue
matchingValue
matchingValue
«relationship»
ValidValuesMapping
«relationship»...
associationDescription : string
confidence : int
steward : string
stewardTypeName : string
stewardPropertyName : string
notes : string
associationDescription : string...
*
*
*
*
*
*
*
*
«relationship»
ReferenceValueAssignment
«relationship»...
attributeName : string
confidence : int
steward : string
stewardTypeName : string
stewardPropertyName : string
notes : string
attributeName : string...
assignedItem
assignedItem
referenceValue
referenceVal...
consistentValue
consistentVa...
consistentValue
consistentValue
«relationship»
ConsistentValidValues
«relationship»...
*
*
*
*Text is not SVG - cannot display \ No newline at end of file diff --git a/site/docs/types/5/area-5-models-and-schema.drawio b/site/docs/types/5/area-5-models-and-schema.drawio index cd95e3df25..354d8c039b 100644 --- a/site/docs/types/5/area-5-models-and-schema.drawio +++ b/site/docs/types/5/area-5-models-and-schema.drawio @@ -1,4 +1,4 @@ - + @@ -2717,8 +2717,8 @@ - - + + @@ -2763,10 +2763,10 @@ - + - + @@ -2784,7 +2784,7 @@ - + @@ -2796,7 +2796,7 @@ - + @@ -2805,7 +2805,7 @@ - + @@ -2814,7 +2814,7 @@ - + @@ -2900,8 +2900,8 @@ - - + + @@ -2909,7 +2909,7 @@ - + @@ -2928,7 +2928,7 @@ - + diff --git a/site/mkdocs.yml b/site/mkdocs.yml index 1ce27ccee6..08829a44ab 100644 --- a/site/mkdocs.yml +++ b/site/mkdocs.yml @@ -700,7 +700,7 @@ nav: - External Reference: concepts/external-reference.md - Favorite Collections: concepts/favorite-things-collection.md - Feedback: concepts/feedback.md - - File Type: concepts/deployed-implementation-type.md + - File Type: concepts/file-type.md - Glossary Term: practices/common-data-definitions/anatomy-of-a-glossary.md - GUID: concepts/guid.md - Governance Action Engine: concepts/governance-action-engine.md @@ -747,6 +747,7 @@ nav: - Note Log: concepts/note-log.md - OMAG Server: concepts/omag-server.md - OMAG Server Platform: concepts/omag-server-platform.md + - OMAG Server Runtime: concepts/omag-server-runtime.md - OMAG Subsystem: concepts/omag-subsystem.md - Open Discovery Engine: concepts/open-discovery-engine.md - Open Discovery Service: concepts/open-discovery-service.md diff --git a/site/snippets/content-packs/deployed-implementation-types.md b/site/snippets/content-packs/deployed-implementation-types.md new file mode 100644 index 0000000000..17a1054d82 --- /dev/null +++ b/site/snippets/content-packs/deployed-implementation-types.md @@ -0,0 +1,20 @@ + + + + + +| Deployed Implementation Type | AssociatedTypeName | Description | +|:-------------------------------|:-----------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Data File | DataFile | A file containing externally accessible data - other fields provide information on the internal format. | +| Program File | DataFile | A file containing program logic. | +| Log File | DataFile | A file containing log records. | +| Data Folder | DataFolder | A directory (folder) that holds files representing a single data source. | +| Archive File | DataFile | A file containing an organized collection of files. | +| PostgreSQL Server | SoftwareServer | A database server running the PostgreSQL software. | +| PostgreSQL Relational Database | RelationalDatabase | A database hosted on a PostgreSQL server. | +| Apache Atlas | SoftwareServer | A data catalog for the Hadoop ecosystem. | +| Apache Kafka | SoftwareServer | An event broker supporting high speed, reliable topic based event exchange. | +| File System | DataManager | A system that manages hierarchically organized files on persistent storage. | +| OMAG Server Platform | SoftwareServerPlatform | An Open Metadata and Governance (OMAG) platform for running one to many OMAG Servers. | +| OMAG Server Runtime | SoftwareServerPlatform | An Open Metadata and Governance (OMAG) platform for running a single OMAG Server. | + diff --git a/site/snippets/content-packs/file-types.md b/site/snippets/content-packs/file-types.md new file mode 100644 index 0000000000..287fcf5b19 --- /dev/null +++ b/site/snippets/content-packs/file-types.md @@ -0,0 +1,134 @@ + + + + +File types associated with specific file names. + +| File Name | File Type Name | encoding Property | assetSubTypeName Property | Deployed Implementation Type | Description | +|------------------------|:-----------------------------------|:------------------|:--------------------------|:-----------------------------|:-----------------------------------------------------------------------------------------------------------------------------| +| manifest.mf | Java Archive Manifest File | | PropertiesFile | Data File | A text file containing a list of name-value property pairs describing the content of a Java Archive (JAR) file. | +| pom.xml | Apache Maven Build File | XML | BuildInstructionFile | Program File | A XML file containing instructions for building a module using Apache Maven. | +| build.gradle | Gradle Build File | | BuildInstructionFile | Program File | A file containing instructions for building a module using gradle. | +| settings.gradle | Gradle Properties File | | BuildInstructionFile | Program File | A file containing instructions and properties for building a module using gradle. | +| package-info.java | Java Package Description | | SourceCodeFile | Program File | A file containing a description of a Java package that is incorporated into Javadoc documentation website. | +| Dockerfile | Docker Container Build Script | | ScriptFile | Program File | A file containing instructions to build a Docker container. | +| .gitignore | Git Ignore File | | PropertiesFile | Data File | A file containing details of the files to exclude from checking to the git tools. | +| CODEOWNERS | Code Owners File | | PropertiesFile | Data File | A file listing the people responsible for maintaining the content of a git repository. | +| LICENSE | License File | | Document | Data File | A file describing the license for using the co-located content in a git repository. | +| NOTICE | License or Copyright File | | Document | Data File | A file describing the license or copyright for using the co-located content in a git repository. | +| application.properties | Spring application.properties File | | PropertiesFile | Data File | A properties file used to configure a Spring application during startup. | +| logback.xml | Logback Configuration File | XML | PropertiesFile | Data File | A properties file used to configure the Logback log file management utilities. | +| README.md | README File | Markdown | Document | Data File | A document of words and linked to diagrams in markdown format that describes the other files in the same directory (folder). | + + +File types associated with specific file extensions. + +| File Extension | File Type Name | encoding Property | assetSubTypeName Property | Deployed Implementation Type | Description | +|------------------|:-------------------------------------------------------------|:------------------|:--------------------------|:-----------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| csv | CSV File | CSV | CSVFile | Data File | A text file with comma-separated values. | +| avro | Avro File | Avro | AVROFile | Data File | A text file containing an Avro object structure. | +| jsn | JSON File | JSON | JSONFile | Data File | A text file formatted using the JSON (JavaScript Object Notation) standard. | +| json | JSON File | JSON | JSONFile | Data File | A text file formatted using the JSON (JavaScript Object Notation) standard. | +| omarchive | Open Metadata Archive File | JSON | ArchiveFile | Data File | A JSON (JavaScript Object Notation) file containing Open Metadata Types and Instances. | +| config | OMAG Server Configuration Document | JSON | PropertiesFile | Data File | A JSON (JavaScript Object Notation) file containing the configuration for an Open Metadata and Governance (OMAG) Server. | +| cohortregistry | OMAG Server Cohort Registry | JSON | PropertiesFile | Data File | A JSON (JavaScript Object Notation) file the cohort registry for an Open Metadata and Governance (OMAG) Server. | +| omalrecord | Open Metadata Audit Log Record File | JSON | LogFile | Data File | A JSON (JavaScript Object Notation) file containing an audit log record. | +| auditlog | Open Metadata Audit Log Folder | | DataFolder | Data Folder | A directory (folder) containing audit log records. | +| openlineageevent | Open Lineage Event File | JSON | LogFile | Data File | A file containing an open lineage event. | +| xml | XML File | XML | XMLFile | Data File | A text file formatted using the XML (eXtended Markup Language) standard. | +| pqt | Parquet File | Parquet | ParquetFile | Data File | A data file formatted in columns using the Apache Parquet standard. | +| parquet | Parquet File | Parquet | ParquetFile | Data File | A data file formatted in columns using the Apache Parquet standard. | +| orc | Optimized Row Columnar (ORC) File | ORC | DataFile | Data File | A data file formatted using Optimized Row Columnar (ORC) file format that provides a highly efficient way to store Hive data. | +| xlsx | Microsoft Excel Spreadsheet File | XML | SpreadsheetFile | Data File | A data file formatted in tabbed sheets of tabular data and formulas for Microsoft Excel. | +| xls | Microsoft Excel Spreadsheet File (Old Format) | | SpreadsheetFile | Data File | A data file formatted in tabbed sheets of tabular data and formulas for Microsoft Excel. | +| numbers | Apple Numbers Spreadsheet File | | SpreadsheetFile | Data File | A data file formatted in tabbed sheets of tabular data and formulas for Apple Numbers. | +| doc | Microsoft Word Document (Old Format) | | Document | Data File | A document of words and diagrams in deprecated Microsoft Word format. | +| docx | Microsoft Word Document | XML | Document | Data File | A document of words and diagrams in Microsoft Word format. | +| odt | OpenDocument File | | Document | Data File | A document of words and diagrams in OpenDocument format. | +| pages | Apple Pages Document | | Document | Data File | A document of words and diagrams in Apple Pages format. | +| drawio | Draw.io Document | XML | Document | Data File | A document of words and diagrams in XML format for the draw.io tool. | +| md | Markdown File | Markdown | Document | Data File | A document of words and linked to diagrams in markdown format. | +| txt | Text File | | Document | Data File | A free text document. | +| tex | Text File | | Document | Data File | A free text document. | +| rtf | Rich-text File | RTF | Document | Data File | A rich text document. | +| pdf | Portable Document Format (PDF) File | PDF | Document | Data File | A container of embedded text and graphics based on a standard developed by Adobe. | +| ppt | Microsoft Powerpoint Presentation File (Old Format) | | Document | Data File | A file containing a presentation for the Microsoft Powerpoint application. | +| pot | Microsoft Powerpoint Presentation Template File (Old Format) | | Document | Data File | A file containing a presentation for the Microsoft Powerpoint application. | +| pptx | Microsoft Powerpoint Presentation File | XML | Document | Data File | A file containing a presentation for the Microsoft Powerpoint application. | +| potx | Microsoft Powerpoint Presentation Template File | XML | Document | Data File | A file containing a presentation for the Microsoft Powerpoint application. | +| key | Apple Keynote Presentation File | | Document | Data File | A file containing a presentation for the Apple Keynote application. | +| key | Encryption keys File | | KeystoreFile | Data File | A file containing encrypted secrets. | +| p12 | Personal Information Exchange File | PKCS#12 | KeystoreFile | Data File | A file containing a digital certificate that uses PKCS#12 (Public Key Cryptography Standard #12) encryption. It is used as a portable format for transferring personal private keys and other sensitive information. | +| pfx | Personal Information Exchange File | PKCS#12 | KeystoreFile | Data File | A file containing a digital certificate that uses PKCS#12 (Public Key Cryptography Standard #12) encryption. It is used as a portable format for transferring personal private keys and other sensitive information. | +| jks | Java Keystore File | | KeystoreFile | Data File | A Java-specific key store containing private keys and certificates, but it cannot be used to store secret keys. | +| jceks | Java Cryptography Extension (JCE) Keystore File | | KeystoreFile | Data File | A Java-specific key store containing private keys and certificates, but it should not be used to store secret keys because it is vulnerable to brute-force cyber attacks. | +| htm | HTML Web Document | HTML | Document | Data File | A document of words and linked to diagrams in Hypertext Markup Language (HTML) format. | +| html | HTML Web Document | HTML | Document | Data File | A document of words and linked to diagrams in Hypertext Markup Language (HTML) format. | +| tif | Tag Image File Format (TIFF) | TIFF | RasterFile | Data File | An device independent image file with tag descriptions, some standard, some proprietary. | +| tiff | Tag Image File Format (TIFF) | TIFF | RasterFile | Data File | An device independent image file with tag descriptions, some standard, some proprietary. | +| tff | Tag Image File Format (TIFF) | TIFF | RasterFile | Data File | An device independent image file with tag descriptions, some standard, some proprietary. | +| gtiff | GeoTIFF File | GeoTIFF | RasterFile | Data File | A TIFF formatted file with additional geo-referencing information embedded. | +| bmp | Windows Bitmap File | | RasterFile | Data File | A device independent formatted image file with no compression (lossless). | +| jpg | JPEG File | JPEG | RasterFile | Data File | A device independent formatted image file from the Joint Photographic Experts Group (JPEG) with lossy compression used by most digital cameras. | +| jpeg | JPEG File | JPEG | RasterFile | Data File | A device independent formatted image file from the Joint Photographic Experts Group (JPEG) with lossy compression used by most digital cameras. | +| gif | Graphics Interchange Format (GIF) File | GIF | RasterFile | Data File | A device independent formatted image file with limited colour depth for graphics with limited colours for simple diagrams, icons, logos and cartoons. It uses lossless compression. | +| png | Portable Network Graphics (PNG) File | PNG | RasterFile | Data File | A device independent formatted image file with an 8-bit (256 colors) palette image and 24-bit truecolor (16 million colors) or 48-bit truecolor with and without alpha channel. It uses lossless compression. | +| heic | High-Efficiency Image Codec (HEIC) File | HEIC | RasterFile | Data File | A device independent formatted image container file following the High Efficiency Image Format (HEIF). It adds High Efficiency Video Coding (HEVC). | +| ico | Computer Icons File | ICO | RasterFile | Data File | A file containing one or more small images of different sizes and colour depths for use as icons. | +| css | Cascading Style Sheet File | CSS | ScriptFile | Program File | A descriptive file used to define the appearance of elements in a web browser user interface. | +| scss | Extended Cascading Style Sheet File | SCSS | ScriptFile | Program File | A descriptive file used to define the appearance of elements in a web browser user interface. A superset of CSS. | +| py | Python Program File | | ScriptFile | Program File | A file containing Python source code. | +| js | Javascript Program File | | ScriptFile | Program File | A file containing Javascript source code. | +| ts | Typescript Program File | | ScriptFile | Program File | A file containing Typescript source code. | +| sql | SQL Program File | | ScriptFile | Program File | A file containing a list of SQL statements used to configure or load data into a relational database. | +| bat | Microsoft DOS Batch File | | ScriptFile | Program File | A file containing a list of Microsoft DOS commands. | +| com | Microsoft DOS Command File | | ScriptFile | Program File | A file containing a list of Microsoft DOS commands. | +| wsf | Microsoft Windows Script File | | ScriptFile | Program File | A file containing a list of Microsoft Windows commands. | +| sh | Bash Shell Script File | | ScriptFile | Program File | A file containing a list of Unix commands that run in the Bash shell. | +| ksh | Korn Shell Script File | | ScriptFile | Program File | A file containing a list of Unix commands that run in the Korn shell. | +| properties | Properties File | | PropertiesFile | Data File | A file containing a list of name-value properties. | +| cfg | Configuration File | | PropertiesFile | Data File | A file containing a list of name-value properties for configuration. | +| conf | Configuration File | | PropertiesFile | Data File | A file containing a list of name-value properties for configuration. | +| log | Log File | | LogFile | Log File | A file where each row represents a log record. | +| msg | Log Message File | | LogFile | Log File | A file where each row represents a log record. | +| err | Stderr Log File | | LogFile | Log File | A file where each row represents a string output on the standard error (stderr) destination of a running process. | +| out | Stdout Log File | | LogFile | Log File | A file where each row represents a string output on the standard out (stdout) destination of a running process. | +| java | Java Program Source File | | SourceCodeFile | Program File | A file containing Java programming language source code and comments. | +| class | Java Program Object File | | ExecutableFile | Program File | A file containing complied Java code. | +| app | MacOS Application Bundle File | | ExecutableFile | Program File | A file containing executable application code for the Apple MacOS operating system. | +| exe | Windows Executable File | | ExecutableFile | Program File | A file containing executable application code for the Microsoft Windows operating system. | +| jar | Java Archive File | | ArchiveFile | Program File | A file containing an organized collection of executable Java code and related resources. | +| tar | Tarball Archive File | | ArchiveFile | Archive File | A file containing an organized collection of files. Often used for install. | +| gz | Compressed Archive File | | ArchiveFile | Archive File | A file containing an organized collection of files that has been compressed to save storage. | +| zip | Zipped Archive File | | ArchiveFile | Archive File | A file containing an organized collection of files that have been compressed to save storage. | +| war | Web Archive File | | ArchiveFile | Program File | A file containing an organized collection of files that represent a web application. | +| ear | Enterprise Archive File | | ArchiveFile | Program File | A file containing an organized collection of files that represent a JEE application. | +| dmg | Apple Disk Image File | | ArchiveFile | Program File | A file containing an organized collection of files that represent an application for Apple MacOS. | +| iso | Windows Disk Image File | | ArchiveFile | Program File | A file containing an organized collection of files that represent an application for Microsoft Windows. | +| yml | YAML File | YAML | YAMLFile | Data File | A file containing a set of properties in the YAML format. | +| yaml | YAML File | YAML | YAMLFile | Data File | A file containing a set of properties in the YAML format. | +| bin | Binary Format File | Binary | DataFile | Data File | A file containing binary data. | +| dat | Application Data File | | DataFile | Data File | A file containing data, typically from an application. The format is known by the application. | +| rpt | Application Report File | | DataFile | Data File | A file containing report data, typically from an application. The format is known by the application. | +| mpp | Microsoft Project File | | DataFile | Data File | A file containing data from Microsoft Project. | +| vcf | Variant Call Format (VCF) File | VCF | DataFile | Data File | A text file format used in bioinformatics for storing gene sequence variations. | +| aif | Audio Interchange File Format (AIFF) File | AIFF | AudioFile | Data File | A file containing audio samples and music. Often used for storing uncompressed, high quality audio on CDs and personal computers. | +| aiff | Audio Interchange File Format (AIFF) File | AIFF | AudioFile | Data File | A file containing audio samples and music. Often used for storing uncompressed, high quality audio on CDs and personal computers. | +| flac | Free Lossless Audio Codec (FLAC) File | FLAC | AudioFile | Data File | A file containing lossless compression audio coding format. | +| wav | Waveform Audio File | WAV | AudioFile | Data File | A file containing lossless compression audio coding format. | +| mp3 | MP3 Audio File | MP3 | AudioFile | Data File | A file containing audio data with Moving Pictures Experts Group (MPEG) compression. | +| wma | Window Media Audio File | WMA | AudioFile | Data File | A file containing audio data with Microsoft's specific compression. | +| mp4 | MP4 Video File | MP4 (H.264) | VideoFile | Data File | A file containing video data with Moving Pictures Experts Group (MPEG) compression. | +| ts | Video Transport System File | | VideoFile | Data File | A file containing MPEG-2-compressed video data. | +| 3gp | Mobile Video File | | VideoFile | Data File | A file containing video data in small file sizes for mobile phones and other devices. | +| 3g2 | Mobile Video File | | VideoFile | Data File | A file containing video data in small file sizes for mobile phones and other devices. | +| mov | Apple MOV Video File | | VideoFile | Data File | A file containing high quality video data. | +| avi | Windows AVI Video File | | VideoFile | Data File | A file containing video data for Microsoft Windows. | +| m4v | Apple M4V Video File | M4V | VideoFile | Data File | A file containing video data for Apple systems, often includes Apple’s FairPlay DRM copyright protection. | +| wmv | Windows Media Video File | WMV | VideoFile | Data File | A file containing video data for Microsoft Windows Media Player. | +| ai | Adobe Illustrator Image File | | VectorFile | Data File | A file containing vector image data for Adobe Illustrator. | +| svg | Scalar Vector Graphics File | SVG | VectorFile | Data File | A file containing vector image data. | +| eps | Encapsulated Postscript File | EPS | VectorFile | Data File | A file containing high quality vector image data. Used in the printing industry. | +| ps | Postscript File | PS | VectorFile | Data File | A file containing an image formatted in the PostScript language. | +| obj | Wavefront 3D Object File | | 3DImageFile | Data File | A file containing a 3 dimensional object. | +| mtl | OBJ Material Template Library File | | 3DImageFile | Data File | A file that describes how a program should apply textures to that object. MTL files contain the names of the texture bitmap files to apply to an object as well as the 3D coordinates at which to apply them. |