Skip to content

Commit

Permalink
Merge branch 'main' into feature-import-export-custom-property
Browse files Browse the repository at this point in the history
  • Loading branch information
Ashish8689 committed Sep 30, 2024
2 parents addac1a + 9219dbe commit 9a9599b
Show file tree
Hide file tree
Showing 42 changed files with 901 additions and 649 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: Database service setup
slug: /how-to-guides/getting-started/day-1/database-service-setup
collate: true
---

## Setting Up a Database Service for Metadata Extraction

You can quickly set up a database service for metadata extraction in OpenMetadata SaaS. Below is an example of how to configure a connection using the `Snowflake` Connector:

1. ### Log in to OpenMetadata SaaS
- Navigate to **Settings > Services > Databases**.
- Click on **Add New Service**.

{% image
src="/images/v1.5/getting-started/add-service.png"
alt="Adding Database Service"
caption="Adding Database Service" /%}

2. ### Select Database Type

- Choose the database type for your service. In this example, select `Snowflake`.
- Enter the required details, such as the **Name** and **Description**, to identify the database.

{% image
src="/images/v1.5/getting-started/select-service.png"
alt="Selecting Database Service"
caption="Selecting Database Service" /%}

3. ### Enter Connection Details
- Provide the necessary connection parameters, such as hostname, port, credentials, etc.
- The side panel offers guidance with available documentation, and you can also refer to the specific `Snowflake` Connector [documentation](/connectors)for more information.

{% image
src="/images/v1.5/getting-started/configure-connector.png"
alt="Updating Connection Details"
caption="Updating Connection Details" /%}

4. ### Test the Connection
Click Test Connection to verify the setup. This will check if OpenMetadata can reach the Snowflake service.

{% image
src="/images/v1.5/getting-started/test-connection.png"
alt="Verifying the Test Connection"
caption="Verifying the Test Connection" /%}
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
---
title: Day 1
slug: /how-to-guides/getting-started/day-1
collate: true
---

# Getting Started: Day 1

Get started with your OpenMetadata service in a few simple steps:

1. Set up a Data Connector: Connect your data sources to begin collecting metadata.
2. Ingest Metadata: Run the metadata ingestion process to gather and push data insights.
3. Invite Users: Add team members to collaborate and manage metadata together.
4. Explore the Features: Dive into OpenMetadata's extensive feature set to unlock the full potential of your data.

**Ready to begin? Let's get started!**

## Requirements

You should receive your initial OpenMetadata credentials from OpenMetadata support, or from your existing OpenMetadata admin. For any questions, please reach us via [slack](https://slack.open-metadata.org/). The following steps will provide initial set up information, with links to more detailed documentation.

## Step 1: Set up a Data Connector

Once you have logged into your OpenMetadata instance, set up a data connector to start ingesting metadata. OpenMetadata provides [80+ turnkey connectors](/connectors) for a wide range of services, including:

- Databases
- Dashboards
- Messaging services
- Pipelines
- ML models
- Storage services
- Other metadata services

For [custom data sources](/connectors/custom-connectors), metadata ingestion can also be set up via API.

There's two options on how to set up a data connector:
1. **Run the connector in OpenMetadata**: In this scenario, an IP will be provided when you add the service. You must allow access to this IP in your data sources.

{% tilesContainer %}
{% tile
title="Run the connector in OpenMetadata"
description="Guide to start ingesting metadata seamlessly from your data sources."
link="/how-to-guides/getting-started/day-1/database-service-setup"
icon="discovery"
/%}
{% /tilesContainer %}

2. **Run the connector in your infrastructure or on a local machine:**: This hybrid model allows organizations to run metadata ingestion components within their own infrastructure. This ensures that OpenMetadata's managed service doesn't need direct access to the underlying data. Only metadata is collected locally and securely transmitted to the platform, maintaining data privacy and security.

## Step 2: Ingest Metadata

Once the connector is set up, configure a [metadata ingestion pipeline](/how-to-guides/admin-guide/how-to-ingest-metadata)
to import metadata into OpenMetadata on a regular schedule.

- Navigate to **Settings > Services > Databases** and select the service you added.
- Go to the **Ingestion** tab and click **Add Metadata Ingestion**.

{% image
src="/images/v1.5/getting-started/add-ingestion.png"
alt="Adding Ingestion"
caption="Adding Ingestion" /%}

- Configure any required settings or filters for the ingestion process. Documentation is available in the side panel for reference.

{% image
src="/images/v1.5/getting-started/ingestion-config.png"
alt="Configure Ingestion"
caption="Configure Ingestion" /%}

- Schedule the pipeline to ingest metadata at regular intervals.

{% image
src="/images/v1.5/getting-started/schedule-ingesgtion.png"
alt="Schedule Ingestion"
caption="Schedule Ingestion" /%}

- In addition to metadata ingestion, you can set up pipelines for lineage, profiler data, or dbt information.
- Once the metadata ingestion is completed, the ingested data assets can be viewed under the **Explore** section in the main menu.

{% image
src="/images/v1.5/getting-started/explore-tab.png"
alt="Ingested Data"
caption="Ingested Data under Explore Tab" /%}

- You can repeat these steps to ingest metadata from other data sources as needed.

## Step 3: Invite Users

After metadata has been ingested into OpenMetadata, you can [invite users](/how-to-guides/admin-guide/teams-and-users/invite-users) to collaborate on the data and assign different roles.

- Go to **Settings > Team & User Management > Users**.

{% image
src="/images/v1.5/getting-started/users.png"
alt="Users Navigation"
caption="Users Navigation" /%}

- Click **Add User**, then enter their email and other required details to grant access to the platform.

{% image
src="/images/v1.5/getting-started/add-users.png"
alt="Adding New User"
height="750px"
caption="Adding New User" /%}

- Organize users into different Teams, and assign them specific Roles.
- Users inherit access permissions from their assigned teams and roles.
- Admin access can be granted to users who need full access to all settings, including the ability to invite new users.

{% image
src="/images/v1.5/getting-started/update-user.png"
alt="Users Profile"
caption="Users Profile" /%}

- New users will receive an email invitation to set up their accounts.

## Step 4: Explore Features of OpenMetadata

OpenMetadata offers a comprehensive platform for data teams to:
- Break down data silos
- Securely share data assets across various sources
- Foster collaboration around trusted data
- Establish a documentation-first data culture within your organization

Explore these features and unlock the full potential of your data using OpenMetadata.

{% tilesContainer %}
{% tile
title="Data Discovery"
description="Discover the right data assets to make timely business decisions."
link="/how-to-guides/data-discovery"
icon="discovery"
/%}
{% tile
title="Data Collaboration"
description="Foster data team collaboration to enhance data understanding."
link="/how-to-guides/data-collaboration"
icon="collaboration"
/%}
{% tile
title="Data Quality & Observability"
description="Trust your data with quality tests & monitor the health of your data systems."
link="/how-to-guides/data-quality-observability"
icon="observability"
/%}
{% tile
title="Data Lineage"
description="Trace the path of data across tables, pipelines, and dashboards."
link="/how-to-guides/data-lineage"
icon="lineage"
/%}
{% tile
title="Data Insights"
description="Define KPIs and set goals to proactively hone the data culture of your company."
link="/how-to-guides/data-insights"
icon="discovery"
/%}
{% tile
title="Data Governance"
description="Enhance your data platform governance using OpenMetadata."
link="/how-to-guides/data-governance"
icon="governance"
/%}
{% /tilesContainer %}

## Deep Dive into OpenMetadata: Guides for Admins and Data Users

{% tilesContainer %}
{% tile
title="Admin Guide"
description="Admin users can get started with OpenMetadata with just three quick and easy steps & know-it-all with the advanced guides."
link="/how-to-guides/admin-guide"
icon="administration"
/%}
{% tile
title="Guide for Data Users"
description="Get to know the basics of OpenMetadata and about the data assets that you can explore in the all-in-one platform."
link="/how-to-guides/guide-for-data-users"
icon="steward"
/%}
{% /tilesContainer %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
title: Getting Started with OpenMetadata for Data cataloging
slug: /how-to-guides/getting-started
---

# Getting Started

Welcome to OpenMetadata's unified platform for data discovery, observability, and governance. Our platform centralizes all data context to help you build high-quality data and AI assets. This guide provides the necessary information to set up your OpenMetadata environment in 30 minutes.

## How Does Collate Work?

OpenMetadata is designed to support both technical and non-technical data practitioners across various use cases, including data discovery, lineage, observability, quality, collaboration, governance, and insights.

The platform includes a library of 80+ turnkey connectors to easily ingest metadata from sources such as data warehouses, data lakes, streaming platforms, dashboards, and ML models. For custom data sources, APIs are available to streamline metadata ingestion. Metadata from these sources is organized into a Unified Metadata Graph, providing a single, comprehensive source of truth for your entire data estate.

This centralized metadata is accessible through a unified user interface, eliminating the need for practitioners to switch between multiple catalogs, quality, or governance tools. OpenMetadata can also be extended with applications, such as AI-driven productivity tools like MetaPilot, or through custom-built workflows that integrate the platform with existing systems.
The platform’s native collaboration features support shared workflows, enabling different teams—data platform engineers, governance professionals, data scientists/analysts, and business users—to collaborate effectively in a single environment.

## Key Features of OpenMetadata

Before we get started, here’s a quick summary of some of Collate’s main features:

### Discovery
- Integrated catalog, data quality, and glossary
- Natural language search, filtering, and faceting
- 80+ turnkey data connectors

### Lineage
- Table and column-level lineage
- Automated data estate mapping with APIs
- Lineage layers, search capabilities
- Governance and PII automation, with manual customization

### Observability
- Alerting and notifications
- Incident management and third-party notifications
- Pipeline monitoring, root cause analysis, and anomaly detection
- Data profiler for performance insights

### Quality
- Table and column-level test cases
- No-code and SQL-based data quality tests
- Test suites, reporting, and dashboards
- Quality lineage maps and widgets for data insights

### Collaboration
- Announcements, tasks, and team conversations
- Slack/Teams integration for communication
- Activity feed and team dashboards for tracking progress

### Governance
- Business glossary and classification tags
- Automated PII classification and description generation

### Insights
- Data asset analytics and app usage metrics
- Coverage KPIs and ownership dashboards
- Data health and governance reports

## Onboarding Journey

In this section, you will find a series of guides to help you get started with OpenMetadata. These guides cover both basic and advanced features of the platform, organized in a step-by-step format and divided into milestones that can be achieved over the course of your onboarding process.

{% tilesContainer %}
{% tile
title="Day 1: Connect your Data Sources and invite users"
description="Discover the right data assets to make timely business decisions."
link="/how-to-guides/getting-started/day-1"
icon="discovery"
/%}
{% /tilesContainer %}
6 changes: 6 additions & 0 deletions openmetadata-docs/content/v1.5.x/menu.md
Original file line number Diff line number Diff line change
Expand Up @@ -697,6 +697,12 @@ site_menu:
color: violet-70
icon: openmetadata

- category: How-to Guides / Getting Started
url: /how-to-guides/getting-started
- category: How-to Guides / Day 1
url: /how-to-guides/getting-started/day-1
- category: How-to Guides / Day 1 / Database Service Setup
url: /how-to-guides/getting-started/day-1/database-service-setup
- category: How-to Guides / Admin Guide
url: /how-to-guides/admin-guide
- category: How-to Guides / Admin Guide / How to Ingest Metadata
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: Database service setup
slug: /how-to-guides/getting-started/day-1/database-service-setup
collate: true
---

## Setting Up a Database Service for Metadata Extraction

You can quickly set up a database service for metadata extraction in OpenMetadata SaaS. Below is an example of how to configure a connection using the `Snowflake` Connector:

1. ### Log in to OpenMetadata SaaS
- Navigate to **Settings > Services > Databases**.
- Click on **Add New Service**.

{% image
src="/images/v1.6/getting-started/add-service.png"
alt="Adding Database Service"
caption="Adding Database Service" /%}

2. ### Select Database Type

- Choose the database type for your service. In this example, select `Snowflake`.
- Enter the required details, such as the **Name** and **Description**, to identify the database.

{% image
src="/images/v1.6/getting-started/select-service.png"
alt="Selecting Database Service"
caption="Selecting Database Service" /%}

3. ### Enter Connection Details
- Provide the necessary connection parameters, such as hostname, port, credentials, etc.
- The side panel offers guidance with available documentation, and you can also refer to the specific `Snowflake` Connector [documentation](/connectors)for more information.

{% image
src="/images/v1.6/getting-started/configure-connector.png"
alt="Updating Connection Details"
caption="Updating Connection Details" /%}

4. ### Test the Connection
Click Test Connection to verify the setup. This will check if OpenMetadata can reach the Snowflake service.

{% image
src="/images/v1.6/getting-started/test-connection.png"
alt="Verifying the Test Connection"
caption="Verifying the Test Connection" /%}
Loading

0 comments on commit 9a9599b

Please sign in to comment.