Update Deployment Docs (#12612)

* Add changes for kubernetes deployment docs * Update Quickstart docker guide * fix: docker quickstart docs * Add troubleshooting guide for quickstart k8s * fix typo * upper case the placeholders * Add FAQs and update EKS deployment docs * fix: docs links * Update Deployment page * fix: update k8s gke docs * fix(chore): Notes as warning type * fix(chore): markdown link and typo * fix(refactor): Docker Deployment Docs * fix(refactor): Docker Deployment Docs * fix: upgrade k8s docs * fix(reviews): remove apache mod_proxy * fix(refactor): docker volumes docs section * fix(refactor): Code Review changes * Merge remote-tracking branch 'origin/main' into 12608 * Add troubleshoot docs for psql create extension pgcypto issue * fix: k8s docs with elasticsearch auth enabled * fix: Update the changes to v1.1.1 docs * fix: updated the docs for 1.1.1 * Add docker and kubernetes v1.1.1 changes in docs * fix: Docker Deployment Link * fix: more fixes --------- Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
open-metadata · Aug 10, 2023 · 60ff64c · 60ff64c
1 parent 62c3a9e
commit 60ff64c
Show file tree

Hide file tree

Showing 49 changed files with 2,717 additions and 2,205 deletions.
diff --git a/...ials/v1.1.0/deployment/configure-external-orchestrator-for-ingestion-service.md b/...ials/v1.1.0/deployment/configure-external-orchestrator-for-ingestion-service.md
@@ -0,0 +1,5 @@
+### Configure External Orchestrator Service (Ingestion Service)
+
+OpenMetadata requires connectors to be scheduled to periodically fetch the metadata or you can use the OpenMetadata APIs to push the metadata as well
+1. OpenMetadata Ingestion Framework is flexible to run on any orchestrator. However we built an ability to deploy and manage connectors as pipelines from the UI. This requires the Airflow container we ship. However, it is recommended to 
+2. If your team prefers to run on any other orchestrator such as prefect, dagster or even github workflows. Please refer to our recent webinar on [How Ingestion Framework works](https://www.youtube.com/watch?v=i7DhG_gZMmE&list=PLa1l-WDhLreslIS_96s_DT_KdcDyU_Itv&index=10)
diff --git a/...metadata-docs/content/partials/v1.1.0/deployment/minimum-sizing-requirements.md b/...metadata-docs/content/partials/v1.1.0/deployment/minimum-sizing-requirements.md
@@ -0,0 +1,9 @@
+## Minimum Sizing Requirements
+
+We recommend you to allocate openmetadata-server with minimum of 2vCPUs and 6 GiB Memory.
+
+For External Services that openmetadata depends on -
+- For the database, minimum 2 vCPUs and 2 GiB RAM (per instance) with 30 GiB of Storage Volume Attached (dynamic expansion up to 100 GiB)
+- For Elasticsearch, minimum 2 vCPUs and 2 GiB RAM (per instance) with 30 GiB of Storage volume attached
+
+These settings apply as well when using managed instances, such as AWS RDS or GCP CloudSQL or AWS OpenSearch.
diff --git a/...ials/v1.1.0/deployment/postgresql-issue-permission-denied-extension-pgcrypto.md b/...ials/v1.1.0/deployment/postgresql-issue-permission-denied-extension-pgcrypto.md
@@ -0,0 +1,19 @@
+If you are facing the below issue with PostgreSQL as Database Backend for OpenMetadata Application,
+
+```
+Message: ERROR: permission denied to create extension "pgcrypto"
+Hint: Must be superuser to create this extension.
+```
+
+It seems the Database User does not have sufficient privileges. In order to resolve the above issue, grant usage permissions to the PSQL User.
+
+```sql
+GRANT USAGE ON SCHEMA schema_name TO <openmetadata_psql_user>;
+GRANT CREATE ON EXTENSION pgcrypto TO <openmetadata_psql_user>;
+```
+
+{%note%}
+
+In the above command, replace `<openmetadata_psql_user>` with the sql user used by OpenMetadata Application to connect to PostgreSQL Database.
+
+{%\note%}
diff --git a/...ials/v1.1.1/deployment/configure-external-orchestrator-for-ingestion-service.md b/...ials/v1.1.1/deployment/configure-external-orchestrator-for-ingestion-service.md
@@ -0,0 +1,5 @@
+### Configure External Orchestrator Service (Ingestion Service)
+
+OpenMetadata requires connectors to be scheduled to periodically fetch the metadata or you can use the OpenMetadata APIs to push the metadata as well
+1. OpenMetadata Ingestion Framework is flexible to run on any orchestrator. However we built an ability to deploy and manage connectors as pipelines from the UI. This requires the Airflow container we ship. However, it is recommended to 
+2. If your team prefers to run on any other orchestrator such as prefect, dagster or even github workflows. Please refer to our recent webinar on [How Ingestion Framework works](https://www.youtube.com/watch?v=i7DhG_gZMmE&list=PLa1l-WDhLreslIS_96s_DT_KdcDyU_Itv&index=10)
diff --git a/...metadata-docs/content/partials/v1.1.1/deployment/minimum-sizing-requirements.md b/...metadata-docs/content/partials/v1.1.1/deployment/minimum-sizing-requirements.md
@@ -0,0 +1,9 @@
+## Minimum Sizing Requirements
+
+We recommend you to allocate openmetadata-server with minimum of 2vCPUs and 6 GiB Memory.
+
+For External Services that openmetadata depends on -
+- For the database, minimum 2 vCPUs and 2 GiB RAM (per instance) with 30 GiB of Storage Volume Attached (dynamic expansion up to 100 GiB)
+- For Elasticsearch, minimum 2 vCPUs and 2 GiB RAM (per instance) with 30 GiB of Storage volume attached
+
+These settings apply as well when using managed instances, such as AWS RDS or GCP CloudSQL or AWS OpenSearch.
diff --git a/...ials/v1.1.1/deployment/postgresql-issue-permission-denied-extension-pgcrypto.md b/...ials/v1.1.1/deployment/postgresql-issue-permission-denied-extension-pgcrypto.md
@@ -0,0 +1,19 @@
+If you are facing the below issue with PostgreSQL as Database Backend for OpenMetadata Application,
+
+```
+Message: ERROR: permission denied to create extension "pgcrypto"
+Hint: Must be superuser to create this extension.
+```
+
+It seems the Database User does not have sufficient privileges. In order to resolve the above issue, grant usage permissions to the PSQL User.
+
+```sql
+GRANT USAGE ON SCHEMA schema_name TO <openmetadata_psql_user>;
+GRANT CREATE ON EXTENSION pgcrypto TO <openmetadata_psql_user>;
+```
+
+{%note%}
+
+In the above command, replace `<openmetadata_psql_user>` with the sql user used by OpenMetadata Application to connect to PostgreSQL Database.
+
+{%\note%}
diff --git a/openmetadata-docs/content/v1.1.0/deployment/bare-metal/index.md b/openmetadata-docs/content/v1.1.0/deployment/bare-metal/index.md
@@ -67,13 +67,7 @@ If you are using AWS OpenSearch Service, OpenMetadata Supports AWS OpenSearch Se
 OpenMetadata performs metadata ingestion using the Ingestion Framework. Learn more about how to deploy and manage
 the ingestion workflows [here](/deployment/ingestion).
 
-## Minimum Sizing Requirements
-
-- Our minimum specs recommendation for the OpenMetadata Deployment (one replica) is 2 vCPUs and 4 Gigs with 20 Gigs of volume size if using persistent volumes for logs.
-- For Elasticsearch, 2 vCPUs and 2 Gigs RAM (per instance) with 30 Gigs of Storage volume attached.
-- For the database, 2 vCPUs and 2 Gigs RAM (per instance) with 30 Gigs of Storage Volume Attached (dynamic expansion up to 100 Gigs).
-
-These settings apply as well when using managed instances, such as RDS or AWS OpenSearch.
+{% partial file="/v1.1.0/deployment/minimum-sizing-requirements.md" /%}
 
 # Procedure
 
@@ -129,25 +123,6 @@ We recommend configuring `serviced` to monitor the OpenMetadata command to resta
 You may put one or more OpenMetadata instances behind a load balancer for reverse proxying.
 To do this you will need to add one or more entries to the configuration file for your reverse proxy.
 
-### Apache mod_proxy
-
-To use the Apache mod_proxy module as a reverse proxy for load balancing, update the VirtualHost tag in your
-Apache config file to resemble the following.
-
-```xml
-<VirtualHost *:80>
-    <Proxy balancer://mycluster>
-        BalancerMember http://127.0.0.1:8585 <!-- First OpenMetadata server -->
-        BalancerMember http://127.0.0.2:8686 <!-- Second OpenMetadata server -->
-    </Proxy>
-
-    ProxyPreserveHost On
-
-    ProxyPass / balancer://mycluster/
-    ProxyPassReverse / balancer://mycluster/
-</VirtualHost>
-```
-
 ### Nginx
 
 To use OpenMetadata behind an Nginx reverse proxy, add an entry resembling the following the http context of your Nginx
@@ -251,6 +226,10 @@ The flag `Xmx` specifies the maximum memory allocation pool for a Java virtual m
 
 Restart the OpenMetadata Application using `./bin/openmetadata.sh start` which will start the service using a linux process.
 
+# PostgreSQL Issue permission denied to create extension "pgcrypto"
+
+{% partial file="/v1.1.0/deployment/postgresql-issue-permission-denied-extension-pgcrypto.md" /%}
+
 ## Enable Security
 
 Please follow our [Enable Security Guide](/deployment/bare-metal/security) to configure security for your OpenMetadata