Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Install and configure pgbackrest on patroni cluster" #3257

Merged
merged 1 commit into from
Nov 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions openshift/patroni-postgis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,12 @@ The WPS pipeline currently assumes the existence of an appropriately tagged patr
### Build and tag an imagestream as follows:

```bash
# Build a patroni-pgbackrest imagestream:
oc -n e1e498-tools process -f openshift/build.yaml -p OBJECT_STORE_SERVER="server" -p OBJECT_STORE_BUCKET="bucket" -p OBJECT_STORE_USER_ID="uid" -p OBJECT_STORE_SECRET="sec" -p GIT_REF="<branch>" | oc -n e1e498-tools apply -f -
# Build a patroni imagestream:
oc -n e1e498-tools process -f openshift/build.yaml | oc -n e1e498-tools apply -f -
# Tag the old imagestream so we can keep it around if we need to revert:
oc -n e1e498-tools tag patroni-postgres-pgbackrest:v12-<current-tag, e.g. 2023-11-22> patroni-postgres-pgbackrest:v12-<date deprecated, e.g. 20200826>
oc -n e1e498-tools tag patroni-postgres:v12 patroni-postgres:v12-<date deprecated, e.g. 20200826>
# Tag the new imagestream (it won't be used until the pods get re-created):
oc -n e1e498-tools tag patroni-postgres-pgbackrest:v12-latest patroni-postgres-pgbackrest:v12-<current date, e.g. 2024-1-1>
# Then update the tag in the patroni yaml files to point to the current date tag
oc -n e1e498-tools tag patroni-postgres:v12-latest patroni-postgres:v12
```

#### Common build failures
Expand Down Expand Up @@ -70,4 +69,4 @@ SELECT postgis_extensions_upgrade();

This script cannot be run as part of the alembic scripts in the api project, the api uses a `wps` user that does not have the appropriate priveleges. Upgrading the image may this require manually running `SELECT postgis_extensions_upgrade();` on the target database once the new image has been applied.

The PostGIS version can be checked with `SELECT postgis_full_version();`
The PostGIS version can be checked with `SELECT postgis_full_version();`
39 changes: 4 additions & 35 deletions openshift/patroni-postgis/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,54 +3,23 @@
FROM image-registry.openshift-image-registry.svc:5000/bcgov/patroni-postgres:12.4-latest

ENV POSTGIS_MAJOR 3
ENV POSTGIS_VERSION 3.3.4+dfsg-1.pgdg100+1
ENV POSTGIS_VERSION 3.3.1+dfsg-1.pgdg100+1

# Switch to root user for package installs
USER 0

# PostGIS requirements taken from https://github.com/postgis/docker-postgis
RUN apt-get update \
&& apt-cache showpkg postgresql-"$PG_MAJOR"-postgis-"$POSTGIS_MAJOR" \
&& apt-cache showpkg postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR \
&& apt-get install -y --no-install-recommends \
# ca-certificates: for accessing remote raster files;
# fix: https://github.com/postgis/docker-postgis/issues/307
ca-certificates \
\
postgresql-"$PG_MAJOR"-postgis-"$POSTGIS_MAJOR"="$POSTGIS_VERSION" \
postgresql-"$PG_MAJOR"-postgis-"$POSTGIS_MAJOR"-scripts \
pgbackrest \
postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR=$POSTGIS_VERSION \
postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR-scripts=$POSTGIS_VERSION \
&& rm -rf /var/lib/apt/lists/*

RUN mkdir -p -m 777 /var/log/pgbackrest \
&& chown postgres:postgres /var/log/pgbackrest \
&& mkdir -p /etc/pgbackrest \
&& mkdir -p /etc/pgbackrest/conf.d \
&& touch /etc/pgbackrest/pgbackrest.conf \
&& chmod 777 /etc/pgbackrest/pgbackrest.conf \
&& chown postgres:postgres /etc/pgbackrest/pgbackrest.conf

# Add the POSTGIS command to the end of the post_init script.
COPY init_postgis /usr/share/scripts/patroni/
RUN cat /usr/share/scripts/patroni/init_postgis >> /usr/share/scripts/patroni/post_init.sh
RUN rm /usr/share/scripts/patroni/init_postgis

RUN echo "[global] \n \
repo1-type=s3 \n \
repo1-storage-verify-tls=n \n \
repo1-s3-endpoint=$OBJECT_STORE_SERVER \n\
repo1-s3-uri-style=path \n\
repo1-s3-bucket=$OBJECT_STORE_BUCKET \n\
repo1-s3-key=$OBJECT_STORE_USER_ID \n \
repo1-s3-key-secret=$OBJECT_STORE_SECRET \n \
repo1-s3-region=eu-west-3 \n \
\n \
repo1-path=/repo1 \n \
repo1-retention-full=3 \n \
start-fast=y \n \
log-level-console=info \n \
log-level-file=debug \n \
delta=y \n \
process-max=2 \n" >> /etc/pgbackrest/pgbackrest.conf

# Switch back to default user
USER 1001
11 changes: 2 additions & 9 deletions openshift/patroni-postgis/docker/init_postgis
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@

if [[ (! -z "$APP_USER") && (! -z "$APP_PASSWORD") && (! -z "$APP_DATABASE") && (! -z "$NAME")]]; then
echo "Initializing archive mode for pgbackrest"
psql -c "alter system set archive_mode=on;"
psql -c "alter system set archive_command=pgbackrest --stanza=$NAME-cluster-1 archive-push '%p';"
if [[ (! -z "$APP_USER") && (! -z "$APP_PASSWORD") && (! -z "$APP_DATABASE")]]; then
echo "Loading PostGIS extensions into $APP_DATABASE"
psql "$1" -w -c "\c ${APP_DATABASE}" -c "CREATE EXTENSION IF NOT EXISTS postgis;"
echo "Restarting postgres to init config"
pg_ctl restart -D /home/postgres/pgdata/pgroot/data
echo "Creating pgbackrest stanza"
pgbackrest --stanza="$NAME-cluster-1" --pg1-path=/home/postgres/pgdata/pgroot/data --pg1-port=5432 stanza-create
fi
fi
23 changes: 1 addition & 22 deletions openshift/patroni-postgis/openshift/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ labels:
app.kubernetes.io/version: "12"
parameters:
- name: NAME
value: patroni-postgres-pgbackrest
value: patroni-postgres
- name: SUFFIX
- name: VERSION
description: Output version
Expand All @@ -23,18 +23,6 @@ parameters:
value: main
- name: POSTGRES_VERSION
value: "12"
- name: OBJECT_STORE_SERVER
description: Backup object store server
required: true
- name: OBJECT_STORE_BUCKET
description: Backup object store bucket
required: true
- name: OBJECT_STORE_USER_ID
description: Backup object store user id
required: true
- name: OBJECT_STORE_SECRET
description: Backup object store secret
required: true
objects:
- apiVersion: v1
kind: ImageStream
Expand Down Expand Up @@ -67,15 +55,6 @@ objects:
type: Git
strategy:
dockerStrategy:
env:
- name: "OBJECT_STORE_SERVER"
value: "${OBJECT_STORE_SERVER}"
- name: "OBJECT_STORE_BUCKET"
value: "${OBJECT_STORE_BUCKET}"
- name: "OBJECT_STORE_USER_ID"
value: "${OBJECT_STORE_USER_ID}"
- name: "OBJECT_STORE_SECRET"
value: "${OBJECT_STORE_SECRET}"
type: Docker
triggers:
- type: ConfigChange
Expand Down
6 changes: 2 additions & 4 deletions openshift/templates/patroni.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -162,8 +162,6 @@ objects:
key: app-db-password
- name: APP_DATABASE
value: ${PARENT_NAME}
- name: NAME
value: ${NAME}
- name: PATRONI_SCOPE
value: ${NAME}
- name: PATRONI_NAME
Expand Down Expand Up @@ -289,12 +287,12 @@ parameters:
- name: IMAGE_NAME
description: |
The Patroni image stream name
value: patroni-postgres-pgbackrest
value: patroni-postgres
- name: IMAGE_TAG
description: |
The image tag used to specify which image you would like deployed.
Don't use `latest`.
value: "v12-2023-11-22"
value: "v12-2022-09-12"
- name: PVC_SIZE
description: The size of the persistent volume to create.
displayName: Persistent Volume Size
Expand Down
4 changes: 2 additions & 2 deletions openshift/templates/patroni_prerequisite.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -179,12 +179,12 @@ parameters:
- name: IMAGE_NAME
description: |
The Patroni image stream name
value: patroni-postgres-pgbackrest
value: patroni-postgres
- name: IMAGE_TAG
description: |
The image tag used to specify which image you would like deployed.
Don't use `latest`.
value: "v12-2023-11-22"
value: "v12"
- name: PVC_SIZE
description: The size of the persistent volume to create.
displayName: Persistent Volume Size
Expand Down
Loading