Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/bump node 9.1 dbsync 13.3 #879

Merged
merged 30 commits into from
Aug 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
29d2c80
chore: updated compose file and added preprod config files
Kammerlo Jul 13, 2024
1ceaa56
chore: updated view creation for db-sync 13.3
Kammerlo Jul 14, 2024
449ccc5
chore: added config files for networks
Kammerlo Jul 14, 2024
e414b71
Added slot point so that ogmios does not start from origin, instead s…
Kartiiyer12 Jul 16, 2024
840caf1
chore: added config files for networks
Kammerlo Jul 26, 2024
52d0521
chore: fixed sonar lint issues
Kammerlo Jul 26, 2024
a02b60a
chore: moved chainsyncpoint to background so that it can be configure…
Kammerlo Jul 29, 2024
be4d1cf
chore: moved chainsyncpoint to background so that it can be configure…
Kammerlo Jul 30, 2024
2e61f3c
chore: removed unused cachedfrom statements
Kammerlo Jul 30, 2024
778a786
chore: fixed docker build format warnings
Kammerlo Jul 30, 2024
2a1eb80
chore: removed unsued console log
Kammerlo Jul 30, 2024
ac8f7d4
chore: splitted cardano-node-ogmios container into two seperate ones …
Kammerlo Jul 30, 2024
ec8ee69
chore: adjusted ogmios server name default value in dockerfile
Kammerlo Jul 31, 2024
463f080
chore: Postgres port now adjustable
Kammerlo Jul 31, 2024
365106f
chore: Postgres port now adjustable
Kammerlo Jul 31, 2024
7c78674
chore: updated urls to use local token registry for all networks by d…
Kammerlo Jul 31, 2024
92b847a
Merge remote-tracking branch 'origin/chore/bump-node-9.0-dbsync-13.3'…
Kammerlo Jul 31, 2024
c0756c8
chore: added docker compose dependency to the token registry that it …
Kammerlo Jul 31, 2024
1e382a8
chore: Token Registry Port is now adjustable and background is using it
Kammerlo Jul 31, 2024
7f90f68
chore: right tokenregistry is used now
Kammerlo Aug 1, 2024
0d8115d
chore: adjusted adapots table name
Kammerlo Aug 1, 2024
d91f945
chore: adjusted adapots table name
Kammerlo Aug 1, 2024
0ae0f31
chore: fixed console log of chainfollower starting point
Kammerlo Aug 1, 2024
6a94332
chore: Testing batchsize for queue size
Kammerlo Aug 2, 2024
643c457
chore: updated metadata info message it can take up to 90mins
Kammerlo Aug 2, 2024
af5cba1
chore: added chain follower vars to docker-compose
Kammerlo Aug 2, 2024
cdb419e
chore: fixed a bug which caused no chainsync point when syncing from …
Kammerlo Aug 2, 2024
e3b6d5c
chore: fixed lint issue
Kammerlo Aug 4, 2024
b306b45
docs: Upgraded documentation
Kammerlo Aug 5, 2024
a855291
docs: disclaimer for db-sync issue
Kammerlo Aug 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
ARG UBUNTU_VERSION=20.04

FROM ubuntu:${UBUNTU_VERSION} as ubuntu-nodejs
FROM ubuntu:${UBUNTU_VERSION} AS ubuntu-nodejs
ARG NODEJS_MAJOR_VERSION=18
ENV DEBIAN_FRONTEND=nonintercative
RUN apt-get update && apt-get install curl -y &&\
curl --proto '=https' --tlsv1.2 -sSf -L https://deb.nodesource.com/setup_${NODEJS_MAJOR_VERSION}.x | bash - &&\
apt-get install nodejs -y

FROM ubuntu-nodejs as nodejs-builder
FROM ubuntu-nodejs AS nodejs-builder
RUN curl --proto '=https' --tlsv1.2 -sSf -L https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - &&\
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list &&\
apt-get update && apt-get install gcc g++ make gnupg2 yarn -y
Expand All @@ -25,21 +25,21 @@ COPY \
tsconfig.json \
/app/

FROM nodejs-builder as cardano-graphql-builder
FROM nodejs-builder AS cardano-graphql-builder
RUN yarn --frozen-lockfile --non-interactive &&\
yarn build

FROM nodejs-builder as cardano-graphql-production-deps
FROM nodejs-builder AS cardano-graphql-production-deps
RUN yarn --frozen-lockfile --non-interactive --production

FROM frolvlad/alpine-glibc:alpine-3.11_glibc-2.30 as downloader
FROM frolvlad/alpine-glibc:alpine-3.11_glibc-2.30 AS downloader
RUN apk add curl
RUN curl --proto '=https' --tlsv1.2 -sSf -L https://github.com/hasura/graphql-engine/raw/stable/cli/get.sh | sh
ENV HASURA_GRAPHQL_ENABLE_TELEMETRY=false
RUN hasura --skip-update-check update-cli --version v2.17.1
RUN hasura --skip-update-check plugins install cli-ext

FROM nodejs-builder as dev
FROM nodejs-builder AS dev
RUN apt-get update && apt-get install yarn -y
RUN mkdir src
RUN mkdir /node-ipc
Expand All @@ -57,7 +57,7 @@ ENV \
POSTGRES_USER_FILE=/run/secrets/postgres_user
WORKDIR /src

FROM ubuntu-nodejs as background
FROM ubuntu-nodejs AS background
ARG NETWORK=mainnet
# using local token registry as default
ARG METADATA_SERVER_URI="http://token-metadata-registry:8091"
Expand All @@ -74,7 +74,7 @@ ENV \
HASURA_URI="http://hasura:8080" \
LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" \
METADATA_SERVER_URI=${METADATA_SERVER_URI} \
OGMIOS_HOST="cardano-node-ogmios" \
OGMIOS_HOST="ogmios" \
OGMIOS_PORT=1337 \
POSTGRES_DB_FILE=/run/secrets/postgres_db \
POSTGRES_HOST=postgres \
Expand All @@ -92,15 +92,15 @@ COPY --from=cardano-graphql-production-deps /app/packages/api-cardano-db-hasura/
WORKDIR /app/packages/api-cardano-db-hasura/dist
CMD ["node", "background.js"]

FROM ubuntu-nodejs as server
FROM ubuntu-nodejs AS server
ARG NETWORK=mainnet
ENV \
CARDANO_NODE_CONFIG_PATH=/config/cardano-node/config.json \
HASURA_GRAPHQL_ENABLE_TELEMETRY=false \
HASURA_URI="http://hasura:8080" \
LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" \
NETWORK=${NETWORK} \
OGMIOS_HOST="cardano-node-ogmios" \
OGMIOS_HOST="ogmios" \
OGMIOS_PORT=1337
COPY --from=cardano-graphql-builder /app/packages/api-cardano-db-hasura/dist /app/packages/api-cardano-db-hasura/dist
COPY --from=cardano-graphql-builder /app/packages/api-cardano-db-hasura/hasura/project /app/packages/api-cardano-db-hasura/hasura/project
Expand All @@ -117,6 +117,6 @@ WORKDIR /app/packages/server/dist
EXPOSE 3100
CMD ["node", "index.js"]

FROM cardanofoundation/cf-token-metadata-registry-api:latest as token-registry
FROM cardanofoundation/cf-token-metadata-registry-api:latest AS token-registry
ADD scripts/token-registry-init.sh /app/entrypoint.sh
ENTRYPOINT sh /app/entrypoint.sh
44 changes: 25 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

Cross-platform, _typed_, and **queryable** API for Cardano. The project contains multiple [packages] for composing
GraphQL services to meet specific application demands, and a [docker compose stack] serving the included
[cardano-graphql-server Dockerfile], the extended [hasura Dockerfile], [cardano-node-ogmios]. The [schema] is defined in
[cardano-graphql-server Dockerfile], the extended [hasura Dockerfile], [ogmios] and [cardano-node]. The [schema] is defined in
native `.graphql`, and used to generate a [TypeScript package for client-side static typing]. A mutation is available to
submit a signed and serialized transaction to the local node.

Expand All @@ -40,7 +40,7 @@ Check the [releases] for the latest version.
``` console
git clone \
--single-branch \
--branch 8.1.0 \
--branch 8.2.0 \
--recurse-submodules \
https://github.com/cardano-foundation/cardano-graphql.git \
&& cd cardano-graphql
Expand All @@ -58,10 +58,13 @@ console session freely. See [Docker Compose docs] to tailor for your use-case
<summary><i>mainnet</i></summary>

Get the most recent weekly snapshot link [here](https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#12/), and set it as `RESTORE_SNAPSHOT` below, or omit if you wish to sync from genesis.
> **Disclaimer:** There is currently a known issue with `cardano-db-sync` which hinders using a snapshot.
> We will provide an update once the issue is resolved.

``` console
DOCKER_BUILDKIT=1 \
COMPOSE_DOCKER_CLI_BUILD=1 \
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10060706-x86_64.tgz \
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.3/db-sync-snapshot-schema-13.3-block-10611621-x86_64.tgz \
docker compose up -d --build &&\
docker compose logs -f
```
Expand Down Expand Up @@ -127,11 +130,12 @@ your use-case.
Get the most recent weekly snapshot link [here](https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#11/), and set it as `RESTORE_SNAPSHOT` below, or omit if you wish to sync from genesis.
``` console
export NETWORK=mainnet &&\
docker pull cardanofoundation/cardano-graphql-server:8.1.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-background:8.1.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-hasura:8.1.0 &&\
docker pull cardanosolutions/cardano-node-ogmios:v6.4.0_8.9.3-${NETWORK} &&\
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10060706-x86_64.tgz \
docker pull cardanofoundation/cardano-graphql-server:8.2.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-background:8.2.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-hasura:8.2.0 &&\
docker pull ghcr.io/intersectmbo/cardano-node:9.1.0 &&\
docker pull cardanosolutions/ogmios:v6.5 &&\
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.3/db-sync-snapshot-schema-13.3-block-10611621-x86_64.tgz \
docker compose up -d &&\
docker compose logs -f
```
Expand All @@ -142,10 +146,11 @@ docker compose logs -f

``` console
export NETWORK=preprod &&\
docker pull cardanofoundation/cardano-graphql-server:8.1.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-background:8.1.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-hasura:8.1.0 &&\
docker pull cardanosolutions/cardano-node-ogmios:v6.4.0_8.9.3-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-server:8.2.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-background:8.2.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-hasura:8.2.0 &&\
docker pull ghcr.io/intersectmbo/cardano-node:9.1.0 &&\
docker pull cardanosolutions/ogmios:v6.5 &&\
API_PORT=3101 \
HASURA_PORT=8091 \
OGMIOS_PORT=1338 \
Expand All @@ -161,10 +166,11 @@ docker compose -p ${NETWORK} logs -f

``` console
export NETWORK=preview &&\
docker pull cardanofoundation/cardano-graphql-server:8.1.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-background:8.1.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-hasura:8.1.0 &&\
docker pull cardanosolutions/cardano-node-ogmios:v6.4.0_8.9.3-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-server:8.2.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-background:8.2.0-${NETWORK} &&\
docker pull cardanofoundation/cardano-graphql-hasura:8.2.0 &&\
docker pull ghcr.io/intersectmbo/cardano-node:9.1.0 &&\
docker pull cardanosolutions/ogmios:v6.5 &&\
API_PORT=3102 \
HASURA_PORT=8092 \
OGMIOS_PORT=1339 \
Expand Down Expand Up @@ -220,7 +226,7 @@ METADATA_SERVER_URI="https://metadata.world.dev.cardano.org"
If you are upgrading from Postgres 11 to 14: A resync will be needed.
To speed up the process you can use the following snapshots:
- [DB-Sync](https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10060706-x86_64.tgz)
- [Node](https://csnapshots.io/about#Manual_download)
- [Node](https://mithril.network/doc/manual/getting-started/bootstrap-cardano-node/)

### Check Cardano DB sync progress
Use the GraphQL Playground in the browser at http://localhost:3100/graphql:
Expand Down Expand Up @@ -288,12 +294,12 @@ See [Building].
[Docker Compose docs]: https://docs.docker.com/compose/
[cardano-graphql-server Dockerfile]: ./Dockerfile
[hasura Dockerfile]: ./packages/api-cardano-db-hasura/hasura/Dockerfile
[cardano-node-ogmios]: https://ogmios.dev/getting-started/docker/
[ogmios]: https://ogmios.dev/getting-started/docker/
[cardano-node]: https://github.com/IntersectMBO/cardano-node
[schema]: ./packages/api-cardano-db-hasura/schema.graphql
[TypeScript package for client-side static typing]: ./packages/client-ts/README.md
[Apollo Server]: https://www.apollographql.com/docs/apollo-server/
[GraphQL Code Generator]: https://graphql-code-generator.com
[available on npm]: https://www.npmjs.com/package/cardano-graphql-ts
[Ogmios]: https://ogmios.dev/
[releases]: https://github.com/cardano-foundation/cardano-graphql/releases
[Wiki :book:]: https://github.com/cardano-foundation/cardano-graphql/wiki
Expand Down
3 changes: 2 additions & 1 deletion config/network/mainnet/cardano-db-sync/config.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"EnableFutureGenesis": true,
"EnableLogMetrics": false,
"EnableLogging": true,
"NetworkName": "mainnet",
Expand Down Expand Up @@ -111,4 +112,4 @@
"scRotation": null
}
]
}
}
Loading
Loading