Add Groovy container w/ examples and Redpanda (#4673)

* Groovy container w/ redpanda and examples. Also README. * Updates from Devin's review - remove apicurio from docker-compose files and change docker-compose to docker compose in shell commands * Updates from Devin's review * Make consistent with Redpanda
deephaven · Jan 17, 2024 · 4f5cc3b · 4f5cc3b
1 parent bc8ae1d
commit 4f5cc3b
Show file tree

Hide file tree

Showing 24 changed files with 204 additions and 178 deletions.
diff --git a/containers/groovy-examples-redpanda/README.md b/containers/groovy-examples-redpanda/README.md
@@ -0,0 +1,22 @@
+# Overview
+
+A Docker Compose deployment for [Deephaven](https://deephaven.io).
+
+## Features
+
+- [Deephaven](https://deephaven.io)
+- [Groovy](https://groovy-lang.org/) scripting
+- [Deephaven Examples](https://github.com/deephaven/examples)
+- [Redpanda](https://vectorized.io/), a Kakfa-compatible event streaming platform.
+
+## Launch Deephaven
+
+For instructions on launching and using Kafka in Deephaven, see[How to connect to a Kafka stream](https://deephaven.io/core/groovy/docs/how-to-guides/data-import-export/kafka-stream/). For instructions on getting started with Deephaven, see the [Quickstart](https://deephaven.io/core/groovy/docs/tutorials/quickstart).
+
+To launch Deephaven, execute the following in your deployment directory:
+
+```bash
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy-examples-redpanda/docker-compose.yml
+docker compose pull
+docker compose up -d
+```
diff --git a/containers/groovy-examples-redpanda/docker-compose.yml b/containers/groovy-examples-redpanda/docker-compose.yml
@@ -0,0 +1,34 @@
+version: "3.4"
+
+services:
+  deephaven:
+    image: ghcr.io/deephaven/server-slim:${VERSION:-latest}
+    ports:
+      - "${DEEPHAVEN_PORT:-10000}:10000"
+    volumes:
+      - ./data:/data
+    environment:
+      - START_OPTS=-Xmx4g
+
+  examples:
+    image: ghcr.io/deephaven/examples
+    volumes:
+      - ./data:/data
+    command: initialize
+
+  redpanda:
+    command:
+    - redpanda
+    - start
+    - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
+    - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
+    - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
+    - --smp 1
+    - --memory 1G
+    - --mode dev-container
+    image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
+    ports:
+    - 8081:8081
+    - 18081:18081
+    - 9092:9092
+    - 19092:19092
diff --git a/containers/groovy-examples/README.md b/containers/groovy-examples/README.md
@@ -15,9 +15,7 @@ For launch instructions, see the [README](https://github.com/deephaven/deephaven
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy-examples/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy-examples/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/groovy-redpanda/README.md b/containers/groovy-redpanda/README.md
@@ -0,0 +1,19 @@
+# Overview
+
+## Features
+
+- [Deephaven](https://deephaven.io)
+- [Groovy](https://groovy-lang.org/) scripting
+- [Redpanda](https://vectorized.io/), a Kafka-compatible event streaming platform
+
+## Launch Deephaven
+
+For instructions on launching and using Kafka in Deephaven, see[How to connect to a Kafka stream](https://deephaven.io/core/groovy/docs/how-to-guides/data-import-export/kafka-stream/). For instructions on getting started with Deephaven, see the [Quickstart](https://deephaven.io/core/groovy/docs/tutorials/quickstart).
+
+To launch Deephaven, execute the following in your deployment directory:
+
+```sh
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy-redpanda/docker-compose.yml
+docker compose pull
+docker compose up -d
+```
diff --git a/containers/groovy-redpanda/docker-compose.yml b/containers/groovy-redpanda/docker-compose.yml
@@ -0,0 +1,28 @@
+version: "3.4"
+
+services:
+  deephaven:
+    image: ghcr.io/deephaven/server-slim:${VERSION:-latest}
+    ports:
+      - "${DEEPHAVEN_PORT:-10000}:10000"
+    volumes:
+      - ./data:/data
+    environment:
+      - START_OPTS=-Xmx4g
+
+  redpanda:
+    command:
+    - redpanda
+    - start
+    - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
+    - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
+    - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
+    - --smp 1
+    - --memory 1G
+    - --mode dev-container
+    image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
+    ports:
+    - 8081:8081
+    - 18081:18081
+    - 9092:9092
+    - 19092:19092
diff --git a/containers/groovy/README.md b/containers/groovy/README.md
@@ -10,14 +10,12 @@ A Docker Compose deployment for [Deephaven](https://deephaven.io).
 
 ## Launch Deephaven
 
-For launch instructions, see the [README](https://github.com/deephaven/deephaven-core/#launch-groovy--java).  For full instructions to work with Deephaven, see the [Quick start](https://deephaven.io/core/docs/tutorials/quickstart).
+For launch instructions, see the [README](https://github.com/deephaven/deephaven-core/#launch-groovy--java).  For full instructions to work with Deephaven, see the [Quick start](https://deephaven.io/core/groovy/docs/tutorials/quickstart).
 
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-examples-redpanda/README.md b/containers/python-examples-redpanda/README.md
@@ -7,18 +7,16 @@ A Docker Compose deployment for [Deephaven](https://deephaven.io).
 - [Deephaven](https://deephaven.io)
 - [Python](https://python.org/) scripting
 - [Deephaven Examples](https://github.com/deephaven/examples)
-- [Redpanda](https://vectorized.io/), a Kakfa-compatible event streaming platform.
+- [Redpanda](https://vectorized.io/), a Kakfa-compatible event streaming platform
 
 ## Launch Deephaven
 
-For instructions on launching and using Kafka, see [Kafka in Deephaven: an introduction](https://deephaven.io/core/docs/conceptual/kafka-in-deephaven/) and [Simple Kafka import](https://deephaven.io/core/docs/how-to-guides/kafka-simple/).  For full instructions to work with Deephaven, see the [Quick start](https://deephaven.io/core/docs/tutorials/quickstart).
+For instructions on launching and using Kafka in Deephaven, see[How to connect to a Kafka stream](https://deephaven.io/core/groovy/docs/how-to-guides/data-import-export/kafka-stream/). For instructions on getting started with Deephaven, see the [Quickstart](https://deephaven.io/core/docs/tutorials/quickstart).
 
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples-redpanda/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples-redpanda/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-examples-redpanda/docker-compose.yml b/containers/python-examples-redpanda/docker-compose.yml
@@ -20,30 +20,15 @@ services:
     command:
     - redpanda
     - start
-    - --smp
-    - '1'
-    - --reserve-memory
-    - 0M
-    - --overprovisioned
-    - --node-id
-    - '0'
-    - --kafka-addr
-    - PLAINTEXT://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
-    - --advertise-kafka-addr
-    - PLAINTEXT://redpanda:29092,OUTSIDE://localhost:9092
-    image: docker.redpanda.com/vectorized/redpanda:v22.2.2
+    - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
+    - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
+    - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
+    - --smp 1
+    - --memory 1G
+    - --mode dev-container
+    image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
     ports:
+    - 8081:8081
+    - 18081:18081
     - 9092:9092
-    - 29092:29092
-
-  registry:
-    image: apicurio/apicurio-registry-mem:1.2.2.Final
-    expose:
-      - 8080
-    ports:
-      - "8081:8080"
-    environment:
-      QUARKUS_PROFILE: prod
-      KAFKA_BOOTSTRAP_SERVERS: redpanda:9092
-      APPLICATION_ID: registry_id
-      APPLICATION_SERVER: localhost:9000
+    - 19092:19092
diff --git a/containers/python-examples/All-AI/README.md b/containers/python-examples/All-AI/README.md
diff --git a/containers/python-examples/All-AI/docker-compose.yml b/containers/python-examples/All-AI/docker-compose.yml
diff --git a/containers/python-examples/NLTK/README.md b/containers/python-examples/NLTK/README.md
@@ -16,9 +16,7 @@ For launch instructions, see the [README](https://github.com/deephaven/deephaven
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/NLTK/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/NLTK/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-examples/PyTorch/README.md b/containers/python-examples/PyTorch/README.md
@@ -16,9 +16,7 @@ For launch instructions, see the [README](https://github.com/deephaven/deephaven
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/PyTorch/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/PyTorch/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-examples/SciKit-Learn/README.md b/containers/python-examples/SciKit-Learn/README.md
@@ -16,9 +16,7 @@ For launch instructions, see the [README](https://github.com/deephaven/deephaven
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/SciKit-Learn/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/SciKit-Learn/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-examples/TensorFlow/README.md b/containers/python-examples/TensorFlow/README.md
@@ -16,9 +16,7 @@ For launch instructions, see the [README](https://github.com/deephaven/deephaven
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/TensorFlow/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/TensorFlow/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-examples/base/README.md b/containers/python-examples/base/README.md
@@ -15,9 +15,7 @@ For launch instructions, see the [README](https://github.com/deephaven/deephaven
 To launch Deephaven, execute the following in your deployment directory:
 
 ```bash
-compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml
-curl  -O "${compose_file}"
-
-docker-compose pull
-docker-compose up -d
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml
+docker compose pull
+docker compose up -d
 ```
diff --git a/containers/python-redpanda/README.md b/containers/python-redpanda/README.md
@@ -0,0 +1,21 @@
+# Overview
+
+A Docker Compose deployment for [Deephaven](https://deephaven.io).
+
+## Features
+
+- [Deephaven](https://deephaven.io)
+- [Python](https://python.org) scripting
+- [Redpanda](https://vectorized.io/), a Kafka-compatible event streaming platform
+
+## Launch Deephaven
+
+For instructions on launching and using Kafka, see [How to connect to a Kafka stream](https://deephaven.io/core/docs/how-to-guides/data-import-export/kafka-stream/). For instructions on getting started with Deephaven, see the [Quickstart](https://deephaven.io/core/docs/tutorials/quickstart).
+
+To launch Deephaven, execute the following in your deployment directory:
+
+```sh
+curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-redpanda/docker-compose.yml
+docker compose pull
+docker compose up -d
+```
diff --git a/containers/python-redpanda/docker-compose.yml b/containers/python-redpanda/docker-compose.yml
@@ -0,0 +1,28 @@
+version: "3.4"
+
+services:
+  deephaven:
+    image: ghcr.io/deephaven/server:${VERSION:-latest}
+    ports:
+      - "${DEEPHAVEN_PORT:-10000}:10000"
+    volumes:
+      - ./data:/data
+    environment:
+      - START_OPTS=-Xmx4g
+
+  redpanda:
+    command:
+    - redpanda
+    - start
+    - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
+    - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
+    - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
+    - --smp 1
+    - --memory 1G
+    - --mode dev-container
+    image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
+    ports:
+    - 8081:8081
+    - 18081:18081
+    - 9092:9092
+    - 19092:19092
diff --git a/containers/python/All-AI/README.md b/containers/python/All-AI/README.md