Skip to content

Commit

Permalink
Revert "Fixed the broken beam python on flink with PortableRunner" (#…
Browse files Browse the repository at this point in the history
…33179)

Co-authored-by: Yi Hu <huuyyi@gmail.com>
  • Loading branch information
damccorm and Abacn authored Nov 20, 2024
1 parent e3948fa commit 025384b
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 11 deletions.
4 changes: 1 addition & 3 deletions runners/flink/job-server-container/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,4 @@ COPY target/LICENSE /opt/apache/beam/
COPY target/NOTICE /opt/apache/beam/

WORKDIR /opt/apache/beam

# Add a conditional check for a mounted volume. This allows passing flink configs.
ENTRYPOINT ["/bin/sh", "-c", "if [ -d \"/flink-conf\" ]; then /opt/apache/beam/flink-job-server.sh --flink-conf-dir /flink-conf; else /opt/apache/beam/flink-job-server.sh; fi"]
ENTRYPOINT ["./flink-job-server.sh"]
11 changes: 3 additions & 8 deletions website/www/site/content/en/documentation/runners/flink.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,17 +207,12 @@ To run a pipeline on an embedded Flink cluster:
{{< /paragraph >}}

{{< paragraph class="language-portable" >}}
(1) Start the JobService endpoint: `docker run --net=host apache/beam_flink1.18_job_server:latest`
(1) Start the JobService endpoint: `docker run --net=host apache/beam_flink1.10_job_server:latest`
{{< /paragraph >}}

{{< paragraph class="language-portable" >}}
The JobService is the central instance where you submit your Beam pipeline to.
It creates a Flink job from your pipeline and executes it.
You might encounter an error message like `Caused by: java.io.IOException: Insufficient number of network buffers:...`.
This can be resolved by providing a Flink configuration file to override the default settings.
You can find an example configuration file [here](https://github.com/apache/beam/blob/master/runners/flink/src/test/resources/flink-conf.yaml).
To start the Job Service endpoint with your custom configuration, mount a local directory containing your Flink configuration to the `/flink-conf` path in the Docker container:
`docker run --net=host -v <your_flink_conf_dir>:/flink-conf beam-flink-runner apache/beam_flink1.18_job_server:latest`
The JobService will create a Flink job for the pipeline and execute the job.
{{< /paragraph >}}

{{< paragraph class="language-portable" >}}
Expand Down Expand Up @@ -248,7 +243,7 @@ To run on a separate [Flink cluster](https://ci.apache.org/projects/flink/flink-
{{< /paragraph >}}

{{< paragraph class="language-portable" >}}
(2) Start JobService with Flink Rest endpoint: `docker run --net=host apache/beam_flink1.18_job_server:latest --flink-master=localhost:8081`.
(2) Start JobService with Flink Rest endpoint: `docker run --net=host apache/beam_flink1.10_job_server:latest --flink-master=localhost:8081`.
{{< /paragraph >}}

{{< paragraph class="language-portable" >}}
Expand Down

0 comments on commit 025384b

Please sign in to comment.