Skip to content

Commit

Permalink
Added inital hook to allow both docker/podman and sudo/runas/"" (adop…
Browse files Browse the repository at this point in the history
…tium#5460)

* Added inital hook to allow both docker/podman and sudo/runas/""

* Better subshell quoting

* Added correct title to dockerfile

* being explicit on default temurin image origin

     [exec] INFO:  podman build  --no-cache -t adoptopenjdk-lucene-solr-test:11-jdk-ubuntu-hotspot-full -f /home/jvanek/git/jvmtest/external/lucene-solr/dockerfile/11/jdk/ubuntu/Dockerfile.hotspot.full /home/jvanek/git/jvmtest/external/
     [exec] #####################################################
     [exec] STEP 1/21: FROM eclipse-temurin:11-jdk
     [exec] Error: creating build container: short-name resolution enforced but cannot prompt without a TTY

Which needs to resolve. The docker.io/library
is already ised in external/external.sh:
     docker_image_name="docker.io/library/eclipse-temurin:${JDK_VERSION}-jdk"

* Fixed target name in README

* Runtime can now use podman too

* Added ability to not clean after run

* Honour JAVA_TOOLS_OPTIONS in external tests

* recognizes->recognizes

Co-authored-by: Martijn Verburg <martijnverburg@gmail.com>

* determining/determines

Co-authored-by: Martijn Verburg <martijnverburg@gmail.com>

* walking/walks

Co-authored-by: Martijn Verburg <martijnverburg@gmail.com>

* Dropped `If used, you are on your own.`

But afaict it should be there...

Co-authored-by: Martijn Verburg <martijnverburg@gmail.com>

* Update external/README.md

Co-authored-by: Martijn Verburg <martijnverburg@gmail.com>

* nothing->empty string

* Used  | isntead of / for exemplar values

* Small debugging nit

* Improved readme in docker x podman context

* Removed smuggled in java_tool_options

* Reset end of file

* Renamed EXTERNAL_AQACLEAN -> EXTERNAL_AQA_CONTAIER_CLEAN

* Rneamned EXTERNAL_AQA_CONTAIER_CLEAN->EXTERNAL_AQA_CONTAINER_CLEAN

* Removed in-buil.xml docekr command

---------

Co-authored-by: Martijn Verburg <martijnverburg@gmail.com>
  • Loading branch information
judovana and karianna committed Aug 16, 2024
1 parent b2c46b9 commit dcef912
Show file tree
Hide file tree
Showing 7 changed files with 124 additions and 24 deletions.
29 changes: 24 additions & 5 deletions external/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,30 @@
# External (Third Party Container) Tests

Third Party container tests help verify that the adoptium binaries are *good* by running a variety of Java applications inside of Docker containers. adoptium/aqa-tests/[Issue #172](https://github.com/adoptium/aqa-tests/issues/172) lists the applications that we have initially targeted to best exercise the adoptium binaries. For each application, we choose to run a selection of their functional tests.
Third Party container tests help verify that the adoptium binaries are *good* by running a variety of Java applications inside of containers. adoptium/aqa-tests/[Issue #172](https://github.com/adoptium/aqa-tests/issues/172) lists the applications that we have initially targeted to best exercise the adoptium binaries. For each application, we choose to run a selection of their functional tests.

## Podman, docker and others. Sudo, runas and others

The toolchain understands two environment variables:
```
EXTERNAL_AQA_RUNNER=docker|podman|...
```
defaults to podman if podman is installed, otherwise to docker

and
```
EXTERNAL_AQA_SUDO=sudo||runas
```
which defaults to empty string
```
EXTERNAL_AQA_CONTAINER_CLEAN=true|false
```
If EXTERNAL_AQA_CONTAINER_CLEAN is false, then the image is not cleaned after the `make _tests...` targets are finished.


## Running External tests locally
To run any AQA tests locally, you follow the same pattern:

0. Ensure your test machine is set up with [test prereqs](https://github.com/adoptium/aqa-tests/blob/master/doc/Prerequisites.md). For external tests, you do need Docker installed.
0. Ensure your test machine is set up with [test prereqs](https://github.com/adoptium/aqa-tests/blob/master/doc/Prerequisites.md). For external tests, you do need Docker or Podman installed.

1. Download/unpack the SDK that you want to test to your test machine
1. `export TEST_JDK_HOME=</pathToWhereYouInstalledSDK>`
Expand All @@ -27,8 +46,8 @@ See the [roadmap](https://github.com/adoptium/aqa-tests/tree/master/external#roa
### Roadmap
Our next steps to improve and expand this set of external tests is divided into 2 categories:
#### Technical Goals
- Verify the docker images that the project produces
- Copy results from Docker container for easier viewing and triage in Jenkins
- Verify the container images that the project produces
- Copy results from container for easier viewing and triage in Jenkins
- Quick compare view, easy comparison of how different implementations stack up
- Parallel testing (to improve execution time)
- Startup-only testing (application startup, but not full runs of app functional testing)
Expand Down Expand Up @@ -61,7 +80,7 @@ There are 4 common triage scenarios, with associated appropriate actions to take
- Replace the example command line at the bottom of this script with the initial command lines that trigger execution of your test.

**build.xml**
- Update the distribution folder paths, docker image name etc according to the name of your application.
- Update the distribution folder paths, container image name etc according to the name of your application.

**playlist.xml**
- Update the name of the example test case to the actual test case of the third party application that you intend to run.
Expand Down
5 changes: 3 additions & 2 deletions external/build.xml
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,9 @@
</target>

<target name="docker_prune" description="Remove all unused images from the machine before running new builds">
<exec executable="docker">
<arg line="system prune --all --force" />
<exec executable="bash">
<arg value="${DEST_EXTERNAL}/external.sh"/>
<arg value="--prune"/>
</exec>
</target>

Expand Down
4 changes: 3 additions & 1 deletion external/build_image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ set -o pipefail

source $(dirname "$0")/common_functions.sh
source $(dirname "$0")/dockerfile_functions.sh
source $(dirname "$0")/provider.sh

buildArg=""
container_build="docker build"
container_build="$(getExternalImageBuildCommand)"

if [ $# -ne 9 ] && [ $# -ne 10 ]; then
echo "The supported tests are ${supported_tests}"
Expand Down
5 changes: 3 additions & 2 deletions external/dockerfile_functions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#

source $(dirname "$0")/common_functions.sh
source $(dirname "$0")/provider.sh

# Generate the common license and copyright header
print_legal() {
Expand Down Expand Up @@ -83,7 +84,7 @@ print_image_args() {
local build=$7
local base_docker_registry_dir="$8"

image_name="eclipse-temurin"
image_name="docker.io/library/eclipse-temurin"
tag=""
if [[ "${package}" == "jre" ]]; then
tag="${version}-jre"
Expand Down Expand Up @@ -126,7 +127,7 @@ print_test_tag_arg() {

print_result_comment_arg() {
local file=$1
echo -e "ENV RESULT_COMMENT=\"IN DOCKER\"\n" >> ${file}
echo -e "ENV RESULT_COMMENT=\"IN CONTAINER($(getProviderTile))\"\n" >> ${file}
}

# Select the ubuntu OS packages
Expand Down
41 changes: 28 additions & 13 deletions external/external.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@
# script runs in 5 modes - prepare / build / run / load / clean

set -e

source $(dirname "$0")/provider.sh

tag=nightly
docker_os=ubuntu
build_type=full
Expand All @@ -33,17 +36,17 @@ node_name=""
node_labels=""
node_label_micro_architecture=""
node_label_current_os=""
container_run="docker run"
container_login="docker login"
container_inspect="docker inspect"
container_cp="docker cp"
container_commit="docker commit"
container_tag="docker tag"
container_logout="docker logout"
container_push="docker push"
container_pull="docker pull"
container_rm="docker rm"
container_rmi="docker rmi"
container_run="$(getExternalImageCommand) run"
container_login="$(getExternalImageCommand) login"
container_inspect="$(getExternalImageCommand) inspect"
container_cp="$(getExternalImageCommand) cp"
container_commit="$(getExternalImageCommand) commit"
container_tag="$(getExternalImageCommand) tag"
container_logout="$(getExternalImageCommand) logout"
container_push="$(getExternalImageCommand) push"
container_pull="$(getExternalImageCommand) pull"
container_rm="$(getExternalImageCommand) rm"
container_rmi="$(getExternalImageCommand) rmi"
docker_registry_required="false"
docker_registry_url=""
docker_registry_dir=""
Expand All @@ -58,7 +61,7 @@ imageArg=""


usage () {
echo 'Usage : external.sh --dir TESTDIR --tag DOCKERIMAGE_TAG --version JDK_VERSION --impl JDK_IMPL [--docker_os docker_os][--platform PLATFORM] [--portable portable] [--node_name node_name] [--node_labels node_labels] [--docker_registry_required docker_registry_required] [--docker_registry_url DOCKER_REGISTRY_URL] [--docker_registry_dir DOCKER_REGISTRY_DIR] [--base_docker_registry_url baseDockerRegistryUrl] [--base_docker_registry_dir baseDockerRegistryDir] [--mount_jdk mount_jdk] [--test_root TEST_ROOT] [--reportsrc appReportDir] [--reportdst REPORTDIR] [--testtarget target] [--docker_args EXTRA_DOCKER_ARGS] [--build|--run|--load|--clean]'
echo 'Usage : external.sh --dir TESTDIR --tag DOCKERIMAGE_TAG --version JDK_VERSION --impl JDK_IMPL [--docker_os docker_os][--platform PLATFORM] [--portable portable] [--node_name node_name] [--node_labels node_labels] [--docker_registry_required docker_registry_required] [--docker_registry_url DOCKER_REGISTRY_URL] [--docker_registry_dir DOCKER_REGISTRY_DIR] [--base_docker_registry_url baseDockerRegistryUrl] [--base_docker_registry_dir baseDockerRegistryDir] [--mount_jdk mount_jdk] [--test_root TEST_ROOT] [--reportsrc appReportDir] [--reportdst REPORTDIR] [--testtarget target] [--docker_args EXTRA_DOCKER_ARGS] [--build|--run|--load|--clean|--prune]'
}

supported_tests="external_custom aot camel criu-functional criu-portable-checkpoint criu-portable-restore criu-ubi-portable-checkpoint criu-ubi-portable-restore derby elasticsearch jacoco jenkins functional-test kafka lucene-solr openliberty-mp-tck payara-mp-tck quarkus quarkus_quickstarts scala system-test tck-ubi-test tomcat tomee wildfly wycheproof netty spring"
Expand Down Expand Up @@ -225,6 +228,9 @@ parseCommandLineArgs() {
"--clean" | "-c" )
command_type=clean;;

"--prune" | "-p" )
command_type=prune;;

"--help" | "-h" )
usage; exit 0;;

Expand Down Expand Up @@ -436,12 +442,21 @@ if [ $command_type == "load" ]; then
fi
fi

if [ $command_type == "clean" ]; then
if [ "${command_type}" == "clean" ] ; then
if [[ ${test} == 'external_custom' ]]; then
test="$(echo ${EXTERNAL_CUSTOM_REPO} | awk -F'/' '{print $NF}' | sed 's/.git//g')"
fi
if [ "${EXTERNAL_AQA_CONTAINER_CLEAN}" == "false" ] ; then
echo "to debug, put '-i --entrypoint /bin/bash' before container name"
container_rm="echo to clean, run manually: $container_rm"
container_rmi="echo to clean, run manually: $container_rmi"
fi
$container_rm -f $test-test; $container_rmi -f adoptopenjdk-$test-test:${JDK_VERSION}-$package-$docker_os-${JDK_IMPL}-$build_type
$container_rm -f restore-test
$container_rmi -f ${docker_registry_url}/${docker_image_source_job_name}/${JDK_VERSION}-${JDK_IMPL}-${docker_os}-${platform}-${node_label_current_os}-${node_label_micro_architecture}:${build_number}
$container_rmi -f ${docker_registry_url}/${docker_image_source_job_name}:${build_number}
fi

if [ "${command_type}" == "prune" ] ; then
$(getExternalImageCommand) system prune --all --force
fi
2 changes: 1 addition & 1 deletion external/lucene-solr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ To run any AQA tests locally, you follow the same pattern:
6. `cd TKG`
7. export required environment variables, BUILD_LIST and EXTRA_DOCKER_ARGS (`export BUILD_LIST=external/lucene-solr` and `export EXTRA_DOCKER_ARGS="-v $TEST_JDK_HOME:/opt/java/openjdk"`
8. `make compile` (This fetches test material and compiles it, based on build.xml files in the test directories)
9. `make lucene-solr_test` (When you defined BUILD_LIST to point to a directory in aqa-tests/external, then this is a testCaseName from the playlist.xml file within the directory you chose)
9. `make _lucene_solr_nightly_smoketest` (When you defined BUILD_LIST to point to a directory in aqa-tests/external, then this is a testCaseName from the playlist.xml file within the directory you chose with underscore)

When [running these from the command-line](https://github.com/adoptium/aqa-tests/blob/master/doc/userGuide.md#local-testing-via-make-targets-on-the-commandline), these tests are grouped under a make target called 'external', so 'make external' would run the entire set of tests found in the aqa-tests/external directory. This is unadvisable! Limit what you compile and run, BUILD_LIST=external/`<someSubDirectory>`, and TARGET=`<testCaseNameFromSubdirPlaylist>`

Expand Down
62 changes: 62 additions & 0 deletions external/provider.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#


###################################################################
## This script walks through all external tests ##
## and determines which virtualisation to use ##
## It recognizes two environment variables: ##
## EXTERNAL_AQA_RUNNER=docker|podman|... ##
## EXTERNAL_AQA_SUDO=sudo||runas ##
## ##
## EXTERNAL_AQA_RUNNER defaults to podman if podman is installed ##
## otherwise to docker ##
## EXTERNAL_AQA_SUDO defaults to empty string ##
###################################################################

if [ -z "${EXTERNAL_AQA_RUNNER}" ]; then
if which podman > /dev/null; then
EXTERNAL_AQA_RUNNER=podman
else
EXTERNAL_AQA_RUNNER=docker
fi
fi

function getExternalImageBuildCommand() {
#"sudo docker build"
echo "$(getExternalImageCommand) build"
}

function getExternalImageCommand() {
#"sudo docker"
echo "${EXTERNAL_AQA_SUDO} ${EXTERNAL_AQA_RUNNER}"
}

function getProviderNice() {
echo "${EXTERNAL_AQA_RUNNER}"
}

function getSudoNice() {
if [ -z "${EXTERNAL_AQA_SUDO}" ]; then
echo "not-as-root"
else
echo "${EXTERNAL_AQA_SUDO}"
fi
}

function getProviderTile() {
echo "$(getSudoNice)/$(getProviderNice)"
}

0 comments on commit dcef912

Please sign in to comment.