diff --git a/build-scripts/README.md b/build-scripts/README.md index e1279aadaecb1..b249938e4f3e9 100644 --- a/build-scripts/README.md +++ b/build-scripts/README.md @@ -1,273 +1,44 @@ -# How to Build `wazuh-indexer` DEB and RPM Packages +# Wazuh Indexer packages generation guide -> [!CAUTION] -> -> Be aware that there might be some problems while following the steps in this guide due to outdated information. -> This document is pending a review. Let us know if you find any issues. +This guide includes instructions to generate distribution packages locally using Docker. -The packages' generation process consists on 2 steps: +Wazuh Indexer supports any of these combinations: -- **Build**: compiles the Java application and bundles it into a package. -- **Assembly**: uses the package from the previous step and inflates it with plugins and - configuration files, ready for production deployment. +- distributions: `['tar', 'deb', 'rpm']` +- architectures: `['x64', 'arm64']` -We usually generate the packages using GitHub Actions, however, the process is designed to -be independent enough for maximum portability. GitHub Actions provides infrastructure, while -the building process is self-contained in the application code. +Windows is currently not supported. -Each section includes instructions to generate packages locally, using Act or Docker. +The process to build packages requires Docker and Docker Compose. -- [Install Act](https://github.com/nektos/act) +- [Install Docker](https://docs.docker.com/engine/install/) +- [Install Docker Compose](https://docs.docker.com/compose/install/linux/) -The names of the packages are managed by the `baptizer.sh` script. +Before you get started, make sure to clean your environment by running `./gradlew clean`. -## Building the plugins +## Building wazuh-indexer packages -Follow the [DEVELOPER_GUIDE.md](https://github.com/wazuh/wazuh-indexer-plugins/blob/master/DEVELOPER_GUIDE.md) instructions to build the plugins. The build scripts expect the plugins in the Maven local repository or under the `artifacts/plugins` folder. - -1. Build the plugins. -2. Publish the plugins to the local Maven repository: run `./gradlew publishToMavenLocal`. - - Alternatively, copy the generated zip files to the `artifacts/plugins` folder. -3. Build and Assemble the `wazuh-indexer` package. - -## Build and Assemble in Act - -Use Act to run the `build.yml` workflow locally. The `act.input.env` file contains the inputs -for the workflow. As the workflow clones the `wazuh-indexer-plugins` repository, the `GITHUB_TOKEN` -is required. You can use the `gh` CLI to authenticate, as seen in the example below. - -```console -act -j build -W .github/workflows/build.yml --artifact-server-path ./artifacts --input-file build-scripts/act.input.env -s GITHUB_TOKEN="$(gh auth token)" -``` - -## Build - -For local package generation, use the `build.sh` script. Take a look at the `build.yml` -workflow file for an example of usage. +Use the script under `wazuh-indexer/docker/builder/builder.sh` to build a package. ```bash -bash build-scripts/build.sh -a x64 -d tar -n $(bash build-scripts/baptizer.sh -a x64 -d tar -m) -``` - -#### Act (GitHub Workflow locally) - -```console -act -j build -W .github/workflows/build.yml --artifact-server-path ./artifacts - -[Build slim packages/build] 🏁 Job succeeded -``` - -#### Running in Docker - -Using the [Docker environment](../docker): - -```console -docker exec -it wi-build_$( By default, `ar` and `tar` tools expect the package to be in `wazuh-indexer/artifacts/tmp/deb`. - > The script takes care of creating the required folder structure, copying also the min package - > and the Makefile. - - Current folder loadout at this stage: - - ``` - artifacts/ - |-- dist - | |-- wazuh-indexer-min_5.0.0_amd64.deb - `-- tmp - `-- deb - |-- Makefile - |-- data.tar.gz - |-- debmake_install.sh - |-- etc - |-- usr - |-- var - `-- wazuh-indexer-min_5.0.0_amd64.deb - ``` - - `usr`, `etc` and `var` folders contain `wazuh-indexer` files, extracted from `wazuh-indexer-min-*.deb`. - `Makefile` and the `debmake_install` are copied over from `wazuh-indexer/distribution/packages/src/deb`. - The `wazuh-indexer-performance-analyzer.service` file is also copied from the same folder. It is a dependency of the SPEC file. - -2. Install the plugins using the `opensearch-plugin` CLI tool. -3. Set up configuration files. - - > Included in `min-package`. Default files are overwritten. - -4. Bundle a DEB file with `debmake` and the `Makefile`. - - > `debmake` and other dependencies can be installed using the `provision.sh` script. - > The script is invoked by the GitHub Workflow. +The resulting package will be stored at `wazuh-indexer/artifacts/dist`. - Current folder loadout at this stage: +> The `STAGE` option defines the naming of the package. When set to `false`, the package will be unequivocally named with the commits' SHA of the `wazuh-indexer`, `wazuh-indexer-plugins` and `wazuh-indexer-reporting` repositories, in that order. For example: `wazuh-indexer_5.0.0-0_x86_64_aff30960363-846f143-494d125.rpm`. - ``` - artifacts/ - |-- artifact_name.txt - |-- dist - | |-- wazuh-indexer-min_5.0.0_amd64.deb - | `-- wazuh-indexer_5.0.0_amd64.deb - `-- tmp - `-- deb - |-- Makefile - |-- data.tar.gz - |-- debmake_install.sh - |-- etc - |-- usr - |-- var - |-- wazuh-indexer-min_5.0.0_amd64.deb - `-- debian/ - | -- control - | -- copyright - | -- rules - | -- preinst - | -- prerm - | -- postinst - ``` +## Building wazuh-indexer docker images -#### Running in Docker - -Pre-requisites: - -- Current directory: `wazuh-indexer/` -- Existing deb package in `wazuh-indexer/artifacts/dist/deb`, as a result of the _Build_ stage. -- Using the [Docker environment](../docker): - -```console -docker exec -it wi-assemble_$( By default, `rpm2cpio` and `cpio` tools expect the package to be in `wazuh-indexer/artifacts/tmp/rpm`. The script takes care of creating the required folder structure, copying also the min package and the SPEC file. - - Current folder loadout at this stage: - - ``` - /rpm/$ARCH - /etc - /usr - /var - wazuh-indexer-min-*.rpm - wazuh-indexer.rpm.spec - ``` - - `usr`, `etc` and `var` folders contain `wazuh-indexer` files, extracted from `wazuh-indexer-min-*.rpm`. - `wazuh-indexer.rpm.spec` is copied over from `wazuh-indexer/distribution/packages/src/rpm/wazuh-indexer.rpm.spec`. - The `wazuh-indexer-performance-analyzer.service` file is also copied from the same folder. It is a dependency of the SPEC file. - -2. Install the plugins using the `opensearch-plugin` CLI tool. -3. Set up configuration files. - - > Included in `min-package`. Default files are overwritten. - -4. Bundle an RPM file with `rpmbuild` and the SPEC file `wazuh-indexer.rpm.spec`. - - > `rpmbuild` is part of the `rpm` OS package. - - > `rpmbuild` is invoked from `wazuh-indexer/artifacts/tmp/rpm`. It creates the {BUILD,RPMS,SOURCES,SRPMS,SPECS,TMP} folders and applies the rules in the SPEC file. If successful, `rpmbuild` will generate the package in the `RPMS/` folder. The script will copy it to `wazuh-indexer/artifacts/dist` and clean: remove the `tmp\` folder and its contents. - - Current folder loadout at this stage: - - ``` - /rpm/$ARCH - /{BUILD,RPMS,SOURCES,SRPMS,SPECS,TMP} - /etc - /usr - /var - wazuh-indexer-min-*.rpm - wazuh-indexer.rpm.spec - ``` - -#### Running in Docker - -Pre-requisites: - -- Current directory: `wazuh-indexer/` -- Existing rpm package in `wazuh-indexer/artifacts/dist/rpm`, as a result of the _Build_ stage. -- Using the [Docker environment](../docker): - -```console -docker exec -it wi-assemble_$( TBD. For manual building refer to [our Docker containers guide](../docker/README.md). diff --git a/build-scripts/REFERENCE.md b/build-scripts/REFERENCE.md new file mode 100644 index 0000000000000..e354f8198e2b2 --- /dev/null +++ b/build-scripts/REFERENCE.md @@ -0,0 +1,224 @@ +# Wazuh Indexer build scripts and flow reference + +## Introduction + +The packages' generation process consists on 2 steps: + +- **Build**: compiles the Java application and bundles it into a package. +- **Assembly**: uses the package from the previous step and inflates it with plugins and configuration files, ready for production deployment. + +We usually generate the packages using GitHub Actions, however, the process is designed to be independent enough for maximum portability. GitHub Actions provides infrastructure, while the building process is self-contained in the application code. + +This guide includes development worthy information about the process to generate packages for Wazuh Indexer. For instructions about how to build Wazuh Indexer packages, refer to the instructions in the [README.md](./README.md). + +## Scripts reference + +There is a script for each of the steps, `build.sh` and `assemble.sh`. Along them, there are some utility scripts, such as `provision.sh` and `baptizer.sh`. This one generates the names for the packages depending on the combination of parameters given. Refer to the definition below for more details about these scripts. + +```yml +scripts: + - file: build.sh + description: | + Generates a distribution package by running the appropiate Gradle task + depending on the parameters. + inputs: + architecture: [x64, arm64] + distribution: [tar, deb, rpm] + name: the name of the package to be generated. + outputs: + package: minimal wazuh-indexer package for the given distribution and architecture. + + - file: assemble.sh + description: | + Bundles the wazuh-indexer package generated in by build.sh with plugins, + configuration files and demo certificates (certificates yet to come). + inputs: + architecture: [x64, arm64] + distribution: [tar, deb, rpm] + revision: revision number. 0 by default. + outputs: + package: final wazuh-indexer package. + + - file: provision.sh + description: Provision script for the assembly of Debian packages. + + - file: baptizer.sh + description: Generates the wazuh-indexer package name depending on the parameters. + inputs: + architecture: [x64, arm64] + distribution: [tar, deb, rpm] + revision: revision number. 0 by default. + plugins_hash: Commit hash of the `wazuh-indexer-plugins` repository. + reporting_hash: Commit hash of the `wazuh-indexer-reporting` repository. + is_release: if set, uses release naming convention. + is_min: if set, the package name will start by `wazuh-indexer-min`. Used on the build stage. + outputs: + package: the name of the wazuh-indexer package. +``` + +## Build and Assemble + +### Build + +The build process is identical for every distribution and architecture. Although the process is driven by the `build.sh` script, the compilation and bundling is performed by Gradle tasks. + +```bash +./build.sh -h +Usage: ./build.sh [args] + +Arguments: +-q QUALIFIER [Optional] Version qualifier. +-s SNAPSHOT [Optional] Build a snapshot, default is 'false'. +-p PLATFORM [Optional] Platform, default is 'uname -s'. +-a ARCHITECTURE [Optional] Build architecture, default is 'uname -m'. +-d DISTRIBUTION [Optional] Distribution, default is 'tar'. +-b BRANCH [Optional] Branch from wazuh/wazuh to download the index template from, default is ' By default, `ar` and `tar` tools expect the package to be in `wazuh-indexer/artifacts/tmp/deb`. + > The script takes care of creating the required folder structure, copying also the min package and the Makefile. + + Current folder loadout at this stage: + + ``` + artifacts/ + |-- dist + | |-- wazuh-indexer-min_5.0.0_amd64.deb + `-- tmp + `-- deb + |-- Makefile + |-- data.tar.gz + |-- debmake_install.sh + |-- etc + |-- usr + |-- var + `-- wazuh-indexer-min_5.0.0_amd64.deb + ``` + + Notes: + + - `usr`, `etc` and `var` folders contain `wazuh-indexer` files, extracted from `wazuh-indexer-min-*.deb`. + - `Makefile` and the `debmake_install` are copied over from `wazuh-indexer/distribution/packages/src/deb`. + - The `wazuh-indexer-performance-analyzer.service` file is also copied from the same folder. It is a dependency of the SPEC file. + +2. Install the plugins using the `opensearch-plugin` CLI tool. +3. Set up configuration files. + + > Included in `min-package`. Default files are overwritten. + +4. Bundle a DEB file with `debmake` and the `Makefile`. + + > `debmake` and other dependencies can be installed using the `provision.sh` script. + > The script is invoked by the GitHub Workflow. + + Current folder loadout at this stage: + + ``` + artifacts/ + |-- artifact_name.txt + |-- dist + | |-- wazuh-indexer-min_5.0.0_amd64.deb + | `-- wazuh-indexer_5.0.0_amd64.deb + `-- tmp + `-- deb + |-- Makefile + |-- data.tar.gz + |-- debmake_install.sh + |-- etc + |-- usr + |-- var + |-- wazuh-indexer-min_5.0.0_amd64.deb + `-- debian/ + | -- control + | -- copyright + | -- rules + | -- preinst + | -- prerm + | -- postinst + ``` + +#### RPM packages + +For RPM packages, the `assemble.sh` script will: + +1. Extract the RPM package using `rpm2cpio` and `cpio` tools. + + > By default, `rpm2cpio` and `cpio` tools expect the package to be in `wazuh-indexer/artifacts/tmp/rpm`. The script takes care of creating the required folder structure, copying also the min package and the SPEC file. + + Current folder loadout at this stage: + + ``` + /rpm/$ARCH + /etc + /usr + /var + wazuh-indexer-min-*.rpm + wazuh-indexer.rpm.spec + ``` + + Notes: + + - `usr`, `etc` and `var` folders contain `wazuh-indexer` files, extracted from `wazuh-indexer-min-*.rpm`. + - `wazuh-indexer.rpm.spec` is copied over from `wazuh-indexer/distribution/packages/src/rpm/wazuh-indexer.rpm.spec`. + - The `wazuh-indexer-performance-analyzer.service` file is also copied from the same folder. It is a dependency of the SPEC file. + +2. Install the plugins using the `opensearch-plugin` CLI tool. +3. Set up configuration files. + + > Included in `min-package`. Default files are overwritten. + +4. Bundle an RPM file with `rpmbuild` and the SPEC file `wazuh-indexer.rpm.spec`. + + > `rpmbuild` is part of the `rpm` OS package. + + > `rpmbuild` is invoked from `wazuh-indexer/artifacts/tmp/rpm`. It creates the {BUILD,RPMS,SOURCES,SRPMS,SPECS,TMP} folders and applies the rules in the SPEC file. If successful, `rpmbuild` will generate the package in the `RPMS/` folder. The script will copy it to `wazuh-indexer/artifacts/dist` and clean: remove the `tmp\` folder and its contents. + + Current folder loadout at this stage: + + ``` + /rpm/$ARCH + /{BUILD,RPMS,SOURCES,SRPMS,SPECS,TMP} + /etc + /usr + /var + wazuh-indexer-min-*.rpm + wazuh-indexer.rpm.spec + ``` + +### Epilogue: Act + +- [Install Act](https://github.com/nektos/act) + +Use Act to run the `build.yml` workflow locally. The `act.input.env` file contains the inputs for the workflow. As the workflow clones the `wazuh-indexer-plugins` repository, the `GITHUB_TOKEN` is required. You can use the `gh` CLI to authenticate, as seen in the example below. + +```bash +act -j build -W .github/workflows/build.yml --artifact-server-path ./artifacts --input-file build-scripts/act.input.env -s GITHUB_TOKEN="$(gh auth token)" +``` diff --git a/build-scripts/build.sh b/build-scripts/build.sh index 559ee09f072b4..6418dfeb8e107 100644 --- a/build-scripts/build.sh +++ b/build-scripts/build.sh @@ -7,7 +7,7 @@ # this file be licensed under the Apache-2.0 license or a # compatible open source license. -set -ex +# set -ex # ==== # Usage diff --git a/docker/README.md b/docker/README.md index 7a92a51b13b38..7e5d72140bb96 100644 --- a/docker/README.md +++ b/docker/README.md @@ -6,14 +6,14 @@ Multipurpose Docker environments to run, test and build `wazuh-indexer`. 1. Install [Docker][docker] as per its instructions. -1. Your workstation must meet the minimum hardware requirements: +2. Your workstation must meet the minimum hardware requirements: - 8 GB of RAM (minimum) - 4 cores The more resources the better ☺ -1. Clone the [wazuh-indexer][wi-repo]. +3. Clone the [wazuh-indexer][wi-repo]. ## Development environments @@ -29,27 +29,32 @@ Once the `wi-dev:x.y.z` container is up, attach a shell to it and run `./gradlew ## Containers to generate packages -Use the `ci/ci.sh` script to start provisioned containers to generate packages. +The `builder` image automates the build and assemble process for the Wazuh Indexer and its plugins, making it easy to create packages on any system. + +In the example below, it will generate a wazuh-indexer package for Debian based systems, for the x64 architecture, using 1 as revision number and using the production naming convention. ```bash -Usage: ./ci.sh {up|down|stop} +# Wihtin wazu-indexer/docker/builder +bash builder.sh -d deb -a x64 -R 1 -s true ``` Refer to [build-scripts/README.md](../build-scripts/README.md) for details about how to build packages. -[docker]: https://docs.docker.com/engine/install -[wi-repo]: https://github.com/wazuh/wazuh-indexer - ## Building Docker images The [prod](./prod) folder contains the code to build Docker images. A tarball of `wazuh-indexer` needs to be located at the same level that the Dockerfile. Below there is an example of the command needed to build the image. Set the build arguments and the image tag accordingly. -```console +```bash docker build --build-arg="VERSION=5.0.0" --build-arg="INDEXER_TAR_NAME=wazuh-indexer-5.0.0-1_linux-x64_cfca84f.tar.gz" --tag=wazuh-indexer:5.0.0 --progress=plain --no-cache . ``` Then, start a container with: -```console -docker run -it --rm wazuh-indexer:5.0.0 +```bash +docker run -it --rm wazuh-indexer:5.0.0 ``` + + + +[docker]: https://docs.docker.com/engine/install +[wi-repo]: https://github.com/wazuh/wazuh-indexer diff --git a/docker/ci/images/.dockerignore b/docker/builder/.dockerignore similarity index 97% rename from docker/ci/images/.dockerignore rename to docker/builder/.dockerignore index 96d12ad527ea5..80e7a76bfe851 100644 --- a/docker/ci/images/.dockerignore +++ b/docker/builder/.dockerignore @@ -65,4 +65,4 @@ testfixtures_shared/ .ci/jobs/ # build files generated -doc-tools/missing-doclet/bin/ \ No newline at end of file +doc-tools/missing-doclet/bin/ diff --git a/docker/builder/Dockerfile b/docker/builder/Dockerfile new file mode 100644 index 0000000000000..2e1c5051e926b --- /dev/null +++ b/docker/builder/Dockerfile @@ -0,0 +1,95 @@ +# Use the official Ubuntu Noble image as the base image +FROM ubuntu:noble + +RUN userdel -r ubuntu + +# Set environment variables for non-interactive installation +ENV DEBIAN_FRONTEND=noninteractive + + +# Update the package list and install necessary tools +RUN apt-get update \ + && apt-get upgrade -y \ + && apt-get install -y \ + wget \ + curl \ + unzip \ + gnupg2 \ + git \ + build-essential \ + debmake \ + debhelper-compat \ + libxrender1 \ + libxtst6 \ + libxi6 \ + libatk1.0-0 \ + libatk-bridge2.0-0 \ + libcups2 \ + libdrm2 \ + libatspi2.0-dev \ + libxcomposite-dev \ + libxdamage1 \ + libxfixes3 \ + libxfixes-dev \ + libxrandr2 \ + libgbm-dev \ + libxkbcommon-x11-0 \ + libpangocairo-1.0-0 \ + libcairo2 \ + libcairo2-dev \ + libnss3 \ + libnspr4 \ + libnspr4-dev \ + aptly \ + cpio \ + rpm \ + rpm2cpio \ + maven \ + && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Install OpenJDK 21 +RUN wget -O- https://download.java.net/openjdk/jdk21/ri/openjdk-21+35_linux-x64_bin.tar.gz | tar xz -C /opt/ + +# Set JAVA_HOME environment variable +ENV JAVA_HOME=/opt/jdk-21 +ENV PATH=$JAVA_HOME/bin:$PATH +ENV GRADLE_VERSION=8.10 +ENV GRADLE_OPTS="-Xmx2048m -XX:ReservedCodeCacheSize=440m" + +# Download gradle +RUN wget https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip -P /tmp \ + && unzip -d /opt/gradle /tmp/gradle-${GRADLE_VERSION}-bin.zip \ + && rm /tmp/gradle-${GRADLE_VERSION}-bin.zip + +# Set GRADLE_HOME environment variable +ENV GRADLE_HOME=/opt/gradle/gradle-${GRADLE_VERSION} +ENV PATH=$GRADLE_HOME/bin/$PATH + +# Clean up APT when done +RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Create a non-root user +RUN addgroup --gid 1000 wazuh-indexer && \ + adduser --ingroup wazuh-indexer --uid 1000 wazuh-indexer + +# Create repositories path + +RUN mkdir -p /repositories/wazuh-indexer-plugins && \ + mkdir -p /repositories/wazuh-indexer-reporting && \ + chown wazuh-indexer:wazuh-indexer /repositories/wazuh-indexer-plugins && \ + chown wazuh-indexer:wazuh-indexer /repositories/wazuh-indexer-reporting + +# Copy your build scripts into the container as root +COPY docker/builder/entrypoint.sh / + +# Change file permissions as root +RUN chmod +x /entrypoint.sh + +# Switch to non-root user +USER wazuh-indexer + +# Set the working directory +WORKDIR /home/wazuh-indexer + +# Entry point to the build script +ENTRYPOINT ["./entrypoint.sh"] diff --git a/docker/builder/builder.sh b/docker/builder/builder.sh new file mode 100755 index 0000000000000..1b0463a21dd4b --- /dev/null +++ b/docker/builder/builder.sh @@ -0,0 +1,117 @@ +#!/bin/bash + +# Start container with required tools to build packages +# Requires Docker +# Script usage: bash ./builder.sh + +set -e + +# ==== +# Checks that the script is run from the intended location +# ==== +function check_project_root_folder() { + current=$(basename "$(pwd)") + + if [[ "$1" != "./builder.sh" && "$1" != "builder.sh" ]]; then + echo "Run the script from its location" + usage + exit 1 + fi + # Change working directory to the root of the repository + cd ../.. +} + +# ==== +# Parse arguments +# ==== +function parse_args() { + + while getopts ":p:r:R:s:d:a:Dh" arg; do + case $arg in + h) + usage + exit 1 + ;; + p) + INDEXER_PLUGINS_BRANCH=$OPTARG + ;; + r) + INDEXER_REPORTING_BRANCH=$OPTARG + ;; + R) + REVISION=$OPTARG + ;; + s) + IS_STAGE=$OPTARG + ;; + d) + DISTRIBUTION=$OPTARG + ;; + a) + ARCHITECTURE=$OPTARG + ;; + D) + DESTROY=$OPTARG + ;; + :) + echo "Error: -${OPTARG} requires an argument" + usage + exit 1 + ;; + ?) + echo "Invalid option: -${arg}" + exit 1 + ;; + esac + done + + ## Set defaults: + [ -z "$INDEXER_PLUGINS_BRANCH" ] && INDEXER_PLUGINS_BRANCH="master" + [ -z "$INDEXER_REPORTING_BRANCH" ] && INDEXER_REPORTING_BRANCH="master" + [ -z "$REVISION" ] && REVISION="0" + [ -z "$IS_STAGE" ] && IS_STAGE="false" + [ -z "$DISTRIBUTION" ] && DISTRIBUTION="rpm" + [ -z "$ARCHITECTURE" ] && ARCHITECTURE="x64" + [ -z "$DESTROY" ] && DESTROY=false +} + +# ==== +# Displays usage +# ==== +function usage() { + echo "Usage: $0 [args]" + echo "" + echo "Arguments:" + echo -e "-p INDEXER_PLUGINS_BRANCH\t[Optional] wazuh-indexer-plugins repo branch, default is 'master'." + echo -e "-r INDEXER_REPORTING_BRANCH\t[Optional] wazuh-indexer-reporting repo branch, default is 'master'." + echo -e "-R REVISION\t[Optional] Package revision, default is '0'." + echo -e "-s STAGE\t[Optional] Staging build, default is 'false'." + echo -e "-d DISTRIBUTION\t[Optional] Distribution, default is 'rpm'." + echo -e "-a ARCHITECTURE\t[Optional] Architecture, default is 'x64'." + echo -e "-D\tDestroy the docker environment" + echo -e "-h\tPrint help" +} + +# ==== +# Main function +# ==== +function main() { + check_project_root_folder $0 + compose_file="docker/${current}/compose.yml" + compose_cmd="docker compose -f $compose_file" + REPO_PATH=$(pwd) + VERSION=$(cat VERSION) + export REPO_PATH + export VERSION + + parse_args "${@}" + + if [[ "$DESTROY" == true || "$DESTROY" == "1" ]]; then + $compose_cmd down -v + exit 0 + fi + + $compose_cmd up +} + +main "$@" diff --git a/docker/builder/compose.yml b/docker/builder/compose.yml new file mode 100644 index 0000000000000..d2b8338fa65c4 --- /dev/null +++ b/docker/builder/compose.yml @@ -0,0 +1,24 @@ +services: + wazuh-indexer-builder: + image: wazuh-indexer-builder:${VERSION} + container_name: wazuh-indexer-builder_${VERSION} + build: + context: ./../.. + dockerfile: ${REPO_PATH}/docker/builder/Dockerfile + environment: + - INDEXER_PLUGINS_BRANCH=${INDEXER_PLUGINS_BRANCH:-master} + - INDEXER_REPORTING_BRANCH=${INDEXER_REPORTING_BRANCH:-master} + - REVISION=${REVISION:-0} + - IS_STAGE=${IS_STAGE:-false} + - DISTRIBUTION=${DISTRIBUTION:-rpm} + - ARCHITECTURE=${ARCHITECTURE:-x64} + volumes: + - ${REPO_PATH}:/home/wazuh-indexer + - wazuh-indexer-plugins:/home/wazuh-indexer/wazuh-indexer-plugins + - wazuh-indexer-reporting:/home/wazuh-indexer/wazuh-indexer-reporting + entrypoint: ["/bin/bash", "/entrypoint.sh"] + user: "1000:1000" + working_dir: /home/wazuh-indexer +volumes: + wazuh-indexer-plugins: + wazuh-indexer-reporting: diff --git a/docker/builder/entrypoint.sh b/docker/builder/entrypoint.sh new file mode 100644 index 0000000000000..7c46b2a86575e --- /dev/null +++ b/docker/builder/entrypoint.sh @@ -0,0 +1,143 @@ +#!/bin/bash + +# Exit immediately if a command exits with a non-zero status. +set -e + +# Set default values for environment variables +INDEXER_PLUGINS_BRANCH=${INDEXER_PLUGINS_BRANCH:-master} +INDEXER_REPORTING_BRANCH=${INDEXER_REPORTING_BRANCH:-master} +REVISION=${REVISION:-0} +IS_STAGE=${IS_STAGE:-false} +DISTRIBUTION=${DISTRIBUTION:-rpm} +ARCHITECTURE=${ARCHITECTURE:-x64} + +# Function to clone repositories +clone_repositories() { + echo "----------------------------------------" + echo "Cloning Repositories" + echo "----------------------------------------" + + PLUGINS_REPO_DIR="/repositories/wazuh-indexer-plugins" + REPORTING_REPO_DIR="/repositories/wazuh-indexer-reporting" + + if [ -d "$PLUGINS_REPO_DIR/.git" ]; then + git -C "$PLUGINS_REPO_DIR" checkout "$INDEXER_PLUGINS_BRANCH" + else + git clone --branch "$INDEXER_PLUGINS_BRANCH" https://github.com/wazuh/wazuh-indexer-plugins --depth 1 "$PLUGINS_REPO_DIR" + fi + + if [ -d "$REPORTING_REPO_DIR/.git" ]; then + git -C "$REPORTING_REPO_DIR" checkout "$INDEXER_REPORTING_BRANCH" + else + git clone --branch "$INDEXER_REPORTING_BRANCH" https://github.com/wazuh/wazuh-indexer-reporting --depth 1 "$REPORTING_REPO_DIR" + fi +} + +# Function to build wazuh-indexer-plugins +build_plugins() { + echo "----------------------------------------" + echo "Building Plugins" + echo "----------------------------------------" + local version="$1" + local revision="$2" + cd /repositories/wazuh-indexer-plugins/plugins/setup + echo "Building setup plugin..." + ./gradlew build -Dversion="$version" -Drevision="$revision" --no-daemon + cd /repositories/wazuh-indexer-plugins/plugins/command-manager + echo "Building command-manager plugin..." + ./gradlew build -Dversion="$version" -Drevision="$revision" --no-daemon +} + +# Function to build wazuh-indexer-reporting +build_reporting() { + echo "----------------------------------------" + echo "Building Reporting" + echo "----------------------------------------" + local version="$1" + local revision="$2" + cd /repositories/wazuh-indexer-reporting + echo "Building reporting..." + ./gradlew build -Dversion="$version" -Drevision="$revision" --no-daemon +} + +# Function to copy builds +copy_builds() { + echo "----------------------------------------" + echo "Copying Builds" + echo "----------------------------------------" + local version="$1" + local revision="$2" + mkdir -p ~/artifacts/plugins + echo "Copying setup plugin..." + cp /repositories/wazuh-indexer-plugins/plugins/setup/build/distributions/wazuh-indexer-setup-"$version"."$revision".zip ~/artifacts/plugins + echo "Copying command-manager plugin..." + cp /repositories/wazuh-indexer-plugins/plugins/command-manager/build/distributions/wazuh-indexer-command-manager-"$version"."$revision".zip ~/artifacts/plugins + echo "Copying reporting..." + cp /repositories/wazuh-indexer-reporting/build/distributions/wazuh-indexer-reports-scheduler-"$version"."$revision".zip ~/artifacts/plugins +} + +# Function for packaging process +package_artifacts() { + echo "----------------------------------------" + echo "Packaging Artifacts" + echo "----------------------------------------" + local architecture="$1" + local distribution="$2" + local revision="$3" + local is_stage="$4" + + local plugins_hash + local reporting_hash + local package_min_name + local package_name + + plugins_hash=$(cd /repositories/wazuh-indexer-plugins && git rev-parse --short HEAD) + reporting_hash=$(cd /repositories/wazuh-indexer-reporting && git rev-parse --short HEAD) + + cd ~ + + echo "Creating package minimum name..." + package_min_name=$(bash build-scripts/baptizer.sh -m \ + -a "$architecture" \ + -d "$distribution" \ + -r "$revision" \ + -l "$plugins_hash" \ + -e "$reporting_hash" \ + "$(if [ "$is_stage" = "true" ]; then echo "-x"; fi)") + + echo "Creating package name..." + package_name=$(bash build-scripts/baptizer.sh \ + -a "$architecture" \ + -d "$distribution" \ + -r "$revision" \ + -l "$plugins_hash" \ + -e "$reporting_hash" \ + "$(if [ "$is_stage" = "true" ]; then echo "-x"; fi)") + + echo "Building package..." + bash build-scripts/build.sh -a "$architecture" -d "$distribution" -n "$package_min_name" + echo "Assembling package..." + bash build-scripts/assemble.sh -a "$architecture" -d "$distribution" -r "$revision" + +} + +# Main script execution +main() { + echo "---------Starting Build Process---------" + clone_repositories + # Set version env var + VERSION=$(cat ~/VERSION) + # Build and assemble the package + build_plugins "$VERSION" "$REVISION" + build_reporting "$VERSION" "$REVISION" + copy_builds "$VERSION" "$REVISION" + package_artifacts "$ARCHITECTURE" "$DISTRIBUTION" "$REVISION" "$IS_STAGE" + + # Clean the environment + echo "----------------------------------------" + echo "Build and Packaging Process Completed Successfully!" + echo "----------------------------------------" +} + +# Execute the main function +main diff --git a/docker/ci/ci.sh b/docker/ci/ci.sh deleted file mode 100755 index 13e84c0881b3f..0000000000000 --- a/docker/ci/ci.sh +++ /dev/null @@ -1,62 +0,0 @@ -#!/bin/bash - -# Start container with required tools to build packages -# Requires Docker -# Script usage: bash ./ci.sh - -set -e - -# ==== -# Checks that the script is run from the intended location -# ==== -function check_project_root_folder() { - current=$(basename "$(pwd)") - - if [[ "$0" != "./ci.sh" && "$0" != "ci.sh" ]]; then - echo "Run the script from its location" - usage - exit 1 - fi - # Change working directory to the root of the repository - cd ../.. -} - -# ==== -# Displays usage -# ==== -function usage() { - echo "Usage: ./ci.sh {up|down|stop}" -} - -# ==== -# Main function -# ==== -function main() { - check_project_root_folder "$@" - compose_file="docker/${current}/ci.yml" - compose_cmd="docker compose -f $compose_file" - REPO_PATH=$(pwd) - VERSION=$(cat VERSION) - export REPO_PATH - export VERSION - - case $1 in - up) - # Main folder created here to grant access to both containers - mkdir -p artifacts - $compose_cmd up -d - ;; - down) - $compose_cmd down - ;; - stop) - $compose_cmd stop - ;; - *) - usage - exit 1 - ;; - esac -} - -main "$@" diff --git a/docker/ci/ci.yml b/docker/ci/ci.yml deleted file mode 100644 index ccf902af07734..0000000000000 --- a/docker/ci/ci.yml +++ /dev/null @@ -1,28 +0,0 @@ -services: - # Essentially wi-dev, but doesn't expose port 9200 - wi-build: - image: wi-build:${VERSION} - container_name: wi-build_${VERSION} - build: - context: ./../.. - dockerfile: ${REPO_PATH}/docker/dev/images/Dockerfile - volumes: - - ${REPO_PATH}:/home/wazuh-indexer - entrypoint: ["tail", "-f", "/dev/null"] - user: "1000:1000" - working_dir: /home/wazuh-indexer - - wi-assemble: - image: wi-assemble:${VERSION} - container_name: wi-assemble_${VERSION} - build: - context: ./../.. - dockerfile: ${REPO_PATH}/docker/ci/images/Dockerfile - volumes: - - ${REPO_PATH}/build-scripts:/home/wazuh-indexer/build-scripts - - ${REPO_PATH}/artifacts:/home/wazuh-indexer/artifacts - - ${REPO_PATH}/distribution/packages/src:/home/wazuh-indexer/distribution/packages/src - - ${REPO_PATH}/buildSrc:/home/wazuh-indexer/buildSrc - entrypoint: ["tail", "-f", "/dev/null"] - user: "1000:1000" - working_dir: /home/wazuh-indexer diff --git a/docker/ci/images/Dockerfile b/docker/ci/images/Dockerfile deleted file mode 100644 index 8704d1a25d83b..0000000000000 --- a/docker/ci/images/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -FROM ubuntu:jammy -RUN mkdir /home/wazuh-indexer && \ - apt-get update -y && \ - apt-get install curl gnupg2 -y && \ - curl -o- https://www.aptly.info/pubkey.txt | apt-key add - && \ - echo "deb http://repo.aptly.info/ squeeze main" | tee -a /etc/apt/sources.list.d/aptly.list && \ - apt-get update -y && \ - apt-get upgrade -y && \ - apt-get install -y aptly build-essential cpio debhelper-compat debmake freeglut3 libasound2 libatk-bridge2.0-0 libatk1.0-0 libatspi2.0-dev libcairo2 libcairo2-dev libcups2 libdrm2 libgbm-dev libgconf-2-4 libnspr4 libnspr4-dev libnss3 libpangocairo-1.0-0 libxcomposite-dev libxdamage1 libxfixes-dev libxfixes3 libxi6 libxkbcommon-x11-0 libxrandr2 libxrender1 libxtst6 rpm rpm2cpio maven && \ - apt-get clean -y && \ - dpkg -r lintian && \ - addgroup --gid 1000 wazuh-indexer && \ - adduser --uid 1000 --ingroup wazuh-indexer --disabled-password --home /home/wazuh-indexer wazuh-indexer && \ - chmod 0775 /home/wazuh-indexer && \ - chown -R 1000:1000 /home/wazuh-indexer -USER wazuh-indexer -WORKDIR /home/wazuh-indexer