Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for ReadOnlyRootFilesystem in Starrocks Chart #390

Open
kapilsingh421 opened this issue Jan 5, 2024 · 2 comments · Fixed by #428
Open

Support for ReadOnlyRootFilesystem in Starrocks Chart #390

kapilsingh421 opened this issue Jan 5, 2024 · 2 comments · Fixed by #428
Labels
bug Something isn't working

Comments

@kapilsingh421
Copy link
Contributor

kapilsingh421 commented Jan 5, 2024

Describe the bug

Enabling ReadOnlyRootFilesystem should be supported.

To Reproduce

Not able to add ReadOnlyRootFilesystem:true

Expected behavior

Both the frontend and compute node are expected to function correctly.

The Persistent Volume (PV) paths are set as follows: for the frontend,
/opt/starrocks/fe/meta and /opt/starrocks/fe/logs.
The compute node uses a similar configuration with paths at /opt/starrocks/cn/storage and /opt/starrocks/cn/logs.

  • Chart Version V1.9.0
@kapilsingh421 kapilsingh421 added the bug Something isn't working label Jan 5, 2024
@yandongxiao
Copy link
Collaborator

yandongxiao commented Jan 31, 2024

StarRocks(3.2) now does not support to run on ReadOnly file system. So we can not
set pod.spec.containers.securityContext.readOnlyRootFilesystem to true.

In order to run StarRocks on read-only file system, we can do the following steps to work around this issue.

  1. Update Dockerfile. Copy the files from the original image to a temporary directory.
# specific starrocks fe-ubuntu image with the provided sha256 digest
FROM starrocks/fe-ubuntu:3.2-latest

# Set the environment variables for the source and destination directories
ENV STARROCKS_ROOT=/opt/starrocks
ENV STARROCKS_TEMP=/opt/starrocks_temp

RUN mkdir -p $STARROCKS_TEMP && cp -a $STARROCKS_ROOT/. $STARROCKS_TEMP/

# rename the original entrypoint script so it can be executed from the your own script
RUN mv $STARROCKS_TEMP/fe_entrypoint.sh $STARROCKS_ROOT/fe_entrypoint_orig.sh

# copy your own script to the container so it matches what the operator will create for the pod spec.cmd
COPY your-own-entrypoint.sh $STARROCKS_ROOT/fe_entrypoint.sh

# Note: this will make the size of the image larger than the original image
RUN chown -R starrocks:starrocks /opt/starrocks_temp/fe/
  1. Update entrypoint script. After we mount the persistent volume to the /opt/starrocks/fe directory, we can copy the
    files from the temporary directory to the persistent volume mount.
# The temporary location where we have our StarRocks files.
STARROCKS_TEMP="/opt/starrocks_temp"

# The persistent volume mount where StarRocks expects to find its files.
STARROCKS_ROOT="/opt/starrocks"

# Function to check if a directory is empty
is_dir_empty() {
    [ -z "$(ls -A "$1" 2>/dev/null)" ]
}

# Function to check and delete PID file
check_and_delete_pid() {
    local pid_dir="$1/bin"
    local pid_file=$(ls $pid_dir/*.pid 2>/dev/null)

    if [ ! -z "$pid_file" ]; then
        rm -f $pid_file
        echo "Deleted PID file: $pid_file"
    else
        echo "No PID file found in $pid_dir"
    fi
}

# Function to update /fe/lib and /fe/bin if the MD5 checksum of starrocks-fe.jar has changed.
update_fe_if_changed() {
    local src_jar="$STARROCKS_TEMP/fe/lib/starrocks-fe.jar"
    local dst_jar="$STARROCKS_ROOT/fe/lib/starrocks-fe.jar"

    if [ -f "$src_jar" ] && [ -f "$dst_jar" ]; then
        local src_md5=$(md5sum "$src_jar" | cut -d' ' -f1)
        local dst_md5=$(md5sum "$dst_jar" | cut -d' ' -f1)

        if [ "$src_md5" != "$dst_md5" ]; then
            echo "MD5 checksum does not match. Updating /fe/lib..."
            rm -rf "$STARROCKS_ROOT"/fe/lib
            cp -r "$STARROCKS_TEMP"/fe/lib "$STARROCKS_ROOT"/fe/
            cp -r "$STARROCKS_TEMP"/fe/bin "$STARROCKS_ROOT"/fe/
            echo "Updated /fe/lib successfully."
        else
            echo "MD5 checksum matches, no update needed for /fe/lib."
        fi
    fi
}

# Main logic for handling 'fe' directory copying and updating based on MD5 checksum comparison.
if [ ! -d "$STARROCKS_ROOT"/fe/bin ] || is_dir_empty "$STARROCKS_ROOT"/fe/bin; then
    echo "Bin directory is empty or does not exist in /fe, copying everything..."
    cp -r "$STARROCKS_TEMP"/fe/* "$STARROCKS_ROOT"/fe/
    echo "Files copied successfully to /fe."
else
    update_fe_if_changed
fi

# Check and delete PID files for 'fe'
check_and_delete_pid "$STARROCKS_ROOT/fe"

bash /opt/starrocks/fe_entrypoint_orig.sh "$FE_SERVICE_NAME"

@kapilsingh421
Copy link
Contributor Author

Thanks its a Good solution @yandongxiao

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants