cronjobs: expand script to run stand-alone

[neteler]

With this PR instructions to set up the source code repo have been turned into executable script chunks. Hence, if the scripts are executed on a new server, the GitHub repo cloning is done as well as needed directories are created.

Furthermore, some minor script tuning.

[neteler]

FYI: Installation of dependencies on grass.osgeo.org:

ssh neteler@grasslxd:~$
cd $HOME/src/main/
git pull
sudo apt install $(cat .github/workflows/apt.txt)

Due to sudo this cannot be automated, though.

[wenzeslaus]

Thanks for looking into this. I didn't get far in a Docker container (just guessing what the setup needs to be):

docker run -it ubuntu bash

apt update
apt install git
git clone https://github.com/neteler/grass-addons.git
cd grass-addons/
git switch cron_standalone_scripts
cd ..
git clone https://github.com/OSGeo/grass
apt install $(cat grass/.github/workflows/apt.txt)
grass-addons/utils/cronjobs_osgeo_lxd/cron_grass_current_stable_build_binaries.sh

grass-addons/utils/cronjobs_osgeo_lxd/cron_grass_current_stable_build_binaries.sh: 141: cd: can't cd to /home//src//releasebranch_8_4/

Perhaps, you can try to get just one of these run in a Docker container and, if that still runs on the lxd server, you can then sync the other ones too.

[wenzeslaus]

How does this relate to instructions in the comment in the PR? A single clear place is needed, maybe in readme, but here is good if it needs to be different for different scripts. (But is it?)

[neteler]

It is partially different due to the different versions.

I'll try to clean this up.

[wenzeslaus]

We do have different branches in the main repo and this is already using them, so perhaps that's one way how to make it different, yet same on the level of these scripts. In grass-addons we also have different branches, so other things can be resolved through that if needed, but that seems less applicable right now.

[neteler]

And instructions simplified in cbdf786

[wenzeslaus]

I don't understand what this is here for. I already had to get grass and grass-addons repos just to get the script running. Maybe this is needed too, but that's not clear to me right now from the setup instructions.

[neteler]

The idea was to automatically check out the code. Of course, a lot of lines for little gain, since this should be done beforehand. But then the script is no longer standalone.

Question: Is it safe to assume that the repo has already been cloned? If so, then I can happily shrink this part.

[wenzeslaus]

My point is that the script and your existing instructions (apt and the /home/src dir) required to have the grass repo downloaded beforehand, so there is no point in downloading it in the script. (Unless your idea was to remove the requirement.) The current instructions look good, so that seems to me like removing the code here.

[neteler]

Agreed. I have removed this part in 2477630

[neteler]

Also dropped "fetch addon repo if needed" in 6514132 and 3ba8276

See README.md for the (updated) docker usage.

[neteler]

... cd: can't cd to /home//src//releasebranch_8_4/

Should be addressed in 250c4a8

I got this running:

# adapt as needed, path to cronjob scripts in addons repo
cd ~/software/grass-addons/utils/cronjobs_osgeo_lxd/

# using ubuntu:22.04 as ubuntu is still missing PDAL
docker run -it --volume="$(pwd)/:/data" ubuntu:22.04 bash
apt update -y && apt install git gettext python3-sphinx doxygen graphviz zip -y

USER=`id -u -n`
MAINDIR=/home/$USER
SOURCE=$MAINDIR/src/
mkdir -p $SOURCE $MAINDIR/cronjobs/
cd $SOURCE

# GRASS GIS addons: get a shallow clone into docker container
cd $SOURCE
git clone --depth=1 https://github.com/OSGeo/grass.git grass8-addons
# add links to cronjob scripts 
(cd $MAINDIR/cronjobs/ && ln -s $SOURCE/grass8-addons/utils/cronjobs_osgeo_lxd/*.sh .)
(cd $MAINDIR/cronjobs/ && ln -s $SOURCE/grass8-addons/utils/cronjobs_osgeo_lxd/*.py .)

# needed for script runs in docker, to avoid major path complexity in scripts
ln -s /home/root/src /root/src

# GRASS GIS release branches: get a shallow clone into docker container
for REPO in releasebranch_7_8 releasebranch_8_3 releasebranch_8_4 ; do
 git clone -b $REPO --single-branch --depth=1 https://github.com/OSGeo/grass.git $REPO
done

# GRASS GIS main: get a shallow clone into docker container
BRANCH=main
git clone --single-branch --depth=1 https://github.com/OSGeo/grass.git $BRANCH

# install dependencies
apt install $(cat $SOURCE/$BRANCH/.github/workflows/apt.txt) -y

# define python3 = python
update-alternatives --install /usr/bin/python python /usr/bin/python3 1

cd /data/

# run all cronjob scripts
bash cron_grass_current_stable_build_binaries.sh && bash cron_grass_current_stable_src_snapshot.sh &&  bash cron_grass_legacy_build_binaries.sh &&  bash cron_grass_legacy_src_snapshot.sh && bash cron_grass_old_build_binaries.sh &&  bash cron_grass_old_src_snapshot.sh &&  bash cron_grass_preview_build_binaries.sh &&  bash cron_grass_preview_src_snapshot.sh

[neteler]

Instructions added to README.md in 0daf05f

[neteler]

@wenzeslaus any objections to merge this PR?
I have a new one on top which better reflects the Python strategy of publishing multiple manual pages which I can only submit once this PR is merged.