Skip to content
/ runman Public
forked from SomajitDey/runman

Linux job scheduler / manager based on task-spooler. Notifies via Telegram on job completion.

License

Notifications You must be signed in to change notification settings

PhyBi/runman

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Run Man(ager)

Run Man (runman) is a humble wrapper around task-spooler (tsp) for easily scheduling and managing jobs in any multicore Linux node. Basically, it's nothing but a poor and lazy man's workload manager. If SLURM, PBS and the likes seem like an overkill for your multiprocessing needs, and vanilla tsp seems too low-level, then runman might be your thing.

Features

  • System-wide task-spooler queue with as many slots as number of CPUs in the system.
  • USER submits .job file that lists her requirements such as
    • command/script/executable to be run
    • number of processors needed
    • paths to store STDOUT and/or STDERR
    • max. execution time (wall clock)
    • working directory
    • dependency on a previous job
  • USER is notified via Telegram on job completion by a bot.
  • Usual job management tools such as - status query, abort/removal, prioritization.
  • Job arrays similar to PBS job arrays

Installation

# Download
curl -LO https://raw.githubusercontent.com/SomajitDey/runman/main/runman

# Set permission
chmod a+rx ./runman

# Install at PATH
sudo install ./runman /usr/local/bin/

# Local installation (for non-sudoers).
export PATH="${HOME}/.bin:${PATH}" # Put this in ~/.bashrc.
install -D -t "${HOME}"/.bin/ ./runman

Also install tsp, telegram, jq and curl.

Usage

runman help

Pro tip

In a trusted environment, the sysadmin may create a single telegram bot to serve all the USERs. The token may be saved in the global config /etc/telegram.sh.conf as the key-value pair:

TELEGRAM_TOKEN='123456:AbcDefGhi-JlkMno'

In such a case, all the USERs must be sent the Telegram address of the bot: @<bot_username> or https://t.me/<bot_username> so they can subscribe to the feed using

runman tgfeed

Known Issue(s)

  • The key value pair stderr=<path> provided in .job file doesn't work, atleast in Ubuntu 20.04. The underlying reason seems to be a bug in task-spooler itself: tsp -E doesn't work as expected.

  • runman can't kill orphaned processes spawned by a job that has finished. The orphaned processes may consume CPU time at the background unbeknownst to runman or its core tsp. Hence, until fixed, it's the job of the user to make sure her job doesn't create (resource consuming) orphaned processes. For example, if her shell script spawns processes in background with &, she must use the wait statement at the end of her script, so that her job finishes only after all her spawned processes have exited.

About

Linux job scheduler / manager based on task-spooler. Notifies via Telegram on job completion.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Shell 100.0%