Better documentation of --shell

[mnlevy1981]

Following the incorporation of #67 we are no longer including shell in self.run_kwargs but it is still used in the context of --launch-command: https://github.com/NCAR/jupyter-forward/blob/ea8a0ec3de63730e96c081a051a60680a44a7ce5/jupyter_forward/core.py#L175

I think there are two options:

1. Update the documentation of shell in cli.py: https://github.com/NCAR/jupyter-forward/blob/ea8a0ec3de63730e96c081a051a60680a44a7ce5/jupyter_forward/cli.py#L68
2. Come up with a shell-agnostic way to incorporate --launch-command and then drop --shell altogether

[mnlevy1981]

@andersy005 -- why does --launch-command generate a shell script to run rather than just running the command directly? (I'm not familiar with --launch-command, but I think the answer is important for deciding between the two different approaches to addressing this issue)

[andersy005]

why does --launch-command generate a shell script to run rather than just running the command directly?

As you may recall, to launch jupyter lab we retrieve the hostname or ip address information and pass this to jupyter lab command:

https://github.com/NCAR/jupyter-forward/blob/20096779c4d562fcd096b6b09b91ccc8b0851146/jupyter_forward/core.py#L178-L182

I tried running the command directly. However, the hostname information I was getting back were for the login node instead of the compute node. For instance, if you were to run this command via srun or qsub:

jupyter lab --no-browser --ip=$(hostname) > $HOME/.jupyter_forward/log.txt 2>&1

The $(hostname) would evaluate to the hostname of the login node or whichever node the qsub or srun command was issued from.

The simplest remedy for this issue was to generate a shell script, and then execute the script once we are on the compute node. My knowledge of batch schedulers is limited :). So, if you know how to get the direct command approach to work, please let me know.

Another thing to keep in mind is that sometimes you need to configure the shell invocation arguments so as to make the shell act as if it had been invoked as a login shell (this is necessary in order to get the conda activate ENV_NAME command to work). I don't know how to do this in a clean and generic way via the direct command approach. For example, here's a sample script generated by jupyter-forward:

$ cat batch-script.2020-11-28T05-01-25
#!/bin/bash -l

conda activate playground && jupyter lab --no-browser --ip=$(hostname) > /glade/u/home/abanihi/.jupyter_forward/log.2020-11-28T05-01-25 2>&1

What would be its equivalent in the direct command approach?

[mgrover1]

@andersy005 - do you think this is required to get this working with a compute node?

[andersy005]

@andersy005 - do you think this is required to get this working with a compute node?

i recall using this on compute nodes in its current form. please try it out and feel free to ask questions if you run into any issues.