Adding Single-Trigger cells?

[oschulz]

Following up on a conversation with @fonsp at JuliaCon I'm creating this issue based on an older discussion:

Using Pluto for use cases the include long(er) running computations is currently a bit tricky:

(In the following, I use "above" and "below" in the sense in dependency-order, not notebook layout cell-order.)

If some Pluto cells have run times on the order of minutes (e.g. statistical inference, machine learning training, etc.), and those (typicall few) cells depends on several input/configuration/parameter values "above", then the cell will be triggered again and again while users fiddle with those "inputs". This can make their system really slow, drain their battery, or (when run on large systems that use accounting) come with resource costs in terms of compute credits or possibly actual money (e.g. when the cells spawns workers on on-demand cloud-resources).

We've had the ability to disable cells temporarily for a while now, which can come in handy. But it doesn't really address the "a few very expensive cells" scenario - one has to manually disable and re-enable the right cells again and again, and while they are disabled everything "below" is grayed out.

Could we add "single-shot" cells to Pluto? They would behave similar to cells that immediately disable themselves after they've been run. But in contrast to actually disabled cells, the content "below" would not be grayed out immediately - not until is becomes invalid - i.e. until the "one-shot" cell should be run again because the values "above" have changed. When that happens, the "one-shot" cell could be marked red, get a big "play" button overlay, or something like that, and everything "below" would be grayed-out - clearly showing that what's "below" is not consistent with what's "above" anymore.

I think this would be in keeping with Pluto philosophy - this is not about adding Jupyter-like cells, it's more like the "single-trigger" mode on an oscilloscope (we already have "continuous trigger" and "trigger off", so to speak).

@Pangoraw came up with a partial workaround based @use_memo and @SimonLafran recently posted a macro-based approach to this (thanks!!), but it can't mark dependent cells as outdated. Having this as a Pluto built-in feature would be great, in my opionion.

[cstjean]

#1079 would be another solution to this for us. The problem isn't necessarily that $LONG_COMPUTATION is rerun, but rather that it is blocking the evaluation of everything else.

It looks like a lot of work to implement that though.

[oschulz]

The problem isn't necessarily that $LONG_COMPUTATION is rerun, but rather that it is blocking the evaluation of everything else.

I don't think parallelizing evaluation will help for the use case I mentioned. If it's really something expensive, like running MCMC for minutes or spawning off SLURM workers that will cost you compute credit, you also don't want many parallel evaluations in the background, you want to control (re-)evaluation. Another use case would we data-acquisition - here you also don't want to start another measurement with your lab system every time you change some things the take = take_data_using(...) cell depends on, you want to trigger that manually once you're done editing all cells "above".

[SimonLafran]

I decided to use some of my free time to try adding the feature. I have never delved into Pluto repo, so I cannot estimate the time needed.

I will however share my thoughts before starting:

Details of the feature

• Clearly mark as stale the single-shot cell, maybe highlighting the run button
• Do not run cells that depend on stale cells and clearly mark them, differentiating from disabled cells

Accessing the feature

I see two potential ways for the user to access the feature:

• As a toggle, similar to disabling a cell
• As a hook-like feature, by returning a special StaleMarker from a cell when the cell should be marked as stale/outdated.

I prefer the hook-like feature because this allows for more flexibility and composability with other hooks.

For example, it would allow staleness to be decided based upon dependency equality, like use_memo, avoiding needlessly marking a cell as stale when we modify a dependency and then go back to its previous value

You could also create a cache of outputs, marking a cell as stale when the cache does not yet contain a precomputed value.

We could create hooks in PlutoHooks for common use cases.

[oschulz]

Thanks so much @SimonLafran !