[WIP] feat: offline-only export

[pankgeorg]

will fix: #2771 (eventually)

[WIP]: this is an attempt to bundle all the assets required for a static export to work completely offline, with all the assets bundled in the HTML.

Parcel is supposed to do this, but there are quite a few caveats that make this not work as-is.

Current state:

creates a 17MB HTML file with all the assets bundled, but

1. parcel is not smart enough to bundle some of the images
2. javascript bundling is also messing up some imports and some MIME types

Needs more work

TODO

• Much of the work from this PR was split into Frontend bundler update nov 2024 #3091 and merged separately
• Offline export should still try to download typefaces from the internet #2898
• Testing
• Update "Check for corrupt files" GHA

[github-actions]

Try this Pull Request!

Open Julia and type:

julia> import Pkg
julia> Pkg.activate(temp=true)
julia> Pkg.add(url="https://github.com/fonsp/Pluto.jl", rev="pg/self-contained-export")
julia> using Pluto

[fonsp]

Interesting! Curious if the bundler issues can be fixed easily!

Did you look into external tools that could do this? (i.e. make an offline-ready snapshot version of this html file.) I tried the "Save complete page" feature from Chrome but that did not work. :(

And did you consider solutions that involve a custom CDN? Maybe you could self-host? You need to statically serve the frontend-dist folder of every Pluto version. Or maybe cdn.jsdelivr.net could be allow-listed?

The solution could also be a Julia script: julia viewnotebook.jl path_to_my_nb.html which starts a static file server and shows the notebook.

[pankgeorg]

Interesting! Curious if the bundler issues can be fixed easily!

me too! some issues seem like noone cares to fix

Did you look into external tools that could do this? (i.e. make an offline-ready snapshot version of this html file.) I tried the "Save complete page" feature from Chrome but that did not work. :(

I've tried parcel with various configurations, monolith and inliner. They all get stuck somewhere in the dynamically loaded images, in loading the modules despite them being text/javascript or some combination

And did you consider solutions that involve a custom CDN? Maybe you could self-host? You need to statically serve the frontend-dist folder of every Pluto version. Or maybe cdn.jsdelivr.net could be allow-listed?

No this is not an option for our use case, unfortunately.

The solution could also be a Julia script: julia viewnotebook.jl path_to_my_nb.html which starts a static file server and shows the notebook.

This might be good to do either way. I'll make a small package that does this, it can be a nice workaround for now.

[pankgeorg]

And, of course, there is the service worker approach

[fonsp]

One problem with the SW is that it only works the second time a page is loaded, but you can work around it by triggering a refresh on the first load. You should check that this will work on file:// though.

[pankgeorg]

One problem with the SW is that it only works the second time a page is loaded, but you can work around it by triggering a refresh on the first load.

Yes, that was my thought too

You should check that this will work on file:// though.

MDN says it should work

> Locally-delivered resources such as those with http://127.0.0.1 URLs, http://localhost and http://*.localhost URLs (e.g. http://dev.whatever.localhost/), and file:// URLs are also considered to have been delivered securely.

But will know more when I have a MWE

Well, it doesn't 🤦🏾w3c/ServiceWorker#578

[fonsp]

Got it! Some changes I had to make:

url in CSS

For the url(https://...svg) in CSS: use two plugins:

"transformers": {
        "*.svg": [
            "...",
            "@parcel/transformer-inline"
        ]
    },
    "optimizers": {
        "*.svg": [
            "...",
            "@parcel/optimizer-data-url"
        ]
    }

And also, I had to learn to remove the frontend/.parcel-cache dir when changing the parcel config, otherwise your changes have no effect.

url in JS

For cases where we were using new URL("https://...", import.meta.url) to tell parcel to bundle these files, I could not find an easy solution to make this work, because parcel uses import.meta.url in the bundle to make this work, but it breaks when loading the JS from a base64 url.

To solve this, I make add these URLs as <link elements to editor.html and use document.head.querySelector(...).href to get the value from code. This lets parcel bundle the files and Julia can base64 them, and it will still work.

fonts

This I still need to figure out!

[fonsp]

Also got mathjax and the fonts :)

[fonsp]

With the JS changes and inlining ionicon svgs, the exports work offline, with mathjax, only the fonts don't load.

But inlining the svgs bring the bundled editor.css from 80kB to 120kB and that's acceptable! So we can do this in the main bundle, and we don't need a separate bundle for offline support.

I'm thinking of tweaking our font stacks a bit to make the fallback fonts look more like Alegreya and Vollkorn, and just keeping it like that! Then the offline html will be just 1.5MB with slightly different looking fonts when viewing without internet. But we don't need a second bundle :)

[fonsp]

#2896 to make the fallback fonts look nicer :) which is something i wanted to do for a long time!

[sjkelly]

Is there any chance this can be merged as-is? This seems like it is useful for procedurally generating notebook pages to HTML even without a UI:

using Pluto

s = Pluto.ServerSession()

nb_file = "plotly_static.jl"

nb = Pluto.SessionActions.open(s, nb_file; run_async=false)

# Generate the HTML file
html_content = Pluto.generate_html(nb)

# Write the HTML content to a file
open("notebook.html", "w") do io
    write(io, html_content)
end

[fonsp]

Hey @sjkelly !

I also like this PR! Btw the code you wrote already works on the current Pluto release :) but the generated HTML does not work offline.

I want to enable this feature by default for all exports, so that our HTML files will always work without relying on the jsdelivr.com CDN like we currently do.

What's left for this PR is #2898. Without that, the generated HTML files will not use Pluto's standard typefaces but always use one of the default system fonts. And that's not nice, because we want a what-you-see-is-what-you-get experience.

[fonsp]

I'm going to merge the cleanup & npm updates & some boilerplate from this PR right now since that's already useful!

EDIT merged in #3091