RFC: Build-level support for dynamic loading of views / apps

[cristiano-belloni]

Right now, Modular can generate full-featured applications, which:

• have an index.html template and an index.ts entrypoint, hardcoded in the configuration, that invokes ReactDOM.render
• have all their dependencies bundled in the dist
• are built with Webpack (optionally esbuild) + plugins

and views , which:

• have a main entrypoint specified in their package.json
• have no dependencies bundled in the dist
• are built with rollup + babel (soon esbuild), with a more concise configuration with less plugins

In the interest of supporting microfrontends and dynamic loading, it would be useful to specify a mechanism which uses one of the previous concepts (or both) to generate a distributable build that can be imported dynamically by a third party platform / client, without knowing of their existence at build time.

Currently, I see two main options:

1. Build views like we build apps. Currently, the fact that Modular builds apps and views differently means that many features usable within the code of an app can't be used in views (eg loading svgs, or web workers) and that, in general, there's no feature parity between the two. Unifying the build process would allow developers to express a complete app as a component using a view, to be dynamically imported it in a client application. The downsides are:

   1. If we build a view as a library, without bundling the dependencies, we would need an additional step to make it effectively dynamically loadable at runtime (for example, rewriting all the dependencies to use a ESM CDN, like Skypack or esm.sh) - we should probably also talk of custom dependency rewriting if we want to go down this way?
   2. It might be desirable to have a "fallback to iframe" functionality, but that's not really a downside, since it's easily solvable by injecting an index.html + index.ts template in the client application.

2. Build apps with an additional entrypoint. Currently, apps are built with one entrypoint, index.ts, which invokes ReactDOM.render on a React component (defined in App.js in the template). What if an application was built with two named entrypoints instead (eg index and app), and the latter was lazily loaded by the first? In theory, a client application could use the app entrypoint to load our app, or fallback to load the whole application's index.html in an iframe. The downsides are:

   1. No module federation - the whole app would be loaded with all of its dependencies bundled (see dependency rewriting discussed in at point 1.i).
   2. Apps would need a specific, immutable template (but maybe we could auto-generate the index.ts and let the developer specify only the app entrypoint? At this point, how different would this solution be from 1), apart for bundling / not bundling the dependencies?)

[benpryke]

Thanks for writing this.

I'd like to understand more clearly why option 1 requires dependencies to be bundled separately. Thinking out loud, I can imagine two scenarios that would not require this, though each has its drawbacks.

1. All dependencies are bundled in a single file. Drawbacks: the size of the file and cache invalidation with every change.
2. Dependencies are chunked and loaded by the main entrypoint. Drawbacks: the hosted URL of the assets may not be known, or may be difficult to know, at build-time, possibly complicating the runtime loader or build.

Did you consider these drawbacks unworkable, or are there further complications that I have not thought of?

One circumstance that was not called out explicitly in your RFC was that of a view that needs to communicate with the client application and carry out initialisation after it has been imported but before it renders. The view-client interface should be general enough such that a view can export its React components or a custom function for the client to call.

In general, I am leaning towards option 1. It seems to me that the "fallback to iframe" functionality you mention is important, and may be a primary use-case for some during local development. This is similar to providing a second entrypoint, except that the entrypoint is essentially built into an index.html template.

[cristiano-belloni]

All dependencies are bundled in a single file / Dependencies are chunked and loaded by the main entrypoint

I guess that Modular should strive to be as generic as possible, but maybe it's not hard to provide both behaviours and generate the appropriate bundledDependencies in each case? Might be behind a flag. I need a bit more exploration but let's keep talking about this.

The view-client interface should be general enough such that a view can export its React components or a custom function for the client to call.

Good point. I feel that the interface between a view and a potential "container" application lies outside of Modular, which only takes care of building packages. But it should be possible to build further functionality on top of what Modular provides. For example, right now views are "expected" to export a default React component, but they can potentially export any other function as named exports.

[cristiano-belloni]

Summary of what we discussed with @cangarugula and we plan to implement:

1. A view will be built as an ES Module, using the Webpack (or optionally Esbuild) configuration that we currently use for apps. They'll be expected to expose a default React component in the "main" entry file in their package.json, that will be the entrypoint for the build process. The result will be a root React component that can be dynamically imported by another web app (since the built view is essentially a library, other non-default functions can be exposed if the consumer, for example, needs initialisation. We will just require the default export to make sure that point 4) and 5) work)
2. Built views will bundle all their dependencies by default; there will be an optional flag which allows them to externalise all their dependencies externalize all dependencies rewriting them to a configurable ESM CDN. A blocklist can be specified to exclude dependencies. This is to make stateful dependencies (ie React) not duplicated on the page and for better performance overall.
3. Separate assets (styles, css, images) will be built separately (not bundled). They can be handled transparently at CI level so that they "just work".
4. A synthetically generated trampoline index.js file will be created when building and put in the js directory of the built view. The file will dynamically natively import the exported view at point 1) and ReactDOM.render() it onto the #root DOM node.
5. A synthetically generated index.html file will be also created and put in the public root directory of the build view, The file will load the trampoline at point 4), providing a way to statically host a view if needed.

[cristiano-belloni]

This was implemented in Modular by adding the esm-view type.