Question: dynamic components

[theSherwood]

I'm experimenting with some dynamic rendering of some components here:
https://codesandbox.io/s/dynamic-components-z69n5

I'm wrapping the dynamic component in a closure in order to make it reactive:

const view = html`
    <h3>Dynamic Components</h3>
    <button onclick=${addToList}>
      Add to List
    </button>
    <button onclick=${switchComponent}>
      Switch Component
    </button>
    <hr />
    ${() =>
      html`
        <${list()} items=${items} />
      `}
  `;

Is there a more idiomatic way of dealing with dynamic components? And if not, would it be worth considering some alternative to a closure with a nested html call?

This feels related to #6 on Conditional Rendering, which faced a similar closure and nested html call issue. As RyanSolid pointed out, the template must be recreated repeatedly.

[luwes]

Thanks for your question, I can't think of a much better way to write this actually.

There's no getting around the closure and nested html call with template literals I'm afraid. (I've been thinking of adding the closures automatically but that will only work for JSX during the compilation step)

It looks a bit nicer if you make the closure a variable, but you lose locality...

const List = () => html`
	<${list()} items=${items} />
`;

const List2 = itemsParam => () => html`
	<${list()} items=${itemsParam} />
`;

const view = html`
    <h3>Dynamic Components</h3>
    <button onclick=${addToList}>
      Add to List
    </button>
    <button onclick=${switchComponent}>
      Switch Component
    </button>
    <hr />
    ${List} // or ${List2(items)}
  `;

[theSherwood]

Interesting. Thanks for the quick reply and sorry to keep bombarding you with questions. But here are some more:

• Apart from the considerations of syntax, what are the performance consequences of the closure and having to recreate these templates (both for dynamic components and for conditional branching)?
• How would you anticipate this looking in JSX?
• Is the issue that this can't be expressed in template literals or that there exists no present convention for doing that?

Again, thank you for answering my questions. My understanding of some of this stuff is limited, but I'm interested in learning more, and potentially contributing if you are looking for contributors.

[luwes]

No worries. Glad to help. Valid questions 👍

The closure is not gonna slow down much, it's negligible I think.

The recreating is easy to work around by hoisting it up and actually creating the DOM node or create a cache of it at the right time. The only thing that will happen with the switch is that the root node of the one component will be replaced by the other.

memoization would work well here I believe.

https://github.com/ryansolid/solid has a built in helper for this I believe.

I might have to take a look a that.

[luwes]

Related Solid issue: solidjs/solid#73

[theSherwood]

That was really informative. All of this inquiry is related to an application I'm planning in which dynamic components are going to be the norm. So I'm looking for a concise way to express dyamic components which is also performant. Since conditional branching exposes a similar issue in sinuous, I'm wanting to find a shorthand that abstracts away the root of both issues. But it looks like it will mean using some alternative to Tagged Templates (or maybe a modified Tagged Templates syntax) that incorporates memoization perhaps.

I'd like to avoid a compilation step, if possible, but it may be the case that it isn't possible.

Is there some idiomatic way of handling memoization in sinuous?

[luwes]

@theSherwood thanks for the feedback, valid requests.

I worked on your example to make this work with memoized components.
https://codesandbox.io/s/dynamic-components-zuwcz

There is no idiomatic way yet to handle this case in sinuous so I put together a working solution that might get a place in the core or a separate pkg. I'll have to think a little about it.

Hope this helps, let me know if you have any questions. All feedback is welcome.

[theSherwood]

@luwes Thanks for that example. That's definitely an improvement. With the design of this memoization functionality, is there a way the memo could be moved to the dynamic component rather than to the components the dynamic component is rendering? Some kind of observable memo?

So, rather than:

const component1 = memo(({ items }) => {
   ...
})
const component2 = memo(({ items }) => {
   ...
})
const list = o(component1);

...we can have:

const component1 = ({ items }) => {
   ...
}
const component2 = ({ items }) => {
   ...
}
const list = dynamic(component1);

...or maybe even:

const view = html`
    <h3>Dynamic Components</h3>
    <button onclick=${addToList}>
      Add to List
    </button>
    <button onclick=${switchComponent}>
      Switch Component
    </button>
    <hr />
    ${dynamic(list()({ items }))}
  `;

...

const component1 = ({ items }) => {
   ...
}
const component2 = ({ items }) => {
   ...
}
const list = o(component1);

dynamic may not be right for the api, but something like this feels a little cleaner than memoizing each of the components fed into the dynamic component. What do you think?

Could something like this also be used for conditional branching?

I also don't yet understand enough about the internals of the sinuous reactivity and rendering models to know whether this is practical.

[luwes]

took a few attempts but got it working!
https://codesandbox.io/s/dynamic-components-gl7ny

memo is the right tool for the job I think, it's possible to build a dynamic utility function like this one:

const dynamic = memo(comp => {
  return memo(props => root(() => comp(props)));
});

Each function component that is passed will be used as the cache key so in this case the body of dynamic will only be ran 2 times ever.

they in turn return a memoized version of the component which is keyed on the props object.

the root function is still needed because of the computations which have to live on even though the component is toggled on and off.

yup, this could all be used for conditional branching as well.

[theSherwood]

It seems to work until you try to use it twice in one component:
https://codesandbox.io/s/dynamic-components-5u5z8

[luwes]

yes, that's because of the first memo function. it will have the first execution cached of dynamic since the argument will be both component1.
easy to get around it by providing a unique argument as 2nd argument.

https://codesandbox.io/s/dynamic-components-2-pc64t

edit: because it is cached it returns the same element, in DOM there cannot be the same element in two places.

[theSherwood]

Thanks. I've iterated on what you've done to try to make a slightly more streamlined api:
https://codesandbox.io/s/dynamic-components-4-0f4b1

So the closure has been cleaned away on the dynamic component and the rebuilding of the dom fragment is gone per your memo function. You can see the minimal function calls being executed in the console.log statements I left in the definition of d. So we now have a performant dynamic component syntax that looks like this:

${d(comp, props, children)}

Notice, that comp must be an observable, and the observable is passed into d(). Not d(comp()).

So this feels like progress to me. However. I'd like to get here:

<${d(comp)} props=${props} >
    <p> ...assorted children... </p>
</${d(comp)}>

And I'm completely stymied as to how to do that. Is this possible without changing the way the tagged templates work?

[ryansolid]

FYI, createMemo in Solid isnt an actual Memo exactly. It's a pure computation. I just took Reacts naming convention. It actually is a computed with it's own reactive lifecycle internally. That being said from the outside pure computeds are essentially memos. Their arguments are just auto tracked dependencies rather than execution arguments.

[luwes]

thanks for checking in @ryansolid, ah yes I misremembered that.
I'll have to look more into createMemo.

I was wondering if Solid has a way to handle this issue of dynamic components?
I guess it is very similar to conditional branches.

The code of sinuous/memo could be used for Solid as well I think, it's pretty much a memoize function with one speciality to handle document fragments.

sinuous/packages/sinuous/memo/src/memo.js

Lines 26 to 32 in b95ae7c

	// Memoizing the contents of a document fragment is impossible because
	// once it is appended, it's cleared of its children leaving an empty
	// shell, on next render the comp would just be cleared.
	// Store the child refs in an array and memo and return this.
	if (computedValue && computedValue.nodeType === 11) {
	computedValue = EMPTY_ARR.slice.call(computedValue.childNodes);
	}

---

@theSherwood this is the best I can get it
https://codesandbox.io/s/dynamic-components-4-s4ton

html`<${dynamic(list)} items="${items}">
	I am a child
<//>`

const memoComp = memo(comp => {
  return memo((props, ...children) => {
    return root(() => comp(props, ...children));
  });
});

const dynamic = comp => {
  return (props, ...children) => {
    return () => memoComp(comp())(props, ...children);
  };
};

[theSherwood]

That looks great! There seems to be some kind of bug if the child isn't a text node, though. For example:

<${dynamic(list)} items="${items}">
   <p>I am a child.</p>
<//>

...causes the dynamic aspect of the component not to function. Did you notice that too? I haven't been able to figure out the cause.

[luwes]

@theSherwood that was a tricky one. The solution for that is here:
https://codesandbox.io/s/blazing-worker-0313z

The problem was that we're memoizing the components including the children.

So when the swap in the DOM happens the children are pulled out of the one component and put in the other. It's similar behavior as you would do an appendChild with a child that is already in the DOM. This was causing issues with clearing and adding of the component.

The solution is to create new children per component even though in this case they are exactly the same. They have to be new instances.

Shame it needs another closure and html tag, will check if this could be made better.

  const view = html`
    <h3>Dynamic Components</h3>
    <button onclick=${addToList}>
      Add to List
    </button>
    <button onclick=${switchComponent}>
      Switch Component
    </button>
    <hr />
    <${dynamic(list)} items="${items}">
      ${() => html`
        <p>I am a child</p>
        <p>I am a child too</p>
      `}
    <//>
  `;

[theSherwood]

Hey, that's great! I hope you guys don't mind if I cannibalize whatever implementation is made in order to get my quine project to work.

[theSherwood]

As a follow up to the HMR issue:

If you have a file that looks something like this:

let state = o(0)

let component = () => html`
   <p>${state}</p>
   <button onclick=${() => state(state()+1)}> + </button>
`

...is it possible to not reset state to 0 when the file is edited and reloaded? I know this is possible with some bundlers in the clojurescript ecosystem and it makes for an incredibly fluid development experience. But clojurescript atoms may be easier for a bundler to detect than sinuous observables. One option may be coopting the o or observable function during dev bundling to add a property to observables (or observables' prototype) that can be detected by the bundler.

[ryansolid]

React and most VDOM libraries have that capability too. That's always the goal, I just have never seen it done in a fine grained reactive library.

Well if you assume that all data needs to be derived downward, we only need to track observables created during the initial component execution in that context and we can avoid everything created within subcontexts (like computeds) atleast initially. While that is wasted work from a rendering perspective since those computeds might hold the actual DOM nodes, atleast assuming all state is a product of props and top level state you could recreate the same DOM from the same state, and presumably since the top level only executes once and is a product of that information it's order would be predictable. Something as simple as an array could track it. The bundler couldn't do it on its own there would have to be a hook into the reactive system. You'd have to add something to the Owning context, but it seems not that much of an overhead I suppose. Have to think of how to make it non-intrusive. There are some trickier edge cases though like computeds that are expecting to run once and write to an observable, but the observable already has the data from the previous state so they might execute again (like async requests) and trigger stuff again when they complete (or say trigger loading states in the middle). Basically it would be hard to ever account for all side effects. This is much trickier than just getting updated Component code.

On an aside beware of hoisting observables like your example. Unless you intend to share them between all instances of your Component (like if there are more than one instance on the page) you want it inside the component function.

[ryansolid]

Yeah that assumption I made for maintaining stated doesn't work. Or at least isn't that simple. Besides the difficulty of handling state changes without a VDOM (or a first pass to know it has in fact changed, so you could be assigning state to the wrong observables), you need to really know all the way down the tree. You couldn't just stop single level. So all hooks for serializing state would have to execute all the way down, and be applied all the way down from the insertion point. You'd need to patch the whole reactive system I think which goes back to what I was originally saying. This comes down to all dependencies being set dynamically at run time, with React Hooks it boils down to a simple equality check on variables. What we'd need to figure out is if we can diff a reactive graph while we are creating it. It might be enough if we could at least assign nodes to the module their code exists in rather than where they are executed. I suspect someone more familiar with how this works in a library like React post Hooks might hold the key but I still am missing something. So I am dropping working on state preservation at least for now.

That being said getting the simple case to work was trivial. I've made it work in Webpack but I want to get Parcel and Rollup too so. I'm still working on getting a universal version working and will let you know. It is really simple though. I'm spending more time setting up an environment to test rather than writing the plugin.

[theSherwood]

Okay. Interesting. I've been wondering a bit about that. On sinuous observables there is a little property to identify it as an observable. Is there an similar mechanism in solid?
https://github.com/luwes/sinuous/blob/master/packages/sinuous/observable/src/observable.js

function observable(value) {
  // Tiny indicator that this is an observable function.
  data.$o = true;
  data._observers = new Set();
  ...
}

What if that $o were instead assigned a uuid? Could that be used by the HMR to diff the state?

That being said getting the simple case to work was trivial. I've made it work in Webpack...

Can you share a link to that?

[ryansolid]

As for the UUID I was thinking of that. But I mean it would essentially be a sequence number, although would probably have to make it into some sort of path to track nesting. What I was getting at is that you need to handle everything below the swap point.

Serialize all node ids, and values down the tree on the dispose step of HMR. Then as you recreate and hit the same ids, set the values instead of the passed-in values. However, all computations would need to run again which is why paths are essential to ensure not new ids assigned each execution.

But how to do this. Since it's creation order in the context that matters not the dependency graph. And you'd need to give computations ids as well since they'd need to be able to give their part of the path on independent updates. You'd almost need to setup a graph beside the actual dependency graph. Or I guess on top, another collection on each context that you could traverse downwards to serialize. In so the computation itself that wraps the component swap could start as the root node. Then how do you read from this? I suppose we could wrap it in a global context while executing but would need to do a bit juggling as you entered and exited context.

This, of course, goes completely south the second you edit the order of observables or computations or add new ones. A VDOM library has the ability to know it has changed to a certain extent and it can throw away the node and revert to the default behaviour of replacing that Component. If we ever failed we could not salvage the rest of the tree. We'd probably want to just restart without preserving the state since we'd never know if we already assigned incorrectly. I'm not clear we could detect failure in many cases. Like if you swapped the order of 2 observables it would probably completely break things but we'd never know unless we checked dependency resolution as well and realized somehow a computation now has different deps than before(id changed when you changed the order). But what is incorrect? The observable value assignment or did you edit that computation on purpose? Basically, to be safe we throw away the whole component and all its descendants. There are many cases we could not safely save state. I do wonder if it could be recoverable.

This is complicated. But I do see one other interesting thing here. The id system and ability to serialize actually could be useful for Dev Tools. It would be an overhead for size and performance, but introducing a dev mode that could be skipped in production builds like React does might be the pattern. You could inspect a component and see its local state. This is all ideas but would definitely be a bit of an undertaking.

---

A webpack loader would look like this, I've dropped your component in as an example (since my version was obviously written in Solid https://github.com/ryansolid/solid-hot-loader). But setting this as a Webpack loader should do the trick:

const loaderUtils = require('loader-utils');

function loadHmr(file) {
  return `
    import { o } from "sinuous";
    import Comp from ${file};

    const obsv = o(Comp),
      Wrapped = (props, ...children) => {
        return () => obsv()(props, () => children);
      };

    export default Wrapped;

    if (module.hot) {
      module.hot.accept(${file}, () => obsv(Comp));
    }
  `;
}

module.exports = function load() {};

module.exports.pitch = function pitch(remainingRequest) {
  const file = loaderUtils.stringifyRequest(this, '!!' + remainingRequest);
  const isProduction = this.minimize || process.env.NODE_ENV === 'production';

  if (this.cacheable) {
    this.cacheable();
  }

  if (isProduction) {
    return `export * from ${file};`;
  }

  return loadHmr(file);
};

This approach relies on the bundler to wrap each component so an approach like this would not work in Parcel anyway as it requires transformation. This does mean you have to be specific with the rules to ensure you are only wrapping Component files. This is an ES6 version, but CJS is possible but need to change to requires.

[luwes]

this would be awesome but sounds pretty complicated 😄

wanted to ping here that I fixed 2 issues that made our memoized version not work.

sinuous/packages/sinuous/h/src/insert.js

Lines 31 to 39 in a0f9e7c

	api.subscribe(function insertContent() {
	current = api.insert(el, value.call({ el }), marker, current, startNode);
	// Save startNode of current. In clearAll() marker.previousSibling
	// is not always accurate if content gets pulled before clearing.
	if (current instanceof Node) {
	startNode = current;
	}
	});

with 0.21.4 this example works like it should with multiple child elements, each component only gets created once. the children of the component are still the same in both components, so it's not possible to have multiple components in the dome at the same time like you would expect. that requires a () => children
https://codesandbox.io/s/dynamic-components-4-u9tu3

edit: actually this does work. unexpected in a good way
https://codesandbox.io/s/dynamic-components-4-l05bu

[theSherwood]

@ryansolid There's a ton to unpack here.

This, of course, goes completely south the second you edit the order of observables or computations or add new ones.

Can that be detected? As in, preserve state in the way you've talked about (with a path) in all instances unless the order of observables/computations has changed? Recreate the subtree if the order of observables has changed? Because that would still be pretty great!

Thanks for the sample webpack plugin. I have no experience with writing anything for bundlers. So that is astonishingly minimal.

This approach relies on the bundler to wrap each component so an approach like this would not work in Parcel anyway as it requires transformation.

I didn't realize that. There's no way to transform code on its way to Parcel? I'm guessing Rollup is more like Webpack in that regard?

This does mean you have to be specific with the rules to ensure you are only wrapping Component files.

I've been meaning to ask about that. How are you handling that for Solid? Does that mean that to use the bundler, only components can ever be default exports from any javascript file?

Once again, thanks for this. This is awesome work.

@luwes Awesome!

Could you explain a little more to me about the performance differences this creates. I'm struggling to create a model of it in my mind.

[luwes]

@theSherwood the difference is that the non-memoized version would still re-create the elements of the component with each swap. the memoized version only creates these once for each component. basically the functions that represent the components are only run once for the memoized version if the arguments stay the same. take a look in the console logs, it will be clear.

[ryansolid]

@theSherwood
(Using JSX since it's a bit less verbose and I'm more familiar with syntax).

function CompA() {
  const count = o(3);
  const name = o("John");
  return <div>
    <span>Clicked {count()} times</span>
    <span>Hello, {name()}<span>
  </div>
}

function CompB() {
  const name = o("John");
  const count = o(3);
  return <div>
    <span>Clicked {count()} times</span>
    <span>Hello, {name()}<span>
  </div>
}

I'm not sure how to tell the difference. You could look at the type of the value but it's unreliable. Using pure pathing it would just assign the wrong state in this scenario. If there is nothing else unique about the call I don't know how to differentiate.

It's possible to do this with either Parcel or Rollup I'm sure as some sort of transformation. It just isn't as simple. Like you could do this with Babel I imagine. Webpack loaders are built so that you return the transformed string makes it pretty easy.

All I did was put all the Components in a "components" folder for starters. Basically though all you need to do is make use of include/exclude paths. It's not ideal but I haven't figured out a different way to do this. It is so much easier to handle this at a modular level with fine-grained approaches.

[theSherwood]

@ryansolid So the problem is doing this statically, right? Or at runtime? Because, I'm wondering if you use the path technique you were talking about, can you also cache the function name (fn.name) or source (fn.toString()). If there is any kind of mismatch, don't try to preserve state. Is that a workable approach?

[ryansolid]

@theSherwood
Which fn? The function I'd be talking about here is o, the generic reactive atom. I'm not sure how we achieve this even at runtime without having users write interesting code (like passing unique names in). If anything we could make a compiler automatically pass in extra debug information without the end-user being any wiser.

If you meant the fn being the component sure we could detect a change but I'm not clear how we could trust anything downstream at that point which pretty much gets us back to where we are right now.

[theSherwood]

@ryansolid Yeah. Sorry, I was unclear. I meant the observable returned from o. But of course that won't work because every observable is the same function. So it basically has to be done through static analysis if it's to be done at all.

[ryansolid]

That's what I got so far. Admittedly I hadn't gotten to that conclusion before this conversation so maybe there are other options. It is within my means with the compiler to pull that off, so that's something. It still seems very complicated, but I will get there at some point I figure as its basically the last feature that keeps Solid from React parity. While not compatible I believe Solid supports more React features currently than even Preact other than HMR. So it is definitely something I'm conscious of. And of course, if I can figure anything else I will share the wealth.

And by all means, keep ideas coming.

[DrSensor]

Hi, anyone knows how to do it using only h function?

h(dynamic(menu))

Seems doesn't work 🤔

(me trying to avoid htm parsing process without using babel)

EDIT:
finally, this works

h([dynamic(menu)])