Trame vtk.js/vtk3d-wasm: Single offscreen canvas serving multiple viewports?

[sophonet]

Hi all,

when doing own steps a few months ago with respect to vtk-wasm and combining it with itk-based processing in an own custom WebAssembly, I have observed that having multiple canvases using wasm is almost impossible (at least very limited), since enscripten compiles WebAssembly functionality for just a single canvas ID. There is example VTK wasm code for multiple canvases (with a vtkConeSource) with some hack that steals the GL canvas ID as soon as the mouse cursor enters a different current view. But besides this aspect, also for vtk.js - or more general, for WebGL, I have read that the number of GL canvases per is limited since a GL canvas is somewhat "heavy". Moreover, for several viewing areas showing the same (big) content from different camera positions, it has to be made available/copied to each canvas, increasing the memory footprint and making synchronization of data an additional overhead (in case it changes due to interactions).

Therefore, the approach of e.g. cornerstone3D (which builds on top of vtk.js) is to just create one big hidden WebGL canvas and - by utilizing VTK's viewport functionality - split that into the visible areas, quickly copying rendered viewports from the hidden canvas to visible plain divs on the Webpage on Render() updates. This would be beneficial in terms of performance and would eliminate the need for data duplication. Moreover, I would think that it might also be possible for vtk3d-wasm (and not only vtk.js).

Is there some example code for trame that demonstrates this? If not, would it be possible in general and what would be the approach to realize it? Any other comments?

Thanks,

sophonet

[jourdain]

Are you sure they are not doing the same technic as we do here with vtk.js?

The technic used above can be applied to trame with either wasm/vtk.js but would require some abstraction to make it trivial to use in the context of trame. But Kitware can help you building such infrastructure. If you are interested in funding such effort, feel free to reach out to Kitware so we can discuss details.

[sophonet]

Thanks for the quick reply - I think cornerstone.js indeed follows a different approach, see here. But good to know that there is no example code available right now in trame. Funding is out of scope for me, it is rather an activity I am doing in my spare time - so I will think about next steps myself. Thanks again - and congratulations, I think choosing python as the backend language for writing web apps will be interesting to many people from the scientific community!

[jourdain]

Indeed interesting pointer, thanks!

BTW, you've listed vtk3d-wasm. Did you find it from what I published yesterday? If so you are fast! I didn't even finish cleaning up the repo... ;-)

[sophonet]

Indeed it was a coincidence - since I was looking for a solution for multiple VTK viewports with WebAssembly showing the same dataset... related to the issues mentioned in my original question above (my own VTK-ITK-and-other-libs-wasm build from a couple of months ago) after having tried trame yesterday for the first time. ;-)

[jourdain]

BTW, I added a multi view example with the known limitation of how many 3D context a browser can support.

[sridhar-mani]

hi,
may i know if the vtk wasm and the vtk js taht is downloaded from the unpkg is able to be run independantly. Because when i try to run this in node environment it throws error that it expects 30mb of size from wasm but the wasm is only 8.4 mb

[jourdain]

It is possible that the rendering code expect a browser API and therefore does not work in node. I'm not sure that is the case, but we don't have much to go on with the information provided.

[sridhar-mani]

i used wasmfs to create a virtual file system and added all the required files but still got the same error that the expected file length for the wasm file is 30mb not 8.4 ass downloaded from the link in the repo.

[jourdain]

Still don't know what to tell you.