Suggestions on how to run this within a Lambda environment

[tavosansal]

Hello,
Thank you for this package. Really cool stuff.
Im running in an issue and wondering if anyone else has encountered it.

I am trying to run webR in a lambda environment with esbuild. I am running into the following error:

"Cannot find module '/var/task/webr-worker.js'","stackTrace":["Error: Cannot find module '/var/task/webr-worker.js'","    
at Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)","    
at Module._load (node:internal/modules/cjs/loader:1051:27)","    
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)","    
at MessagePort.<anonymous> (node:internal/main/worker_thread:186:26)","    
at [nodejs.internal.kHybridDispatch] (node:internal/event_target:820:20)","    
at MessagePort.<anonymous> (node:internal/per_context/messageport:23:28)"],"code":"MODULE_NOT_FOUND","requireStack":[]}}

I am bundling my lambda with esbuild using CDK with the default esbuild options for a NodejsFunction. I managed to track this down to this line (although there are other similar ones for different workers).

I took a look at my file after esbuild does the bundling and it looks like only the main file from the package is getting bundled:

// ../node_modules/webr/dist/webr.cjs
var require_webr = __commonJS({
  "../node_modules/webr/dist/webr.cjs"(exports2, module2) {

There are no other entries in the file about webr definitions.

Wondering if anyone has run into this and has any ideas on how to proceed? Appreciate the time.

[tavosansal]

Following up. I was able to create a Lambda layer with the node_modules folder that includes WebR and that worked. So I am thinking it would be something with the configuration of the package that esbuild does not pick up the right parts? I am by no means a package export configuration expert so I will decline to comment further.

[georgestagg]

I don't know much about executing JavaScript in AWS lambda, though I will say generally that it's unlikely that with our current setup we can bundle everything into a single JavaScript file, because we rely on starting a Web Worker to execute R Wasm computation and the worker has it's own script: webr-worker.js.

It looks like this is the file that's getting missed by the bundler, your esbuild script might need to be configured to copy web-worker.js from the webR distribution (in node_modules/webr, also available as downloads on the Releases page of the GitHub repo) to the output directory.

It's not impossible to embed everything into a single file, but would require some setup to tell esbuild to embed the contents of webr-worker.js into webr.cjs/webr.mjs. There are also lots of additional support files that webR relies on finding to startup, including non-JavaScript filesystem images that are required so that R thinks it's running under a standard Unix filesystem. Again, I think embedding those files into a single output would be tricky, but perhaps esbuild can be told to copy them from node_modules/webr/vfs to a reasonable directory as part of bundler output.

I will leave the issue open in case others have any further advice or suggestions to add.

[tavosansal]

Thanks for the reply.
I think the layer approach is ok for what I am trying to do. Technically you can expose the files the webr package needs via esbuild config for lambda. But I found it cumbersome to work with (I am using AWS CDK). So for now I am going to work with the layer.