Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support V8 C++ APIs for "nan" addons and other packages to work #4290

Open
51 of 83 tasks
xhyrom opened this issue Aug 24, 2023 · 61 comments
Open
51 of 83 tasks

Support V8 C++ APIs for "nan" addons and other packages to work #4290

xhyrom opened this issue Aug 24, 2023 · 61 comments
Assignees
Labels
tracking An umbrella issue for tracking big features

Comments

@xhyrom
Copy link
Collaborator

xhyrom commented Aug 24, 2023

Affected packages

  • better-sqlite3
  • node-canvas@v2 (node-canvas v3 is supported)
  • midi
  • libxmljs
  • bignum
  • zeromq
  • node-libcurl
  • bson-ext
  • heapdump
  • @datadog/native-metrics
  • tiny-secp256k1
  • tree-sitter-c-sharp
  • chacha-native
  • @sentry/profiling-node
  • pdfjs-dist
  • gl
  • leveldown / pouchdb
  • sleep
  • cpu-features
  • odbc
  • msgpackr
  • re2
  • node-pty
  • i2c

Functions implemented

  • v8::Isolate::GetCurrent()
  • v8::Isolate::TryGetCurrent()
  • v8::Isolate::GetCurrentContext()
  • v8::Number::New(v8::Isolate*, double)
  • v8::Number::Value() const
  • v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int)
  • v8::String::WriteUtf8(v8::Isolate*, char*, int, int*, int) const
  • v8::api_internal::ToLocalEmpty()
  • v8::String::Length() const
  • v8::External::New(v8::Isolate*, void*)
  • v8::External::Value() const
  • v8::Object::New(v8::Isolate*)
  • v8::Object::Set(v8::Local<v8::Context>, v8::Local<v8::Value>, v8::Local<v8::Value>)
  • v8::Object::SetInternalField(int, v8::Local<v8::Data>)
  • v8::Object::SlowGetInternalField(int)
  • v8::HandleScope::HandleScope(v8::Isolate*)
  • v8::HandleScope::~HandleScope()
  • v8::FunctionTemplate::GetFunction(v8::Local<v8::Context>)
  • v8::FunctionTemplate::New(v8::Isolate*, void (*)(v8::FunctionCallbackInfo<v8::Value> const&), v8::Local<v8::Value>, v8::Local<v8::Signature>, int, v8::ConstructorBehavior, v8::SideEffectType, v8::CFunction const*, unsigned short, unsigned short, unsigned short) (only when all parameters past data are their default values)
  • v8::ObjectTemplate::NewInstance(v8::Local<v8::Context>)
  • v8::ObjectTemplate::SetInternalFieldCount(int)
  • v8::ObjectTemplate::InternalFieldCount() const
  • v8::ObjectTemplate::New(v8::Isolate*, v8::Local<v8::FunctionTemplate>) (only when no FunctionTemplate is provided)
  • v8::EscapableHandleScopeBase::EscapeSlot(unsigned long*)
  • v8::EscapableHandleScopeBase::EscapableHandleScopeBase(v8::Isolate*)
  • node_module_register
  • v8::Function::SetName(v8::Local<v8::String>)
  • v8::Value::IsBoolean() const
  • v8::Boolean::Value() const
  • v8::Value::FullIsTrue() const
  • v8::Value::FullIsFalse() const
  • v8::EscapableHandleScope::EscapableHandleScope(v8::Isolate*)
  • v8::EscapableHandleScope::~EscapableHandleScope()
  • v8::Value::IsObject() const
  • v8::Value::IsNumber() const
  • v8::Value::IsUint32() const
  • v8::Value::Uint32Value(v8::Local<v8::Context>) const
  • v8::Value::IsUndefined() const
  • v8::Value::IsNull() const
  • v8::Value::IsNullOrUndefined() const
  • v8::Value::IsTrue() const
  • v8::Value::IsFalse() const
  • v8::Value::IsString() const
  • v8::Boolean::New(v8::Isolate*, bool)
  • v8::Object::GetInternalField(int)
  • node::AddEnvironmentCleanupHook(v8::Isolate*, void (*)(void*), void*)
  • node::RemoveEnvironmentCleanupHook(v8::Isolate*, void (*)(void*), void*)
  • v8::HandleScope::CreateHandle(v8::internal::Isolate*, unsigned long)
  • v8::internal::IsolateFromNeverReadOnlySpaceObject(unsigned long)
  • v8::Array::New(v8::Isolate*, v8::Local<v8::Value>*, unsigned long)
  • v8::Object::Get(v8::Local<v8::Context>, v8::Local<v8::Value>)

Major areas of unimplemented functionality

  • Array
  • ObjectTemplate with a constructor function
  • Less-simple cases of FunctionTemplate
  • ArrayBuffer
  • bun install fixes (need to pass node-gyp the right version)
  • Doubles
  • UTF-16 strings
  • Persistent handles

What version of Bun is running?

0.8.0+ad326b77342dd3d8585a30b7da803d32f9c11fe2

What platform is your computer?

Linux 6.4.10-artix1-1 x86_64 unknown

What steps can reproduce the bug?

import Database from 'better-sqlite3';
const db = new Database('foobar.db', {});

console.log(db.prepare("CREATE TABLE TEST").run());

What is the expected behavior?

Run as normally

What do you see instead?

/usr/bin/bun: symbol lookup error: /home/hyro/Workspace/booo/node_modules/better-sqlite3/build/Release/better_sqlite3.node: undefined symbol: node_module_register

Additional information

Related thread: 1144181146809270332
Message: https://discord.com/channels/876711213126520882/1144181146809270332/1144184224006557747

All versions are affected

@xhyrom xhyrom added the bug Something isn't working label Aug 24, 2023
@floydjones1
Copy link

This also happens for node-libcurl

@xhyrom xhyrom changed the title better-sqlite3 doesn't work, missing node_module_register multiple modules doesn't work, missing node_module_register Aug 24, 2023
@xhyrom xhyrom changed the title multiple modules doesn't work, missing node_module_register multiple modules don't work, missing node_module_register Aug 24, 2023
@xhyrom xhyrom changed the title multiple modules don't work, missing node_module_register node_module_register is not implemented Aug 25, 2023
@xhyrom xhyrom changed the title node_module_register is not implemented node_module_register is not implemented (better_sqlite3) Aug 25, 2023
@robobun robobun added bun:sqlite Something to do with bun:sqlite napi Compatibility with the native layer of Node.js and removed bun:sqlite Something to do with bun:sqlite napi Compatibility with the native layer of Node.js labels Sep 10, 2023
@issadarkthing
Copy link

I also encounter this error

@Zykatious
Copy link

this also happens for bson-ext

@autoscatto
Copy link

also for heapdump

@vroudge
Copy link

vroudge commented Sep 15, 2023

same for @datadog/native-metrics

@uditdc
Copy link

uditdc commented Sep 16, 2023

Likewise for tiny-secp256k1

@ozyman42
Copy link

Also for cdktf-cli #5306

@Electroid Electroid added the napi Compatibility with the native layer of Node.js label Sep 24, 2023
@obedm503
Copy link

Also happens with tree-sitter-c-sharp

@Scitz0
Copy link

Scitz0 commented Sep 28, 2023

Same error for chacha-native

@metawrap-dev
Copy link

same for node-canvas

@4i8
Copy link

4i8 commented Oct 28, 2023

bun: symbol lookup error: /root/rut/node_modules/canvas/build/Release/canvas.node: undefined symbol: node_module_register
same for canvas

@edi9999
Copy link

edi9999 commented Oct 28, 2023

Same for libxmljs

@floydjones1
Copy link

floydjones1 commented Nov 13, 2023

How can we get a little more attention to this. This is holding me back from using bun in production. 😢
node-libcurl

bun: symbol lookup error: /home/.../node_modules/node-libcurl/lib/binding/node_libcurl.node: undefined symbol: node_module_register

@M-Gonzalo
Copy link

bun: symbol lookup error: /root/rut/node_modules/canvas/build/Release/canvas.node: undefined symbol: node_module_register same for canvas

me too

@frani
Copy link

frani commented Nov 26, 2023

similar error:

bun: symbol lookup error: /usr/src/app/node_modules/@sentry/profiling-node/lib/sentry_cpu_profiler-linux-x64-glibc-115.node: undefined symbol: _ZN2v87Isolate10GetCurrentEv

@xbura1han
Copy link

xbura1han commented Nov 27, 2023

Also happening for Sentry & bcrypt
Sentry code:
Code:

const Sentry = require('@sentry/bun');
const { ProfilingIntegration } = require("@sentry/profiling-node")

Error:

bun: symbol lookup error: /root/projects/userBackend/node_modules/@sentry/profiling-node/lib/sentry_cpu_profiler-linux-x64-glibc-115.node: undefined symbol: _ZN2v87Isolate10GetCurrentEv

@dayblox
Copy link

dayblox commented Aug 8, 2024

same for cap

@190n
Copy link
Contributor

190n commented Aug 15, 2024

Now that #12821 is merged, the next version of Bun will include some of the first working V8 functions. See the top of this issue for more detailed status. Packages more complex than cpu-features or sleep are likely to still run into cases I have not covered yet, and there may be functions where we're never able to bridge the gap between how V8 and JavaScriptCore represent data, but we're excited to have the first version of this functionality landing soon for testing.

@chriskrogh
Copy link

@190n thank you for the hard work! based on #5835 (comment) this gets us closer to supporting node-canvas which unblocks jsdom! we can't wait to use bun at @Faire

@Jarred-Sumner
Copy link
Collaborator

@190n thank you for the hard work! based on #5835 (comment) this gets us closer to supporting node-canvas which unblocks jsdom! we can't wait to use bun at @Faire

@chriskrogh you can use canvas@next instead of the latest version and it should work, as that uses napi instead of v8 APIs

@chriskrogh
Copy link

@Jarred-Sumner it works! 🥳 no more canvas issues on bun install!!!
we only have 2 more offending packages:

  • llamaindex
    • tries to link @datastax/astra-db-ts
    • we don't have that many usages. might try to switch to ai by vercel
  • @grpc/grpc-js
    • this one is actually used by most of our app servers, so we might need to wait for support here

but after removing those two, I was able to get a clean install which was lightning fast! thanks again for all the progress here! hopefully grpc doesn't take that much more? 🤞

@KilianB
Copy link
Contributor

KilianB commented Aug 22, 2024

Is grpc related to v8 incompatibilities? I thought it was blocked by https2 server support

@rgillan
Copy link

rgillan commented Aug 28, 2024

@Jarred-Sumner et al, please note I've deleted a comment re ssh2 server issues as it is not related to the V8 APIs, will raise a separate ticket shortly (have identified a potential issue)

@jerrywoo96
Copy link

Is there a list or is anyone able to make a list of packages that are not yet resolved or implemented? This issue thread is becoming too large to keep track of which packages are still outstanding.

@190n
Copy link
Contributor

190n commented Aug 29, 2024

That exists in the top comment on this issue, but it probably deserves to be the most prominent part instead of the list of functions. I'll move it to the top.

@rgillan
Copy link

rgillan commented Aug 29, 2024

@190n if you could add msgpackr to the list as well please, although will be ok once Array::New is done I expect. Really appreciate all your efforts on this.

@190n
Copy link
Contributor

190n commented Aug 29, 2024

Right, thanks for the reminder.

@etcd
Copy link

etcd commented Sep 2, 2024

@190n thoughts on also adding re2 to the list (given that issue #12560 was marked as a duplicate of this one)?

@karlhorky
Copy link

@nachat-ayoub
Copy link

how much time could it take to support the "gl" module?

@adworacz
Copy link

adworacz commented Oct 4, 2024

@190n if you could add msgpackr to the list as well please, although will be ok once Array::New is done I expect. Really appreciate all your efforts on this.

For anyone else who's confused by this - technically msgpackr does support Bun, it's just the "native acceleration" feature that msgpackr has that won't work.

@rgillan
Copy link

rgillan commented Oct 5, 2024

@190n if you could add msgpackr to the list as well please, although will be ok once Array::New is done I expect. Really appreciate all your efforts on this.

For anyone else who's confused by this - technically msgpackr does support Bun, it's just the "native acceleration" feature that msgpackr has that won't work.

so there's been investigation by @190n back in July when we first found this, and you can find more in #12754 with their findings. One of the great things with msgpackr is the native acceleration, and that needs v8::Array::New

@190n
Copy link
Contributor

190n commented Oct 5, 2024

One of the great things with msgpackr is the native acceleration, and that needs v8::Array::New

As an update on that, I'm not sure if we'll be able to support their specific use of v8::Array::New, because they're using it by casting an array of NAPI values into an array of V8 values. That only works if NAPI values are V8 values, which is true in Node but not Bun. So while we can get v8::Array::New working for packages using the V8 API normally, for msgpackr specifically I think the solution would be more along the lines of finding out why they detect us as Node and stopping that from happening. So ideally their native acceleration can still work in Bun but it should be using exclusively NAPI functions instead of mixing NAPI and V8.

@Frtrillo
Copy link

Frtrillo commented Oct 16, 2024

Same issue for @newrelic/native-metrics and symbol lookup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tracking An umbrella issue for tracking big features
Projects
None yet
Development

No branches or pull requests