DuckDB Support

[KyleAMathews]

DuckDB could be an interesting alternative to SQLIte to support as many data-heavy UIs would benefit from its feature-set/speed. It also has a WASM build https://duckdb.org/docs/api/wasm/overview.html

[thruflo]

Hey, thanks, this is a great idea. I think we have one major technical blocker atm which is that DuckDB doesn’t support triggers.

Are you aware of any mechanisms we can use to hook into writes / commits / transactions / a WAL or replication stream?

[KyleAMathews]

I'm not super familiar with DuckDB — they do have a WAL apparently https://duckdb.org/docs/sql/statements/checkpoint.html

Thinking about this more — DuckDB would perhaps be best run alongside SQLite as a read-only copy as it's intended for analytics not transactional. Potentially this could just be a userland concern — load duckdb as needed for a feature and sync data from sqlite in.

[thruflo]

Interesting

[AntonOfTheWoods]

I'm lazy... what are they using for storage with their wasm build? This is the key issue for OLAP with wasm sqlite - IDB-based storage layers are slow, and OLAP is for "big data" (right?). The sync file handle layers are super fast, but in the browser require having the sqlite engine in a web worker. That introduces quite a bit of extra complexity, particularly surrounding concurrent access, etc.

[KyleAMathews]

in 2021 at least on release they skipped on persistence — https://news.ycombinator.com/item?id=29039480 Not sure they've revisited it since

[balegas]

Turning this issue into a feature request discussion

[KyleAMathews]

👋 we've been working the last month on a rebuild of the Electric server over at a temporary repo https://github.com/electric-sql/electric-next/

You can read more about why we made the decision at https://next.electric-sql.com/about

We're really excited about all the new possibilities the new server brings and we hope you'll check it out soon and give us your feedback.

We're now moving the temporary repo back here. As part of that migration we're closing all the old issues, PRs, and discussions. We really appreciate you taking the time to start this discussion! If it's still relevant in the new system, please reopen.