SQLite Observability?

[KyleAMathews]

I was thinking about this a bit today. With all the sqlite-in-client projects out there, a general observability collector for monitoring queries (frequency/duration/etc), size on client of tables, indexdb usage, etc. would be really useful. Googling around, it doesn't seem like sqlite gives you the tools itself to monitor it (though I'd be happy to be proven wrong) so there'd probably need to be a generic sqlite pass-through wrapper which could monitor the instance?

Anyone have any thoughts on this?

[thruflo]

This would be great. I haven’t come across something that exists. I wonder how generic it could be and how best to implement and package it.

There are the status methods as per http://www.sqlite.org/c3ref/c_status_malloc_count.html If drivers have a transaction queue that could be a place to monitor query execution. Maybe in the browser the VFS is best place to monitor storage.

[KyleAMathews]

await navigator.storage.estimate() gets you the indexeddb usage e.g.

How do you run the status methods in the browser?

Yeah, maybe electricsql could just expose a hook or two for observing queries/usage — you're probably right that making this truly generic isn't possible/worth it (though the underlying methods should be easily portable).

So there could be a general library that just hooks into each sqlite system.

[thruflo]

The c-level functions need to be exposed by the driver. So the native module or WASM interface need to wrap and expose functions to the JS.

[KyleAMathews]

👋 we've been working the last month on a rebuild of the Electric server over at a temporary repo https://github.com/electric-sql/electric-next/

You can read more about why we made the decision at https://next.electric-sql.com/about

We're really excited about all the new possibilities the new server brings and we hope you'll check it out soon and give us your feedback.

We're now moving the temporary repo back here. As part of that migration we're closing all the old issues, PRs, and discussions. We really appreciate you taking the time to start this discussion! If it's still relevant in the new system, please reopen.