Skip to content

Commit

Permalink
libsql, local first
Browse files Browse the repository at this point in the history
  • Loading branch information
btholt committed Aug 15, 2024
1 parent 7620e37 commit d60b25e
Show file tree
Hide file tree
Showing 6 changed files with 208 additions and 2 deletions.
3 changes: 3 additions & 0 deletions lessons/07-performance/meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"icon": "gauge-high"
}
46 changes: 44 additions & 2 deletions lessons/09-scaling-sqlite/C-libsql.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,47 @@
---
title: "libsql"
title: "libSQL"
---

TODO
## What is libSQL and why fork SQLite?

[libSQL][libsql] bills itself as "SQLite for modern applications". It is a fork of the SQLite codebase that [Turso][turso] made to make its SQLite offering in the cloud. [SQLite is open source but not open contribution][oss] which makes it hard if you want to take SQLite and extend it further (as the Turso folks found.) In the end they decided to fork SQLite and make their own version on top of it. [The publish a manifesto][manifesto] which is a good read if you're curious.

libSQL is the same in terms all of your SQLite queries will work as-is and many of the extensions you would expect to work do. They've committed to make the delta between the two as low as possible.

Where libSQL differs is a few things:

- libSQL works as a server, more similar to Postgres and MySQL. We'll talk about that in the next section.
- Due to it working more as a server, it has a [user management system][user].
- libSQL works over HTTP which means you'll need a different client to work with it. Luckily Turso maintains a few [including a JavaScript one][js].
- Because it works as a server, it's easy to expose it to the Internet so you can have remote servers.
- They allow you to do more standard [ALTER TABLE][alter] actions.
- A few other things and always adding more. They do aim to be the modern SQLite database.

## sqld

So let's try it. [Install sqld][sqld] first. Then run it with `sqld`. This will start a sqld server on localhost:8080.

Now we can start running queries against its HTTP API. I use [Bruno][bruno] for API requests but Postman or Insomnia are just fine. From here we need to see our sqld server with Chinook. I took the liberty or writing a Postman collection for this

- [Click here][postman] for a Postman-compatible collection of API requests (both Insomnia and Bruno can use Postman-style collections too.)
- [Click here][query] for just the JSON used to query the API.

Run the query to add all the rows to the database. Once you've done that, [clone this repo][repo], run `npm install`, and `npm run dev` to get the app and running. Now we have our same app running on libSQL and sqld!

- We just used the [libsql sqlite3][sqlite3] driver here. It's a drop-in replacement which means we didn't have to rewrite any code, just change the URL.
- There is a better SDK, [the official libSQL JS SDK][sdk]. This one is closer to better-sqlite3 and has promises built-in.
- Beyond that, we're 100% the same app!

[libsql]: https://turso.tech/libsql
[oss]: https://www.sqlite.org/copyright.html
[manifesto]: https://turso.tech/libsql-manifesto
[js]: https://github.com/tursodatabase/libsql-js
[alter]: https://github.com/tursodatabase/libsql/blob/main/libsql-sqlite3/doc/libsql_extensions.md#altering-columns
[sqld]: https://github.com/tursodatabase/libsql/blob/main/docs/BUILD-RUN.md#build-and-run-sqld
[user]: https://github.com/tursodatabase/libsql/blob/main/docs/USER_GUIDE.md
[bruno]: https://www.usebruno.com/
[query]: /public/chinook-sqld.json
[postman]: /public/sqld-collection.json
[repo]: https://github.com/btholt/sqlite-app-libsql
[sqlite3]: https://github.com/libsql/libsql-node-sqlite3
[sdk]: https://github.com/tursodatabase/libsql-js
31 changes: 31 additions & 0 deletions lessons/09-scaling-sqlite/D-local-first.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
One of the amazing things that libSQL can do is thing called "local first" development. This is a confusing name but think of this way: when you're developing locally, you are running at the speed of "local". Everything is happening on the machine so you have no roundtrip latency between your app and your database. As soon as you start connecting to a database, your responses are guaranteed to be hundreds of milliseconds longer, if not more. Any time you hit the network you are working with the speed of light: your data can't travel any faster than the speed of light and your database is at best in the same data center and at worst across the world.

What if we could just load our database onto the same server seamlessly? LiteFS did this as well but this makes it even easier.

```bash
sqld --grpc-listen-addr=127.0.0.1:5001
```

```javascript
import { createClient } from "@libsql/client";

const db = createClient({
url: "file:local-data.db",
syncUrl: "http://localhost:5001",
syncPeriod: 60,
});

const rep = await db.sync();
```

This is enough set up that libSQL via its SDK it will load a local copy of it onto your app server and now you can start querying at the speed of localhost. Pretty compelling, right? Local first can go a lot further but I wanted to give you a very fast intro to it.

I rewrote the invoice endpoint to use this feature under [invoice-local-first.js][js].

You won't a difference now because we were already doing local dev but you would if it was remote!

Quite a few companies are chasing this local-first paradigm and most of them are using SQLite. [Turso][turso] and [Electric SQL][electric] are two to look at if you're interested.

[js]: https://github.com/btholt/sqlite-app-libsql/blob/main/invoice-local-first.js
[turso]: https://docs.turso.tech/introduction
[electric]: https://electric-sql.com/
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
---
title: "Take SQLite to the Cloud"
---

TODO
Loading

0 comments on commit d60b25e

Please sign in to comment.