scaling sqlite

lessons/03-basic-sql/meta.json

@@ -1,4 +1,4 @@
 {
     "title": "Basic SQL",
-    "icon": "code"
-}
+    "icon": "database"
+}

lessons/08-json/C-jsonb.md

@@ -1,3 +1,7 @@
+---
+title: "jsonb"
+---
+
 So we've been doing everything with the JSON functions but in reality we really want to use JSONB. It's a more compact way to represent JSON and it's faster to use. It makes everything a little harder to read but otherwise is just better.
 
 ```sql

lessons/08-json/meta.json

@@ -1,4 +1,4 @@
 {
   "title": "JSON",
-  "icon": "package"
+  "icon": "file-code"
 }

lessons/09-scaling-sqlite/A-backups.md

@@ -0,0 +1,46 @@
+A lot of fantastic tooling has been built around SQLite, and in the spirit of the project they've been made straightforward and "lite." One of the most interesting ones in my opinion is [Litestream][litestream]. It allows you with very little effort to be able to stream point-in-time backups to somewhere else. This can be incredibly useful as this provides some security that if you have a bad query wipe out data you can restore it from an earlier version.
+
+We're going to quickly set this up using Docker. This isn't a Docker course so just stick with me. If you want to more about Docker, I teach a [pretty cool course on it for Frontend Masters][containers] as well. If you like this course, I try to take the same approach to demystifying containers.
+
+We're going to be using [MinIO][minio] to stream our backups too. You can think of MinIO like a mini [AWS S3][s3] bucket that runs on your computer.
+
+Make sure that you have [Litestream][install] and [Docker][docker] installed.
+
+Now let's get our MinIO container running.
+
+```bash
+docker run -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"
+```
+
+Next go to your local instance of MinIO running at [http://localhost:9001/]() and log in with the default username and password, minioadmin / minioadmin.
+
+Click the "Create Bucket" and call it "chinook-backup". Click into the bucket and make sure it's set to be "Public" instead of "Private".
+
+Next, from where-ever your database folder is, run
+
+```bash
+export LITESTREAM_ACCESS_KEY_ID=minioadmin
+export LITESTREAM_SECRET_ACCESS_KEY=minioadmin
+
+# be sure to replace data.db with something else if you called the file something else
+litestream replicate data.db s3://chinook-backup.localhost:9000/data.db
+```
+
+DONE! You are now streaming your backups to the bucket. As you can imagine, instead of a local MinIO bucket, you can stream it to a cloud bucket like S3 or Azure Blob Storage. Backing up SQLite is as easy as that!
+
+Let's go ahead and do a restore!
+
+```bash
+export LITESTREAM_ACCESS_KEY_ID=minioadmin
+export LITESTREAM_SECRET_ACCESS_KEY=minioadmin
+litestream restore -o data2.db s3://chinook-backup.localhost:9000/data.db
+```
+
+And just like that we were able to pull down the most recent copy of your data, no problem.
+
+[litestream]: https://litestream.io/
+[containers]: https://frontendmasters.com/teachers/brian-holt/
+[minio]: https://min.io/
+[s3]: https://aws.amazon.com/s3/
+[install]: https://litestream.io/install/
+[docker]: https://www.docker.com/products/docker-desktop/

lessons/09-scaling-sqlite/B-replication.md

@@ -0,0 +1,43 @@
+Okay, so we've been building locally with SQLite and most people end here. Now that they've built their app as big as they want to locally, they'll move to Postgres or MySQL. And frankly this is a totally acceptable thing to do if you are so inclined. I've been using Postgres myself for years like this. However I think this is sort of a vim thing: I've used Postgres for so many years and know it so well that I already know well how to use it (like vim users, who have already suffered through the hard part of learning.)
+
+In any case, you no longer have to switch off of SQLite ever if you are so inclined. There is now something called [litefs][litefs]. Litefs is a tool that simulates a virtual filesystem and then replicates it across machines. In otherwords, it tricks SQLite into thinking it's just reading and writing to a file and underneath it is replicating it out to other replicas.
+
+You end up with a primary node that you can read and write to and secondary nodes that you can write to. It's a smart, elegant system that has some great upside in how simple it is to manage and how simple SQLite is, and how fast it is since your database is running on the same node as your app so reads are local-speed instead of having to go over a network.
+
+The downside is that your secondary nodes do have a bit of lag in receiving updates. Normally it's fast enough for most apps that this eventual consistency isn't a problem (it's usually like 200ms, depending on network speed and other things.) However if you're doing things were strong consistency is essential, this probably not your solution.
+
+You're also using SQLite, for all the good and bad that means. That are lots of advantages that other databases have, so just keep in that mind.
+
+Okay, so, let's adapt our little project and make it work in a distributed fashion. [Clone this repo][repo] and let's explore what's going on here.
+
+- This already pre-completed so no code needed to write here.
+- This is a [Docker Compose project][compose]. This means it will orchestrate multiple containers together to get this project. If you want to learn more, [watch my segment on it on Frontend Masters][compose-video].
+- In our case we'll have an NGINX container that will act as ingress for traffic and also round-robin the connections to our primary and secondary nodes.
+- We have two types of app nodes. The primary node (which there were only ever be one) has the SQLite node that you can write to. The secondary node(s) will have read only replica copies. You could have many of these but for simplicity's sake I hard coded it to be one copy
+- [Notice][db-connection] that I'm connecting to `/litefs/data.db`. This is the virtual filesystem. As long as we're doing that, we'll be connecting to the distributed SQLite database.
+- Feel free to look at the Dockerfile, docker-compose.yml and etc/litefs.yml. Those are probably the most interesting config files.
+
+Let's run it. Run
+
+```bash
+docker compose up --build
+```
+
+You should be able to see the app running at [http://localhost:8080](http://localhost:8080).
+
+> 🚨 If you see a weird SQLITE_ERROR, errno 1 error, delete you node_modules on your local computer and re-run Docker so that it builds the container again.
+
+Let's try connecting to the nodes themselves.
+
+```bash
+docker exec -it sqlite-project-litefs-primary-1 sqlite3 /litefs/data.db
+docker exec -it sqlite-project-litefs-replica1-1 sqlite3 /litefs/data.db
+```
+
+Pretty cool right? We're now working with a distributed SQLite database!!
+
+[litefs]: https://github.com/superfly/litefs
+[repo]: https://github.com/btholt/sqlite-app-litefs
+[db-connection]: https://github.com/btholt/sqlite-app-litefs/blob/main/invoice.js#L3
+[compose]: https://docs.docker.com/compose/
+[compose-video]: https://frontendmasters.com/courses/complete-intro-containers-v2/docker-compose/

lessons/09-scaling-sqlite/C-libsql.md

@@ -0,0 +1,5 @@
+---
+title: "libsql"
+---
+
+TODO

lessons/09-scaling-sqlite/D-take-sqlite-to-the-cloud.md

@@ -0,0 +1,3 @@
+---
+title: "Take SQLite to the Cloud"
+---

lessons/09-scaling-sqlite/meta.json

@@ -0,0 +1,4 @@
+{
+  "title": "Scaling SQLite",
+  "icon": "server"
+}