Housaku (豊作 「ほうさく」)

Housaku is a personal search engine built on top of SQLite's FTS5 that lets you search your documents and favorite feeds in one place.

Housaku is currently in early development, so you can expect some incompatible changes and other minor issues when updating. Once version v1.0.0 is reached, my goal is to focus on stability and avoiding breaking changes.

Features

• Support for the following file formats:
  • Plain text files.
  • Markdown.
  • CSV.
  • PDF.
  • EPUB.
  • DOCX.
  • XLSX.
  • PPTX.
• Support for RSS/Atom feeds parsing and indexing.
• Parallel file processing.
• Concurrent feed processing.
• Web UI.
• Modern TUI.
• Easy-to-use CLI.
• Relevant results powered by the BM25 algorithm.
• Automatically updates files that had been modified since the last indexing session.

Support for other file formats like ODT is coming.

Technologies used

• aiohttp.
• click.
• FastAPI.
• feedparser.
• pydantic.
• pymupdf
• rich.
• SQLite.
• SQLite's FTS5 extension.
• textual.

Why

Every time I need to search for something, I find myself feeling a bit frustrated with the experience. Web search results have become increasingly inconsistent, and I often spend more time looking for what I truly want or need than I did before. Searching my personal files is also not a great experience. While programs like Obsidian, which I use for the majority of my personal notes, are somewhat better, the experience is still slower, and the results rely on a simple pattern matching. Additionally, searching for a specific piece of content in documents outside my vault, such as my university notes, PDFs, presentations, or my personal library of books, becomes nearly impossible. That is why I decided to build Housaku. I wanted an easy-to-use and easy-to-maintain program that would allow me to search all my documents and favorite feeds from a single location without having to worry about format or location. I also wanted my results to be relevant to my search queries, not just based on basic pattern matching or a regular expression.

Installation

At the moment Housaku is only compatible with Python 3.12.* versions.

Using uv

The recommended way of installing Housaku is by using uv:

uv tool install housaku

Now you just need to run:

housaku --help

To upgrade, use:

uv tool upgrade housaku

# Or

uv tool upgrade housaku --reinstall

Using pipx

To install Housaku using pipx, simply run:

pipx install housaku

Via pip

You can also install Housaku using pip, but the exact command will depend on how your environment is set up. In this case, the command should look something like this:

python3 -m pip install housaku

Configuration

Before you start using Housaku, the first step is to edit the config.toml file located at your $XDG_CONFIG_HOME/housaku/config.toml. This file is generated automatically the first time you run housaku and will look something like this:

# Welcome! This is the configuration file for Housaku.

[files]
# Directories to include for indexing.
# Example: include = ["/home/<user>/documents/notes"]
include = []

# Patterns to exclude from the indexing
# Example: exclude = ["*.tmp", "backup", "*.png"]
exclude = []

[feeds]
# List of RSS/Atom feeds to index
# Example: urls = ["https://example.com/feed", "https://anotherexample.com/rss"]
urls = []

The folder that holds the configuration file as well as the SQLite database is determined by the get_app_dir utility. You can read more about it here.

An easy way to open your config.toml file is to run the following command:

housaku config

Theming

You can also adapt the default theme of Housaku, which is based on the Dracula theme, by adding the following section to your config.toml file and updating the values of the following variables:

[theme]
primary = "#ff79c6"
foreground = "#f8f8f2"
background= "#1E1F29"
warning= "#ffb86c"
error= "#ff5555"
success= "#50fa7b"
accent= "#bd93f9"
surface= "#44475a"
boost= "#44475a"

Usage

Help

The best way to see which commands are available is to run housaku with the --help flag.

housaku --help

You can also learn more about what a specific command does by running:

housaku [command] --help

# Like for example

housaku index --help

Config

The config command is a very simple command that just open the config.toml file using the default editor.

housaku config

Index

After you have configured the list of directories containing the documents you want to index, as well as the list of feeds from which you want to fetch the posts, you can run:

housaku index

Note that you don't need to specify both files and feeds to start indexing.

You can also change the number of threads being used when indexing your files and documents:

housaku index -t 8

I recommend to stick with the default number of threads.

At the moment, indexing files is done in parallel, which makes the process faster but also introduces some complications. For example, canceling the indexing process is not recommended at the moment. My advice is to index small folders if you want to test the tool, or simply allow the indexing process to finish. In my case, I have about 7,000 documents, including markdown files, PDF, and EPUB files, as well as a large list of approximately 150 feeds. The entire process takes about 10 to 15 minutes.

Search

The search command

The simplest way to start searching your documents and posts is by using the search command:

houskau searh --query "Django AND Postgres"

You can also limit the number of results by using the --limit option which, by default, is set to 10:

housaku search --query "Django AND Postgres" --limit 20

If you don't specify a query using the --query/-q options you will be prompted to enter one.

You can learn more about the query syntax here.

Using the TUI

My favorite and recommended way to search is by using the TUI. To start it, just run:

housaku tui

To exit the TUI just press ctrl + q, and to open a search result, press Enter while the result is highlighted.

Using the Web UI

Housaku also has a very simple Web UI that you can access by running:

housaku web

The default port is 4242.

This searching method have some limitations. For example, you can't open results that link to your personal files. In the future, I will try to solve this limitations, but for now please keep this in mind.

vacuum and purge

The vacuum command is used to optimize the SQLite database by reclaiming unused space and improving performance. To run the vacuum command, simply execute:

housaku vacuum

The purge command is used to completely clear all data from the database. This command is useful when you want to reset the database to its initial state.

housaku purge

Be careful before using both of these commands since they will have a direct impact on the data you hold in your database.

Contributing

Contributions are welcomed! If you have any suggestions feel free to open an issue.