Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add README for char_count_zig, updated status/roadmap #5

Merged
merged 3 commits into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,11 @@ pgzx is currently under heavy development by the [Xata](https://xata.io) team. I
* [ ] Double list
* [ ] Hash tables
* Development environment
* [ ] Download and vendor Postgres source code
* [ ] Compile extensions against the Postgres source code
* [x] Download and vendor Postgres source code
* [x] Compile example extensions against the Postgres source code
* [x] Build target to run Postgres regression tests
* [ ] Run Zig unit tests in the Postgres environment
* [ ] Provide a standard way to test extensions from separate repos
* Packaging
* [x] Add support for Zig packaging

Expand Down Expand Up @@ -201,7 +203,7 @@ $ psql -U postgres -c 'select version()'

This project has a few example extensions. We will install and test the `char_count_zig` extension next:

```
```sh
$ cd examples/char_count_zig
$ zig build -freference-trace -p $PG_HOME
$ psql -U postgres -c 'CREATE EXTENSION char_count_zig;'
Expand Down
109 changes: 109 additions & 0 deletions examples/char_count_zig/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# char_count_zig - Minimal PostgreSQL extension using Zig

This is a sample PostgreSQL extension written in Zig. It provides a function `char_count_zig` that counts the number of occurrences of a character in a string. The code is a port using pgzx of the sample `char_count` C extension from [this tutorial](https://www.highgo.ca/2019/10/01/a-guide-to-create-user-defined-extension-modules-to-postgres/).

## Functionality

The function `char_count_zig` takes two arguments: a string and a character. It returns the number of occurrences of the character in the string:

```sql
SELECT char_count_zig('Hello, World', 'o');
char_count_zig
----------------
2
(1 row)
```

## Running

To test the extension, follow first the development shell instructions in the [pgzx README][pgzx_Development]. The following commands assume you are in the nix shell (run `nix develop`).

Run in the folder of the extension:

```sh
cd examples/char_count_zig
zig build -freference-trace -p $PG_HOME
```

This will build the extension and install the extension in the Postgres instance.

Then, connect to the Postgres instance:

```sh
psql -U postgres
```
tsg marked this conversation as resolved.
Show resolved Hide resolved

At the Postgres prompt, create the extension:

```sql
CREATE EXTENSION char_count_zig;
```
tsg marked this conversation as resolved.
Show resolved Hide resolved

## Code walkthrough

### Control files

The overall structure of the extension looks very similar to a C extension:

```
├── extension
│ ├── char_count_zig--0.1.sql
│ └── char_count_zig.control
```

The `extension` folder contains the control files, which are used by Postgres to manage the extension. The `char_count_zig.control` file contains metadata about the extension, such as its name and version. The `char_count_zig--0.1.sql` file contains the SQL commands to create and drop the extension.

### Zig code

The `main.zig` file starts with the following `comptime` block:

```zig
comptime {
pgzx.PG_MODULE_MAGIC();

pgzx.PG_FUNCTION_V1("char_count_zig", char_count_zig);
}
```

The [pgzx.PG_MODULE_MAGIC][docs_PG_MODULE_MAGIC] function returns an exported `PG_MAGIC` struct that PostgreSQL uses to recognize the library as a Postgres extension.

The [pgzx.PG_FUNCTION_V1][docs_PG_FUNCTION_V1] macro defines the `char_count_zig` function as a Postgres function. This function does the heavy lifting of deserializing the input arguments and transforming them in Zig slices.

This means the implementation of the `char_count_zig` function is quite simple:

```zig
fn char_count_zig(input_text: []const u8, target_char: []const u8) !u32 {
if (target_char.len > 1) {
return pgzx.elog.Error(@src(), "Target char is more than one byte", .{});
}

pgzx.elog.Info(@src(), "input_text: {s}\n", .{input_text});
pgzx.elog.Info(@src(), "target_char: {s}\n", .{target_char});
pgzx.elog.Info(@src(), "Target char len: {}\n", .{target_char.len});

var count: u32 = 0;
for (input_text) |char| {
if (char == target_char[0]) {
count += 1;
}
}
return count;
}
```

In the above, note the use of [pgzx.elog.Error][docs_Error] and [pgzx.elog.Info][docs_Info] to report errors and info back to the user. The rest of the function is idiomatic Zig code.

### Testing

The extensions contains regression tests using the `pg_regress` tool, see the `sql` and `expected` folders. To run the regression tests, use the following command:

```sh
zig build pg_regress
```

[pgzx_Development]: https://github.com/xataio/pgzx/tree/main?tab=readme-ov-file#develpment-shell-and-local-installation
[docs_PG_MODULE_MAGIC]: https://xataio.github.io/pgzx/#A;pgzx:fmgr.PG_MAGIC
[docs_PG_FUNCTION_V1]: https://xataio.github.io/pgzx/#A;pgzx:PG_FUNCTION_V1
[docs_Error]: https://xataio.github.io/pgzx/#A;pgzx:elog.Error
[docs_Info]: https://xataio.github.io/pgzx/#A;pgzx:elog.Info

10 changes: 0 additions & 10 deletions examples/char_count_zig/src/main.zig
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,3 @@ fn char_count_zig(input_text: []const u8, target_char: []const u8) !u32 {
}
return count;
}

// TODO:
// how can we compile these into the lib and run them from within postgres?
test "char_count_zig happy path" {
const input_text = "Hello World";
const target_char = "l";
const expected_count: u32 = 3;
const actual_count = try char_count_zig(input_text, target_char);
try std.testing.expectEqual(expected_count, actual_count);
}
43 changes: 42 additions & 1 deletion examples/pgaudit_zig/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,45 @@ It logs, as json:

Note that tables (relations in internal Postgres terminology) accessed are logged in an array.

## Running

To test the extenstion, follow first the development shell instructions from the [pgzx README][pgzx_Development]. The following commands assume you are in the nix shell (run `nix develop`).

Run in the folder of the extension:

```sh
cd examples/pgaudit_zig
zig build -freference-trace -p $PG_HOME
```

This will build the extension and install the extension in the Postgres instance.

Then, connect to the Postgres instance:

```sh
psql -U postgres
```

At the Postgres prompt, load the library and create the extension:

```sql
LOAD 'pg_audit_zig.dylib';
CREATE EXTENSION pgaudit_zig;
```

Now, create a table and run a query:

```sql
CREATE TABLE foo(id text);
SELECT * FROM foo;
```

You should see a set of logs, including:

```
NOTICE: Internal [debug] default pgaudit_zig: logAuditEvent: {"operation": "CMD_SELECT", "relations": [{"relOid": 16403, "relname": "foo", "namespaceOid": 2200, "relnamespaceName": "public"}], "commandText": "SELECT * FROM foo;"}
```

## Code walkthrough

Here are the key code elements and how they make use of the pgzx helpers. When reviewing the code, it is helpful to have a look at the C code in the pgaudit extension, as well as at the Postgres source code.
Expand Down Expand Up @@ -223,4 +262,6 @@ The `executorCheckPermsHook` function receives the list of tables (`rangeTable`)
while (it.next()) |rte| {
...
}
```
```

[pgzx_Development]: https://github.com/xataio/pgzx/tree/main?tab=readme-ov-file#develpment-shell-and-local-installation