Skip to content

Commit

Permalink
Update README and move codegen to cmd directory.
Browse files Browse the repository at this point in the history
  • Loading branch information
KevinJoiner committed Mar 4, 2024
1 parent c4c2d41 commit 85c75a6
Show file tree
Hide file tree
Showing 4 changed files with 37 additions and 1 deletion.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
*.so
*.dylib

codegen

# Test binary, built with `go test -c`
*.test

Expand Down
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,37 @@ Welcome to the **Model Garage**, a Golang toolkit for managing and working with

3. **Usage**:
Explore the documentation to start using Model Garage in your project.


## Repo structure

### Codegen
The `codegen` directory contains the code generation tool for creating models from vspec CSV schemas. The tool is a standalone application that can be run from the command line.

Example usage:
```bash
go run ./cmd/codegen -output=./pkg/vss -spec=./schema/vss_rel_4.2-DIMO.csv -migrations=./schema/migrations.json -package=vss
package main
```

#### Generation Info
The Model generation is handled by packages in `internal/codegen`. They are responsible for creating Go structs, Clickhouse tables, and conversion functions from the vspec CSV schema and migrations file.

**Vspec Schema** The vspec schema is a CSV file that contains the signal definitions for the model. This schema is generated using vss-tools in this https://github.com/KevinJoiner/DIMO-VSS repository.

**Migrations File** The migrations file is a JSON file that contains the signal definitions that are to be included in the model. This file is manually created. With the following structure:

- **vspecName**: The name of the signal field in the vspec. Only fields specified in the vspec will be included in the model.

- **conversion**: (optional) Details about the conversion from the original data to the vspec field. If not specified, the conversion is assumed to be a direct copy.
- **originalName**: The original name of the field in the data.

- **originalType**: (optional) The original data type of the field. If not specified, the original type is assumed to be the same as the vspec type.
##### Generation Process
1. First, the vspec CSV schema and migrations file are parsed.
2. Then a struct is created for each signal in the vpsec schema that is specified in the migrations file. With Clickhouse and JSON tags for each field. The CH and JSON names are the same as the vspec except `.` are replaced with `_`.
3. Next, a Clickhouse table is created for the struct. The table name is the same as the package name. The table is created with the same fields as the struct with corresponding Clickhouse types.
4. Finally, conversion functions are created for each struct. These functions convert the original data in the form of a JSON document to the struct.

**Conversion Functions**
For each field, a conversion function is created. If a conversion is specified in the migrations file, the conversion function will use the specified conversion. If no conversion is specified, the conversion info function will assume a direct copy. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten.
File renamed without changes.
2 changes: 1 addition & 1 deletion generate.go
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
//go:generate go run ./internal/codegen/cmd -output=./pkg/vss -spec=./schema/vss_rel_4.2-DIMO.csv -migrations=./schema/migrations.json -package=vss
//go:generate go run ./cmd/codegen -output=./pkg/vss -spec=./schema/vss_rel_4.2-DIMO.csv -migrations=./schema/migrations.json -package=vss
package main

0 comments on commit 85c75a6

Please sign in to comment.