Transfer repo to DIMO-Network.

* Removes errors on not found data points.

README.md

@@ -1,8 +1,8 @@
 # Model Garage
 
 ![GitHub license](https://img.shields.io/badge/license-Apache%202.0-blue.svg)
-[![GoDoc](https://godoc.org/github.com/KevinJoiner/model-garage?status.svg)](https://godoc.org/github.com/KevinJoiner/model-garage)
-[![Go Report Card](https://goreportcard.com/badge/github.com/KevinJoiner/model-garage)](https://goreportcard.com/report/github.com/KevinJoiner/model-garage)
+[![GoDoc](https://godoc.org/github.com/DIMO-Network/model-garage?status.svg)](https://godoc.org/github.com/DIMO-Network/model-garage)
+[![Go Report Card](https://goreportcard.com/badge/github.com/DIMO-Network/model-garage)](https://goreportcard.com/report/github.com/DIMO-Network/model-garage)
 
 Welcome to the **Model Garage**, a Golang toolkit for managing and working with DIMO models generated from vspec CSV schemas. Model Garage provides the following features:
 
@@ -17,48 +17,55 @@ Welcome to the **Model Garage**, a Golang toolkit for managing and working with
 ## Getting Started
 
 1. **Installation**:
+
    ```bash
    go get github.com/your-username/model-garage
    ```
 
 2. **Import in Your Code**:
+
    ```go
    import "github.com/your-username/model-garage"
    ```
 
 3. **Usage**:
    Explore the documentation to start using Model Garage in your project.
 
-
 ## Repo structure
 
 ### Codegen
+
 The `codegen` directory contains the code generation tool for creating models from vspec CSV schemas. The tool is a standalone application that can be run from the command line.
 
 Example usage:
+
 ```bash
 go run ./cmd/codegen -output=./pkg/vss -spec=./schema/vss_rel_4.2-DIMO.csv -definitions=./schema/definitions.json -package=vss
 package main
 ```
 
 #### Generation Info
+
 The Model generation is handled by packages in `internal/codegen`. They are responsible for creating Go structs, Clickhouse tables, and conversion functions from the vspec CSV schema and definitions file.
 
-**Vspec Schema** The vspec schema is a CSV file that contains the signal definitions for the model. This schema is generated using vss-tools in this https://github.com/KevinJoiner/DIMO-VSS repository.
+**Vspec Schema** The vspec schema is a CSV file that contains the signal definitions for the model. This schema is generated using vss-tools in this https://github.com/DIMO-Network/DIMO-VSS repository.
 
 **Definitions File** The definitions file is a JSON file that contains the signal definitions that are to be included in the model. This file is manually created. With the following structure:
 
 - **vspecName**: The name of the signal field in the vspec. Only fields specified in the vspec will be included in the model.
 
 - **conversion**: (optional) Details about the conversion from the original data to the vspec field. If not specified, the conversion is assumed to be a direct copy.
-	- **originalName**: The original name of the field in the data.
 
-	- **originalType**: (optional) The original data type of the field. If not specified, the original type is assumed to be the same as the vspec type.
+  - **originalName**: The original name of the field in the data.
+
+  - **originalType**: (optional) The original data type of the field. If not specified, the original type is assumed to be the same as the vspec type.
+
 ##### Generation Process
+
 1. First, the vspec CSV schema and definitions file are parsed.
 2. Then a struct is created for each signal in the vpsec schema that is specified in the definitions file. With Clickhouse and JSON tags for each field. The CH and JSON names are the same as the vspec except `.` are replaced with `_`.
 3. Next, a Clickhouse table is created for the struct. The table name is the same as the package name. The table is created with the same fields as the struct with corresponding Clickhouse types.
-4. Finally, conversion functions are created for each struct. These functions convert the original data in the form of a JSON document to the struct. 
+4. Finally, conversion functions are created for each struct. These functions convert the original data in the form of a JSON document to the struct.
 
 **Conversion Functions**
-For each field, a conversion function is created. If a conversion is specified in the definitions file, the conversion function will use the specified conversion. If no conversion is specified, the conversion info function will assume a direct copy. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten.
+For each field, a conversion function is created. If a conversion is specified in the definitions file, the conversion function will use the specified conversion. If no conversion is specified, the conversion info function will assume a direct copy. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten.

cmd/codegen/codegen.go

@@ -5,10 +5,10 @@ import (
 	"flag"
 	"log"
 
-	"github.com/KevinJoiner/model-garage/internal/codegen"
-	"github.com/KevinJoiner/model-garage/internal/codegen/clickhouse"
-	"github.com/KevinJoiner/model-garage/internal/codegen/convert"
-	"github.com/KevinJoiner/model-garage/internal/codegen/model"
+	"github.com/DIMO-Network/model-garage/internal/codegen"
+	"github.com/DIMO-Network/model-garage/internal/codegen/clickhouse"
+	"github.com/DIMO-Network/model-garage/internal/codegen/convert"
+	"github.com/DIMO-Network/model-garage/internal/codegen/model"
 )
 
 func main() {

go.mod

@@ -1,4 +1,4 @@
-module github.com/KevinJoiner/model-garage
+module github.com/DIMO-Network/model-garage
 
 go 1.22.0
 

internal/codegen/clickhouse/clickhouse.go

@@ -10,7 +10,7 @@ import (
 	"strings"
 	"text/template"
 
-	"github.com/KevinJoiner/model-garage/internal/codegen"
+	"github.com/DIMO-Network/model-garage/internal/codegen"
 )
 
 // clickhouseFileName is the name of the ClickHouse table file that will be generated.

internal/codegen/convert/convert.go

@@ -13,7 +13,7 @@ import (
 	"strings"
 	"text/template"
 
-	"github.com/KevinJoiner/model-garage/internal/codegen"
+	"github.com/DIMO-Network/model-garage/internal/codegen"
 )
 
 var (

internal/codegen/convert/convert.tmpl

@@ -4,16 +4,8 @@ package {{ .PackageName }}
 var (
 	// errInvalidType is returned when a field is not of the expected type or not found.
 	errInvalidType = errors.New("invalid type")
-
-	// errNotFound is returned when a field is not found.
-	errNotFound = errors.New("not found")	
 )
 
-// IsNotFound returns true if the error is of type errNotFound.
-func IsNotFound(err error) bool {
-	return errors.Is(err, errNotFound)
-}
-
 // IsInvalidType returns true if the error is of type errInvalidType.
 func IsInvalidType(err error) bool {
 	return errors.Is(err, errInvalidType)
@@ -23,49 +15,48 @@ func IsInvalidType(err error) bool {
 // FromData creates a new {{ .ModelName }} from a map of data. Using defined conversion functions.
 // If skipNotFound is true, the function will not return an error if a key is not found. 
 // instead the field will be set to the zero value of the type.
-func FromData(jsonData []byte, skipNotFound bool) (*{{ .ModelName }}, error) {
-	{{ lower .ModelName }} := {{ .ModelName }}{}
+func FromData(jsonData []byte) (*{{ .ModelName }}, error) {
+	{{ $varName := lower .ModelName}}
+	{{ $varName }} := {{ .ModelName }}{}
 
 {{ $first := true }}
-{{- range .Signals }}
-	{{ if not .Conversion }} {{ continue }} {{ end }}
+{{- range $idx, $sig := .Signals }}
+	{{ if not $sig.Conversion }} {{ continue }} {{ end }}
 	{{ if $first }}
 	var err error
 	var result gjson.Result
 	{{ $first = false }} {{ end }}
 
-	// convert {{ .Conversion.OriginalName }} to {{ .GOName }}
-	result = gjson.GetBytes(jsonData, "{{ .Conversion.OriginalName }}")
+	// convert {{ $sig.Conversion.OriginalName }} to {{ $sig.GOName }}
+	result = gjson.GetBytes(jsonData, "{{ $sig.Conversion.OriginalName }}")
 	if result.Exists() {
-		{{ if .Conversion.IsArray -}}
+		{{ if $sig.Conversion.IsArray -}}
 		if !result.IsArray() {
-			return nil, fmt.Errorf("%w, field '{{ .Conversion.OriginalName }}' is not an array", errInvalidType)
+			return nil, fmt.Errorf("%w, field '{{ $sig.Conversion.OriginalName }}' is not an array", errInvalidType)
 		}
-		slice{{ .GOName}} := make([]{{ .Conversion.OriginalType }}, len(result.Array()))
+		slice{{ $sig.GOName}} := make([]{{ $sig.Conversion.OriginalType }}, len(result.Array()))
 		for i, res := range result.Array() {
-			v, ok := res.Value().({{ .Conversion.OriginalType }})
+			v, ok := res.Value().({{ $sig.Conversion.OriginalType }})
 			if !ok{
-				return nil, fmt.Errorf("%w, field '{{ .Conversion.OriginalName }}' array element %d is not of type {{ .Conversion.OriginalType }}", errInvalidType, i)
+				return nil, fmt.Errorf("%w, field '{{ $sig.Conversion.OriginalName }}' array element %d is not of type {{ $sig.Conversion.OriginalType }}", errInvalidType, i)
 			}
-			slice{{ .GOName}}[i] = v
+			slice{{ $sig.GOName}}[i] = v
 		}
-		vehicle.{{ .GOName }}, err = {{ convertName . }}(slice{{ .GOName}} )
+		{{ $varName }}.{{ $sig.GOName }}, err = {{ convertName $sig }}(slice{{ $sig.GOName}} )
 		if err != nil {
-			return nil, fmt.Errorf("failed to convert '{{ .Conversion.OriginalName }}': %w", err)
+			return nil, fmt.Errorf("failed to convert '{{ $sig.Conversion.OriginalName }}': %w", err)
 		}
 	{{ else -}}
-		val{{ .GOName}}, ok := result.Value().({{ .Conversion.OriginalType }})
+		val{{ $sig.GOName}}, ok := result.Value().({{ $sig.Conversion.OriginalType }})
 		if !ok {
-			return nil, fmt.Errorf("%w, field '{{ .Conversion.OriginalName }}' is not of type {{ .Conversion.OriginalType }}", errInvalidType)
+			return nil, fmt.Errorf("%w, field '{{ $sig.Conversion.OriginalName }}' is not of type {{ $sig.Conversion.OriginalType }}", errInvalidType)
 		}
-		vehicle.{{ .GOName }}, err = {{ convertName . }}(val{{ .GOName}})
+		{{ $varName }}.{{ $sig.GOName }}, err = {{ convertName $sig }}(val{{ $sig.GOName}})
 		if err != nil {
-			return nil, fmt.Errorf("failed to convert '{{ .Conversion.OriginalName }}': %w", err)
+			return nil, fmt.Errorf("failed to convert '{{ $sig.Conversion.OriginalName }}': %w", err)
 		}
 	{{ end -}}
-	} else if !skipNotFound{
-		return nil, fmt.Errorf("%w, field '{{ .Conversion.OriginalName }}'", errNotFound)
 	}
 {{- end }}
-	return &vehicle, nil
+	return &{{ $varName }}, nil
 }

internal/codegen/model/model.go

@@ -9,7 +9,7 @@ import (
 	"strings"
 	"text/template"
 
-	"github.com/KevinJoiner/model-garage/internal/codegen"
+	"github.com/DIMO-Network/model-garage/internal/codegen"
 )
 
 // structFileName is the name of the Go file that will contain the vehicle struct.

pkg/vss/convert_test.go

@@ -4,13 +4,13 @@ import (
 	"testing"
 	"time"
 
-	"github.com/KevinJoiner/model-garage/pkg/vss"
+	"github.com/DIMO-Network/model-garage/pkg/vss"
 	"github.com/stretchr/testify/require"
 )
 
 func TestFullFromDataConversion(t *testing.T) {
 	t.Parallel()
-	vehicle, err := vss.FromData([]byte(fullInputJSON), false)
+	vehicle, err := vss.FromData([]byte(fullInputJSON))
 	require.NoErrorf(t, err, "error converting full input data: %v", err)
 	require.Equalf(t, fullVehicle, vehicle, "converted vehicle does not match expected vehicle")
 }