v0.4.0 #711

2024-05-09T12:08:31Z

github-actions[bot]
bot May 9, 2024

v0.4.0 - 2024-05-09

Bug Fixes

(ci) Fix broken nighly jobs (#659)

Fixes the broken nightly builds. Look at solved results
here.

Motivation and context

Nightly builds were broken due to oversight during the #571 fixes.

Migration notes

No changes required

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(gh-tests) Fix local npm registry config (#692)

Fix the NPM registry config in the Github tests.

Migration notes

N/A

Set max log level based on verbose flag (#664)

Set max log level based on verbose flag

Motivation and context

MET-445

Migration notes

n/a

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Fix secret passing (#675)

Fix secret passing in examples and documentation.

Motivation and context

Followup to #666.

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Website and headline (#691)

Motivation and context

Fix the CSS issue introduced by docusaurus 3.2.0
(facebook/docusaurus#10005). 3.2.1 should fix
it but the affected version maybe loaded by dependencies, so we will
have to wait a bit more.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Do not override log level when no verbosity flag is present (#694)

Remove log level override by the verbosity flag when no flag is present.
It will default to the configured env_logger default level (or env
variable).

...

The default log level became "error" after #664, and RUST_LOG
environment variable where ignored.

Migration notes

N/A

Documentation

Start rebranding (#641)

Motivation and context

Changing the intro.

Migration notes

None.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Add `reference/programmatic-deployment/` (#686)

Motivation and context

Add missing docs for tgDeploy, tgRemove

Migration notes

None

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Add examples for each command (#684)

Motivation and context

Getting started with meta cli should be easy

Migration notes

None

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Improve `/tutorials/quick-start` section. (#681)

Improve Layout
Include a simple project.
Add the result/outputs to running CLI commands.
Remove Metatype cloud registration form.
Separate the CLI commands to separate code blocks
Add links to references and concepts.
Add playground.

Motivation and context

Docs
Meta-task

Migration notes

No Migration Needed

Checklist

Test the commands and the examples.
The change come with new or modified tests

Improve `/docs/tutorials/metatype-basics` (#688)

Improve /docs/tutorials/metatype-basics

Motivation and context

Docs
Meta-task

Migration notes

No Migrations Needed

Checklist

The change come with new or modified tests

Improve `/reference/runtimes/` (#676)

Motivation and context

Better documentation

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Improve `/guides/external-functions` (#677)

Improvements to the /guides/external-functions page.
Adds a configuration file for git-town

Motivation and context

N/A

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Improve `docs/reference/types` (#685)

Improves docs/reference/types

Motivation and context

N/A

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Add a comparison b/n metatype and other similar solutions/products. (#697)

Adds a comparison table between metatype and other similar services.
Add artifact upload protocol to Architecture section in docs.

MET-443

Migration notes

No Migration Needed

Features

(cli) Long running discovery (#599)

Delegate serialize, deploy, undeploy, unpack work to SDK.

Motivation and context

Remove duplicate logic, thinking of cli as a convenience on top of the
SDK.

Migration notes

When meta cli is used, Migration files are unpacked/resolved relative to
the typegraph's path, not the process's workdir.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(cli) `meta gen` (#636)

Adds a command to meta-cli to invoke metagen.

Motivation and context

MET-424

Migration notes

No changes required

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(cli) Timeout loader process (#693)

(cli,sdk) Codegen command (#661)

Motivation and context

Enable back codegen on current cli implementation.

Migration notes

None

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(cli,sdk) Better error messages (#689)

Motivation and context

Make it more clear where failures happen

Migration notes

None

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(gate) Wasmtime support (#669) - BREAKING: wasmtime support (#669)

Motivation and context

Enable support for
wit
and facilitate readiness for the upcoming specs.

Migration notes

#[wasmedge_bindgen] are replaced by wit bindings.

In the old version we were restricted to#[wasmedge_bindgen], which was
only available in Rust and unique to WasmEdge.

#[wasmedge_bindgen]
fn add(a: u32, b: u32) -> u32 {
   a + b
}

In the new implementation, wasm modules written in any language that
uses the wit interface are now natively supported within typegate.

// wit/example.wit
package example:host;
world host {
  export add: func(a: u32, b: u32) -> u32;
}

An implementation (eg. in Rust) may look like this..

// src/lib.rs
wit_bindgen::generate!({ world: "host" });
struct MyLib;
impl Guest for MyLib {
    fn add(a: u32, b: u32) -> u32 {
        a + b
    }
}
export!(MyLib);

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(gate,cli) `$DENO_V8_FLAGS` (#647)

Enables the DENO_V8_FLAGS env var for tuning v8.

Motivation and context

MET-435 or #621

Migration notes

This just exposes the deno paramter directly. Refer to deno or v8 docs
for more details.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(metagen) Metagen mdk rust (#624)

Implements the general framework for metagen including a generator for
rust based wasm mat functions modules.

Motivation and context

MET-420

Migration notes

No breaking changes

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(sdk) .tgignore file support (#633)

Motivation and context

Set what files/folders should be ignored when using the custom
expand_path function in an external .tgignore file.
.tgignore will behave similarly to most .ignore files with basic glob
syntax support.

Migration notes

expand_glob has been renamed to expand_path

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(sdk) Introduce flag for disabling hashing artifacts + move hash to rust (#645)

Motivation and context

The mdk codegen and typegraph mutually depends on each other (typegraph
needs a concrete mdk.wasm for hashing, and for the mdk.wasm to be built,
it needs type generation based on the typegraph)
Add a flag to enable processing a partial typegraph when using meta gen (partial == no artifact resolution).

Migration notes

get_file_hash has been moved to core sdk (under the name hash_file)

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

(sdk, gate, cli) Upload protocol poc uploading wasm file for `WasmEdge Runtime` for single replica mode (#631)

Upload protocol for wasm files and atrifacts for WasmEdge Runtime for
single replica mode

Motivation and context

Upload WasmEdge Runtime artifacts during typegraph deploy
Access and load WasmEdge Runtime artifacts from the local file system
from typegate

Migration notes

No Migrations Needed

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Raw prisma query through the typegate runtime (#634)

Enable prisma query execution through the typegate runtime

Motivation and context

Console.

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Store the typegraph on s3 (#620) - BREAKING: Store the typegraph on s3 (#620)

Store the typegraph on s3 for multiple instance support mode.

Motivation and context

Reduce Redis data.

Migration notes

Environment variables:

REDIS_URL has been removed
For multiple instance support, the following variables are required:
SYNC_REDIS_URL, SYNC_S3_HOST, SYNC_S3_REGION, SYNC_S3_BUCKET,
SYNC_S3_ACCESS_KEY, SYNC_S3_SECRET_KEY; and the following variables
are optional: SYNC_REDIS_PASSWORD, SYNC_S3_PATH_STYLE. Otherwise,
none of them can be set.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Remove secret definitions through env vars (#666) - BREAKING: Remove secret definitions through env vars (#666)

Remove the ability to define secrets in the env vars of the typegate.

Secrets can now only be defined in the metatype config file and the
--secret CLI option.

Motivation and context

MET-370

Security
Better DX

Migration notes

Metatype config file: On the node configuration, secrets are
defined at secrets.<tg_name>.key:

# before
typegates:
  dev:
    env:
      TG_CONSOLE_POSTGRES_CONN: postgresql://postgres:password@localhost:5432/db?schema=console
      TG_CONSOLE_BASIC_ADMIN: password

#after
typegates:
  dev:
    secrets:
      console:
        POSTGRES_CONN: postgresql://postgres:password@localhost:5432/db?schema=console
        BASIC_ADMIN: password

Secret override option on meta/cli

# before
meta deploy -f my-tg.py --secret TG_CONSOLE_POSTGRES_CONN=postgresql://postgres:password@localhost:5432/db?schema=console

# after
meta deploy -f my-tg.py --secret POSTGRES_CONN=postgresql://postgres:password@localhost:5432/db?schema=console
# or - with the typegraph name
meta deploy -f my-tg.py --secret console:POSTGRES_CONN=postgresql://postgres:password@localhost:5432/db?schema=console

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Add queryPrismaModel in the typegate runtime (#635)

The queryPrismaModel function on the typegate typegraph queries rows
from a prisma model.

Motivation and context

Console.

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Upload artifacts to s3 (#638)

Upload artifacts to S3 when sync-mode is enabled

Motivation and context

Sharing artifacts between replicas without including it in the typegraph
(and sync through redis)

Migration notes

No changes needed.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Upload `PythonRuntime` artifacts and deps (#672)

Motivation and context

Track artifact/module dependencis for PythonRuntime

Migration notes

python.import(...) and python.import_(...) accept an optional parameter deps that accepts list of dependencies for the python module.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Upload `DenoRuntime` artifacts and deps (#674)

Track deno runtime artifacts(also dependencies)
Upload artifacts during deploy to either local(single replica) or
shared(s3)
resolve artifacts(module and deps) upon typegate runtime.

Motivation and context

Persisting deno runtime artifacts to a local/shared storage.

Migration notes

deno.import(...) and deno.import_(...) accept an optional parameter
that accepts list of dependencies for the deno/ts module.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Enable batch prisma queries in the typegate runtime (#682)

Enable batch prisma queries (and transaction) in the typegate runtime

Motivation and context

Console

MET-381

Migration notes

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Artifact removal (#668)

Add GC: remove artifacts when unreferenced by any deployed typegraph
Improve resource management: use AsyncDisposable and
AsyncDisposableStack
Improve testability (for parallel testing): always read the tmpDir
config from the Typegate object

Motivation and context

MET-433

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Summary by CodeRabbit

New Features
- Enhanced search functionality with the addition of a new search bar.
- Introduced new test configurations to improve script execution.
Updated artifact storage documentation to clarify management
processes.
- Added new extensions to support improved code commenting.
Bug Fixes
Removed outdated Deno import mapping settings to streamline
development environment setup.
Documentation
Expanded documentation on artifact tracking and management, including
reference counting and garbage collection mechanisms.
Refactor
Implemented interface changes in QueryEngine for better async
disposal management.
Code restructuring in artifact management for enhanced performance and
maintainability.
Chores
Adjusted settings and configurations in the development environment to
align with current best practices.
Tests
Introduced new test cases for artifact upload and management
functionalities.

Miscellaneous Tasks

(release) Prepare 0.4.0 (#710)

Bumps version to release 0.4.0.

Summary by CodeRabbit

New Features
Updated the software across various components to version 0.4.0,
enhancing functionality and potentially introducing new features or
fixes.
Documentation
Updated version documentation in multiple configuration files to
reflect new version 0.4.0.
Bug Fixes
Adjusted version constants and dependencies to ensure compatibility
and stability with the new software version 0.4.0.

(sdk,gate) Bump wasmtime to 20.0.0 and wit-bindgen to 0.24.0 (#695)

Refactor

(gate) Wasi 0.2 pyrt (#687)

Rewrites the PythonRuntime host using a componentize-py based
component.
Leaf through this
memo for a
mental model.

(libs/xtask,gate) Remove xtask/codegen (#700)

Faster build time

Migration notes

Make sure to sync typegate/src/types.ts when an update is made on the
typegraph schema.

(sdk,gate) Improve temporal rt (#642) - BREAKING: improve temporal rt (#642)

Improve the temporal runtime exposing more parameters and adding more
tests.

Motivation and context

MET-397. The old implementation was bug-ridden and did not expose
necessary params.

Migration notes

API changes to TemporalRuntime ctor, methods and generated
materializers.

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Pass test options in the first parameter (#667)

Pass the test options in the first parameter along with the test
name/description.

Motivation and context

Avoid scrolling to the end of the test function to see/update the test
options.

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

Testing

Use local npm registry for tests (#646)

Use verdaccio local npm registry for tests:

The @typegraph/sdk package is published to the local npm registry,
and can now be consumed like any npm package from Nodejs or Deno.

Motivation and context

...

Migration notes

N/A

Checklist

The change come with new or modified tests
Hard-to-understand functions have explanatory comments
End-user documentation is updated to reflect the change

This discussion was created from the release v0.4.0.

v0.4.0 #711

github-actions[bot] bot May 9, 2024

v0.4.0 - 2024-05-09

Bug Fixes

Motivation and context

Migration notes

Checklist

Migration notes

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Checklist

Migration notes

Documentation

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Migration notes

Features

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

Motivation and context

Migration notes

Checklist

github-actions[bot]
bot May 9, 2024