CHANGELOG.md

Changelog

[0.23.0] - 2024-08-26

Breaking

• Introduce tree-sitter-language crate for grammar crates to depend on (tree-sitter#3069)
• Revert interning of a sequence or choice of a single rule (tree-sitter#3548)
• bindings: Use capsules in python (tree-sitter#3384)
• dsl: Support other JS runtimes (tree-sitter#3355)

Features

• Add fuzz subcommand (tree-sitter#3385)
• Allow external scanners to use the logger (tree-sitter#3204)
• bindings: Add query constants to python
• bindings: Add node, python, swift tests (tree-sitter#3178)
• bindings: Update npm scripts (tree-sitter#3210)
• cli: Bump unicode data to v15.1.0
• cli: Add debug build flag (tree-sitter#3279)
• cli: Attach helpful context when grammar.json cannot be found (tree-sitter#3405)
• cli: Add --show-fields flag to test command (tree-sitter#3502)
• lib: Add ts_query_end_byte_for_pattern (tree-sitter#3451)
• lib: Support no_std
• zig: Update outdated path syntax (tree-sitter#3383)

Bug Fixes

• Always reset to the first language when iterating over language attributes (tree-sitter#3375)
• Better error when a supertype rule is invalid (tree-sitter#3400)
• Intern a sequence or choice of a single element the same as the element itself
• Do not "absorb" rules that consist of a single terminal if the rule is hidden (tree-sitter#2577)
• bindings: Update go bindings (tree-sitter#3544)
• cli: Installation via authenticated proxy (tree-sitter#3414)
• cli: Dedup preceding_auxiliary_symbols (tree-sitter#3550)
• dsl: Improve error message when a rule function returns undefined (tree-sitter#3452)
• generate: Rename cargo.toml template (tree-sitter#3532)
• go: Update parser name in binding files, add to docs (tree-sitter#3547)
• lib: A null clock must have tv_nsec be 0 as well (tree-sitter#3372)
• lib: Restrict pattern_map optimization when a wildcard step has an immediate first child (tree-sitter#3440)
• lib: An empty root node should not precede an empty range (tree-sitter#3450)
• lib: Fix api header C++ interop (tree-sitter#3534)
• make: Fail properly on Windows (tree-sitter#3418)
• rust: Fetch CARGO_MANIFEST_DIR at runtime in build script (tree-sitter#3352)
• rust: Fix new clippy warnings (tree-sitter#3491)
• test: Multi-grammar corpus tests are now in the repo root (tree-sitter#3342)
• wasm: Update test

Performance

• Hoist out common subexpressions in satisfies_text_predicates (tree-sitter#3397)

Documentation

• Update changelog
• Remove duplicate pr # in changelog
• Add note for bullet
• Fix syntax highlighting unit testing example (tree-sitter#3434)
• Add tsserver annotation to example (tree-sitter#3460)
• Fix tree cursor documentation (tree-sitter#3324)
• Document rust library features (tree-sitter#3395)
• Clean up binding & parser lists (tree-sitter#3443)

Refactor

• Remove ansi_term dependency (tree-sitter#3387)
• Remove difference dependency (tree-sitter#3388)
• scripts: Clean up bash scripts (tree-sitter#3231)

Testing

• Modernize scanner files (tree-sitter#3340)

Build System and CI

• deps: bump wasmtime, cc, and wasmparser (tree-sitter#3529
• bindings: Use language version in soname (tree-sitter#3308)
• lib: Include the minor in the soname
• loader: Make dependencies optional (tree-sitter#1638)
• swift: Declare header search path (tree-sitter#3474)
• wasm: Don't minify JS (tree-sitter#3380)
• wasm: Bump emscripten to 3.1.64 (tree-sitter#3497)
• wasm: Support big endian machines (tree-sitter#3492)
• zig: Git ignore updated Zig cache directory (tree-sitter#3408)

Other

• Swap sprintf() for snprintf() (tree-sitter#3430)
• Add .build to gitignore (tree-sitter#3498)
• Reset language when resetting wasm store (tree-sitter#3495)
• Clone wasm store engine (tree-sitter#3542)
• bindings: Fix indent & line endings (tree-sitter#3284)

[0.22.6] — 2024-05-05

Features

• Improve handling of serialization buffer overflows (tree-sitter#3318)
• Reverse iteration through node parents (tree-sitter#3214)
• cli: Support NO_COLOR (tree-sitter#3299)
• cli: Add test listing and allow users to parse a specific test number (tree-sitter#3067)
• grammar: Add "inherits" field if available (tree-sitter#3295)

Bug Fixes

• Correctly load field data from wasm languages
• Improve error message when the tree-sitter field is malformed
• Don't error out on package.json lookup errors if --no-bindings is passed (tree-sitter#3323)
• cli: Keep default cc flags in build
• cli: Properly account for multi-grammar repos when using docker to build a wasm parser (tree-sitter#3337)
• generate: Don't check arbitrarily named dirs
• generate: Take AsRef<Path> for the path parameter to avoid clones (tree-sitter#3322)
• highlight: Correct signature of ts_highlighter_add_language
• lib: Do not return field names for extras (tree-sitter#3330)
• lib: Advance the lookahead end byte by 4 when there's an invalid code point (tree-sitter#3305)
• rust: Update README example (tree-sitter#3307)
• rust: Use unix + wasi cfg instead of not windows for fd (tree-sitter#3304)
• test: Allow newlines in between test name and attribute
• wasm: Correct childrenFromFieldXXX method signatures (tree-sitter#3301)
• xtask: Always bump every crate in tandem
• zig: Make usable as a zig dependency (tree-sitter#3315)

Documentation

• Mention build command variables
• Swap \s for \\s in query example
• highlight: Typo (tree-sitter#3290)

Refactor

• tests: Migrate remaining grammar.json tests to grammar.js (tree-sitter#3325)

Build System and CI

• Add nightly rustfmt to workflow for linting (tree-sitter#3333)
• Fix address sanitizer step (tree-sitter#3188)
• deps: Bump cc from 1.0.92 to 1.0.94 in the cargo group (tree-sitter#3298)
• deps: Bump the cargo group with 6 updates (tree-sitter#3313)
• xtask: Bump build.zig.zon version when bumping versions

[0.22.5] — 2024-04-14

Bug Fixes

• Avoid generating unused character set constants
• cli: Test parsing on windows (tree-sitter#3289)
• rust: Compilation on wasm32-wasi (tree-sitter#3293)

[0.22.4] — 2024-04-12

Bug Fixes

• Fix sorting of transitions within a lex state
• Include 2-character ranges in array-based state transitions

Build System and CI

• Always bump at least the patch version in bump xtask

[0.22.3] — 2024-04-12

Features

• Add strncat to wasm stdlib
• Generate simpler code for matching large character sets (tree-sitter#3234)
• When loading languages via WASM, gracefully handle memory errors and leaks in external scanners (tree-sitter#3181)

Bug Fixes

• bindings: Add utf-8 flag to python & node (tree-sitter#3278)
• bindings: Generate parser.c if missing (tree-sitter#3277)
• bindings: Remove required platforms for swift (tree-sitter#3264)
• cli: Fix mismatched parenthesis when accounting for && (tree-sitter#3274)
• lib: Do not consider childless nodes for ts_node_parent (tree-sitter#3191)
• lib: Properly account for aliased root nodes and root nodes with children in ts_subtree_string (tree-sitter#3191)
• lib: Account for the root node of a tree cursor being an alias (tree-sitter#3191)
• lib: Use correct format specifier in log message (tree-sitter#3255)
• parser: Fix variadic macro (tree-sitter#3229)
• render: Proper function prototypes (tree-sitter#3277)
• windows: Add /utf-8 flag for parsers using unicode symbols (tree-sitter#3223)
• Add a semicolon after SKIP macros (tree-sitter#3264)
• Add back build-wasm temporarily (tree-sitter#3203)
• Add lifetime to matches function (tree-sitter#3254)
• Default output directory for build --wasm should use current_dir (tree-sitter#3203)
• Fix sorting of wasm stdlib symbols
• Insert "tree-sitter" section in current directory's package.json if it exists (tree-sitter#3224)
• Tie the lifetime of the cursor to the query in QueryCursor::captures() (tree-sitter#3266)
• Wrong flag check in build.rs

Performance

• cli: Reduced the compile time of generated parsers by generating C code with fewer conditionals (tree-sitter#3234)

Documentation

• Add NGINX grammar

Refactor

• parser: Make REDUCE macro non-variadic (tree-sitter#3280)
• js: Misc fixes & tidying
• rust: Misc fixes & tidying

Testing

• Add regression test for node parent + string bug (tree-sitter#3191)
• test: Allow colons in test names (tree-sitter#3264)

Build System and CI

• Upgrade wasmtime
• Update emscripten version (tree-sitter#3272)
• dependabot: Improve PR labels (tree-sitter#3282)

[0.22.2] — 2024-03-17

Breaking

• cli: Add a separate build command to compile parsers

Features

• bindings/rust: Expose Parser::included_ranges
• Lower the lib's MSRV (tree-sitter#3169)
• lib: Implement Display for Node (tree-sitter#3177)

Bug Fixes

• bindings/wasm: Fix Parser.getIncludedRanges() (tree-sitter#3164)
• lib: Makefile installation on macOS (tree-sitter#3167)
• lib: Makefile installation (tree-sitter#3173)
• lib: Avoid possible UB of calling memset on a null ptr when 0 is passed into array_grow_by (tree-sitter#3176)
• lib: Allow hiding symbols (tree-sitter#3180)

Documentation

• Fix typo (tree-sitter#3158)
• licensfe: Update year (tree-sitter#3183)

Refactor

• Remove dependency on which crate (tree-sitter#3172)
• Turbofish styling

Testing

• Fix header writes (tree-sitter#3174)

Build System and CI

• Simplify workflows (tree-sitter#3002)
• lib: Allow overriding CFLAGS on the commandline (tree-sitter#3159)

[0.22.1] — 2024-03-10

Bug Fixes

• Cli build script behavior on release

[0.22.0] — 2024-03-10

Breaking

• Remove top-level corpus dir for tests The cli will now only look in test/corpus for tests
• Remove redundant escape regex & curly brace regex preprocessing (tree-sitter#2838)
• bindings: Convert node bindings to NAPI (tree-sitter#3077)
• wasm: Make current*, is*, and has* methods properties (tree-sitter#3103)
• wasm: Keep API in-line with upstream and start aligning with node (tree-sitter#3149)

Features

• Add xtasks to assist with bumping crates (tree-sitter#3065)
• Improve language bindings (tree-sitter#2438)
• Expose the allocator and array header files for external scanners (tree-sitter#3063)
• Add typings for the node bindings
• Replace nan with node-addon-api and conditionally print logs
• bindings: Add more make targets
• bindings: Add peerDependencies for npm
• bindings: Add prebuildify to node
• bindings: Remove dsl types file (tree-sitter#3126)
• node: Type tag the language (tree-sitter#3109)
• test: Add attributes for corpus tests

Bug Fixes

• Apply some scan-build suggestions (unused assignment/garbage access) (tree-sitter#3056)
• Wrap || comparison in parentheses when && is used (tree-sitter#3070)
• Ignore unused variables in the array macros (tree-sitter#3083)
• binding.cc overwrite should replace PARSER_NAME (tree-sitter#3116)
• Don't use __declspec(dllexport) on windows (tree-sitter#3128)
• Parsers should export the language function on windows
• Allow the regex v flag (tree-sitter#3154)
• assertions: Case shouldn't matter for comment node detection
• bindings: Editorconfig and setup.py fixes (tree-sitter#3082)
• bindings: Insert types after main if it exists (tree-sitter#3122)
• bindings: Fix template oversights (tree-sitter#3155)
• cli: Only output the sources with --no-bindings (tree-sitter#3123)
• generate: Add .npmignore, populate Swift's exclude list (tree-sitter#3085)
• generate: Extern allocator functions for the template don't need to be "exported" (tree-sitter#3132)
• generate: Camel case name in Cargo.toml description (tree-sitter#3140)
• lib: Include api.h so ts_set_allocator is visible (tree-sitter#3092)

Documentation

• Add GitHub user and PR info to the changelog
• Add css for inline code (tree-sitter#2844)
• Document test attributes
• Add Ohm language parser
• Remove duplicate the's (tree-sitter#3120)
• Add discord and matrix badges (tree-sitter#3148)

Refactor

• Rename TS_REUSE_ALLOCATOR flag (tree-sitter#3088)
• Remove extern/const where possible
• array: Use pragma GCC in clang too
• bindings: Remove npmignore (tree-sitter#3089)

Testing

• Don't use TS_REUSE_ALLOCATOR on Darwin systems (tree-sitter#3087)
• Add test case for parse stack merging with incorrect error cost bug (tree-sitter#3098)

Build System and CI

• Improve changelog settings (tree-sitter#3064)
• Unify crate versions via workspace (tree-sitter#3074)
• Update cc to remove annoying debug output (tree-sitter#3075)
• Adjust dependabot settings (tree-sitter#3079)
• Use c11 everywhere
• Add uninstall command
• Don't skip tests on failing lint (tree-sitter#3102)
• Remove unused deps, bump deps, and bump MSRV to 1.74.1 (tree-sitter#3153)
• bindings: Metadata improvements
• bindings: Make everything c11 (tree-sitter#3099)
• dependabot: Update weekly instead of daily (tree-sitter#3112)
• deps: Bump the cargo group with 1 update (tree-sitter#3081)
• deps: Bump the cargo group with 1 update (tree-sitter#3097)
• deps: Bump deps & lockfile (tree-sitter#3060)
• deps: Bump the cargo group with 4 updates (tree-sitter#3134)
• lint: Detect if Cargo.lock needs to be updated (tree-sitter#3066)
• lint: Make lockfile check quiet (tree-sitter#3078)
• swift: Move 'cLanguageStandard' behind 'targets' (tree-sitter#3101)

Other

• Make Node.js language bindings context aware (tree-sitter#2841) They don't have any dynamic global data, so all it takes is just declaring them as such
• Fix crash when attempting to load ancient languages via wasm (tree-sitter#3068)
• Use workspace dependencies for internal crates like Tree-sitter (tree-sitter#3076)
• Remove vendored wasmtime headers (tree-sitter#3084) When building rust binding, use wasmtime headers provided via cargo by the wasmtime-c-api crate.
• Fix invalid parse stack recursive merging with mismatched error cost (tree-sitter#3086) Allowing this invalid merge caused an invariant to be violated later on during parsing, when handling a later error.
• Fix regression in subtree_compare (tree-sitter#3111)
• docs: Add Ohm language parser (tree-sitter#3114)
• Delete binding_files.rs (tree-sitter#3106)
• bindings: Consistent wording (tree-sitter#3096)
• bindings: Ignore more artifacts (tree-sitter#3119)

[0.21.0] — 2024-02-21

Breaking

• Remove the apply-all-captures flag, make last-wins precedence the default

  NOTE: This change might cause breakage in your grammar's highlight tests. Just flip the order around of the relevant queries, and keep in mind that the last query that matches will win.

Features

• Use lockfiles to dedup recompilation
• Improve error message for files with an unknown grammar path (tree-sitter#2475)
• Implement first-line-regex (tree-sitter#2479)
• Error out if an empty string is in the extras array
• Allow specifying an external scanner's files (tree-sitter#3031)
• Better error info when a scanner is missing required symbols
• cli: Add an optional grammar-path argument for the playground (tree-sitter#3014)
• cli: Add optional config-path argument (tree-sitter#3050)
• loader: Add more commonly used default parser directories

Bug Fixes

• Prettify xml output and add node position info (tree-sitter#2970)
• Inherited grammar generation
• Properly error out when the word property is an invalid rule
• Update schema for regex flags (tree-sitter#3006)
• Properly handle Query.matches when filtering out results (tree-sitter#3013)
• Sexp format edge case with quoted closed parenthesis (tree-sitter#3016)
• Always push the default files if there's no externals
• Don't log NUL characters (tree-sitter#3037)
• Don't throw an error if the user uses map in the grammar (tree-sitter#3041)
• Remove redundant imports (tree-sitter#3047)
• cli: Installation via a HTTP tunnel proxy (tree-sitter#2824)
• cli: Don't update tests automatically if parse errors are detected (tree-sitter#3033)
• cli: Don't use long for grammar_path
• test: Allow writing updates to tests without erroneous nodes instead of denying all of them if a single error is found
• test: Edge case when parsing UNEXPECTED/MISSING nodes with an indentation level greater than 0
• wasm: Remove C++ mangled symbols (tree-sitter#2971)

Documentation

• Create issue template (tree-sitter#2978)
• Document regex limitations
• Mention that token($.foo) is illegal
• Explicitly mention behavior of walking outside the given "root" node for a TSTreeCursor (tree-sitter#3021)
• Small fixes (tree-sitter#2987)
• Add Tact language parser (tree-sitter#3030)
• web: Provide deno usage information (tree-sitter#2498)

Refactor

• Extract regex check into a function and lower its precedence
• &PathBuf -> &Path (tree-sitter#3035)
• Name anonymous types in api.h (tree-sitter#1659)

Testing

• Add quotes around bash variables (tree-sitter#3023)
• Update html tests

Build System and CI

• Only create release for normal semver tags (tree-sitter#2973)
• Add useful development targets to makefile (tree-sitter#2979)
• Remove minimum glibc information in summary page (tree-sitter#2988)
• Use the native m1 mac runner (tree-sitter#2995)
• Add editorconfig (tree-sitter#2998)
• Remove symbolic links from repository (tree-sitter#2997)
• Move common Cargo.toml keys into the workspace and inherit them (tree-sitter#3019)
• Remove reviewers when drafting or closing a PR (tree-sitter#2963)
• Enable creating changelogs with git-cliff (tree-sitter#3040)
• Cache fixtures (tree-sitter#3038)
• Don't cancel jobs on master (tree-sitter#3052)
• Relax caching requirements (tree-sitter#3051)
• deps: Bump clap from 4.4.18 to 4.5.0 (tree-sitter#3007)
• deps: Bump wasmtime from v16.0.0 to v17.0.1 (tree-sitter#3008)
• deps: Bump wasmtime to v18.0.1 (tree-sitter#3057)
• sanitize: Add a timeout of 60 minutes (tree-sitter#3017)
• sanitize: Reduce timeout to 20 minutes (tree-sitter#3054)

Other

• Document preferred language for scanner (tree-sitter#2972)
• Add java and tsx to corpus tests (tree-sitter#2992)
• Provide a CLI flag to open log.html (tree-sitter#2996)
• Some more clippy lints (tree-sitter#3010)
• Remove deprecated query parsing mechanism (tree-sitter#3011)
• Print out full compiler arguments ran when it fails (tree-sitter#3018)
• Deprecate C++ scanners (tree-sitter#3020)
• Add some documentation to the playground page (tree-sitter#1495)
• Update relevant rust tests (tree-sitter#2947)
• Clippy lints (tree-sitter#3032)
• Error out when multiple arguments are passed to token/token.immediate (tree-sitter#3036)
• Tidying
• Prefer turbofish syntax where possible (tree-sitter#3048)
• Use published wasmtime crates
• Cleaner cast
• Update Cargo.lock
• Get rid of github_issue_test file (tree-sitter#3055)
• cli: Use spawn to display emcc's stdout and stderr (tree-sitter#2494)
• cli: Warn users when a query path needed for a subcommand isn't specified in a grammar's package.json
• generate: Dedup and warn about duplicate or invalid rules (tree-sitter#2994)
• test: Use different languages for async tests (tree-sitter#2953)
• wasm: Use SIDE_MODULE=2 to silence warning (tree-sitter#3003)