Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update oniguruma-to-es and use new api #886

Merged
merged 1 commit into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/references/engine-js-compat.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@

Compatibility reference of all built-in grammars with the [JavaScript RegExp engine](/guide/regex-engines#javascript-regexp-engine).

> Generated on Thursday, January 2, 2025
> Generated on Wednesday, January 8, 2025
>
> Version `1.24.4`
> Version `1.26.1`
>
> Runtime: Node.js v20.18.0
> Runtime: Node.js v22.11.0

## Report Summary

| | Count |
| :-------------- | --------------------------: |
| Total Languages | 219 |
| Supported | [213](#supported-languages) |
| Supported | [212](#supported-languages) |
| Mismatched | [1](#mismatched-languages) |
| Unsupported | [5](#unsupported-languages) |
| Unsupported | [6](#unsupported-languages) |

## Supported Languages

Expand Down Expand Up @@ -56,7 +56,6 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| cmake | ✅ OK | 23 | - | |
| cobol | ✅ OK | 863 | - | |
| codeowners | ✅ OK | 4 | - | |
| codeql | ✅ OK | 151 | - | |
| coffee | ✅ OK | 469 | - | |
| common-lisp | ✅ OK | 60 | - | |
| coq | ✅ OK | 26 | - | |
Expand Down Expand Up @@ -259,6 +258,7 @@ Languages that throw with the JavaScript RegExp engine, either because they cont

| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
| ---------- | :-------------- | ----------------: | --------------: | ---: |
| codeql | ✅ OK | 150 | 1 | |
| hack | ❌ Error | 947 | 1 | 114 |
| purescript | ❌ Error | 72 | 1 | |
| csharp | ❌ Error | 306 | 3 | 137 |
Expand Down
18 changes: 10 additions & 8 deletions packages/engine-javascript/src/engine-compile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,23 @@ export interface JavaScriptRegexEngineOptions extends JavaScriptRegexScannerOpti
/**
* The target ECMAScript version.
*
* Oniguruma-To-ES uses RegExp features from later versions of ECMAScript to provide improved
* accuracy and add support for more grammars. If using target `ES2024` or later, the RegExp `v`
* flag is used which requires Node.js 20+ or Chrome 112+.
* Oniguruma-To-ES uses RegExp features from later versions of ECMAScript to add support for a
* few more grammars. If using target `ES2024` or later, the RegExp `v` flag is used which
* requires Node.js 20+ or Chrome 112+.
* @see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/unicodeSets
*
* For maximum compatibility, you can set it to `ES2018` which uses the RegExp `u` flag but
* supports a few less grammars.
* For maximum compatibility, you can set it to `ES2018` which uses the RegExp `u` flag.
*
* Set to `auto` to automatically detect the latest version supported by the environment.
*
* @default 'auto'
*/
target?: 'auto' | 'ES2025' | 'ES2024' | 'ES2018'
}

/**
* The default RegExp constructor for JavaScript regex engine.
* The default regex constructor for the JavaScript RegExp engine.
*/

export function defaultJavaScriptRegexConstructor(pattern: string, options?: OnigurumaToEsOptions): RegExp {
return toRegExp(
pattern,
Expand All @@ -42,11 +41,15 @@ export function defaultJavaScriptRegexConstructor(pattern: string, options?: Oni
// Removing `\G` anchors in cases when they're not supported for emulation allows
// supporting more grammars, but also allows some mismatches
ignoreUnsupportedGAnchors: true,
// Oniguruma uses depth limit `20`; lowered here to keep regexes shorter and maybe
// sometimes faster, but can be increased if issues reported due to low limit
recursionLimit: 5,
},
...options,
},
)
}

/**
* Use the modern JavaScript RegExp engine to implement the OnigScanner.
*
Expand All @@ -55,7 +58,6 @@ export function defaultJavaScriptRegexConstructor(pattern: string, options?: Oni
* unsupported patterns, and when the grammar includes patterns that use invalid Oniguruma syntax.
* Set `forgiving` to `true` to ignore these errors and skip any unsupported or invalid patterns.
*/

export function createJavaScriptRegexEngine(options: JavaScriptRegexEngineOptions = {}): RegexEngine {
const _options: JavaScriptRegexEngineOptions = Object.assign(
{
Expand Down
14 changes: 7 additions & 7 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pnpm-workspace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ catalog:
monaco-editor-core: ^0.52.2
ofetch: ^1.4.1
ohash: ^1.1.4
oniguruma-to-es: 0.10.0
oniguruma-to-es: ^1.0.0
picocolors: ^1.1.1
pinia: ^2.3.0
pnpm: ^9.15.2
Expand Down
Loading