diff --git a/README.md b/README.md index cf6218ac..87834b3c 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,3 @@ - - - - -[![CI-test](https://github.com/mathiasrw/rexreplace/workflows/CI-test/badge.svg)](https://github.com/mathiasrw/rexreplace/actions) -[![NPM downloads](https://img.shields.io/npm/dm/rexreplace.svg?style=flat&label=npm%20downloads)](https://npm-stat.com/charts.html?package=rexreplace) -[![npm version](https://badge.fury.io/js/rexreplace.svg)](https://www.npmjs.com/package/rexreplace) -[![FOSSA Status](https://img.shields.io/badge/license-MIT-brightgreen.svg)](https://app.fossa.io/projects/git%2Bgithub.com%2Fmathiasrw%2Frexreplace?ref=badge_shield) -[![OPEN open source software](https://img.shields.io/badge/Open--OSS-%E2%9C%94-brightgreen.svg)](http://open-oss.com) - - -


RexReplace mascot Benny on the RexReplace logo @@ -18,13 +6,15 @@

---- - # RexReplace +[![CI-test](https://github.com/mathiasrw/rexreplace/workflows/CI-test/badge.svg)](https://github.com/mathiasrw/rexreplace/actions) +[![NPM downloads](https://img.shields.io/npm/dm/rexreplace.svg?style=flat&label=npm%20downloads)](https://npm-stat.com/charts.html?package=rexreplace) +[![npm version](https://badge.fury.io/js/rexreplace.svg)](https://www.npmjs.com/package/rexreplace) +[![FOSSA Status](https://img.shields.io/badge/license-MIT-brightgreen.svg)](https://app.fossa.io/projects/git%2Bgithub.com%2Fmathiasrw%2Frexreplace?ref=badge_shield) +[![OPEN open source software](https://img.shields.io/badge/Open--OSS-%E2%9C%94-brightgreen.svg)](http://open-oss.com) - -RexReplace is a versatile tool to search and replace text in files from the command line. Its inspired by how developers often need to do quick fixes or one-liners for build scripts. +RexReplace is a versatile tool to search and replace text in files from the command line. It's inspired by how developers often need to do quick fixes or one-liners for build scripts. **Key features**: @@ -33,9 +23,8 @@ RexReplace is a versatile tool to search and replace text in files from the comm - Pinpoint the exact files with [glob notation](https://www.tcl.tk/man/tcl8.5/tutorial/Tcl16a.html) (`docs/*.md` represents each markdown file in `docs/`) - No more brute-forcing the right combination of `find`, `cat`, `sed`, `tr`, and `awk` to replace a text pattern in the right files - - ## Install + To use RexReplace from your command line ```bash @@ -44,18 +33,15 @@ To use RexReplace from your command line To use RexReplace from an npm build script: - ```bash > yarn add rexreplace --dev # or > npm install rexreplace --save-dev ``` - - ## Examples -Let 'foobar' become 'xxxbar' in myfile.md +Let 'foobar' become 'xxxbar' in myfile.md ```bash > rexreplace 'Foo' 'xxx' myfile.md @@ -67,68 +53,61 @@ Hard for your fingers to write on your keyboard? We got you covered with the `rr > rr Foo xxx myfile.md ``` +--- ----- - -#### Catch the beginning +#### Catch the beginning Let all markdown files in the `docs/` dir get headlines moved one level deeper ```bash -> rexreplace '^#' '##' docs/*.md +> rexreplace '^#' '##' docs/*.md ``` - - ---- + +--- #### Using glob notation to pinpoint files -Fix a spell error in all javascript and typescript files in the folders `src/` and `test/` recursively. +Fix a spell error in all javascript and typescript files in the folders `src/` and `test/` recursively. ```bash > rexreplace 'foubar' 'foobar' '{src,test}/**/*.{js,ts}' ``` - ----- - +--- #### Dynamically generated content Let the version number from package.json get into your distribution js files (use the string `VERSION` in your source files). - + ```bash -> rexreplace 'VERSION' 'require("./package.json").version' -j dist/*.js +> rexreplace 'VERSION' 'require("./package.json").version' -j dist/*.js ``` -Require have been given the alias `r` and both are expanded to understand relative paths even without `./` prepended. As the file extension is not needed eighter you will get the same result writing: +Require have been given the alias `r` and both are expanded to understand relative paths even without `./` prepended. As the file extension is not needed eighter you will get the same result writing: ```bash -> rexreplace 'VERSION' 'r("package").version' -j dist/*.js +> rexreplace 'VERSION' 'r("package").version' -j dist/*.js ``` - #### Reference a matching group -Let 'foobar' become 'barfoo' in myfile.md +Let 'foobar' become 'barfoo' in myfile.md ```bash > rexreplace '(foo)(.*)' '$2$1' myfile.md ``` - RexReplace defaults to treating `€` as an alias for `$` so the following will do the same as the previous example ```bash -> rexreplace '(foo)(.*)' '€2€1' myfile.md +> rexreplace '(foo)(.*)' '€2€1' myfile.md ``` - - ----- +--- #### Surviving backslash-escape hell -Let `foo[bar]` become `foo.0.[bar]` in myfile.md +Let `foo[bar]` become `foo.0.[bar]` in myfile.md ```bash > rexreplace '\[' '.0.[' myfile.md @@ -142,7 +121,6 @@ The `[` as a literate char must be escaped according to regex. If you run the co } ``` - RexReplace defaults to treating `§` as an alias for `\` so the following give same result: ```bash @@ -155,19 +133,16 @@ RexReplace defaults to treating `§` as an alias for `\` so the following give s } ``` +--- - ----- - - -#### More relevant examples +#### More relevant examples **Per file info** Add creation time, name of the file and human readable file size as the first line in each file in `test-run` recursively. ```bash -> rexreplace '^' 'ctime_ + name_ + size + nl' -j -M 'dist/**/*.*' +> rexreplace '^' 'ctime_ + name_ + size + nl' -j -M 'dist/**/*.*' ``` **Matching pairs** @@ -178,97 +153,92 @@ Let both `"foo 'is' bar"` and `' foo "was" bar'` ' become `» foo ... bar «` in > rexreplace '(['"])([^\1]+)\1' '» $2 «' myfile.md ``` +--- ----- - +## Usage -## Usage ```bash > rexreplace pattern replacement [fileGlob|option]+ ``` - -Flag | Effect ----- | ---- -`-v` | **`--version`** Print rexreplace version (can be given as only argument) [boolean] -`-V` | **`--verbose`** More chatty output [boolean] -`-L` | **`--literal`** Literal string search (no regex used when searching) [boolean] -`-I` | **`--void-ignore-case`** Void case insensitive search pattern. [boolean] -`-G` | **`--void-global`** Void global search (stop looking after the first match). [boolean] -`-s` | **`--dot-all`** Have `.` also match newline. [boolean] -`-M` | **`--void-multiline`** Void multiline search pattern. Makes ^ and $ match start/end of whole content rather than each line. [boolean] -`-u` | **`--unicode`** Treat pattern as a sequence of unicode code points. [boolean] -`-e` | **`--encoding`** Encoding of files/piped data. [default: "utf8"] -`-E` | **`--engine`** What regex engine to use: [choices: "V8"] [default: "V8"] -`-q` | **`--quiet`** Only display errors (no other info) [boolean] -`-Q` | **`--quiet-total`** Never display errors or info [boolean] -`-H` | **`--halt`** Halt on first error [boolean] [default: false] -`-d` | **`--debug`** Print debug info [boolean] -`-€` | **`--void-euro`** Void having `€` as alias for `$` in pattern and replacement parameters [boolean] -`-§` | **`--void-section`** Void having `§` as alias for `\` in pattern and replacement parameters [boolean] -`-o` | **`--output`** Output the final result instead of saving to file. Will also output content even if no replacement has taken place. [boolean] -`-A` | **`--void-async`** Handle files in a synchronous flow. Good to limit memory usage when handling large files. [boolean] -`-B` | **`--void-backup`** Avoid temporary backing up file. Works async (independent of -A flag) and will speed up things but at one point data lives only in memory, and you will lose the content if the process is abrupted. [boolean] -`-b` | **`--keep-backup`** Keep a backup file of the original content. [boolean] -`-m` | **`--output-match`** Output each match on a new line. Will not replace any content but you still need to provide a dummy value (like `_`) as replacement parameter. If search pattern does not contain matching groups the full match will be outputted. If search pattern does contain matching groups only matching groups will be outputted (same line with no delimiter). [boolean] -`-T` | **`--trim-pipe`** Trim piped data before processing. If piped data only consists of chars that can be trimmed (new line, space, tabs...) it will become an empty string. [boolean] -`-R` | **`--replacement-pipe`** Replacement will be piped in. You still need to provide a dummy value (like `_`) as replacement parameter. [boolean] -`-j` | **`--replacement-js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. If the javascript source code references to the full match or a captured group the code will run once per match. Otherwise, it will run once per file. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command (null if no piped data), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] -`-x` | **`--exclude-re`** Exclude files with a path that matches this regular expression. Will follow same regex flags and setup as the main search. Can be used multiple times. [string] -`-X` | **`--exclude-glob`** Exclude files found with this glob. Can be used multiple times. [string] -`-h` | **`--help`** Display help. [boolean] - -## Good to know - -### Features +| Flag | Effect | +| ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `-v` | **`--version`** Print rexreplace version (can be given as only argument) [boolean] | +| `-V` | **`--verbose`** More chatty output [boolean] | +| `-L` | **`--literal`** Literal string search (no regex used when searching) [boolean] | +| `-I` | **`--void-ignore-case`** Void case insensitive search pattern. [boolean] | +| `-G` | **`--void-global`** Void global search (stop looking after the first match). [boolean] | +| `-s` | **`--dot-all`** Have `.` also match newline. [boolean] | +| `-M` | **`--void-multiline`** Void multiline search pattern. Makes ^ and $ match start/end of whole content rather than each line. [boolean] | +| `-u` | **`--unicode`** Treat pattern as a sequence of unicode code points. [boolean] | +| `-e` | **`--encoding`** Encoding of files/piped data. [default: "utf8"] | +| `-E` | **`--engine`** What regex engine to use: [choices: "V8"] [default: "V8"] | +| `-q` | **`--quiet`** Only display errors (no other info) [boolean] | +| `-Q` | **`--quiet-total`** Never display errors or info [boolean] | +| `-H` | **`--halt`** Halt on first error [boolean] [default: false] | +| `-d` | **`--debug`** Print debug info [boolean] | +| `-€` | **`--void-euro`** Void having `€` as alias for `$` in pattern and replacement parameters [boolean] | +| `-§` | **`--void-section`** Void having `§` as alias for `\` in pattern and replacement parameters [boolean] | +| `-o` | **`--output`** Output the final result instead of saving to file. Will also output content even if no replacement has taken place. [boolean] | +| `-A` | **`--void-async`** Handle files in a synchronous flow. Good to limit memory usage when handling large files. [boolean] | +| `-B` | **`--void-backup`** Avoid temporary backing up file. Works async (independent of -A flag) and will speed up things but at one point data lives only in memory, and you will lose the content if the process is abrupted. [boolean] | +| `-b` | **`--keep-backup`** Keep a backup file of the original content. [boolean] | +| `-m` | **`--output-match`** Output each match on a new line. Will not replace any content but you still need to provide a dummy value (like `_`) as replacement parameter. If search pattern does not contain matching groups the full match will be outputted. If search pattern does contain matching groups only matching groups will be outputted (same line with no delimiter). [boolean] | +| `-T` | **`--trim-pipe`** Trim piped data before processing. If piped data only consists of chars that can be trimmed (new line, space, tabs...) it will become an empty string. [boolean] | +| `-R` | **`--replacement-pipe`** Replacement will be piped in. You still need to provide a dummy value (like `_`) as replacement parameter. [boolean] | +| `-j` | **`--replacement-js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. If the javascript source code references to the full match or a captured group the code will run once per match. Otherwise, it will run once per file. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command (null if no piped data), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] | +| `-x` | **`--exclude-re`** Exclude files with a path that matches this regular expression. Will follow same regex flags and setup as the main search. Can be used multiple times. [string] | +| `-X` | **`--exclude-glob`** Exclude files found with this glob. Can be used multiple times. [string] | +| `-h` | **`--help`** Display help. [boolean] | + +## Good to know + +### Features - Patterns are described as javascript notation regex - Pattern defaults to global multiline case-insensitive search -- Supports regex lookaheads in the pattern -- Supports backreference to matching groups in the replacement -- Data to be treated can be piped in -- See the [release note](https://github.com/mathiasrw/rexreplace/releases) for a log of changes. Descriptions are given in latest patch version. - +- Supports regex lookaheads in the pattern +- Supports backreference to matching groups in the replacement +- Data to be treated can be piped in +- See the [release note](https://github.com/mathiasrw/rexreplace/releases) for a log of changes. Descriptions are given in latest patch version. ### Quirks -- Options can only be set after the replacement parameter. "_But I like to put my options as the first thing, so I know what I am doing_" I agree, but we must sometimes sacrifice habits for consistency. -- Per default `€` is treated as an alias for `$` in the CLI input. The main reason is for you not to worry about how command line tools often have a special relationship with the `$` char. You can escape your way out of this old love story, but it often pops up in unexpected ways. Use the `-€` flag if you need to search or replace the actual euro char. +- Options can only be set after the replacement parameter. "_But I like to put my options as the first thing, so I know what I am doing_" I agree, but we must sometimes sacrifice habits for consistency. -- Per default `§` is treated as an alias for `\` in the CLI input. The main reason is for you not to worry about escaping the escape of an escape. Use the `-§` flag if you need to search or replace the actual section char. +- Per default `€` is treated as an alias for `$` in the CLI input. The main reason is for you not to worry about how command line tools often have a special relationship with the `$` char. You can escape your way out of this old love story, but it often pops up in unexpected ways. Use the `-€` flag if you need to search or replace the actual euro char. +- Per default `§` is treated as an alias for `\` in the CLI input. The main reason is for you not to worry about escaping the escape of an escape. Use the `-§` flag if you need to search or replace the actual section char. ### Limitations + - RexReplace reads each file fully into memory, so working on your 8Gb log files will probably not be ideal. - For versions of Node prior to 6, please use version 2.2.x. For versions of Node prior to 0.12, please use [the legacy version of RexReplace called rreplace](https://www.npmjs.com/package/rreplace) - - ### Priorities -- Flexibility regarding text pattern matching + +- Flexibility regarding text pattern matching - Easy to filter what files to be treated - Helpful interface - Tests (if you know how to do a test cover report on javascript code ran via the command line, please let me know) - ### Not a priority -- Speed. Well... obviously, speed is important, but to what extent does a 29 millisecond command really satisfy the user compared to a 294 millisecond command? See _test->speed_ for more info. + +- Speed. Well... obviously, speed is important, but to what extent does a 29 millisecond command really satisfy the user compared to a 294 millisecond command? See _test->speed_ for more info. ```bash > time cat README.md | sed 's/a/x/g' > /dev/null cat myfile 0,00s user 0,00s system 45% cpu 0,011 total sed 's/a/x/g' > /dev/null 0,00s user 0,00s system 43% cpu 0,029 total -> time rr a x README.md -o > /dev/null +> time rr a x README.md -o > /dev/null rr x y myfile -o > /dev/null 0,21s user 0,04s system 86% cpu 0,294 total ``` - - - -## Test +## Test ### Regression + All CLI end to end tests are defined in [test/cli/run.sh](https://github.com/mathiasrw/rexreplace/blob/main/test/cli/run.sh) and all unit test are described in [`test/*.js`](https://github.com/mathiasrw/rexreplace/tree/main/test). After `git clone`'ing the repo and `npm install`'ing you can invoke them with: ```bash @@ -277,26 +247,26 @@ All CLI end to end tests are defined in [test/cli/run.sh](https://github.com/mat ## Performance -_**tl;dr**:_ +_**tl;dr**:_ _Files over 5 Mb are faster with `rr` than with `sed` - but - it does not matter as any file under 25 Mb has less than 0.7 seconds in difference._ -The speed test is initiated by `npm run test-speed`. The test takes files in different sizes and compares the processing time for RexReplace (`rr`) and the Unix tool `sed`. The test uses the sources of a website displaying [the book _1984_ by George Orwell](http://1984.surge.sh). The task for the tests is to remove all HTML tags by search-and-replace, so only the final text is left. The source is 888Kb, so all files up to 500Kb are generated directly from the source, while larger files are created by combining the first 500Kb several times. Each test runs 10 times to even out any temporary workload fluctuations. -Results from the latest speed test can always be seen in the [speed test log](https://github.com/mathiasrw/rexreplace/blob/main/test/speed/testlog.speed.md). +The speed test is initiated by `npm run test-speed`. The test takes files in different sizes and compares the processing time for RexReplace (`rr`) and the Unix tool `sed`. The test uses the sources of a website displaying [the book _1984_ by George Orwell](http://1984.surge.sh). The task for the tests is to remove all HTML tags by search-and-replace, so only the final text is left. The source is 888Kb, so all files up to 500Kb are generated directly from the source, while larger files are created by combining the first 500Kb several times. Each test runs 10 times to even out any temporary workload fluctuations. +Results from the latest speed test can always be seen in the [speed test log](https://github.com/mathiasrw/rexreplace/blob/main/test/speed/testlog.speed.md).

-The graph visualises speed as relative to fastest overall run (`sed` on a 1kb file). This chart also has [an interactive version in log scale](https://plot.ly/~mathiasrw/1.embed), so the details in the low end can be studied better. Interestingly files of 1Kb, 5Kb takes longer for `rr` than 10Kb files. +The graph visualises speed as relative to fastest overall run (`sed` on a 1kb file). This chart also has [an interactive version in log scale](https://plot.ly/~mathiasrw/1.embed), so the details in the low end can be studied better. Interestingly files of 1Kb, 5Kb takes longer for `rr` than 10Kb files. -Now, what is relevant to notice is how `sed` only takes 4.9 seconds longer for the 100Mb file - even if the difference looks drastic on the graph. +Now, what is relevant to notice is how `sed` only takes 4.9 seconds longer for the 100Mb file - even if the difference looks drastic on the graph. ``` Speed relative to fastest tool for each file size --------------------------------------------------- Bytes sed rr Time it took longer (seconds) -1 1 39 0,5 <= sed is 39x faster -5 1 32 0,4 +1 1 39 0,5 <= sed is 39x faster +5 1 32 0,4 10 1 27 0,4 100 1 19 0,3 500 1 7 0,3 @@ -310,39 +280,41 @@ Bytes sed rr Time it took longer (seconds) So even though the speed evolves very differently, there is only little practical use of the focus on speed for most use cases. Replacing in 10.000 small files? Use RexReplace and get yourself a cup of coffee - or spend half an hour getting `sed` to work as you want it to and enjoy the thrilling few seconds it takes to do its magic. -Please note that speeds might look very different when files get as large as the memory available. +Please note that speeds might look very different when files get as large as the memory available. Sure, I can help sharpen up the text for better clarity and impact. While I don't have specific knowledge about the rexreplace project, I understand the general idea conveyed in the text. Here's a revised version: --- +Here's an improved version of the text for your README: + ### Tips and Tricks for Performance -It is much more time expensive to read lots of files, than to have a long search regex. +Reading many files multiple times is far more time-consuming than creating a complex regex and reading each file once. -> **Illustrative Anecdote Time** -> -> Say you've got a bunch of files you want to duplicate, but you also have a list of keys that need to be unique across all the duplicated files. One way to do this is to add `_v2` to each key in the duplicate files. +> **Anecdote time** > -> Using rexreplace for each individual key would be the "default" approach. In a real-world example with 5,000 keys and 10,000 files, this method took 57 minutes. The slow part is that each of the 10,000 files is opened and read 5,000 times plus opening node 5000 times. +> Imagine you need to duplicate a set of files, but each duplicate must have unique keys. To achieve this, you can append `_v2` to each key in the duplicated files. > -> Doing the same task, but with all the keys bundled into a single long search regex using a `(key1|key2|key3|...)` format it only took 2.14 seconds! Even though this creates an nasty 300Kb search regex string, it’s way faster. +> Running a separate `rexreplace` command for each key is a logical approach. However, in a real-world scenario with 5,000 keys across 10,000 files, this method took **57 minutes**. The bottleneck was identifying the 10,000 files 5,000 times, opening, and reading each one repeatedly, plus the startup time for node 5,000 times. > -> ``` -> > rexreplace key1 key1_v2 src/v2/**/*.* -> > rexreplace key2 key2_v2 src/v2/**/*.* -> > rexreplace key3 key3_v2 src/v2/**/*.* +> Alternatively, bundling all keys into a single regex search pattern like `(key1|key2|key3|...)'` reduced the runtime to surprising **2.14 seconds**! Although this resulted in a 300Kb regex string, it was significantly faster. +> +> ```sh +> # Separate commands for each key +> rexreplace key1 key1_v2 src/v2/**/*.* +> rexreplace key2 key2_v2 src/v2/**/*.* +> rexreplace key3 key3_v2 src/v2/**/*.* > ... -> > rexreplace key5000 key5000_v2 src/v2/**/*.* -> => 57 minutes -> -> > rexreplace '(key1|key2|key3|...|key5000)' €1_v2 src/v2/**/*.* -> => 2.14 seconds +> rexreplace key5000 key5000_v2 src/v2/**/*.* +> # Total time: 57 minutes +> +> # Single command for all keys +> rexreplace '(key1|key2|key3|...|key5000)' '$1_v2' src/v2/**/*.* +> # Total time: 2.14 seconds > ``` -> Moral: Choose the right approach might save you time... - - - +> +> Moral: Choosing the right approach can save you a significant amount of time. ## Rumours @@ -350,7 +322,6 @@ It is much more time expensive to read lots of files, than to have a long search _.oO(What should "sed" have looked like by now?)_ - ### Future ideas - Default to RegExmoji lax syntax @@ -360,43 +331,40 @@ _.oO(What should "sed" have looked like by now?)_ - Let search and replace be within the names of the files (ask for overwriting. -Y = no questions) - Let search and replace be within the path of the files (ask for overwriting. -Y = no questions) - Let pattern and globs be piped -- Let pipe be placed into search or replace. +- Let pipe be placed into search or replace. - Let Pattern, replacement, and globs come from a file - Let pattern and glob be javascript code returning a string as the result - Auto string search / replace if no regex magic is used (and verify that speed is better) - Error != warning - Flag for String-literal (no regex, no special chars, no escape chars) to avoid backslashes or remembering which characters needs to be escaped - Check if https://github.com/eugeneware/replacestream is good to rely on -- Check if regex engine from spider monkey can be wrapped in something that does not need node +- Check if regex engine from spider monkey can be wrapped in something that does not need node - Check if sd can be wrapped in a WASM module - Set engine to RE2 https://www.npmjs.com/package/re2 - Set engine to run on streams with https://www.npmjs.com/package/replacestream - Implement in go so that all platforms can be supported with no need for node (might be based on) - Let https://github.com/dthree/vorpal deal with the interface? Or maybe https://www.npmjs.com/package/pretty-cli - ## Related projects Many projects are seeking to solve the same problem as RexReplace. I my oppinion they all lack the flexibility of the RexReplace CLI interface and diverse the replacement can be, but have other strong features. If our way does not suit you, we suggest you have a look at: - [sd](https://github.com/chmln/sd) - Blazingly fast! 5.9x **faster** than RexReplace. Almost perfect. Using _"Good ol' unix philosophy to the rescue."_ for pinpointing files so not always easy. Get it via cargo for Rust. -- [replace-in-file](https://www.npmjs.com/package/replace-in-file) - same speed as RexReplace. Very good support to be used as a module for a node application. A bit cumbersome CLI notation for pinpointing files. Get via npm. +- [replace-in-file](https://www.npmjs.com/package/replace-in-file) - same speed as RexReplace. Very good support to be used as a module for a node application. A bit cumbersome CLI notation for pinpointing files. Get via npm. - [replace-x](https://www.npmjs.com/package/replace-x) - 3.8x slower than RexReplace. A bit cumbersome CLI notation for pinpointing files. Get it via npm. - [replace](https://www.npmjs.com/package/replace) - 2.6x slower than RexReplace. Very chatty default output. A bit more cumbersome CLI notation for pinpointing files. Get it via npm. - ----- - +--- RexReplace mascot Benny on the RexReplace logBo -Please note that RexReplace is an [OPEN open source software](http://open-oss.com) project. +Please note that RexReplace is an [OPEN open source software](http://open-oss.com) project. This means that individuals making significant and valuable contributions are given commit access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project. -[![OPEN open source software](https://img.shields.io/badge/Open--OSS-%E2%9C%94-brightgreen.svg)](http://open-oss.com) - +[![OPEN open source software](https://img.shields.io/badge/Open--OSS-%E2%9C%94-brightgreen.svg)](http://open-oss.com) _Icon inspired by Freepik from www.flaticon.com_ ## License + [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fmathiasrw%2Frexreplace.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fmathiasrw%2Frexreplace?ref=badge_large)