Docs and static bin release

.github/workflows/cd.yml

@@ -29,12 +29,9 @@ jobs:
           token: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Build binary release
-        run: cargo build --features cli-bin --release --locked
-
-      # TODO: troubleshoot static binaries
-      #run: |
-      #  rustup target add x86_64-unknown-linux-musl
-      #  cargo build --target x86_64-unknown-linux-musl --release --locked
+        run: |
+          rustup target add x86_64-unknown-linux-musl
+          cargo build --target x86_64-unknown-linux-musl --features cli-bin --release --locked
 
       - name: Create Release
         id: create_release

README.md

@@ -4,7 +4,7 @@
 ![GitHub Actions](https://github.com/entropic-security/xgadget/workflows/test/badge.svg)
 
 Fast, parallel, cross-variant ROP/JOP gadget search for x86 (32-bit) and x64 (64-bit) binaries.
-Uses official Rust bindings for the [zydis disassembler library](https://github.com/zyantific/zydis).
+Uses the [iced-x86 disassembler library](https://github.com/0xd4d/iced).
 
 **Current state:** decent test coverage, but still in beta. Issues/PRs welcome :)
 
@@ -59,7 +59,7 @@ let cross_reg_write_gadgets = xgadget::filter_stack_set_regs(&cross_gadgets);
 Run `xgadget --help`:
 
 ```
-xgadget v0.3.0
+xgadget v0.4.0
 
 About:  Fast, parallel, cross-variant ROP/JOP gadget search for x86/x64 binaries.
 Cores:  8 logical, 8 physical
@@ -73,7 +73,8 @@ FLAGS:
     -d, --dispatcher       Filter to potential JOP 'dispatcher' gadgets [default: all gadgets]
     -e, --extended-fmt     Print in terminal-wide format [default: only used for partial match search]
     -h, --help             Prints help information
-    -i, --imm16            Include '{ret, ret far} imm16' (e.g. add to stack ptr) [default: don't include]
+        --inc-call         Include gadgets containing a call [default: don't include]
+        --inc-imm16        Include '{ret, ret far} imm16' (e.g. add to stack ptr) [default: don't include]
     -j, --jop              Search for JOP gadgets only [default: ROP, JOP, and SYSCALL]
     -n, --no-color         Don't color output, useful for UNIX piping [default: color output]
     -m, --partial-match    Include cross-variant partial matches [default: full matches only]
@@ -98,17 +99,16 @@ ARGS:
 Build from source and install locally:
 
 ```bash
-sudo apt-get install cmake                  # Ubuntu-specific, adjust for your package manager
 cargo install xgadget --features cli-bin    # Build on host (pre-req: https://www.rust-lang.org/tools/install)
 ```
 
 ### CLI Binary Releases for Linux
 
-Commits to this repo's `master` branch automatically run integration tests and build a dynamically-linked binary for 64-bit Linux.
+Commits to this repo's `master` branch automatically run integration tests and build a statically-linked binary for 64-bit Linux.
 You can [download it here](https://github.com/entropic-security/xgadget/releases) and use the CLI immediately, instead of building from source.
-Static binaries for Linux and Windows may be supported in the future.
+Static binaries for Windows may also be supported in the future.
 
-### ~~Yeah, but can it do 10 OS kernels in 30 seconds?!~~ Repeatable Benchmark Harness
+### ~~Yeah, but can it do 10 OS kernels under 10 seconds?!~~ Repeatable Benchmark Harness
 
 ```bash
 bash ./benches/bench_setup_ubuntu.sh    # Ubuntu-specific, download/build 10 kernel versions
@@ -118,11 +118,7 @@ cargo bench                             # Grab a coffee, this'll take a while...
 * `bench_setup_ubuntu.sh` downloads and builds 10 consecutive Linux kernels (versions `5.0.1` to `5.0.10` - with `x86_64_defconfig`).
 * `cargo bench`, among other benchmarks, searches all 10 kernels for common gadgets.
 
-On an i7-9700K (8C/8T, 3.6GHz base, 4.9 GHz max, e.g. an older-gen consumer CPU) machine with `gcc` version 8.4.0: the average runtime, to process *all ten 54MB kernels simultaneously* with a max gadget length of 5 instructions and full-match search for all gadget types (ROP, JOP, and syscall gadgets), is *only 31 seconds*!
-
-Note this is a statistical benchmark that samples from many iterations, and requires a lot of RAM (> 32GB). If you just want to run `xgadget` on the 10 kernels once, use `./benches/run_on_bench_kernels.sh`.
-
-Searching all 10 kernels for *both* partial and full matches is still in beta, no benchmarks yet (implemented but not yet optimized). Because of the performance hit and the lower utility of partial gadget matches, this search option is disabled by default. It can be enabled with the `--partial-match` flag for the CLI, or via setting a configuration bit, e.g. `search_config |=  xgadget::SearchConfig::PART`, for the library API. Conversely, removing default options improves performance: searching all 10 kernels for only ROP gadgets (ignoring JOP and syscall gadgets) takes just 22 seconds. `xgadget` is designed to scale for large binaries while being easily configurable.
+On an i7-9700K (8C/8T, 3.6GHz base, 4.9 GHz max) machine with `gcc` version 8.4.0: the average runtime, to process *all ten 54MB kernels simultaneously* with a max gadget length of 5 instructions and full-match search for all gadget types (ROP, JOP, and syscall gadgets), is *only 5.8 seconds*! Including partial matches as well takes *just 7.2 seconds*.
 
 ### Acknowledgements
 
@@ -134,5 +130,5 @@ This project started as an optimized solution to Chapter 8, exercise 3 of "Pract
 * [2] [`rayon` crate by Josh Stone, Niko Matsakis](https://crates.io/crates/rayon)
 * [3] [`xgadget/.github/workflows`](https://github.com/entropic-security/xgadget/tree/master/.github/workflows)
 * [4] [`criterion` crate by Brook Heisler, Jorge Aparicio](https://crates.io/crates/criterion)
-* [5] [`zydis` bindings by Joel Honer, Timo von Hartz](https://crates.io/crates/zydis)
+* [5] [`iced-x86` crate by 0xd4d](https://crates.io/crates/iced-x86)
 * [6] ["Practical Binary Analysis" by Dennis Andreisse](https://practicalbinaryanalysis.com/)

benches/bench_3_elf_kernels.rs

@@ -17,11 +17,22 @@ fn elf_kernel_bench(c: &mut Criterion) {
         .map(|path| xgadget::Binary::from_path_str(path.to_str().unwrap()).unwrap())
         .collect();
 
-    c.bench_function("10_kernel_search", |b| {
+    c.bench_function("10_kernel_search_full_match", |b| {
         b.iter(|| {
             xgadget::find_gadgets(&bins, MAX_GADGET_LEN, xgadget::SearchConfig::DEFAULT).unwrap()
         })
     });
+
+    c.bench_function("10_kernel_search_part_match", |b| {
+        b.iter(|| {
+            xgadget::find_gadgets(
+                &bins,
+                MAX_GADGET_LEN,
+                xgadget::SearchConfig::DEFAULT | xgadget::SearchConfig::PART,
+            )
+            .unwrap()
+        })
+    });
 }
 
 // Runner --------------------------------------------------------------------------------------------------------------

src/cli/cli.rs

@@ -93,7 +93,7 @@ struct CLIOpts {
     stack_pivot: bool,
 
     /// Filter to potential JOP 'dispatcher' gadgets [default: all gadgets]
-    #[structopt(short, long, conflicts_with = "stack_pivot")]
+    #[structopt(short, long, conflicts_with_all = &["rop", "stack_pivot"])]
     dispatcher: bool,
 
     /// Filter to 'pop {reg} * 1+, {ret or ctrl-ed jmp/call}' gadgets [default: all gadgets]

src/filters.rs

@@ -1,6 +1,5 @@
 use std::collections::{BTreeMap, BTreeSet};
 
-use iced_x86;
 use rayon::prelude::*;
 
 use crate::gadget;

src/gadget.rs

@@ -2,8 +2,6 @@ use std::cmp::Ordering;
 use std::collections::{BTreeMap, BTreeSet};
 use std::hash::{Hash, Hasher};
 
-use iced_x86;
-
 use crate::binary;
 
 // TODO: implement Ord for binary, use BTReeSet instead of Vector to maintain sorted order on insertion - will have nicer output at partial match at cost of speed (how much?)
@@ -48,11 +46,7 @@ impl<'a> Gadget<'a> {
         match self.partial_matches.get_mut(&addr) {
             Some(bins) => bins.push(bin),
             None => {
-                // TODO: Use unwrap_none() once on stable
-                match self.partial_matches.insert(addr, vec![bin]) {
-                    Some(_) => return,
-                    None => return,
-                }
+                self.partial_matches.insert(addr, vec![bin]);
             }
         };
     }

src/lib.rs

@@ -1,7 +1,7 @@
 //!# xgadget
 //!
-//!Fast, parallel, cross-variant ROP/JOP gadget search for x86 (32-bit), and x64 (64-bit) binaries.
-//!Uses official Rust bindings for the [zydis disassembler library](https://github.com/zyantific/zydis).
+//!Fast, parallel, cross-variant ROP/JOP gadget search for x86 (32-bit) and x64 (64-bit) binaries.
+//!Uses the [iced-x86 disassembler library](https://github.com/0xd4d/iced).
 //!
 //!**Current state:** decent test coverage, but still in beta. Issues/PRs welcome :)
 //!
@@ -56,7 +56,7 @@
 //!Run `xgadget --help`:
 //!
 //!```ignore
-//!xgadget v0.3.0
+//!xgadget v0.4.0
 //!
 //!About:   Fast, parallel, cross-variant ROP/JOP gadget search for x86/x64 binaries.
 //!Cores:   8 logical, 8 physical
@@ -70,7 +70,8 @@
 //!    -d, --dispatcher       Filter to potential JOP 'dispatcher' gadgets [default: all gadgets]
 //!    -e, --extended-fmt     Print in terminal-wide format [default: only used for partial match search]
 //!    -h, --help             Prints help information
-//!    -i, --imm16            Include '{ret, ret far} imm16' (e.g. add to stack ptr) [default: don't include]
+//!        --inc-call         Include gadgets containing a call [default: don't include]
+//!        --inc-imm16        Include '{ret, ret far} imm16' (e.g. add to stack ptr) [default: don't include]
 //!    -j, --jop              Search for JOP gadgets only [default: ROP, JOP, and SYSCALL]
 //!    -n, --no-color         Don't color output, useful for UNIX piping [default: color output]
 //!    -m, --partial-match    Include cross-variant partial matches [default: full matches only]
@@ -95,17 +96,16 @@
 //!Build from source and install locally:
 //!
 //!```bash
-//!sudo apt-get install cmake                  # Ubuntu-specific, adjust for your package manager
 //!cargo install xgadget --features cli-bin    # Build on host (pre-req: https://www.rust-lang.org/tools/install)
 //!```
 //!
 //!### CLI Binary Releases for Linux
 //!
-//!Commits to this repo's `master` branch automatically run integration tests and build a dynamically-linked binary for 64-bit Linux.
+//!Commits to this repo's `master` branch automatically run integration tests and build a statically-linked binary for 64-bit Linux.
 //!You can [download it here](https://github.com/entropic-security/xgadget/releases) and use the CLI immediately, instead of building from source.
-//!Static binaries for Linux and Windows may be supported in the future.
+//!Static binaries for Windows may also be supported in the future.
 //!
-//!### ~~Yeah, but can it do 10 OS kernels in 30 seconds?!~~ Repeatable Benchmark Harness
+//!### ~~Yeah, but can it do 10 OS kernels under 10 seconds?!~~ Repeatable Benchmark Harness
 //!
 //!```bash
 //!bash ./benches/bench_setup_ubuntu.sh    # Ubuntu-specific, download/build 10 kernel versions
@@ -115,11 +115,7 @@
 //!* `bench_setup_ubuntu.sh` downloads and builds 10 consecutive Linux kernels (versions `5.0.1` to `5.0.10` - with `x86_64_defconfig`).
 //!* `cargo bench`, among other benchmarks, searches all 10 kernels for common gadgets.
 //!
-//!On an i7-9700K (8C/8T, 3.6GHz base, 4.9 GHz max, e.g. an older-gen consumer CPU) machine with `gcc` version 8.4.0: the average runtime, to process *all ten 54MB kernels simultaneously* with a max gadget length of 5 instructions and full-match search for all gadget types (ROP, JOP, and syscall gadgets), is *only 31 seconds*!
-//!
-//!Note this is a statistical benchmark that samples from many iterations, and requires a lot of RAM (> 32GB). If you just want to run `xgadget` on the 10 kernels once, use `./benches/run_on_bench_kernels.sh`.
-//!
-//!Searching all 10 kernels for *both* partial and full matches is still in beta, no benchmarks yet (implemented but not yet optimized). Because of the performance hit and the lower utility of partial gadget matches, this search option is disabled by default. It can be enabled with the `--partial-match` flag for the CLI, or via setting a configuration bit, e.g. `search_config |=  xgadget::SearchConfig::PART`, for the library API. Conversely, removing default options improves performance: searching all 10 kernels for only ROP gadgets (ignoring JOP and syscall gadgets) takes just 22 seconds. `xgadget` is designed to scale for large binaries while being easily configurable.
+//!On an i7-9700K (8C/8T, 3.6GHz base, 4.9 GHz max) machine with `gcc` version 8.4.0: the average runtime, to process *all ten 54MB kernels simultaneously* with a max gadget length of 5 instructions and full-match search for all gadget types (ROP, JOP, and syscall gadgets), is *only 5.8 seconds*! Including partial matches as well takes *just 7.2 seconds*.
 //!
 //!### Acknowledgements
 //!
@@ -131,7 +127,7 @@
 //!* \[2\] [`rayon` crate by Josh Stone, Niko Matsakis](https://crates.io/crates/rayon)
 //!* \[3\] [`xgadget/.github/workflows`](https://github.com/entropic-security/xgadget/tree/master/.github/workflows)
 //!* \[4\] [`criterion` crate by Brook Heisler, Jorge Aparicio](https://crates.io/crates/criterion)
-//!* \[5\] [`zydis` bindings by Joel Honer, Timo von Hartz](https://crates.io/crates/zydis)
+//!* \[5\] [`iced-x86` crate by 0xd4d](https://crates.io/crates/iced-x86)
 //!* \[6\] ["Practical Binary Analysis" by Dennis Andreisse](https://practicalbinaryanalysis.com/)
 
 // Macro Import --------------------------------------------------------------------------------------------------------

src/search.rs

@@ -1,7 +1,6 @@
 use std::collections::BTreeSet;
 use std::error::Error;
 
-use iced_x86;
 use rayon::prelude::*;
 use rustc_hash::{FxHashMap as HashMap, FxHashSet as HashSet};
 

src/str_fmt.rs

@@ -1,7 +1,6 @@
 use std::collections::BTreeMap;
 
 use colored::Colorize;
-use iced_x86;
 use rayon::prelude::*;
 
 use crate::binary;
@@ -190,7 +189,7 @@ fn set_color(s: &str, kind: iced_x86::FormatterTextKind) -> colored::ColoredStri
         iced_x86::FormatterTextKind::Directive | iced_x86::FormatterTextKind::Keyword => s.blue(),
         iced_x86::FormatterTextKind::Prefix | iced_x86::FormatterTextKind::Mnemonic => s.cyan(),
         iced_x86::FormatterTextKind::Register => s.yellow(),
-        iced_x86::FormatterTextKind::Number => s.white().bold(),
+        iced_x86::FormatterTextKind::Punctuation => s.bright_magenta(),
         _ => s.white(),
     }
 }

tests/test_semantics.rs

@@ -121,7 +121,10 @@ fn test_sys_semantics() {
 fn test_rw_semantics() {
     let add_rax_0x08: [u8; 4] = [0x48, 0x83, 0xc0, 0x08];
     let instr = common::decode_single_x64_instr(0, &add_rax_0x08);
-    assert!(xgadget::semantics::is_reg_rw(&instr, &iced_x86::Register::RAX));
+    assert!(xgadget::semantics::is_reg_rw(
+        &instr,
+        &iced_x86::Register::RAX
+    ));
 }
 
 #[test]