perfect

JIT playground for microbenchmarking, one-off experiments, and other half-baked ideas. Unlike eigenform/lamina, this relies on the "raw events" exposed via the Linux perf API.

All of this relies heavily on CensoredUsername/dynasm-rs for generating code during runtime, and you will probably want to read the dynasm-rs documentation.

perfect/         - Main library crate
perfect-zen2/    - Zen2 experiments
perfect-tremont/ - Tremont experiments
scripts/         - Miscellaneous scripts

Experiments

All of the experiments here are [sometimes very contrived] programs used to demonstrate, observe, and document different microarchitectural implementation details. Apart from being executable, these are all written with the intention of being read and understood.

Note that most of the interesting experiments here are probably only relevant for the Zen 2 microarchitecture (and potentially later Zen iterations).

Optimizations

• Memory Renaming Eligibility
• Store-to-Load Forwarding Eligibility
• Move Elimination and Zero Idioms

Resources

• Integer PRF Capacity
• FP/Vector PRF Capacity
• Store Queue Capacity

Predictors

• Branch Direction Prediction
• Branch Target Prediction
• Direction Predictor Stimulus/Response

Miscellania

• Validating/Discovering PMC Events
• Observing CVE-2023-20593 (Zenbleed)
• Observing Speculative Loads with Timing

Environment

There are a bunch of scripts that you're expected to use to configure your environment before running any experiments:

• Most [if not all] experiments rely on the RDPMC instruction, which you'll need to enable with ./scripts/rdpmc.sh

• Most [if not all] experiments are intended to be used with SMT disabled, see ./scripts/smt.sh

• Most [if not all] experiments rely on vm.mmap_min_addr being set to zero, see ./scripts/low-mmap.sh

• ./scripts/freq.sh will disable cpufreq boost and change the governor; you probably want to change this for your setup

You can also use the perfect-env binary to check/validate some of this:

$ cargo build --release --bin perfect-env
...

$ sudo ./target/release/perfect-env
[*] 'perfect' environment summary:
  online cores                            : 32
  isolated cores                          : disabled
  nohz_full cores                         : disabled
  simultaneous multithreading (SMT)       : enabled [!!]
  cpufreq boost                           : enabled [!!]
  userspace rdpmc                         : disabled [!!]
  vm.mmap_min_addr                        : 65536

Most of the experiments are intended to be used while booting Linux with the following kernel command-line options (where N is the core you expect to be running experiments on):

isolcpus=nohz,domain,managed_irq,N nohz_full=N

This should [mostly] prevent interrupts, and [mostly] prevent Linux from scheduling tasks on the core.

WARNING:

Under normal circumstances (without isolcpus), the Linux watchdog timer relies on counter #0 being configured automatically by the perf subsystem.

Our use of the perf-event crate only ever configures the first available counter. This means that uses of RDPMC in measured code must read from counter #1. However, while using an isolated core, the watchdog timer is not configured, and measured code must use RDPMC to read from counter #0 instead.

You're expected to keep this in mind while writing experiments.

Harness Configuration

See ./perfect/src/harness.rs for more details.

1. The default configuration tries allocate the low 256MiB of virtual memory (from 0x0000_0000_0000_0000 to 0x0000_0000_1000_0000). This is used to simplify some things by allowing us to emit loads and stores with simple immediate addressing. If the vm.mmap_min_addr sysctl knob isn't set to zero, this will cause you to panic when emitting the harness.

2. The default configuration tries to allocate 64MiB at virtual address 0x0000_1337_0000_0000 for emitting the harness itself.

3. The default configuration also pins the current process to core #15. You may want to change this to something suitable for your own setup, ie.

   use perfect::*;
   
   fn main() {
       let harness = HarnessConfig::default_zen2()
           .pinned_core(3)
           .emit();
       ...
   }

Running Experiments

Typical usage looks something like this:

$ sudo ./scripts/smt.sh off
$ sudo ./scripts/rdpmc.sh on
$ sudo ./scripts/freq.sh on
$ sudo ./scripts/low-mmap.sh on

# Run an experiment
$ cargo run --release -p perfect-zen2 --bin <experiment>
...