Transformer Benchmark for Oxc and Swc

A transformer is also known as a transpiler, similar to Babel transform.

The purpose of this benchmark is for people who wants to evaluate and compare the performance characteristics of the two transformer.

The benchmark measures the whole parse -> transform -> codegen pipeline as a real-word scenario.

The numbers indicate that Oxc is at least 2 times faster than Swc.

Results

Codspeed

Codspeed measures performance by cpu instructions.

Mac mini M2 8 cores

cal.com.tsx

	oxc	swc
no-drop	14.4 ms (1.00x)	31.5 ms (2.20x)
parallel	26.9 ms (1.00x)	69.4 ms (2.59x)
single-thread	14.4 ms (1.00x)	32.3 ms (2.25x)

typescript.js

	oxc	swc
no-drop	106.7 ms (1.00x)	213.8 ms (2.00x)
parallel	195.8 ms (1.00x)	437.2 ms (2.23x)
single-thread	106.0 ms (1.00x)	221.7 ms (2.09x)

MacBook Pro M3 Max

cal.com.tsx

	oxc	swc
no-drop	12.4 ms (1.00x)	31.1 ms (2.50x)
parallel	21.5 ms (1.00x)	58.2 ms (2.70x)
single-thread	12.4 ms (1.00x)	32.1 ms (2.59x)

typescript.js

	oxc	swc
no-drop	95.5 ms (1.00x)	204.0 ms (2.14x)
parallel	140.0 ms (1.00x)	373.0 ms (2.66x)
single-thread	95.5 ms (1.00x)	211.8 ms (2.22x)

Run benchmark locally

Run the following command on your machine for replication.

cargo bench

Generate the table

pnpm i
pnpm run table

Maximum Resident Set Size

./memory.sh

./files/cal.com.tsx
oxc 41.7 mb
swc 31.1 mb

./files/typescript.js
oxc 224.4 mb
swc 160.1 mb

Setup

• Uses mimalloc as the global allocator
• Uses the following release profile

[profile.release]
opt-level     = 3
lto           = "fat"
codegen-units = 1
strip         = "symbols"
debug         = false
panic         = "abort"

single-thread

This is the standard benchmark run in a single thread.

group.bench_with_input(id, &source, |b, source| {
    b.iter(|| Self::run(source))
});

no-drop

This uses the iter_with_large_drop function, which does not take AST drop time into account.

AST drop time can become a bottleneck in applications such as as bundler, where there are a few thousands of files need to be parsed.

group.bench_with_input(id, &source, |b, source| {
    b.iter_with_large_drop(|| Self::run(source))
});

parallel

This benchmark uses the total number of physical cores as the total number of files to parse per bench iteration. For example it parses 6 files in parallel on my Mac i7 6 cores.

This can indicate the existence of resource contention.

let cpus = num_cpus::get_physical();
group.bench_with_input(id, &source, |b, source| {
    b.iter(|| {
        (0..cpus).into_par_iter().for_each(|_| {
            Self::run(source);
        });
    })
});