Slower compared to got and fetch

[VoidMonk]

Hi, I'm trying got-scraping for a project, and noticed that it's almost 2x slower compared to got and native fetch.

Here's some basic test code:

import { got, gotScraping } from 'got-scraping';

const url = 'https://httpbin.org/html';
let t, r;

t = performance.now();
r = await fetch(url);
console.log(`fetch took ${performance.now() - t}ms`);

t = performance.now();
r = await got(url);
console.log(`got took ${performance.now() - t}ms`, JSON.stringify(r.timings.phases));

t = performance.now();
r = await gotScraping(url);
console.log(`gotScraping took ${performance.now() - t}ms`, JSON.stringify(r.timings.phases));

Results (similar timing difference on multiple runs, even with other URLs):

fetch took 987.2981999991462ms

got took 960.8908999999985ms {"wait":2,"dns":2,"tcp":234,"tls":477,"request":1,"firstByte":235,"download":3,"total":954}

gotScraping took 1676.9162999996915ms {"wait":719,"request":null,"firstByte":239,"download":2,"total":960}

My test machine is running Windows 11 and Node.js 18.12.1.

Why is got-scraping is slower, if not any faster, and can it be improved?

[B4nan]

This library is an extension on top of got - it can only be slower than that as it does more. The purpose is to mimic real browser requests, so not to be faster than the original library, but rather to get around some bot detections, etc.

Btw such tests are not very accurate, I ran your script now 3 times, and things were wildly different on each run, got scraping being faster than got in 2 out of 3 runs. I can observe ~2x slowdown on a less popular site with better performance, but as mentioned above - the goal is to get around blocking, not to be faster.