Proxy config seems to be causing problems related to memory leak

[cybairfly]

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/playwright (PlaywrightCrawler)

Issue description

I have problems with an actor only when a proxy config is used. The crawler gets stuck on loading the page infinitely, and there are memory overload warnings in the logs - all is good when the proxy config is not used. Not sure what's going on and perhaps it is related to this as the closest thing I could find but honestly didn't look into it much: apify/proxy-chain#146 @szmarczak

Code sample

const proxyConfiguration = input.proxy && await Actor.createProxyConfiguration({
	apifyProxyGroups: input.proxy.apifyProxyGroups,
	apifyProxyCountry: input.proxy.apifyProxyCountry,
});
...

const crawler = new PlaywrightCrawler({
	// proxyConfiguration,

Package version

latest

Node.js version

21

Operating system

Win10

Apify platform

• Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

Running on platform there are no memory warnings showing up but instead getting this constantly:

WARN  PlaywrightCrawler: Reclaiming failed request back to the list or queue. Navigation timed out after 60 seconds.

[cybairfly]

Input schema:

"proxy": {
            "sectionCaption": "Proxy",
            "title": "Proxy",
            "type": "object",
            "description": "Define proxy to pass data through",
            "prefill": {
                "useApifyProxy": false
            },
            "default": {
                "useApifyProxy": false
            },
            "editor": "proxy"
        }

Runtime proxy config:

{
  isManInTheMiddle: false,
  nextCustomUrlIndex: 0,
  proxyUrls: undefined,
  tieredProxyUrls: undefined,
  usedProxyUrls: {
  },
  newUrlFunction: undefined,
  log: {
    LEVELS: {
      "0": "OFF",
      "1": "ERROR",
      "2": "SOFT_FAIL",
      "3": "WARNING",
      "4": "INFO",
      "5": "DEBUG",
      "6": "PERF",
      OFF: 0,
      ERROR: 1,
      SOFT_FAIL: 2,
      WARNING: 3,
      INFO: 4,
      DEBUG: 5,
      PERF: 6,
    },
    options: {
      level: 4,
      maxDepth: 4,
      maxStringLength: 2000,
      prefix: "ProxyConfiguration",
      suffix: null,
      logger: {
        _events: {
        },
        _eventsCount: 0,
        _maxListeners: undefined,
        options: {
          skipTime: true,
        },
      },
      data: {
      },
    },
    warningsOnceLogged: {
    },
  },
  domainTiers: {
  },
  config: {
    options: {
    },
    services: {
    },
    storageManagers: {
    },
  },
  groups: [
  ],
  countryCode: undefined,
  hostname: "proxy.apify.com",
  port: 8000,
  usesApifyProxy: true,
}

[cybairfly]

I am now only able to reproduce this consistently, only locally, not on the platform. Presence of proxy config leads to memory warnings and thousands of network-level errors in the browser, while its absence is problem free (with the proxy config prop commented out). Target is techcrunch.com

Perhaps the proxy config is somehow corrupt locally, when the platform proxy input is not used?

[B4nan]

Without a complete reproduction, I am not sure how we could help here. The presence of a proxy config itself is not the culprit (most likely), we would be well aware if that would be the case.