feat: auto browser download #1029

KuznetsovRoman · 2024-11-13T01:22:26Z

What is done

Added ability to automatically download browsers and/or webdrivers for these browsers.

Supported browsers to download:

chrome > 73
firefox

Supported drivers to download:

chromedriver (for chrome)
geckodriver (for firefox)
edgedriver (for edge)

With "--local" (or gridUrl: "local") and webdriver protocol, webdiver instances are automatically run for these browsers:

chrome (>73)
firefox
edge (if the browser itself is installed. Driver will be installed automatically)
safari (mac only)

When launching tests with "--local" (and optionally webdriver protocol) all supported browsers/drivers will be installed automatically, if necessary. With "--local" we at all cost are trying to avoid network requests to google/mozilla/... servers.

Example

Set up a testplane config, using webdriver protocol and describe some chrome/firefox browsers with defined versions
Launch eigher using npx testplane --local, either by setting "gridUrl": "local"

In order to just download browser, npx testplane install-browsers could be used. This way we also check if our browsers are up-to-date, so if we have 114.0.5696.0 downloaded and user wants chrome@114.0, we check for newest chrome@114.0 version and see it is "114.0.5735.133". Then this new version is downloaded

If used without arguments, all of the config browsers will be installed
If argument is browserId from config, the corresponding browser will be installed
If argument is @, it will be installed.
These can be combined: npx testplane install-browsers my-chrome chrome@115

Directory to save testplane browsers and drivers can be set with TESTPLANE_BROWSERS_PATH env variable. By default, ~/.testplane is used.

KuznetsovRoman · 2024-11-13T01:23:20Z

package.json

@@ -54,6 +54,7 @@
    "@babel/code-frame": "7.24.2",
    "@gemini-testing/commander": "2.15.4",
    "@jspm/core": "2.0.1",
+    "@puppeteer/browsers": "2.4.0",


download chrome

download chromium (replaces chrome for versions >73, <115)

download firefox

KuznetsovRoman · 2024-11-13T01:25:50Z

package.json

@@ -64,12 +65,16 @@
    "bluebird": "3.5.1",
    "chalk": "2.4.2",
    "clear-require": "1.0.1",
+    "cli-progress": "3.12.0",


multiple progress bar

KuznetsovRoman · 2024-11-13T01:26:22Z

package.json

    "debug": "2.6.9",
    "devtools": "8.39.0",
+    "edgedriver": "5.6.1",


download edge driver

KuznetsovRoman · 2024-11-13T01:27:02Z

package.json

    "error-stack-parser": "2.1.4",
    "expect-webdriverio": "3.6.0",
+    "extract-zip": "2.0.1",


extract zip archive with chromedriver, which is installed manually (>73, <115)

KuznetsovRoman · 2024-11-13T01:27:21Z

package.json

    "fastq": "1.13.0",
    "fs-extra": "5.0.0",
+    "geckodriver": "4.5.0",


download and run geckodriver

KuznetsovRoman · 2024-11-13T01:46:23Z

src/browser-installer/utils.ts

+    if (platform === BrowserPlatform.MAC_ARM && Number(milestone) < MIN_CHROMIUM_MAC_ARM_VERSION) {
+        return BrowserPlatform.MAC;
+    }


for chrome, use mac64 platform, if chromium version is old and mac_arm version does not exist

KuznetsovRoman · 2024-11-13T01:47:04Z

src/browser-installer/utils.ts

+export const getFirefoxBrowserDir = (): string => getBrowsersDir(); // path is set by @puppeteer/browsers.install
+export const getChromeBrowserDir = (): string => getBrowsersDir(); // path is set by @puppeteer/browsers.install
+
+export const retryFetch = async (


As we fetch some data from github, retries wont hurt

KuznetsovRoman · 2024-11-13T01:47:45Z

src/browser-pool/basic-pool.ts

@@ -26,10 +28,11 @@ export class BasicPool implements Pool {

        this._activeSessions = {};
        this._cancelled = false;
+        this._wdPool = new WebdriverPool();


all browsers of the same browser-pool share WebdriverPool,
which is passed by reference to NewBrowser instance

KuznetsovRoman · 2024-11-13T01:48:59Z

src/browser-pool/webdriver-pool.ts

+type ChildProcessWithStatus = { process: ChildProcess; gridUrl: string; isBusy: boolean };
+export type WdProcess = { gridUrl: string; free: () => void; kill: () => void };
+
+export class WebdriverPool {


We have N browsers.
This class handles webdriver processes:

we can reuse existing webdriver process, which is free because browser, which requested webdriver process, no longer needs it (browser.quit is called)

we need to launch new webdriver process (if all existing webdriver processes are busy and we need to launch NewBrowser)

WebdriverPool can reuse webdriver process between browsers with different browserId, if their browser and browserVersion matches

port in this class acts like process id as it is unique

KuznetsovRoman · 2024-11-13T01:49:54Z

src/browser/new-browser.ts

            warn(`WARNING: Can not close session: ${(e as Error).message}`);
+            this._wdProcess?.kill();


If session can't be closed, we kill webdriver process, so later we could launch the new one

KuznetsovRoman · 2024-11-13T10:45:10Z

src/browser/new-browser.ts

+        if (this._isLocalGridUrl() && config.automationProtocol === "webdriver") {
+            gridUrl = await this._getLocalWebdriverGridUrl();
+        } else {
+            // if automationProtocol is not "webdriver", fallback to default grid url from "local"
+            // because in "devtools" protocol we dont need gridUrl, but it still has to be valid URL
+            gridUrl = config.gridUrl === LOCAL_GRID_URL ? DEFAULT_GRID_URL : config.gridUrl;
+        }


"isLocalGridUrl" === either "gridUrl" is "local", either testplane is launched with --local

If isLocalGridUrl == true and automationProtocol === "webdriver", we get webdriver process and use its gridUrl

if isLocalGridUrl == true and automationProtocol is "devtools", gridUrl won't be used either way, but we need to pass any valid url

KuznetsovRoman · 2024-11-13T10:45:57Z

src/browser/new-browser.ts

+        return this._isLocalGridUrl()
+            ? this._addExecutablePath(config, capabilitiesWithAddedHeadless)


Here we don't check for automationProtocol, so we could use our binaries for "devtools" automation protocol

KuznetsovRoman · 2024-11-13T10:47:36Z

src/browser/new-browser.ts

+        const executablePath = await installBrowser(
+            this._config.desiredCapabilities?.browserName as SupportedBrowser,
+            this._config.desiredCapabilities?.browserVersion as string,
+        );


Installs browser (if necessary) and returns its executable path
For safari and edge null is returned (we won't set binary path in capabilities, but wdio will be able to find them, if they are installed)

KuznetsovRoman · 2024-11-13T10:56:55Z

src/browser-installer/chrome/browser.ts

+            unpack: true,
+        }).then(result => result.executablePath);
+
+    return installBinary(Browser.CHROME, platform, buildId, installFn);


registry.installBinary does not mean we will call "installFn" function.
If multiple "installBinary" are called with same browser platform and buildId, "installFn" would only be called at most once. And if we already have it installed, "installFn" wouldn't be called

KuznetsovRoman commented Nov 13, 2024

View reviewed changes

KuznetsovRoman force-pushed the TESTPLANE-269.auto_browsers branch from 46071d9 to a3693cd Compare November 13, 2024 11:07

feat: auto browser download

58c27d2

KuznetsovRoman force-pushed the TESTPLANE-269.auto_browsers branch from a3693cd to 58c27d2 Compare November 13, 2024 11:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: auto browser download #1029

feat: auto browser download #1029

KuznetsovRoman commented Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

		warn(`WARNING: Can not close session: ${(e as Error).message}`);
		this._wdProcess?.kill();

		return this._isLocalGridUrl()
		? this._addExecutablePath(config, capabilitiesWithAddedHeadless)

feat: auto browser download #1029

Are you sure you want to change the base?

feat: auto browser download #1029

Conversation

KuznetsovRoman commented Nov 13, 2024 • edited Loading

What is done

Example

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

KuznetsovRoman Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman commented Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading