Artillery + Playwright: Login once using before

[staramani-ut]

Hi, all! It's me again, your novice who is trying to use artillery + playwright.

I am using the Playwright engine with artillery and I am running into a scenario where I want to authenticate just once and then have all the vusers share that authenticated session. I tried a before but that did not seem to work.

I also have my Playwright code using a setup method.

Any help would be much appreciated. I think I am missing something obvious.

Thank you for the help!

[angelo-loria]

I'm also confused on how before/beforeScenario works with the Playwright engine. This starts executing testFunction, skipping the beforeScenario:

scenarios:
    - engine: playwright
      beforeScenario: "login"
      testFunction: "testFunction"

[angelo-loria]

Alright figured this out, although there might be a cleaner way to do it. I'm using a before section with a function that POSTs to a login API and then writes the response token I need to a JSON file (emulating the storageState function in Playwright.) This JSON file is then specified in the Playwright contextOptions.

My Artillery config:

config:
    target: https://siteUnderTest.com/
    engines:
        playwright: {
                contextOptions: { storageState: "state.json" },
            }
    processor: "./processors/playwright-flows.js"

before:
    flow:
        - function: postLogin

scenarios:
    - name: playwright flow
      engine: playwright
      testFunction: playwrightFlow

My JS script:

module.exports = { playwrightFlow, postLogin };
const playwright = require("playwright");

async function postLogin(context, ee, next) {
    const apiContext = await playwright.request.newContext({
        storageState: undefined,
    });
    const response = await apiContext.post(
        "https://siteUnderTest.com/api/v1/sign-in",
        {
            data: {
                username: "userName",
                password: "password",
            },
        }
    );
    const loginResponse = await response.json();
    var stateJson = {
        cookies: [
            {
                name: "token",
                value: loginResponse.user.token,
                domain: "https//www.siteUnderTest.com",
                path: "/",
                expires: -1,
                httpOnly: false,
                sameSite: "Lax",
                secure: true,
            },
        ],
    };
    const fs = require("fs");
    fs.writeFile(
        "state.json",
        JSON.stringify(stateJson, null, 2),
        function (err) {
            if (err) throw err;
        }
    );
    next();
}

async function playwrightFlow(page) {
    await page.goto(
        "https://siteUnderTest.com/home"
    );
    ...

[bernardobridge]

👋 You can now use engine: playwright correctly in before/after, since Artillery v2.0.4: https://github.com/artilleryio/artillery/releases/tag/artillery-2.0.4

Example of reusing authentication with storageState: https://github.com/artilleryio/artillery/tree/main/examples/browser-playwright-reuse-authentication

Note: beforeScenario is still not supported in the Playwright engine, but I believe a before hook better suits what was being attempted in this topic!

[bernardobridge]

@alexandre-embie, to use this with Artillery, an empty object in the specified storage.json file needs to exist before running the test ({}), as mentioned here: https://github.com/artilleryio/artillery/tree/main/examples/browser-playwright-reuse-authentication#example

That should be the issue. 🤞

---

P.S. I also noticed includeFiles is on the root, rather than under config in your example. If you're doing this using Fargate, you'll need to set that correctly config.includeFiles).

[alexandre-embie]

@bernardobridge I have the file in my load dir as can be seen here 🤔

[bernardobridge]

@alexandre-embie looks like it might be a bug on our side, I can reproduce it. We will tackle this soon . Thanks! 🙇

[priyankshivhare-mb]

Do you happen to have any updates on this? I'm facing similar issues when I run Artillery inside ADO pipeline, it works locally.

[bernardobridge]

Hi @priyankshivhare-mb , yes, planning on providing a workaround with updated example for the next release (should be May 13th - we release every 2 weeks).

[bernardobridge]

Hello @priyankshivhare-mb , @alexandre-embie and others 👋 . This should now be fixed in Artillery v2.0.12.

The example has been updated with the needed changes: https://github.com/artilleryio/artillery/tree/main/examples/browser-playwright-reuse-authentication

[HugoImaios]

@bernardobridge @hassy Any thoughts on this?

I have this working where a list of users are used for the same scenario. I also try and reuse storageState, if already logged in before.

This works for the most part except I don't get the web vitals, as I need to create a new context (to reuse storageState in Playwright) and that doesn't have the init scripts like the following (I am trying to avoid copying and pasting code from the Artillery Playwright node_modules):

await context.addInitScript(() => {
          ['onLCP', 'onFCP', 'onCLS', 'onTTFB', 'onFID', 'onINP'].forEach(
          ....

I ran into the same issue, I once tried to create a new context (mandatory to add storageState) and indeed I lost web core vitals (and status codes, and other metrics). I personnaly rolledback this to keep the artillery's context, and the metrics

[bernardobridge]

Hey @sthavamani-ut and @HugoImaios 👋

So just to make sure I fully understand what you're trying to do.

• You have username/password in a CSV, you want to login from that CSV where the first time it logs in, it will save the storageState file with the username dynamically. Any further logins of that user would result in reusing the storageState

Is this correct?

[austin-patterson]

Hey @sthavamani-ut and @HugoImaios 👋

So just to make sure I fully understand what you're trying to do.

• You have username/password in a CSV, you want to login from that CSV where the first time it logs in, it will save the storageState file with the username dynamically. Any further logins of that user would result in reusing the storageState

Is this correct?

If we can dare to dream, I think this sounds like an amazing enhancement!!! I can't speak for @sthavamani-ut or @HugoImaios, but I believe you have interpreted correctly.

[HugoImaios]

yep that's correct ! Sorry I only "reacted" back then and haven't replied since.
The main goal was to run a load test on the login/auth scenario, then reuse the dozens of storage state with cookies to run some other scenario, skipping the login step

[MariaStadnik]

@bernardobridge I have a question 'm using Artillery for browser load testing and am struggling with how to configure dynamic contexts for each test scenario. Specifically, I need to set extraHTTPHeader for my browser dynamically without hardcoding token values. I want each scenario to create its own context that includes a unique authorization token, so each test can authenticate individually.

Currently, I can't find a way to pass a new token for every scenario without embedding the token directly in the configuration. Is there a method in Artillery to dynamically generate or fetch tokens per scenario, or to set headers on a per-scenario basis before the browser context is created? I also don't want to store the hardcoded token value in my repository anywhere, so would like to utilize a mechanism of retrieving a token value dynamically via some js function.

Here is my Artillery yaml below:

  target: 'myTargetUrl'
  phases:
    - duration: 10
      arrivalCount: 2
      name: phase-for-ui-test
  engines:
    playwright: 
      extendedMetrics: true
      launchOptions:
        headless: false
      contextOptions:
        extraHTTPHeaders:
          Authorization: 'Bearer myTokenValue
  processor: 'processor.js'

scenarios:
  - engine: playwright
    testFunction: 'myTestFunction'

Thanks for any help