Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(playwright-engine): add UI and CLI guides and migration recipe #4013

Merged
merged 5 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions docs/docs/cli/creating-tests-playwright-engine.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
id: creating-tests-playwright-engine
title: Defining Playwright Engine Tests as Text Files
description: Tracetest enables developers to define tests as text files and run them using a CLI. Integrate the execution of Playwright Engine tests in your existing CI pipeline.
keywords:
- tracetest
- trace-based testing
- observability
- distributed tracing
- testing
- playwright
- playwright engine
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

:::info How is Playwright Engine different from the Playwright npm module integration?
Read the migration guide [here](/examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine).
:::

The Playwright Testing Engine executes your testing scripts natively in the platform, by specifying three things:

- The target URL of your website (private or public).
- A JavaScript file with your tests.
- And the exported method you want to execute.

Tracetest Playwright Engine test definition YAML:

```yaml
trigger:
type: playwrightengine
playwrightEngine:
target: ${env:TARGET_URL}
script: ./script.js
method: importPokemon
```

Playwright `script.js` definition:

```js
const { expect } = require("@playwright/test");

// [...]

async function importPokemon(page) {
expect(await page.getByText("Pokeshop")).toBeTruthy();

await page.click("text=Import");
await page.getByLabel("ID").fill("143");

await Promise.all([
page.waitForResponse(
(resp) => resp.url().includes("/pokemon/import") && resp.status() === 200
),
page.getByRole("button", { name: "OK", exact: true }).click(),
]);
}

module.exports = { importPokemon };
```
4 changes: 3 additions & 1 deletion docs/docs/examples-tutorials/recipes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ Recipes are short, self-contained, guides for getting started with building popu
These guides show how to get the best of your instrumentation using the native Tracetest Triggers.

- [True End-To-End Trace-Based Tests with the Tracetest Playwright Engine](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine)
- [Trace-Based Tests with the Tracetest GrapqhQL Trigger](/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop)
- [Migrating tests from Playwright integration to Playwright Engine
](/examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine)
- [Trace-Based Tests with the Tracetest GraphQL Trigger](/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop)

## Synthetic Monitoring

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
---
id: migrating-tests-from-playwright-integration-to-playwright-engine
title: Migrating Tests from Playwright Integration to Playwright Engine
description: Quickstart on how to Migrating tests from Playwright integration to Playwright Engine
hide_table_of_contents: false
keywords:
- tracetest
- trace-based testing
- observability
- distributed tracing
- end-to-end testing
- tracetest
- playwright
- trace-based-testing
- trigger
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

:::info Tracetest x Playwright Frontend Instrumentation Requirements
Find out the requirements for your instrumented app to start using [Tracetest x Playwright](https://tracetest.io/blog/tracetest-tip-instrumentation-for-end-to-end-tests).
:::

:::info Version Compatibility
The features described here are compatible with the [Tracetest CLI v1.4.1](https://github.com/kubeshop/tracetest/releases/tag/v1.4.1) and above.
:::

:::note
[Check out the full recipe on using the Playwright Engine here.](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine)
:::

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.

[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.

## Why is this important?

### Playwright Engine

The [Tracetest Playwright Engine](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine) trigger enables you to combine the power of end-to-end tests with trace-based testing to easily capture a full distributed trace from your OpenTelemetry instrumented front-end and back-end system.

### Playwright Integration

The [Tracetest integration for Playwright](/tools-and-integrations/playwright) enables your current Playwright tests to easily capture a full distributed trace from your OpenTelemetry instrumented frontend and backend system. You can embed a Tracetest in this Playwright test, and allow trace-based testing assertions to be applied across this entire flow, enabling true end-to-end tests across your entire system.

### Why Migrate to Playwright Engine?

By creating a Tracetest Playwright Engine test, you will be able to create trace-based assertions to be applied across the entire flow like any other Tracetest test. It also allows you to mix and match it with your existing Monitors, Test Suites and CI/CD validations.

You do not need to edit your existing Playwright tests. Only import your existing tests into Tracetest.

![diffrenrences](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725956860/docs/66b22fa08831a4fafdf23579_66971d81fc6dbc9929228702_Screenshot_25202024-07-16_2520at_25209.24.58_2520PM_td79ry.png)

## Tracetest Playwright Integration Test

Running a Tracetest test with the [Playwright integration](/tools-and-integrations/playwright) requires you to do 3 things:

1. Install the `@tracetest/playwright` npm module and edit your existing Playwright test scripts.
2. Load the Tracetest test definition YAML file.
3. Add three lifecycle hooks to the Playwright test script: `beforeAll`, `beforeEach`, `afterAll`. They are used to trigger Tracetest.

```js
// 1. Install the Tracetest npm module
import { test, expect } from "@playwright/test";
import Tracetest, { Types } from "@tracetest/playwright";

const { TRACETEST_API_TOKEN = "", TRACETEST_SERVER_URL = "https://app.tracetest.io" } = process.env;

let tracetest: Types.TracetestPlaywright | undefined = undefined;

test.describe.configure({ mode: "serial" });

// 2. Load the Tracetest test definition YAML file
const definition = `
type: Test
spec:
id: UGxheXdyaWdodDogaW1wb3J0cyBhIHBva2Vtb24=
name: "Playwright: imports a pokemon"
trigger:
type: playwright
specs:
- selector: span[tracetest.span.type="http"]
name: "All HTTP Spans: Status code is 200"
assertions:
- attr:http.status_code = 200
- selector: span[tracetest.span.type="database"]
name: "All Database Spans: Processing time is less than 100ms"
assertions:
- attr:tracetest.span.duration < 2s
outputs:
- name: MY_OUTPUT
selector: span[tracetest.span.type="general" name="Tracetest trigger"]
value: attr:name
`;

// 3. Add lifecycle hooks
test.beforeAll(async () => {
tracetest = await Tracetest({ apiToken: TRACETEST_API_TOKEN, serverUrl: TRACETEST_SERVER_URL, serverPath: "" });

await tracetest.setOptions({
"Playwright: imports a pokemon": {
definition,
},
});
});

test.beforeEach(async ({ page }, info) => {
await page.goto("/");
await tracetest?.capture(page, info);
});

// optional step to break the playwright script in case a Tracetest test fails
test.afterAll(async ({}, testInfo) => {
testInfo.setTimeout(80000);
await tracetest?.summary();
});

test("Playwright: creates a pokemon", async ({ page }) => {
expect(await page.getByText("Pokeshop")).toBeTruthy();

await page.click("text=Add");

await page.getByLabel("Name").fill("Charizard");
await page.getByLabel("Type").fill("Flying");
await page
.getByLabel("Image URL")
.fill("https://upload.wikimedia.org/wikipedia/en/1/1f/Pok%C3%A9mon_Charizard_art.png");
await page.getByRole("button", { name: "OK", exact: true }).click();
});

test("Playwright: imports a pokemon", async ({ page }) => {
expect(await page.getByText("Pokeshop")).toBeTruthy();

await page.click("text=Import");

await page.getByLabel("ID").fill("143");

await Promise.all([
page.waitForResponse((resp) => resp.url().includes("/pokemon/import") && resp.status() === 200),
page.getByRole("button", { name: "OK", exact: true }).click(),
]);
});

test("Playwright: deletes a pokemon", async ({ page }) => {
await page.locator('[data-cy="pokemon-list"]');

await page.locator('[data-cy="pokemon-card"]').first().click();
await page.locator('[data-cy="pokemon-card"] [data-cy="delete-pokemon-button"]').first().click();
});
```

## Tracetest Playwright Engine Test

Running a Tracetest test with the [Playwright Engine](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine) requires you to do 3 things:

1. Add the target URL of your website.
2. Upload your existing unedited Playwright test script.
3. Select the exported method to execute from the Playwright test script.

Tracetest now natively supports Playwright tests without including third-party libraries or code snippets that are not 100% related to your tests.

You upload your existing Playwright test script without adding any additional npm modules.

```js
const { expect } = require("@playwright/test");

async function importPokemon(page) {
expect(await page.getByText("Pokeshop")).toBeTruthy();
await page.click("text=Import");
await page.getByLabel("ID").fill("143");
await Promise.all([
page.waitForResponse(
(resp) => resp.url().includes("/pokemon/import") && resp.status() === 200
),
page.getByRole("button", { name: "OK", exact: true }).click(),
]);
}

module.exports = { importPokemon };
```

Add the target URL and select the method to run.

![](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725957754/docs/66b22fa08831a4fafdf2357c_66971c53720e374b82ff4a43_trigger4_ggrzxa.png)

## What's Next?

Follow the [Playwright Engine recipe to learn how to run your Playwright tests in Tracetest](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine).

[Read the announcement blog post](https://tracetest.io/blog/tracetest-playwright-engine-the-future-of-end-to-end-tests-is-trace-based-testing) to learn why we decided to implement the Playwright Engine.


## Learn More

Please visit our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
id: running-tests-with-tracetest-graphql-pokeshop
title: Trace-Based Tests with the Tracetest GrapqhQL Trigger
title: Trace-Based Tests with the Tracetest GraphQL Trigger
description: Quickstart on how to create Trace-Based Tests with the Tracetest GraphQL Trigger
hide_table_of_contents: false
keywords:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ The features described here are compatible with the [Tracetest CLI v1.4.1](https
:::

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.

[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.

## Why is this important?
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/web-ui/creating-tests-graphql.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_T

Click the GraphQL option to continue:

![Choose Trigger](../img/choose-trigger-0.16.png)
![Choose Trigger](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725452405/Blogposts/Trace-based-Tests-with-GraphQL-in-Action/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_tests_page_1_cgdeys.png)

In this example, a GaphQL Request has been chosen.
In this example, a GraphQL Request has been chosen.

![Choose Example](../img/graphql-choose-trigger.png)

Expand Down
53 changes: 53 additions & 0 deletions docs/docs/web-ui/creating-tests-playwright-engine.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
id: creating-tests-playwright-engine
title: Creating Tests - Playwright Engine
description: Tracetest enables creating and running native Playwright tests visually in the Tracetest Web UI.
hide_table_of_contents: false
keywords:
- tracetest
- trace-based testing
- observability
- distributed tracing
- testing
- playwright
- playwright engine
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

:::info How is Playwright Engine different from the Playwright npm module integration?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:::info How is Playwright Engine different from the Playwright npm module integration?
:::info How is the Playwright Engine different from the Playwright npm module integration?

Read the migration guide [here](/examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine).
:::

[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.

👉 **Join our [shared Pokeshop API Demo environment](https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/invites/invite_760904a64b4b9dc9/accept) and try creating a Test yourself!**

Click the Playwright Engine option to continue:

![Choose Trigger](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725949257/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_tests_page_1_2_sl75n1.png)

You can choose an example test in the shared Pokeshop API Demo environment.

![Choose Example Pokemon](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725951354/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_test_create_playwrightengine_ktazl2.png)

Choose the **Pokeshop - Import** example and click **Run**.

The Playwright Testing Engine executes your testing scripts natively in the platform, by specifying three things:

- The target URL of your website (private or public).
- A JavaScript file with your tests.
- And the exported method you want to execute.

When the test finishes, you will get the following results:

- The Playwright script output.
- The trace from both the front-end UI and back-end processes.

![test result](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725951959/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_test_IwCdBElIR_run_1385_trigger_1_plxh33.png)

![trace result](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725951905/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_test_IwCdBElIR_run_1385_trigger_kflqrv.png)

Please visit the [Test Results](/web-ui/test-results) document for an explanation of viewing the results of a test.

👉 **Join our [shared Pokeshop API Demo environment](https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/invites/invite_760904a64b4b9dc9/accept) and try creating a Test yourself!**

17 changes: 16 additions & 1 deletion docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,15 @@ const sidebars = {
id: "examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine",
label: "True End-To-End Trace-Based Tests with the Tracetest Playwright Engine",
},
{
type: "doc",
id: "examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine",
label: "Migrating tests from Playwright integration to Playwright Engine",
},
{
type: "doc",
id: "examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop",
label: "Trace-Based Tests with the Tracetest GrapqhQL Trigger",
label: "Trace-Based Tests with the Tracetest GraphQL Trigger",
},
],
},
Expand Down Expand Up @@ -986,6 +991,11 @@ const sidebars = {
id: "web-ui/creating-tests-kafka",
label: "Creating Tests - Kafka",
},
{
type: "doc",
id: "web-ui/creating-tests-playwright-engine",
label: "Creating Tests - Playwright Engine",
},
{
type: "doc",
id: "web-ui/creating-tests-graphql",
Expand Down Expand Up @@ -1134,6 +1144,11 @@ const sidebars = {
id: "cli/creating-tests-kafka",
label: "Creating Tests - Kafka",
},
{
type: "doc",
id: "cli/creating-tests-playwright-engine",
label: "Creating Tests - Playwright Engine",
},
{
type: "doc",
id: "cli/creating-tests-graphql",
Expand Down
Loading