docs(playwright-engine): add UI and CLI guides and migration recipe (#…

…4013) * docs(pw engine): add ui and cli test creation docs * docs(pw-engine): add migration guide * docs(pw-engine): update links * Apply suggestions from code review Co-authored-by: Julianne Fermi <julianne@kubeshop.io> * docs(pwe): fix links --------- Co-authored-by: Julianne Fermi <julianne@kubeshop.io>
kubeshop · Sep 10, 2024 · 0039562 · 0039562
1 parent d75ac63
commit 0039562
Show file tree

Hide file tree

Showing 8 changed files with 329 additions and 5 deletions.
diff --git a/docs/docs/cli/creating-tests-playwright-engine.mdx b/docs/docs/cli/creating-tests-playwright-engine.mdx
@@ -0,0 +1,59 @@
+---
+id: creating-tests-playwright-engine
+title: Defining Playwright Engine Tests as Text Files
+description: Tracetest enables developers to define tests as text files and run them using a CLI. Integrate the execution of Playwright Engine tests in your existing CI pipeline.
+keywords:
+  - tracetest
+  - trace-based testing
+  - observability
+  - distributed tracing
+  - testing
+  - playwright
+  - playwright engine
+image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
+---
+
+:::info How is Playwright Engine different from the Playwright npm module integration?
+Read the migration guide [here](/examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine).
+:::
+
+The Playwright Testing Engine executes your testing scripts natively in the platform, by specifying three things:
+
+- The target URL of your website (private or public).
+- A JavaScript file with your tests.
+- And the exported method you want to execute.
+
+Tracetest Playwright Engine test definition YAML:
+
+```yaml
+trigger:
+  type: playwrightengine
+  playwrightEngine:
+    target: ${env:TARGET_URL}
+    script: ./script.js
+    method: importPokemon
+```
+
+Playwright `script.js` definition:
+
+```js
+const { expect } = require("@playwright/test");
+
+// [...]
+
+async function importPokemon(page) {
+  expect(await page.getByText("Pokeshop")).toBeTruthy();
+
+  await page.click("text=Import");
+  await page.getByLabel("ID").fill("143");
+
+  await Promise.all([
+    page.waitForResponse(
+      (resp) => resp.url().includes("/pokemon/import") && resp.status() === 200
+    ),
+    page.getByRole("button", { name: "OK", exact: true }).click(),
+  ]);
+}
+
+module.exports = { importPokemon };
+```
diff --git a/docs/docs/examples-tutorials/recipes.mdx b/docs/docs/examples-tutorials/recipes.mdx
@@ -23,7 +23,9 @@ Recipes are short, self-contained, guides for getting started with building popu
 These guides show how to get the best of your instrumentation using the native Tracetest Triggers.
 
 - [True End-To-End Trace-Based Tests with the Tracetest Playwright Engine](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine)
-- [Trace-Based Tests with the Tracetest GrapqhQL Trigger](/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop)
+- [Migrating tests from Playwright integration to Playwright Engine
+](/examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine)
+- [Trace-Based Tests with the Tracetest GraphQL Trigger](/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop)
 
 ## Synthetic Monitoring
 

diff --git a/...ls/recipes/migrating-tests-from-playwright-integration-to-playwright-engine.mdx b/...ls/recipes/migrating-tests-from-playwright-integration-to-playwright-engine.mdx
@@ -0,0 +1,194 @@
+---
+id: migrating-tests-from-playwright-integration-to-playwright-engine
+title: Migrating Tests from Playwright Integration to Playwright Engine
+description: Quickstart on how to Migrating tests from Playwright integration to Playwright Engine
+hide_table_of_contents: false
+keywords:
+  - tracetest
+  - trace-based testing
+  - observability
+  - distributed tracing
+  - end-to-end testing
+  - tracetest
+  - playwright
+  - trace-based-testing
+  - trigger
+image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
+---
+
+:::info Tracetest x Playwright Frontend Instrumentation Requirements
+Find out the requirements for your instrumented app to start using [Tracetest x Playwright](https://tracetest.io/blog/tracetest-tip-instrumentation-for-end-to-end-tests).
+:::
+
+:::info Version Compatibility
+The features described here are compatible with the [Tracetest CLI v1.4.1](https://github.com/kubeshop/tracetest/releases/tag/v1.4.1) and above.
+:::
+
+:::note
+[Check out the full recipe on using the Playwright Engine here.](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine)
+:::
+
+[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.
+
+[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.
+
+## Why is this important?
+
+### Playwright Engine
+
+The [Tracetest Playwright Engine](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine) trigger enables you to combine the power of end-to-end tests with trace-based testing to easily capture a full distributed trace from your OpenTelemetry instrumented front-end and back-end system.
+
+### Playwright Integration
+
+The [Tracetest integration for Playwright](/tools-and-integrations/playwright) enables your current Playwright tests to easily capture a full distributed trace from your OpenTelemetry instrumented frontend and backend system. You can embed a Tracetest in this Playwright test, and allow trace-based testing assertions to be applied across this entire flow, enabling true end-to-end tests across your entire system.
+
+### Why Migrate to Playwright Engine?
+
+By creating a Tracetest Playwright Engine test, you will be able to create trace-based assertions to be applied across the entire flow like any other Tracetest test. It also allows you to mix and match it with your existing Monitors, Test Suites and CI/CD validations.
+
+You do not need to edit your existing Playwright tests. Only import your existing tests into Tracetest.
+
+![diffrenrences](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725956860/docs/66b22fa08831a4fafdf23579_66971d81fc6dbc9929228702_Screenshot_25202024-07-16_2520at_25209.24.58_2520PM_td79ry.png)
+
+## Tracetest Playwright Integration Test
+
+Running a Tracetest test with the [Playwright integration](/tools-and-integrations/playwright) requires you to do 3 things:
+
+1. Install the `@tracetest/playwright` npm module and edit your existing Playwright test scripts.
+2. Load the Tracetest test definition YAML file.
+3. Add three lifecycle hooks to the Playwright test script: `beforeAll`, `beforeEach`, `afterAll`. They are used to trigger Tracetest.
+
+```js
+// 1. Install the Tracetest npm module
+import { test, expect } from "@playwright/test";
+import Tracetest, { Types } from "@tracetest/playwright";
+
+const { TRACETEST_API_TOKEN = "", TRACETEST_SERVER_URL = "https://app.tracetest.io" } = process.env;
+
+let tracetest: Types.TracetestPlaywright | undefined = undefined;
+
+test.describe.configure({ mode: "serial" });
+
+// 2. Load the Tracetest test definition YAML file
+const definition = `
+type: Test
+spec:
+  id: UGxheXdyaWdodDogaW1wb3J0cyBhIHBva2Vtb24=
+  name: "Playwright: imports a pokemon"
+  trigger:
+    type: playwright
+  specs:
+    - selector: span[tracetest.span.type="http"]
+      name: "All HTTP Spans: Status  code is 200"
+      assertions:
+      - attr:http.status_code   =   200
+    - selector: span[tracetest.span.type="database"]
+      name: "All Database Spans: Processing time is less than 100ms"
+      assertions:
+      - attr:tracetest.span.duration < 2s
+  outputs:
+    - name: MY_OUTPUT
+      selector: span[tracetest.span.type="general" name="Tracetest trigger"]
+      value: attr:name
+`;
+
+// 3. Add lifecycle hooks
+test.beforeAll(async () => {
+  tracetest = await Tracetest({ apiToken: TRACETEST_API_TOKEN, serverUrl: TRACETEST_SERVER_URL, serverPath: "" });
+
+  await tracetest.setOptions({
+    "Playwright: imports a pokemon": {
+      definition,
+    },
+  });
+});
+
+test.beforeEach(async ({ page }, info) => {
+  await page.goto("/");
+  await tracetest?.capture(page, info);
+});
+
+// optional step to break the playwright script in case a Tracetest test fails
+test.afterAll(async ({}, testInfo) => {
+  testInfo.setTimeout(80000);
+  await tracetest?.summary();
+});
+
+test("Playwright: creates a pokemon", async ({ page }) => {
+  expect(await page.getByText("Pokeshop")).toBeTruthy();
+
+  await page.click("text=Add");
+
+  await page.getByLabel("Name").fill("Charizard");
+  await page.getByLabel("Type").fill("Flying");
+  await page
+    .getByLabel("Image URL")
+    .fill("https://upload.wikimedia.org/wikipedia/en/1/1f/Pok%C3%A9mon_Charizard_art.png");
+  await page.getByRole("button", { name: "OK", exact: true }).click();
+});
+
+test("Playwright: imports a pokemon", async ({ page }) => {
+  expect(await page.getByText("Pokeshop")).toBeTruthy();
+
+  await page.click("text=Import");
+
+  await page.getByLabel("ID").fill("143");
+
+  await Promise.all([
+    page.waitForResponse((resp) => resp.url().includes("/pokemon/import") && resp.status() === 200),
+    page.getByRole("button", { name: "OK", exact: true }).click(),
+  ]);
+});
+
+test("Playwright: deletes a pokemon", async ({ page }) => {
+  await page.locator('[data-cy="pokemon-list"]');
+
+  await page.locator('[data-cy="pokemon-card"]').first().click();
+  await page.locator('[data-cy="pokemon-card"] [data-cy="delete-pokemon-button"]').first().click();
+});
+```
+
+## Tracetest Playwright Engine Test
+
+Running a Tracetest test with the [Playwright Engine](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine) requires you to do 3 things:
+
+1. Add the target URL of your website.
+2. Upload your existing unedited Playwright test script.
+3. Select the exported method to execute from the Playwright test script.
+
+Tracetest now natively supports Playwright tests without including third-party libraries or code snippets that are not 100% related to your tests.
+
+You upload your existing Playwright test script without adding any additional npm modules.
+
+```js
+const { expect } = require("@playwright/test");
+
+async function importPokemon(page) {
+  expect(await page.getByText("Pokeshop")).toBeTruthy();
+  await page.click("text=Import");
+  await page.getByLabel("ID").fill("143");
+  await Promise.all([
+    page.waitForResponse(
+      (resp) => resp.url().includes("/pokemon/import") && resp.status() === 200
+    ),
+    page.getByRole("button", { name: "OK", exact: true }).click(),
+  ]);
+}
+
+module.exports = { importPokemon };
+```
+
+Add the target URL and select the method to run.
+
+![](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725957754/docs/66b22fa08831a4fafdf2357c_66971c53720e374b82ff4a43_trigger4_ggrzxa.png)
+
+## What's Next?
+
+Follow the [Playwright Engine recipe to learn how to run your Playwright tests in Tracetest](/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine).
+
+[Read the announcement blog post](https://tracetest.io/blog/tracetest-playwright-engine-the-future-of-end-to-end-tests-is-trace-based-testing) to learn why we decided to implement the Playwright Engine.
+
+
+## Learn More
+
+Please visit our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!
diff --git a/...cs/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop.mdx b/...cs/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop.mdx
@@ -1,6 +1,6 @@
 ---
 id: running-tests-with-tracetest-graphql-pokeshop
-title: Trace-Based Tests with the Tracetest GrapqhQL Trigger
+title: Trace-Based Tests with the Tracetest GraphQL Trigger
 description: Quickstart on how to create Trace-Based Tests with the Tracetest GraphQL Trigger
 hide_table_of_contents: false
 keywords:

diff --git a/...s/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine.mdx b/...s/examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine.mdx
@@ -29,6 +29,7 @@ The features described here are compatible with the [Tracetest CLI v1.4.1](https
 :::
 
 [Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.
+
 [Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.
 
 ## Why is this important?

diff --git a/docs/docs/web-ui/creating-tests-graphql.mdx b/docs/docs/web-ui/creating-tests-graphql.mdx
@@ -19,9 +19,9 @@ image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_T
 
 Click the GraphQL option to continue:
 
-![Choose Trigger](../img/choose-trigger-0.16.png)
+![Choose Trigger](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725452405/Blogposts/Trace-based-Tests-with-GraphQL-in-Action/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_tests_page_1_cgdeys.png)
 
-In this example, a GaphQL Request has been chosen.
+In this example, a GraphQL Request has been chosen.
 
 ![Choose Example](../img/graphql-choose-trigger.png)
 

diff --git a/docs/docs/web-ui/creating-tests-playwright-engine.mdx b/docs/docs/web-ui/creating-tests-playwright-engine.mdx
@@ -0,0 +1,53 @@
+---
+id: creating-tests-playwright-engine
+title: Creating Tests - Playwright Engine
+description: Tracetest enables creating and running native Playwright tests visually in the Tracetest Web UI.
+hide_table_of_contents: false
+keywords:
+  - tracetest
+  - trace-based testing
+  - observability
+  - distributed tracing
+  - testing
+  - playwright
+  - playwright engine
+image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
+---
+
+:::info How is Playwright Engine different from the Playwright npm module integration?
+Read the migration guide [here](/examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine).
+:::
+
+[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.
+
+👉 **Join our [shared Pokeshop API Demo environment](https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/invites/invite_760904a64b4b9dc9/accept) and try creating a Test yourself!**
+
+Click the Playwright Engine option to continue:
+
+![Choose Trigger](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725949257/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_tests_page_1_2_sl75n1.png)
+
+You can choose an example test in the shared Pokeshop API Demo environment.
+
+![Choose Example Pokemon](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725951354/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_test_create_playwrightengine_ktazl2.png)
+
+Choose the **Pokeshop - Import** example and click **Run**.
+
+The Playwright Testing Engine executes your testing scripts natively in the platform, by specifying three things:
+
+- The target URL of your website (private or public).
+- A JavaScript file with your tests.
+- And the exported method you want to execute.
+
+When the test finishes, you will get the following results:
+
+- The Playwright script output.
+- The trace from both the front-end UI and back-end processes.
+
+![test result](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725951959/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_test_IwCdBElIR_run_1385_trigger_1_plxh33.png)
+
+![trace result](https://res.cloudinary.com/djwdcmwdz/image/upload/v1725951905/docs/app.tracetest.io_organizations_ttorg_2179a9cd8ba8dfa5_environments_ttenv_231b49e808c29e6a_test_IwCdBElIR_run_1385_trigger_kflqrv.png)
+
+Please visit the [Test Results](/web-ui/test-results) document for an explanation of viewing the results of a test.
+
+👉 **Join our [shared Pokeshop API Demo environment](https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/invites/invite_760904a64b4b9dc9/accept) and try creating a Test yourself!**
+
diff --git a/docs/sidebars.js b/docs/sidebars.js
@@ -43,10 +43,15 @@ const sidebars = {
               id: "examples-tutorials/recipes/running-tests-with-tracetest-playwright-engine",
               label: "True End-To-End Trace-Based Tests with the Tracetest Playwright Engine",
             },
+            {
+              type: "doc",
+              id: "examples-tutorials/recipes/migrating-tests-from-playwright-integration-to-playwright-engine",
+              label: "Migrating tests from Playwright integration to Playwright Engine",
+            },
             {
               type: "doc",
               id: "examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop",
-              label: "Trace-Based Tests with the Tracetest GrapqhQL Trigger",
+              label: "Trace-Based Tests with the Tracetest GraphQL Trigger",
             },
           ],
         },
@@ -986,6 +991,11 @@ const sidebars = {
                       id: "web-ui/creating-tests-kafka",
                       label: "Creating Tests - Kafka",
                     },
+                    {
+                      type: "doc",
+                      id: "web-ui/creating-tests-playwright-engine",
+                      label: "Creating Tests - Playwright Engine",
+                    },
                     {
                       type: "doc",
                       id: "web-ui/creating-tests-graphql",
@@ -1134,6 +1144,11 @@ const sidebars = {
                       id: "cli/creating-tests-kafka",
                       label: "Creating Tests - Kafka",
                     },
+                    {
+                      type: "doc",
+                      id: "cli/creating-tests-playwright-engine",
+                      label: "Creating Tests - Playwright Engine",
+                    },
                     {
                       type: "doc",
                       id: "cli/creating-tests-graphql",