Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(e2e): add discard action test #8196

Closed
wants to merge 1 commit into from
Closed

test(e2e): add discard action test #8196

wants to merge 1 commit into from

Conversation

RitaDias
Copy link
Contributor

@RitaDias RitaDias commented Jan 6, 2025

Description

Add e2e test for the discard document action

What to review

Does the test make sense, should we implement it in a different way?

Notes for release

N/A, it's internal testing

Copy link

vercel bot commented Jan 6, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 6, 2025 2:08pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 6, 2025 2:08pm
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 6, 2025 2:08pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 6, 2025 2:08pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Jan 6, 2025 2:08pm

Copy link
Contributor

github-actions bot commented Jan 6, 2025

No changes to documentation

Copy link
Contributor

github-actions bot commented Jan 6, 2025

Component Testing Report Updated Jan 6, 2025 1:38 PM (UTC)

❌ Failed Tests (1) -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 3s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 13s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 37s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 50s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 25s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 14s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 6s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 29s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 42s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 13s 3 9 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ❌ Failed (Inspect) 2m 24s 20 0 1
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Jan 6, 2025

⚡️ Editor Performance Report

Updated Mon, 06 Jan 2025 13:48:22 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 24.1 efps (42ms) 22.7 efps (44ms) +3ms (+6.0%)
article (body) 46.0 efps (22ms) 48.2 efps (21ms) -1ms (-4.6%)
article (string inside object) 24.4 efps (41ms) 25.6 efps (39ms) -2ms (-4.9%)
article (string inside array) 22.5 efps (45ms) 23.3 efps (43ms) -2ms (-3.4%)
recipe (name) 47.6 efps (21ms) 55.6 efps (18ms) -3ms (-14.3%)
recipe (description) 55.6 efps (18ms) 58.8 efps (17ms) -1ms (-5.6%)
recipe (instructions) 99.9+ efps (7ms) 99.9+ efps (7ms) +0ms (-/-%)
synthetic (title) 18.2 efps (55ms) 18.5 efps (54ms) -1ms (-1.8%)
synthetic (string inside object) 17.9 efps (56ms) 20.0 efps (50ms) -6ms (-10.7%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 42ms 59ms 72ms 612ms 887ms 11.7s
article (body) 22ms 26ms 72ms 244ms 539ms 7.1s
article (string inside object) 41ms 43ms 54ms 223ms 325ms 7.1s
article (string inside array) 45ms 47ms 53ms 170ms 128ms 7.2s
recipe (name) 21ms 23ms 24ms 49ms 0ms 8.2s
recipe (description) 18ms 20ms 22ms 33ms 0ms 4.6s
recipe (instructions) 7ms 9ms 10ms 12ms 0ms 3.2s
synthetic (title) 55ms 58ms 60ms 188ms 453ms 12.6s
synthetic (string inside object) 56ms 58ms 79ms 403ms 1033ms 9.3s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 44ms 63ms 86ms 490ms 1060ms 11.6s
article (body) 21ms 24ms 38ms 98ms 184ms 6.3s
article (string inside object) 39ms 41ms 45ms 171ms 158ms 6.6s
article (string inside array) 43ms 45ms 48ms 167ms 150ms 7.2s
recipe (name) 18ms 20ms 22ms 32ms 0ms 7.4s
recipe (description) 17ms 19ms 20ms 42ms 0ms 4.5s
recipe (instructions) 7ms 8ms 10ms 12ms 0ms 3.2s
synthetic (title) 54ms 59ms 66ms 346ms 702ms 13.0s
synthetic (string inside object) 50ms 54ms 67ms 479ms 863ms 9.1s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

Copy link
Contributor

@pedrobonamin pedrobonamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this Rita
Not blocking, but added a suggestion, would love your thoughts on this.

Comment on lines +19 to +23
// Focus the publish button to trigger the tooltip showing the keyboard shortcut
page.getByTestId('action-publish').focus()

// There is a delay before the tooltip opens, let's explicitly wait for it
await page.waitForTimeout(300)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to focus on the publish action? Not sure why we need to wait for the tooltip to open.

If we need the timeout to wait until the change has been saved, I think it will be a good idea to validate that the Saved shows in the footer.
Screenshot 2025-01-07 at 09 58 36

Or maybe we could we just depend on the following step page.getByTestId('action-publish').click() ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good question, I added this as a copy paste straight of the publish test (since I needed the document to be published) so I didn't even pass my eyes over it 🤔

That being said, let me have a look, I'll change the publish test too. good catch!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I think it wiser to not rely on the following step just by itself simply because I don't want to add more areas of potential flakiness which we know sometimes can happen in steps like those)

@RitaDias
Copy link
Contributor Author

RitaDias commented Jan 8, 2025

Actually, closing this since I just now realised (when looking through the feedback given by Pedro) that we already have some discard (DiscardChanges) tests. It just wasn't where I expected. One of the tests there already tests this use case, so I'm closing this 🙏
Sorry for the noise!

@RitaDias RitaDias closed this Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants