Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(sanity): ReleaseSummary tests #7870

Merged
merged 8 commits into from
Nov 26, 2024
Merged

test(sanity): ReleaseSummary tests #7870

merged 8 commits into from
Nov 26, 2024

Conversation

RitaDias
Copy link
Contributor

Description

What to review

Testing

Notes for release

Copy link

vercel bot commented Nov 25, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 25, 2024 8:01pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 25, 2024 8:01pm
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 25, 2024 8:01pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 25, 2024 8:01pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Nov 25, 2024 8:01pm

Copy link
Contributor

No changes to documentation

Copy link
Contributor

github-actions bot commented Nov 25, 2024

Component Testing Report Updated Nov 25, 2024 8:36 PM (UTC)

✅ All Tests Passed -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 8s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 13s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 39s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 53s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 27s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 15s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 2m 8s 2 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 8s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 2m 38s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 44s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 13s 3 9 0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx ✅ Passed (Inspect) 39s 9 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 54s 12 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Nov 25, 2024

⚡️ Editor Performance Report

Updated Mon, 25 Nov 2024 20:39:53 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 15.4 efps (65ms) 11.9 efps (84ms) +19ms (+29.2%) 🔴
article (body) 56.5 efps (18ms) 53.5 efps (19ms) +1ms (+5.6%)
article (string inside object) 18.2 efps (55ms) 12.6 efps (80ms) +25ms (+44.5%) 🔴
article (string inside array) 15.4 efps (65ms) 10.3 efps (97ms) +32ms (+49.2%) 🔴
recipe (name) 29.9 efps (34ms) 20.0 efps (50ms) +17ms (+49.3%) 🔴
recipe (description) 31.3 efps (32ms) 21.7 efps (46ms) +14ms (+43.8%) 🔴
recipe (instructions) 99.9+ efps (7ms) 99.9+ efps (6ms) -1ms (-/-%)
synthetic (title) 14.1 efps (71ms) 5.7 efps (174ms) +103ms (+145.1%) 🔴
synthetic (string inside object) 14.1 efps (71ms) 6.0 efps (168ms) +97ms (+135.9%) 🔴

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 65ms 74ms 121ms 167ms 652ms 13.6s
article (body) 18ms 20ms 33ms 193ms 291ms 6.1s
article (string inside object) 55ms 60ms 88ms 252ms 285ms 8.8s
article (string inside array) 65ms 72ms 118ms 286ms 756ms 9.8s
recipe (name) 34ms 41ms 72ms 81ms 0ms 9.6s
recipe (description) 32ms 35ms 41ms 171ms 0ms 6.4s
recipe (instructions) 7ms 10ms 12ms 15ms 3ms 3.5s
synthetic (title) 71ms 73ms 85ms 183ms 1530ms 15.3s
synthetic (string inside object) 71ms 77ms 143ms 516ms 1784ms 10.6s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 84ms 94ms 114ms 327ms 3117ms 18.3s
article (body) 19ms 23ms 47ms 167ms 360ms 6.3s
article (string inside object) 80ms 84ms 94ms 405ms 2679ms 11.9s
article (string inside array) 97ms 103ms 118ms 405ms 3815ms 13.2s
recipe (name) 50ms 51ms 73ms 123ms 897ms 11.8s
recipe (description) 46ms 48ms 55ms 107ms 692ms 7.8s
recipe (instructions) 6ms 8ms 9ms 21ms 0ms 3.2s
synthetic (title) 174ms 181ms 190ms 1035ms 9865ms 26.8s
synthetic (string inside object) 168ms 175ms 268ms 986ms 9930ms 20.9s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

Comment on lines +188 to +189
timeout: 5000,
interval: 500,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jordanl17 seriously, was the issue the whole time just a timeout? 🥲

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite - this seemed to be some flakiness in the tests, and I'm not quite sure why. Adding in a longer timeout seems to resolve the flake, but there were quite a few other parts necessary to actually get these tests to work:

  1. awaiting the waitFor for the spinner to go away and the table to render
  2. Updating the mock for DefaultPreview to pass the props through
  3. Something else that I can't remember...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

God, this is so frustrating 😭 😂

@RitaDias RitaDias changed the title test(releases): ReleaseSummary tests test(sanity): ReleaseSummary tests Nov 26, 2024
@RitaDias RitaDias requested review from a team and pedrobonamin November 26, 2024 11:56
Copy link
Member

@bjoerge bjoerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Added some comments/questions you can consider, but feel free to merge.

useBundleDocumentsMockReturnWithResults,
} from './__mocks__/useBundleDocuments.mock'

vi.mock('../../../index', () => ({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we be more specific about the path here? I think it's preferable to mock the file where the function is defined, e..g where useDocumentPresence is actually defined. Same with useDocumentPreviewStore. That would ensure these gets mocked also in a (potential future case) where these are imported from a more specific internal path.

}),
useDocumentPreviewStore: vi.fn().mockReturnValue({
unstable_observeDocumentIdSet: vi.fn(() => ({
pipe: vi.fn(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the unstable_observeDocumentIdSet mock instead return an observable? It's not clear to me what mocking pipe() here does 🤔

vi.mock('../../../../preview/components/_previewComponents', async () => {
return {
_previewComponents: {
default: vi.fn((arg) => <DefaultPreview {...arg} />),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be a dummy component or are we asserting anything "default preview" specific?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are asserting on the passing through and rendering of the document preview itself. this mock only exists as the _previewComponents has an as yet unknown quirk that during test runs all the exports are set to undefined rather than their correct components

@jordanl17 jordanl17 merged commit 735f831 into corel Nov 26, 2024
142 of 148 checks passed
@jordanl17 jordanl17 deleted the fix-ReleaseSummary-test branch November 26, 2024 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants