Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support test screenshot configuration options #82

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sanghun0724
Copy link

@sanghun0724 sanghun0724 commented Nov 23, 2024

Add configuration for scroll performance testing

Background

When measuring scroll performance using FPS in Xcode, the current test configuration options in Tuist don't provide accurate measurement environments. For example, without disabling certain settings, what should be 60 FPS is incorrectly measured as 30 FPS.

Tuist Configurable Options Checklist

Tips for creating performance tests:

  • Create a separate test scheme for your Performance XCTest ✅
  • Use Release Build Configuration and uncheck Debug executable ✅
  • Switch off Automatic Screenshots and Code Coverage ❌
  • Turn off all diagnostic options, checkers, sanitizers and memory management ✅

ref: https://developer.apple.com/videos/play/wwdc2020/10077/

Proposed Solution

I propose to add two test configuration options:

  1. captureScreenshotsAutomatically: This will allow users to disable automatic screenshots during testing, which is crucial for accurate scroll performance measurements.

  2. deleteScreenshotsWhenEachTestSucceeds: This option will work in conjunction with captureScreenshotsAutomatically to configure AttachmentLifeTime (currently not utilized in Tuist).

These additions will help developers obtain more accurate performance measurements and better manage test artifacts.

Impact

  • Enables accurate FPS measurements for scroll performance testing
  • Provides more granular control over test screenshot behavior
  • Improves overall test environment configuration capabilities

Note

The following PR will be submitted to Tuist based on this work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant