Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add info to automate runtime-tests #156

Merged
merged 3 commits into from
Dec 13, 2023
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 76 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,4 +162,79 @@ and define the following in your `csproj`:
These attributes will ask for the runtime test engine to replace the ones defined by the `Uno.UI.RuntimeTests.Engine` package.

## Running the tests automatically during CI
dr1rrb marked this conversation as resolved.
Show resolved Hide resolved
_TBD_
When your application references the runtime-test engine, as soon as you start it with the following environement variables, the runtime-test engine will automatically run the tests on application startup and then kill the app.

dr1rrb marked this conversation as resolved.
Show resolved Hide resolved
* **UNO_RUNTIME_TESTS_RUN_TESTS**: This can be either
* A json serialized [test configuration](https://github.com/unoplatform/uno.ui.runtimetests.engine/blob/main/src/Uno.UI.RuntimeTests.Engine.Library/Engine/UnitTestEngineConfig.cs) (use `{}` to run with default configuration);
* A filter string.
* **UNO_RUNTIME_TESTS_OUTPUT_PATH**: This is the output path of the test result file

You can also define some other configurations variables:
dr1rrb marked this conversation as resolved.
Show resolved Hide resolved

* **UNO_RUNTIME_TESTS_OUTPUT_KIND**: Selects the kind of the test result file, possible values are `NUnit` (default) or `UnoRuntimeTests` (cf. [`TestResultKind`](https://github.com/unoplatform/uno.ui.runtimetests.engine/blob/main/src/Uno.UI.RuntimeTests.Engine.Library/Engine/ExternalRunner/RuntimeTestEmbeddedRunner.cs#L41))

Currently the easiest way to run runtime-tests on the CI is using the skia-GTK head. Here an example of a azure pipeline configuration file:
dr1rrb marked this conversation as resolved.
Show resolved Hide resolved

```yaml
jobs:
- job: Skia_Tests
displayName: 'Runtime Tests - Skia GTK'
timeoutInMinutes: 60

pool:
vmImage: 'ubuntu-20.04'

variables:
NUGET_PACKAGES: $(build.sourcesdirectory)/.nuget

steps:
- checkout: self
clean: true

- task: UseDotNet@2
displayName: 'Use .NET'
inputs:
packageType: 'sdk'
version: '7.x'

- script: |
dotnet tool install -g uno.check
uno-check --target skiagtk --fix --non-interactive --ci

displayName: 'Run uno-check'

- script: dotnet build Uno.Extensions.RuntimeTests.Skia.Gtk.csproj -c Release -p:UnoTargetFrameworkOverride=net7.0 -p:GeneratePackageOnBuild=false -bl:$(Build.ArtifactStagingDirectory)/skia-gtk-runtime-test-build.binlog
displayName: 'Build Runtime Tests app (GTK)'
workingDirectory: $(Build.SourcesDirectory)/src/Uno.Extensions.RuntimeTests/Uno.Extensions.RuntimeTests.Skia.Gtk

- task: PublishBuildArtifacts@1
displayName: Publish Build Logs
retryCountOnTaskFailure: 3
condition: always()
inputs:
PathtoPublish: $(build.artifactstagingdirectory)/skia-gtk-runtime-test-build.binlog
ArtifactName: skia-runtime-test-build
ArtifactType: Container

- script: xvfb-run --auto-servernum --server-args='-screen 0 1280x1024x24' dotnet Uno.Extensions.RuntimeTests.Skia.Gtk.dll
displayName: 'Run Runtime Tests (GTK)'
workingDirectory: $(Build.SourcesDirectory)/src/Uno.Extensions.RuntimeTests/Uno.Extensions.RuntimeTests.Skia.Gtk/bin/Debug/net7.0
env:
UNO_RUNTIME_TESTS_RUN_TESTS: '{}'
UNO_RUNTIME_TESTS_OUTPUT_PATH: '$(Common.TestResultsDirectory)/skia-gtk-runtime-tests-results.xml'

- task: PublishTestResults@2
displayName: 'Publish GTK Runtime Tests Results'
condition: always()
retryCountOnTaskFailure: 3
inputs:
testRunTitle: 'GTK Runtime Tests Run'
testResultsFormat: 'NUnit'
testResultsFiles: '$(Common.TestResultsDirectory)/skia-gtk-runtime-tests-results.xml'
failTaskOnFailedTests: true
```

Notes:
* This is running the GTK head using a virtual x-server (xvfb).
* We use `{}` for the `UNO_RUNTIME_TESTS_RUN_TESTS` in order to run all tests with default configuration.
* If you want to test hot-relaod scenarios (usually relevant only for library developers), you need to build your test project in debug (`-c Debug`).
dr1rrb marked this conversation as resolved.
Show resolved Hide resolved
Loading