The test runner gets stuck in the "Testing in playmode" phase #188
Labels
bug
Something isn't working
Comments
|
I have messed with same behaviour... Test runner just freezing on "Testing in playmode" |
|
Same problem here. Locally the tests take around 7s to complete. |
|
We're getting the same problem with both edit and playmode tests. |
|
Any update with this problem? I've got the same issue after update unity 2020.3.38f1 |
Updates would be posted in this thread. It doesn't look like a complete failure for projects and all versions, since there's only 5 reports here in 2 to 3 months. What would help is if everyone added their unity versions, and ideally also some stats on memory usage (you could use atop or other measuring tools). Either way it may be hard to debug the cause here because the Unity Editor isn't giving us much to work with. Perhaps some kind of debug output would also help. Not sure if anyone managed to get more output than nothing at all? |
|
We experienced failures using unity 2019.4.40f1 - name: Run Tests
id: tests
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
run: |
docker run --gpus all --rm \
--env UNITY_LICENSE \
--volume="${PWD}:/opt/project" \
unityci/editor:ubuntu-2019.4.40f1-linux-il2cpp-1 bash -c /opt/project/ci.shci.sh #!/bin/bash
FILE_PATH=UnityLicenseFile.ulf
echo "$UNITY_LICENSE" | tr -d '\r' >$FILE_PATH
unity-editor \
-batchmode \
-nographics \
-logFile /dev/stdout \
-quit \
-manualLicenseFile $FILE_PATH
platforms=(editmode playmode)
COVERAGE_OPTIONS="pathFilters:+/**/Assets/Scripts/**;assemblyFilters:+assets;generateAdditionalMetrics;generateHtmlReport;generateBadgeReport"
for platform in "${platforms[@]}"; do
unity-editor \
-runTests \
-projectPath /opt/project \
-testResults /opt/project/Logs/${platform}_test_results.xml \
-logfile /dev/stdout \
-debugCodeOptimization \
-enableCodeCoverage \
-testPlatform $platform \
-coverageResultsPath /opt/project/CodeCoverage \
-coverageOptions $COVERAGE_OPTIONS
done |
|
just found solution in my case, just changing |
|
this solves our issue |
That worked, thanks! |
|
Looks like one of the releases caused this. It would be extremely helpful if you could help us find out which one. Do the versions |
I have tested |
|
I am using Unity version 2019.4.28f1. The project does not have any playmode tests only editmode tests. test-runner@v2.0.0 it hangs at "Testing in Editmode" Trying to run the tests on window-latests also failed: Looks like the simlink is broken? |
The windows issue is related to this issue it seems. |
|
I also have this problem with play-mode tests. I tried the "2.0.0" solution but it doesn't work for me. It would also be helpful to flush log messages as they happen instead of dumping them all at the end. Even before this bug the entire bunch of tests log messages would be dumped at one time point--after all the tests were complete--and all having almost the same time stamp. |
|
Grouping and flushing output is tracked in game-ci/cli#12 Contributions are welcomed :) |
|
I just tried running the test on 2021.3.14f1 and instead of getting stuck on "Testing in Playmode" it fails when trying to fetch private git packages. I wonder if that might always be the reason it fails and just does not handle it properly in earlier versions. |
I still have the bug with Unity 2021.3.14f1. It sounds to me that your fetch failing should not have anything to do with the Unity version. |
Probably, you need this step for running test (important line # Test
- name: Run Tests
uses: game-ci/unity-test-runner@v2
id: tests
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
githubToken: ${{ secrets.GITHUB_TOKEN }}or: # Build
- uses: game-ci/unity-builder@v2
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
targetPlatform: ${{ matrix.targetPlatform }}
buildsPath: 'Builds'
buildName: ${{ needs.setup.outputs.completeName }}Of course, you need to add Github secret token which has access of private repos (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) |
Tested yesterday |
I do have the gitPrivateToken. I am using the same config that works for the builder: # Test
- name: Test 🔬
uses: game-ci/unity-test-runner@v2
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
with:
githubToken: ${{ secrets.GITHUB_TOKEN }}
gitPrivateToken: ${{ secrets.PRIVATE_TOKEN }}
customParameters: -username='${{ secrets.USERNAME }}' -password='${{ secrets.PASSWORD }}' |
If possible, please send here your log file. Also, be sure that you gave enough access for you generated Github access token |
|
The log just says: |
|
Not sure how you are fetching git projects but in our case, we do through UPM as far as I remember. If you are fetching through git directly, this might be useful https://stackoverflow.com/a/73229045/2122457 Just in case I put my current CI configuration that might be useful: # See: https://game.ci/docs/github/getting-started
name: CI
on:
workflow_dispatch: { }
push:
branches: [ master ]
pull_request:
types: [review_requested, ready_for_review]
branches: [ master ]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
name: Unity Tests Workflow
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
matrix:
unityVersion:
- 2020.3.19f1
testMode:
# - playmode
- editmode
steps:
# Checkout (without LFS)
- name: Checkout Repository
uses: actions/checkout@v2
# Git LFS
- name: Create LFS file list
run: git lfs ls-files -l | cut -d' ' -f1 | sort > .lfs-assets-id
- name: Restore LFS cache
uses: actions/cache@v2
id: lfs-cache
with:
path: .git/lfs
key: ${{ runner.os }}-lfs-${{ hashFiles('.lfs-assets-id') }}
- name: Git LFS Pull
run: |
git lfs pull
git add .
git reset --hard
# Cache
- uses: actions/cache@v2
with:
path: Library
key: Library-${{ hashFiles('Assets/**', 'Packages/**', 'ProjectSettings/**') }}
restore-keys: |
Library-
# Test
- name: Run Tests
uses: game-ci/unity-test-runner@v2.0.0
id: tests
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
githubToken: ${{ secrets.GITHUB_TOKEN }}
# Test Results
- name: Upload Test Results
uses: actions/upload-artifact@v2
if: always()
with:
name: Test Restuls
path: ${{ steps.tests.outputs.artifactsPath }}
If you still experiencing some problems, If I were you, I would try to run Github Actions locally (for example, https://github.com/nektos/act) and try to debug your DevOps scenario on a local machine. |
|
I talked with my friend, so in our project, we are not using other private git repos anymore, we have embedded them into Unity project. We have done that as not all team members use Git or have access to all private repos. In your case, you need to follow |
Thanks for your help. As mentioned the same settings work for the Build step to get the private packages. |
|
I'd like to return to the error in the OP - the test runner gets stuck. |
Have you tried |
|
Yes. Unfortunately it didn't help in my case. (I wrote this above) |
desplesda
added a commit
to YarnSpinnerTool/YarnSpinner-Unity
that referenced
this issue
Mar 2, 2023
Attempting a workaround for this problem: game-ci/unity-test-runner#188
|
I've got the same hang with Workflow file: https://github.com/Ecilos/ecilos-unity-assets-sandbox/actions/runs/4288532708/workflow |
|
@kenorb try: - name: Set up Unity
uses: game-ci/unity-test-runner@v2.0.0 # <--
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
projectPath: .
githubToken: ${{ secrets.GITHUB_TOKEN }} |
Thanks. Seems like it worked using Since v2.0.0 is older, do you know what's the (known) issue with the newer version? |
|
No problem. I am not sure what is wrong. The maintainer should answer this question. FIY, I have tested |
desplesda
added a commit
to YarnSpinnerTool/YarnSpinner-Unity
that referenced
this issue
Mar 6, 2023
Attempting a workaround for this problem: game-ci/unity-test-runner#188
|
I am also facing this issue with playmode tests. I am using unity version |
|
Any chance your tests involve a coroutine that has |
|
Yes, there are multiple coroutines which has |
|
Hey, again, tested Somehow fixed it the last time, but here it happens again. |
|
@asyasmi what output do you get? |
|
To this day I am using Some advice from experience using this action:
|
|
@ugnelis
that's all before the cancellation of the job :( Regarding UPD: Eventually that were our tests. So I'd really be waiting for game-ci/cli#12 to be realized. |
|
I'm not sure if the progress logs get printed to the Unity log, but I do this in my tests to see what test caused a hang (works in my Core tests). public static class TestHelper
{
private static Stopwatch _stopwatch;
public static void Setup()
{
if (_stopwatch == null)
{
// Other global setup
_stopwatch = Stopwatch.StartNew();
}
// Other per-test setup
TestContext.Progress.WriteLine("Begin time: " + _stopwatch.Elapsed.ToString() + ", test: " + TestContext.CurrentContext.Test.FullName);
}
public static void Cleanup()
{
// Other per-test cleanup
TestContext.Progress.WriteLine("Success time: " + _stopwatch.Elapsed.ToString() + ", test: " + TestContext.CurrentContext.Test.FullName);
}
}Then in all of my test classes, I put this: [SetUp]
public void Setup()
{
TestHelper.Setup();
}
[TearDown]
public void Teardown()
{
TestHelper.Cleanup();
} |
|
Thanks, @timcassell ! |
|
I guess this could be interesting: https://docs.unity3d.com/2022.3/Documentation/Manual/CLIBatchmodeCoroutines.html |
|
Edit: I tried using the unityVersion I still have this error. :( I just added a simple empty test like that: using NUnit.Framework;
public class SampleEditorTest
{
[Test]
public void SampleEditorTestSimplePasses()
{
Assert.Pass();
}
}the code: name: Testing
on:
workflow_dispatch:
pull_request:
push:
jobs:
test:
name: Testing in ${{ matrix.testMode }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
testMode:
- editmode
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
lfs: true
- name: Try caching Library folder for faster builds
uses: actions/cache@v3
with:
path: ${{ matrix.projectPath }}/Library
key: Library-${{ matrix.projectPath }}
restore-keys: |
Library-
- name: Setup GameCI-Environment for testing and test
uses: game-ci/unity-test-runner@v4
id: testStep
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
with:
unityVersion: 2023.2.9f1
testMode: ${{ matrix.testMode }}Log: |
|
Nevermind, I waited now like 13 mins and it got it done. It does stuff in the background, it just does not print it to the console. Because of this, I thought, that it does not work. Just after it ran all tests, it shows the whole output. Maybe this should be changed, so that the output gets shown right away, that a user can see, that something happens. :) In the performance information is logged, that the initialization of the project took like 6mins. That seems pretty long for just 1 test without any extra dependencies. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug description
The test runner gets stuck in the "Testing in playmode" phase. I have tried
ubuntu-latestandwindows-2022docker images.CI configuration is almost the same as described in https://game.ci/docs/github/getting-started
Run Tests logs:
I used secret var
ACTIONS_STEP_DEBUG=truefor enabling verbose logs.Where could be the problem?
The text was updated successfully, but these errors were encountered: