This action works in conjunction with another step that runs cypress run --reporter mochawesome. This action takes the generated results json file and creates a Status Check or PR Comment depending on the flags set with the test outcome.
This action does not run the Cypress tests itself and it can only process one results file at a time.
- process-cypress-test-results
The test status & action's conclusion can be viewed in multiple places:
- In the body of a PR comment this action generates
- Next to the name of one of the status checks under the
Checkssection of a PR - Next to the name of one of the status checks under the
Jobssection of the workflow run - In the body of a status check listed on the workflow run
If the test results contain failures, the status check's conclusion will be set to failure. If the status check is required and its conclusion is failure the PR cannot be merged. If this required status check behavior is not desired, the ignore-test-failures input can be set and the conclusion will be marked as neutral if test failures are detected. The status badge that is shown in the comment or status check body will still indicate it was a failure though.
GitHub does have a size limitation of 65535 characters for a Status Check body or a PR Comment. This action will fail if the test results exceed the GitHub limit. To mitigate this size issue only failed tests are included in the output.
If you have multiple workflows triggered by the same pull_request or push event, GitHub creates one checksuite for that commit. The checksuite gets assigned to one of the workflows randomly and all status checks for that commit are reported to that checksuite. That means if there are multiple workflows with the same trigger, your status checks may show on a different workflow run than the run that created them.
This is shown on the pull request when the create-pr-comment is set to true and there is a PR associated with the commit.
This is shown on the pull request when the create-status-check is set to true and there is a PR associated with the commit.
This is shown on the workflow run when the create-status-check is set to true.
For failed test runs you can expand each failed test and view more details about the failure
| Parameter | Is Required | Default | Description |
|---|---|---|---|
github-token |
true | N/A | Used for the GitHub Checks API. Value is generally: secrets.GITHUB_TOKEN. |
results-file |
true | N/A | The json results file generated by cypress & mochawesome reporter. |
report-name |
false | Cypress Test Results |
The desired name of the report that is shown on the PR Comment and inside the Status Check. |
create-status-check |
false | true |
Flag indicating whether a status check with cypress test results should be generated. |
ignore-test-failures |
false | false |
If there are test failures, the check's conclusion is set to neutral so it will not block pull requests.Only applicable when create-status-check is true. |
create-pr-comment |
false | true |
Flag indicating whether a PR comment with cypress test results should be generated. When true the default behavior is to update an existing comment if one exists. |
update-comment-if-one-exists |
false | true |
This flag determines whether a new comment is created or if the action updates an existing comment (if one is found). Only applicable when create-pr-comment is true. |
comment-identifier |
false | ${{ env.GITHUB-JOB }}_${{ env.GITHUB-ACTION }} |
A unique identifier which will be added to the generated markdown as a comment (it will not be visible in the PR comment). This identifier enables creating then updating separate results comments on the PR if more than one instance of this action is included in a single job. This can be helpful when there are multiple test projects that run separately but are part of the same job. Only applicable when create-pr-comment is true. |
timezone |
false | UTC |
IANA time zone name (e.g. America/Denver) to display dates in. |
| Output | Description |
|---|---|
test-outcome |
Test outcome based on presence of failing tests: Failed,Passed If exceptions are thrown or if it exits early because of argument errors, this is set to Failed. |
test-results-truncated |
Flag indicating whether test results were truncated due to markdown exceeding character limit of 65535. |
test-results-file-path |
File path for the file that contains the pre-truncated test results in markdown format. This is the same output that is posted in the PR comment. |
status-check-id |
The ID of the Status Check that was created. This is only set if create-status-check is true and a status check was created successfully. |
pr-comment-id |
The ID of the PR comment that was created. This is only set if create-pr-comment is true and a PR was created successfully. |
-
These examples require a few npm packages to be installed, generally as dev dependencies:
- mochawesome
- mochawesome-merge
- mochawesome-report-generator
-
Some npm scripts have also been added to the project
"scripts": { "precypressrun": "rm -rf mochawesome-report/", "cypressrun": "cypress run --project ./ --reporter mochawesome --reporter-options overwrite=false,html=false,json=true", "cypressreport": "mochawesome-merge mochawesome-report/*.json > raw-results.json && marge --reportDir cypress-results raw-results.json" }
cypressrun:overwrite=false: This flag is set to false so a separate json file is created per test/suite. If not set, each test overwrites the previous test results so a comprehensive report cannot be generated.html=false: This tells mochawesome not to generate an html report since we'll be doing that after the test run.json=true: This tells mochawesome to generate json result files which we will be merged together in thecypressreportscript.
precypressrun:- Since the
overwriteflag is false, we need a way to remove results between test runs which is what this script does. By default mochawesome puts the results in themochawesome-reportdirectory.
- Since the
cypressreport:mochawesome-merge mochawesome-report/*.json > raw-results.json: merges all json files into onemarge --reportDir cypress-results raw-results.json: Generates the html report from the merged json file and puts it in thecyress-resultsdirectory
A step by step tutorial that sets up mochawesome can be found on Test reporting with Cypress and Mochawesome.
jobs:
advanced-cypress:
runs-on: [windows-2019]
permissions:
contents: read
checks: write
statuses: write
pull-requests: write
env:
APP_DIR: './src/MyProject/FrontEnd'
defaults:
run:
shell: bash
working-directory: ${{ env.APP_DIR }}
steps:
- uses: actions/checkout@v4
- name: Restore npm packages
run: npm install
- name: dotnet build & run the app
working-directory: ./
run: |
dotnet build DemoApp15.sln
dotnet run
- name: Run cypress
id: cypress
continue-on-error: true
run: npm run cypressrun
# This creates the json file needed for this action
- name: Merge results into a single json and create html report
run: npm run cypressreport
- name: Create Status check based on merged cypress results
if: always()
id: process-cypress
# You may also reference just the major or major.minor version
uses: im-open/process-cypress-test-results@v3.0.2
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
results-file: '${{ env.APP_DIR }}/raw-results.json' # Name set up in npm script `cypressreport`
report-name: 'Cypress ${{ github.run_number }}' # Default: Cypress Test Results
create-status-check: true # Default: true
create-pr-comment: false # Default: true
update-comment-if-one-exists: false # Default: true
ignore-test-failures: true # Default: false
timezone: 'america/denver' # Default: UTC
- name: Annotate Test Outcome
if: steps.process-cypress.outputs.test-outcome-truncated == 'true'
run: cat ${{ steps.process-cypress.outputs.test-outcome-file-path }} > $GITHUB_STEP_SUMMARY
# Optional
- name: Upload Outcome as artifact if character limit reached
if: steps.process-cypress.outputs.test-outcome-truncated == 'true'
uses: actions/upload-artifact@v4
with:
name: Cypress-Results
path: |
${{ steps.process-cypress.outputs.test-outcome-file-path }}
retention-days: 7
- name: Fail if there were errors in the cypress tests
if: steps.process-cypress.outputs.test-outcome == 'Failed'
run: |
echo "There were cypress test failures."
exit 1- The
test-outcome-truncatedoutput was renamed totest-results-truncated - The
test-outcome-file-pathoutput was renamed totest-results-file-path
When creating PRs, please review the following guidelines:
- The action code does not contain sensitive information.
- At least one of the commit messages contains the appropriate
+semver:keywords listed under Incrementing the Version for major and minor increments. - The action has been recompiled. See Recompiling Manually for details.
- The README.md has been updated with the latest version of the action. See Updating the README.md for details.
This repo uses git-version-lite in its workflows to examine commit messages to determine whether to perform a major, minor or patch increment on merge if source code changes have been made. The following table provides the fragment that should be included in a commit message to active different increment strategies.
| Increment Type | Commit Message Fragment |
|---|---|
| major | +semver:breaking |
| major | +semver:major |
| minor | +semver:feature |
| minor | +semver:minor |
| patch | default increment type, no comment needed |
The files and directories that are considered source code are listed in the files-with-code and dirs-with-code arguments in both the build-and-review-pr and increment-version-on-merge workflows.
If a PR contains source code changes, the README.md should be updated with the latest action version and the action should be recompiled. The build-and-review-pr workflow will ensure these steps are performed when they are required. The workflow will provide instructions for completing these steps if the PR Author does not initially complete them.
If a PR consists solely of non-source code changes like changes to the README.md or workflows under ./.github/workflows, version updates and recompiles do not need to be performed.
This command utilizes esbuild to bundle the action and its dependencies into a single file located in the dist folder. If changes are made to the action's source code, the action must be recompiled by running the following command:
# Installs dependencies and bundles the code
npm run buildIf changes are made to the action's source code, the usage examples section of this file should be updated with the next version of the action. Each instance of this action should be updated. This helps users know what the latest tag is without having to navigate to the Tags page of the repository. See Incrementing the Version for details on how to determine what the next version will be or consult the first workflow run for the PR which will also calculate the next version.
This project has adopted the im-open's Code of Conduct.
Copyright © 2024, Extend Health, LLC. Code released under the MIT license.