Add support for SPM projects (#204)

* WIP: Test case in place, not working.

* Use project dir for working directory

* Fix tests

* Remove test repetition mode from test

* Update go-xcode

* Improve input validation

* Use forked sample repo

* Cleanup

* Add more tests

* Update README

* Update go-xcode to final version

* Code cleanup

Co-authored-by: Michael Matranga <michael.matranga@bitrise.io>

README.md

@@ -7,19 +7,19 @@ Runs your project's pre-defined Xcode tests on every build.
 <details>
 <summary>Description</summary>
 
-This Steps runs all those Xcode tests that are included in your project. 
-The Step will work out of the box if your project has test targets and your Workflow has the **Deploy to Bitrise.io** Step which exports the test results and (code coverage files if needed) to the Test Reports page. 
+This Steps runs all those Xcode tests that are included in your project.
+The Step will work out of the box if your project has test targets and your Workflow has the **Deploy to Bitrise.io** Step which exports the test results and (code coverage files if needed) to the Test Reports page.
 This Step does not need any code signing files since the Step deploys only the test results to [bitrise.io](https://www.bitrise.io).
 
 ### Configuring the Step
-If you click into the Step, there are some required input fields whose input must be set in accordance with the Xcode configuration of the project.  
+If you click into the Step, there are some required input fields whose input must be set in accordance with the Xcode configuration of the project.
 The **Scheme** input field must be marked as Shared in Xcode.
 
 ### Troubleshooting
 If the **Deploy to Bitrise.io** Step is missing from your Workflow, then the **Xcode Test for iOS** Step will not be able to export the test results on the Test Reports page and you won't be able to view them either.
-The xcpretty output tool does not support parallel tests. 
+The xcpretty output tool does not support parallel tests.
 If parallel tests are enabled in your project, go to the Step's Debug section and set the **Log formatter** input's value to xcodebuild.
-If the Xcode test fails with the error `Unable to find a destination matching the provided destination specifier`, then check our [system reports](https://github.com/bitrise-io/bitrise.io/tree/master/system_reports) to see if the requested simulator is on the stack or not. 
+If the Xcode test fails with the error `Unable to find a destination matching the provided destination specifier`, then check our [system reports](https://github.com/bitrise-io/bitrise.io/tree/master/system_reports) to see if the requested simulator is on the stack or not.
 If it is not, then pick a simulator that is on the stack.
 
 ### Useful links
@@ -44,7 +44,7 @@ You can also run this step directly with [Bitrise CLI](https://github.com/bitris
 
 | Key | Description | Flags | Default |
 | --- | --- | --- | --- |
-| `project_path` | Xcode Project (`.xcodeproj`) or Workspace (`.xcworkspace`) path.  The input value sets xcodebuild's `-project` or `-workspace` option. | required | `$BITRISE_PROJECT_PATH` |
+| `project_path` | Xcode Project (`.xcodeproj`) or Workspace (`.xcworkspace`) path. The input value sets xcodebuild's `-project` or `-workspace` option.  If this is a Swift package, this should be the path to the `Package.swift` file. | required | `$BITRISE_PROJECT_PATH` |
 | `scheme` | Xcode Scheme name.  The input value sets xcodebuild's `-scheme` option. | required | `$BITRISE_SCHEME` |
 | `destination` | Destination specifier describes the device to use as a destination.  The input value sets xcodebuild's `-destination` option. | required | `platform=iOS Simulator,name=iPhone 8 Plus,OS=latest` |
 | `test_plan` | Run tests in a specific Test Plan associated with the Scheme.  Leave this input empty to run the default Test Plan or Test Targets associated with the Scheme.  The input value sets xcodebuild's `-testPlan` option. |  |  |
@@ -53,13 +53,13 @@ You can also run this step directly with [Bitrise CLI](https://github.com/bitris
 | `relaunch_tests_for_each_repetition` | If this input is set, tests will launch in a new process for each repetition.  By default, tests launch in the same process for each repetition.  The input value sets xcodebuild's `-test-repetition-relaunch-enabled` option. |  | `no` |
 | `should_retry_test_on_fail` | If this input is set, the Step will rerun the tests in the case of failed tests.  Note that all the tests will be rerun, not just the ones that failed.  This input is not available if you are using Xcode 13+. In that case, we recommend using the `retry_on_failure` Test Repetition Mode (`test_repetition_mode`). | required | `no` |
 | `xcconfig_content` | Build settings to override the project's build settings.  Build settings must be separated by newline character (`\n`).  Example:  ``` COMPILER_INDEX_STORE_ENABLE = NO ONLY_ACTIVE_ARCH[config=Debug][sdk=*][arch=*] = YES ```  The input value sets xcodebuild's `-xcconfig` option. |  | `COMPILER_INDEX_STORE_ENABLE = NO` |
-| `perform_clean_action` |  | required | `no` |
-| `xcodebuild_options` |  |  |  |
+| `perform_clean_action` | If this input is set, `clean` xcodebuild action will be performed besides the `test` action. | required | `no` |
+| `xcodebuild_options` | Additional options to be added to the executed xcodebuild command. |  |  |
 | `log_formatter` | Defines how xcodebuild command's log is formatted.  Available options: - `xcpretty`: The xcodebuild command's output will be prettified by xcpretty. - `xcodebuild`: Only the last 20 lines of raw xcodebuild output will be visible in the build log.  The raw xcodebuild log will be exported in both cases. | required | `xcpretty` |
-| `xcpretty_options` |  |  | `--color --report html --output "${BITRISE_DEPLOY_DIR}/xcode-test-results-${BITRISE_SCHEME}.html"` |
+| `xcpretty_options` | Additional options to be added to the executed xcpretty command. |  | `--color --report html --output "${BITRISE_DEPLOY_DIR}/xcode-test-results-${BITRISE_SCHEME}.html"` |
 | `cache_level` | Defines what cache content should be automatically collected.  Available options: - `none`: Disable collecting cache content. - `swift_packages`: Collect Swift PM packages added to the Xcode project. |  | `swift_packages` |
-| `verbose_log` |  |  | `no` |
-| `collect_simulator_diagnostics` |  |  | `never` |
+| `verbose_log` | If this input is set, the Step will print additional logs for debugging. |  | `no` |
+| `collect_simulator_diagnostics` | If this input is set, the simulator verbose logging will be enabled and the simulator diagnostics log will be exported. |  | `never` |
 | `headless_mode` | In headless mode the simulator is not launched in the foreground.  If this input is set, the simulator will not be visible but tests (even the screenshots) will run just like if you run a simulator in foreground. |  | `yes` |
 </details>
 
@@ -68,11 +68,11 @@ You can also run this step directly with [Bitrise CLI](https://github.com/bitris
 
 | Environment Variable | Description |
 | --- | --- |
-| `BITRISE_XCODE_TEST_RESULT` |  |
+| `BITRISE_XCODE_TEST_RESULT` | Result of the tests. 'succeeded' or 'failed'. |
 | `BITRISE_XCRESULT_PATH` | The path of the generated `.xcresult`. |
 | `BITRISE_XCRESULT_ZIP_PATH` | The path of the zipped `.xcresult`. |
 | `BITRISE_XCODE_TEST_ATTACHMENTS_PATH` | This is the path of the test attachments zip. |
-| `BITRISE_XCODEBUILD_BUILD_LOG_PATH` | If `single_build` is set to false, the step runs `xcodebuild build` before the test,   and exports the raw xcodebuild log. |
+| `BITRISE_XCODEBUILD_BUILD_LOG_PATH` | If `single_build` is set to false, the step runs `xcodebuild build` before the test, and exports the raw xcodebuild log. |
 | `BITRISE_XCODEBUILD_TEST_LOG_PATH` | The step exports the `xcodebuild test` command output log. |
 </details>
 

e2e/bitrise.yml

@@ -2,6 +2,22 @@ format_version: "11"
 default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
 
 workflows:
+  test_swift_package:
+    envs:
+    - TEST_APP_URL: https://github.com/bitrise-io/swift-package-deckofplayingcards
+    - TEST_APP_BRANCH: main
+    - BITRISE_PROJECT_PATH: Package.swift
+    - BITRISE_SCHEME: DeckOfPlayingCards
+    - TEST_PLAN: ""
+    - DESTINATION: platform=iOS Simulator,name=iPhone 8,OS=latest
+    - LOG_FORMATTER: xcpretty
+    - RETRY_ON_FAILURE: "no"
+    - EXPECT_TEST_FAILURE: "false"
+    - CACHE_LEVEL: swift_packages
+    after_run:
+    - _run
+    - _check_outputs
+
   test_objc_xcpretty:
     envs:
     - TEST_APP_URL: https://github.com/bitrise-io/sample-apps-ios-simple-objc-with-uitest.git

go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/bitrise-io/go-utils v1.0.1
 	github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.2
 	github.com/bitrise-io/go-xcode v1.0.2
-	github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.3
+	github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.12
 	github.com/hashicorp/go-version v1.3.0
 	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
 	github.com/stretchr/testify v1.7.0

go.sum

@@ -20,8 +20,8 @@ github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.2/go.mod h1:sy+Ir1X8P3tAAx/qU/r+h
 github.com/bitrise-io/go-xcode v1.0.1/go.mod h1:Y0Wu2dXm0MilJ/4D3+gPHaNMlUcP+1DjIPoLPykq7wY=
 github.com/bitrise-io/go-xcode v1.0.2 h1:Uv/cBOJ/qZpitjOpyS8orafee3wk66OwvRTbqA2fr+4=
 github.com/bitrise-io/go-xcode v1.0.2/go.mod h1:Y0Wu2dXm0MilJ/4D3+gPHaNMlUcP+1DjIPoLPykq7wY=
-github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.3 h1:LnqktaZ9CC+rTjxKbWSpYSXfGuoTElmteThO++ZtiwE=
-github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.3/go.mod h1:6YbvyYwZgSTt96CQSQ6QlrkcRiv3ssX8zLijh2TPnbU=
+github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.12 h1:F0o2zLo19ZvA0KKo8+diY9+cfA0Vn4HyuIdE3xUrHEA=
+github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.12/go.mod h1:6YbvyYwZgSTt96CQSQ6QlrkcRiv3ssX8zLijh2TPnbU=
 github.com/bitrise-io/goinp v0.0.0-20210504152833-8559b0680ab1/go.mod h1:iRbd8zAXLeNy+0gic0eqNCxXvDGe8ZEY/uYX2CCeAoo=
 github.com/bitrise-io/gows v0.0.0-20210505125306-dd92ff463938/go.mod h1:3Cp9ceJ8wHl1Av6oEE2ff1iWaYLliQuD+oaNdyM0NWQ=
 github.com/bitrise-io/pkcs12 v0.0.0-20211108084543-e52728e011c8/go.mod h1:UiXKNs0essbC14a2TvGlnUKo9isP9m4guPrp8KJHJpU=

step.yml

@@ -52,11 +52,12 @@ inputs:
 - project_path: $BITRISE_PROJECT_PATH
   opts:
     title: Project path
-    summary: Xcode Project (`.xcodeproj`) or Workspace (`.xcworkspace`) path.
+    summary: Path of the Xcode Project (`.xcodeproj`), Workspace (`.xcworkspace`) or Swift package (`Package.swift`)
     description: |-
       Xcode Project (`.xcodeproj`) or Workspace (`.xcworkspace`) path.
-
       The input value sets xcodebuild's `-project` or `-workspace` option.
+
+      If this is a Swift package, this should be the path to the `Package.swift` file.
     is_required: true
 
 - scheme: $BITRISE_SCHEME
@@ -274,7 +275,8 @@ inputs:
 outputs:
 - BITRISE_XCODE_TEST_RESULT:
   opts:
-    title: Result of the tests. 'succeeded' or 'failed'.
+    title: Test result
+    description: Result of the tests. 'succeeded' or 'failed'.
     value_options:
     - succeeded
     - failed

step/step.go

@@ -156,8 +156,9 @@ func (s XcodeTestRunner) ProcessConfig() (Config, error) {
 	if err != nil {
 		return Config{}, fmt.Errorf("failed to get absolute project path: %w", err)
 	}
-	if filepath.Ext(projectPath) != ".xcodeproj" && filepath.Ext(projectPath) != ".xcworkspace" {
-		return Config{}, fmt.Errorf("invalid project file (%s), extension should be (.xcodeproj/.xcworkspace)", projectPath)
+	fileExtension := filepath.Ext(projectPath)
+	if fileExtension != ".xcodeproj" && fileExtension != ".xcworkspace" && filepath.Base(projectPath) != "Package.swift" {
+		return Config{}, fmt.Errorf("invalid project path: should be an .xcodeproj/.xcworkspace or Package.swift file (actual: %s)", projectPath)
 	}
 
 	sim, err := s.getSimulatorForDestination(input.Destination)
@@ -413,7 +414,6 @@ func (s XcodeTestRunner) runTests(cfg Config) (Result, int, error) {
 		return result, -1, fmt.Errorf("could not create test output temporary directory: %w", err)
 	}
 	xcresultPath := path.Join(tempDir, fmt.Sprintf("Test-%s.xcresult", cfg.Scheme))
-
 	var swiftPackagesPath string
 	if cfg.XcodeMajorVersion >= 11 {
 		var err error

vendor/modules.txt

@@ -37,7 +37,7 @@ github.com/bitrise-io/go-utils/v2/pathutil
 github.com/bitrise-io/go-xcode/models
 github.com/bitrise-io/go-xcode/simulator
 github.com/bitrise-io/go-xcode/utility
-# github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.3
+# github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.12
 ## explicit
 github.com/bitrise-io/go-xcode/v2/destination
 github.com/bitrise-io/go-xcode/v2/xcconfig

xcodebuild/utils.go

@@ -68,13 +68,14 @@ type TestParams struct {
 	AdditionalOptions              string
 }
 
-func (b *xcodebuild) runXcodebuildCmd(args ...string) (string, int, error) {
+func (b *xcodebuild) runXcodebuildCmd(workDir string, args ...string) (string, int, error) {
 	var outBuffer bytes.Buffer
 
 	cmd := b.commandFactory.Create("xcodebuild", args, &command.Opts{
 		Stdout: &outBuffer,
 		Stderr: &outBuffer,
 		Env:    xcodeCommandEnvs,
+		Dir:    workDir,
 	})
 
 	b.logger.Printf("$ %s", cmd.PrintableCommandArgs())
@@ -89,7 +90,7 @@ func (b *xcodebuild) runXcodebuildCmd(args ...string) (string, int, error) {
 	return outBuffer.String(), exitCode, err
 }
 
-func (b *xcodebuild) runPrettyXcodebuildCmd(useStdOut bool, xcprettyArgs []string, xcodebuildArgs []string) (string, int, error) {
+func (b *xcodebuild) runPrettyXcodebuildCmd(workDir string, xcprettyArgs []string, xcodebuildArgs []string) (string, int, error) {
 	// build outputs:
 	// - write it into a buffer
 	// - write it into the pipe, which will be fed into xcpretty
@@ -98,15 +99,13 @@ func (b *xcodebuild) runPrettyXcodebuildCmd(useStdOut bool, xcprettyArgs []strin
 	buildOutWriters := []io.Writer{pipeWriter}
 	buildOutWriter := CreateBufferedWriter(&buildOutBuffer, buildOutWriters...)
 	//
-	var prettyOutWriter io.Writer
-	if useStdOut {
-		prettyOutWriter = os.Stdout
-	}
+	prettyOutWriter := os.Stdout
 
 	buildCmd := b.commandFactory.Create("xcodebuild", xcodebuildArgs, &command.Opts{
 		Stdout: buildOutWriter,
 		Stderr: buildOutWriter,
 		Env:    xcodeCommandEnvs,
+		Dir:    workDir,
 	})
 
 	prettyCmd := b.commandFactory.Create("xcpretty", xcprettyArgs, &command.Opts{
@@ -147,11 +146,16 @@ func (b *xcodebuild) runPrettyXcodebuildCmd(useStdOut bool, xcprettyArgs []strin
 }
 
 func (b *xcodebuild) createXcodebuildTestArgs(params TestParams) ([]string, error) {
-	projectFlag := "-project"
-	if filepath.Ext(params.ProjectPath) == ".xcworkspace" {
-		projectFlag = "-workspace"
+	var xcodebuildArgs []string
+
+	fileExtension := filepath.Ext(params.ProjectPath)
+	if fileExtension == ".xcodeproj" {
+		xcodebuildArgs = append(xcodebuildArgs, "-project", params.ProjectPath)
+	} else if fileExtension == ".xcworkspace" {
+		xcodebuildArgs = append(xcodebuildArgs, "-workspace", params.ProjectPath)
 	}
-	xcodebuildArgs := []string{projectFlag, params.ProjectPath, "-scheme", params.Scheme}
+	xcodebuildArgs = append(xcodebuildArgs, "-scheme", params.Scheme)
+
 	if params.PerformCleanAction {
 		xcodebuildArgs = append(xcodebuildArgs, "clean")
 	}
@@ -243,8 +247,12 @@ func (b *xcodebuild) runTest(params TestRunParams) (string, int, error) {
 		return "", 1, err
 	}
 
-	b.logger.Infof("Running the tests...")
+	b.logger.Donef("Running the tests...")
 
+	// When the project path input is set to an SPM Package.swift file, we need to execute the xcodebuild command
+	// within the working directory of the project. This is optional for regular workspaces and projects,
+	// because we use the `-project` flag to point to the .xcproj/xcworkspace, but we do it for consistency.
+	workDir := filepath.Dir(params.TestParams.ProjectPath)
 	var rawOutput string
 	var exit int
 	var testErr error
@@ -253,13 +261,14 @@ func (b *xcodebuild) runTest(params TestRunParams) (string, int, error) {
 		if err != nil {
 			return "", 1, err
 		}
-
-		rawOutput, exit, testErr = b.runPrettyXcodebuildCmd(true, xcprettyArgs, xcodebuildArgs)
+		rawOutput, exit, testErr = b.runPrettyXcodebuildCmd(workDir, xcprettyArgs, xcodebuildArgs)
 	} else {
-		rawOutput, exit, testErr = b.runXcodebuildCmd(xcodebuildArgs...)
+		rawOutput, exit, testErr = b.runXcodebuildCmd(workDir, xcodebuildArgs...)
 	}
 
-	fmt.Println("exit: ", exit)
+	if exit != 0 {
+		fmt.Println("Exit code: ", exit)
+	}
 
 	if testErr != nil {
 		return b.handleTestRunError(params, testRunResult{xcodebuildLog: rawOutput, exitCode: exit, err: testErr})

xcodebuild/xcodebuild_test.go

@@ -62,15 +62,24 @@ func Test_GivenXcodebuild_WhenInvoked_ThenUsesCorrectArguments(t *testing.T) {
 				parameters := runParameters()
 				parameters.TestParams.RelaunchTestsForEachRepetition = false
 
+				return parameters
+			},
+		},
+		{
+			name: "Swift package",
+			input: func() TestRunParams {
+				parameters := runParameters()
+				parameters.TestParams.ProjectPath = "MyPackage/Package.swift"
+
 				return parameters
 			},
 		},
 	}
 
 	for _, test := range tests {
-		t.Log(test.name)
-
-		runArgumentsTest(t, test.input())
+		t.Run(test.name, func(t *testing.T) {
+			runArgumentsTest(t, test.input())
+		})
 	}
 }
 
@@ -266,7 +275,7 @@ func createXcodebuildAndMocks(stdoutProvider func() string) (Xcodebuild, testing
 
 func runParameters() TestRunParams {
 	testParams := TestParams{
-		ProjectPath:                    "ProjectPath",
+		ProjectPath:                    "ProjectPath.xcodeproj",
 		Scheme:                         "Scheme",
 		Destination:                    "Destination",
 		TestPlan:                       "TestPlan",
@@ -292,11 +301,14 @@ func runParameters() TestRunParams {
 }
 
 func argumentsFromRunParameters(parameters TestRunParams) []string {
-	arguments := []string{
-		"-project", parameters.TestParams.ProjectPath,
-		"-scheme", parameters.TestParams.Scheme,
+	var arguments []string
+
+	if !strings.HasSuffix(parameters.TestParams.ProjectPath, "Package.swift") {
+		arguments = append(arguments, "-project", parameters.TestParams.ProjectPath)
 	}
 
+	arguments = append(arguments, "-scheme", parameters.TestParams.Scheme)
+
 	if parameters.TestParams.PerformCleanAction {
 		arguments = append(arguments, "clean")
 	}