Initiating the Extractor and Publisher Pipelines deployed within Azure DevOps from a GitHub Repository

[anotherRedbeard]

Release version

v4.10.1

Describe the bug

I would like to be able to run the AzDO yaml pipelines to extract and publish my changes from API Management while having my code be stored in a GitHub Repository. I tried to use the existing pipelines to accomplish this, but was unsuccessful.

Expected behavior

The expectation would be for extractor to extract the files, and create a pull request and once the pull request is merged the publisher would publish the changes to the API Management environments.

Actual behavior

What actually happened when I ran the extractor was this:

Cloning branch main in repository <repo-name>...

ERROR: TF401019: The Git repository with name or identifier <repo-name> does not exist or you do not have permissions for the operation you are attempting.

Clone URL is

Cloning into 'artifacts-from-portal'...

fatal: 'D:\a\_temp\artifacts-from-portal' does not appear to be a git repository

fatal: Could not read from remote repository.

 

Please make sure you have the correct access rights

and the repository exists.

Cloning branch main in repository <repo-name> failed.

At D:\a\_temp\<guid>.ps1:24 char:28

+ … DE -ne 0) { throw "Cloning branch $branchName in repository $reposito …

+               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo          : OperationStopped: (Cloning branch main…<repo-name> failed.:String) [], RuntimeException

+ FullyQualifiedErrorId : Cloning branch main in repository <repo-name> failed.

##[debug]Exit code: 1

##[debug]Leaving Invoke-VstsTool.

##[error]PowerShell exited with code '1'.

Reproduction Steps

For the setup, I am following the steps in the document for the AzDO setup and then ran the extractor manually.

[github-actions]

🎉 Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the closed issues in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes. 🎉

[waelkdouh]

@anotherRedbeard when you say "while having my code be stored in a GitHub Repository" you mean you would like to store the pipelines, the PRs, as well as the extracted artifacts on the Github side? So you would basically have azure devops run the pipelines only?

[anotherRedbeard]

@waelkdouh , yes that is correct!

[anotherRedbeard]

I've been reviewing the pipelines and specifically the 'Create Pull Request' powershell and believe I've been able to make some minor changes to the publisher.yaml and extractor.yaml pipelines and got this to run successfully. From a high level, I used a Github Personal Access Token(PAT) to create the PR instead of using the az cli with the azure-devops extension. Here are the detailed steps if anyone else is interested in doing this

• Create a Github PAT that you can use to authenticate using the github API (there are other ways to authenticate to the GH api, this is just the way I chose). How to create a PAT
• Add the GITHUB_PAT and GITHUB_OWNER variables to the variable group you are using apim-automation if you are following the docs.
  
• Update the Create pull request powershell task and replace the az cli calls with github api calls. Here is my run-extractor.yaml file
• Update the run-extractor.yaml and run-publisher.yaml to disable pull request triggers. This will keep your pipelines from triggering with the PR is created. YAML PR triggers are only supported in GitHub, which is why you don't have to do this in Azure DevOps. See this document

That should be it. With that I'll close this issue.