diff --git a/README.md b/README.md index 2ee51ed0..9536b8fb 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ The goal of ```GARM``` is to be simple to set up, simple to configure and simple GARM supports creating pools on either GitHub itself or on your own deployment of [GitHub Enterprise Server](https://docs.github.com/en/enterprise-server@3.5/admin/overview/about-github-enterprise-server). For instructions on how to use ```GARM``` with GHE, see the [credentials](/doc/github_credentials.md) section of the documentation. -Through the use of providers, `GARM` can create runners in a variety of environments using the same `GARM` instance. Want to create pools of runners in your OpenStack cloud, your Azure cloud and your Kubernetes cluster? No problem! Just install the appropriate providers, configure them in `GARM` and you're good to go. Create zero-runner pools for instances with high costs (large VMs, GPU enabled instances, etc) and have them spin up on demand, or create large pools of k8s backed runners that can be used for your CI/CD pipelines at a moment's notice. You can mix them up and create pools in any combination of providers or resource allocations you want. +Through the use of providers, `GARM` can create runners in a variety of environments using the same `GARM` instance. Whether you want to create pools of runners in your OpenStack cloud, your Azure cloud and your Kubernetes cluster, that is easily achieved by just installing the appropriate providers, configuring them in `GARM` and creating pools that use them. You can create zero-runner pools for instances with high costs (large VMs, GPU enabled instances, etc) and have them spin up on demand, or you can create large pools of k8s backed runners that can be used for your CI/CD pipelines at a moment's notice. You can mix them up and create pools in any combination of providers or resource allocations you want. :warning: **Important note**: The README and documentation in the `main` branch are relevant to the not yet released code that is present in `main`. Following the documentation from the `main` branch for a stable release of GARM, may lead to errors. To view the documentation for the latest stable release, please switch to the appropriate tag. For information about setting up `v0.1.4`, please refer to the [v0.1.4 tag](https://github.com/cloudbase/garm/tree/v0.1.4) diff --git a/doc/using_garm.md b/doc/using_garm.md index 564a8bf8..6880330c 100644 --- a/doc/using_garm.md +++ b/doc/using_garm.md @@ -7,34 +7,55 @@ While using the GARM cli, you will most likely spend most of your time listing p - [Using GARM](#using-garm) - - [Listing controller info](#listing-controller-info) - - [Listing configured providers](#listing-configured-providers) - - [Listing github credentials](#listing-github-credentials) - - [Adding a new repository](#adding-a-new-repository) - - [Managing repository webhooks](#managing-repository-webhooks) - - [Listing repositories](#listing-repositories) - - [Removing a repository](#removing-a-repository) - - [Adding a new organization](#adding-a-new-organization) - - [Adding an enterprise](#adding-an-enterprise) - - [Creating a runner pool](#creating-a-runner-pool) - - [Listing pools](#listing-pools) - - [Showing pool info](#showing-pool-info) - - [Deleting a pool](#deleting-a-pool) - - [Update a pool](#update-a-pool) - - [Listing runners](#listing-runners) - - [Showing runner info](#showing-runner-info) - - [Deleting a runner](#deleting-a-runner) + - [Controller operations](#controller-operations) + - [Listing controller info](#listing-controller-info) + - [Updating controller settings](#updating-controller-settings) + - [Providers](#providers) + - [Listing configured providers](#listing-configured-providers) + - [Github Endpoints](#github-endpoints) + - [Creating a GitHub Endpoint](#creating-a-github-endpoint) + - [Listing GitHub Endpoints](#listing-github-endpoints) + - [Getting information about an endpoint](#getting-information-about-an-endpoint) + - [Deleting a GitHub Endpoint](#deleting-a-github-endpoint) + - [GitHub credentials](#github-credentials) + - [Adding GitHub credentials](#adding-github-credentials) + - [Listing GitHub credentials](#listing-github-credentials) + - [Getting detailed information about credentials](#getting-detailed-information-about-credentials) + - [Deleting GitHub credentials](#deleting-github-credentials) + - [Repositories](#repositories) + - [Adding a new repository](#adding-a-new-repository) + - [Listing repositories](#listing-repositories) + - [Removing a repository](#removing-a-repository) + - [Organizations](#organizations) + - [Adding a new organization](#adding-a-new-organization) + - [Enterprises](#enterprises) + - [Adding an enterprise](#adding-an-enterprise) + - [Managing webhooks](#managing-webhooks) + - [Pools](#pools) + - [Creating a runner pool](#creating-a-runner-pool) + - [Listing pools](#listing-pools) + - [Showing pool info](#showing-pool-info) + - [Deleting a pool](#deleting-a-pool) + - [Update a pool](#update-a-pool) + - [Runners](#runners) + - [Listing runners](#listing-runners) + - [Showing runner info](#showing-runner-info) + - [Deleting a runner](#deleting-a-runner) - [The debug-log command](#the-debug-log-command) - [Listing recorded jobs](#listing-recorded-jobs) -## Listing controller info +## Controller operations + +The `controller` is essentially GARM itself. Every deployment of GARM will have its own controller ID which will be used to tag runners in github. The controller is responsible for managing runners, webhooks, repositories, organizations and enterprises. There are a few settings at the controller level which you can tweak and we will cover them below. + +### Listing controller info You can list the controller info by running the following command: ```bash -garm-cli controller-info show +garm-cli controller show +------------------------+----------------------------------------------------------------------------+ | FIELD | VALUE | +------------------------+----------------------------------------------------------------------------+ @@ -51,17 +72,46 @@ There are several things of interest in this output. * `Controller ID` - This is the unique identifier of the controller. Each GARM installation, on first run will automatically generate a unique controller ID. This is important for several reasons. For one, it allows us to run several GARM controllers on the same repos/orgs/enterprises, without accidentally clasing with each other. Each runner started by a GARM controller, will be tagged with this controller ID in order to easily identify runners that we manage. * `Hostname` - This is the hostname of the machine where GARM is running. This is purely informative. -* `Metadata URL` - This URL is configured by the user in the GARM config file, and is the URL that is presented to the runners via userdata when they get set up. Runners will connect to this URL and retrieve information they might need to set themselves up. GARM cannot automatically determine this URL, as it is dependent on the user's network setup. GARM may be hidden behind a load balancer or a reverse proxy, in which case, the URL by which the GARM controller can be accessed may be different than the IP addresses that are locally visible to GARM. -* `Callback URL` - This URL is configured by the user in the GARM config file, and is the URL that is presented to the runners via userdata when they get set up. Runners will connect to this URL and send status updates and system information (OS version, OS name, github runner agent ID, etc) to the controller. -* `Webhook Base URL` - This is the base URL for webhooks. It is configured by the user in the GARM config file. This URL can be called into by GitHub itself when hooks get triggered by a workflow. GARM needs to know when a new job is started in order to schedule the createion of a new runner. Job webhooks sent to this URL will be recorded by GARM and acter upon. While you can configure this URL directly in your GitHub repo settings, it is advised to use the `Controller Webhook URL` instead, as it is unique to each controller, and allows you to potentially install multiple GARM controller inside the same repo. -* `Controller Webhook URL` - This is the URL that GitHub will call into when a webhook is triggered. This URL is unique to each GARM controller and is the preferred URL to use in order to receive webhooks from GitHub. It serves the same purpose as the `Webhook Base URL`, but is unique to each controller, allowing you to potentially install multiple GARM controllers inside the same repo. +* `Metadata URL` - This URL is configured by the user, and is the URL that is presented to the runners via userdata when they get set up. Runners will connect to this URL and retrieve information they might need to set themselves up. GARM cannot automatically determine this URL, as it is dependent on the user's network setup. GARM may be hidden behind a load balancer or a reverse proxy, in which case, the URL by which the GARM controller can be accessed may be different than the IP addresses that are locally visible to GARM. Runners must be able to connect to this URL. +* `Callback URL` - This URL is configured by the user, and is the URL that is presented to the runners via userdata when they get set up. Runners will connect to this URL and send status updates and system information (OS version, OS name, github runner agent ID, etc) to the controller. Runners must be able to connect to this URL. +* `Webhook Base URL` - This is the base URL for webhooks. It is configured by the user in the GARM config file. This URL can be called into by GitHub itself when hooks get triggered by a workflow. GARM needs to know when a new job is started in order to schedule the createion of a new runner. Job webhooks sent to this URL will be recorded by GARM and acter upon. While you can configure this URL directly in your GitHub repo settings, it is advised to use the `Controller Webhook URL` instead, as it is unique to each controller, and allows you to potentially install multiple GARM controller inside the same repo. Github must be able to connect to this URL. +* `Controller Webhook URL` - This is the URL that GitHub will call into when a webhook is triggered. This URL is unique to each GARM controller and is the preferred URL to use in order to receive webhooks from GitHub. It serves the same purpose as the `Webhook Base URL`, but is unique to each controller, allowing you to potentially install multiple GARM controllers inside the same repo. Github must be able to connect to this URL. We will see the `Controller Webhook URL` later when we set up the GitHub repo to send webhooks to GARM. -## Listing configured providers +### Updating controller settings + +Like we've mentioned before, there are 3 URLs that are very important for normal operations: + +* `metadata_url` - Must be reachable by runners +* `callback_url` - Must be reachable by runners +* `webhook_url` - Must be reachable by GitHub + +These URLs depend heavily on how GARM was set up and what the network topology of the user is set up. GARM may be behind a NAT or reverse proxy. There may be different hostnames/URL paths set up for each of the above, etc. The short of it is that we cannot determine these URLs reliably and we must ask the user to tell GARM what they are. + +We can assume that the URL that the user logs in at to manage garm is the same URL that the rest of the URLs are present at, but that is just an assumption. By default, when you initialize GARM for the first time, we make this assumption to make things easy. It's also safe to assume that most users will do this anyway, but in case you don't, you will need to update the URLs in the controller and tell GARM what they are. + +In the previous section we saw that most URLs were set to `https://garm.example.com`. The URL path was the same as the routes that GARM sets up. For example, the `metadata_url` has `/api/v1/metadata`. The `callback_url` has `/api/v1/callbacks` and the `webhook_url` has `/webhooks`. This is the default setup and is what most users will use. + +If you need to update these URLs, you can use the following command: + +```bash +garm-cli controller update \ + --metadata-url https://garm.example.com/api/v1/metadata \ + --callback-url https://garm.example.com/api/v1/callbacks \ + --webhook-url https://garm.example.com/webhooks +``` + +The `Controller Webhook URL` you saw in the previous section is automatically calculated by GARM and is essentially the `webhook_url` with the controller ID appended to it. This URL is unique to each controller and is the preferred URL to use in order to receive webhooks from GitHub. + +After updating the URLs, make sure that they are properly routed to the appropriate API endpoint in GARM **and** that they are accessible by the interested parties (runners or github). + +## Providers GARM uses providers to create runners. These providers are external executables that GARM calls into to create runners in a particular IaaS. +### Listing configured providers + Once configured (see [provider configuration](/doc/providers.md)), you can list the configured providers by running the following command: ```bash @@ -87,124 +137,237 @@ ubuntu@garm:~$ garm-cli provider list Each of these providers can be used to set up a runner pool for a repository, organization or enterprise. -## Listing github credentials +## Github Endpoints -GARM needs access to your GitHub repositories, organizations or enterprise in order to manage runners. This is done via a [GitHub personal access token](/doc/github_credentials.md). You can configure multiple tokens with access to various repositories, organizations or enterprises, either on GitHub or on GitHub Enterprise Server. +GARM can be used to manage runners for repos, orgs and enterprises hosted on `github.com` or on a GitHub Enterprise Server. -The credentials sections allow you to override the API URL, Upload URL and base URLs, unlocking the ability to use GARM with GitHub Enterprise Server. +Endpoints are the way that GARM identifies where the credentials and entities you create, are located and where the API endpoints for the GitHub API can be reached, along with a possible CA certificate that validates the connection. There is a default endpoint for `github.com`, so you don't need to add it. But if you're using GHES, you'll need to add an endpoint for it. -To list existing credentials, run the following command: +### Creating a GitHub Endpoint + +To create a GitHub endpoint, you can run the following command: ```bash -ubuntu@garm:~$ garm-cli credentials list -+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+ -| NAME | DESCRIPTION | BASE URL | API URL | UPLOAD URL | -+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+ -| gabriel | github token or user gabriel | https://github.com | https://api.github.com/ | https://uploads.github.com/ | -+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+ -| gabriel_org | github token with org level access | https://github.com | https://api.github.com/ | https://uploads.github.com/ | -+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+ +garm-cli github endpoint create \ + --base-url https://ghes.example.com \ + --api-base-url https://api.ghes.example.com \ + --upload-url https://upload.ghes.example.com \ + --ca-cert-path $HOME/ca-cert.pem \ + --name example \ + --description "Just an example ghes endpoint" ++----------------+------------------------------------------------------------------+ +| FIELD | VALUE | ++----------------+------------------------------------------------------------------+ +| Name | example | +| Base URL | https://ghes.example.com | +| Upload URL | https://upload.ghes.example.com | +| API Base URL | https://api.ghes.example.com | +| CA Cert Bundle | -----BEGIN CERTIFICATE----- | +| | MIICBzCCAY6gAwIBAgIQX7fEm3dxkTeSc+E1uTFuczAKBggqhkjOPQQDAzA2MRkw | +| | FwYDVQQKExBHQVJNIGludGVybmFsIENBMRkwFwYDVQQDExBHQVJNIGludGVybmFs | +| | IENBMB4XDTIzMDIyNTE4MzE0NloXDTMzMDIyMjE4MzE0NlowNjEZMBcGA1UEChMQ | +| | R0FSTSBpbnRlcm5hbCBDQTEZMBcGA1UEAxMQR0FSTSBpbnRlcm5hbCBDQTB2MBAG | +| | ByqGSM49AgEGBSuBBAAiA2IABKat241Jzvkl+ksDuPq5jFf9wb5/l54NbGYYfcrs | +| | 4d9/sNXtPP1y8pM61hs+hCltN9UEwtxqr48q5G7Oc3IjH/dddzJTDC2bLcpwysrC | +| | NYLGtSfNj+o/8AQMwwclAY7t4KNhMF8wDgYDVR0PAQH/BAQDAgIEMB0GA1UdJQQW | +| | MBQGCCsGAQUFBwMCBggrBgEFBQcDATAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQW | +| | BBSY+cSG07sIU2UC+fOniODKUGqiUTAKBggqhkjOPQQDAwNnADBkAjBcFz3cZ7vO | +| | IFVzqn9eqXMmZDGp58HGneHhFhJsJtQE4BkxGQmgZJ2OgTGXDqjXG3wCMGMQRALt | +| | JxwlI1PJJj7M0g48viS4NjT4kq2t/UFIbTy78aarFynUfykpL9FD9NOmiQ== | +| | -----END CERTIFICATE----- | +| | | ++----------------+------------------------------------------------------------------+ ``` -These credentials are configured in the GARM config file. You can add, remove or modify them as needed. When using GitHub, you don't need to explicitly set the API URL, Upload URL or base URL, as they are automatically set to the GitHub defaults. When using GitHub Enterprise Server, you will need to set these URLs explicitly. See the [github credentials](/doc/github_credentials.md) section for more details. +The name of the endpoint needs to be unique within GARM. -## Adding a new repository +### Listing GitHub Endpoints -To add a new repository we need to use credentials that has access to the repository. We've listed credentials above, so let's add our first repository: +To list existing GitHub endpoints, run the following command: ```bash -ubuntu@garm:~$ garm-cli repository add \ - --name garm \ - --owner gabriel-samfira \ - --credentials gabriel \ - --install-webhook \ - --random-webhook-secret -+----------------------+--------------------------------------+ -| FIELD | VALUE | -+----------------------+--------------------------------------+ -| ID | be3a0673-56af-4395-9ebf-4521fea67567 | -| Owner | gabriel-samfira | -| Name | garm | -| Credentials | gabriel | -| Pool manager running | true | -+----------------------+--------------------------------------+ +garm-cli github endpoint list ++------------+--------------------------+-------------------------------+ +| NAME | BASE URL | DESCRIPTION | ++------------+--------------------------+-------------------------------+ +| github.com | https://github.com | The github.com endpoint | ++------------+--------------------------+-------------------------------+ +| example | https://ghes.example.com | Just an example ghes endpoint | ++------------+--------------------------+-------------------------------+ ``` -Lets break down the command a bit and explain what happened above. We added a new repository to GARM, that belogs to the user `gabriel-samfira` and is called `garm`. When using GitHub, this translates to `https://github.com/gabriel-samfira/garm`. +### Getting information about an endpoint -As part of the above command, we used the credentials called `gabriel` to authenticate to GitHub. If those credentials didn't have access to the repository, we would have received an error when adding the repo. +To get information about a specific endpoint, you can run the following command: -The other interesting bit about the above command is that we automatically added the `webhook` to the repository and generated a secure random secret to authenticate the webhooks that come in from GitHub for this new repo. Any webhook claiming to be for the `gabriel-samfira/garm` repo, will be validated against the secret that was generated. +```bash +garm-cli github endpoint show github.com ++--------------+-----------------------------+ +| FIELD | VALUE | ++--------------+-----------------------------+ +| Name | github.com | +| Base URL | https://github.com | +| Upload URL | https://uploads.github.com/ | +| API Base URL | https://api.github.com/ | ++--------------+-----------------------------+ +``` + +### Deleting a GitHub Endpoint + +You can delete an endpoint unless one of the following conditions is met: -### Managing repository webhooks +* The endpoint is the default endpoint for `github.com` +* The endpoint is in use by a repository, organization or enterprise +* There are credentials defined against the endpoint you are trying to remove -The `webhook` URL that was used, will correspond to the `Controller Webhook URL` that we saw earlier when we listed the controller info. Let's list it and see what it looks like: +To delete an endpoint, you can run the following command: ```bash -ubuntu@garm:~$ garm-cli repository webhook show be3a0673-56af-4395-9ebf-4521fea67567 -+--------------+----------------------------------------------------------------------------+ -| FIELD | VALUE | -+--------------+----------------------------------------------------------------------------+ -| ID | 460257636 | -| URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 | -| Events | [workflow_job] | -| Active | true | -| Insecure SSL | false | -+--------------+----------------------------------------------------------------------------+ +garm-cli github endpoint delete example ``` -We can see that it's active, and the events to which it subscribed. +## GitHub credentials -The `--install-webhook` and `--random-webhook-secret` options are convenience options that allow you to quickly add a new repository to GARM and have it ready to receive webhooks from GitHub. If you don't want to install the webhook, you can add the repository without it, and then install it later using the `garm-cli repository webhook install` command (which we'll show in a second) or manually add it in the GitHub UI. +GARM needs access to your GitHub repositories, organizations or enterprise in order to manage runners. This is done via a [GitHub personal access token or via a GitHub App](/doc/github_credentials.md). You can configure multiple tokens or apps with access to various repositories, organizations or enterprises, either on GitHub or on GitHub Enterprise Server. -To uninstall a webhook from a repository, you can use the following command: +### Adding GitHub credentials + +There are two types of credentials: + +* PAT - Personal Access Token +* App - GitHub App + +To add each of these types of credentials requires slightly different command line arguments (obviously). I'm going to give you an example of both. + +To add a PAT, you can run the following command: ```bash -garm-cli repository webhook uninstall be3a0673-56af-4395-9ebf-4521fea67567 +garm-cli github credentials add \ + --name deleteme \ + --description "just a test" \ + --auth-type pat \ + --pat-oauth-token gh_yourTokenGoesHere \ + --endpoint github.com ``` -After which listing the webhook will show that it's inactive: +To add a GitHub App (only available for repos and orgs), you can run the following command: ```bash -ubuntu@garm:~$ garm-cli repository webhook show be3a0673-56af-4395-9ebf-4521fea67567 -Error: [GET /repositories/{repoID}/webhook][404] GetRepoWebhookInfo default {Error:Not Found Details:hook not found} +garm-cli github credentials add \ + --name deleteme-app \ + --description "just a test" \ + --endpoint github.com \ + --auth-type app \ + --app-id 1 \ + --app-installation-id 99 \ + --private-key-path /etc/garm/yiourGarmAppKey.2024-12-12.private-key.pem ``` -You can always add it back using: +Notice that in both cases we specified the github endpoint for which these credentials are valid. + +### Listing GitHub credentials + +To list existing credentials, run the following command: ```bash -ubuntu@garm:~$ garm-cli repository webhook install be3a0673-56af-4395-9ebf-4521fea67567 -+--------------+----------------------------------------------------------------------------+ -| FIELD | VALUE | -+--------------+----------------------------------------------------------------------------+ -| ID | 460258767 | -| URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 | -| Events | [workflow_job] | -| Active | true | -| Insecure SSL | false | -+--------------+----------------------------------------------------------------------------+ +ubuntu@garm:~$ garm-cli github credentials ls ++----+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+------+ +| ID | NAME | DESCRIPTION | BASE URL | API URL | UPLOAD URL | TYPE | ++----+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+------+ +| 1 | gabriel | github token or user gabriel | https://github.com | https://api.github.com/ | https://uploads.github.com/ | pat | ++----+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+------+ +| 2 | gabriel_org | github token with org level access | https://github.com | https://api.github.com/ | https://uploads.github.com/ | app | ++----+-------------+------------------------------------+--------------------+-------------------------+-----------------------------+------+ ``` -To allow GARM to manage webhooks, the PAT you're using must have the `admin:repo_hook` and `admin:org_hook` scopes. Webhook management is not available for enterprises. For enterprises you will have to add the webhook manually. +For more information about credentials, see the [github credentials](/doc/github_credentials.md) section for more details. -To manually add a webhook, see the [webhooks](/doc/webhooks.md) section. +### Getting detailed information about credentials + +To get detailed information about one specific credential, you can run the following command: -## Listing repositories +```bash +garm-cli github credentials show 2 ++---------------+------------------------------------+ +| FIELD | VALUE | ++---------------+------------------------------------+ +| ID | 2 | +| Name | gabriel_org | +| Description | github token with org level access | +| Base URL | https://github.com | +| API URL | https://api.github.com/ | +| Upload URL | https://uploads.github.com/ | +| Type | app | +| Endpoint | github.com | +| | | +| Repositories | gsamfira/garm-testing | +| | | +| Organizations | gsamfira | ++---------------+------------------------------------+ +``` + +### Deleting GitHub credentials + +To delete a credential, you can run the following command: + +```bash +garm-cli github credentials delete 2 +``` + +Note, you may not delete credentials that are currently associated with a repository, organization or enterprise. You will need to first replace the credentials on the entity, and then you can delete the credentials. + +## Repositories + +### Adding a new repository + +To add a new repository we need to use credentials that has access to the repository. We've listed credentials above, so let's add our first repository: + +```bash +ubuntu@garm:~$ garm-cli repository add \ + --name garm \ + --owner gabriel-samfira \ + --credentials gabriel \ + --install-webhook \ + --pool-balancer-type roundrobin \ + --random-webhook-secret ++----------------------+--------------------------------------+ +| FIELD | VALUE | ++----------------------+--------------------------------------+ +| ID | 0c91d9fd-2417-45d4-883c-05daeeaa8272 | +| Owner | gabriel-samfira | +| Name | garm | +| Pool balancer type | roundrobin | +| Credentials | gabriel | +| Pool manager running | true | ++----------------------+--------------------------------------+ +``` + +Lets break down the command a bit and explain what happened above. We added a new repository to GARM, that belogs to the user `gabriel-samfira` and is called `garm`. When using GitHub, this translates to `https://github.com/gabriel-samfira/garm`. + +As part of the above command, we used the credentials called `gabriel` to authenticate to GitHub. If those credentials didn't have access to the repository, we would have received an error when adding the repo. + +The other interesting bit about the above command is that we automatically added the `webhook` to the repository and generated a secure random secret to authenticate the webhooks that come in from GitHub for this new repo. Any webhook claiming to be for the `gabriel-samfira/garm` repo, will be validated against the secret that was generated. + +Another important aspect to remember is that once the entity (in this case a repository) is created, the credentials associated with the repo at creation time, dictates the GitHub endpoint in which this repository exists. + +When updating credentials for this entity, the new credentials **must** be associated with the same endpoint as the old ones. An error is returned if the repo is associated with `github.com` but the new credentials you're trying to set are associated with a GHES endpoint. + +### Listing repositories To list existing repositories, run the following command: ```bash ubuntu@garm:~$ garm-cli repository list -+--------------------------------------+-----------------+---------+------------------+------------------+ -| ID | OWNER | NAME | CREDENTIALS NAME | POOL MGR RUNNING | -+--------------------------------------+-----------------+---------+------------------+------------------+ -| be3a0673-56af-4395-9ebf-4521fea67567 | gabriel-samfira | garm | gabriel | true | -+--------------------------------------+-----------------+---------+------------------+------------------+ ++--------------------------------------+-----------------+--------------+------------------+--------------------+------------------+ +| ID | OWNER | NAME | CREDENTIALS NAME | POOL BALANCER TYPE | POOL MGR RUNNING | ++--------------------------------------+-----------------+--------------+------------------+--------------------+------------------+ +| be3a0673-56af-4395-9ebf-4521fea67567 | gabriel-samfira | garm | gabriel | roundrobin | true | ++--------------------------------------+-----------------+--------------+------------------+--------------------+------------------+ ``` This will list all the repositories that GARM is currently managing. -## Removing a repository +### Removing a repository To remove a repository, you can use the following command: @@ -216,7 +379,9 @@ This will remove the repository from GARM, and if a webhook was installed, will Note: GARM will not remove a webhook that points to the `Base Webhook URL`. It will only remove webhooks that are namespaced to the running controller. -## Adding a new organization +## Organizations + +### Adding a new organization Adding a new organization is similar to adding a new repository. You need to use credentials that have access to the organization, and you can add the organization to GARM using the following command: @@ -242,7 +407,9 @@ Managing webhooks for organizations is similar to managing webhooks for reposito All the other operations that exist on repositories, like listing, removing, etc, also exist for organizations and enterprises. Have a look at the help for the `garm-cli organization` subcommand for more details. -## Adding an enterprise +## Enterprises + +### Adding an enterprise Enterprises are a bit special. Currently we don't support managing webhooks for enterprises, mainly because the level of access that would be required to do so seems a bit too much to enable in GARM itself. And considering that you'll probably ever only have one enterprise with multiple organizations and repositories, the effort/risk to benefit ratio makes this feature not worth implementing at the moment. @@ -263,7 +430,66 @@ All the other operations that exist on repositories, like listing, removing, etc At that point the enterprise will be added to GARM and you can start managing runners for it. -## Creating a runner pool +## Managing webhooks + +Webhook management is available for repositories and organizations. I'm going to show you how to manage webhooks for a repository, but the same commands apply for organizations. See `--help` for more details. + +When we added the repository in the previous section, we specified the `--install-webhook` and the `--random-webhook-secret` options. These two options automatically added a webhook to the repository and generated a random secret for it. The `webhook` URL that was used, will correspond to the `Controller Webhook URL` that we saw earlier when we listed the controller info. Let's list it and see what it looks like: + +```bash +ubuntu@garm:~$ garm-cli repository webhook show be3a0673-56af-4395-9ebf-4521fea67567 ++--------------+----------------------------------------------------------------------------+ +| FIELD | VALUE | ++--------------+----------------------------------------------------------------------------+ +| ID | 460257636 | +| URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 | +| Events | [workflow_job] | +| Active | true | +| Insecure SSL | false | ++--------------+----------------------------------------------------------------------------+ +``` + +We can see that it's active, and the events to which it subscribed. + +The `--install-webhook` and `--random-webhook-secret` options are convenience options that allow you to quickly add a new repository to GARM and have it ready to receive webhooks from GitHub. As long as you configured the URLs correctly (see previous sections for details), you should see a green checkmark in the GitHub settings page, under `Webhooks`. + +If you don't want to install the webhook, you can add the repository without it, and then install it later using the `garm-cli repository webhook install` command (which we'll show in a second) or manually add it in the GitHub UI. + +To uninstall a webhook from a repository, you can use the following command: + +```bash +garm-cli repository webhook uninstall be3a0673-56af-4395-9ebf-4521fea67567 +``` + +After which listing the webhook will show that it's inactive: + +```bash +ubuntu@garm:~$ garm-cli repository webhook show be3a0673-56af-4395-9ebf-4521fea67567 +Error: [GET /repositories/{repoID}/webhook][404] GetRepoWebhookInfo default {Error:Not Found Details:hook not found} +``` + +You can always add it back using: + +```bash +ubuntu@garm:~$ garm-cli repository webhook install be3a0673-56af-4395-9ebf-4521fea67567 ++--------------+----------------------------------------------------------------------------+ +| FIELD | VALUE | ++--------------+----------------------------------------------------------------------------+ +| ID | 460258767 | +| URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 | +| Events | [workflow_job] | +| Active | true | +| Insecure SSL | false | ++--------------+----------------------------------------------------------------------------+ +``` + +To allow GARM to manage webhooks, the PAT or app you're using must have the `admin:repo_hook` and `admin:org_hook` scopes (or equivalent). Webhook management is not available for enterprises. For enterprises you will have to add the webhook manually. + +To manually add a webhook, see the [webhooks](/doc/webhooks.md) section. + +## Pools + +### Creating a runner pool Now that we have a repository, organization or enterprise added to GARM, we can create a runner pool for it. A runner pool is a collection of runners of the same type, that are managed by GARM and are used to run workflows for the repository, organization or enterprise. @@ -322,7 +548,7 @@ ubuntu@garm:~$ garm-cli runner list 9daa34aa-a08a-4f29-a782-f54950d8521a +----+------+--------+---------------+---------+ ``` -## Listing pools +### Listing pools To list pools created for a repository you can run: @@ -337,7 +563,20 @@ ubuntu@garm:~$ garm-cli pool list --repo=be3a0673-56af-4395-9ebf-4521fea67567 If you want to list pools for an organization or enterprise, you can use the `--org` or `--enterprise` options respectively. -## Showing pool info +You can also list **all** pools from all configureg github entities by using the `--all` option. + +```bash +ubuntu@garm:~/garm$ garm-cli pool list --all ++--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+ +| ID | IMAGE | FLAVOR | TAGS | BELONGS TO | LEVEL | ENABLED | RUNNER PREFIX | PRIORITY | ++--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+ +| 8935f6a6-f20f-4220-8fa9-9075e7bd7741 | windows_2022 | c3.small.x86 | self-hosted x64 Windows windows equinix | gsamfira/scripts | repo | false | garm | 0 | ++--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+ +| 9233b3f5-2ccf-4689-8f86-a8a0d656dbeb | runner-upstream:latest | small | self-hosted x64 Linux k8s org | gsamfira | org | false | garm | 0 | ++--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+ +``` + +### Showing pool info You can get detailed information about a pool by running the following command: @@ -365,7 +604,7 @@ ubuntu@garm:~$ garm-cli pool show 9daa34aa-a08a-4f29-a782-f54950d8521a +--------------------------+----------------------------------------+ ``` -## Deleting a pool +### Deleting a pool In order to delete a pool, you must first make sure there are no runners in the pool. To ensure this, we can first disable the pool, to make sure no new runners are created, remove the runners or allow them to be user, then we can delete the pool. @@ -401,7 +640,7 @@ If there are no runners in the pool, you can then remove it: ubuntu@garm:~$ garm-cli pool delete 9daa34aa-a08a-4f29-a782-f54950d8521a ``` -## Update a pool +### Update a pool You can update a pool by using the `garm-cli pool update` command. Nearly every aspect of a pool can be updated after it has been created. To demonstrate the command, we can enable the pool we created earlier: @@ -429,6 +668,8 @@ ubuntu@garm:~$ garm-cli pool update 9daa34aa-a08a-4f29-a782-f54950d8521a --enabl +--------------------------+----------------------------------------+ ``` +See `garm-cli pool update --help` for a list of settings that can be changed. + Now that the pool is enabled, GARM will start creating runners for it. We can list the runners in the pool to see if any have been created: ```bash @@ -453,7 +694,9 @@ root@incus:~# incus list Awesome! This runner will be able to pick up bobs that match the labels we've set on the pool. -## Listing runners +## Runners + +### Listing runners You can list runners for a pool, for a repository, organization or enterprise, or for all of them. To list all runners, you can run: @@ -474,7 +717,7 @@ ubuntu@garm:~$ garm-cli runner list --all Have a look at the help command for the flags available to the `list` subcommand. -## Showing runner info +### Showing runner info You can get detailed information about a runner by running the following command: @@ -508,7 +751,7 @@ ubuntu@garm:~$ garm-cli runner show garm-BFrp51VoVBCO +-----------------+------------------------------------------------------------------------------------------------------+ ``` -## Deleting a runner +### Deleting a runner You can delete a runner by running the following command: