Add option to rabbitmq.conf to pass access token to load definitions from a private remote repository

[F-Joachim]

Is your feature request related to a problem? Please describe.

I would like to load a definition file from a private repository (as described here). For this, the ability to pass an access token would be necessary.

Describe the solution you'd like

Introduce a new property in rabbitmq.conf, so that an access token can be passed, e.g. definitions.https.token

Describe alternatives you've considered

No response

Additional context

No response

[lukebakken]

What exactly does "access token" mean? Are you talking about basic auth, OAuth, etc?

[michaelklishin]

@F-Joachim you haven't provided any details for anyone to be able to implement this feature. You are welcome to do it since you have all the context.

The definition fetching mechanism supports HTTPS, including peer verification and lots of other TLS-related options. The idea has been that network segmentation plus HTTPS with peer verification would be sufficient for most environments.

There's nothing wrong with using JTW tokens but what exactly that means for various environments, I won't guess. We do not guess in this community.

[F-Joachim]

@michaelklishin | @lukebakken - Thanks for your quick replies. Sorry, I was really very imprecise in my description.

With my very vaguely described (former) issue I did not want to trigger an implementation directly, but rather ask you what possibilities there are to access a private repository (e.g. GitHub Enterprise Organization) to load the definitions.json from there. Maybe there is already a possibility and I have just overlooked it.

But let me explain my question based on my use case. We have a private GitHub Enterprise organization in our company for our project. In this organization we want to create a repository for the definitions (json files). At the time of deploying the (dockerized) rabbitmq to our container platform, the rabbitmq.conf with corresponding definitions.* parameterization (where definitions.https.url points to a file of a specific commit) should be passed to the container. Since it is - as mentioned - a private organization, we need a way to pass e.g. a GitHub fine-grained (or classic) token.

I hope that I have now been able to make my concern clearer. Please let me know if you need any further information.

Many thanks in advance.

[michaelklishin]

You can use any file that the node can download using HTTPS with (optionally) enabled peer verification.

Support for an optional Authorization header per GitHub docs can be added but it's going to be a hell of a lot of fun to test (our team heavily relies on integration tests).

[F-Joachim]

I will try it with a reverse proxy, so that the rabbitmq container connects to this proxy container and forwards the request (enriched with the necessary headers) to the GitHub API.

[michaelklishin]

That's a great idea. We'll look into setting the header but I cannot promise any ETA.

[F-Joachim]

I got it to work. For all those who also want to use a rabbitmq definition from a private external repository, here is my Traefik and the minimal rabbitmq.conf configuration to connect to the GitHub API

Note

The setup only contains a self-signed certificate. Please use only trusted certificates in production 😊

traefik.yaml - Static Traefik configuration (via file provider)

entryPoints:
  github:
    address: ":9090"
providers:
  file:
    filename: "/app-config/dynamic/conf/dynamic.yaml"

dynamic.yml - Dynamic Traefik configuration (via file provider)

http:
  routers:
    github-router:
      rule: Host(`traefik`)
      service: github-service
      entryPoints:
        - github
      middlewares:
        - github-headers
      tls: {}
  services:
    github-service:
      loadBalancer:
        servers:
          - url: "https://api.github.com/"
        passHostHeader: false
  middlewares:
    github-headers:
      headers:
        customRequestHeaders:
          Authorization: "Bearer <insert your GitHub token here>"
          Accept: "application/vnd.github.v3.raw"
          User-Agent: "my-awesome-app" # required by github: https://docs.github.com/en/rest/using-the-rest-api/getting-started-with-the-rest-api?apiVersion=2022-11-28#user-agent
tls:
  certificates:
    - certFile: /etc/ssl/local/pubkey.pem
      keyFile: /etc/ssl/local/private.key

Important

User-Agent header is required by GitHub. See here.

rabbitmq.conf

definitions.import_backend = https
definitions.https.url = https://traefik:9090/repos/<OWNER or ORGANIZATION>/<REPO name>/contents/<PATH>[?ref=sha/tag/branch]
definitions.tls.verify = verify_none # Do not use this in production ;-)

Please see the GitHub documentation for more information of how to address your repository content.

compose.yaml

services:
  traefik:
    image: traefik:v3.1.2
    command: --configFile=/app-config/traefik.yaml
    container_name: traefik
    volumes:
      - $PWD/dynamic.yaml:/app-config/dynamic/conf/dynamic.yaml
      - $PWD/traefik.yaml:/app-config/traefik.yaml
      - $PWD/cert.pem:/etc/ssl/local/pubkey.pem
      - $PWD/key.pem:/etc/ssl/local/private.key
    restart: always

  rabbitmq:
    container_name: rabbitmq
    image: rabbitmq:3.13.6-management
    hostname: rabbitmq
    ports:
      - 15672:15672
    volumes:
      - rabbitmq_data:/var/lib/rabbitmq
      - $PWD/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf

volumes:
  rabbitmq_data:

For test purposes you can generate a self-signed certificate as described here, for example, or use tls-gen.

Thanks a lot @lukebakken and @michaelklishin for the discussion on this topic.

Also many thanks to the Traefik Forum for helping me to fine-tune the Traefik configuration.

[lukebakken]

This is awesome! Thank you for being an excellent, above-and-beyond community user of RabbitMQ ⭐

[F-Joachim]

Thanks for the review and enhancement @lukebakken