Skip to content

Commit

Permalink
Add Proposal 2: Run
Browse files Browse the repository at this point in the history
Co-authored-by: locomundo <flavia@aknostic.com>
Signed-off-by: nikimanoledaki <niki.manoledaki@grafana.com>
  • Loading branch information
nikimanoledaki and locomundo committed May 15, 2024
1 parent d4f4522 commit 6759913
Showing 1 changed file with 199 additions and 0 deletions.
199 changes: 199 additions & 0 deletions website/content/docs/proposals/proposal-002-run.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
# Run the bechmark tests as part of the automated pipeline

## Authors

- @locomundo
- @nikimanoledaki

## Status

Draft

## Table of Contents
<!-- toc -->
- [Run the bechmark tests as part of the automated pipeline](#run-the-bechmark-tests-as-part-of-the-automated-pipeline)
- [Authors](#authors)
- [Status](#status)
- [Table of Contents](#table-of-contents)
- [Summary](#summary)
- [Motivation](#motivation)
- [Goals](#goals)
- [Non-Goals](#non-goals)
- [Linked Docs](#linked-docs)
- [Proposal](#proposal)
- [User Stories (Optional)](#user-stories-optional)
- [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional)
- [Risks and Mitigations](#risks-and-mitigations)
- [Design Details](#design-details)
- [Graduation Criteria (Optional)](#graduation-criteria-optional)
- [Drawbacks (Optional)](#drawbacks-optional)
- [Alternatives](#alternatives)
- [Infrastructure Needed (Optional)](#infrastructure-needed-optional)
<!-- /toc -->

## Summary

<!--
A good summary is at least a paragraph in length and should be written with a wide audience
in mind.
It should encompass the entire document, and serve as both future documentation
and as a quick reference for people coming by to learn the proposal's purpose
without reading the entire thing.
Both in this section and below, follow the guidelines of the [documentation
style guide]. In particular, wrap lines to a reasonable length, to make it
easier for reviewers to cite specific portions, and to minimize diff churn on
updates.
[documentation style guide]: https://github.com/kubernetes/community/blob/master/contributors/guide/style-guide.md
-->

## Motivation

<!--
This section is for explicitly listing the motivation, goals and non-goals of
this proposal. Describe why the change is important, how it fits into the project's
goals and the benefits to users.
It is helpful to frame this to answer the question: "What is the problem this proposal
is trying to solve?"
-->

This proposal is part of the pipeline automation of the Green Review for Falco. Currently, we are using Flux to watch the upstream Falco repository and run the benchmark tests constantly. For example, [this benchmark test](https://github.com/falcosecurity/cncf-green-review-testing/blob/main/kustomize/falco-driver/ebpf/stress-ng.yaml#L27-L32) is setup as a Kubernetes Deployment that runs an endless loop of [`stress-ng`](https://wiki.ubuntu.com/Kernel/Reference/stress-ng), which applies stress to the kernel. Instead, this proposal aims to provide a solution for how to deploy the benchmark tests only when they are needed.

### Goals

<!--
List the specific goals of the proposal. What is it trying to achieve? How will we
know that this has succeeded?
-->

- Describe the actions to take immediately after the trigger from Proposal 1 https://github.com/cncf-tags/green-reviews-tooling/issues/84
- Describe how the pipeline should _fetch_ the benchmark tests either from this repository (`cncf-tags/green-reviews-tooling`) or from an upstream repository (Falco's [`falcosecurity/cncf-green-review-testing`](https://github.com/falcosecurity/cncf-green-review-testing)).
- Describe how the pipeline should _run_ the benchmark tests through GitHub Actions for a specific project e.g. Falco
- Communicate the changes needed to be made by the Falco team to change the benchmark test to a GitHub Action file.
- Provide _modularity_ for the benchmark tests.

### Non-Goals

<!--
What is out of scope for this proposal? Listing non-goals helps to focus discussion
and make progress.
It is important to remember that non-goals are still equally important things
which will be dealt with one day but are not things which need to be dealt with immediately
within the scope of this work. This helps make sure everyone is crystal clear on the outcomes.
-->

### Linked Docs

<!--
Provide links to previous discussions, Slack threads, motivation issues or any other document
with context. It is really helpful to provide a "source of truth" for the work
so that people aren't searching all over the place for lost context.
-->

## Proposal

<!--
This is where we get down to the specifics of what the proposal actually is:
outlining your solution to the problem described in the Motivation section.
This should have enough detail that reviewers can understand exactly what
you're proposing, but should not include things like API designs or
implementation. The "Design Details" section below is for the real
nitty-gritty.
-->

### User Stories (Optional)

<!--
Detail the things that people will be able to do if this proposal is implemented.
Include as much detail as possible so that people can understand the "how" of
the system. The goal here is to make this feel real for users without getting
bogged down.
-->


* As a CNCF Project Maintainer,

* As a Green Review WG reviewer,

* As a Green Review WG cluster maintainer,

### Notes/Constraints/Caveats (Optional)

<!--
What are the caveats to the proposal?
What are some important details that didn't come across above?
Go in to as much detail as necessary here.
This might be a good place to talk about core concepts and how they relate.
-->

### Risks and Mitigations

<!--
What are the risks of this proposal, and how do we mitigate?
Think broadly. For example, consider how this will impact or be impacted
by scaling to support more CNCF Projects.
How will this affect the benchmark tests, CNCF Project Maintainers, pipeline maintainers, etc?
-->

## Design Details

<!--
This section should contain enough information that the specifics of your
change are understandable. This may include manifests or workflow examples
(though not always required) or even code snippets. If there's any ambiguity
about HOW your proposal will be implemented, this is the place to discuss them.
-->

There is a way to (re)use workflows from other orgs/repos: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_iduses

```
jobs:
call-workflow-in-another-repo:
uses: octo-org/another-repo/.github/workflows/workflow.yml@v1
```

This can also be set to use `@main` or `@another-branch` which would be nice for versioning and testing specific releases.

Compared to self-hosted runners, this reduces the need to authenticate to only one repo - green reviews tooling. By using public runners, we would still need to authenticate to run kubectl e.g. through by storing a kubeconfig as a secret in the repo.

We may also want to consider a starter workflow (creates a template that looks like this) for the top-level workflow.

### Graduation Criteria (Optional)

<!--
List criteria which would allow progression from one maturity level to another.
eg. What needs to have been accomplished/demonstrated to move from Alpha to Beta.
If applicable, what is the milestone marker which will allow deprecation of the
replaced capability?
-->

## Drawbacks (Optional)

<!--
What other approaches did you consider, and why did you rule them out? These do
not need to be as detailed as the proposal, but should include enough
information to express the idea and why it was not acceptable.
-->

## Alternatives

<!--
What other approaches did you consider, and why did you rule them out? These do
not need to be as detailed as the proposal (pros and cons are fine),
but should include enough information to express the idea and why it was not acceptable
as well as illustrate why the final solution was selected.
-->

## Infrastructure Needed (Optional)

<!--
Use this section if you need things from the project/SIG. Examples include a
new subproject, repos requested, or GitHub details. Listing these here allows a
SIG to get the process for these resources started right away.
-->

0 comments on commit 6759913

Please sign in to comment.