Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for base and target trees in git ingest, add .tar.gz bundler #5181

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

evankanderson
Copy link
Member

Summary

Prepare for running external tools via Minder rego. This adds two features (sorry, I got on a roll!):

  • git ingest during a pull request can access the PR base branch (what you want to merge to) via base_file.* analogues of file.* operations.
  • Added a file.bundle operation that takes a list of file paths and produces a .tar.gz as a string.

Change Type

Mark the type of change your PR introduces:

  • Bug fix (resolves an issue without affecting existing features)
  • Feature (adds new functionality without breaking changes)
  • Breaking change (may impact existing functionalities or require documentation updates)
  • Documentation (updates or additions to documentation)
  • Refactoring or test improvements (no bug fixes or new functionality)

Testing

Unit testing for lib and some light manual testing for the git ingest.

Review Checklist:

  • Reviewed my own code for quality and clarity.
  • Added comments to complex or tricky code sections.
  • Updated any affected documentation.
  • Included tests that validate the fix or feature.
  • Checked that related changes are merged.

@evankanderson evankanderson requested a review from a team as a code owner December 11, 2024 08:36
@coveralls
Copy link

coveralls commented Dec 11, 2024

Coverage Status

coverage: 55.422% (+0.07%) from 55.352%
when pulling ed1cb1d on evankanderson:file-machinery
into f2d6feb on mindersec:main.

func BaseFileArchive(res *interfaces.Result) func(*rego.Rego) {
return rego.Function1(
&rego.Function{
Name: "base_file.archive",
Copy link
Contributor

@JAORMX JAORMX Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description
says it would be file.bundle. is this naming difference intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talking with Ed, he suggested archive instead.

// a tarball. It takes one argument: a list of file or directory paths to
// include, and outputs the tarball as a string.
// It's exposed as 'base_file.archive`.
func BaseFileArchive(res *interfaces.Result) func(*rego.Rego) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to expose this function to non PR/diff contexts too? I think it would be handy in general.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See FileArchive, above. 😁

// If the branch is not provided in the rule-type
// configuration, use the default branch
return defaultBranch
checkpoint := checkpoints.NewCheckpointV1Now().WithBranch(ent.GetTargetRef()).WithCommitHash(head.Hash().String())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to fix this in this PR... But... I wonder if we should have a base and target checkpoints checksum checkpoints?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since "base" is the branch that the PR will be merged into, I think that only changes if the target/head branch changes.

@@ -57,30 +70,47 @@ func FileExists(res *interfaces.Result) func(*rego.Rego) {
Name: "file.exists",
Decl: types.NewFunction(types.Args(types.S), types.B),
},
func(_ rego.BuiltinContext, op1 *ast.Term) (*ast.Term, error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diff looks really bad, but I just moved this to the return function from fsExists

func BaseFileArchive(res *interfaces.Result) func(*rego.Rego) {
return rego.Function1(
&rego.Function{
Name: "base_file.archive",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talking with Ed, he suggested archive instead.

// a tarball. It takes one argument: a list of file or directory paths to
// include, and outputs the tarball as a string.
// It's exposed as 'base_file.archive`.
func BaseFileArchive(res *interfaces.Result) func(*rego.Rego) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See FileArchive, above. 😁

// If the branch is not provided in the rule-type
// configuration, use the default branch
return defaultBranch
checkpoint := checkpoints.NewCheckpointV1Now().WithBranch(ent.GetTargetRef()).WithCommitHash(head.Hash().String())
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since "base" is the branch that the PR will be merged into, I think that only changes if the target/head branch changes.

JAORMX
JAORMX previously approved these changes Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants