Enhance artifact pull and push commands

.gitignore

@@ -8,7 +8,7 @@ input_dockerfile.json
 Dockerfile*
 genval
 cosign
-.todo
+todo.md
 !.devcontainer/Dockerfile
 results.json
 .env

.goreleaser.yaml

@@ -30,7 +30,16 @@ source:
 
 # Adds Changelog to the release
 changelog:
+  use: github
+  format: "{{.SHA}}: {{.Message}} (@{{.AuthorUsername}})"
   sort: asc
+  groups: # Regex use RE2 syntax as defined here: https://github.com/google/re2/wiki/Syntax.
+    - title: 'Updates'
+      regexp: ^(?i).*?update\w*(\([[:word:]]+\))??!?:.+$
+      order: 100
+    - title: 'Bug fixes'
+      regexp: ^(?i).*?fix\w*(\([[:word:]]+\))??!?:.+$
+      order: 200
   filters:
     exclude:
       - "^docs:"

README.md

@@ -127,7 +127,7 @@ If verification is successful, you'll see "**Verified OK.**"
 ## Quick Start
 
 
-For a quick start, pre-built templates for Dockerfile generation for popular languages can be found in the `./templates/inputs/dockerfile_input` folder.
+For a quick start, pre-built templates for Dockerfile generation for popular languages can be found in the `./templates/inputs/dockerfile_input` folder. We also maintain all the default policies and input templates in a dedicated [repository](https://github.com/intelops/policyhub).
 
 
 ## Building from Source
@@ -188,12 +188,52 @@ $ genval dockerfile --reqinput=./templates/inputs/dockerfile_input/golang_input.
 
 All the rego policies are housed in a hirearchy containing a `rego` policy and a `JSON` file containing all the metadata related to policy. A user needs to pass the directory containing boththe `.rego` and `.json` files. Genval also accpets a top leval directory containing multiple sub-directories containing multiple rego and accompanied JSON files for validating with more than one policy.
 
+Users can use policies to validate input JSON as well as generated Dockerfile with policies stored in their OCI registries
+or with Genval's default Rego policies. Behind the scenes, this action iteracts with OCI registries for pulling the policies.
 
+To facilitate authentication with OCI compliant container registries,
+Users can provide credentials through --credentials flag. The creds can be provided via <$USER:$PAT> or <REGISTRY_PAT> format.
+If no credentials are provided, Genval searches for the "./docker/config.json" file in the user's $HOME directory.
+If this file is found, Genval utilizes it for authentication.
+
+**Validating with Default policies**
+
+```shell
+./genval dockerfile --reqinput https://github.com/intelops/genval-security-policies/blob/patch-1/input-templates/dockerfile_input/clang_input.json \
+--output ./output/Dockefile-cobra
+// No credntials provided, will default to $HOME/.docker/config.json for credentials
+```
+
+**Validating with policies stored in OCI compliant container registries**
+
+```shell
+./genval dockerfile --reqinput https://github.com/intelops/genval-security-policies/blob/patch-1/input-templates/dockerfile_input/clang_input.json \
+--output ./output/Dockefile-cobra \
+inputpolicy oci://ghcr.io/intelops/policyhub/genval/input_policies:v0.0.1 \
+--outputpolicy oci://ghcr.io/intelops/policyhub/genval/dockerfile_policies:v0.0.1 \
+--credentials <$GITHUB_PAT> or <$USER:$PAT> format
+```
 **Review Feedback**: Genval provides feedback based on best practice validation. Use this feedback to refine your Dockerfile.
 
 ### Validation of the Dockerfile, Kubernetes manifests and Terraform files using Rego policies
 
-Genval manages validation with Rego polcies with `regoval` command and for validation of each of the technology a separate subcommand is provided:
+Users can leverage Genval's feature of Validating of resources using policies stored in OCI compliant registries or provide policies stored in their own OCI compliant registries.
+
+To facilitate authentication with OCI compliant container registries, Users can provide credentials through `--credentials` flag while invoking a regoval subcommand. The credentials can
+be provided via <$USER:$PAT> or <$REGISTRY_PAT> format. If no credentials are provided, Genval searches for the `./docker/config.json` file in the user's `$HOME` directory. If this file is found, Genval utilizes it for authentication.
+
+**Example**:
+
+./genval regoval dockerfileval --reqinput=Dockerfile \
+--policy oci://ghcr.io/intelops/policyhub/genval/dockerfile_policies:v0.0.1
+--credentials <GITHUB_PAT> or <USER:PAT>
+
+
+Users can you use default policies maintained by the community stored in the https://github.com/intelops/policyhub repo
+
+./genval regoval dockerfileval --reqinput <Path to Dockerfile>
+// No credntials provided, will default to $HOME/.docker/config.json for credentials
+
 
 #### Validation of Dockerfiles with Rego policies
 

cmd/artifact_pull.go

@@ -22,10 +22,10 @@ The artifact pull command pull the artifact stored in the remote container regis
 If --verify flag is set to true (false by default), Genval will verify the artifact's signature
 which is signed by Cosign in Keyless mode and pull the artifact, and unpack the tar.gz archive in desired path.
 
-To facilitate authentication with container registries, Genval initially searches for the "./docker/config.json"
-file in the user's $HOME directory. If this file is found, Genval utilizes it for authentication.
-However, if the file is not present, users must set the ARTIFACT_REGISTRY_USERNAME and ARTIFACT_REGISTRY_PASSWORD
-environment variables to authenticate with the container registry.
+To facilitate authentication with OCI compliant container registries,
+Users can provide credentials through --creds flag. The creds can be provided via <USER:PAT> or <REGISTRY_PAT> format.
+If no credentials are provided, Genval searches for the "./docker/config.json" file in the user's $HOME directory.
+If this file is found, Genval utilizes it for authentication.
 
 
 `,
@@ -45,12 +45,14 @@ environment variables to authenticate with the container registry.
 --path ./output \
 --verify true \
 --pub-key ./cosign/cosign.pub
+--credentials <GITHUB_PAT> or <USER:PAT>
 
 # Uses can also pull the artifact with verifying the signatures of the artifact
   in the container registry and unpack the archive in desired path
 
 ./genval artifact pull --dest ghcr.io/santoshkal/artifacts/genval:test \
 --path ./output
+// No credentials provided, will default to $HOME/.docker/config.json for credentials
 `,
 	RunE: runPullArtifactCmd,
 }
@@ -103,7 +105,7 @@ func runPullArtifactCmd(cmd *cobra.Command, args []string) error {
 	spin := utils.StartSpinner("Pulling Artifact...")
 	defer spin.Stop()
 
-	if err := oci.PullArtifact(context.Background(), pullArgs.dest, pullArgs.path); err != nil {
+	if err := oci.PullArtifact(context.Background(), pullArgs.creds, pullArgs.dest, pullArgs.path); err != nil {
 		color.Red("Error pulling artifact from remote : %v", err)
 		return err
 	}

cmd/artifact_push.go

@@ -6,12 +6,13 @@ import (
 	"path/filepath"
 	"time"
 
+	"github.com/google/go-containerregistry/pkg/authn"
 	"github.com/google/go-containerregistry/pkg/compression"
 	"github.com/google/go-containerregistry/pkg/crane"
-	"github.com/google/go-containerregistry/pkg/name"
 	v1 "github.com/google/go-containerregistry/pkg/v1"
 	"github.com/google/go-containerregistry/pkg/v1/empty"
 	"github.com/google/go-containerregistry/pkg/v1/mutate"
+	"github.com/google/go-containerregistry/pkg/v1/remote/transport"
 	"github.com/google/go-containerregistry/pkg/v1/tarball"
 	"github.com/google/go-containerregistry/pkg/v1/types"
 	"github.com/intelops/genval/pkg/oci"
@@ -27,10 +28,10 @@ var pushCmd = &cobra.Command{
 The artifact push command takes in a tar.gz bundle of configuration files generated/validated by Genval
 and pushes them into a OCI complient container registry
 
-To facilitate authentication with container registries, Genval initially searches for the "./docker/config.json"
-file in the user's $HOME directory. If this file is found, Genval utilizes it for authentication.
-However, if the file is not present, users must set the ARTIFACT_REGISTRY_USERNAME and ARTIFACT_REGISTRY_PASSWORD
-environment variables to authenticate with the container registry.
+To facilitate authentication with OCI compliant container registries,
+Users can provide credentials through --creds flag. The creds can be provided via <USER:PAT> or <REGISTRY_PAT> format.
+If no credentials are provided, Genval searches for the "./docker/config.json" file in the user's $HOME directory.
+If this file is found, Genval utilizes it for authentication.
 `,
 	Example: `
 # Build and push the provided file/files as OCI artifact to registry
@@ -42,13 +43,15 @@ environment variables to authenticate with the container registry.
 ./genval artifact push --reqinput ./templates/defaultpolicies/rego \
 --dest ghcr.io/santoshkal/artifacts/genval:test \
 --sign true
+// No credentials provided, will default to $HOME/.docker/config.json for credentials
 
 # Alternatively, users may provide the Cosign generated private key for signing the artifact
 
 ./genval artifact push --reqinput ./templates/defaultpolicies/rego \
 --dest ghcr.io/santoshkal/artifacts/genval:test \
 --sign true
 --cosign-key <Path to Cosign private Key>
+--credentials <GITHUB_PAT> or <USER:PAT>
 
 # User can pass additional annotations in <key=value> pair while pushing the artifact
 
@@ -114,21 +117,20 @@ func runPushCmd(cmd *cobra.Command, args []string) error {
 	if err := oci.CreateTarball(inputPath, outputPath); err != nil {
 		return fmt.Errorf("creating tarball: %w", err)
 	}
-	log.Println("✔ Artifact created successfully")
+	log.Info("✔ Artifact created successfully")
 
-	ref, err := name.ParseReference(source)
+	ref, err := oci.ParseOCIReference(source)
 	if err != nil {
-		log.Printf("Error parsing source: %v", err)
+		log.Errorf("Error parsing source: %v", err)
 	}
 
 	remoteURL, err := oci.GetRemoteURL()
-	fmt.Printf("Remote Name: %v", remoteURL)
 	if err != nil {
-		return err
+		return fmt.Errorf("error fetching credentials: %v", err)
 	}
 	annotations, err := oci.ParseAnnotations(pushArgs.annotations)
 	if err != nil {
-		return err
+		return fmt.Errorf("error parsing annotations %s: %v", pushArgs.annotations, err)
 	}
 
 	img := mutate.MediaType(empty.Image, types.OCIManifestSchema1)
@@ -155,9 +157,17 @@ func runPushCmd(cmd *cobra.Command, args []string) error {
 	}
 	spin := utils.StartSpinner("pushing artifact")
 	defer spin.Stop()
-	opts, err := oci.GetCreds()
+	auth, err := oci.GetCreds(pushArgs.creds)
 	if err != nil {
-		log.Errorf("Error reading credentials: %v", err)
+		return fmt.Errorf("error getting credentials: %v", err)
+	}
+	opts, err := oci.GenerateCraneOptions(cmd.Context(), ref, auth, []string{ref.Context().Scope(transport.PushScope)})
+	if err != nil {
+		log.Errorf("Error creating options required for push: %v", err)
+	}
+	opts = append(opts, crane.WithAuth(auth))
+	if pushArgs.creds == "" {
+		opts = append(opts, crane.WithAuthFromKeychain(authn.DefaultKeychain))
 	}
 
 	if err := crane.Push(img, ref.String(), opts...); err != nil {
@@ -174,11 +184,11 @@ func runPushCmd(cmd *cobra.Command, args []string) error {
 	if pushArgs.sign {
 		err := oci.SignCosign(digestURL, pushArgs.cosignKey)
 		if err != nil {
-			return err
+			return fmt.Errorf("error signing artifact %s: %v", digestURL, err)
 		}
 	}
 
-	log.Printf("✔ Artifact pushed successfully to: %v\n, with Digest: %v\n", source, digest)
-	log.Printf("Digest URL: %v\n", digestURL)
+	log.Infof("✔ Artifact pushed successfully to: %v\n,Artifact Digest: %v\n", source, digest)
+	log.Infof("Digest URL: %v\n", digestURL)
 	return nil
 }

cmd/completion.go

@@ -0,0 +1,86 @@
+/*
+Copyright © 2024 NAME HERE <EMAIL ADDRESS>
+*/
+package cmd
+
+import (
+	"fmt"
+	"os"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/cobra"
+)
+
+var completionCmd = &cobra.Command{
+	Use:   "completion [bash|zsh|fish|powershell]",
+	Short: "Generate completion script",
+	Long: fmt.Sprintf(`To load completions:
+
+Bash:
+
+  $ source <(%[1]s completion bash)
+
+  # To load completions for each session, execute once:
+  # Linux:
+  $ %[1]s completion bash > /etc/bash_completion.d/%[1]s
+  # macOS:
+  $ %[1]s completion bash > $(brew --prefix)/etc/bash_completion.d/%[1]s
+
+Zsh:
+
+  # If shell completion is not already enabled in your environment,
+  # you will need to enable it.  You can execute the following once:
+
+  $ echo "autoload -U compinit; compinit" >> ~/.zshrc
+
+  # To load completions for each session, execute once:
+  $ %[1]s completion zsh > "${fpath[1]}/_%[1]s"
+
+  # You will need to start a new shell for this setup to take effect.
+
+fish:
+
+  $ %[1]s completion fish | source
+
+  # To load completions for each session, execute once:
+  $ %[1]s completion fish > ~/.config/fish/completions/%[1]s.fish
+
+PowerShell:
+
+  PS> %[1]s completion powershell | Out-String | Invoke-Expression
+
+  # To load completions for every new session, run:
+  PS> %[1]s completion powershell > %[1]s.ps1
+  # and source this file from your PowerShell profile.
+`, rootCmd.Name()),
+	DisableFlagsInUseLine: true,
+	ValidArgs:             []string{"bash", "zsh", "fish", "powershell"},
+	Args:                  cobra.MatchAll(cobra.ExactArgs(1), cobra.OnlyValidArgs),
+	Run: func(cmd *cobra.Command, args []string) {
+		if len(args) == 0 {
+			log.Error("No shell specified")
+		}
+		var err error
+		switch args[0] {
+		case "bash":
+			err = rootCmd.GenBashCompletion(os.Stdout)
+		case "zsh":
+			err = rootCmd.GenZshCompletion(os.Stdout)
+		case "fish":
+			err = rootCmd.GenFishCompletion(os.Stdout, true)
+		case "powershell":
+			err = rootCmd.GenPowerShellCompletionWithDesc(os.Stdout)
+		default:
+			log.Printf("Unknown shell: %s", args[0])
+			return
+		}
+		if err != nil {
+			log.Printf("Error generating completion for %s: %v", args[0], err)
+			return
+		}
+	},
+}
+
+func init() {
+	rootCmd.AddCommand(completionCmd)
+}