BCR Provenance attestations

[loosebazooka]

Bazel BCR Provenance

Goals

• Add security metadata (build provenance) when publishing bazel modules on BCR.
• Define a well known format for attestation storage and discovery on BCR that is flexible for both open source and google requirements
• Provide tooling so the "easy path" to publishing on BCR will generate provenance automatically for open source developers.
• Create tooling for verification at the client side (at user module consumption time)

Non-Goals

• Define a policy controller for BCR for checking uploaded attestations
• Provenance verification embedded directly in bazel tooling
• Define what it means to "trust" a SLSA attestation

Background

BCR

The Bazel Central Registry (BCR) is the default and recommended repository for Bazel modules used in the Bzlmod external dependency system. It's a centralized location where you can find and publish your preferred Bazel projects. The BCR website allows you to easily browse available modules.

Layout

From https://docs.bazel.build/versions/5.1.0/bzlmod.html

/modules/$MODULE/$VERSION  
                 ├── MODULE.bazel  
                 ├── source.json  
                 └── patches  
                      ├── 1.patch  
                      ├── …  
                      └── N.patch

Each published module to /modules/$MODULE/$VERSION contains the following files (see https://bazel.build/external/registry for more details):

MODULE.bazel: The bazel module file of this module version. It contains the metadata about this model and its dependencies to other Bazel modules.

source.json: A JSON file containing information on how to fetch the source of this module version, with the following fields:

• url: The URL of the source archive.
• integrity: The Subresource Integrity checksum of the archive.
• strip_prefix: A directory prefix to strip when extracting the source archive.
• patches: The name and digest of all the patch files under the patches/ directory for this module version. The patch files will be applied to the extracted source archive. .
• patch_strip: Same as the --strip argument of Unix patch.

patches/: An optional directory containing patch files

Here is an example push to BCR

SLSA Build Attestations

Build Attestations are signed metadata about how (in this case) bazel modules were built and released to BCR. With build attestations, module consumers (bazel users or package repositories) can make some assertions about the source and packaging process of a module.. While other security measures exist for BCR, currently BCR provides no guidance on signatures or signed attestations over the releases.

For the purposes of this document we will be talking about SLSA provenance, but BCR may accept other types of metadata formats (ex: VSA)

Proposal: Provenance on Module build/release

Our processes for attestation generation must cover the following critical files:

1. Source archive
2. source.json
3. MODULE.bazel

Users may optionally choose to cover all files attached to a module release. However these are not necessary to make security assertions over the module release.

1. patches (all patches are included in source.json)
2. presubmit.yml (not used by module consumers)

Any build/release process for a module must generate provenance attestations over all three objects and any consumer of a module that wants to verify provenance check that all files used are covered by a provenance file.

Build provenance over the source archive

Users should be able to trace back the source (source.json#url) archive to a repository and a trusted packaging system. An attestation over the source archive should contain a single subject (the source archive) and should exist next to the source archive. It may be replicated in other places (like BCR), but should be accessible without BCR.

The source typically exists as a release artifact in the Project repository. This is an archive (.tar/.zip) over the source at a specific commit, generated by git archive. BCR does not contain the source archive. An attestation reference over this object MUST point to the source archive location for it to be accessible from BCR.

Build Provenance over the critical BCR module objects (source.json, MODULE.bazel)

Users should be able to make cryptographic assertions about the provenance of module metadata (MODULE.bazel and source.json : example). These files contain enough information to make assertions over the entire release (including patches). Since they are created by a releaser (rather than source owner – which may or may not be the same person/machine id) the releaser would have to generate an attestation/signature at release time.

Build provenance over other files

To make assertions about a release without being bazel-aware, it might make sense to just attest to all objects uploaded as part of the release. If a developer wants to attach attestations to other files in addition to security critical files they can.

Attestation Details

Generation

The bazel release reusable workflow currently generated the source archive, simple modifications to this reusable workflow should be able to apply github artifact attestations directly to the source archival process, signed with the repositories workflow identity.

On the "easy path", the publish-to-bcr bot generates/hydrates source.json, MODULE.bazel and patches from repository templates. However this needs to be moved into a reusable workflow (either a new publish workflow or integrated into the existing release workflow) so that attestations can be produced using github actions workflow tokens and attributed to the repository directly. Apps such as the publish-to-bcr app do not have access to workflow identity tokens .

On github workflow id is a OIDC token associated with the developer/releasers github repository. In addition to the information embedded in the provenance, any signatures associated with provenances should have an identity tied to <organization>/<repository> and potentially can be refined using other OIDs embedded in the fulcio certificate.

Format

Each attestation bundle is an intoto-bundle which stores all attestations in a single file as a json-list. Where each line is a json formatted attestation in the file. Each attestation is a

1. DSSE envelope
2. Sigstore bundle

Consumers of the bundle should ignore attestation types they don't understand.

The current intoto-bundle format only specifies how to list DSSE bundles, but the spec should be updated to allow sigstore-bundles and remote attestations in the format.

{ "..." } // sigstore signed provenance
{ "..." } // dsse envelope

We want the attestation format to support both the open source usecase (signed with sigstore) and vsa/enterprise/google use case (signed with keys). All build attestations should standardize on DSSE signatures and may be wrapped in a sigstore bundle. Open source attestations on the "easy path" using bazel-build tooling are expected to be sigstore wrapped DSSE signatures generated by github signed using workflow identity.

An example bundle for open source projects using keyless sigstore signing might look like:

{
  "mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json"
  "verificationMaterial": {
    "certificate": {
        "rawBytes": "MIIDNDCCAtqgAwIBAgIEE...B1pm9i+q7bMMl+X2Cm51odIrMI9PbjMw="
    }
    "tlogEntries": [
      {
        "logIndex": "4288993",
        "kindVersion": {
          "kind": "dsse",
          "version": "0.0.1"
        },
        ...
      }
    ],
  },
  "dsseEnvelope": {
    "payload": "ewog...cy83M0KICB9Cn0K", // this is the actual attestation
    "payloadType": "application/vnd.in-toto+json",
    "signatures": [
      {
        "sig": "MEUCIE1...ZBFIhC53qw=",
        "keyid": "" // there is no keyid hint for sigstore keyless
      }
    ]
  }
}

where the payload could look like

{
  "_type": "https://in-toto.io/Statement/v1",
  "subject": [ // the files that we care to attest for the release
    {
      "name": "source.json",
      "digest": {
        "sha256": "abc..."
      }
    },
    {
      "name": "MODULE.bazel",
      "digest": {
        "sha256": "123..."
      }
    }
  ],
  "predicateType": "https://slsa.dev/provenance/v1",
  "predicate": {
	..."whatever github gives us"
  }
}

Extended support for other formats (VSA?)

Formats that are native DSSE without attached sigstore verification materials can simply include the DSSE bundle as the attestation.

{
  "payload": "ewog...cy83M0KICB9Cn0K", // this is the actual vsa
  "payloadType": "application/vnd.in-toto+json",
  "signatures": [
    {
      "sig": "MEUCIE1...ZBFIhC53qw=",
      "keyid" : "some key hint"
    }
  ]
}

where the VSA payload would look like

{
    "_type": "https://in-toto.io/Statement/v1",
    "predicateType": "https://slsa.dev/verification_summary/v1",
    "predicate": {
         // whatever the vsa producer wants to put in here
    }
}

Storage and Discovery

Attestations will be stored according to the file naming convection defined in the intoto project

Attestation for files will be named <filename>.intoto.jsonl and may contain one or more types of attestations. For our purposes, this may contain one slsa attestation, one vsa or both to start with. Other types of attestations may be added for each file as the need arises.

Attestations will be stored and found both on github (or wherever the source archive is stored) and bcr. The attestation will be stored at the primary source of each module object.

An attestation may be reused if generated by the same process.This will likely lead to situations where attestations are duplicated for all the files in a module and that's fine.

Github/Source Archive

The attestation for the source archive should live with the source archive on Github. Typically this means as a release artifact in a github release.

https://github.com/$ORG/$REPO/releases/download/$VERSION/  
├── source-archive.tar.gz  
└── **source-archive.tar.gz.intoto.jsonl**

BCR

Attestations related to the module release should exist with the module information, for the case of BCR this means they are published alongside module files.

/modules/$MODULE/$VERSION  
                 ├── MODULE.bazel  
                 ├── MODULE.bazel.intoto.jsonl
                 ├── source.json  
                 ├── source.json.intoto.jsonl
                 └── patches  
                      ├── 1.patch  
                      ├── 1.patch.intoto.jsonl (optional)
                      ├── …  
                      └── N.patch

Workflow examples

Workflow 1: Separate release and publish on github

Workflow 2: A single release and publish from github

Workflow 3: An enterprise generating VSA attestations for BCR release

BCR Checks

• Bcr should presubmit check that attestations are valid
  • Attestations are either plain DSEE or Sigstore Bundles
  • Attestations signatures can be verified by slsa-verifier
    • ephemeral sigstore based signatures do not need key discovery
    • signatures using keys will need a discovery mechanism (like VSAs)
• For BCR to consider a module verified it will require slsa attestations over the security critical files (source.json, MODULE.bazel and source archive). Each file should have an attestation with the file in question in the "subjects". Any other attestations or subjects can be ignored for the purposes of BCR.

Future checks (not covered by this document)

• Bcr will have a policy engine to make assertions about the incoming modules – like (but not limited to):
  • verifying the identity of the signer matches the expected source repository (somewhat like trusted publishers)
  • all required files (source.json, MODULE.bazel, source-archive) are covered by some attestation

BCR also surfaces provenance in their UI

The Metadata section can include build information of some kind. This is an example of what is possible with this new metadata, not a suggestion of what/how we surface this information.

Client Verification

Bazel clients wishing to verify attestations can check the attestation list and select the attestations that

1. it understands (SLSA, or VSAs or whatever)
2. cover the range of files it is required to attestation (source.json, module.bzl, source-archive)

It can then verify those attestations against a policy defined by the client. Tooling to do verification will not be built into bazel yet. A bazel-module-audit tools could do it.

Existing workflows for slsa verification

1. SLSA: https://slsa.dev/spec/v1.1/verifying-artifacts
2. VSA: https://slsa.dev/spec/v1.1/verification_summary#how-to-verify

[peakschris]

I'm pleased you're following SLSA

• How hard will it be to convince rules authors to add provenance to their gitlab release processes?
• Is there a way to generate provenance for rules or tools downloaded via http_archive?
• Can provenance be modified for an override_module case pointing at a clone of a ruleset (either via command line, or via archive_override)
• How about provenance of a module refenced by hash?
• Can provenance of other tools (e.g. binaries directly referenced, or downloaded via http_archive, or vendored into the monorepo) be included in this scheme?

Thanks! Chris

[loosebazooka]

How hard will it be to convince rules authors to add provenance to their gitlab release processes?

I'm not sure about the build processes for gitlab, I believe they have a lot of the parts already necessary for this in their CI product. It's a matter of providing an easy path that will be low effort for developers. Ideally, we can cover a large number of developers first on github and then helping other repos will be easy because all the mechanisms will already be in place.

Is there a way to generate provenance for rules or tools downloaded via http_archive?

I think this is easy, but the focus of this doc is BCR. Reusing the source achive action that we would be using will generate an attestation for free.

Can provenance be modified for an override_module case pointing at a clone of a ruleset (either via command line, or via archive_override)

I don't know? Provenance cannot be modified after the fact, typically the publishing provenance is attested with a signature. The signature providing some security metadata so you can make assertions.

How about provenance of a module refenced by hash?

Can you elaborate on this? Maybe this is a bazel thing (I come from the security metadata side) that I'm not following?

Can provenance of other tools (e.g. binaries directly referenced, or downloaded via http_archive, or vendored into the monorepo) be included in this scheme?

I think we can explore this in the future. We really want to solve the BCR problem at the repository level first. People hosting their own archives and stuff can generate slsa attestation right now if they want (how easily is somewhat subjective)

[peakschris]

Can you elaborate on this? Maybe this is a bazel thing (I come from the security metadata side) that I'm not following?

So typically in a real bazel project, you are not using a set of rulesets at official release numbers, because things don't work together first time. You need to piece together a set of rules, some using local clones, some using patches, and some using a specific commit hash ahead of a released version.

There has been no moment in the last 2 years that I was using a full set of released rules.

Here is an example of using a temporary fork of a ruleset that I was using earlier in the year when I needed a feature not supported at that time by aspect_rules_lint:

bazel_dep(name = "aspect_rules_lint", version = "0.0.0")
ASPECT_RULES_LINT_TAG="00cb22382bf7c2d7d6e4b2614b51ee8a69551244"
ASPECT_RULES_LINT_INTEGRITY="sha256-MNKMxPqu9tuSJ+S5eIqTm/wCafLcPP3EvylKo2wH6cE="
archive_override(
    module_name = "aspect_rules_lint",
    strip_prefix = "rules_lint-"+ASPECT_RULES_LINT_TAG,
    urls = ["https://github.com/myfork/rules_lint/archive/"+ASPECT_RULES_LINT_TAG+".zip"],
    integrity = ASPECT_RULES_LINT_INTEGRITY,
)

Using a ruleset at a commit hash follows exactly the same pattern as above.

[loosebazooka]

Yeah this is a bit of a complicated question. What we want to do first is add provenance to publicly published rules on BCR. What you're doing in this example seems a little bit out of scope. However, there's no reason that "myfork/.../...zip" can't have provenance attached to it.

[peakschris]

Yeah, this is fair. Actually, thinking about it, the provenance of my 'hacks' of 3rd party things probably just references the location of my internal repository and its hash.

[loosebazooka]

I mean if they're signed, they are an attestation to "something"?, slsa is just a standarized format. This proposal also allows for any number of attestations (dsse/sigstore arranged) attached to an artifact in the *.attestations.jsonl. The tooling may not be able to verify anything but slsa/vsa standards yet... but it should be extensible.

[peakschris]

I think we can explore this in the future. We really want to solve the BCR problem at the repository level first. People hosting their own archives and stuff can generate slsa attestation right now if they want (how easily is somewhat subjective)

I guess I'm asking for the client side tooling built into bazel (that you mention as a separate project) handles both BCR cases, and all other cases of pulling build tooling into bazel, in a unified way.

[loosebazooka]

So, we can write an audit tool that does this. I don't see how it's extra hard, but it is more work. Some of this work is scoped to specific amount of work based on a fixed budget. As we build out the initial functionality and tooling (in opensource), I believe making additions to the tooling should be easier for everyone?