Testing Interop

[mkhraisha]

We should discuss what our tests should look like for actual interop tests.

My current proposal is we create individual test credential fixtures, that are signed by each implementer, which will then be submitted to each verifier, creating a matrix of Issuer-Verifier results.

Open question:

Currently the spec requires VC-JOSE so should all these fixtures be JWTs?

[PatStLouis]

The fixtures should match the spec so they could be individual JWTs or a single file containing an array of JWTs, whichever is the most convenient to testing.

One thing I would like to see is each implementer submit a non revoked credential and a revoked credential to test BitstringStatusList implementation interop, not just signatures.

My suggestion would be to replicate the Business Card Workflow. This seems like a simple scenario that matches what we want to achieve.

Each implementer can submit a valid business card and a revoked business card.

[PatStLouis]

Also, are implementers submitting credentials or presentations? Technically it should be a presentation to be posted to an oauth protected endpoint.

[PatStLouis]

Great discussions. I just want to capture my train of thoughts around error codes and what we could test.

For the /presentations endpoint:

200 -> The VC is verified, valid, unrevoked
400 -> Bad Request (not a presentation, bad content-type, error with data model, invalid proof)
401 -> Unauthenticated Request
422 -> Unprocessable Content: when the verifier can't process the payload because its expired or its revoked

These 4 response code could cover the need to asses if the implementation is doing sufficient testing and to test some interop for the implementers libraries. If more granularity is beneficial we could define error titles in the specification in addition to status to leverage.
ex: 400: Content Type, 400: Proof, 422: Status, 422: Validity Period

Can we confirm the fixtures will have this format:

{
  "@context": [
    "https://www.w3.org/ns/credentials/v2",
    "https://www.w3.org/ns/credentials/examples/v2"
  ],
  "type": ["VerifiablePresentation", "ExamplePresentation"],
  "verifiableCredential": [{
    "@context": "https://www.w3.org/ns/credentials/v2",
    "id": "data:application/vc+ld+json+sd-jwt;QzVjV...RMjU",
    "type": "EnvelopedVerifiableCredential"
  }]
}

Looking forward furthering this dicussion

[mkhraisha]

Proposal is to create a /verify endpoint that takes the same payload as the /presentations endpoint.

The /verify endpoint will be used to test interop and compliance to the spec.

I agree with the fixtures that Patrick is proposing.

Please react with thumbs up or down to this comment

[OR13]

Can we use CURL-like examples to be a bit clearer about what we mean here?

I think this is what is being proposed:

POST /presentations --content-type application/vp+ld+json+jwt { ...body is token } 
Response is application/json of some unknown schema 

POST /verify --content-type application/vp+ld+json+jwt { ...body is token } 
Response is application/json of some unknown schema 

You will need a way to pass parameters for presentation verify internal endpoint, I would use query strings like ?aud=...&nonce=...

You would also need to define the shape of the response...

{ "verified": true, content: ... what was verified, ... status stuff, ... schema stuff, }

Our validation functions return stuff that looks like this:

{
        "verified": true,
        "content": {
          "@context": [
            "https://www.w3.org/ns/credentials/v2",
            "https://vendor.example/api/context/v2"
          ],
          "id": "https://vendor.example/api/credentials/3732",
          "type": [
            "VerifiableCredential",
            "ExampleDegreeCredential"
          ],
          "issuer": {
            "id": "did:example:123",
            "name": "Example University"
          },
          "validFrom": "2024-07-11T20:15:51.249Z",
          "credentialSchema": {
            "id": "https://vendor.example/api/schemas/product-passport",
            "type": "JsonSchema"
          },
          "credentialStatus": [
            {
              "id": "https://vendor.example/credentials/status/3#0",
              "type": "BitstringStatusListEntry",
              "statusPurpose": "revocation",
              "statusListIndex": "0",
              "statusListCredential": "https://vendor.example/credentials/status/3"
            }
          ],
          "credentialSubject": {
            "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
            "degree": {
              "type": "ExampleBachelorDegree",
              "subtype": "Bachelor of Science and Arts"
            }
          }
        },
        "schema": {
          "https://vendor.example/api/schemas/product-passport": {
            "validation": "succeeded"
          }
        },
        "status": {
          "https://vendor.example/credentials/status/3#0": {
            "revocation": true
          }
        },
        "warnings": [
          {
            "message": "Identifier will not be well understood:  did:example:123",
            "pointer": "/issuer/id",
            "reference": "https://www.w3.org/TR/2024/CRD-vc-data-model-2.0-20240205/#identifiers"
          },
          ...
        ]
      }

This validation and this /verify endpoint would be internal APIs, in contrast to /presentations ...

I don't think it makes a lot of sense to standardize them... of course there are many different ways a vendor might return validation information, and they might also do other checks like look at dates, or check RDF, or join with private data sets, or process claims with AI...

[TallTed]

This validation and this /verify endpoint would be internal APIs, in contrast to /presentations

We agreed some time ago (years?) that we would not specify anything as an internal (or external) API; rather, that all APIs would be designed and specified to support authentication, so as to support functioning across trust boundaries, whether those exist within a single server, office, division, company, or otherwise. (We also adopted "within" and "across" trust boundaries as clearer and more accurate replacements for "internal" and "external".)

[OR13]

After some discussion with @mkhraisha perhaps we can make it clearer that this API does not mutate state, but provides validation results that might be useful when determining if a presentation will ultimately be acceptable to a verifier.

A few details to consider:

• confirmation validation
• needs to be authenticated since its expensive
• need to be clear that this is not mutating state on the server.

Perhaps the following:

Authorization: Bearer <same token as would be used for /presentations>
GET /validate?token=...&aud=...&nonce=...
Response:
{
  "verified": true,
  ...
  // exact schema TBD
}

This way the entire request and response can be safely cached base on the URL alone.
We are being clear that this endpoint is useful for "dry run" / "testing of a future presentation".
The endpoint would also be useful in front end applications that need to display validation details (check marks, warnings, errors)
The endpoint would work for both "credentials" and "presentations".
It seems like the goal is to provide more than just a verification answer here, which is why I suggest calling this validate, and giving answers to the schema and status checks which are not part of verify (they come immediately after it).

A command line version of this kind of interface might look like:

$ command --dry-run

Where the --dry-run indicates that the command should not actually change things, but that the caller wants to know how the command would be interpreted before committing to running it.

[PatStLouis]

@OR13 if the spec recommends the vcdm 2.0 with vc-jose securing mechanism, wouldn't the body of the request be a EnvelopedVerifiablePresentation? That's how I'm currently understanding this requirement.

curl -X 'POST' \
  'https://my.application/presentations' \
  -H 'Authorization: Bearer ...' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json+ld' \
  -d '{
  "@context": "https://www.w3.org/ns/credentials/v2",
  "id": "data:application/vp+jwt;eyJraWQiO...zhwGfQ",
  "type": "EnvelopedVerifiablePresentation"
}'

+1 for the need to differentiate validating from verifying.

However I think these steps could live on a 'verification' endpoint, as long as the boolean for verified isn't affected by anything else than a successful signature check. Every raised validation issue would be returned as a warning.

For the returned schema, the way I'm approaching the issue is to leverage the ProblemDetails as objects in the warnings/errors arrays. This is based on RFC9457

So a response could look like:

{
  "verified": true,
  "document": {unsecured_document},
  "errors": [],
  "warnings": [
    {
      "type": "https://www.w3.org/TR/vc-bitstring-status-list/#STATUS_VERIFICATION_ERROR",
      "title": "STATUS_VERIFICATION_ERROR",
      "message": "The status with purpose revocation is set to true."
    },
    {
      "type": "https://www.w3.org/TR/vc-json-schema/#SCHEMA_VALIDATION_ERROR",
      "title": "SCHEMA_VALIDATION_ERROR",
      "message": "The provided schema did not validate."
    }
  ]
}

or

{
  "verified": false,
  "document": {unsecured_document},
  "errors": [
    {
      "type": "https://www.w3.org/TR/vc-data-model#CRYPTOGRAPHIC_SECURITY_ERROR",
      "title": "CRYPTOGRAPHIC_SECURITY_ERROR",
      "message": "The signature is invalid."
    }
  ],
  "warnings": []
}

Now with this being said, I like what is being proposed with having respective status/schema properties in the response body with their id's, however a risk is that the id is optional for a status entry. So we are creating a test requirement on an optional feature. Maybe we want to be opinionated a make the status entry id a MUST.

We can tangibly start with just testing for the signature to get something in place, and define how to assert status/schema as we go along...

[OR13]

Typically POST is used to create resources, GET is used retrieve them... The resource you are retrieving is validation results.

A resource is a URL including query string.

I suppose you could look at this, like you are producing a validation result (and possibly retaining it indefinitely), in which case POST makes sense, since you are creating a resource.

It's true that id is optional all over the place in the vcdm.

But statusListCredential and statusListIndex are required... For some RDF types...

RFC9457, is a fine base to build on.

It looks like you've pasted a validation result from data integrity libraries, but missing some other details (such as the per credential verification checks)...

We could also include those in the response.

[mkhraisha]

Just had a chat with Brent and Nis on a call, and we've found that really we don't think there's enough value in continuing work on these tests compared to the effort required, and as long as testing isn't a blocker for FCGS then we should close this issue and #657

We should also remove the Interoperability testing section.