feat: add experimental conformance support

[Xunzhuo]

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

• add experimental conformance support
• generate conformance report

[codecov]

Codecov Report

Merging #1780 (3d1ebac) into main (5cb8697) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1780      +/-   ##
==========================================
+ Coverage   65.32%   65.35%   +0.02%     
==========================================
  Files          86       86              
  Lines       12529    12529              
==========================================
+ Hits         8185     8188       +3     
+ Misses       3826     3824       -2     
+ Partials      518      517       -1     

see 2 files with indirect coverage changes

[arkodg]

thanks for working on this @Xunzhuo !
do you think we should manually run this, run this in CI on every merge and copy into the latest release notes, run this before every semvar release ?
and how often should we publish this data upstream in the gateway api repo

[Xunzhuo]

  apiVersion: gateway.networking.k8s.io/v1alpha1
  date: "2023-08-10T03:35:39Z"
  gatewayAPIVersion: v0.8.0-rc1
  implementation:
    contact:
    - https://github.com/envoyproxy/gateway/blob/main/GOVERNANCE.md
    organization: envoyproxy
    project: envoy-gateway
    url: https://github.com/envoyproxy/gateway
    version: latest
  kind: ConformanceReport
  profiles:
  - core:
      result: success
      statistics:
        Failed: 0
        Passed: 29
        Skipped: 0
      summary: ""
    extended:
      result: success
      statistics:
        Failed: 0
        Passed: 9
        Skipped: 0
      summary: ""
      supportedFeatures:
      - HTTPRouteSchemeRedirect
      - HTTPRoutePathRewrite
      - HTTPRouteQueryParamMatching
      - HTTPResponseHeaderModification
      - HTTPRoutePortRedirect
      - HTTPRoutePathRedirect
      - HTTPRouteHostRewrite
      - HTTPRouteRequestMirror
      - HTTPRouteMethodMatching
    name: HTTP
  - core:
      result: success
      statistics:
        Failed: 0
        Passed: 11
        Skipped: 0
      summary: ""
    name: TLS

[Xunzhuo]

thanks for working on this @Xunzhuo ! do you think we should manually run this, run this in CI on every merge and copy into the latest release notes, run this before every semvar release ?

I think experimental conformance can run in main CI, and having conformance report in every commit.

and how often should we publish this data upstream in the gateway api repo

I think each time we bump gwapi version ?

[arkodg]

should we only run & publish this every time we import a new gateway api version ?
so we can add a new GHA that checks if charts/gateway-helm/crds/gatewayapi-crds.yaml path has changed and if so it runs this specific make target as well as publishes the Conformance Profile as a GH comment on the PR 😎 , which we can then copy over to upstream

[Xunzhuo]

should we only run & publish this every time we import a new gateway api version ? so we can add a new GHA that checks if charts/gateway-helm/crds/gatewayapi-crds.yaml path has changed and if so it runs this specific make target as well as publishes the Conformance Profile as a GH comment on the PR 😎 , which we can then copy over to upstream

Sounds good : )

[Xunzhuo]

I tested and made changes to gwapi.yaml, https://github.com/envoyproxy/gateway/actions/runs/6117087341?pr=1780, it worked normally.

[arkodg]

any way to descope this a bit ?

[arkodg]

you probably want to add a lot more https://github.com/kubernetes-sigs/gateway-api/blob/8ab4e18ca73533793f8a5d4e8c4248626b4690bf/conformance/utils/suite/features.go#L195

[arkodg]

thanks for adding another great tool to EG @Xunzhuo !
minor comment which can be done as a follow up