feat: Wasm extension HTTP code source #3164

zhaohuabing · 2024-04-10T09:54:45Z

This PR implements the HTTP code source for Wasm extension.

zhaohuabing · 2024-04-11T07:33:15Z

api/v1alpha1/wasm_types.go

-	// SHA256 *string `json:"sha256,omitempty"`
+	//
+	// kubebuilder:validation:Pattern=`^[a-f0-9]{64}$`
+	SHA256 string `json:"sha256"`


Make SHA256 mandatory for now to avoid downloading wasm and calculate SHA in the control plane.

We can make it optional without breaking changes if we opt to build a cache in the control plane in the future.

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

api/v1alpha1/envoyextensionypolicy_types.go

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

This reverts commit 64b76a9.

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

codecov · 2024-04-16T06:03:18Z

Codecov Report

Attention: Patch coverage is 44.44444% with 135 lines in your changes are missing coverage. Please review.

Project coverage is 66.48%. Comparing base (29946b0) to head (67977a3).
Report is 67 commits behind head on main.

❗ Current head 67977a3 differs from pull request most recent head 0c52d50. Consider uploading reports for the commit 0c52d50 to get more accurate results

Files	Patch %	Lines
internal/xds/translator/wasm.go	15.44%	97 Missing and 7 partials ⚠️
internal/gatewayapi/envoyextensionpolicy.go	74.69%	16 Missing and 5 partials ⚠️
internal/xds/translator/utils.go	70.00%	6 Missing and 3 partials ⚠️
internal/xds/translator/jwt.go	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3164      +/-   ##
==========================================
- Coverage   66.51%   66.48%   -0.03%     
==========================================
  Files         161      163       +2     
  Lines       22673    22994     +321     
==========================================
+ Hits        15080    15288     +208     
- Misses       6720     6824     +104     
- Partials      873      882       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing · 2024-04-18T11:06:01Z

api/v1alpha1/wasm_types.go

@@ -31,14 +31,16 @@ type Wasm struct {
 	// RootID is a unique ID for a set of extensions in a VM which will share a
 	// RootContext and Contexts if applicable (e.g., an Wasm HttpFilter and an Wasm AccessLog).
 	// If left blank, all extensions with a blank root_id with the same vm_id will share Context(s).
-	// RootID *string `json:"rootID,omitempty"`
+	// RootID must match the root_id parameter used to register the Context in the Wasm code.
+	RootID *string `json:"rootID,omitempty"`


RootID is needed and it can't be generated by EG. See: https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/921039ae983ce053bf5cba78a85a3c08ff9791e5/proxy_wasm_intrinsics.cc#L27-L28

zhaohuabing · 2024-04-18T11:09:52Z

test/e2e/testdata/wasm.yaml

+    code:
+      type: HTTP
+      http:
+        url: https://raw.githubusercontent.com/envoyproxy/envoy/main/examples/wasm-cc/lib/envoy_filter_http_wasm_example.wasm


@arkodg The HTTP code source does support HTTPS schema.

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

arkodg · 2024-04-18T21:41:08Z

api/v1alpha1/envoyextensionypolicy_types.go

@@ -46,12 +46,12 @@ type EnvoyExtensionPolicySpec struct {
 	// TargetRef
 	TargetRef gwapiv1a2.PolicyTargetReferenceWithSectionName `json:"targetRef"`

-	// WASM is a list of Wasm extensions to be loaded by the Gateway.
+	// Wasm is a list of Wasm extensions to be loaded by the Gateway.


pascal case semantics ?

Following the name convention here：https://webassembly.org/

WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications.

api/v1alpha1/wasm_types.go

arkodg · 2024-04-18T21:43:31Z

api/v1alpha1/wasm_types.go

@@ -31,14 +31,16 @@ type Wasm struct {
 	// RootID is a unique ID for a set of extensions in a VM which will share a
 	// RootContext and Contexts if applicable (e.g., an Wasm HttpFilter and an Wasm AccessLog).
 	// If left blank, all extensions with a blank root_id with the same vm_id will share Context(s).
-	// RootID *string `json:"rootID,omitempty"`
+	// RootID must match the root_id parameter used to register the Context in the Wasm code.
+	RootID *string `json:"rootID,omitempty"`


why can't we reuse name internally here ?

In theory, yes, Name can be the same as RootID. But while RootID must match the root_id in the wasm code, Name can be any meaningful name the user wants to name this wasm extension. They are set in different phases:

RootID: by the developer when writing the wasm source code, can't be changed after wasm module is published

Name: by the admin when writing the EnvoyExtensionPolicy, can be changed on the fly

name: envoy.filters.http.wasm/envoyextensionpolicy/envoy-gateway/policy-for-gateway/0 typedConfig: '@type': type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm config: configuration: '@type': type.googleapis.com/google.protobuf.StringValue value: '{"parameter1":{"key1":"value1","key2":"value2"},"parameter2":"value3"}' name: some-meaningful-name # can be anything rootId: my-root-id. # must be the same root_id in the wasm source code vmConfig: code: remote: httpUri: cluster: www_example_com_443 timeout: 10s uri: https://www.example.com/wasm-filter-1.wasm sha256: 746df05c8f3a0b07a46c0967cfbc5cbe5b9d48d0f79b6177eeedf8be6c8b34b5 runtime: envoy.wasm.runtime.v8 vmId: envoyextensionpolicy/envoy-gateway/policy-for-gateway/0

If we can generate this field internally, using name, my vote is to do that and not expose another field the user needs to think about how to populate

We can't generate RootID, See: https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/921039ae983ce053bf5cba78a85a3c08ff9791e5/proxy_wasm_intrinsics.cc#L27-L28

And it can be optional if the wasm source code leaves it empty.

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com>

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

arkodg

LGTM thanks !

zhaohuabing requested a review from a team as a code owner April 10, 2024 09:54

zhaohuabing marked this pull request as draft April 10, 2024 09:54

zhaohuabing force-pushed the wasm-http branch 3 times, most recently from ea8e717 to 07495ec Compare April 11, 2024 07:30

zhaohuabing commented Apr 11, 2024

View reviewed changes

zhaohuabing force-pushed the wasm-http branch 7 times, most recently from d81834d to adaa4c8 Compare April 11, 2024 07:50

zhaohuabing marked this pull request as ready for review April 11, 2024 07:50

zhaohuabing marked this pull request as draft April 11, 2024 07:52

wasm http impl

d0cb214

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing force-pushed the wasm-http branch from adaa4c8 to d0cb214 Compare April 11, 2024 09:25

zhaohuabing marked this pull request as ready for review April 11, 2024 09:26

zhaohuabing requested review from arkodg and guydc April 11, 2024 09:28

zhaohuabing commented Apr 11, 2024

View reviewed changes

api/v1alpha1/envoyextensionypolicy_types.go Show resolved Hide resolved

zhaohuabing added 2 commits April 12, 2024 16:35

fix order

b6cc5a6

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

address comments

64b76a9

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing requested a review from Xunzhuo April 12, 2024 08:42

zhaohuabing added 2 commits April 12, 2024 19:10

Revert "address comments"

d05c8ec

This reverts commit 64b76a9.

Merge remote-tracking branch 'upstream/main' into wasm-http

0b47909

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing force-pushed the wasm-http branch from 5bc9369 to 0b47909 Compare April 14, 2024 06:36

zhaohuabing added 2 commits April 16, 2024 13:56

Merge remote-tracking branch 'upstream/main' into wasm-http

164f6d8

add enum types

67977a3

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing dismissed guydc’s stale review via 4ba3921 April 18, 2024 00:29

zhaohuabing added 5 commits April 18, 2024 02:50

fix gen

c48eadc

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

fix gen

0c4400c

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

fix gen

0369f4f

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

minor wording

e82e79b

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

fix gen

3b9510b

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing commented Apr 18, 2024

View reviewed changes

fix test

67baf73

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing force-pushed the wasm-http branch from 0f1e7c2 to 67baf73 Compare April 18, 2024 12:53

zhaohuabing added 3 commits April 18, 2024 05:54

Merge remote-tracking branch 'upstream/main' into wasm-http

416eeb9

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

minor wording

20038ab

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

add log for test

6a4942e

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing requested a review from guydc April 18, 2024 14:00

arkodg reviewed Apr 18, 2024

View reviewed changes

api/v1alpha1/wasm_types.go Outdated Show resolved Hide resolved

arkodg reviewed Apr 18, 2024

View reviewed changes

zhaohuabing added 4 commits April 18, 2024 15:17

move out configmap source

bfb649d

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

fix flaky test

6086333

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

Merge branch 'main' into wasm-http

cb2b993

fix lint

8d536ba

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

zhaohuabing requested a review from arkodg April 19, 2024 01:34

zhaohuabing added 2 commits April 18, 2024 22:08

Merge branch 'main' into wasm-http

a13e258

Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com>

fix lint

0c52d50

Signed-off-by: huabing zhao <zhaohuabing@gmail.com>

arkodg approved these changes Apr 19, 2024

View reviewed changes

arkodg requested review from a team April 19, 2024 22:10

zirain approved these changes Apr 20, 2024

View reviewed changes

zirain merged commit fcfeefd into envoyproxy:main Apr 20, 2024
20 checks passed

eguzki mentioned this pull request Apr 23, 2024

Envoy Gateway Support Kuadrant/kuadrant-operator#325

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Wasm extension HTTP code source #3164

feat: Wasm extension HTTP code source #3164

zhaohuabing commented Apr 10, 2024 •

edited

Loading

zhaohuabing Apr 11, 2024 •

edited

Loading

codecov bot commented Apr 16, 2024 •

edited

Loading

zhaohuabing Apr 18, 2024 •

edited

Loading

zhaohuabing Apr 18, 2024

arkodg Apr 18, 2024

zhaohuabing Apr 18, 2024 •

edited

Loading

arkodg Apr 18, 2024

zhaohuabing Apr 18, 2024 •

edited

Loading

arkodg Apr 18, 2024

zhaohuabing Apr 18, 2024 •

edited

Loading

arkodg left a comment

feat: Wasm extension HTTP code source #3164

feat: Wasm extension HTTP code source #3164

Conversation

zhaohuabing commented Apr 10, 2024 • edited Loading

zhaohuabing Apr 11, 2024 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2024 • edited Loading

Codecov Report

zhaohuabing Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

zhaohuabing Apr 18, 2024

Choose a reason for hiding this comment

arkodg Apr 18, 2024

Choose a reason for hiding this comment

zhaohuabing Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

arkodg Apr 18, 2024

Choose a reason for hiding this comment

zhaohuabing Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

arkodg Apr 18, 2024

Choose a reason for hiding this comment

zhaohuabing Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

arkodg left a comment

Choose a reason for hiding this comment

zhaohuabing commented Apr 10, 2024 •

edited

Loading

zhaohuabing Apr 11, 2024 •

edited

Loading

codecov bot commented Apr 16, 2024 •

edited

Loading

zhaohuabing Apr 18, 2024 •

edited

Loading

zhaohuabing Apr 18, 2024 •

edited

Loading

zhaohuabing Apr 18, 2024 •

edited

Loading

zhaohuabing Apr 18, 2024 •

edited

Loading