Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Wasm extension HTTP code source #3164

Merged
merged 26 commits into from
Apr 20, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions api/v1alpha1/envoyextensionypolicy_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@ type EnvoyExtensionPolicySpec struct {
// TargetRef
TargetRef gwapiv1a2.PolicyTargetReferenceWithSectionName `json:"targetRef"`

// WASM is a list of Wasm extensions to be loaded by the Gateway.
// Wasm is a list of Wasm extensions to be loaded by the Gateway.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pascal case semantics ?

Copy link
Member Author

@zhaohuabing zhaohuabing Apr 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the name convention here:https://webassembly.org/

WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications.

// Order matters, as the extensions will be loaded in the order they are
// defined in this list.
//
// +optional
WASM []Wasm `json:"wasm,omitempty"`
Wasm []Wasm `json:"wasm,omitempty"`
zhaohuabing marked this conversation as resolved.
Show resolved Hide resolved

// ExtProc is an ordered list of external processing filters
// that should added to the envoy filter chain
Expand Down
16 changes: 11 additions & 5 deletions api/v1alpha1/wasm_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,16 @@ type Wasm struct {
// RootID is a unique ID for a set of extensions in a VM which will share a
// RootContext and Contexts if applicable (e.g., an Wasm HttpFilter and an Wasm AccessLog).
// If left blank, all extensions with a blank root_id with the same vm_id will share Context(s).
// RootID *string `json:"rootID,omitempty"`
// RootID must match the root_id parameter used to register the Context in the Wasm code.
RootID *string `json:"rootID,omitempty"`
Copy link
Member Author

@zhaohuabing zhaohuabing Apr 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can't we reuse name internally here ?

Copy link
Member Author

@zhaohuabing zhaohuabing Apr 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, yes, Name can be the same as RootID. But while RootID must match the root_id in the wasm code, Name can be any meaningful name the user wants to name this wasm extension. They are set in different phases:

  • RootID: by the developer when writing the wasm source code, can't be changed after wasm module is published
  • Name: by the admin when writing the EnvoyExtensionPolicy, can be changed on the fly
          name: envoy.filters.http.wasm/envoyextensionpolicy/envoy-gateway/policy-for-gateway/0
          typedConfig:
            '@type': type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
            config:
              configuration:
                '@type': type.googleapis.com/google.protobuf.StringValue
                value: '{"parameter1":{"key1":"value1","key2":"value2"},"parameter2":"value3"}'
              name: some-meaningful-name # can be anything
              rootId: my-root-id. # must be the same root_id in the wasm source code
              vmConfig:
                code:
                  remote:
                    httpUri:
                      cluster: www_example_com_443
                      timeout: 10s
                      uri: https://www.example.com/wasm-filter-1.wasm
                    sha256: 746df05c8f3a0b07a46c0967cfbc5cbe5b9d48d0f79b6177eeedf8be6c8b34b5
                runtime: envoy.wasm.runtime.v8
                vmId: envoyextensionpolicy/envoy-gateway/policy-for-gateway/0

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can generate this field internally, using name, my vote is to do that and not expose another field the user needs to think about how to populate

Copy link
Member Author

@zhaohuabing zhaohuabing Apr 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't generate RootID, See: https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/921039ae983ce053bf5cba78a85a3c08ff9791e5/proxy_wasm_intrinsics.cc#L27-L28

And it can be optional if the wasm source code leaves it empty.


// Code is the wasm code for the extension.
Code WasmCodeSource `json:"code"`

// Config is the configuration for the Wasm extension.
// This configuration will be passed as a JSON string to the Wasm extension.
Config *apiextensionsv1.JSON `json:"config"`
// +optional
Config *apiextensionsv1.JSON `json:"config,omitempty"`

// FailOpen is a switch used to control the behavior when a fatal error occurs
// during the initialization or the execution of the Wasm extension.
Expand All @@ -61,7 +63,7 @@ type WasmCodeSource struct {
// Type is the type of the source of the wasm code.
// Valid WasmCodeSourceType values are "HTTP" or "Image".
//
// +kubebuilder:validation:Enum=HTTP;Image
// +kubebuilder:validation:Enum=HTTP;Image;ConfigMap
// +unionDiscriminator
Type WasmCodeSourceType `json:"type"`

Expand All @@ -78,8 +80,9 @@ type WasmCodeSource struct {
Image *ImageWasmCodeSource `json:"image,omitempty"`

// SHA256 checksum that will be used to verify the wasm code.
// +optional
// SHA256 *string `json:"sha256,omitempty"`
//
// kubebuilder:validation:Pattern=`^[a-f0-9]{64}$`
SHA256 string `json:"sha256"`
Copy link
Member Author

@zhaohuabing zhaohuabing Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make SHA256 mandatory for now to avoid downloading wasm and calculate SHA in the control plane.

We can make it optional without breaking changes if we opt to build a cache in the control plane in the future.

}

// WasmCodeSourceType specifies the types of sources for the wasm code.
Expand All @@ -92,6 +95,9 @@ const (

// ImageWasmCodeSourceType allows the user to specify the wasm code in an OCI image.
ImageWasmCodeSourceType WasmCodeSourceType = "Image"

// ConfigMapCodeSourceType allows the user to specify the wasm code in a ConfigMap.
zhaohuabing marked this conversation as resolved.
Show resolved Hide resolved
ConfigMapWasmCodeSourceType WasmCodeSourceType = "ConfigMap"
)

// HTTPWasmCodeSource defines the HTTP URL containing the wasm code.
Expand Down
9 changes: 7 additions & 2 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@ spec:
rule: '!has(self.sectionName)'
wasm:
description: |-
WASM is a list of Wasm extensions to be loaded by the Gateway.
Wasm is a list of Wasm extensions to be loaded by the Gateway.
Order matters, as the extensions will be loaded in the order they are
defined in this list.
items:
Expand Down Expand Up @@ -394,6 +394,13 @@ spec:
- pullSecret
- url
type: object
sha256:
description: |-
SHA256 checksum that will be used to verify the wasm code.


kubebuilder:validation:Pattern=`^[a-f0-9]{64}$`
type: string
type:
allOf:
- enum:
Expand All @@ -402,11 +409,13 @@ spec:
- enum:
- HTTP
- Image
- ConfigMap
description: |-
Type is the type of the source of the wasm code.
Valid WasmCodeSourceType values are "HTTP" or "Image".
type: string
required:
- sha256
- type
type: object
config:
Expand All @@ -430,9 +439,15 @@ spec:
Wasm extension if multiple extensions are handled by the same vm_id and root_id.
It's also used for logging/debugging.
type: string
rootID:
description: |-
RootID is a unique ID for a set of extensions in a VM which will share a
RootContext and Contexts if applicable (e.g., an Wasm HttpFilter and an Wasm AccessLog).
If left blank, all extensions with a blank root_id with the same vm_id will share Context(s).
RootID must match the root_id parameter used to register the Context in the Wasm code.
type: string
required:
- code
- config
- name
type: object
type: array
Expand Down
115 changes: 97 additions & 18 deletions internal/gatewayapi/envoyextensionpolicy.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
package gatewayapi

import (
"errors"
"fmt"
"sort"
"strings"
Expand Down Expand Up @@ -293,24 +294,34 @@ func resolveEEPolicyRouteTargetRef(policy *egv1a1.EnvoyExtensionPolicy, routes m

func (t *Translator) translateEnvoyExtensionPolicyForRoute(policy *egv1a1.EnvoyExtensionPolicy, route RouteContext,
xdsIR XdsIRMap, resources *Resources) error {
var (
extProcs []ir.ExtProc
wasms []ir.Wasm
err, errs error
)

if extProcs, err = t.buildExtProcs(policy, resources); err != nil {
errs = errors.Join(errs, err)
}
if wasms, err = t.buildWasms(policy); err != nil {
errs = errors.Join(errs, err)
}

// Apply IR to all relevant routes
prefix := irRoutePrefix(route)
for _, ir := range xdsIR {
for _, http := range ir.HTTP {
for _, r := range http.Routes {
// Apply if there is a match
if strings.HasPrefix(r.Name, prefix) {
if extProcs, err := t.buildExtProcs(policy, resources); err == nil {
r.ExtProcs = extProcs
} else {
return err
}
r.ExtProcs = extProcs
r.Wasms = wasms
}
}
}
}

return nil
return errs
}

func (t *Translator) buildExtProcs(policy *egv1a1.EnvoyExtensionPolicy, resources *Resources) ([]ir.ExtProc, error) {
Expand All @@ -320,21 +331,24 @@ func (t *Translator) buildExtProcs(policy *egv1a1.EnvoyExtensionPolicy, resource
return nil, nil
}

if len(policy.Spec.ExtProc) > 0 {
for idx, ep := range policy.Spec.ExtProc {
name := irConfigNameForEEP(policy, idx)
extProcIR, err := t.buildExtProc(name, utils.NamespacedName(policy), ep, idx, resources)
if err != nil {
return nil, err
}
extProcIRList = append(extProcIRList, *extProcIR)
for idx, ep := range policy.Spec.ExtProc {
name := irConfigNameForEEP(policy, idx)
extProcIR, err := t.buildExtProc(name, utils.NamespacedName(policy), ep, idx, resources)
if err != nil {
return nil, err
}
extProcIRList = append(extProcIRList, *extProcIR)
}
return extProcIRList, nil
}

func (t *Translator) translateEnvoyExtensionPolicyForGateway(policy *egv1a1.EnvoyExtensionPolicy,
gateway *GatewayContext, xdsIR XdsIRMap, resources *Resources) error {
var (
extProcs []ir.ExtProc
wasms []ir.Wasm
err, errs error
)

irKey := t.getIRKey(gateway.Gateway)
// Should exist since we've validated this
Expand All @@ -345,9 +359,11 @@ func (t *Translator) translateEnvoyExtensionPolicyForGateway(policy *egv1a1.Envo
string(policy.Spec.TargetRef.Name),
)

extProcs, err := t.buildExtProcs(policy, resources)
if err != nil {
return err
if extProcs, err = t.buildExtProcs(policy, resources); err != nil {
errs = errors.Join(errs, err)
}
if wasms, err = t.buildWasms(policy); err != nil {
errs = errors.Join(errs, err)
}

for _, http := range ir.HTTP {
Expand All @@ -360,13 +376,21 @@ func (t *Translator) translateEnvoyExtensionPolicyForGateway(policy *egv1a1.Envo
// targeting a lesser specific scope(Gateway).
for _, r := range http.Routes {
// if already set - there's a route level policy, so skip
if r.ExtProcs != nil ||
r.Wasms != nil {
continue
}

if r.ExtProcs == nil {
r.ExtProcs = extProcs
}
if r.Wasms == nil {
r.Wasms = wasms
}
}
}

return nil
return errs
}

func (t *Translator) buildExtProc(
Expand Down Expand Up @@ -428,3 +452,58 @@ func irConfigNameForEEP(policy *egv1a1.EnvoyExtensionPolicy, idx int) string {
utils.NamespacedName(policy).String(),
idx)
}

func (t *Translator) buildWasms(policy *egv1a1.EnvoyExtensionPolicy) ([]ir.Wasm, error) {
var wasmIRList []ir.Wasm

if policy == nil {
return nil, nil
}

for idx, wasm := range policy.Spec.Wasm {
name := irConfigNameForEEP(policy, idx)
wasmIR, err := t.buildWasm(name, wasm)
if err != nil {
return nil, err
}
wasmIRList = append(wasmIRList, *wasmIR)
}
return wasmIRList, nil
}

func (t *Translator) buildWasm(name string, wasm egv1a1.Wasm) (*ir.Wasm, error) {
var (
failOpen = false
httpWasmCode *ir.HTTPWasmCode
)

if wasm.FailOpen != nil {
failOpen = *wasm.FailOpen
}

switch wasm.Code.Type {
case egv1a1.HTTPWasmCodeSourceType:
httpWasmCode = &ir.HTTPWasmCode{
URL: wasm.Code.HTTP.URL,
SHA256: wasm.Code.SHA256,
}
case egv1a1.ConfigMapWasmCodeSourceType:
return nil, fmt.Errorf("ConfigMap Wasm code source is not supported yet")
case egv1a1.ImageWasmCodeSourceType:
return nil, fmt.Errorf("OCI image Wasm code source is not supported yet")
default:
// should never happen because of kubebuilder validation, just a sanity check
return nil, fmt.Errorf("unsupported Wasm code source type %q", wasm.Code.Type)
}

wasmIR := &ir.Wasm{
Name: name,
RootID: wasm.RootID,
WasmName: wasm.Name,
Config: wasm.Config,
FailOpen: failOpen,
HTTPWasmCode: httpWasmCode,
}

return wasmIR, nil
}
Loading
Loading