Validating a Request Payload with Wasm

What is a Wasm Plugin?

A Wasm plugin lets you easily extend the functionality of your service mesh by adding custom code to the data path. Plugins can be written in the language of your choice. At present, there are Proxy-Wasm SDKs for AssemblyScript (TypeScript-ish), C++, Rust, Zig, and Go.

In this blog post we describe how to use a Wasm plugin to validate a request payload. This is an important use case for Wasm with Istio and an example of the many ways in which you can extend Istio using Wasm. You may be interested in reading our blog posts on using Wasm with Istio and viewing the recording of our free workshop on using Wasm in Istio and Envoy.

When Should You Use a Wasm Plugin?

You should use a Wasm plugin when you need to add custom functionality that isn’t supported natively by Envoy or Istio. Some ideas of how these can be used are to add custom validation, authentication, logging or manage quotas.

In this example we will build and run a Wasm plugin that validates that the request body is JSON and that it contains two required keys, id and token.

Writing a Wasm Plugin

This example uses tinygo to compile into Wasm. Ensure that you have the tinygo compiler installed.

Set Up Wasm Contexts

First we set up the Wasm contexts so our tinygo file can operate on HTTP requests:

package main

import (
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
	"github.com/tidwall/gjson"
)

func main() {
	// SetVMContext is the entrypoint for setting up this entire Wasm VM.
	// Please make sure that this entrypoint be called during "main()" function,
	// otherwise this VM would fail.
	proxywasm.SetVMContext(&vmContext{})
}

// vmContext implements types.VMContext interface of proxy-wasm-go SDK.
type vmContext struct {
	// Embed the default VM context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultVMContext
}

// Override types.DefaultVMContext.
func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext {
	return &pluginContext{}
}

// pluginContext implements types.PluginContext interface of proxy-wasm-go SDK.
type pluginContext struct {
	// Embed the default plugin context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultPluginContext
}

// Override types.DefaultPluginContext.
func (ctx *pluginContext) NewHttpContext(contextID uint32) types.HttpContext {
	return &payloadValidationContext{}
}

// payloadValidationContext implements types.HttpContext interface of proxy-wasm-go SDK.
type payloadValidationContext struct {
	// Embed the default root http context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultHttpContext
	totalRequestBodySize int
}

Validating the Payload

The content type header is validated by implementing OnHttpRequestHeaders, which is called once the request headers have been received from the client.

proxywasm.SendHttpResponse is used to respond with a 403 forbidden error code and message if the content type is missing.

func (ctx *payloadValidationContext) OnHttpRequestHeaders(numHeaders int, endOfStream bool) types.Action {
	contentType, err := proxywasm.GetHttpRequestHeader("content-type")
	if err != nil || contentType != "application/json" {
		// If the header doesn't have the expected content type, send a 403 response
		if err := proxywasm.SendHttpResponse(403, nil, []byte("content-type must be provided"), -1); err != nil {
			proxywasm.LogErrorf("failed to send the 403 response: %v", err)
		}
		// and terminates the further processing of this traffic by ActionPause.
		return types.ActionPause
	}

	// ActionContinue lets the host continue processing the body.
	return types.ActionContinue
}

The request body is validated by implementing OnHttpRequestBody, which is called each time a chunk of the request is received from the client. This is done by waiting until endOfStream is true and recording the total size of all the received chunks. Once the whole body is received, it is read using proxywasm.GetHttpRequestBody and can then be validated using golang.

This example uses gjson as tinygo does not support golang’s default JSON library. It checks that the payload is valid JSON, and that the keys id and token are present.

func (ctx *payloadValidationContext) OnHttpRequestBody(bodySize int, endOfStream bool) types.Action {
	ctx.totalRequestBodySize += bodySize
	if !endOfStream {
		// OnHttpRequestBody may be called each time a part of the body is received.
		// Wait until we see the entire body.
		return types.ActionPause
	}

	body, err := proxywasm.GetHttpRequestBody(0, ctx.totalRequestBodySize)
	if err != nil {
		proxywasm.LogErrorf("failed to get request body: %v", err)
		return types.ActionContinue
	}

	if !validatePayload(body) {
		// If the validation fails, send the 403 response,
		if err := proxywasm.SendHttpResponse(403, nil, []byte("invalid payload"), -1); err != nil {
			proxywasm.LogErrorf("failed to send the 403 response: %v", err)
		}
		// and terminates this traffic.
		return types.ActionPause
	}

	return types.ActionContinue
}

// validatePayload validates the given json payload.
// Note that this function parses the json data by gjson, since TinyGo doesn't support encoding/json.
func validatePayload(body []byte) bool {
	if !gjson.ValidBytes(body) {
		proxywasm.LogErrorf("body is not a valid json: %v", body)
		return false
	}
	jsonData := gjson.ParseBytes(body)

	// Do any validation on the json. Check if required keys exist here as an example.
	for _, requiredKey := range []string{"id", "token"} {
		if !jsonData.Get(requiredKey).Exists() {
			proxywasm.LogErrorf("required key (%v) is missing: %v", requiredKey, jsonData)
			return false
		}
	}

	return true
}

Compiling to Wasm

This can now be compiled to wasm using the tinygo compiler via:

tinygo build -o main.wasm -scheduler=none -target=wasi main.go

Deploying the Wasm Plugin

Deploying to Envoy in Docker

For development this plugin can be deployed to Envoy in docker. The following Envoy configuration file will set up Envoy to listen on localhost:18000, run the provided Wasm plugin, and respond with a HTTP 200 and the text “hello from server” on success. The highlighted portion is where the Wasm plugin is configured.

static_resources:
  listeners:
    - name: main
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 18000
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: auto
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains:
                        - "*"
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: web_service
 
                http_filters:
                 - name: envoy.filters.http.wasm
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          vm_config:
                            runtime: "envoy.wasm.runtime.v8"
                            code:
                              local:
                                filename: "./main.wasm"
                  - name: envoy.filters.http.router

    - name: staticreply
      address:
        socket_address:
          address: 127.0.0.1
          port_value: 8099
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: auto
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains:
                        - "*"
                      routes:
                        - match:
                            prefix: "/"
                          direct_response:
                            status: 200
                            body:
                              inline_string: "hello from the server\n"
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config: {}

  clusters:
    - name: web_service
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: mock_service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8099

admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8001

Then run the Envoy container via:

docker run --rm -p 18000:18000 \
  -v $PWD/envoy.yaml:/envoy.yaml \
  -v $PWD/main.wasm:/main.wasm \
  --entrypoint envoy containers.istio.tetratelabs.com/proxyv2:1.9.7-tetrate-v0 \
  -l debug \
  -c /envoy.yaml

This can be tested using curl. First, with no content type, a 403 is returned as expected.

% curl -i -X POST localhost:18000
HTTP/1.1 403 Forbidden
content-length: 29
content-type: text/plain
date: Sun, 13 Mar 2022 22:13:37 GMT
server: envoy

content-type must be provided

Then, with a request body that isn’t JSON, another 403.

% curl -i -X POST localhost:18000 -H 'Content-Type: application/json' --data 'not JSON'
HTTP/1.1 403 Forbidden
content-length: 15
content-type: text/plain
date: Sun, 13 Mar 2022 22:15:53 GMT
server: envoy

invalid payload

With a JSON payload missing the token field, a further 403.

% curl -i -X POST localhost:18000 -H 'Content-Type: application/json' --data '{"id": "xxx"}'
HTTP/1.1 403 Forbidden
content-length: 15
content-type: text/plain
date: Sun, 13 Mar 2022 22:17:18 GMT
server: envoy

invalid payload

When both the id and token fields are provided a successful response is returned.

% curl -i -X POST localhost:18000 -H 'Content-Type: application/json' --data '{"id": "xxx", "token": "xxx", "anotherField": "yyy"}'
HTTP/1.1 200 OK
content-length: 22
content-type: text/plain
date: Sun, 13 Mar 2022 22:18:37 GMT
server: envoy
x-envoy-upstream-service-time: 1

hello from the server

Deploying to Istio

Set Up a Cluster with Istio and the httpbin Sample App

For this example I’m using kind to create a test cluster, but this will work the same for any other Kubernetes cluster. To create a cluster using kind:

kind create cluster

Once your cluster is created, install Istio (I’m using version 1.12.3) and the Istio httpbin sample app.

istioctl install --set profile=demo
kubectl label namespace default istio-injection=enabled
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/httpbin/httpbin.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/httpbin/httpbin-gateway.yaml

In another terminal, forward port 80 of the ingress gateway to port 8080 on your local machine.

kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80

Let’s check the service has been set up correctly by sending a curl request and you should see a successful response.

curl -X POST -i https://localhost:8080/post

Now there are two ways to install a wasm module in Istio. Istio 1.12 and newer supports the WasmPlugin resource. For older Istio versions an EnvoyFilter is needed instead.

Install using WasmPlugin

The WasmPlugin resource pulls the wasm module from a container registry. So let’s first build and push a docker image for our wasm module. The following Dockerfile allows building a docker image from your wasm module.

FROM scratch

COPY main.wasm ./

Then build and push the image to your container registry.

export HUB=your_registry # e.g. docker.io/tetrate
docker build . -t $HUB/json-validation:v1
docker push $HUB/json-validation:v1

Now we create the WasmPlugin resource. This will apply to all routes exposed via the Istio ingress gateway and apply our validation to them. Ensure you replace {your_registry} with the container registry that you uploaded the wasm image to.

apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: json-validation
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  url: oci://{your_registry}/json-validation:v3
  imagePullPolicy: IfNotPresent
  phase: AUTHN

Install using EnvoyFilter

To use an EnvoyFilter, we create a config map containing the compiled wasm plugin, mount the config map into the gateway pod, and then configure Envoy via an EnvoyFilter to load the wasm plugin from a local file. A limitation of this approach is that larger and more complicated Wasm modules may not fit in the config map size limit of 1MiB.

First, create a config map containing the compiled wasm module.

kubectl -n istio-system create configmap wasm-plugins --from-file=main.wasm

Then patch the Istio ingress gateway deployment to mount this config map.

kubectl -n istio-system patch deployment istio-ingressgateway --patch='
spec:
  template:
    spec:
      containers:
        - name: istio-proxy
          volumeMounts:
            - name: wasm-plugins
              mountPath: /var/local/lib/wasm-plugins
              readOnly: true
      volumes:
        - name: wasm-plugins
          configMap:
            name: wasm-plugins'

Now that the wasm module is mounted into the gateway pod, apply an EnvoyFilter to use it.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: json-validation
  namespace: istio-system
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
    patch:
      operation: INSERT_BEFORE
      value:
        name: json-validation
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          config:
            vm_config:
              code:
                local:
                  filename: /var/local/lib/wasm-plugins/main.wasm
              runtime: envoy.wasm.runtime.v8
              vm_id: json-validation

Testing the Wasm Plugin

Let’s repeat our curl request from before.

% curl -X POST -i https://localhost:8080/post
HTTP/1.1 403 Forbidden
content-length: 29
content-type: text/plain
date: Tue, 15 Mar 2022 22:04:35 GMT
server: istio-envoy

content-type must be provided

Now the request will only succeed if the content type and json payload are provided.

curl -i https://localhost:8080/post  -H 'Content-Type: application/json' --data '{"id": "xxx", "token": "xxx"}'

Making the Required Fields Configurable

Instead of hard coding the required JSON fields in the compiled golang code, it can be useful to allow configuring these via Envoy configuration.

When running with Envoy in docker, this can be done by adding configuration to the Wasm http_filter created earlier.

http_filters:
                  - name: envoy.filters.http.wasm
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          configuration:
                            "@type": type.googleapis.com/google.protobuf.StringValue
                            value: |
                              { "requiredKeys": ["id", "token"] }
                          vm_config:
                            runtime: "envoy.wasm.runtime.v8"
                            code:
                              local:
                                filename: "./main.wasm"

When run in Istio using a WasmPlugin, they should instead be set in the pluginConfig field.

apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: json-validation
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  url: oci://{your_registry}/json-validation:v3
  imagePullPolicy: IfNotPresent
  phase: AUTHN
  pluginConfig:
    requiredKeys: ["id", "token"]

Lastly, when run in Istio using an EnvoyFilter, add it to the filter configuration.

 value:
        name: json-validation
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          config:
            configuration:
              "@type": type.googleapis.com/google.protobuf.StringValue
              value: |
                { "requiredKeys": ["id", "token"] }
            vm_config:
              code:
                local:
                  filename: /var/local/lib/wasm-plugins/main.wasm
              runtime: envoy.wasm.runtime.v8
              vm_id: json-validation

To use these in code, implement OnPluginStart and use proxywasm.GetPluginConfiguration to load them.

// pluginContext implements types.PluginContext interface of proxy-wasm-go SDK.
type pluginContext struct {
	// Embed the default plugin context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultPluginContext
	configuration *pluginConfiguration
}

// pluginConfiguration is a type to represent an example configuration for this wasm plugin.
type pluginConfiguration struct {
	// Example configuration field.
	// The plugin will validate if those fields exist in the json payload.
	requiredKeys []string
}

// Override types.DefaultPluginContext.
func (ctx *pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus {
	data, err := proxywasm.GetPluginConfiguration()
	if err != nil {
		proxywasm.LogCriticalf("error reading plugin configuration: %v", err)
		return types.OnPluginStartStatusFailed
	}
	config, err := parsePluginConfiguration(data)
	if err != nil {
		proxywasm.LogCriticalf("error parsing plugin configuration: %v", err)
		return types.OnPluginStartStatusFailed
	}
	ctx.configuration = config
	return types.OnPluginStartStatusOK
}

// parsePluginConfiguration parses the json plugin configuration data and returns pluginConfiguration.
// Note that this parses the json data by gjson, since TinyGo doesn't support encoding/json.
// You can also try https://github.com/mailru/easyjson, which supports decoding to a struct.
func parsePluginConfiguration(data []byte) (*pluginConfiguration, error) {
	config := &pluginConfiguration{}
	if !gjson.ValidBytes(data) {
		return nil, fmt.Errorf("the plugin configuration is not a valid json: %v", data)
	}

	jsonData := gjson.ParseBytes(data)
	requiredKeys := jsonData.Get("requiredKeys").Array()
	for _, requiredKey := range requiredKeys {
		config.requiredKeys = append(config.requiredKeys, requiredKey.Str)
	}

	return config, nil
}

Now that they are included in the pluginConfiguration struct, they can be used during validation like any other field.

// validatePayload validates the given json payload.
// Note that this function parses the json data by gjson, since TinyGo doesn't support encoding/json.
func (ctx *payloadValidationContext) validatePayload(body []byte) bool {
	if !gjson.ValidBytes(body) {
		proxywasm.LogErrorf("body is not a valid json: %v", body)
		return false
	}
	jsonData := gjson.ParseBytes(body)

	// Do any validation on the json. Check if required keys exist here as an example.
	// The required keys are configurable via the plugin configuration.
	for _, requiredKey := range ctx.requiredKeys {
		if !jsonData.Get(requiredKey).Exists() {
			proxywasm.LogErrorf("required key (%v) is missing: %v", requiredKey, jsonData)
			return false
		}
	}

	return true
}

This can then be compiled and tested using the same commands as before.

In Summary

To summarize, to use a Wasm plugin on Istio 1.12 and newer requires three steps:

1. Implement the plugin functionality in the language of your choice. I used Golang in this tutorial.
2. Compile the Wasm plugin and push to a container registry.
3. Configure Istio to load and use the plugin from the registry.

The tutorial also detailed how to run a Wasm plugin in an Envoy container using Docker for faster development, and how to deploy it into older Istio versions.

Appendix

Full sources: https://github.com/tetratelabs/proxy-wasm-go-sdk/tree/main/examples/json_validation