WasmPlugin API has recently been added to the Istio project as a new and improved mechanism for extensibility. Here at Tetrate, we’ve recently conducted a successful workshop called Istio Wasm workshop. Click here to watch the recording of the workshop and join the conversation on Slack.
We talked about WebAssembly and its importance in Istio and Envoy projects, and went through multiple practical labs using Proxy-Wasm Go SDK and func-e.
We here at Tetrate have been concerned about Istio extensibility for a long time. Tetrate engineers Takeshi Yoneda and Lizan Zhou have been instrumental in making the changes described in this post possible, and we’re very happy to see Istio extensibility improving dramatically as a result.
In this blog post, I describe the status of Istio and Envoy extensibility before the introduction of WasmPlugin API; the current, much-improved situation; and the changes that will more or less complete this arc of extensibility improvements, which we expect in upcoming releases that should arrive soon.
History of Istio and Wasm
| Pre- Istio 1.4 | Istio 1.5 | Istio 1.12 and future |
|---|---|---|
| Maintain your own Envoy proxy build with C++ extension | New Wasm extensibility model introduced using the EnvoyFilter resource (still complex) | Dedicated WasmPlugin API introduced |
| Uses Mixer (inefficient) | Supports local or HTTP locations only | Includes support for OCI registries |
Before Istio 1.4 (released in November 2019), there was no good mechanism to run plugins. At that time, Istio maintained their own fork of the Envoy proxy to run custom plugins, such as the RBAC and JWT filters written in C++ and built together with the Envoy proxy.
At the time, Istio was using a Mixer component that provided a layer between the application code and the infrastructure backends. Using Mixer, one was able to enforce auth policies, collect telemetry, and manage quotas. In this model, Envoy proxies called out to the Mixer component before making requests to the backends to perform any precondition checks – e.g., “can service A call service B” – and called Mixer again after each request was completed to report the telemetry. Using this model led to inefficient resource usage, and impacted latencies as well.
The extensibility in Envoy relied on developers knowing how to write extensions in C++. Additionally, any C++ extensions had to be built with the Envoy proxy. That’s the reason why Istio maintained their own Envoy proxy build at the time.
In this model, one had to replace the entire fleet of existing Envoy proxy instances with new binaries that included the updated extension whenever an extension was updated.
History of Envoy and Wasm
There was a desire from the Envoy side to converge on a single extensibility stack that allows decoupling Envoy releases from the extensions ecosystem and enabling extension developers to use something other than C++. Adding this functionality to Envoy would consequently let Istio (and any other Envoy proxy users) roll out new features and capabilities whenever they wanted to, without the need of maintaining a separate Envoy build. With strong motivation from both the Envoy and Istio sides, the work to support WebAssembly in Envoy started in 2018.
WebAsssembly introduced in Envoy
The hard work on extensibility that was done in both Envoy and Istio was included in the Istio 1.5 release (in 2020). The Istio 1.5 release included a new model for extensibility that uses WebAssembly. With the introduction of WebAssembly, the need to run a separate Mixer component was gone, which also resulted in simplified Istio deployments – one less thing to deploy, and one less thing to worry about.
The release included the generic Application Binary Interface (ABI) and the C++, Rust, and AssemblyScript SDKs. The existing Istio functionality such as stats, metadata exchange, and others was also implemented as WebAssembly extensions. A resource called EnvoyFilter was introduced to, amongst other things, handle the deployment of these extensions.
The EnvoyFilter resource was a very light abstraction over the Envoy configuration, and it allowed Istio operators to modify Envoy proxy behavior. However, the light abstraction still meant the operators had to be familiar with details of the Envoy APIs and how to configure them effectively.
To have Envoy proxies load and use the extensions, you had to bring the Wasm binary inside the same pod as the Envoy proxy container. Operators had an option of either pointing to a local file (i.e., a file inside the cluster, accessible to the Envoy proxy container) or providing an HTTP location where the proxy could download the extension from.
There were multiple improvements in subsequent releases of Istio, notably the change where the Istio agent started intercepting EnvoyFilter resources and fetching the binaries on behalf of the Envoy proxy.
Istio 1.12 and WasmPlugin API
The most significant breakthrough features were introduced recently in Istio 1.12. A dedicated API for Wasm plugins called the WasmPlugin API is introduced that uses a new method to fetch the Wasm binaries from an OCI-compliant registry.
The introduction of the new API removes the need to use EnvoyFilter for deploying the extensions. Extension developers can now use a resource called WasmPlugin to specify the workloads where the plugins are to be deployed. The new support for OCI-compliant registries allows developers to use existing tools (e.g., Docker) to build images containing their extensions and push them to OCI-compliant registries. This allows treating the Wasm plugins in the same way container images are treated – for example, using image tags and different repositories.
Conclusion
There’s still work in progress on the WasmPlugin API. Some features – including support for image pull secrets, better image cache support, and support for singleton extensions – are not available yet.
To get started with Wasm, watch the recording of the Wasm workshop and join the Wasm conversation on Slack.
###
If you’re new to service mesh, Tetrate has a bunch of free online courses available at Tetrate Academy that will quickly get you up to speed with Istio and Envoy.
Are you using Kubernetes? Tetrate Enterprise Gateway for Envoy (TEG) is the easiest way to get started with Envoy Gateway for production use cases. Get the power of Envoy Proxy in an easy-to-consume package managed by the Kubernetes Gateway API. Learn more ›
Getting started with Istio? If you’re looking for the surest way to get to production with Istio, check out Tetrate Istio Subscription. Tetrate Istio Subscription has everything you need to run Istio and Envoy in highly regulated and mission-critical production environments. It includes Tetrate Istio Distro, a 100% upstream distribution of Istio and Envoy that is FIPS-verified and FedRAMP ready. For teams requiring open source Istio and Envoy without proprietary vendor dependencies, Tetrate offers the ONLY 100% upstream Istio enterprise support offering.
Need global visibility for Istio? TIS+ is a hosted Day 2 operations solution for Istio designed to simplify and enhance the workflows of platform and support teams. Key features include: a global service dashboard, multi-cluster visibility, service topology visualization, and workspace-based access control.
Get a Demo