In his 2019 talks at KubeCon Barcelona, Tetrate Engineer and Envoy Senior Maintainer Lizan Zhou presented an overview of Envoy and a deep dive into its extensibility. The service proxy solves a host of operational problems related to observability and networking in large distributed systems, and its extensibility allows it to be adapted to a large variety of end use cases. Tetrate’s GetEnvoy, which provides enterprise with certified and tested Envoy proxy builds, launches next week.
Large organizations have moved en masse toward microservice architectures to meet their business requirements. But the shift has been messy.
The problem
If you’re running a large, distributed architecture, you’ve probably encountered operational problems related to controlling, securing and monitoring a system with very heterogeneous components.
At this high level, we’re going to walk through Envoy and the problem it solves as it was articulated by Envoy’s creator, Matt Klein.
You’re likely using multiple languages and frameworks, and your networking may be handled by per language libraries to make service calls. You’re maybe using multiple protocols, databases, and caching layers, different types of physical and virtual infrastructures, and multiple load balancers. Your observability outputs are inconsistent. Library upgrades can be incredibly painful. In such abstruse systems, authentication, authorization, and implementation of best practices like retries, circuit breaking and rate limiting will be spotty. Developers lose their trust in the network. And if you can’t actively understand what’s going on in your system, debugging is perhaps the biggest disaster of all.
The solution is in the mesh
The basic purpose of Envoy is to push all of that complexity out of your applications to be handled in the space between your applications. Originally built at Lyft, Envoy is a C++ distributed proxy that runs alongside each application and abstracts the network, providing common, platform-agnostic features. Envoy is also a data plane that is specially designed for large service mesh architectures. Envoy sidecar proxies deployed to your applications or as an edge proxy can be controlled using Istio or another control plane. With all service traffic flowing through an Envoy mesh, it becomes possible to consistently control and observe what’s going on in your network. Developers can deliver services quickly and continuously using their choice of languages and technologies, and their focus can be on the business logic of their applications. The Envoy sidecar proxy will handle most network features– service discovery, logging, monitoring, tracing, authentication and authorization– configured via the control plane.
Envoy’s popularity belies its short history. In just a few years– from its creation in 2015 to its 2018 graduation from the CNCF– it’s grown a devoted open source community and has been adopted by an array of tech giants– cloud providers, large internet providers, and startups that are building businesses on top of it.
Envoy’s high-level features
Out of process architecture: Envoy is not a library. It was designed to run as a self-contained service proxy, or sidecar, that runs alongside every application server. Applications exchange messages with the localhost and are unaware of the network topology. This means that it works with any application language. And, in contrast to the library approach to service communications, Envoy can be deployed and upgraded transparently.
High-performance, low-latency code base: Envoy is written in C++14, which will allow very high performance and especially low tail latency.
L3/L4 filter architecture: Envoy is an L3/L4, or byte-oriented network proxy with a pluggable filter mechanism that have been written to support tasks like TLS client certificate authentication, RBAC, and API management. It can be used for HTTP, TCP, Mongo MySQL, Redis, Dubbo, and others.
HTTP L7 routing and filter architecture: In HTTP mode, Envoy supports a routing subsystem that can route and redirect requests based on path, authority, content type, runtime values, and more that’s an especially powerful tool in a service mesh. Envoy supports an additional HTTP L7 filter layer for an especially rich set of tasks, such as buffering, rate limiting, and routing/forwarding.
First class HTTP/2 support: Envoy was built with HTTP/2 first, so it supports both HTTP/1.1 and HTTP/2, and can bridge any combination of HTTP clients and target servers. HTTP/2 is not just for edge. It can also provide efficiency for service-to-service calls. Envoy supports gRPC both upstream and downstream, and also supports MongoDB and DynamoDB.
Service discovery and health checking: Envoy has rich features for service discovery (what are your backend hosts?) and health checking (are those hosts healthy?). It can optionally consume a layered set of dynamic configuration APIs for centralized management, but for simpler deployment, backend host discovery can be done through DNS resolution and static config files. Envoy supports consistent service discovery and both active and passive health checking, and can use service discovery and health checking information to determine healthy load balancing targets.
Advanced load balancing: Envoy’s out of process architecture enables it to implement advanced load balancing techniques for any application in a single place. This includes automatic retries, rate limiting, request shadowing, outlier detection, circuit breaking, etc.
Observability: The primary goal of Envoy is to make the network transparent, providing rich stats output, logging and tracing. Envoy includes statistics support for all subsystems. Envoy also supports distributed tracing via third-party vendors. Tetrate recommends SkyWalking, which was built especially for service mesh. By running Envoy throughout the organization, you get more meaningful, apples-to-apples metrics.
Authentication and authorization: Mutual TLS is the basic authentication for the service-to-service call. Envoy also supports RBAC and JWT authentication.
((Front/edge proxy support:** With features like TLS termination, H1/H2 support, and L7 routing, Envoy can be used as an edge proxy for most web application use cases.
Configuration
Envoy serves as a universal data plane and its configuration is managed by the control plane. It discovers its dynamic resources via the filesystem or by querying one or more management servers. These discovery services and their corresponding configuration APIs are referred to as xDS. Each xDS API is concerned with resources of a given type, including:
listener (LDS) cluster (CDS) route (RDS) endpoint (EDS) and TLS certificates that are secret (SDS).
Envoy supports gRPC streaming and JSON/YAML REST via proto3. The central management system allows users to control a fleet of Envoys and avoid the tedium of configuring individual proxies. A global, standard bootstrap config is used for every Envoy, and the rest is taken care of by the management server. The control plane can push a new configuration to each Envoy.
Envoy’s extension system
Extensibility is a key feature of Envoy. As you can see in the diagram, beginning from the workers, each request runs through a stack of filters: listener filters, connection filter chains, L3 and L4 filters– to HTTP filters if it’s a protocol Envoy can operate as an L7 layer– and finally on to the routers and backend services. The same chain, vice versa, applies in the response from backend services through the filters, the listener and finally to the downstream clients. These filters are powerful and configurable. They can modify data at the L4 layer, insert HTTP headers, check and guide traffic, call out to an auth service, or transcode between protocols. Custom filters can be added via the data plane API, and Envoy has a plug-in architecture for observability outputs.
Envoy is designed to have multiple extension points: L4/L7 filters Acccess loggers Tracers Health checkers Transport sockets Retry policy Resource monitors Stats sink
Envoy’s extension repo is prodigious and updated frequently. TAP is a newly introduced extension with a socket capture feature that allows you to capture and debug at the L4 or L7 layer; you open a request to see traffic that meets given conditions and get a sample of the traffic from the admin endpoints. MySQL, Kafka and Dubbo are three new L4 filters introduced this year, adding support for those protocols.
Custom extensions
You can write your own extension using C++, in two parts: the config part and the real implementation. The config part is hooked into the Envoy config parsing process. That tells Envoy that this name belongs to this extension. Envoy will take the configuration to initiate your extension with the implementation. This is typically done in protocol buffers (protobuf). Envoy doesn’t currently provide high-level extension developer documentation, but provides an example of how to add a network filter, structure the repository and build dependencies. In the example, the build file introduces a new Envoy static binary target so that Envoy knows how to initiate the new filter.
In the simple example above, the Gzip filter compresses the response Envoy receives downstream from the backend service. The config is created from the protobuf, and the logic of the encodeData, a key part of the filter, is here:
It sends the buffer to the compressor. The data is modified by the compresser and it will send it back downstream. Based on these Boolean conditions, the compression will be skipped when a request does not contain accept-encoding header.
Other ways to extend
The Lua filter is now only operated on HTTP. You can write and embed Lua scripts in the Envoy config, and the Lua filter will allow you to do header and data modification. WebAsssembly work in also ongoing in a separate WASM repo. For details on this work, check out the Envoy+WASM session from KubeCon Barcelona 2019 by Tetrate’s Dhi Aurrahman. There is also a proposal to do out of process extensions via gRPC– but we’re still evaluating those options.
GetEnvoy
Tetrate, the provider of enterprise-ready service mesh, offers GetEnvoy to organizations that want to have Envoy proxies built for them by service mesh experts, with tested and certified packages, binaries and images.
