In a microservices architecture, API gateways often need to perform high-level request and response processing tasks such as authentication, data transformation, and security checks. Envoy’s ext_proc external processing filter is a powerful tool that enables flexible request and response handling by interacting with a gRPC service. This article delves into its features, configuration, and performance optimization strategies to help developers and DevOps engineers effectively utilize this capability.
Tetrate offers an enterprise-ready, 100% upstream distribution of Envoy Gateway, Tetrate Enterprise Gateway for Envoy (TEG). TEG is the easiest way to get started with Envoy for production use cases. Get access now ›
Relationship Between ext_proc and Other Filters
The ext_proc filter shares similarities with other gRPC-based filters in Envoy, such as ext_authz. However, ext_proc offers more extensive capabilities, supporting full request and response processing. This makes it particularly suitable for scenarios requiring deep content inspection and modification.
How ext_proc Works and Its Configuration
Definition and Functionality
ext_proc is an Envoy HTTP filter that delegates request and response processing to a gRPC service, allowing complex logic implementation in external services to meet specific business requirements.
Working Mechanism
ext_proc uses a bidirectional gRPC stream to communicate with an external service, enabling real-time interaction for request and response processing. This allows Envoy to offload complex tasks such as authentication, data transformation, and custom header manipulation to an external service, improving flexibility and scalability. Envoy sends ProcessingRequest messages, and the external service returns ProcessingResponse messages. Importantly, this gRPC stream is created per HTTP request stream; it is not shared among multiple requests. Each HTTP request handled by Envoy creates its own dedicated gRPC stream, ensuring isolation and precise request-response management. This design allows the external service to intervene at various stages of the request and response lifecycle, even generating entirely new responses.
Here’s a sequence diagram illustrating the process:
Key Features:
- Request and Response Handling: Read and modify HTTP request and response headers, bodies, and trailers.
- Flexibility: Define custom logic based on business needs, extending built-in functionality.
- Asynchronous Processing: Support for asynchronous processing to prevent request blocking.
Envoy Configuration Example
Below is a basic Envoy configuration example:
static_resources:
listeners:
- name: listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
log_format:
text_format: "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%RESP(X-EXTPROC-HELLO)%\" \"%RESP(CONTENT-TYPE)%\" \"%RESP(CONTENT-LENGTH)%\" %DURATION% ms\n"
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: [ "*" ]
routes:
- match:
prefix: "/"
route:
host_rewrite_literal: www.envoyproxy.io
cluster: service_envoyproxy_io
http_filters:
- name: envoy.filters.http.ext_proc
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
grpc_service:
envoy_grpc:
cluster_name: ext_proc_cluster
failure_mode_allow: true
processing_mode:
request_header_mode: SKIP
response_header_mode: SEND
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: ext_proc_cluster
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
load_assignment:
cluster_name: ext_proc_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 9000
- name: service_envoyproxy_io
type: LOGICAL_DNS
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: service_envoyproxy_io
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: www.envoyproxy.io
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
sni: www.envoyproxy.io
To understand the connection between Envoy configuration and a gRPC service, consider how the following configuration items influence traffic processing:
- grpc_service: Defines the target address and cluster name for communication with the gRPC service, corresponding to ext_proc_cluster in Envoy’s configuration.
- processing_mode: Controls when to trigger processing stages such as request headers, request bodies, and response headers, determining when to invoke the gRPC service.
- failure_mode_allow: Specifies whether to continue request processing when the gRPC service fails, ensuring high availability in partial failure scenarios.
- listeners: Defines the network address and port where Envoy listens for incoming requests.
- filter_chains: Configures the request processing chain, including the HTTP connection manager and external processing filters.
- http_filters: Lists the enabled filters, such as ext_proc and router.
- clusters: Defines upstream services and the location of the external processing gRPC service.
You can see the configuration instructions in the Envoy documentation.
Envoy uses these configuration options to offload request and response processing to the gRPC service. The results are returned through a bidirectional streaming protocol, influencing request forwarding behavior.
gRPC Service Example
Below is a simple gRPC external processing server implementation demonstrating how to add custom response headers using ext_proc. This example highlights key method choices and design decisions, such as using the Process method to continuously receive requests and send responses, ensuring seamless processing. It also leverages the HeaderMutation configuration to modify HTTP response headers, illustrating the tight integration between gRPC messages and Envoy’s configuration for dynamic extension and flexible management.
package main
import (
"log"
"net"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
configPb "github.com/envoyproxy/go-control-plane/envoy/config/core/v3"
extProcPb "github.com/envoyproxy/go-control-plane/envoy/service/ext_proc/v3"
)
type extProcServer struct {
extProcPb.UnimplementedExternalProcessorServer
}
// Process handles external processing requests from Envoy.
// It listens for incoming requests, modifies response headers,
// and sends the updated response back to Envoy.
//
// When a request with response headers is received, it adds a custom header
// "x-extproc-hello" with the value "Hello from ext_proc" and returns the modified headers.
//
// Note: The `RawValue` field is used instead of `Value` because it supports
// setting the header value as a byte slice, allowing precise handling of binary data.
//
// This function is called once per HTTP request to process gRPC messages from Envoy.
// It exits when an error occurs while receiving or sending messages.
func (s *extProcServer) Process(
srv extProcPb.ExternalProcessor_ProcessServer,
) error {
for {
req, err := srv.Recv()
if err != nil {
return status.Errorf(codes.Unknown, "error receiving request: %v", err)
}
log.Printf("Received request: %+v\n", req)
// Prepare the response to be returned to Envoy.
resp := &extProcPb.ProcessingResponse{}
// Only process response headers, not request headers.
if respHeaders := req.GetResponseHeaders(); respHeaders != nil {
log.Println("Processing Response Headers...")
resp = &extProcPb.ProcessingResponse{
Response: &extProcPb.ProcessingResponse_ResponseHeaders{
ResponseHeaders: &extProcPb.HeadersResponse{
Response: &extProcPb.CommonResponse{
HeaderMutation: &extProcPb.HeaderMutation{
SetHeaders: []*configPb.HeaderValueOption{
{
Header: &configPb.HeaderValue{
Key: "x-extproc-hello",
RawValue: []byte("Hello from ext_proc"),
},
},
},
},
},
},
},
}
log.Printf("Sending response: %+v\n", resp)
// Send the response back to Envoy.
if err := srv.Send(resp); err != nil {
return status.Errorf(codes.Unknown, "error sending response: %v", err)
}
} else {
// If it is not a callback in the response header stage, do not make any modifications and continue processing the next event.
// For request_headers or other events, do not modify & ensure that Envoy will not be stuck.
// An empty processing can be returned for request_headers, or it can be skipped in envoy.yaml.
// Here, simply continue to wait for the next event.
continue
}
}
}
func main() {
lis, err := net.Listen("tcp", ":9000")
if err != nil {
log.Fatalf("Failed to listen: %v", err)
}
grpcServer := grpc.NewServer()
// Register the ExternalProcessorServer with the gRPC server.
extProcPb.RegisterExternalProcessorServer(grpcServer, &extProcServer{})
log.Println("Starting gRPC server on :9000...")
if err := grpcServer.Serve(lis); err != nil {
log.Fatalf("Failed to serve: %v", err)
}
}
Running Envoy and the gRPC service locally, you can test with:
envoy -c envoy.yaml go run main.go curl -v http://localhost:8080
You’ll see a response including the custom header x-extproc-hello: Hello from ext_proc.
Expected Result: The request returns a status code of 200, with a custom header x-extproc-hello: Hello from ext_proc in the response. If the header is missing, check the following:
- Is the gRPC service running? Ensure the gRPC server has started and is listening on port :9000.
- Is the Envoy configuration correct? Verify the Envoy configuration file to ensure the ext_proc filter is enabled and that the ext_proc_cluster configuration is correct.
- Logs and Error Investigation: Check the logs of both Envoy and the gRPC server for potential errors.
Use Cases and Optimization Strategies
Common Use Cases
- Authentication: External authentication services check user credentials.
- Data Auditing: Log request and response data for compliance purposes.
- Traffic Management: Adjust traffic routing based on dynamic analysis.
Configuration Tips and Optimization Strategies
- Failure Recovery and Load Balancing: Deploy multiple instances of the gRPC service with load balancing.
- Message Timeouts and Retries:
http_filters:
- name: envoy.filters.http.ext_proc
config:
grpc_service:
envoy_grpc:
cluster_name: ext_proc_server
processing_mode:
request_header_mode: SEND
response_header_mode: SEND
message_timeout: 500ms
max_message_timeout: 1000ms
failure_mode_allow: false
- Metadata Options and Security Policies:
metadata_options:
forwarding_namespaces:
untyped: ["custom_namespace"]
Conclusion
Envoy’s ext_proc filter provides robust support for service governance, data transformation, and request inspection through flexible request and response processing capabilities. Correctly configuring and optimizing ext_proc can significantly enhance system flexibility and scalability to meet diverse business requirements.
References
###
If you’re new to service mesh, Tetrate has a bunch of free online courses available at Tetrate Academy that will quickly get you up to speed with Istio and Envoy.
Are you using Kubernetes? Tetrate Enterprise Gateway for Envoy (TEG) is the easiest way to get started with Envoy Gateway for production use cases. Get the power of Envoy Proxy in an easy-to-consume package managed by the Kubernetes Gateway API. Learn more ›
Getting started with Istio? If you’re looking for the surest way to get to production with Istio, check out Tetrate Istio Subscription. Tetrate Istio Subscription has everything you need to run Istio and Envoy in highly regulated and mission-critical production environments. It includes Tetrate Istio Distro, a 100% upstream distribution of Istio and Envoy that is FIPS-verified and FedRAMP ready. For teams requiring open source Istio and Envoy without proprietary vendor dependencies, Tetrate offers the ONLY 100% upstream Istio enterprise support offering.
Need global visibility for Istio? TIS+ is a hosted Day 2 operations solution for Istio designed to simplify and enhance the workflows of platform and support teams. Key features include: a global service dashboard, multi-cluster visibility, service topology visualization, and workspace-based access control.
Get a Demo