Advanced Request and Response Processing in Microservice Architecture with Envoy’s External Processing Filter
In a microservices architecture, API gateways often need to perform high-level request and response processing tasks such as authentication, data tran

In a microservices architecture, API gateways often need to perform high-level request and response processing tasks such as authentication, data transformation, and security checks. Envoy’s ext_proc external processing filter is a powerful tool that enables flexible request and response handling by interacting with a gRPC service. This article delves into its features, configuration, and performance optimization strategies to help developers and DevOps engineers effectively utilize this capability.
Tetrate offers an enterprise-ready, 100% upstream distribution of Istio, Tetrate Istio Subscription (TIS). TIS is the easiest way to get started with Istio for production use cases. TIS+, a hosted Day 2 operations solution for Istio, adds a global service registry, unified Istio metrics dashboard, and self-service troubleshooting.
Relationship Between ext_proc
and Other Filters
The ext_proc filter shares similarities with other gRPC-based filters in Envoy, such as ext_authz. However, ext_proc offers more extensive capabilities, supporting full request and response processing. This makes it particularly suitable for scenarios requiring deep content inspection and modification.
How ext_proc
Works and Its Configuration
Definition and Functionality
ext_proc is an Envoy HTTP filter that delegates request and response processing to a gRPC service, allowing complex logic implementation in external services to meet specific business requirements.
Working Mechanism
ext_proc uses a bidirectional gRPC stream to communicate with an external service, enabling real-time interaction for request and response processing. This allows Envoy to offload complex tasks such as authentication, data transformation, and custom header manipulation to an external service, improving flexibility and scalability. Envoy sends ProcessingRequest messages, and the external service returns ProcessingResponse messages. Importantly, this gRPC stream is created per HTTP request stream; it is not shared among multiple requests. Each HTTP request handled by Envoy creates its own dedicated gRPC stream, ensuring isolation and precise request-response management. This design allows the external service to intervene at various stages of the request and response lifecycle, even generating entirely new responses.
Here’s a sequence diagram illustrating the process:
Key Features:
- Request and Response Handling: Read and modify HTTP request and response headers, bodies, and trailers.
- Flexibility: Define custom logic based on business needs, extending built-in functionality.
- Asynchronous Processing: Support for asynchronous processing to prevent request blocking.
Envoy Configuration Example
Below is a basic Envoy configuration example:
static_resources:
listeners:
- name: listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
log_format:
text_format: "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%RESP(X-EXTPROC-HELLO)%\" \"%RESP(CONTENT-TYPE)%\" \"%RESP(CONTENT-LENGTH)%\" %DURATION% ms\n"
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: [ "\*" ]
routes:
- match:
prefix: "/"
route:
host_rewrite_literal: www.envoyproxy.io
cluster: service_envoyproxy_io
http_filters:
- name: envoy.filters.http.ext_proc
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
grpc_service:
envoy_grpc:
cluster_name: ext_proc_cluster
failure_mode_allow: true
processing_mode:
request_header_mode: SKIP
response_header_mode: SEND
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: ext_proc_cluster
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
load_assignment:
cluster_name: ext_proc_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 9000
- name: service_envoyproxy_io
type: LOGICAL_DNS
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: service_envoyproxy_io
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: www.envoyproxy.io
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
sni: www.envoyproxy.io
To understand the connection between Envoy configuration and a gRPC service, consider how the following configuration items influence traffic processing:
- grpc_service: Defines the target address and cluster name for communication with the gRPC service, corresponding to ext_proc_cluster in Envoy’s configuration.
- processing_mode: Controls when to trigger processing stages such as request headers, request bodies, and response headers, determining when to invoke the gRPC service.
- failure_mode_allow: Specifies whether to continue request processing when the gRPC service fails, ensuring high availability in partial failure scenarios.
- listeners: Defines the network address and port where Envoy listens for incoming requests.
- filter_chains: Configures the request processing chain, including the HTTP connection manager and external processing filters.
- http_filters: Lists the enabled filters, such as ext_proc and router.
- clusters: Defines upstream services and the location of the external processing gRPC service.
You can see the configuration instructions in the Envoy documentation.
Envoy uses these configuration options to offload request and response processing to the gRPC service. The results are returned through a bidirectional streaming protocol, influencing request forwarding behavior.
gRPC Service Example
Below is a simple gRPC external processing server implementation demonstrating how to add custom response headers using ext_proc. This example highlights key method choices and design decisions, such as using the Process method to continuously receive requests and send responses, ensuring seamless processing. It also leverages the HeaderMutation configuration to modify HTTP response headers, illustrating the tight integration between gRPC messages and Envoy’s configuration for dynamic extension and flexible management.
package main
import (
"log"
"net"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
configPb "github.com/envoyproxy/go-control-plane/envoy/config/core/v3"
extProcPb "github.com/envoyproxy/go-control-plane/envoy/service/ext_proc/v3"
)
type extProcServer struct {
extProcPb.UnimplementedExternalProcessorServer
}
// Process handles external processing requests from Envoy.
// It listens for incoming requests, modifies response headers,
// and sends the updated response back to Envoy.
//
// When a request with response headers is received, it adds a custom header
// "x-extproc-hello" with the value "Hello from ext_proc" and returns the modified headers.
//
// Note: The `RawValue` field is used instead of `Value` because it supports
// setting the header value as a byte slice, allowing precise handling of binary data.
//
// This function is called once per HTTP request to process gRPC messages from Envoy.
// It exits when an error occurs while receiving or sending messages.
func (s \*extProcServer) Process(
srv extProcPb.ExternalProcessor_ProcessServer,
) error {
for {
req, err := srv.Recv()
if err != nil {
return status.Errorf(codes.Unknown, "error receiving request: %v", err)
}
log.Printf("Received request: %+v\n", req)
// Prepare the response to be returned to Envoy.
resp := &extProcPb.ProcessingResponse{}
// Only process response headers, not request headers.
if respHeaders := req.GetResponseHeaders(); respHeaders != nil {
log.Println("Processing Response Headers...")
resp = &extProcPb.ProcessingResponse{
Response: &extProcPb.ProcessingResponse_ResponseHeaders{
ResponseHeaders: &extProcPb.HeadersResponse{
Response: &extProcPb.CommonResponse{
HeaderMutation: &extProcPb.HeaderMutation{
SetHeaders: []\*configPb.HeaderValueOption{
{
Header: &configPb.HeaderValue{
Key: "x-extproc-hello",
RawValue: []byte("Hello from ext_proc"),
},
},
},
},
},
},
},
}
log.Printf("Sending response: %+v\n", resp)
// Send the response back to Envoy.
if err := srv.Send(resp); err != nil {
return status.Errorf(codes.Unknown, "error sending response: %v", err)
}
} else {
// If it is not a callback in the response header stage, do not make any modifications and continue processing the next event.
// For request_headers or other events, do not modify & ensure that Envoy will not be stuck.
// An empty processing can be returned for request_headers, or it can be skipped in envoy.yaml.
// Here, simply continue to wait for the next event.
continue
}
}
}
func main() {
lis, err := net.Listen("tcp", ":9000")
if err != nil {
log.Fatalf("Failed to listen: %v", err)
}
grpcServer := grpc.NewServer()
// Register the ExternalProcessorServer with the gRPC server.
extProcPb.RegisterExternalProcessorServer(grpcServer, &extProcServer{})
log.Println("Starting gRPC server on :9000...")
if err := grpcServer.Serve(lis); err != nil {
log.Fatalf("Failed to serve: %v", err)
}
}
Running Envoy and the gRPC service locally, you can test with:
envoy -c envoy.yaml
go run main.go
curl -v http://localhost:8080
You’ll see a response including the custom header x-extproc-hello: Hello from ext_proc.


Expected Result: The request returns a status code of 200, with a custom header x-extproc-hello: Hello from ext_proc in the response. If the header is missing, check the following:
- Is the gRPC service running? Ensure the gRPC server has started and is listening on port :9000.
- Is the Envoy configuration correct? Verify the Envoy configuration file to ensure the ext_proc filter is enabled and that the ext_proc_cluster configuration is correct.
- Logs and Error Investigation: Check the logs of both Envoy and the gRPC server for potential errors.
Use Cases and Optimization Strategies
Common Use Cases
- Authentication: External authentication services check user credentials.
- Data Auditing: Log request and response data for compliance purposes.
- Traffic Management: Adjust traffic routing based on dynamic analysis.
Configuration Tips and Optimization Strategies
- Failure Recovery and Load Balancing: Deploy multiple instances of the gRPC service with load balancing.
- Message Timeouts and Retries:
http_filters:
- name: envoy.filters.http.ext_proc
config:
grpc_service:
envoy_grpc:
cluster_name: ext_proc_server
processing_mode:
request_header_mode: SEND
response_header_mode: SEND
message_timeout: 500ms
max_message_timeout: 1000ms
failure_mode_allow: false
- Metadata Options and Security Policies:
metadata_options:
forwarding_namespaces:
untyped: ["custom_namespace"]
Conclusion
Envoy’s ext_proc filter provides robust support for service governance, data transformation, and request inspection through flexible request and response processing capabilities. Correctly configuring and optimizing ext_proc can significantly enhance system flexibility and scalability to meet diverse business requirements.