Tetrate & Bloomberg Announce the First Release of Envoy AI Gateway

Learn more › close
Tetrate Enterprise ready service mesh
Get Demo
Pricing
Home

|

Blog

|

Implementing HTTP/2 CONNECT Tunnels with Envoy: Concepts and Practical Guide

Implementing HTTP/2 CONNECT Tunnels with Envoy: Concepts and Practical Guide

| Author: Jimmy Song

Service Mesh Architecture

Article content

​​​​​​As part of my recent exploration into Istio Ambient mode, I discovered that the HTTP/2 CONNECT method is a core technology for creating tunnels that enable transparent traffic interception and forwarding. HTTP/2 CONNECT tunnel is a powerful tool that can create an efficient tunnel in an existing HTTP/2 connection for transmitting raw TCP data. This article shows how to implement the basic functions of HTTP/2 CONNECT tunnel using Envoy through a simple Demo.

What Is HTTP/2 CONNECT and HBONE Tunnel?

The HTTP/2 CONNECT method provides a standardized approach for creating tunnels to transparently transmit data. In Istio Ambient mode, it offers an efficient way for proxy data planes to communicate. HBONE (HTTP-Based Overlay Network Environment) tunnel, built on HTTP/2 CONNECT, is Istio’s implementation for transparent traffic interception and forwarding. Using HBONE, data can securely traverse the tunnel via HTTP/2, offering a sidecar-less alternative. This innovative design simplifies the management and deployment of service meshes.

HBONE, specific to Istio, is a secure tunneling protocol for communication between Istio components. Its current implementation incorporates three open standards:

  • HTTP/2
  • HTTP CONNECT
  • Mutual TLS (mTLS)

HTTP CONNECT establishes the tunnel, mTLS encrypts the connection, and HTTP/2 multiplexes streams within the secure tunnel while transmitting stream-level metadata. For more details, see the HBONE documentation.

Basic Principles of HTTP/2 CONNECT Tunnel Creation

The HTTP/2 CONNECT method creates a VPN-like tunnel for secure data transmission. The basic steps are as follows:

  1. The client sends a standard TCP connection or HTTP request to the proxy.
  2. The proxy sends an HTTP/2 CONNECT request to the target server on behalf of the client.
  3. If the server allows tunnel creation, it responds with an HTTP/2 200 OK.
  4. Data flows bidirectionally between the client, proxy, and server through the tunnel.

This method ensures secure and transparent data transmission, particularly suited for scenarios requiring efficient communication and end-to-end encryption.

Below is a diagram showing the basic process of creating a tunnel using the HTTP/2 CONNECT method:

Figure 1: The basic process of establishing a tunnel using the HTTP2 Connect method.

Demo: Establishing an HTTP/2 CONNECT Tunnel with Envoy

In this demo:

  • The client sends a text message to the Envoy proxy, which forwards it to the server through an HTTP/2 CONNECT tunnel.
  • The server processes and replies to the message, with Envoy facilitating secure communication.
  • The client remains unaware of the tunnel’s existence, experiencing a seamless connection.

Below is the architecture for this demo. To simplify the setup, we will configure TLS only for the server:

Figure 2: Architecture diagram

Directory Structure

The complete directory structure of this example is as follows:

envoy-http2-tunnel/
├── certs/
│   ├── openssl.cnf
│   ├── server.crt
│   ├── server.key
├── client/
│   └── client.js
├── docker-compose.yml
├── envoy.yaml
└── server/
	├── Dockerfile
	└── server.js

Key Steps in the Demo Setup

  1. Generate Certificates: Create self-signed certificates using OpenSSL.
  2. Configure Envoy: Define Envoy settings to accept client TCP connections and establish tunnels with the server.
  3. Implement the Server: Use Node.js to build an HTTP/2 server that processes client messages.
  4. Implement the Client: Use Node.js to build a client that sends messages to the Envoy proxy.
  5. Deploy Services: Use Docker Compose to orchestrate Envoy and the server.
  6. Test Communication: Verify message flow from client to server and back through the HTTP/2 CONNECT tunnel.

By following this guide, you can establish efficient, transparent, and secure traffic tunnels using HTTP/2 CONNECT, a cornerstone of Istio’s Ambient mode and HBONE technology.

Environment Preparation

1. Install Node.js

Ensure that Node.js (version >= 10.10.0) is installed on your system, as the http2 module became stable starting with this version.

2. Install Docker and Docker Compose

3. Create a Project Directory

Create a new directory in your workspace and navigate to it:

mkdir envoy-http2-tunnel
cd envoy-http2-tunnel

Generating a Self-Signed Certificate

To enable encrypted communication between Envoy and the server, we need to generate a self-signed certificate with the correct configuration.

1. Create the Certificates Directory and OpenSSL Configuration File

Create the certs directory:

mkdir certs
cd certs

Create an openssl.cnf file with the following content:

[ req ]
default_bits   	= 2048
default_md     	= sha256
prompt         	= no
distinguished_name = dn
req_extensions 	= req_ext

[ dn ]
C        	= US
ST       	= California
L        	= San Francisco
O        	= My Company
OU       	= My Division
CN       	= server

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1   = server
DNS.2   = localhost

2. Generate the Key and Certificate

Run the following command to generate the key and certificate:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout server.key -out server.crt -config openssl.cnf

This will generate the server.key and server.crt files in the certs directory.

Configuring the Envoy Proxy

We need to configure Envoy to accept plain TCP connections from the client and forward the data to the server through an HTTP/2 CONNECT tunnel.

1. Create the Envoy Configuration File

In the project root directory, create a file named envoy.yaml with the following content:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.filters.network.tcp_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
          stat_prefix: tcp_stats
          cluster: tunnel_cluster
          tunneling_config:
            hostname: server:8080
          access_log:
          - name: envoy.access_loggers.stdout
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              log_format:
                json_format:
                  start_time: "%START_TIME%"
                  method: "%REQ(:METHOD)%"
                  path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
                  protocol: "%PROTOCOL%"
                  response_code: "%RESPONSE_CODE%"
                  response_flags: "%RESPONSE_FLAGS%"
                  bytes_received: "%BYTES_RECEIVED%"
                  bytes_sent: "%BYTES_SENT%"
                  duration: "%DURATION%"
                  upstream_service_time: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
                  x_forwarded_for: "%REQ(X-FORWARDED-FOR)%"
                  user_agent: "%REQ(USER-AGENT)%"
                  request_id: "%REQ(X-REQUEST-ID)%"
                  upstream_host: "%UPSTREAM_HOST%"
                  upstream_cluster: "%UPSTREAM_CLUSTER%"
                  downstream_local_address: "%DOWNSTREAM_LOCAL_ADDRESS%"
                  downstream_remote_address: "%DOWNSTREAM_REMOTE_ADDRESS%"
  clusters:
  - name: tunnel_cluster
    connect_timeout: 5s
    type: LOGICAL_DNS
    lb_policy: ROUND_ROBIN
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        sni: server
        common_tls_context:
          validation_context:
            trusted_ca:
              filename: "/certs/server.crt"
          alpn_protocols: [ "h2" ]
    http2_protocol_options: {}
    load_assignment:
      cluster_name: tunnel_cluster
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: server
                port_value: 8080

2. Key Highlights of the Configuration

  • TCP Proxy Listener: Listens for incoming TCP connections on port 10000 and forwards traffic to the upstream server.
  • HTTP/2 Tunnel: Configures Envoy to use HTTP/2 CONNECT for tunneling traffic to the server.
  • TLS Encryption: Enables secure communication between Envoy and the upstream server using the generated server.crt and server.key.
  • Access Logs: Configures detailed logs for request and response metrics.

Next, proceed to set up the server and client components to test the tunnel functionality.

Implementing the Server

1. Create the Server Directory and Files

Create the server directory in the project root:

mkdir server

Inside the server directory, create server.js and Dockerfile.

2. Write server.js

Add the following code to server/server.js:

const http2 = require('http2');
const fs = require('fs');

const server = http2.createSecureServer({
  key: fs.readFileSync('/certs/server.key'),
  cert: fs.readFileSync('/certs/server.crt'),
});

server.on('stream', (stream, headers) => {
  const method = headers[':method'];
  const path = headers[':path'];

  if (method === 'CONNECT') {
    console.log(`Received CONNECT request for ${path}`);

    // Respond with 200 to establish the tunnel
    stream.respond({
      ':status': 200,
    });

    // Handle data within the tunnel
    stream.on('data', (chunk) => {
      const message = chunk.toString();
      console.log(`Received from client: ${message}`);

      // Respond to the client
      const response = `Echo from server: ${message}`;
      stream.write(response);
    });

    stream.on('end', () => {
      console.log('Stream ended by client.');
      stream.end();
    });
  } else {
    // Return 404 for non-CONNECT requests
    stream.respond({
      ':status': 404,
    });
    stream.end();
  }
});

// Start the server and listen on port 8080
server.listen(8080, () => {
  console.log('Secure HTTP/2 server is listening on port 8080');
});

Notes:

  • It listens for the secureConnection event to process TLS-secured sockets.
  • The server reads incoming data from the socket, processes client messages, and sends responses.

3. Create the Dockerfile

Add the following content to server/Dockerfile:

FROM node:14

WORKDIR /app

COPY server.js .

EXPOSE 8080

CMD ["node", "server.js"]

Implementing the Client

1. Create the Client Directory and Files

Create the client directory in the project root:

const net = require('net');

// Create a TCP connection to Envoy
const client = net.createConnection({ port: 10000 }, () => {
  console.log('Connected to Envoy.');

  // Send messages to the server
  let counter = 1;
  const interval = setInterval(() => {
	const message = `Message ${counter} from client!`;
	client.write(message);
	counter += 1;
  }, 1000);

  // Close the connection
  setTimeout(() => {
	clearInterval(interval);
	client.end();
  }, 5000);
});

client.on('data', (data) => {
  console.log(`Received from server: ${data.toString()}`);
});

client.on('end', () => {
  console.log('Disconnected from server.');
});

client.on('error', (err) => {
  console.error('Client error:', err);
});

Explanation:

  • The client establishes a standard TCP connection with Envoy and sends text messages.
  • The existence of this client is only to trigger Envoy to establish a tunnel with the server.

Creating the Docker Compose File

Create docker-compose.yml in the project root:

version: '3.8'

services:
  envoy:
    image: envoyproxy/envoy:v1.32.1
    volumes:
      - ./envoy.yaml:/etc/envoy/envoy.yaml
      - ./certs:/certs  # Mount the certificates directory
    ports:
      - "10000:10000"
    networks:
      - envoy_network
    depends_on:
      - server
    command: /usr/local/bin/envoy -c /etc/envoy/envoy.yaml --service-cluster envoy --log-level debug

  server:
    build:
      context: ./server
    networks:
      - envoy_network
    expose:
      - "8080"
    volumes:
      - ./certs:/certs  # Mount the certificates directory

networks:
  envoy_network:

This configuration sets up both the Envoy proxy and the server to operate within the same Docker network, facilitating seamless communication.

Running the Example

1. Start Docker Compose

In the project root directory, run:

docker-compose up --build

Expected Output:

  • Envoy Container: Displays startup information and debug logs.
  • Server Container: Displays Secure HTTP/2 server is listening on port 8080.

2. Run the Client

Open a new terminal, navigate to the client directory:

cd client

Run the client:

node client.js

Expected Output:

Connected to Envoy.
Received from server: Echo from server: Message 1 from client!
Received from server: Echo from server: Message 2 from client!
Received from server: Echo from server: Message 3 from client!
Received from server: Echo from server: Message 4 from client!
Received from server: Echo from server: Message 5 from client!

Disconnected from server.

3. Check Server Logs

In the Docker Compose output, you should see logs from the server:

envoy_1   | {"downstream_remote_address":"192.168.65.1:46306","path":null,"request_id":null,"bytes_sent":160,"protocol":null,"upstream_service_time":null,"bytes_received":88,"response_code":0,"user_agent":null,"downstream_local_address":"172.21.0.3:10000","upstream_host":"172.21.0.2:8080","start_time":"2024-12-03T11:37:59.542Z","upstream_cluster":"tunnel_cluster","duration":5012,"response_flags":"-","method":null,"x_forwarded_for":null}
server_1  | Secure HTTP/2 server is listening on port 8080
server_1  | New secure connection established.
server_1  | Received from client: Message 1 from client!
server_1  | Received from client: Message 2 from client!
server_1  | Received from client: Message 3 from client!
server_1  | Received from client: Message 4 from client!
server_1  | Received from client: Message 5 from client!
server_1  | Connection ended by client.

4. Check Envoy Logs

In Envoy’s logs, you can see records of the connection established with the server using the HTTP/2 CONNECT tunnel.

Testing the Communication

  • The client sends text messages to Envoy over a TCP connection.
  • Envoy forwards the client’s TCP traffic to the server via an HTTP/2 CONNECT tunnel.
  • The server receives the messages, processes them, and replies to the client.
  • Envoy relays the server’s responses back to the client through the tunnel.
  • The client receives the responses from server.

Notes

  • Certificate Management: Ensure the certificates are properly configured and used in both Envoy and the server.
  • Docker Networking: Use Docker Compose-defined networks to allow containers to communicate using service names.
  • Port Conflicts: Ensure ports 10000 (Envoy) and 8080 (server) are not occupied.
  • TLS Configuration: In this example, communication between Envoy and the server is secured with TLS and HTTP/2.

Tunnel Establishment Process

The following diagram illustrates the interaction between the client, Envoy proxy, and server, showing how data is transmitted and the tunnel connection is established:

Figure 3: Tunnel Establishment Process

Explanation:

  1. Client connects to Envoy via TCP:
  • The client sends a TCP connection request to Envoy.
  • Envoy accepts the connection and creates a new TCP proxy session (ConnectionId: 0).
  • Envoy establishes a connection with the server:
    • Envoy connects to the upstream cluster tunnel_cluster, creating a new connection (ConnectionId: 1).
  • Establishing the HTTP/2 CONNECT tunnel:
    • Envoy establishes an HTTP/2 connection with the server.
    • Envoy sends an HTTP/2 CONNECT request with the target hostname server:8080.
    • The server responds with 200 OK, successfully establishing the tunnel.
  • Data transmission:
    • Message exchange loop:
      • The client sends data (Message N) to Envoy.
      • Envoy forwards the data through the tunnel to the server.
      • The server processes the data and responds (Echo Message N).
      • Envoy forwards the response back to the client.
    • Logging:
      • The server logs each received message, e.g., Received from client: Message N from client!.
  • Closing the connection:
    • The client sends a FIN request to close the connection.
    • Envoy forwards the FIN request to the server, closing the tunnel.
    • The server acknowledges the FIN with an ACK.
    • Envoy sends the ACK to the client, completing the connection closure.
  • Logging:
    • Envoy logs the closure of connections, including ConnectionId and stats.
    • The server logs the closure, e.g., Stream ended by client..

Conclusion

Although this is an introductory example, it provides a solid foundation for understanding and further exploring the HTTP/2 CONNECT tunnel function. In the next blog, I will explain the tunnel implemented by two Envoy proxies and take you to further understand the HBONE transparent tunnel in the Istio ambient mode.

References

###

If you’re new to service mesh, Tetrate has a bunch of free online courses available at Tetrate Academy that will quickly get you up to speed with Istio and Envoy.

Are you using Kubernetes? Tetrate Enterprise Gateway for Envoy (TEG) is the easiest way to get started with Envoy Gateway for production use cases. Get the power of Envoy Proxy in an easy-to-consume package managed by the Kubernetes Gateway API. Learn more ›

Getting started with Istio? If you’re looking for the surest way to get to production with Istio, check out Tetrate Istio Subscription. Tetrate Istio Subscription has everything you need to run Istio and Envoy in highly regulated and mission-critical production environments. It includes Tetrate Istio Distro, a 100% upstream distribution of Istio and Envoy that is FIPS-verified and FedRAMP ready. For teams requiring open source Istio and Envoy without proprietary vendor dependencies, Tetrate offers the ONLY 100% upstream Istio enterprise support offering.

Need global visibility for Istio? TIS+ is a hosted Day 2 operations solution for Istio designed to simplify and enhance the workflows of platform and support teams. Key features include: a global service dashboard, multi-cluster visibility, service topology visualization, and workspace-based access control.

Get a Demo

Let's Get In Touch

For more information or a demo request, just drop us a message.

Contact Us