In-Pod IPtables Rule Injection in Istio Ambient Mode Explained

In my previous blog post, I provided an overview of the IPtables rules injected within pod network namespaces in Istio ambient mode. This article takes a closer look at these rules, explaining how they achieve transparent traffic interception and redirection within pods.

IPtables Rules Inside the Pod

In a pod’s network namespace, the Istio CNI Node Agent sets up a series of IPtables rules to enable transparent traffic interception and redirection. The following rules, injected into the mangle and nat tables, demonstrate how Istio processes inbound and outbound traffic.

# Generated by iptables-save v1.8.9 (nf_tables) on Thu Nov 14 08:43:17 2024
*mangle
:PREROUTING ACCEPT [99138:22880045]  # Default ACCEPT policy for the PREROUTING chain in the mangle table.
:INPUT ACCEPT [0:0]                  # Default ACCEPT policy for the INPUT chain in the mangle table.
:FORWARD ACCEPT [0:0]                # Default ACCEPT policy for the FORWARD chain in the mangle table.
:OUTPUT ACCEPT [100900:34940164]     # Default ACCEPT policy for the OUTPUT chain in the mangle table.
:POSTROUTING ACCEPT [0:0]            # Default ACCEPT policy for the POSTROUTING chain in the mangle table.
:ISTIO_OUTPUT - [0:0]                # Custom ISTIO_OUTPUT chain for handling outbound traffic.
:ISTIO_PRERT - [0:0]                 # Custom ISTIO_PRERT chain for handling prerouting traffic.
-A PREROUTING -j ISTIO_PRERT         # Direct all PREROUTING traffic to the ISTIO_PRERT chain.
-A OUTPUT -j ISTIO_OUTPUT            # Direct all OUTPUT traffic to the ISTIO_OUTPUT chain.
-A ISTIO_OUTPUT -m connmark --mark 0x111/0xfff -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
# Restore connection mark 0x111/0xfff for consistent connection tracking.
​
-A ISTIO_PRERT -m mark --mark 0x539/0xfff -j CONNMARK --set-xmark 0x111/0xfff
# Set connection mark to 0x111/0xfff for packets marked 0x539/0xfff in PREROUTING.
​
COMMIT  # Apply mangle table rules.
# Completed on Thu Nov 14 08:43:17 2024
​
# Generated by iptables-save v1.8.9 (nf_tables) on Thu Nov 14 08:43:17 2024
*nat
:PREROUTING ACCEPT [2:120]           # Default ACCEPT policy for the PREROUTING chain in the nat table.
:INPUT ACCEPT [0:0]                  # Default ACCEPT policy for the INPUT chain in the nat table.
:OUTPUT ACCEPT [119:9344]            # Default ACCEPT policy for the OUTPUT chain in the nat table.
:POSTROUTING ACCEPT [0:0]            # Default ACCEPT policy for the POSTROUTING chain in the nat table.
:ISTIO_OUTPUT - [0:0]                # Custom ISTIO_OUTPUT chain for handling outbound NAT traffic.
:ISTIO_PRERT - [0:0]                 # Custom ISTIO_PRERT chain for handling prerouting NAT traffic.
-A PREROUTING -j ISTIO_PRERT         # Direct all PREROUTING traffic to the ISTIO_PRERT chain.
-A OUTPUT -j ISTIO_OUTPUT            # Direct all OUTPUT traffic to the ISTIO_OUTPUT chain.
-A ISTIO_OUTPUT -d 169.254.7.127/32 -p tcp -m tcp -j ACCEPT
# Allow TCP traffic destined for 169.254.7.127 (likely an internal Istio address).
​
-A ISTIO_OUTPUT -p tcp -m mark --mark 0x111/0xfff -j ACCEPT
# Allow TCP traffic marked as 0x111/0xfff.
​
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -j ACCEPT
# Allow traffic to the loopback interface excluding 127.0.0.1.
​
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -p tcp -m mark ! --mark 0x539/0xfff -j REDIRECT --to-ports 15001
# Redirect outbound TCP traffic (not marked 0x539/0xfff) destined outside 127.0.0.1 to port 15001 (outbound socket).
​
-A ISTIO_PRERT -s 169.254.7.127/32 -p tcp -m tcp -j ACCEPT
# Allow TCP traffic originating from 169.254.7.127 in the PREROUTING chain.
​
-A ISTIO_PRERT ! -d 127.0.0.1/32 -p tcp -m tcp ! --dport 15008 -m mark ! --mark 0x539/0xfff -j REDIRECT --to-ports 15006
# Redirect inbound TCP traffic (not marked 0x539/0xfff) to port 15006 (inbound socket) if its destination port is not 15008.
​
COMMIT  # Apply nat table rules.
# Completed on Thu Nov 14 08:43:17 2024

Role of Specific Ports

These IPtables rules differentiate and handle various types of traffic using specific ports:

• 15008 (HBONE socket): Handles HTTP-based traffic transparently using the HBONE protocol.
• 15006 (plaintext socket): Manages unencrypted traffic within the mesh for inter-pod communication.
• 15001 (outbound socket): Controls outbound traffic and enforces policies for accessing external services.

By leveraging these ports, Istio enables transparent management and control of inbound, outbound, and internal traffic, enforcing fine-grained security policies and traffic controls. For more information, refer to Istio Application Requirements.

Significance of 0x539 Mark

The 0x539 mark identifies traffic originating from Istio proxies (e.g., ztunnel). This mark is applied to distinguish packets processed by proxies, ensuring they are not reprocessed or misrouted.

Significance of 0x111 Mark

The 0x111 mark is used for connection-level marking within the Istio mesh, indicating that a connection has been processed by a proxy. The CONNMARK module in IPtables extends this mark to the entire connection, speeding up subsequent packet matching.

Visualizing IPtables Rules

For further details on how Istio CNI handles IPtables, refer to the source code: istio/cni/pkg/iptables/iptables.go at master · istio/istio · GitHub.

Traffic Routing Visualization

The following diagram illustrates the path of cross-node encrypted L4 traffic:

1. Application Sends Request: Traffic is initiated by the application process and enters the pod’s network namespace.
2. iptables Rule Matching:
   • Outbound traffic is matched in the OUTPUT chain, where eligible traffic is redirected to the ISTIO_OUTPUT chain.
   • In the ISTIO_OUTPUT chain, the traffic is marked and accepted.
3. REDIRECT Action: Traffic is captured by iptables and redirected to ztunnel’s listening port (15006 for plaintext, 15008 for encrypted traffic).
4. ztunnel Processes Traffic: ztunnel receives the traffic, performs policy checks, encryption, and other operations.
5. Traffic is Forwarded to Target Service: ztunnel sends the processed traffic through the HBONE tunnel to the ztunnel on the destination node, which then decrypts and delivers it to the target service.

Conclusion

By analyzing the IPtables rules in Istio ambient mode, we see how Istio’s CNI plugin establishes a transparent traffic interception mechanism within pods. These rules ensure that traffic entering and leaving pods is correctly handled by ztunnel, enabling finer-grained traffic management and security policy enforcement. Stay tuned for more deep dives into Istio ambient mode networking!