Designing Traffic Flow via Tier-1 and Tier-2 Ingress Gateways

Introduction

Tetrate’s application connectivity platform, Tetrate Service Bridge (TSB), offers two gateway types, called Tier-1 and Tier-2, that are both based on Envoy and help to achieve different goals. Let’s explore how each type of gateway functions, and when to choose one gateway type versus another.

Tier-1 Gateways sit at the application edge and are used in multi-cluster environments to route traffic between clusters hosting the same application.

Tier-2 Gateways sit at the cluster edge and route traffic to the mesh-managed services inside the cluster.

Tier-1 and Tier-2 Gateways Explained

The design of an application deployment hosted in a TSB-managed cluster is very similar to the open-source Istio model. It requires the same constructs and will use an ingress gateway to route  incoming traffic. The Tier-2 Gateway is equivalent to the Istio ingress gateway. As shown in Figure 1, it’s  logically the same as the open-source model.

×

Tetrate Service Bridge uses Istio and Envoy as the control and data planes in a mesh-managed cluster and is not itself present in the application data path. Compare a packet capture between an open source Istio-managed cluster and a TSB-managed cluster and you will see no difference between the two. TSB-configured manifests are consumed and used by Istio. In this way, TSB acts similarly to CI/CD automation logic, wherein the deployment process affects the behavior of the application, but not the application logic itself.

TSB adds constructs to manage the installation and configuration of scoped gateways per set of applications to speed up developer and operator workflows, separate responsibilities between infrastructure and application, and to isolate the impact of a misconfigured gateway config from other apps/business groups.

When to Deploy a Tier-1 Gateway

When you have two or more Kubernetes clusters that serve the same application—for increased capacity, blue-green deployments, failover, etc.—the question always arises: how is inbound traffic distributed across those clusters?

An Tier-2 Gateway at each cluster edge allows direct access to the application–for example, Cluster A will be listening to service1A.example.com and Cluster X will be listening to service1X.example.com. In turn, a Tier-2 Gateway provides global load balancing across clusters. Traffic routing distribution across clusters is based on a weight value between 1-100 that refers to the percentage of the traffic being sent to a particular cluster.

Below is a simple example of a Tier-1 Gateway configuration. The example represents a complete Tier-1 Gateway manifest to demonstrate the simplicity of the solution. For details on the specific settings, please refer to Tetrate API documentation.

---
apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: service1-tier1
  group: demo-gw-group
  organization: demo-org
  tenant: demo-tenant
  workspace: demo-ws
spec:
  workloadSelector:
    namespace: service1-tier1
    labels:
      app: tsb-gateway-service1-tier1
      istio: ingressgateway
  externalServers:
  - name: service1
    hostname: service1.cx.example.com
    port: 80
    tls: {}
    clusters:
    - name: site-1-gcp
      weight: 75
    - name: site-2-aws
      weight: 25

In this example, 75% of user calls from the Internet to service1.cx.example.com are forwarded to site-1 in GCP and the rest to site-2 in AWS. The traffic in this example arrives at the plain-text port 80, after which all communication between the Tier-1 Gateway and application clusters is mTLS encrypted.

×

Cloud Vendor Gateway Consolidation

Istio users often implement the ingress gateway per application model. This approach assures secure, separate management of an application and its artifacts.

One of the most common pain points noticed here is that a cloud vendor load balancer must  be used for each application. This approach comes with the monetary and resource management costs associated with maintaining a high number of load balancers placed in front of Envoy ingress gateway pods.

TSB allows service discovery and communication via the NodePort service type instead of a LoadBalancer, which means that no cloud vendor load balancers are required; services within TSB clusters can be reached via NodePort. The Tier-1 Gateway allows us to collapse the usage of cloud vendor load balancers to a single ingress point.

×

Figure 3 above demonstrates the simplification of a cloud setup by moving the intra-cluster service connectivity to TSB, rather than requiring the use of cloud vendor load balancers. To implement the setup above without TSB would require the use of external load balancers. TSB also maintains the list of the Kubernetes nodes.

Resource Requirements

In terms of the resources needed for a Tier-2 Gateway, the requirements are no different between open source Istio and TSB. In fact, the implementation is the same–the Gateway and VirtualService manifests may be created manually or via an automation tool in open source. In Tetrate’s case, TSB creates the manifests for Istio consumption.

The Tier-1 Gateway does require a dedicated control plane which means that mesh-managed applications and the Tier-1 Gateway can not operate in the same cluster, although a Kubernetes cluster that hosts a Tier-1 Gateway can also host  applications that are outside of the service mesh. As an example, some Tetrate customers place the Tier-1 Gateway on the same cluster as the TSB management plane.

Architecture Considerations

It’s common for new requirements to arise as an environment grows and matures. A Tier-1 Gateway can be planned and implemented as part of an initial service mesh implementation, or it can be added later. The addition of a Tier-1 Gateway only affects the entry point inbound traffic, but does not require any changes to existing clusters.

Figure 4 demonstrates a setup without a Tier-1 Gateway.

×

When a Tier-1 Gateway is introduced (Figure 5), DNS records must be updated to point to the Tier-1 Gateway with no changes required in the setup of the application clusters.

×

Note: TSB is not a DNS management tool and changes of DNS records are performed outside of TSB. (There are multiple automation tools and techniques that are available for that operation).

However, the switch from using a LoadBalancer to a NodePort architecture when adding a Tier-2 Gateway does require a slight change in application clusters as described in Tetrate’s Documentation

While the Tier-1 Gateway is the front-end for incoming traffic at the application edge, it can be deployed in a high-availability configuration (Figure 6).

×

There is no limit on the number of Tier-1 gateways you can use for high availability. This flexible architecture allows Tetrate customers to build robust designs that are adjustable to meet a  wide range of requirements.

Summary

This article covers the most common architectural concerns that service mesh architects have when designing a TSB deployment in an enterprise environment. Here are the most important takeaways:

• TSB Tier-1 and Tier-2 Gateways use Istio ingress gateway pods and services. No additional proprietary components are introduced here.
• TSB supports the open source model of ingress gateways. There is merely a change in nomenclature such that TSB ingress gateways are called Tier-2 Gateways.
• A single gateway may be for all applications or a gateway-per-application model may be used.
• TSB can lower cloud costs by reducing the  number of cloud vendor load balancers used by utilizing a Kubernetes NodePort instead of a LoadBalancer for intra-cluster communication.
• TSB Tier-1 Gateways offer cross-cluster load balancing functionality.
• Since cross-cluster load balancing may not be required at the early stages of an implementation, Tier-1 Gateways need not be part of an initial deployment and may be added later with no significant impact on existing applications.
• Multiple Tier-1 Gateways may be deployed in front of the same application for high availability.