Maximum Throughput, Minimum Resources: Envoy Gateway + AWS Load Balancing

Envoy Gateway, now at its 1.1 release, is the reference cloud-native load-balancer. It provides Layer 7 traffic routing and observability, i.e., it understands the HTTP protocol, and can direct traffic based on request paths.

A lot of us run on AWS. AWS also offers three primary load balancing options: the old and deprecated CLB – Classic Load Balancer (neé ELB); the Application Load Balancer – ALB; and the Network Load Balancer – NLB. Countless column-pixels have been dedicated to which you should choose, including articles from AWS themselves. The very short version is: ALBs work at Layer 7, duplicating many of Envoy Gateway’s features.

You’ll always need some kind of AWS load-balancer to get traffic in from the internet (routed into a public subnet) to your worker nodes (hopefully situated in a private subnet). However, if you’re running Envoy Gateway, you don’t need the complexity, cost, and relatively high latency of an ALB; an NLB is optimal.

When you configure a Gateway resource, an AWS load-balancer is automatically created to expose your cluster’s ingress layer – Envoy Gateway or otherwise – to the outside world. I won’t go into the mechanisms powering this as it’s fairly complicated, but suffice to just understand that there’s a Kubernetes Service pointing at all the managed Envoy Proxy pods. The type: field of this Service is set to “LoadBalancer,” and a Kubernetes component called the cloud-controller spots that and calls out to AWS APIs to spin up an AWS load-balancer which sends traffic to all the Pods in that Kubernetes Service.

However, by default you’ll get a Classic Load Balancer – the worst! In this article I’ll take you through all the steps you need to use an NLB instead. The instructions I show are for EKS, but they should work on self-installed clusters with a few changes.

Setup

First let’s spin up an EKS cluster to use (if you want to use an existing cluster make sure you have the owners’ permission!). I’ll be using eksctl for this – if you don’t have it installed, instructions are on its site.

Verify eksctl is installed correctly:

$ eksctl version
0.187.0

Note: Version 0.187.0 is the version I tested with. As of this writing, the default EKS version is Kubernetes 1.30, which again is the version these instructions are verified with.

Create a cluster:

$ eksctl create cluster --name envoy-gateway-demo –with-oidc

Note: You’ll want to get a coffee while this happens.

Our cluster needs to use the AWS VPC CNI plugin – again a very technical detail, but for our purposes this determines the kind of network connections made between Pods in the cluster, and is necessary for the NLB to be able to connect to our managed Envoy Proxy pods. In an EKS cluster this is the default option, and the eksctl command above will have installed it; if you’re managing your own cluster you’ll need to use this CNI plugin, no e.g. Cilium.

Now our cluster is ready, we can install the various components into it. Annoyingly the order of the following steps is important, otherwise things go wrong, often in quite subtle ways, but if you follow these steps you’ll be fine!

First, the reason we’re here: Envoy Gateway 1.1:

$ helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.0 -n envoy-gateway-system --create-namespace

If you have any problems installing EG, more detailed installation instructions are available. 

Configure GatewayClass

Next we need to configure a GatewayClass resource so that we can associate our EG config with the right instance – it’s possible to have multiple copies of EG running, or EG plus a different ingress controller.

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    namespace: envoy-gateway-system
    name: custom-proxy-config

This resource contains a reference to an EnvoyProxy-type resource called custom-proxy-config, which will alter the Kubernetes configuration of all the actual Envoy instances that EG deploys to handle our traffic. So next, let’s deploy that:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-type: external
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
          service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing

This resource is a little more complicated, but essentially it’s giving a set of annotations to apply to the Kubernetes Service object which gets traffic to the Envoy proxies deployed by EG. This isn’t really any different from a Service you’d deploy to get traffic to your own apps, except remember this one will have its type field set to LoadBalancer.

Using NLB Instead of CLB

Most distributions of Kubernetes ship with some code that interacts with the cloud provider the cluster is running on. This is how it’s able to do things like create storage buckets for PersistentVolumeClaims, and how it’s able to make cloud load-balancers for Services of type LoadBalancer. However the code that’s built into Kubernetes to do this is a little old and only makes the CLBs I mentioned before. Next, we’ll deploy a newer replacement for that code, written by AWS, which will see our Service object and make an NLB. The aws-load-balancer-type: external annotation on the Service tells the built-in code to ignore it and not make a CLB.

This installation is a little complicated, as we’ll need to create a couple of AWS IAM resources. Luckily the policy documents already exist, and we can use eksctl to deploy them and wire them up to service accounts etc.

# This only needs to be run once per AWS account. On subsequent deployments you’ll get an already exists error.
$ aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file://<(curl -s https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.8.1/docs/install/iam_policy.json)

$ export AWS_ACCOUNT_ID=<your account ID>
$ eksctl create iamserviceaccount \
  --cluster=envoy-gateway-demo \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::${AWS_ACCOUNT_ID}:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve

$ helm repo add eks https://aws.github.io/eks-charts
$ helm repo update eks

$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=envoy-gateway-demo \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

You should now be able to see two Pods for the AWS load-balancer controller in the kube-system namespace. As of this writing, the helm chart we used instals version v2.8.1.

$ kubectl get pods -n kube-system
NAME                                        	READY   STATUS	RESTARTS   AGE
aws-load-balancer-controller-5ffb88ccf8-4dx2f   1/1 	Running   0      	31s
aws-load-balancer-controller-5ffb88ccf8-kqpb9   1/1 	Running   0      	31s
…

Configure Envoy Gateway

We can now configure EG as normal – I’ll give a quick overview here but more detailed guides are available. First, a Gateway resource, representing the actual Envoys that will be deployed:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: apps
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      protocol: HTTP
      port: 80

And an HTTPRoute, sending requests for www.example.com/httpbin to an httpbin Pod.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: apps
  hostnames:
    - "www.example.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /httpbin/
      filters:
        - type: URLRewrite
          urlRewrite:
            path:
              type: ReplacePrefixMatch
              replacePrefixMatch: /
      backendRefs:
        - group: ""
          kind: Service
          name: httpbin
          port: 8000
          weight: 1

Finally, we deploy a copy of httpbin itself, in the default namespace where the HTTPRoute above is expecting it:

$ kubectl apply -f https://raw.githubusercontent.com/istio/istio/master/samples/httpbin/httpbin.yaml

Review Load Balancers

Ever since we deployed the Gateway resource, the Envoy Gateway control plane will have started spinning up Envoy instances to actually proxy our requests, as well as deploying peripheral resources like the Service type=LoadBalancer. Moments after that was created, the NLB creation should have started, and with luck, will now be complete.

If your AWS CLI is configured for the correct account and region, you can see the NLB with:

$ aws elbv2 describe-load-balancers

Alternatively, it’s visible in the AWS web console.

Nothing should be too surprising, but take a note of the DNS name that it got, as that’s what we’ll need to connect to in order to test it. If you want to do that programmatically, you can use this rather long command:

$ export NLB_DNS_NAME=$(kubectl get service -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-namespace=default -l gateway.envoyproxy.io/owning-gateway-name=apps -o jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}')

We can use curl to connect to the NLB, which in turn will forward all traffic to EG inside the EKS cluster. EG will then parse the HTTP protocol being spoken, and direct us to a Pod depending on the HTTP host and path used. Recall that our Gateway and HTTPRoute are expecting a hostname of www.example.com, but because we need to physically connect to the NLB, the curl command looks like this:

$ /usr/bin/curl --header "Host: www.example.com" ${NLB_DNS_NAME}/httpbin/headers
{
  "headers": {
    "Accept": "*/*",
    "Host": "www.example.com",
    "User-Agent": "curl/8.1.2",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000",
    "X-Envoy-External-Address": "143.58.214.48",
    "X-Envoy-Original-Path": "/httpbin/headers"
  }
}

Hopefully, this has all worked and given you everything you need to use the latest greatest NLBs. From here, you might want to read around all the config we’ve set, or make this into a more production-ready setup, e.g., by adding a CDN in front of it, adding TLS, or injecting the Coraza WAF into the EG proxies.

Cleanup

If you now wish to tear everything down, you can run:

$ kubectl delete gateway apps
$ eksctl delete cluster --name envoy-gateway-demo

The first command is important, to remove the NLB, Security Group, etc that EG dynamically creates. Having run that, the eksctl delete command should complete successfully, however this is always a little hit-and-miss, and manual cleanup may be necessary.

Further Resources

• Envoy Gateway documentation ›
• What’s new in Envoy Gateway 1.1 ›
• Envoy Gateway 1.1 release notes ›

If you’re new to service mesh and Kubernetes security, we have a bunch of free online courses available at Tetrate Academy that will quickly get you up to speed with Istio and Envoy.

Once you have Istio up and running, you will probably need simpler ways to manage and secure your services beyond what’s available in Istio, that’s where Tetrate Service Bridge comes in. You can learn more about how Tetrate Service Bridge makes service mesh more secure, manageable, and resilient here, or contact us for a quick demo.