Image: rawpixel.com

Introduction

In this article, we will discuss different load balancer options in AWS that may be confusing because we have several choices.

The following simplified diagram shows a typical network setup for an EKS cluster (without showing private and public subnets, which is usually the case).

A typical setup has the worker nodes running in private subnets and with private IP addresses as part of the VPC CIDR block.

The pods are also running on the worker nodes and have private IP addresses as part of the VPC CIDR block.

To allow the pods to communicate with external resources, such as the Internet or other resources outside the VPC, the cluster uses a load balancer.

The load balancer runs in the public subnet and has a public IP address that is reachable from the Internet.

Now the question is, which type of load balancer should we choose?

There are new offerings and some that will be decommissioned, and we need to consider the use cases when selecting a load balancer.

This article discusses the architectural aspects of load balancers rather than the implementation, although we will also touch on that.

The goal is to guide you in the right direction when deciding on an AWS load balancer for your needs.

Specify Your Requirements

Before you could even select the correct option, you need to define the proper requirements. Those requirements need to be based on what type of application you intend to run in your cluster, your security constraints, what protocol you need to support (again, back to the type of application you're running)

Try to answer the following questions: 

1. Do you just need to support the HTTP traffic?
2. Do you use gRPC (open-source remote procedure call (RPC))  between your services and also for client > server communication?
3. Should the Load Balancer handle authentication/redirect?
4. Do you need to use a Web Application Firewall (WAF) for enhanced security? (inspect traffic, SQL injections, reject some requests to protect your cluster) ? 
5. Any need to support non-HTTP protocols?
6. Is your application handling end-2-end encryption?
7. Do you have a huge amount of traffic spikes?

OSI Model - Network Layers

Before we can determine which type of load balancer to use, it's helpful to review the OSI model. You don't need to study it in depth, but it's important to understand the difference between L4 (Layer 4) and L7 (Layer 7) because it plays a significant role in selecting different load balancer types. The description of the layers in the following diagram comes from ISO/IEC 7498:

❕From Wikipedia

Recommendation X.200 describes seven layers, labeled 1 to 7. Layer 1 is the lowest layer in this model.

diagram - OSI model

• Layer 1: Physical -  responsible for the physical connection between devices, including cables and switches.
• Layer 2: Data Link - responsible for the transmission of data between devices on the same network. 
• Layer 3: Network - responsible for routing and addressing data packets between networks. 
• Layer 4: Transport - responsible for end-to-end communication and error checking. 
• Layer 5: Session -  responsible for establishing, maintaining, and terminating communication sessions between devices.
• Layer 6: Presentation - responsible for translating data between different formats. 
• Layer 7: Application - responsible for providing the interface between the application and the network.

Load Balancer Types

Based on the questionary we've seen before, you could conclude the following:

• If you answered yes to any of questions 1-4, you should choose a load balancer that supports Layer 7. The reason is that you probably need redirection, packet inspection, the ability to reject requests, and the ability to send a request to an authentication service, among other things. Layer 7 load balancers provide the intelligence to handle these types of requests.
  • ALB (Application Load Balancer) is a type of load balancer that operates at the application layer (Layer 7) of the OSI model and is capable of routing traffic based on the content of the requests. One of the features of an ALB is that it can perform TLS/SSL (Secure Sockets Layer/Transport Layer Security) offloading, which means that it can handle the encryption and decryption of traffic on behalf of the target instances.
  • ALB can forward requests based on rules that you can define.
  • ALB may also use a web application firewall (WAF) to protect your applications from common web vulnerabilities.
  • ALB can also redirect requests to different targets based on the contents of the requests.
• If you only answered yes to questions 5-7, you should choose a Layer 4 type of load balancer. NLB (Network Load Balancer) is a load balancer type that simply takes a request and sends it to your target. 
  • NLB (Network Load Balancer) can also support TLS offloading, but since it operates at the transport layer (Layer 4) of the OSI mode, it does not have the ability to examine the contents of the packets it transmits. As a result, it cannot perform content-based routing or other advanced traffic management features that are available with an ALB.
  • Layer 4 load balancers are also the right choice if you run any non-HTTP services, such as an SQL service, VoIP, DNS service, etc. In general, they can handle any type of traffic, not just HTTP traffic, and are particularly well-suited for non-HTTP services that rely on persistent connections or require low latency.

The following diagram may define the type of load balancer you need:

AWS Load Balancers

AWS offers the following type of load balancers:

• NLB - Network Load Balancer (Level 4)
  • very fast, manages millions of requests per second
• ALB - Application Load Balancer (Level 7)
  • forward requests based on different rules
  • use a web application firewall (WAF)
  • redirects requests
• ELB - Elastic or Classic Load Balancer
  • Classic Load Balancer is an old implementation
  • hybrid of L4 and L7 capabilities.

 The following decision tree diagram can help us to choose the correct load balancer type:

NLB is cheaper than Classic Load Balancer if you just need a Layer-4 load balancer.

Comparison

The following table outlines the most essential features that each load balancer type offers:

For the full list, please check the official documentation here.

I wouldn't recommend implementing a classic load balancer as it's subject to depreciation.

Read more about Classic Load Balancer and end of life here 

How do you implement a load balancer with EKS?

You may be using an Nginx ingress controller that receives all incoming requests from a load balancer. The load balancer itself is defined in the service manifest of the ingress controller.

Creating a load balancer in AWS is straightforward. You can create a load balancer using the AWS management console, the AWS CLI, or through an API. Once you have created a load balancer, you can use an annotation in the ingress controller service to configure the load balancer and trigger its creation. The exact syntax and format of the annotation will depend on the specific configuration options that you need to set and the version of the ingress controller that you are using. It is recommended to consult the documentation for your ingress controller and the AWS load balancer service to learn more about the available options and how to use annotations to create and configure a load balancer.

The following annotation is used to specify the type of load balancer that should be created for a Kubernetes service running in AWS. In this case, the annotation specifies that an NLB (Network Load Balancer) should be created.

example - annotation to create a load balancer

service.beta.kubernetes.io/aws-load-balancer-type: nlb

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer

You can find all the annotations you can set under this link

❗If you don't specify any annotation, and your service is defined as a Load Balancer type, AWS will provision a classic load balancer for you.

SNAT for Pod Communication

SNAT (Source Network Address Translation) is used in network load balancing to allow the target instances in a load balancer pool to communicate with the Internet or other resources outside the pool*. When a target instance initiates a connection to a resource outside the pool*, the load balancer translates the source IP address of the request from the target instance's private IP address to the load balancer's public IP address. This allows the response from the external resource to be sent back to the load balancer, which can then forward it to the target instance.

• SNAT is used when the target instances do not have public IP addresses or are not directly accessible from the Internet. It allows the target instances to communicate with external resources without exposing their private IP addresses, which helps to improve security and reduce the risk of attacks.
• SNAT is used with network load balancers, such as NLBs (Network Load Balancers) in AWS, but it can also be used with other types of load balancers that support this feature.

❗SNAT is not applicable if your cluster is using the IPv6 family since the IPv6 addresses are not network-translated.

❗The pool refers to the group of target instances that the load balancer distributes traffic to. The target instances in the pool can be Amazon Elastic Compute Cloud (EC2) instances, containers, or other resources that are capable of handling incoming traffic.

Usually, each pod in an EKS cluster has a private IP address that is part of the CIDR block. This means that traffic can only be initiated by the pod itself and not from external resources. Even if you have resources that are connected via VPC peering, transit VPC, or Direct Connect, these resources cannot initiate communication with the pods in your cluster. If your pods are running in a private subnet (which is strongly recommended), the only way to reach the internet-facing resources is through a NAT gateway.

You need to have SNAT (Source Network Address Translation) enabled if there are resources that connect to your VPC (where your cluster resides) via VPC peering, transit VPC, or Direct Connect and need to initiate communication directly with your pods.

Please note! SNAT is not enabled by default.

SNAT ENABLED

You can enable SNAT via the following command:

kubectl set env daemonset -n kube-system aws-node AWS_VPC_K8S_CNI_EXTERNALSNAT=true

The following diagram shows the difference between when SNAT=enabled vs disabled:

Diagram - SNAT enabled

Diagram - SNAT disabled:

The differences between Enabled and Disabled:

• When SNAT (Source Network Address Translation) is enabled, the IP translation is typically performed by a load balancer rather than by the CNI (Container Network Interface) plugin. The CNI plugin is responsible for configuring the networking for the pods in the cluster, but it does not typically perform SNAT itself.

• When SNAT is enabled, traffic from the pods to destinations outside of the VPC CIDR (such as the Internet) will generally be NATed to the public IP address of the load balancer, rather than to the private IP address of the worker node. This allows the traffic to reach external resources without exposing the private IP addresses of the pods.

• SNAT is not typically used for Pod-to-Pod communication within the same VPC, as the pods can communicate directly using their private IP addresses.

TLS Certificates

Both ALB (Application Load Balancer) and NLB (Network Load Balancer) support TLS decryption and re-encryption of certificates on the load balancer level. This can be set via an annotation. If you want to perform TLS termination on your load balancer, you can use AWS Certificate Manager or set up your own certificate manually. Both also support end-to-end TLS termination, which means your application handles the termination instead of the load balancer.

There is a difference in how the traffic is passed through, however. In NLB, the TCP session remains from the client to the pod. In the ALB case, there is a new TCP session from the load balancer to the pod. This means the traffic is re-encrypted. To achieve this, you can offload signed certificates on the ALB and then re-encrypt the traffic.

With NLB, you get an easier passthrough to your application since the TCP session from the client can be sustained, as the NLB maintains the TCP session between the client and the target instance.

Conclusion

As we have learned, the choice of load balancer depends on your use case.

In summary:

• Choose ALB (Application Load Balancer) or NLB (Network Load Balancer) based on your use case. There is no need to use the Classic Load Balancer anymore.
• Determine if you need a Layer 4 or Layer 7 load balancer based on the OSI model.
• If you expect a lot of request spikes, NLB may be a better choice.
• If you just want to pass through traffic, NLB is sufficient. If you need extra application protection (such as AWS WAF), select ALB.
• NLB supports TCP and UDP protocols, while ALB does not.
• ALB supports HTTP, HTTPS, and gRPC protocols.
• The exporter forwarder header added to ALB will be sent directly to the pod, allowing you to see the client's IP address.

About the Author

Aleksandro Matejic, a Cloud Architect, began working in the IT industry over 21 years ago as a technical specialist, right after his studies. Since then, he has worked in various companies and industries in various system engineer and IT architect roles. He currently works on designing Cloud solutions, Kubernetes, and other DevOps technologies.

In his spare time, Aleksandro works on different development projects such as developing devoriales.com, a blog and learning platform launching in 2022/2023. In addition, he likes to read and write technical articles about software development and DevOps methods and tools. You can contact Aleksandro by visiting his LinkedIn Profile.