Independent software vendor (ISV) users often provide end-user solutions on a multi-tenant architecture to cut costs and reduce operational management. However, this approach can lead to resource exhaustion or network sparsity issues in Kubernetes clusters, affecting neighboring workloads. Kubernetes offers capabilities to enforce resource availability like processor and memory limits by default to prevent computer starvation. Yet, evolving workloads now utilize other resources like network bandwidth to enhance performance, which can cause bandwidth exhaustion and impact neighboring pods.
To address this Kubernetes challenge, the Amazon Virtual Private Cloud (VPC) CNI plugin comes into play. This plugin restricts pod usage in the network for ingress and egress bandwidth, preventing network starvation and ensuring network stability and quality of service. Developed by AWS for Amazon Elastic Kubernetes Service (EKS), the Amazon VPC CNI plugin allows container networks to use Amazon VPC networking and security features directly.
Provisioning a node with this plugin assigns a group of slots from the primary ENI node subnet, enabling connectivity and networking of Kubernetes pods on Amazon EKS while integrating VPC networking functionality into the pods. Bandwidth throttling is achieved by the Amazon VPC CNI plugin relying on a bandwidth plugin to control the ingress and egress bandwidth limits using Linux traffic control utilities.
Before configuring pod bandwidth limits, prerequisites such as an Amazon EKS Cluster v1.24 and higher, Amazon VPC CNI v1.15.0 and higher, kubectl, and eksctl are required. The process involves creating an EKS cluster (optional), enabling the CNI Bandwidth Plugin on an EC2 instance, and installing iperf and tc CLI tools.
Testing the module on output/input limits involves checking the current qdisc on the EC2 instance using the tc command and using iperf to measure the maximum achievable bandwidth. Redeploying pods with bandwidth restrictions includes specifying ingress and egress bandwidth limits in the deployment manifest and redeploying the application.
However, it is important to note the Bandwidth plugin does not support Amazon VPC CNI-based network policy as of now, as it conflicts with the Network Policy Agent. But efforts are being made to resolve this issue and support both features simultaneously.
In conclusion, the use of the Amazon VPC CNI plugin and its capabilities in limiting ingress and egress bandwidth for pods on Amazon EKS can help prevent network starvation in Kubernetes clusters. This functionality allows users to control the usage of their pods on the network and avoid issues caused by excessive network consumption by neighboring pods.
Article Source
https://aws.amazon.com/blogs/containers/ensuring-fair-bandwidth-allocation-for-amazon-eks-workloads/