The innovative architecture of the Ingress Controller eliminates the dependency on a standalone etcd cluster, greatly simplifying maintenance costs and system complexity.

Background​

Ingress Controller is a Kubernetes Ingress Controller based on Apache . It has the capability to convert Ingress/CRDs resources from Kubernetes into routing rules for Apache , synchronizing them with the Apache cluster. As a result, users can harness the robust functionalities of Apache , including plugins, load balancing, health checks, and more, for the management of inbound traffic in Kubernetes.

In previous versions, deploying an Ingress Controller cluster necessitated the additional maintenance of a highly available etcd cluster. In practice, it proved to be less user-friendly and posed several challenges:

1. High Maintenance Costs for the etcd Cluster: Setting up a highly available cluster involves significant learning and maintenance costs, including system resource consumption like memory. Deploying an etcd cluster in Kubernetes requires careful attention to various factors, often leading to challenges for those unfamiliar with etcd, and necessitating concerns about memory and other system resource consumption.

2. High Utilization Costs: Deploying an Ingress Controller cluster entails three components. Compared to a single-component ingress-nginx, the Ingress Controller demands higher learning and debugging costs. It is notably less straightforward to use, presenting an additional burden for first-time users.

3. Data Redundancy and Inconsistency: Both Kubernetes etcd and etcd clusters retain a copy of the data. During usage, efforts are often needed to prevent discrepancies between the two datasets. Due to and the Ingress controller being decoupled, addressing and mitigating such situations becomes challenging.

4. Obstacles in Implementing Gateway API: The Gateway API dynamically manages the full lifecycle of a set of Gateways (). Because configuration primarily stems from etcd, the Ingress Controller must simultaneously monitor the etcd clusters and , which makes maintenance and management very complicated.

In the overall architecture, Apache does not rely on the Ingress Controller. The Ingress Controller performs the role of pushing configuration but lacks the ability to manage effectively. These issues are challenging to address within the existing architecture. To tackle these challenges and provide solutions, a new Ingress Controller architecture needs to be designed.

Design of New Architecture​

To address the issues in the existing architecture, it is necessary to remove the etcd component. Currently, we are considering two main approaches:

1. Rendering the .yaml configuration file: Generate .yaml configuration files based on the CRD. , in YAML deployment mode, periodically reads the entire .yaml configuration file every second.

2. Simulating the etcd server API: Building a KV in-memory database based on the CRD and simulating the etcd server API for use by . will attempt to watch the resource configurations provided by the Controller and notify all instances.

Clearly, the first approach is simpler, but it is not suitable for scenarios where the gateway directly connects to backend Pods. The reason is that in Kubernetes, Pod IPs have dynamic scalability features, and the Ingress Controller generates the .yaml configuration continuously.

This causes the routing tree to be rebuilt frequently, causing long-term performance jitter. Finally, after discussion, the community decided to adopt the second option. Its architecture is shown in the figure below:

Ingress Controller implements a new architecture in Release v1.7.0, which has the following advantages:

• Sole reliance on declarative configuration: will exclusively rely on the configuration information provided by the Control Plane and take it as the sole source. This approach, commonly used in Kubernetes, greatly reduces operational complexity.

• No need to maintain a separate etcd cluster: The new architecture eliminates the dependency on a standalone etcd cluster. This significantly reduces maintenance costs and complexity for users, making it easier to deploy and use.

• Advancement of the Kubernetes Gateway API standard: relies on the Ingress Controller and enables it to manage the lifecycle of the gateway. This contributes to the complete implementation of the Gateway API.

Ideal Architecture for Gateway API​

Gateway API is the next-generation version of the Ingress API, offering enhanced functionality and expressiveness. Currently, Gateway API has gained support from numerous vendors and projects. As one of the implementers of Gateway API, Ingress Controller not only adheres to the standard specifications of Gateway API but also combines the rich features of Apache to provide users with a broader range of gateway configurations and policy options.

The implementation of the new architecture further advances the realization of Gateway API, enabling better routing configuration and policies while reducing maintenance costs. This makes it easier to deploy and use, while also leveraging the advantages of Gateway API to improve the management efficiency of the API gateway.

How to Deploy and Use the New Ingress Controller​

In this chapter, we will explain the high-available installation and deployment of Ingress Controller in Kubernetes, and demonstrate how to configure Route to access the httpbin application service in an example.

Install Ingress Controller​

1. You can run the following command to clone the source code from Github:

git clone --depth 1 --branch 1.7.0 https://github.com/apache/-ingress-controller.git ingress--1.7.0cd ingress--1.7.0

2. Install CRDs

kubectl apply -k samples/deploy/crd/v1/

3. Install Ingress Controller

kubectl apply -f samples/deploy/composite.yaml

4. Check deployment status

• Check service

  kubectl get service  -n ingress- # check service

  Copy

  NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGEingress--gateway      NodePort    10.99.236.58     <none>        80:31143/TCP,443:30166/TCP   90s

  Copy

• Check pods

  kubectl get pods -n ingress- # check pod

  Copy

  NAME                                                 READY   STATUS    RESTARTS   AGEingress--composite-deployment-5df9bc99c7-xxpvq   2/2     Running   0          100s

  Copy

Highly Available Deployment​

• Deploy 3 instances and you can achieve high availability by configuring replicas.

 kubectl scale deployment ingress--composite-deployment --replicas=3 -n ingress-

• Check deployment status

kubectl get pods -n ingress-

NAME                                                   READY   STATUS    RESTARTS   AGEingress--composite-deployment-6bfdc5d6f6-gjgql   2/2     Running   0          20singress--composite-deployment-6bfdc5d6f6-jb24q   2/2     Running   0          20singress--composite-deployment-6bfdc5d6f6-sjpzr   2/2     Running   0          45h

Example of Usage​

Route is the CRDs resource of Ingress, used to represent routing traffic to specific backend services.

The following example shows how to configure Route to route traffic to the httpbin backend service.

Deploy httpbin Application Service​

• Deploy httpbin application service and configure Route:

kubectl apply -f samples/httpbin/httpbin-route.yaml

• The specific Route configuration is as follows:

  All requests with Host: httpbin.org will be routed to the httpbin service

  apiVersion: .apache.org/v2kind: Routemetadata:  name: httpbin-routespec:  http:    - name: route-1      match:        hosts:          - httpbin.org        paths:          - /*      backends:        - serviceName: httpbin          servicePort: 80

  Copy

Visit httpbin Test​

Access the ingress--gateway service through local port forwarding, and requests will be routed from ingress--gateway to the httpbin application.

# forward port 9080 -> service 80kubectl port-forward service/ingress--gateway 9080:80 -n ingress- &# acesss httpbincurl http://127.0.0.1:9080/headers -H 'Host: httpbin.org'

{  "headers": {    "Accept": "*/*",    "Host": "httpbin.org",    "User-Agent": "curl/7.74.0",    "X-Forwarded-Host": "httpbin.org"  }}

Summary​

We thoroughly discussed the innovative architecture of the Ingress Controller, liberating it from dependency on the etcd cluster. This greatly simplifies maintenance costs and system complexity. Simultaneously, the Ingress Controller actively advances the implementation of the Kubernetes Gateway API standard within the Ingress Controller, aiming to provide more extensive and consistent traffic management capabilities.

In conclusion, whether it be the new architecture of Ingress Controller or the implementation of the Kubernetes Gateway API, the goal is to offer users a more robust, flexible, and user-friendly Ingress Controller solution to meet the ever-changing demands of cloud-native application deployment and traffic management.