Jintao Zhang, Apache Committer, Kubernetes Ingress Nginx Reviewer, and contributor to several cloud-native open source projects.

Apache Ingress Overview​

Apache Ingress Definition​

In the K8s ecosystem, Ingress is a resource that represents the entry point for K8s traffic. To make it effective, an Ingress Controller is needed to listen to the Ingress resources in K8s and resolve the rules and actually carry the traffic for those resources. The most widely used Ingress Controller implementations in today's trends are Kubernetes Ingress Nginx.

Ingress is another implementation of the Ingress Controller. The main difference with Kubernetes Ingress Nginx is that Ingress uses Apache as the actual data plane that carries business traffic. As shown in the figure below, when a user requests a specific service/API/web page, the entire business traffic/user request is transferred to the K8s cluster through an external proxy and then processed by Ingress.

As you can see from the above diagram, Ingress is divided into two parts. One part is the Ingress Controller, which is the control plane that will do the configuration management and distribution. The other part is the Proxy Pod, which is responsible for carrying business traffic and is implemented by means of CRD (Custom Resource Definitions).

Apache in a nutshell​

We mentioned earlier that Ingress uses Apache as the data surface that actually carries business traffic, so what does the Apache project do?

Apache is the top open source project of the Apache Foundation and is currently the most active open source gateway project. As a dynamic, real-time, high-performance open source API gateway, Apache provides rich traffic management features such as load balancing, dynamic upstream, canary release, service meltdown, authentication, observability, and more.

Apache helps enterprises handle API and microservice traffic quickly and securely with features such as flow-limiting authentication, logging security features, and support for rich custom plug-ins. There are also currently relevant integrations with many open source projects such as Apache SkyWalking, Prometheus and other such components.

Ingress vs K8s Ingress Nginx​

Since I am involved in the development and maintenance of both Ingress and K8s Ingress Nginx, many people ask me how to choose between these two projects. Or why do you want to do Ingress when you have K8s Ingress Nginx.

Configuration level​

In Ingress, we have added some rich and flexible configurations, such as canary release deployment through a single configuration file. However, in K8s Ingress Nginx, you need at least two Ingress resource files to achieve the above effect.

Richness​

In terms of richness, because Apache is feature-rich and allows for multiple plugins, using Ingress eliminates the need to configure additional features yourself, allowing you to spend more time on actual development.

Architecture separation​

Ingress uses a separate architecture for the data plane and control plane, so users can choose to deploy the data plane inside/outside the K8s cluster. However, K8s Ingress Nginx puts the control plane and the data plane in the same Pod, so if something happens to the Pod or the control plane, the whole Pod will hang, which will affect business traffic.

This architectural separation gives users a more convenient deployment option and facilitates data migration and usage in business architecture adjustment scenarios.

Ingress Features Explained​

Since Apache is a fully dynamic, high-performance gateway, Ingress itself supports full dynamics, including routing, SSL certificates, upstream, plug-ins, and more.

Ingress also has the following features.

• CRD support for easier understanding of declarative configurations; while state checking ensures quick access to the synchronized state of declarative configurations
• Support for advanced route matching rules and custom resources, which can be extended with more than 50 official Apache plug-ins & customer-defined plug-ins
• Support for K8s native Ingress configuration
• Support for traffic slicing
• Support for gRPC plaintext and TCP Layer 4 proxy
• Automatic service registration and discovery, no fear of scaling up or down
• More flexible load balancing policy with health check function

The following is a detailed introduction to CRD and custom resources.

CRD Extensions​

We mentioned CRD in the previous introduction, but how does Ingress use CRD extensions?

At the user level, when a request is made by a Client and arrives at Apache , the corresponding service traffic is directly transferred to the backend (e.g., Service Pod), thus completing the forwarding process. This process does not need to go through the Ingress Controller, which can ensure that if there are problems, or changes, expansion and contraction or migration processing, it will not affect the users and business traffic.

On the configuration side, users can apply custom CRD configurations to K8s clusters via kubectl apply, and Ingress Controller will continuously watch these resource changes to apply the corresponding configurations to Apache .

Custom resources​

Ingress currently supports the following 5 categories of custom resources related to routing, upstream, consumer, certificate related and cluster public configuration.

Route​

The top-level configuration for the spec attribute in the custom resource Route is http. But actually spec supports both configurations, spec.http as in the example below, which is mainly used for Layer 7 proxies, and spec.stream, which is used for Layer 4 proxies. In the configuration file, we first customize a rule for it, namely the relevant parameter under match.

As the above back-end configuration example uses the same Service, you can adjust it according to the scenario in actual use. Note that the weight attribute is used to configure the relevant service weight. With the above configuration, a complete set of routing customization resources can be implemented.

Upstream (Upstream)​

When configuring Upstream, you need to pay attention to the content of name to be consistent with the Service of K8s cluster, so as to ensure that the subsequent Ingress Controller can accurately match its corresponding traffic.

In the configuration file, spec.loadbalancer is mainly responsible for setting the load balancing policy, and there are various policy modes to choose from. spec.scheme is for the protocol type configuration, currently only HTTP and gRPC protocols are supported. spec.healthCheck is mainly for setting the health check function, such as setting its active status, effective protocol and path and final feedback and other parameters configuration.

Consumer​

The Consumer configuration mainly adds authentication-related features, such as spec.authParameter, which currently supports BasicAuth and KeyAuth, the two more common types of authentication.

You can configure the associated username and password directly with value, or directly with secret, which is more secure than the plaintext configuration of the former.

TLS (certificate)​

TLS is mainly for certificate management. As the example shows, users can configure multiple domains via hosts, and the parameters under secret are the corresponding configuration certificates.

Apache TLS also comes with spec.client for configuring mTLS two-way authentication.

Config​

The Config types supported by custom resources are described in two ways.

One type is Cluster Config, which is mainly used for some generic configurations. Currently it supports global use of Prometheus plug-in/global configuration SkyWalking in K8s or Apache , and we will add some other generic configurations in subsequent development.

Another one is the Plugin Config that we are currently working on in PR. If you are interested, you can also click the link to join the discussion. Plugin Config is mainly a unified collection of common plug-in configurations, for example, some of the same configuration, users can use Plugin Config to apply to multiple routes at the same time, eliminating the tedious steps of multiple independent configurations.

Ingress Hands-On Practice​

Currently, you can deploy Ingress via Helm Charts. You can deploy both Apache and Ingress, including the etcd required for Apache , with a single command, which is very simple.

Practice Scenario 1: Traffic Segmentation​

By using Ingress, you can achieve the effect of proportional traffic slicing, as follows.

Step 1: Configure Upstream​

Step 2: Configure Route​

Configure subset and weight in backends to split the incoming user request traffic. The example below shows that 90% of the traffic will go to v1 and 10% of the traffic will go to v2.

With the above two steps, it is very easy to slice and dice traffic proportionally to achieve scenarios like canary release. For more details, please refer to: Traffic Segmentation in Apache Ingress Controller.

Practice Scenario 2: Configuring Authentication​

If you want to configure Basic Auth for certain routes in Ingress, you can refer to the following actions.

Step 1: Create Consumer resource​

As mentioned earlier, you can add basicAuth to the Consumer configuration and specify a username and password for it.

Step 2: Configure the Route and add authentication-related parameters​

In the custom resource Route, simply turn it on and specify the authentication type by adding authentication to the backend.

With the above steps, it is possible to use Consumer to complete the relevant configuration authentication.

Practice Scenario 3: K8s Resource Extensions​

As we mentioned at the beginning, Ingress supports not only custom resources, but also K8s native Ingress resources.

The image above shows the K8s Ingress resource. Normally if you want to do rewrite on a resource, you can add annotation configuration attributes. This way when the user carries the httpbin.org request, it can be redirected to /ip via the path /sample.

When the above requirement uses Ingress, simply add a kubernetes.io/ingress.class: to Ingress to specify the Ingress Controller to listen to this resource, and configure k8s..apache.org/rewrite-target: "/ip" to complete the redirection to the /ip path.

The above example is just one of the ways Ingress currently supports native K8s Ingress, for more examples you can check the specific documentation for reference and use.

Future Plans​

Ingress will continue to be updated in terms of functionality and ecology, and the current phase has been completed Ingress and Cert-manager integration, and the following goals will be achieved step by step.

1. complete Kubernetes V1.22+ and CRD V1 version adaptation support (completed, soon to be released in Ingress V1.3 version)
2. support for Gateway API (expected in Q4)
3. extend the new architecture to allow users to use Ingress without the need to use etcd
4. enrich the product ecology and expand the Ingress community

Finally, we hope you can participate in the project more often, for example, there will be an Ingress community meeting every two weeks on Wednesday at 2 pm, and we will synchronize the current progress of the project or the problems we encountered. Keep an eye on the Apache video to participate in the live community meetings.

Click here for more details about the Ingress community meetings.