Almost all applications workloads need to interact with world outside Kubernetes cluster to function as intended. Exposing Services objects using LoadBalancer or NodePort type are one of the common ways. However, at times, you may need functionality like SSL/TLS termination, virtual hosting functions etc. To use such features, you can use Ingress resource type. It exposes exposes HTTP and HTTPS routes from outside the cluster to Services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.

Do note that you would need an Ingress Controller to implement the Ingress Resource. There is no standard Ingress controller that is built into kubernetes, so the user must install one of many optional implementations. You can think of Ingress Controllers as pluggable mechanisms.

There are multiple reasons that Ingress resource has been created like this. The most common of them all is that there is no single http load balancer that can satisfy the requirement of all application workloads. Then there are on-premise and cloud based versions and hardware based load balancers as well.

You can find list of supported ingress controllers here. You can also use multiple ingress controllers in the same cluster, although that is for advanced scenarios.

For the purpose of this post, we’ll use traefik as Ingress Controller. We can see more information about same using below command:

cloud_user@d7bfd02ab81c:~$ kubectl describe service traefik -n kube-system
Name:                     traefik
Namespace:                kube-system
Labels:                   app.kubernetes.io/instance=traefik
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=traefik
                          helm.sh/chart=traefik-9.18.2
Annotations:              meta.helm.sh/release-name: traefik
                          meta.helm.sh/release-namespace: kube-system
Selector:                 app.kubernetes.io/instance=traefik,app.kubernetes.io/name=traefik
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.43.228.176
IPs:                      10.43.228.176
LoadBalancer Ingress:     172.18.0.2, 172.18.0.3, 172.18.0.4, 172.18.0.5
Port:                     web  80/TCP
TargetPort:               web/TCP
NodePort:                 web  31295/TCP
Endpoints:                10.42.0.89:8000
Port:                     websecure  443/TCP
TargetPort:               websecure/TCP
NodePort:                 websecure  31127/TCP
Endpoints:                10.42.0.89:8443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Creating an Ingress Resource

The simplest way to use Ingress is to have it just blindly pass everything that it sees through to an upstream service. Consider the below sample ingress manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
spec:
  defaultBackend:
    service:
      name: test
      port:
        number: 8080

Lets go ahead and create the ingress resource:

cloud_user@d7bfd02ab81c:~$ kubectl apply -f simple-ingress.yaml 
ingress.networking.k8s.io/test-ingress created

cloud_user@d7bfd02ab81c:~$ kubectl get ingress 
NAME           CLASS    HOSTS   ADDRESS   PORTS   AGE
test-ingress   <none>   *                 8080      25s

Do note that creating an ingress resource like above, is not sufficient. We need to have upstream pods and services to satisfy the traffic routed from ingress as well. We can see the more details about ingress state using kubectl describe command:

cloud_user@d7bfd02ab81c:~$ kubectl describe ingress test-ingress
Name:             test-ingress
Namespace:        default
Address:          
Default backend:  test:8080 (<error: endpoints "test" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           *     test:8080 (<error: endpoints "test" not found>)
Annotations:  <none>
Events:       <none>

We can see that ingress resource is complaining about not able to find upstream service named test.

Lets create an service and deployment controller to create and maintain pods with below manifests:

apiVersion: v1
kind: Service
metadata:
   name: test
spec:
  selector:
    app: demoui
  ports:
  - name: http
    port: 8080
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  labels:
    env: dev
    app: demoui
spec:
  replicas: 3
  selector:
    matchLabels:
      app: demoui
  template:
    metadata:
      labels:
        app: demoui
    spec:
      containers:
      - name: demoui
        image: docker.io/mohitgoyal/demo-ui01:1.0.0
        imagePullPolicy: Always
        ports:
        - name: http
          containerPort: 80
          protocol: TCP

After this, we should be able to see proper endpoints reflects in the ingress:

cloud_user@d7bfd02ab81c:~$ kubectl describe ingress test-ingress
Name:             test-ingress
Namespace:        default
Address:          
Default backend:  test:8080 (10.42.1.58:80,10.42.2.26:80,10.42.3.23:80)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           *     test:8080 (10.42.1.58:80,10.42.2.26:80,10.42.3.23:80)
Annotations:  <none>
Events:       <none>

We can now access our application from outside cluster:

cloud_user@d7bfd02ab81c:~$ curl http://localhost:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to Demo Web App 01!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to Demo Web App 01</h1>
<p>If you see this page, the underlying nginx web server is successfully installed and
working. Further configuration is required.</p>

Define Ingress with Minimal Rules

Now that we are familiar with Ingress resource, lets define an ingress controller with minimal rules for traffic distribution. Consider below example where we have defined that incoming traffic on port 8080 should be redirected to upstream service test (same as above use case):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: test
            port:
              number: 8080

Lets create the controller and see it in action:

# update ingress configuration
cloud_user@d7bfd02ab81c:~/workspace$ kubectl apply -f ingress-minimal.yaml 
ingress.networking.k8s.io/minimal-ingress configured


# check if ingress seems alive
cloud_user@d7bfd02ab81c:~/workspace$ kubectl get ingress 
NAME              CLASS    HOSTS   ADDRESS                                       PORTS   AGE
minimal-ingress   <none>   *       172.18.0.2,172.18.0.3,172.18.0.4,172.18.0.5   80      2m53s


# verify new rules on ingress are working
cloud_user@d7bfd02ab81c:~/workspace$ kubectl describe ingress minimal-ingress
Name:             minimal-ingress
Namespace:        default
Address:          172.18.0.2,172.18.0.3,172.18.0.4,172.18.0.5
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           
              /   test:8080 (10.42.1.61:80,10.42.2.29:80,10.42.3.26:80)
Annotations:  <none>
Events:       <none>


# verify that we are able to access application
cloud_user@d7bfd02ab81c:~/workspace$ curl http://localhost:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to Demo Web App 01!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to Demo Web App 01</h1>
<p>If you see this page, the underlying nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Few things to note about ingress configuration is how Rules and Default backend properties looks different. We no longer seem to have working default backend.

Default Backend

An Ingress with no rules sends all traffic to a single default backend. The defaultBackend is conventionally a configuration option of the Ingress controller and is not specified in Ingress resources. However what we can do is to make sure that we define a rule for path /. This matches all incoming traffic by default.

If none of the hosts or paths match the HTTP request in the Ingress objects, the traffic is routed to default backend.

Incoming Traffic Rules

Each traffic rule contains the following information:

• An optional host. In this example, no host is specified, so the rule applies to all inbound HTTP traffic through the IP address specified. If a host is provided (for example, foo.bar.com), the rules apply to that host.
• A list of paths (for example, /testpath), each of which has an associated backend defined with a service.name and a service.port.name or service.port.number. Both the host and path must match the content of an incoming request before the load balancer directs traffic to the referenced Service.
• A backend is a combination of Service and port names 

Note that we keep saying HTTP traffic, as Ingress is designed to handle only HTTP traffic. TCP traffic has to be handled by using Service types of LoadBalancer or NodePort.

Distributing Traffic based on URI

An Ingress can be used to route traffic from a single IP address to more than one service, based on the HTTP URI being requested. It helps you to keep the number of load balancers down to a minimum. For example, consider below manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-path
spec:
  rules:
  - http:
      paths:
      - path: /test01
        pathType: Prefix
        backend:
          service:
            name: test
            port:
              number: 8080
      - path: /test02
        pathType: Prefix
        backend:
          service:
            name: test02
            port:
              number: 8080

Lets create ingress with above manifest:

cloud_user@d7bfd02ab81c:~/workspace$ kubectl describe ingress ingress-path
Name:             ingress-path
Namespace:        default
Address:          172.18.0.2,172.18.0.3,172.18.0.4,172.18.0.5
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           
              /test01   test:8080 (10.42.1.62:80,10.42.2.30:80,10.42.3.27:80)
              /test02   test02:8080 (10.42.1.64:80,10.42.2.32:80,10.42.3.29:80)
Annotations:  <none>
Events:       <none>

The Ingress controller provisions an implementation-specific load balancer that satisfies the Ingress, as long as the Services (test, test02) exist. When it has done so, you can see the address of the load balancer at the Address field.

Do note that as requests get proxied to the upstream service, the path remains unmodified. That means that the upstream service needs to be ready to serve traffic on that subpath.

Distributing Traffic Based on Hostname

An Ingress can be used to route traffic from a single IP address to more than one service, based on the HTTP hostname being requested. For example, consider below ingress manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-hostname
spec:
  rules:
  - host: ui01.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: test
            port:
              number: 8080
  - host: ui02.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: test02
            port:
              number: 8080

which will create ingress with below properties:

cloud_user@d7bfd02ab81c:~/workspace$ kubectl apply -f ingress-host.yaml 
ingress.networking.k8s.io/ingress-hostname created

cloud_user@d7bfd02ab81c:~/workspace$ kubectl describe ingress ingress-host
Name:             ingress-hostname
Namespace:        default
Address:          172.18.0.2,172.18.0.3,172.18.0.4,172.18.0.5
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  ui01.com    
              /   test:8080 (10.42.1.62:80,10.42.2.30:80,10.42.3.27:80)
  ui02.com    
              /   test02:8080 (10.42.1.64:80,10.42.2.32:80,10.42.3.29:80)
Annotations:  <none>
Events:       <none>

If you do not have a domain or if you are using a local solution such as kind or K3s, you can set up a local configuration by editing your /etc/hosts file to add an IP address. You will need root privileges on your workstation to do so. The location or name of the file may differ depending on your OS distribution.

Serving TLS

Serving websites over TLS is one of the necessities. You can secure an Ingress by specifying a Secret that contains a TLS private key and certificate. The Ingress resource only supports a single TLS port, 443, and assumes TLS termination at the ingress point (traffic to the service and its pods is in plaintext).

The TLS secret must contain keys named tls.crt and tls.key that contain the certificate and private key to use for TLS. For example consider below manifest for creating a secret:

apiVersion: v1
kind: Secret
metadata:
  name: testsecret-tls
data:
  tls.crt: <base64 encoded cert>
  tls.key: <base64 encoded key>
type: kubernetes.io/tls

When creating a TLS Secret using kubectl, you can use the tls subcommand as shown in the following example:

kubectl create secret tls my-tls-secret \
  --cert=path/to/cert/file \
  --key=path/to/key/file

cloud_user@d7bfd02ab81c:~/workspace$ kubectl create secret tls tlssecret-tls --key tls.key --cert tls.crt
secret/tlssecret-tls created

Once you have the certificate uploaded, you can reference it in an Ingress object. This specifies a list of certificates along with the hostnames that those certificates should be used for. For example, consider below manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-example-ingress
spec:
  tls:
  - hosts:
      - https-example.foo.com
    secretName: testsecret-tls
  rules:
  - host: https-example.foo.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 8080

Do note that hostname used should be specified in the FQDN in the certificate properties.