← Back to Recording

Summary

Overview

This course module provides a comprehensive overview of Kubernetes ingress and the evolving Gateway API ecosystem, with a focus on practical deployment and troubleshooting. It contrasts the legacy Ingress API with the modern Gateway API, explains the role of CNI plugins (particularly Cilium), and walks through hands-on configuration of an Nginx-based ingress controller in a minikube environment. The session concludes with a transition to Cilium for advanced networking, security, and encryption capabilities, emphasizing real-world migration challenges and operational best practices.

Topic (Timeline)

1. Ingress and Gateway API Fundamentals [00:00:03 - 00:03:47]

  • Introduces Ingress as a Kubernetes resource for routing external traffic to services using defined rules, managed via the Kubernetes API.
  • Clarifies confusion between the Ingress API (legacy) and Gateway API (modern successor); notes that Gateway API is not a drop-in replacement and requires migration of Ingress resources.
  • Explains that Gateway API relies on two core custom resource definitions (CRDs): Gateway and HTTPRoute, enabling more flexible, standardized, and scalable traffic management.
  • Mentions that Gateway API supports TLSRoute for direct database-to-gateway connections, though this is increasingly uncommon due to native pod-to-pod and node-to-node encryption in modern clusters.
  • Notes that the Ingress API’s Nginx-based controller (used for basic authentication on management pages) lacks equivalent functionality in Gateway API, prompting a pending upstream feature request for authentication layer replacement.

2. Gateway API Implementation and CNI Integration [00:03:48 - 00:06:50]

  • Highlights that Gateway API implementations are typically provided by CNI vendors, not core Kubernetes.
  • Introduces Cilium as a modern CNI plugin that supports both Ingress and Gateway API, offering layer 3–7 network policies, IPAM, and native encryption for pod-to-pod and node-to-node traffic.
  • Notes Cilium’s ability to replace Envoy proxy, improving cluster reliability by eliminating single points of failure inherent in older proxy-based solutions.
  • Discusses historical friction between CNI teams and MetalLB, reflecting ecosystem fragmentation and differing engineering priorities.
  • Emphasizes that modern CNIs like Cilium improve cluster resilience by eliminating dependency on unstable components like Envoy proxy.

3. Practical Ingress Deployment with Nginx Controller [00:06:50 - 00:17:14]

  • Walks through enabling the minikube Nginx ingress controller, revealing three components: Admission Create (Job), Admission Patch (Job), and Nginx Controller (Deployment).
  • Demonstrates that completed Jobs persist indefinitely in the cluster without TTL, a potential operational concern.
  • Deploys an example application (EngineX) with a NodePort service and verifies connectivity via minikube service.
  • Creates an Ingress resource targeting the EngineX service, but encounters a 503 error due to namespace mismatch: Ingress is deployed in coupsystem, while the service resides in default.
  • Troubleshoots by inspecting Ingress events and service endpoints, identifying the namespace misalignment as the root cause.
  • Fixes the issue by modifying the Ingress YAML to target the correct namespace (default) and reapplies; confirms successful routing via browser access to the FQDN (engine-x.example).

4. Transition to Cilium and Cluster Environment Reset [00:17:14 - 00:20:12]

  • Shifts focus to Cilium as a superior, all-in-one networking and security solution, contrasting it with complex alternatives like Istio (which requires sidecars and specialized expertise).
  • Notes Cilium’s ease of adoption for teams migrating from simpler CNI solutions like kube-proxy.
  • Demonstrates cluster reset: stops minikube, deletes existing resources, and restarts to ensure a clean environment.
  • Verifies Cilium deployment across all three minikube nodes, showing Cilium Agent pods running on each node and a single Cilium Operator managing control plane functions.
  • Confirms presence of Envoy pods (likely remnants or sidecars), indicating Cilium’s compatibility with existing proxy patterns while offering native alternatives.

Appendix

Key Principles

  • Ingress vs. Gateway API: Ingress is legacy; Gateway API is the future, requiring CRDs and manual migration.
  • Namespace Isolation: Ingress resources must match the namespace of their target services unless explicitly shared via Gateway API labels or multi-namespace configurations.
  • CNI Choice Matters: Modern CNIs like Cilium provide integrated security, encryption, and proxy replacement, reducing operational complexity.
  • Job Persistence: Admission Jobs created during ingress enablement persist indefinitely unless cleaned up manually.

Tools Used

  • minikube: Local Kubernetes environment for testing.
  • Nginx Ingress Controller: Legacy ingress implementation used for demonstration.
  • Cilium: Modern CNI with built-in Gateway API support, encryption, and policy enforcement.
  • kubectl: Used for inspecting pods, services, ingress, and events.
  • nslookup: Used to verify DNS resolution for FQDNs during ingress testing.

Common Pitfalls

  • Namespace mismatch between Ingress and Service → 503 errors.
  • Assuming Gateway API is backward-compatible with Ingress → Requires full resource conversion.
  • Leaving completed Jobs (e.g., Admission Create/Patch) in cluster indefinitely → Resource bloat.
  • Relying on Envoy proxy → Single point of failure; replaced by Cilium’s native datapath.

Practice Suggestions

  • Practice migrating a simple Ingress resource to Gateway API using Gateway and HTTPRoute CRDs.
  • Deploy Cilium on minikube and compare its network policies with Calico or Flannel.
  • Create a multi-namespace Ingress scenario using Gateway API’s shared gateway feature.
  • Monitor and clean up orphaned Jobs after ingress controller installation.
  • Test DNS resolution for Ingress-hosted services using local /etc/hosts entries before relying on external DNS.