Cloud Desktop Teaching Platforms

This course module provides a comprehensive overview of Kubernetes ingress and the evolving Gateway API ecosystem, with a focus on practical deployment and troubleshooting. It contrasts the legacy Ingress API with the modern Gateway API, explains the role of CNI plugins (particularly Cilium), and walks through hands-on configuration of an Nginx-based ingress controller in a minikube environment. The session concludes with a transition to Cilium for advanced networking, security, and encryption capabilities, emphasizing real-world migration challenges and operational best practices.

Topic (Timeline)

1. Ingress and Gateway API Fundamentals [00:00:03 - 00:03:47]

Introduces Ingress as a Kubernetes resource for routing external traffic to services using defined rules, managed via the Kubernetes API.
Clarifies confusion between the Ingress API (legacy) and Gateway API (modern successor); notes that Gateway API is not a drop-in replacement and requires migration of Ingress resources.
Explains that Gateway API relies on two core custom resource definitions (CRDs): Gateway and HTTPRoute, enabling more flexible, standardized, and scalable traffic management.
Mentions that Gateway API supports TLSRoute for direct database-to-gateway connections, though this is increasingly uncommon due to native pod-to-pod and node-to-node encryption in modern clusters.
Notes that the Ingress API’s Nginx-based controller (used for basic authentication on management pages) lacks equivalent functionality in Gateway API, prompting a pending upstream feature request for authentication layer replacement.

2. Gateway API Implementation and CNI Integration [00:03:48 - 00:06:50]

Highlights that Gateway API implementations are typically provided by CNI vendors, not core Kubernetes.
Introduces Cilium as a modern CNI plugin that supports both Ingress and Gateway API, offering layer 3–7 network policies, IPAM, and native encryption for pod-to-pod and node-to-node traffic.
Notes Cilium’s ability to replace Envoy proxy, improving cluster reliability by eliminating single points of failure inherent in older proxy-based solutions.
Discusses historical friction between CNI teams and MetalLB, reflecting ecosystem fragmentation and differing engineering priorities.
Emphasizes that modern CNIs like Cilium improve cluster resilience by eliminating dependency on unstable components like Envoy proxy.

3. Practical Ingress Deployment with Nginx Controller [00:06:50 - 00:17:14]

Walks through enabling the minikube Nginx ingress controller, revealing three components: Admission Create (Job), Admission Patch (Job), and Nginx Controller (Deployment).
Demonstrates that completed Jobs persist indefinitely in the cluster without TTL, a potential operational concern.
Deploys an example application (EngineX) with a NodePort service and verifies connectivity via minikube service.
Creates an Ingress resource targeting the EngineX service, but encounters a 503 error due to namespace mismatch: Ingress is deployed in coupsystem, while the service resides in default.
Troubleshoots by inspecting Ingress events and service endpoints, identifying the namespace misalignment as the root cause.
Fixes the issue by modifying the Ingress YAML to target the correct namespace (default) and reapplies; confirms successful routing via browser access to the FQDN (engine-x.example).

4. Transition to Cilium and Cluster Environment Reset [00:17:14 - 00:20:12]

Shifts focus to Cilium as a superior, all-in-one networking and security solution, contrasting it with complex alternatives like Istio (which requires sidecars and specialized expertise).
Notes Cilium’s ease of adoption for teams migrating from simpler CNI solutions like kube-proxy.
Demonstrates cluster reset: stops minikube, deletes existing resources, and restarts to ensure a clean environment.
Verifies Cilium deployment across all three minikube nodes, showing Cilium Agent pods running on each node and a single Cilium Operator managing control plane functions.
Confirms presence of Envoy pods (likely remnants or sidecars), indicating Cilium’s compatibility with existing proxy patterns while offering native alternatives.

Appendix

Key Principles

Ingress vs. Gateway API: Ingress is legacy; Gateway API is the future, requiring CRDs and manual migration.
Namespace Isolation: Ingress resources must match the namespace of their target services unless explicitly shared via Gateway API labels or multi-namespace configurations.
CNI Choice Matters: Modern CNIs like Cilium provide integrated security, encryption, and proxy replacement, reducing operational complexity.
Job Persistence: Admission Jobs created during ingress enablement persist indefinitely unless cleaned up manually.

Tools Used

minikube: Local Kubernetes environment for testing.
Nginx Ingress Controller: Legacy ingress implementation used for demonstration.
Cilium: Modern CNI with built-in Gateway API support, encryption, and policy enforcement.
kubectl: Used for inspecting pods, services, ingress, and events.
nslookup: Used to verify DNS resolution for FQDNs during ingress testing.

Common Pitfalls

Namespace mismatch between Ingress and Service → 503 errors.
Assuming Gateway API is backward-compatible with Ingress → Requires full resource conversion.
Leaving completed Jobs (e.g., Admission Create/Patch) in cluster indefinitely → Resource bloat.
Relying on Envoy proxy → Single point of failure; replaced by Cilium’s native datapath.

Practice Suggestions

Practice migrating a simple Ingress resource to Gateway API using Gateway and HTTPRoute CRDs.
Deploy Cilium on minikube and compare its network policies with Calico or Flannel.
Create a multi-namespace Ingress scenario using Gateway API’s shared gateway feature.
Monitor and clean up orphaned Jobs after ingress controller installation.
Test DNS resolution for Ingress-hosted services using local /etc/hosts entries before relying on external DNS.

Kubernetes Comprehensive 2-Day - lane-nbsb-20250627-030635

Visit NobleProg websites for related course

Summary

Overview