Beyond Annotation Spaghetti: How to Architect Multi-Tenant Traffic with the Kubernetes Gateway API
The legacy Kubernetes Ingress resource attempted to force three distinct organizational personas into a single monolithic API object:


Cluster/Infra Operators: Managing public IPs, TLS certificates, firewall rules, and cloud load balancers.


Platform Engineers: Enforcing routing policies, rate limits, and authentication guards.


Application Developers: Directing URI paths to specific internal service ports.


Because Ingress lacked expressive native primitives for advanced routing (like canary splits, header rewriting, or multi-namespace attachment), platforms became dependent on custom controller annotations.


The Gateway API decomposes monolithic ingress into a role-oriented, decoupled resource hierarchy:


[❌ Monolithic Ingress Anti-Pattern]
Single 'Ingress' Manifest (Annotated to death)
├── Infra Config (TLS, LoadBalancer IP)
├── Routing Rules (/api, /auth)
└── App Secrets
Result: No RBAC isolation; single namespace leak affects cluster-wide gateway.


[✅ Production Gateway API Architecture]
[Cluster Admin] --> GatewayClass (Defines controller: Cilium, Envoy, Cloud ALB)
│
[Platform / DevOps] --> Gateway (Manages listeners, VIP, TLS certs, allowed namespaces)
│
┌───────────────┴───────────────┐
▼ ▼
[App Team A / HTTPRoute] [App Team B / HTTPRoute]
Namespace: 'checkout' Namespace: 'catalog'
Path: /api/v1/checkout Path: /api/v1/products
Canary: 90% v1 / 10% v2 Direct Service Forwarding
To migrate production traffic cleanly without downtime, follow this 3-step transition pattern:


1. Decouple Infrastructure from Application Routing
Define the Gateway in an administrative namespace (e.g., infra-gateways) and declare explicit listener constraints:


YAML
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: external-edge-gateway
namespace: infra-gateways
spec:
gatewayClassName: envoy-gateway
listeners:
- name: https-prod
protocol: HTTPS
port: 443
tls:
mode: Terminate
certificateRefs:
- kind: Secret
name: wildcard-production-tls
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
gateway-access: "true"


2. Delegate Routing Authority via Scoped HTTPRoute
Application teams define their own routing inside their application namespaces by attaching to the parent gateway:


YAML
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: checkout-route
namespace: checkout-workloads
spec:
parentRefs:
- name: external-edge-gateway
namespace: infra-gateways
rules:
- matches:
- path:
type: PathPrefix
value: /api/v1/checkout
backendRefs:
- name: checkout-svc-v1
port: 8080
weight: 90
- name: checkout-svc-v2
port: 8080
weight: 10
Canary traffic splitting is now an expressive, first-class native specification with weighted backend references—no brittle custom controller annotations required.


3. Control Cross-Namespace References with ReferenceGrant
By default, the Gateway API blocks cross-namespace routing to enforce multi-tenant security. If an HTTPRoute in namespace A needs to forward traffic to a backend Service in namespace B, namespace B must explicitly issue a ReferenceGrant. This eliminates cross-tenant privilege escalation and unauthorized service snooping.


Discussion Question
Has your platform team started migrating production workloads from legacy Ingress controllers to Gateway API implementations (Envoy Gateway, Cilium, or Cloud-managed Gateways), and what was your primary blocker during DNS cutover?


CTA
Join Cloud, DevOps & Open Source to explore hands-on Kubernetes architecture patterns, GitOps workflows, and production infrastructure design with engineers worldwide.