Lesson 31 of 66 · 13 min read · Updated

Installing Envoy Gateway and Exposing Your First .NET API

In the previous article you saw why Ingress is being left behind and audited what a migration has to cover. The audit ended with an empty result: on a fresh minikube cluster, kubectl get crd | grep gateway.networking.k8s.io printed nothing. Gateway API is not built into Kubernetes, so the first job is to install it.

You do not install Gateway API on its own. You install an implementation, which brings the CRDs, a controller that watches them, and the proxies that actually carry traffic. This series uses Envoy Gateway, the Envoy project's own Gateway API controller. Everything you write in the Gateway API resources would work the same on any other conformant implementation.

By the end of this article a request to storefront.local will travel through an Envoy proxy and land on one of the three kiosk-api Pods, without a single annotation.

We will cover the following topics:

  1. How Envoy Gateway is structured: a control plane and per-Gateway data planes
  2. Installing Envoy Gateway v1.9.2 with Helm, and what the chart installs
  3. Creating the eg GatewayClass
  4. Creating the storefront-gateway Gateway with an HTTP listener
  5. Writing your first HTTPRoute for kiosk-api
  6. Reaching the Gateway with kubectl port-forward and with minikube tunnel
  7. Reading the status conditions that tell you whether everything was accepted

Technical Requirements

  1. A minikube cluster and kubectl.
  2. Helm 3.8 or later, because the chart is published as an OCI artifact.
  3. The kiosk-api:1.0 image, Deployment, and Service in the storefront namespace from the previous article. Step 1 recreates them if needed.
  4. curl for testing.

Core Concepts

Control Plane and Data Plane

Envoy Gateway has two layers, and it helps to keep them apart from the start.

The control plane is a single Deployment called envoy-gateway in the envoy-gateway-system namespace. It watches GatewayClasses, Gateways, and Routes, translates them into Envoy configuration, and pushes that configuration to the proxies. It never handles a user request.

The data plane is made of Envoy proxies. For every Gateway you create, Envoy Gateway creates a Deployment of Envoy Pods and a Service in front of them. Those Pods are what your users actually connect to.

LayerWhat it isCreated byCarries traffic?
Control planeenvoy-gateway DeploymentHelm installNo
Data planeEnvoy Deployment and Service per GatewayEnvoy Gateway, when you create a GatewayYes

So creating a Gateway is not just configuration. It provisions real infrastructure: Pods, a Service, and on a cloud cluster a load balancer.

What the Helm Chart Installs

The default Helm install applies three things:

  1. The Gateway API CRDs: GatewayClass, Gateway, HTTPRoute, GRPCRoute, ReferenceGrant, and others.
  2. The Envoy Gateway CRDs for its own extensions, such as EnvoyProxy, ClientTrafficPolicy, BackendTrafficPolicy, and SecurityPolicy.
  3. The control plane Deployment, its RBAC, and its webhooks.

The bundled Gateway API CRDs come from the experimental channel, which is a superset of the Standard channel. That lets Envoy Gateway offer alpha kinds such as TCPRoute. Everything this series uses is in the Standard channel, so your manifests stay portable.

If your platform already manages the Gateway API CRDs, for example a managed cloud cluster, you can tell the chart not to install them with --set crds.gatewayAPI.enabled=false. On minikube, let the chart do it.

The Three Resources You Will Create

GatewayClass is cluster-scoped. It names a controller. When you create a GatewayClass whose controllerName is gateway.envoyproxy.io/gatewayclass-controller, Envoy Gateway claims it and marks it Accepted. Any Gateway that references this class is then handled by Envoy Gateway.

Gateway is namespaced. It describes a point where traffic enters: which ports to listen on, with which protocol, for which hostnames. Each entry is a listener. A Gateway with one HTTP listener on port 80 is the simplest useful Gateway.

HTTPRoute is namespaced too. It attaches to a Gateway through parentRefs, declares which hostnames it serves, and contains rules. Each rule has matches (path, headers, query parameters, method) and backendRefs (the Services that receive the request).

ResourceScopeOwned byAnswers the question
GatewayClass egClusterPlatform/infrastructureWhich software implements my Gateways?
Gateway storefront-gatewayNamespacePlatform teamWhere does traffic enter, on which ports and hostnames?
HTTPRoute kiosk-apiNamespaceApplication teamWhich requests go to which Service?

How a Request Flows

Once everything is in place, a request follows this path:

  1. The client connects to the Envoy Service created for storefront-gateway.
  2. The Service forwards to one of the Envoy Pods.
  3. Envoy matches the Host header against the listener and the HTTPRoute hostnames, then the path against the rule matches.
  4. Envoy picks one of the kiosk-api Pod IPs directly. It reads the Service's EndpointSlices and load balances itself. It does not go through the kiosk-api ClusterIP.
  5. A kiosk-api Pod answers.

Step 4 surprises people. The backendRefs field names a Service, but Envoy uses the Service only to discover endpoints. That is why a Pod that fails its readiness probe disappears from Envoy's rotation just as it does from the Service.

Reaching the Gateway on minikube

The Envoy Service is of type LoadBalancer. On a cloud cluster it gets an external IP and the Gateway reports that IP in status.addresses. On minikube nothing provisions load balancers, so the address stays pending. There are two ways around that:

OptionHowWhen to use
kubectl port-forwardForward a local port to the Envoy ServiceQuick tests. Dies with the command.
minikube tunnelRun in a separate terminal. It assigns an IP to every LoadBalancer Service.When you want the Gateway to get a real address in its status.

Both are shown in the Step-by-Step section.

On Docker Desktop's built-in kind cluster the situation is different. Docker Desktop provisions LoadBalancer Services itself: the Envoy Service gets an address from Docker's 172.18.x.x network, the Gateway becomes Programmed=True without any tunnel, and every listener port is published on localhost. That address is not reachable from the host, so use http://localhost/ instead of the Gateway address. There is one catch: only one LoadBalancer Service can own a given localhost port. If another one already holds port 80, such as an Ingress NGINX controller from the previous article, the Envoy Service stays <pending> and the Gateway reports Programmed=False (AddressNotAssigned). The port-forward works in every case. On a plain kind cluster, run cloud-provider-kind to get LoadBalancer addresses.

Status Conditions

Every Gateway API resource reports what the controller thought of it. This is one of the biggest improvements over Ingress, and you should get into the habit of reading it.

ResourceConditionMeaning when True
GatewayClassAcceptedA controller claimed this class.
GatewayAcceptedThe controller understood the Gateway spec.
GatewayProgrammedThe proxies are configured and ready to receive traffic.
HTTPRoute (per parent)AcceptedThe route attached to that Gateway.
HTTPRoute (per parent)ResolvedRefsEvery backendRef points to something that exists and is allowed.

The next article looks at these conditions in depth, including the reasons you see when they are False.

Hands-On: Kubernetes Commands

Install Envoy Gateway with Helm

Installs the CRDs and the control plane into envoy-gateway-system.

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.9.2 -n envoy-gateway-system --create-namespace

Wait for the control plane

Blocks until the envoy-gateway Deployment is available.

kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available

List GatewayClasses with their acceptance

The ACCEPTED column should say True.

kubectl get gatewayclass

List Gateways with their address and readiness

PROGRAMMED tells you whether the proxies are ready.

kubectl get gateway -n storefront

List HTTPRoutes

Shows each route and the hostnames it serves.

kubectl get httproute -n storefront

Find the Envoy Service of a Gateway

Envoy Gateway labels the Services it creates with the Gateway's name and namespace.

kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=storefront,gateway.envoyproxy.io/owning-gateway-name=storefront-gateway

Print a route's status per parent

The quickest way to see why a route does not work.

kubectl get httproute kiosk-api -n storefront -o jsonpath='{range .status.parents[*].conditions[*]}{.type}={.status} ({.reason}){"\n"}{end}'

Uninstall Envoy Gateway

Removes the control plane. Delete your Gateways first so their proxies are cleaned up.

helm uninstall eg -n envoy-gateway-system

Step-by-Step Example

The Scenario

The storefront platform team has picked Envoy Gateway to replace Ingress NGINX. Your job is to install it, give the cluster a GatewayClass and a shared Gateway, and expose kiosk-api on storefront.local with an HTTPRoute that the kiosk team owns.

Step 1 — Make Sure kiosk-api Is Running

You need the storefront namespace, the kiosk-api Deployment with three replicas, and the kiosk-api ClusterIP Service on port 80. If you followed the previous article they are already there:

kubectl get deploy,svc -n storefront
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kiosk-api 3/3 3 3 14m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kiosk-api ClusterIP 10.105.41.212 <none> 80/TCP 14m

If not, follow Step 1 of the previous article. The app is the .NET 10 minimal API from the Services series. It returns the name and IP of the Pod that served the request, which makes load balancing visible.

Step 2 — Install Envoy Gateway

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.9.2 -n envoy-gateway-system --create-namespace
kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available
deployment.apps/envoy-gateway condition met

Check that the Gateway API CRDs now exist:

kubectl api-resources --api-group=gateway.networking.k8s.io
NAME SHORTNAMES APIVERSION NAMESPACED KIND
backendtlspolicies btlspolicy gateway.networking.k8s.io/v1 true BackendTLSPolicy
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1 true Gateway
grpcroutes gateway.networking.k8s.io/v1 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1 true ReferenceGrant
...

The exact list depends on the CRD version bundled with the chart. What matters is that GatewayClass, Gateway, and HTTPRoute are served as v1.

Step 3 — Create the GatewayClass

Save as 3-gatewayclass.yaml. This is usually done once per cluster by the platform team.

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
kubectl apply -f 3-gatewayclass.yaml
kubectl get gatewayclass
NAME CONTROLLER ACCEPTED AGE
eg gateway.envoyproxy.io/gatewayclass-controller True 6s

ACCEPTED: True means Envoy Gateway recognised its own controller name and claimed the class. If you mistype the controller name, the class stays unaccepted forever, because no controller claims it.

Step 4 — Create the Gateway

Save as 4-storefront-gateway.yaml. One listener, plain HTTP on port 80.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: storefront-gateway
namespace: storefront
spec:
gatewayClassName: eg
listeners:
- name: http # routes can target this listener by name
protocol: HTTP
port: 80
kubectl apply -f 4-storefront-gateway.yaml
kubectl get gateway -n storefront
NAME CLASS ADDRESS PROGRAMMED AGE
storefront-gateway eg False 5s

PROGRAMMED: False with an empty address is expected on minikube. The proxies are being created, and the LoadBalancer Service has no external IP. Look at what Envoy Gateway provisioned for you:

kubectl get deploy,svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-name=storefront-gateway
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envoy-storefront-storefront-gateway-5f2a8c1e 1/1 1 1 31s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envoy-storefront-storefront-gateway-5f2a8c1e LoadBalancer 10.98.170.21 <pending> 80:31652/TCP 31s

You never wrote a Deployment or a Service for Envoy. Creating the Gateway was enough. The names include the Gateway's namespace, its name, and a hash.

Step 5 — Create the HTTPRoute

Save as 5-kiosk-api-route.yaml. This is the object the kiosk team owns.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: kiosk-api
namespace: storefront
spec:
parentRefs:
- name: storefront-gateway # attach to this Gateway, same namespace
hostnames:
- storefront.local
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: kiosk-api # the ClusterIP Service
port: 80

Compare it with the Ingress from the previous article. Host, path, and backend are all still there, now as typed fields. The IngressClass is gone, replaced by a reference to a specific Gateway.

kubectl apply -f 5-kiosk-api-route.yaml
kubectl get httproute -n storefront
NAME HOSTNAMES AGE
kiosk-api ["storefront.local"] 4s

Step 6 — Check the Route Status

Before sending traffic, ask whether the route was accepted:

kubectl get httproute kiosk-api -n storefront -o jsonpath='{range .status.parents[*].conditions[*]}{.type}={.status} ({.reason}){"\n"}{end}'
Accepted=True (Accepted)
ResolvedRefs=True (ResolvedRefs)

Accepted means the route attached to storefront-gateway. ResolvedRefs means the kiosk-api Service exists. If either were False, the reason would tell you why. You will break both on purpose in the next article.

Step 7 — Send Traffic with port-forward

Find the Envoy Service by its labels and forward local port 8888 to its port 80:

export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=storefront,gateway.envoyproxy.io/owning-gateway-name=storefront-gateway -o jsonpath='{.items[0].metadata.name}')
kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &

Send a few requests with the right Host header:

for i in 1 2 3 4 5 6; do curl -s -H "Host: storefront.local" http://localhost:8888/; echo; done
{"servedByPod":"kiosk-api-6b9d7f5c9-2lq8v","podIp":"10.244.0.14","node":"minikube","path":"/"}
{"servedByPod":"kiosk-api-6b9d7f5c9-wz4tn","podIp":"10.244.0.16","node":"minikube","path":"/"}
{"servedByPod":"kiosk-api-6b9d7f5c9-hp7rd","podIp":"10.244.0.15","node":"minikube","path":"/"}
{"servedByPod":"kiosk-api-6b9d7f5c9-2lq8v","podIp":"10.244.0.14","node":"minikube","path":"/"}
{"servedByPod":"kiosk-api-6b9d7f5c9-hp7rd","podIp":"10.244.0.15","node":"minikube","path":"/"}
{"servedByPod":"kiosk-api-6b9d7f5c9-wz4tn","podIp":"10.244.0.16","node":"minikube","path":"/"}

All three Pods answer. Envoy load balances across the endpoints of the kiosk-api Service.

Step 8 — Prove That the Hostname Matters

Send the same request without the Host header, or with a different host:

curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8888/
curl -s -o /dev/null -w "%{http_code}\n" -H "Host: other.local" http://localhost:8888/
404
404

The listener accepts any hostname, but the only route attached to it serves storefront.local. Envoy finds no matching route and answers 404 itself. Your Pods never see these requests.

Stop the port-forward when you are done:

kill %1

Step 9 — Give the Gateway a Real Address with minikube tunnel

In a second terminal, start the tunnel and leave it running. It may ask for your password, because it changes the host's routing table.

minikube tunnel

Back in the first terminal, the Gateway now has an address and is fully programmed:

kubectl get gateway -n storefront
NAME CLASS ADDRESS PROGRAMMED AGE
storefront-gateway eg 10.98.170.21 True 6m

Read the address from the Gateway status and call it directly:

export GATEWAY_HOST=$(kubectl get gateway/storefront-gateway -n storefront -o jsonpath='{.status.addresses[0].value}')
curl -s -H "Host: storefront.local" http://$GATEWAY_HOST/

On Docker Desktop's kind cluster you need neither the tunnel nor the address: curl -s -H "Host: storefront.local" http://localhost/ reaches the Gateway as soon as it is Programmed.

On a cloud cluster this is the normal state: the address is the cloud load balancer, and you point the DNS record for storefront.local at it.

Step 10 — Clean Up

Delete the route and the Gateway. Deleting the Gateway also deletes the Envoy Deployment and Service that were created for it. Keep the GatewayClass, Envoy Gateway, and kiosk-api, because the next article builds on them.

kubectl delete httproute kiosk-api -n storefront
kubectl delete gateway storefront-gateway -n storefront
kubectl get deploy,svc -n envoy-gateway-system

Stop minikube tunnel with Ctrl+C.

Summary

  1. Gateway API is installed by an implementation. This series uses Envoy Gateway v1.9.2, installed with one Helm command that also brings the Gateway API CRDs.
  2. Envoy Gateway has a control plane (envoy-gateway) and creates a data plane of Envoy Pods and a Service for every Gateway.
  3. A GatewayClass names the controller. eg with gateway.envoyproxy.io/gatewayclass-controller is claimed by Envoy Gateway.
  4. A Gateway declares listeners. One HTTP listener on port 80 is enough to start.
  5. An HTTPRoute attaches to a Gateway with parentRefs and sends matching requests to backendRefs. Envoy load balances directly across the Service's Pod endpoints.
  6. On minikube, reach the Gateway with kubectl port-forward or give it an address with minikube tunnel.
  7. Always read status conditions: Accepted and Programmed on the Gateway, Accepted and ResolvedRefs on each route parent.

In the next article you will take the resource model apart: listeners, allowedRoutes, sectionName, hostname matching, and what each status condition says when something goes wrong.