Kubernetes Labels and Annotations

As your application grows, you end up with dozens or hundreds of Kubernetes objects: Pods, Deployments, Services and more. You need a way to say "all the production Pods" or "everything that belongs to the Product API". Kubernetes gives you two built-in tools for this: labels and annotations.

Think of a filing cabinet. Labels are the coloured stickers on the folders: short, standard tags you can sort and search by. Annotations are the notes written on the back of a folder: longer details that people and tools read, but that you never search by.

Both are simple key/value pairs in an object's metadata. Labels are what connect objects to each other. A Service finds its Pods through labels, and a Deployment finds the Pods it manages through labels too.

What you will learn

1. Why labels exist and the rules for writing label keys and values
2. How to filter objects with equality-based, set-based and existence selectors
3. How Deployments, Services and other objects use selectors to find each other
4. When to use an annotation instead of a label
5. How to design a simple label strategy for a multi-environment .NET application

Why labels exist

Kubernetes was built to run software at scale. In production you rarely have one of anything. You have several replicas, several versions and several environments, and labels let you reason about sets of objects instead of single items.

Imagine you run a Product API in three environments (prod, staging and dev) and you are testing two versions at the same time. How do you delete only the staging Pods? How does a Service know which Pods should get traffic? Labels answer both questions.

Two ideas shaped the design of labels:

1. Production abhors a singleton. A single instance always grows into a set. Labels let Kubernetes work with sets instead of isolated objects.
2. No fixed hierarchy fits everyone. The structure of an application changes over time. Labels can express any grouping without locking you into a rigid tree.

Label syntax

A label is a key/value pair where both parts are strings. The key has an optional prefix and a required name, separated by a slash: prefix/name.

1. Prefix (optional): a valid DNS subdomain of at most 253 characters, such as acme.com or app.kubernetes.io.
2. Name (required): at most 63 characters. It must start and end with a letter or digit, and may contain dashes (-), underscores (_) and dots (.) in between.
3. Value: at most 63 characters and may be empty. A non-empty value follows the same character rules as the name.

A prefix signals ownership. The kubernetes.io/ and k8s.io/ prefixes are reserved for Kubernetes itself. Use your own domain as a prefix (for example acme.com/) for keys your organisation or your tools own, so they never collide with keys from other tools.

Kubernetes also recommends a shared set of keys, such as app.kubernetes.io/name, app.kubernetes.io/version and app.kubernetes.io/part-of. Tools such as Helm and dashboards understand them. This lesson uses the short keys app, env and ver to keep the examples readable; the ideas are the same.

Labels in a manifest

Labels live in the metadata.labels field of any object. In a Deployment they usually appear in two places:

1. On the Deployment itself (metadata.labels): they describe the Deployment object.
2. On the Pod template (spec.template.metadata.labels): they are copied to every Pod the Deployment creates. The Deployment's selector must match these labels.

Here is a Deployment for a .NET Product API with three labels: app, env and ver.

The selector.matchLabels block ties the Deployment to its Pods. When Kubernetes counts replicas or rolls out an update, it finds the Pods by querying for app=product-api,env=prod.

Notice that ver is on the Pods but not in the selector. Keep the selector to labels that never change for the life of the Deployment, and put changing details such as the version only on the template.

Label selectors

A label selector is a query that matches objects by their labels. You use selectors in two places: in kubectl commands, and inside object definitions (this is how a Service finds its Pods).

On the command line you pass a selector with --selector or its short form -l. Both mean the same thing.

Equality-based selectors

The simplest form matches one key/value pair:

Separate several conditions with a comma. A comma means AND, so every condition must be true:

Exclude a value with !=:

Set-based selectors

Match a key against a list of values with in and notin:

Existence selectors

Check whether a label key exists at all, whatever its value:

Or select objects that do not have the label. Use single quotes so your shell does not treat ! as a special character:

Selector operator reference

Note that != and notin also match objects that do not have the key at all.

Selectors inside API objects

Inside a manifest, selectors are written in YAML. Most objects, including Deployments, ReplicaSets, DaemonSets and Jobs, use the newer form with matchLabels and matchExpressions:

Every condition in matchLabels and matchExpressions is combined with AND. The valid operators are In, NotIn, Exists and DoesNotExist.

Older object types, such as Services and ReplicationControllers, only support equality. Their selector is a plain key/value map:

This means app=product-api AND env=prod. It is simpler, but it cannot express set-based conditions.

How Kubernetes uses labels

Kubernetes is deliberately decoupled. No single component owns everything; objects find each other through labels and selectors:

1. A Deployment uses a selector to find and manage its Pods through a ReplicaSet.
2. A Service uses a selector to decide which Pods receive its traffic.
3. A NetworkPolicy uses selectors to define which Pods may talk to each other.
4. Pod affinity rules use selectors to place a Pod near (or away from) other Pods.

Annotations

Annotations are also key/value pairs, but they serve a different purpose. Labels identify and group objects; annotations describe them. You cannot use annotations in a selector. They are there for people and tools that read the object.

Common uses:

1. A Git commit hash or build tag, for example acme.com/git-commit: a3f2c1d
2. A link to the CI/CD pipeline run that created the deployment
3. A human-readable reason for the last change, for example kubernetes.io/change-cause: "Bumped memory for Black Friday"
4. Contact details for the team that owns the service
5. Configuration read by other tools, such as Prometheus, Istio or Fluentd
6. Settings for alpha features that do not yet have a proper API field

Annotation syntax

Annotation keys follow the same format as label keys: an optional DNS subdomain prefix and a required name. Because tools usually write annotations, a prefix matters even more to avoid collisions. Keys you will meet in practice:

1. kubernetes.io/change-cause: records why a rollout happened; kubectl rollout history shows it
2. deployment.kubernetes.io/revision: set by the Deployment controller to track revisions for rollback
3. acme.com/build-pipeline: your own CI/CD system's metadata

Annotation values are free-form strings. You can store a URL, a short note or even a JSON document, and the API server does not check the format. The only limit is size: all annotations on one object together must stay under 256 KiB.

Annotations go in metadata.annotations, next to the labels:

This snippet shows only the metadata part; the spec is the same as in the earlier Deployment.

Labels or annotations?

Everyday commands

Two details are easy to miss. A trailing minus sign (canary-) removes a key. And changing a key that already exists needs --overwrite; without it, kubectl refuses and leaves the old value in place.

Hands-on: label a multi-environment .NET application

You will deploy two .NET microservices across two environments, connect a Service to the production Pods with a selector, filter everything with selectors, and record build details with annotations.

Step 1 — Build the Product API image

Save this Dockerfile in the root of your ASP.NET Core project:

Build and push it to your registry (repeat with tag 2.0 for the staging version, and do the same for the Inventory API):

Step 2 — Create the production Product API Deployment

Save the Deployment from earlier in this lesson as product-api-prod-deployment.yaml:

Step 3 — Create the staging Product API Deployment

The staging copy runs version 2 with one replica. Only the name, labels, image, environment and CPU limit change. Save it as product-api-staging-deployment.yaml:

Step 4 — Create the production Inventory API Deployment

A second service in production, already on version 2. Save it as inventory-api-prod-deployment.yaml:

Step 5 — Route traffic to production with a Service

Save this Service as product-api-service.yaml:

The spec.selector uses the plain equality form. The Service sends traffic to any Pod that has both app=product-api and env=prod, so the staging Pods are left out automatically.

Step 6 — Apply the manifests

Step 7 — Look at the labels

List the Deployments with all their labels:

Expected output:

Do the same for Pods. Each Pod got its labels from the template of its Deployment:

Expected output (Pod name suffixes will differ; Kubernetes also adds a pod-template-hash label, left out here):

A long LABELS column is hard to read. -L turns chosen labels into their own columns:

Expected output:

Step 8 — Filter with selectors

Each command below answers a different question about the same five Pods:

For example, --selector="ver=2" returns the staging Product API Pod and both Inventory API Pods:

Now check that the Service picked the right Pods. The EndpointSlice of a Service lists the Pod IPs it selected:

Expected output (two addresses, the two production Product API Pods; your IPs will differ):

Compare those IPs with kubectl get pods -l "app=product-api,env=prod" -o wide. The staging Pod is not in the list.

Step 9 — Mark a canary with a label

Add a canary label to the staging Deployment:

List only the Deployments that do not carry it:

Expected output:

Combine conditions, for example version 2 objects that are not canaries. Run it against Deployments, because the label is only on the Deployment and not on its Pods:

Change the value, then remove the label again:

Step 10 — Record build details with annotations

Add operational metadata to the production Deployment. One command can set several annotations:

Check that they were stored:

Look for the Annotations: section near the top of the output:

Update an annotation (note --overwrite) and remove one you no longer need:

Step 11 — Clean up by label

Labels make cleanup easy. Preview what a selector matches first, then delete:

Common mistakes

1. Unquoted values. ver: 1 or canary: true in YAML is not a string, and the object is rejected. Quote them.
2. Selector and template out of sync. A Deployment whose selector does not match spec.template.metadata.labels is rejected by the API server.
3. Trying to change a Deployment selector. In apps/v1 the selector is immutable after creation. Delete and recreate the Deployment instead, so choose stable selector labels from the start.
4. Expecting Pods to inherit new Deployment labels. They do not; change the Pod template instead.
5. Forgetting --overwrite. Changing an existing label or annotation fails without it.
6. Using ! in double quotes. Shells such as bash may expand it. Put selectors with ! in single quotes.

Summary

Labels are short key/value pairs that identify and group Kubernetes objects. Deployments use them to manage Pods, Services use them to route traffic, and you use them to filter and clean up with kubectl. Annotations are key/value pairs too, but they carry descriptive metadata for tools and people and can never be used in a selector.

1. Give every object a consistent set of labels: at least the application name, the environment and the version.
2. Select with =, !=, in, notin, key and !key; commas mean AND.
3. Deployments, ReplicaSets and Jobs use matchLabels and matchExpressions; Services use a plain equality map.
4. Keep selectors to stable labels; they are immutable on a Deployment.
5. Use annotations for Git hashes, pipeline links, owners, change causes and tool configuration.
6. kubectl label and kubectl annotate add, change (with --overwrite) and remove (with a trailing -) metadata on live objects.

With labels and annotations in place, you can organise any cluster your way, without forcing a rigid hierarchy on your application.