Kubernetes | Kristian M. Eschenburg

Kubernetes for ML Engineering, Part 2: Isolation — Networking and RBAC

Mon, 21 Sep 2026 09:00:00 -0700

Introduction

In the last post, I set up a two-service architecture, containing a frontend dashboard, and a backend API. My demo included the K8s Deployment setup, the frontend NodePort service type, and covered some of the similarities of Kubernetes to AWS ECS, the platform I’m very familiar with.

In this post, I’ll expand on topics related to authentication, authorization, and networking. See below for a rough architecture diagram of this post’s content. Again, this is not a tutorial. This is a series of posts documenting my path to development with Kubernetes. In each post, I’ll progressively integrate more and more K8s functionality, culminating in an application towards ML training and inference pipelines.

Code for this post can be found here.

Architecture and Overview

The theme of this post is networking, roles, and permission boundaries. A shared cluster has multiple boundaries that matter:

the networking path between workloads
the permission surface against the K8s API server
the rules dictating cluster access (though technically, we could offload this to outside the cluster itself)

Part 2 architecture diagram. We still have the frontend dashboard, but I’ve added two backend services – one in the same namespace as the frontend (salary-backend), and another in a different namespace (user-backend). There’s also a third namespace, ‘dex’, with it’s own service for mocking OIDC authentication.

Given the networking theme, we need a service architecture that requires these types of components. We’ll have a few services this time, spanning different namespaces and permissions boundaries:

frontend service: displays user and user salary information via Python Dash application
salary service: gets salary data associated with users
users service: gets personal information about users
rates “service”: applies currency transforms to salaries
Dex service: mocks OIDC authentication

Part One: the network

Services across namespaces

The frontend and salary API exist in the client-a namespace, and the user API exists in the platform namespace. We can envision these two services as something a specific tenant or team might want to deploy. The salary API has within-namespace endpoints (e.g. access data within its own Pod), and across-namespace endpoints (e.g. calls users API in the platform namespace, calls rates service outside cluster).

Like we demonstrated last time, services within a namespace can resolve one another using the service names themselves. However, across namespaces, for one service to find another, we need DNS resolution. This means that, within the client-a namespace, “salary-backend” resolves just fine, but “users-backend” does not. In order for the salary service to find the users service, we need to provide the full A Record (since it maps directly to an IP address) user-backend.platform.svc.cluster.local as an environment variable in the client-a ConfigMap (since ConfigMaps are namespace-scoped).

apiVersion: v1
kind: ConfigMap
metadata:
 name: client-a
 namespace: client-a
data:
 # user-backend lives in the `platform` namespace, so the short name won't resolve from here
 user_backend_url: "http://user-backend.platform.svc.cluster.local:8000"
 salary_backend_url: "http://salary-backend:8001"
 # set to http://172.18.0.101 (fx-decoy), egress policy will block that access
 fx_url: "http://172.18.0.100"

To address some of those concerns, we’ve set up a few roles: We’ve got a few different roles associated with this architecture:

platform role (ClusterRole)
client-a role (ClusterRole, restricted to namespace)
observer role (ClusterRole, minimal permissions)

To handle the rules dictating cluster access, we’ve also implemented a system for creating users, setting up OIDC authentication, and establishing mTLS for cluster access using Dex.

Why are we even concerned with all of this? In a production grade ML system, we’ll likely have a multi-tenant architecture, with multiple tenants setting up their own training and inference systems. We’ll need a platform or admin-scoped role that let’s us control the cluster, and we’ll likely also want some roles related to observing the system, scoped more narrowly than the platform roles and restricted primarily to accessing logs.

Ingress and the Gateway API

Initially, I built this out using the IngressController but quickly saw that in a real-world application, this would become chaotic. Each IngressController belongs to its own namespace and sets up both the infrastructure and the network rules. Any change to any of those components requires rebuilding the entire resource suite.

I switched to using the Gateway Class, GatewayAPI, and HTTPRoute resources. GatewayClass is a cluster-scoped definition of the routing, while the GatewayAPI is a namespace-scoped instance of that infrastructure. The HTTPRoute resources mirror the “path” rules of the IngressControllers. If we want to add new paths, or additional rules, we set up new routes or route types, and these are automatically picked up by the GatewayAPI infrastructure (assuming label selectors are defined correctly). This quickly allows individual teams to deploy their own applications without disrupting existing services.

I set the Gateway up in the platform namespace to only accept routes from namespaces with specific labels e.g. “tier=platform” or “tier=tenant”. I created a “fake” hostname called “apps.part-2.test”, and created an entry for this in my local /etc/hosts file. Only the frontend paths are exposed. The frontend-service HTTPRoute lives in the client-a namespace, and attaches to the Gateway vie the parentRefs name + namespace label selectors.

Generic GatewayAPI architecture.

# would be owned by the platform / admin team
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
 name: apps
 namespace: platform
spec:
 gatewayClassName: envoy-gateway
 listeners:
 - name: http
 protocol: HTTP
 port: 8080
 hostname: apps.part-2.test
 allowedRoutes:
 namespaces:
 from: Selector
 selector:
 matchExpressions:
 - key: tier
 operator: In
 values: [platform, tenant]

---
# only frontend is exposed outside the cluster
# attaches to platform-owned Gateway
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
 name: frontend
 namespace: client-a
spec:
 parentRefs:
 - name: apps
 namespace: platform
 hostnames:
 - apps.part-2.test
 rules:
 - matches:
 - path:
 type: PathPrefix
 value: /frontend
 backendRefs:
 - name: part-2-frontend
 port: 8050

NetworkPolicy

We want to follow the principle of least privilege. Without any explicit policies, every Pod can reach every other Pod in every namespace. This obviously should not happen. One team’s services should not be able to call another team’s services unless that communication is explicitly allowed. We also don’t want non-admin resources to be able to access admin / platform resources directly.

To make these rules explicit, we use NetworkPolicy resources. NetworkPolicies can set rules at the Pod or Namespace level using label selectors, and/or at the CIDR level, and can specify Ingress and Egress rules. Anything that isn’t explicitly allowed, is denied. For our architecture, we have the following needs:

frontend (client-a namespace) can be accessed from outside cluster via the Gateway (platform namespace)
frontend (client-a namespace) needs to be able to call salary API (client-a namespace)
salary API (client-a namespace) needs to be able to call user API (platform namespace)
salary API needs to be able to call the fx-rates service (outside cluster)

client-a-ingress: frontend is only reachable through the gateway

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
 name: client-a-ingress
 namespace: client-a
spec:
 podSelector:
 matchLabels:
 app: part-2-frontend
 policyTypes:
 - Ingress
 ingress:
 - from:
 # for the gateway api
 - namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: envoy-gateway-system
 # for the nginx ingress controller (if we want to test both)
 - namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: ingress-nginx
 ports:
 - protocol: TCP
 port: 8050

Ingress traffic will explicitly come through the Gateway, so we need to only allow this traffic (envoy-gateway-system namespace).

salary-egress: what salary-backend can call

# salary-backend is reachable only from the frontend, and can only call
# user-backend, the fx-rates container, and cluster DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
 name: salary-egress
 namespace: client-a
spec:
 podSelector:
 matchLabels:
 app: part-2-salary
 policyTypes:
 - Ingress
 - Egress
 ingress:
 # allow ingress from frontend (same namespace)
 - from:
 - podSelector:
 matchLabels:
 app: part-2-frontend
 # specify the ports
 ports:
 - protocol: TCP
 port: 8001

 egress:
 # DNS so user-backend.platform.svc.cluster.local resolves from salary backend
 - to:
 - namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: kube-system
 podSelector:
 matchLabels:
 k8s-app: kube-dns
 ports:
 - protocol: UDP
 port: 53
 - protocol: TCP
 port: 53

 # fx-rates sits outside the cluster
 # the fx-decoy IP (172.18.0.101) isn't listed so it's blocked
 - to:
 - ipBlock:
 cidr: 172.18.0.100/32
 ports:
 - protocol: TCP
 port: 80

 # user-backend in the platform namespace
 - to:
 - namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: platform
 podSelector:
 matchLabels:
 app: part-2-users
 ports:
 - protocol: TCP
 port: 8000

We have to allow egress from the salary service to the kube-system namespace so that DNS resolution of the user API works correctly (remember cross-namespace service calls). We also specifically allow egress from the salary service to the 172.18.0.100/32 CIDR range, which just maps to the IP address 172.18.0.100, where the fx-rates services is running in the Docker network but outside of the Kind cluster.

platform-ingress: user-backend only accepts calls from client-a’s salary-backend

# user-backend is reachable only from client-a's salary-backend
# The frontend's direct call to user-backend is blocked here on purpose, so the dashboard
# shows one blocked edge next to the allowed ones.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
 name: platform-ingress
 namespace: platform
spec:
 podSelector:
 matchLabels:
 app: part-2-users
 policyTypes:
 - Ingress
 - Egress
 ingress:
 - from:
 - namespaceSelector:
 matchLabels:
 kubernetes.io/metadata.name: client-a
 podSelector:
 matchLabels:
 app: part-2-salary
 ports:
 - protocol: TCP
 port: 8000
 # no egress rules, user-backend doesn't make outbound calls

Here, the namespaceSelector and the podSelector are one “block” e.g. form an “AND” rule, meaning that we only allow ingress from things both in the client-a namespace AND the part-2-salary app.

Watching the policies work

Belong are two demonstrations of querying a user. I’ve set up two user request patterns:

frontend requests user API directly: this is explicitly not allowed via the NetworkPolicies
frontend requests salary API, which requests user API: this is allowed via the NetworkPolicies

Requesting a user that exists.

Requesting a user that does not exist.

We can also see that a pod in the platform namespace is not visible to the pod in the client-a namespace – this is the consequence of the RBAC (see below).

Part Two: Identity and the API server

The RBAC objects and how they relate

I explored RBAC in more depth here, incorporating Roles, ClusterRoles, RoleBindings, and ClusterRoleBindings. Roles + RoleBindings are namespace restricted – they only apply to objects within the namespace to which they belong. ClusterRoles + ClusterRoleBindings can be scoped to the entire cluster or they can be scoped to a specific namespace. The domain of role permissions for ClusterRoles is more expansive than those of Roles. ClusterRoles allow access to cluster-scopes resources e.g. Nodes, PersistentVolumes, while Roles restrict permissions to resources within a namespace e.g. ConfigMaps, Pods, Services.

Role diagram. ClusterRoles and ClusterRoleBindings span the entire cluster, while Roles and RoleBindings are namespace-restriced.

Authenticating people with Dex

Generally, we want to limit what a user can do based on what groups they belong to. In my current corporate environment, we have an external IdP and use AWS Cognito for authentication. K8s doesn’t have a “User” resource, so we need to authenticate and authorize outside the cluster. I’m using a tool called Dex, which acts as a mocked IdP + OIDC server. I created three different users, and associated each with a unique “group”. When creating the Kind cluster, we can provide some additional parameters pointing at the Dex service, such as a self-signed Certificate Authority (CA) file.

In the cluster manifest file, we’d point at the OIDC issuer URL (here a local Dex service running in the cluster, in its own namespace), indicate how a user will login (oidc-username-claim), indicate how we assign users to groups (oidc-groups-claim), and point at the certs file (oidc-ca-file).

# cluster.yaml
...
 kind: ClusterConfiguration
 apiServer:
 extraArgs:
 # Must be https and must match `issuer` in dex-config.yaml exactly.
 oidc-issuer-url: "https://kind-control-plane:32000"
 oidc-client-id: "kubernetes"
 oidc-username-claim: "email"
 oidc-groups-claim: "groups"
 oidc-ca-file: "/etc/kubernetes/dex/ca.crt"
 extraVolumes:
 - name: dex-ca
 hostPath: /etc/kubernetes/dex/ca.crt
 mountPath: /etc/kubernetes/dex/ca.crt
 readOnly: true
 pathType: File
...

# dex-config.yaml
# has to match the oidc-issuer-url in cluster.yaml exactly.
issuer: https://kind-control-plane:32000
storage:
 type: memory
web:
 https: 0.0.0.0:5556
 tlsCert: /etc/dex/tls/tls.crt
 tlsKey: /etc/dex/tls/tls.key
staticClients:
- id: kubernetes
 redirectURIs:
 # kubelogin's default local callback ports
 - http://localhost:8000
 - http://localhost:18000
 name: 'Kubernetes CLI'
 secret: ZXhhbXBsZS1hcHAtc2VjcmV0
enablePasswordDB: true

staticPasswords:
- email: "admin@example.com"
 # Clean pre-compiled bcrypt hash for 'password'
 hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"
 username: "platform-admin"
 userID: "admin-id-001"
 groups:
 - "platform-admins"

- email: "client-a@example.com"
 hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"
 username: "client-a-user"
 userID: "client-a-id-002"
 groups:
 - "clients-a"

- email: "observer@example.com"
 hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"
 username: "global-observer"
 userID: "observer-id-003"
 groups:
 - "observers"

We created three users: platform-admin, client-a-user, and global-observer, each with their own email, password, and groups. The groups determine

Authorizing people: three groups, three bindings

For each Dex group (again, a stand-in for groups provided by your own IdP), you can assign ClusterRoles or Roles. We have three group + role combinations:

# cluster-roles.yaml

# platform admin has full global admin cluster-wide
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: oidc-platform-admin-binding
subjects:
- kind: Group
 name: "platform-admins" # for Dex
 apiGroup: rbac.authorization.k8s.io
roleRef:
 kind: ClusterRole
 name: cluster-admin # K8s superuser role
 apiGroup: rbac.authorization.k8s.io

---
# client-a isolated to it's namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: oidc-client-a-binding
 namespace: client-a # namespace-restricted
subjects:
- kind: Group
 name: "clients-a" # for Dex
 apiGroup: rbac.authorization.k8s.io
roleRef:
 kind: ClusterRole
 name: edit # K8s role allowing read/write on workloads, but no RBAC manipulation
 apiGroup: rbac.authorization.k8s.io

---
# observer has cluster-wide read-only access
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: oidc-observer-binding
subjects:
- kind: Group
 name: "observers" # for Dex
 apiGroup: rbac.authorization.k8s.io
roleRef:
 kind: ClusterRole
 name: view # K8s role allowing global read-only access
 apiGroup: rbac.authorization.k8s.io

The cluster-admin and observer roles can access resources across the whole cluster via a ClusterRoleBinding, while we’ve restricted the client-a role to the client-a namespace via a RoleBinding.

Authorizing workloads: the frontend’s ServiceAccount

I wanted to demonstrate RBAC in action, and built this in to the dashboard itself. The frontend’s pod panel (pods.py) lists pods with the frontend’s own ServiceAccount token. The pod-reader Role and RoleBinding allow that in client-a namespace only, so the panel shows client-a pods and a 403 platform pods.

# roles.yaml

#### service accounts for each namespace
apiVersion: v1
kind: ServiceAccount
metadata:
 name: user-backend-sa
 namespace: platform
# the backends never call the API server, so don't mount a token
automountServiceAccountToken: false

---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: salary-backend-sa
 namespace: client-a
automountServiceAccountToken: false

---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: part-2-frontend-sa
 namespace: client-a

---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: observer-sa
 namespace: observer

---
#### frontend callback list pods in its own namespace (src/frontend/pods.py).
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 name: pod-reader
 namespace: client-a
rules:
- apiGroups: [""] # "" indicates the core API group
 resources: ["pods"]
 verbs: ["get", "watch", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: part-2-frontend-pod-reader
 namespace: client-a
subjects:
- kind: ServiceAccount
 name: part-2-frontend-sa
 namespace: client-a
roleRef:
 kind: Role
 name: pod-reader
 apiGroup: rbac.authorization.k8s.io

Testing it

To see if the RBAC are working as we expect them to, we can run some tests impersonating a user in a specific group:

# people (OIDC groups)
kubectl auth can-i create deployments -n client-a --as=client-a@example.com --as-group=clients-a
kubectl auth can-i create deployments -n platform --as=client-a@example.com --as-group=clients-a
kubectl auth can-i list secrets -A --as=observer@example.com --as-group=observers

# workloads (ServiceAccounts)
kubectl auth can-i list pods -n client-a --as=system:serviceaccount:client-a:part-2-frontend-sa
kubectl auth can-i list pods -n platform --as=system:serviceaccount:client-a:part-2-frontend-sa

Wrapping up

Sources of Confusion

Ingress vs. Gateway: I got hung up at first on how the GatewayClass + GatewayAPI + HTTPRoute approach differed from the Ingress + IngressController approach, and why exactly the Gateway approach was superior. However, once I understood that the Gateway approach allowed decoupling of the infrastructure from the routing rules, and how this enabled isolation across teams / clients / tenants, it made much more sense.
Dex and OIDC: while not stritly specific to K8s, I had to read a number of blog posts about Dex to understand how exactly to incorporate the OIDC mocking into this architecture. It took me a while to wrap my head around the different communication flows. The tls.crt file is provided by the Dex pod once to the local host (via kubelogin) and again to the K8s API server. The ca.crt file needs to exist on both the local host, and in the cluster (with --oidc-ca-file flag). The private keys aren’t shared anywhere. There are two handshakes: one for user authentication with Dex, and another to identify Dex to the API server, so the API server can verify the user JWT.

What’s next

So far we’ve built stateless services. These accept requests and return responses without long-lived state. Model training is different, in that we’ll want to log training performance, and generate various artefacts along the way.

In next post, I’ll cover K8s Jobs, how to stage training and inference data, and how to persist artifacts. I’ll also cover GPU allocation syntax, as well as when/where a single Job stops being enough. That will lead nicely into ramping up to higher-level orchestration concepts like PyTorchJob and KubeRay.

Kubernetes for ML Engineering, Part 1: Why, and Two Services Talking

Sun, 13 Sep 2026 09:00:00 -0700

Why Kubernetes

I come from an ML-heavy academic research background, where I applied machine learning methods to medical imaging and brain network analysis. My work focused heavily on 3- and 4-D image and mesh-based segmentation and computer vision, as well as on the development and application of graph neural network architectures to better understand brain architecture. In my current role at Just-Evotec Biologics, I’ve built and deployed protein language models for antibody optimization and antibody property prediction. In all of these cases, I’ve had access to on-prem GPU compute and used schedulers like Slurm and Sun Grid Engine for training and inference.

My more recent work has moved toward the infrastructure side of ML: data engineering, cloud architecture, and data platform design.

I wanted to learn Kubernetes, specifically for machine learning training, inference, and monitoring. ECS has worked well for us, but a lot of the open-source tooling for ML platforms is built for Kubernetes. I also wanted to see how Kubernetes compares to the different HPC schedulers I’ve used for training. This series covers my exploration of Kubernetes (K8s) for machine learning and AI engineering. It starts with foundational topics like Pods and Services, moves to intermediate topics like role-based access control (RBAC) and networking, and ends with topics specific to ML platforms like horizontal pod autoscaling (HPA), multi-tenant namespaces, and job queuing.

I want to stress that this won’t be a tutorial. There are plenty of those online already. This is simply going to be a documentation of my path to learning K8s, with a specific application towards ML and AI.

What’s coming

I’ll post my progress as a series of five blog posts:

Part 1: Why, and Two Services Talking
Part 2: Networking and RBAC
Part 3: Model Training as Jobs
Part 4: Slurm vs. Kubernetes
Part 5: Capstone ML Platform

Part 1: Two services need to find each other

One of the most basic capabilities I’ve had to enable for our platform is letting two services talk to each other. In a monorepo, we’d define separate modules and import one from another. But we often want to reuse one service across many applications, and in that case it makes sense to extract the reusable code into its own deployed application. That leaves the question: how do multiple deployed services communicate with each other?

You can find the full repo for my code here.

The naive approach

At the most basic level, we can deploy two services to the same or different EC2 instances (if using AWS) and allow traffic between the IP addresses of those instances. This approach is easy. We set up each instance, define the necessary subnets and security groups, and voila, the instances can talk to each other.

But it’s not a robust way to deploy things. We have to hardcode IP addresses and, often, environment variables, and things break as soon as an instance is replaced, whether during a redeploy or when scaling up.

In K8s, the equivalent easy approach would also hardcode values, like node IPs or Pod IPs stored in environment variables. The first time a Pod is replaced, the applications would stop being able to communicate. Kubernetes doesn’t move a Pod to a new node. It deletes the Pod and creates a new one, which usually gets a new IP address. Hardcoding is easy, but it is not robust.

The Kubernetes way

Instead, we’ll implement a more robust system for inter-service communication. I’ve set up two applications here: a very simple Python Dash frontend and a FastAPI backend. The backend loads a small CSV file containing the names and ages of individuals, and has an endpoint that returns the age of a requested person. The frontend accepts a name and displays that person’s age, if they are in the table. External users don’t need to know about the backend; they just need to interact with the frontend.

We’ll create two Deployments, a ConfigMap, and two Services. The backend and frontend each run in their own Pods, and the frontend reaches the backend through the backend Service’s name, part-1-backend, rather than a hardcoded IP address.

Deployment

We’ll create two Deployments. One runs the backend container, with the label selector app: part-1-backend. The other runs the frontend, with the label selector app: part-1-frontend. Since the images are built locally, each one has to be loaded into kind with kind load docker-image ${image_name}:${tag} --name ${cluster_name}, and each container sets imagePullPolicy: Never so Kubernetes doesn’t try to pull the image from a registry.

For each container, we specify a liveness probe at /healthz and a readiness probe at /ready, along with some environment variables pulled from the ConfigMap (see below). The backend uses DATA_FILE to choose which CSV file to load, and SLEEP_DURATION to simulate a slow startup. The first call to /ready blocks for SLEEP_DURATION seconds, so the backend isn’t marked ready until that call finishes.

# deployment.yaml

# deployment for the backend
apiVersion: apps/v1
kind: Deployment
metadata:
 labels:
 app: part-1-backend
 name: part-1-backend
spec:
 replicas: 1
 selector:
 matchLabels:
 app: part-1-backend
 strategy: {}
 template:
 metadata:
 labels:
 app: part-1-backend
 spec:
 containers:
 - image: part-1-backend:latest
 ports:
 - containerPort: 8000
 name: backend
 resources: {}
 imagePullPolicy: Never
 livenessProbe:
 httpGet:
 path: /healthz
 port: 8000
 initialDelaySeconds: 5 # Time to wait before first check
 periodSeconds: 10 # How often to check
 timeoutSeconds: 2 # Timeout for response
 failureThreshold: 3 # Number of failures before restarting
 readinessProbe:
 httpGet:
 path: /ready
 port: 8000
 initialDelaySeconds: 5 # Time to wait before first check
 periodSeconds: 5 # How often to check
 timeoutSeconds: 2 # Timeout for response
 failureThreshold: 3 # Number of failures before marking the container not ready
 env:
 - name: DATA_FILE
 valueFrom:
 configMapKeyRef:
 name: part-1
 key: data_file
 - name: SLEEP_DURATION
 valueFrom:
 configMapKeyRef:
 name: part-1
 key: sleep_duration
status: {}

---
# deployment for the frontend
apiVersion: apps/v1
kind: Deployment
metadata:
 labels:
 app: part-1-frontend
 name: part-1-frontend
spec:
 replicas: 1
 selector:
 matchLabels:
 app: part-1-frontend
 strategy: {}
 template:
 metadata:
 labels:
 app: part-1-frontend
 spec:
 containers:
 - image: part-1-frontend:latest
 ports:
 - containerPort: 8050
 name: frontend
 resources: {}
 imagePullPolicy: Never
 livenessProbe:
 httpGet:
 path: /healthz
 port: 8050
 initialDelaySeconds: 5 # Time to wait before first check
 periodSeconds: 10 # How often to check
 timeoutSeconds: 2 # Timeout for response
 failureThreshold: 3 # Number of failures before restarting
 readinessProbe:
 httpGet:
 path: /ready
 port: 8050
 initialDelaySeconds: 5 # Time to wait before first check
 periodSeconds: 5 # How often to check
 timeoutSeconds: 2 # Timeout for response
 failureThreshold: 3 # Number of failures before marking the container not ready
 env:
 - name: BACKEND_URL
 valueFrom:
 configMapKeyRef:
 name: part-1
 key: backend_url
status: {}

Services

We create two Services, one for each application. By default (if no type is specified), a Service is of type ClusterIP, which means it can be reached from inside the cluster but not from outside it. We only want external users to be able to access the frontend application. For all intents and purposes, they don’t even need to know that the backend exists. To do this, we make the frontend a NodePort Service, which also opens a port on every node in the cluster. In the manifest, port is the port the Service listens on inside the cluster, and targetPort is the container port that traffic is forwarded to.

# services.yaml
apiVersion: v1
kind: Service
metadata:
 name: part-1-backend
spec:
 selector:
 app: part-1-backend
 ports:
 - port: 8000
 targetPort: 8000

---
apiVersion: v1
kind: Service
metadata:
 name: part-1-frontend
spec:
 type: NodePort
 selector:
 app: part-1-frontend
 ports:
 - port: 8050
 targetPort: 8050

ConfigMap

As I mentioned above, a naive approach would be to hardcode environment variables into the image. Instead, we can define them in a ConfigMap and redeploy the Pods whenever we want to change them. Environment variables from a ConfigMap are set when the container starts, so after editing the ConfigMap we restart the Deployment that uses the changed value (for example, kubectl rollout restart deployment part-1-backend after changing data_file). This ConfigMap defines backend_url (used by the frontend application), data_file, which points to the desired CSV file of names and ages, and sleep_duration, which sets how long a backend container takes to become ready.

# config-map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
 name: part-1
data:
 # property-like keys; each key maps to a simple value
 backend_url: "http://part-1-backend:8000"
 data_file: "special_names.csv"
 sleep_duration: "10"

We apply the resources in this order:

kubectl apply -f ./manifests/config-map.yaml
kubectl apply -f ./manifests/deployment.yaml
kubectl apply -f ./manifests/services.yaml

Are we ready? Not yet. On macOS, kind runs its nodes as Docker containers inside Docker Desktop’s VM, so the NodePort isn’t reachable from the host right off the bat. So instead we port-forward to the frontend Service:

kubectl port-forward service/part-1-frontend 8051:8050

Port-forwarding works with any K8s Service type and connects to a single Pod behind the Service, so it skips the load balancing (if implemented). That’s fine for checking the app locally, but it isn’t how external traffic would reach the frontend in a real cluster. I’ll get into load balancing in the next post.

We’ve set the base path of the Dash application to /frontend/, so once the port-forward is running we can access the frontend at http://127.0.0.1:8051/frontend/.

Bridging the gap between ECS and K8s

Coming from ECS, my instinct was to try to map each Kubernetes object onto something I already knew well. Most Kubernetes objects / resources have a rough ECS counterpart, but the boundaries between them are drawn in different places, and there wasn’t always a direct analog for me to connect the dots with. What helped me bridge the gap the most was just starting to play around with KinD and K8s locally, rather than viewing ECS as a template (I think that may have been more confusing actually).

ECS	Kubernetes	Notes
Task Definition	Pod template (`spec.template` in a Deployment)	ECS Task Definitions are numbered revisions. K8s tracks revisions through ReplicaSets, so we can roll back to a previous revision with `kubectl rollout undo`.
Task	Pod	In `awsvpc` mode, all the containers in a task also share a network namespace, so the `localhost` path from my first attempt (see Gotchas below) also works in ECS.
Service	Deployment + Service	An ECS Service keeps N tasks running and registers them with a load balancer. K8s splits this concept into two resources.
Service Connect or Cloud Map	Cluster DNS	Service discovery in ECS must be specified, whereas every K8s Service gets a DNS name by default.
Application Load Balancer	Ingress	A `type: LoadBalancer` Service is closer to a Network Load Balancer.
Task role	ServiceAccount	The ECS Execution Role pulls images and fetches secrets, and maps to a K8s node’s IAM role and image pull secrets.
Environment variables and secrets in the Task Definition	ConfigMap and Secret	Changing a ConfigMap doesn’t restart anything; you have to actually run the `rollout`, just like with changing environment variables in a Task Definition.
Container and load balancer health checks	Liveness and readiness probes	Both are defined on the Task / Pod OR the Target Group (EC2).
Task CPU and memory	Container requests and limits	Required on Fargate, optional in K8s.

Each platform’s definition of “Service” was a bit confusing at first. An ECS Service keeps a desired number of tasks running, handles deploys, and attaches tasks to target groups. A K8s Service is just a name and IP in front of the Pods that match its label selector while the actual controlling of the Pods is the Deployment’s job. Kubernetes also explicitly refers to objects as “Controllers” – from that perspective it makes a lot more sense e.g. “Your job is to maintain a certain state, or to watch for certain things…”.

Speedbumps

I ran into one gotcha along the way:

Single Deployment vs. separate Deployments for the two Services. My initial approach was to put both Services in a single Deployment, where both the frontend and backend containers were in each Pod. Since containers in a Pod share a network namespace, the frontend just referred to the backend using localhost:8000. But this nixed the independence of each Service, and meant the two Services could not scale independently. Splitting them into separate Deployments, each with its own label, meant the backend Service only selected backend Pods. The frontend now reaches the backend through that Service’s name, and Kubernetes load-balances requests across any ready backend Pod, not just the backend container in the same Pod.

What this looks like at scale

This setup works on a laptop, but there are a few things I’d do differently in a production cluster.

Data wouldn’t live in the container image. Right now the backend’s CSV files are copied into the image, so changing the data means rebuilding it. In production, the data would live in S3 or in a database like RDS. For S3 access, we’d give the backend’s ServiceAccount an IAM role rather than storing long-lived AWS keys in a Kubernetes Secret.
Service-to-service traffic would be explicitly allowed, not open by default. a.k.a. principle of least privilege a.k.a. need-to-know. I didn’t implement any authentication or network policies. By default, every Pod in a K8s cluster can reach every other Pod. This should be an explicit decision, since it’s very likely not every service should be able to communicate with every other. The first step is a NetworkPolicy that denies all traffic by default and only allows the frontend to reach the backend. Also, a service mesh with mutual TLS could be used for authenticating and encrypting each request between services. Similarly, we don’t have any authentication at the cluster boundary, which would sit in front of the frontend at the Ingress. I’ll dig into these topics in Part 2.
Probes would account for slow startup. The backend’s SLEEP_DURATION mocks actual startup work. For an ML service, that work might be loading a model, which, depending on the model size, could take quite long. If the liveness probe starts checking before the model finishes loading, Kubernetes restarts the container and the load starts over. A startupProbe holds off the liveness and readiness probes until the container has finished starting.