Azure Kubernetes Service (AKS) Guide: Architecture, Deployment, and Best Practices

Containers have transformed how applications are built and deployed. They provide consistency across environments, efficient resource utilization, and simplified dependency management. But as your application grows from a few containers to dozens or hundreds, a new set of challenges emerges:

• How do you schedule containers across multiple machines?
• How do you handle container failures automatically?
• How do you scale containers up and down based on demand?
• How do you manage service discovery and networking?
• How do you perform rolling updates without downtime?
• How do you maintain high availability across failure domains?

This is where Kubernetes comes in. Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides the control plane for scheduling workloads, self-healing, service discovery, load balancing, and declarative configuration management.

But running Kubernetes yourself is operationally complex. You need to manage the control plane components (API server, etcd, scheduler, controller manager), handle upgrades, monitor cluster health, and secure the infrastructure. This is a significant operational burden that many organizations would rather avoid.

Azure Kubernetes Service (AKS) addresses this challenge. AKS is a managed Kubernetes service that reduces the complexity and operational overhead of managing Kubernetes by offloading much of that responsibility to Azure. The Azure platform manages the AKS control plane, which is responsible for the Kubernetes objects and worker nodes that you deploy to run your applications.

AKS is managed Kubernetes, not "Kubernetes without operations." Organizations still need to design and operate workloads, networking, security, node pools, identities, storage, observability, upgrades, and application reliability. AKS reduces control-plane management but does not eliminate cluster and workload operations.

AKS is an ideal platform for deploying and managing containerized applications that require high availability, scalability, and portability, and for deploying applications to multiple regions, using open-source tools, and integrating with existing DevOps tools.

What Is Azure Kubernetes Service?

Azure Kubernetes Service is a managed Kubernetes service that you can use to deploy and manage containerized applications. It provides a fully managed Kubernetes control plane with customer-managed worker nodes.

The architecture

AKS separates the Microsoft-managed control plane from the customer-managed nodes (the node plane). Azure operates and abstracts Kubernetes control plane components, such as:

• API server (kube-apiserver) – The frontend for the Kubernetes control plane
• etcd – The distributed key-value store for cluster state
• Scheduler (kube-scheduler) – Assigns pods to nodes
• Controller manager (kube-controller-manager) – Runs controller processes
• Cloud controller manager – Integrates with Azure infrastructure

Your nodes are organized into node pools in your subscription. Each node runs the kubelet and a container runtime, which is containerd on supported AKS node pools.

Basic architecture diagram

Developer
   |
   v
Kubernetes API
   |
   v
AKS Control Plane (Microsoft-managed)
   |
   +-------------------+
   |                   |
   v                   v
Node Pool A         Node Pool B (Customer-managed)
   |                   |
   v                   v
Application Pods    Application Pods

What Azure manages versus what you manage

Responsibility	Microsoft Azure	Customer
Kubernetes control plane	✅ Fully managed	—
Control plane upgrades	✅ Automated	Schedule maintenance
Worker nodes (VMs)	Provisioning	Configuration, scaling, upgrades
Node OS patching	Automated	—
Kubernetes workloads	—	✅ Deployment, configuration
Application configuration	—	✅ ConfigMaps, Secrets
Networking design	Underlying infrastructure	✅ VNet, subnet, network policies
Identity and RBAC	Entra ID integration	✅ Permissions, service accounts
Monitoring	Azure Monitor integration	✅ Alerting, actionable insights
Application reliability	—	✅ Probes, pod disruption budgets
Storage	Azure storage services	✅ PVCs, storage classes

When to use AKS

Common use cases for AKS include:

• Lift and shift to containers – Migrate existing applications to containers and run them in a fully managed Kubernetes environment
• Microservices – Simplify the deployment and management of microservices-based applications with streamlined horizontal scaling, self-healing, load balancing, and secret management
• Secure DevOps – Implement secure DevOps with Kubernetes
• Bursting with ACI – Use virtual nodes to provision pods on Azure Container Instances for burst capacity

AKS at a Glance

Concept	AKS / Kubernetes Component	Purpose
Cluster	AKS cluster	Kubernetes execution environment with managed control plane
Control plane	Managed Kubernetes control plane	Cluster orchestration, scheduling, and state management
Worker compute	Node pools	Virtual machines that run your workloads
Pod	Kubernetes pod	Smallest deployable workload unit; one or more containers
Deployment	Kubernetes Deployment	Desired application state; manages replica sets
Service	Kubernetes Service	Stable network endpoint and load balancing for pods
Ingress	Ingress / Gateway	External HTTP/HTTPS routing to services
Namespace	Kubernetes namespace	Logical isolation for workloads within a cluster
Container registry	Azure Container Registry	Store and manage private container images
Identity	Microsoft Entra ID / Workload Identity	Authentication and authorization for cluster and workloads
Observability	Azure Monitor / Container Insights	Metrics, logs, and monitoring for cluster and applications

AKS Cluster Architecture

Control plane components

The AKS control plane is fully managed by Azure. It includes:

API server (kube-apiserver) – The central management component that exposes the Kubernetes API. All administrative and operational interactions with the cluster go through the API server.

etcd – A distributed key-value store that holds the cluster's configuration and state. Azure manages etcd for you, including backups and high availability.

Scheduler (kube-scheduler) – Watches for newly created pods that have no assigned node and selects a node for them to run on, based on resource requirements, policy constraints, and other factors.

Controller manager – Runs controller processes that regulate the state of the cluster. Controllers watch the state of the cluster through the API server and make changes to move the current state toward the desired state.

Node plane (worker nodes)

Your nodes are organized into node pools in your subscription. AKS supports node pools backed by Virtual Machine Scale Sets and the newer Virtual Machines node pool type.

Each node runs:

• kubelet – The primary node agent that communicates with the control plane
• Container runtime – containerd on supported AKS node pools
• kube-proxy – Manages network rules on nodes (not present when using Cilium eBPF dataplane)

Cluster hierarchy

Cluster
  └── Node Pool
        └── Node (VM)
              └── Pod
                    └── Container

Networking

Networking is provided by a configurable Azure Container Networking Interface (CNI) plugin. Azure CNI Overlay is the recommended option for most clusters; other Azure CNI variants are also supported.

Node Pools

Nodes of the same configuration are grouped together into node pools. Node pools contain the underlying virtual machines that run your applications.

System node pools vs. user node pools

AKS supports two distinct node pool modes:

System node pools serve the primary purpose of hosting critical system pods such as CoreDNS and metrics-server. Key characteristics:

• Must use Linux nodes (Ubuntu or Azure Linux)
• Require a VM SKU with at least 4 vCPUs and 4 GB of memory
• B-series VMs aren't supported
• Should be tainted with CriticalAddonsOnly=true:NoSchedule to keep application pods off system pools
• For production clusters, a single system node pool should contain at least two nodes; three nodes are recommended for improved fault tolerance

User node pools serve the primary purpose of hosting your application pods. Key characteristics:

• Can use Ubuntu Linux, Azure Linux, or Windows
• Can contain zero or more nodes
• Choose current-generation v5 or later VM families that match CPU, memory, storage, and GPU requirements

Why separate node pools?

Use separate system and user node pools so cluster services and workloads can scale and upgrade independently. This separation prevents an application from causing instability with your cluster's system node pool.

Example node pool architecture

AKS Cluster
├── System Node Pool (Linux, 3 nodes, D4ds_v5)
│   └── CoreDNS, metrics-server, konnectivity-agent
│
├── Application Node Pool (Linux, 5-20 nodes, D8ds_v5)
│   └── Web and API workloads
│
├── Compute Node Pool (Linux, 2-10 nodes, F-series)
│   └── CPU-intensive batch workloads
│
└── GPU Node Pool (Linux, 0-4 nodes, NC-series)
    └── AI inference and ML workloads

Scaling node pools

Node pools can scale manually or automatically:

• Manual scaling – Adjust the node count through the Azure portal, CLI, or API
• Cluster Autoscaler – Automatically adjusts the number of nodes in a node pool based on pending pods

Smart VM Defaults

If you leave the VM SKU parameter blank when creating a node pool, AKS dynamically selects an appropriate SKU based on available regional capacity and subscription quota. This capability, called Smart VM Defaults, became generally available in May 2025.

Deploying Applications to AKS

The developer workflow

Source Code
   |
   v
Build Container Image (Dockerfile)
   |
   v
Azure Container Registry
   |
   v
AKS (kubectl apply)
   |
   v
Deployment
   |
   v
Pods

Key Kubernetes resources

Deployment – Defines the desired state for your application: which container image to run, how many replicas, update strategy, and resource requirements.

Service – Provides a stable network endpoint for accessing pods. Service types include:

• ClusterIP – Internal cluster IP (default)
• LoadBalancer – External load balancer from Azure
• NodePort – Exposes the service on each node's IP at a static port

ConfigMap – Stores configuration data as key-value pairs. Used to decouple configuration from container images.

Secret – Stores sensitive information like passwords, tokens, and TLS certificates. Kubernetes Secrets are base64-encoded, not encrypted by default.

Ingress – Manages external access to services, typically HTTP/HTTPS. Provides host-based and path-based routing, TLS termination, and load balancing.

Conceptual deployment YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: myregistry.azurecr.io/my-app:latest
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

Azure Container Registry Integration

AKS commonly integrates with Azure Container Registry (ACR) for private container image storage and management.

Integration patterns

Private container images – Store container images in ACR and pull them into AKS. This ensures your images are secure and accessible only to your cluster.

Image lifecycle management – ACR provides image retention policies, tag management, and automated image cleanup.

Authentication

ACR integration uses Microsoft Entra ID and managed identities for authentication:

CI/CD
  |
  v
Azure Container Registry
  |
  | Managed Identity / Workload Identity
  v
AKS (pulls images)

Best practice: Use managed identities for ACR authentication rather than storing registry credentials in Kubernetes Secrets. Use the acrpull role to grant AKS cluster identity permission to pull images from ACR.

Image security

ACR provides:

• Vulnerability scanning – Detect vulnerabilities in container images
• Content trust – Ensure image integrity
• Quarantine – Prevent vulnerable images from being pulled

Networking

Networking is one of the most important architecture decisions in AKS.

Kubernetes networking model

The Kubernetes networking model requires:

• Pods can communicate with all other pods without NAT
• Nodes can communicate with all pods without NAT
• IP addresses assigned to pods are visible to other components

AKS networking options

Azure CNI Overlay (recommended) – The recommended option for most clusters. Pods get IP addresses from a private CIDR range that's separate from the VNet, preserving VNet IP space. Supports network policies.

Azure CNI (node subnet) – Pods get IP addresses directly from the VNet subnet. Requires careful IP address planning. Supports network policies.

kubenet (legacy, retiring in 2028) – Basic networking with IP address translation (NAT). Pods get IPs from a private CIDR, and outbound traffic goes through the node's IP. Do not use for new clusters—kubenet will be retired on March 31, 2028.

Network policy

Network policies control traffic between pods. Options include:

• Azure Network Policies – Azure-native implementation
• Calico – Open-source network policy engine
• Cilium – eBPF-based networking with advanced policy capabilities

Ingress and application traffic

External traffic reaches applications through:

Internet
   |
   v
Azure Load Balancer / Application Gateway
   |
   v
Ingress Controller (NGINX, Gateway API)
   |
   v
Kubernetes Service
   |
   v
Pods

Load Balancer vs. Ingress vs. API Management:

	Load Balancer	Ingress	API Management
Purpose	L4 traffic distribution	L7 HTTP routing	API gateway, security, governance
Protocol	TCP/UDP	HTTP/HTTPS	HTTP/HTTPS
Features	Port-based routing	Host/path routing, TLS	Authentication, rate limiting, developer portal
Best for	Simple traffic distribution	HTTP routing, TLS termination	API publishing, security, governance

Private clusters

For production workloads requiring network isolation, deploy AKS as a private cluster:

• The API server has a private IP address only
• No public endpoint is exposed
• Access requires VNet connectivity (VPN, ExpressRoute, or VNet peering)

Hub and spoke network topology

The baseline AKS architecture uses a hub and spoke network topology. Deploy the hub and spokes in separate virtual networks connected through virtual network peering.

Advantages:

• Enable segregated management with least privilege
• Minimize direct exposure of Azure resources to the public internet
• Support workloads that span multiple subscriptions
• Add new spokes without redesigning the topology

Identity and Security

Identity architecture

AKS identity spans three distinct areas:

1. Control-plane authentication – Who can access the Kubernetes API (Azure RBAC + Kubernetes RBAC)
2. Cluster-to-Azure authentication – How the AKS cluster authenticates to Azure services (managed identities)
3. Pod-to-Azure authentication – How applications running in pods authenticate to Azure resources (Workload Identity)

Microsoft Entra ID and RBAC

Azure RBAC controls access to the AKS resource itself and Azure resources it integrates with. Roles like Azure Kubernetes Service Cluster Admin grant Azure-level permissions.

Kubernetes RBAC controls access to Kubernetes resources within the cluster (pods, services, deployments, etc.). Use Kubernetes Role and ClusterRole resources with RoleBinding and ClusterRoleBinding.

Managed identities

AKS clusters require a Microsoft Entra identity to access Azure resources, like load balancers and managed disks. Use system-assigned or user-assigned managed identities for:

• Load balancer creation
• Managed disk provisioning
• ACR image pulls
• Key Vault access

Microsoft Entra Workload Identity

Workload Identity is the recommended approach for pod-to-Azure authentication. It enables Kubernetes applications to access Azure resources securely with Microsoft Entra ID, based on annotated service accounts.

How it works:

1. A Kubernetes service account is annotated with the Azure managed identity
2. The pod mounts the service account token
3. The token is exchanged for an Azure access token
4. The application uses the token to access Azure resources

Benefits over other approaches:

• No credentials stored in pods
• Supports federated identity
• Works with Azure Identity client libraries
• Uses OIDC federation for secure token exchange

Security best practices

Least privilege – Grant only the permissions required. Use minimal RBAC roles for both Azure and Kubernetes.

Pod security – Use pod security standards, security contexts, and network policies.

Image security – Scan images for vulnerabilities. Use trusted registries (ACR). Sign images with content trust.

Node security – Keep nodes patched. Use Azure Linux or Ubuntu with security updates. Minimize node access.

Audit logging – Enable audit logs for both Azure and Kubernetes. Monitor for suspicious activity.

Secrets and Azure Key Vault

Kubernetes Secrets

Kubernetes Secrets store sensitive information. However:

• Secrets are base64-encoded, not encrypted by default
• Secrets are stored in etcd (which is managed by Azure in AKS)
• Anyone with API server access can read Secrets

Azure Key Vault

For production workloads, use Azure Key Vault for sensitive configuration:

• Centralized secret management – One place for all secrets, keys, and certificates
• Access control – Granular permissions through Azure RBAC
• Auditing – Full audit trail of secret access
• Rotation – Automated or scheduled secret rotation

Key Vault integration with AKS

Use the Azure Key Vault provider for Secrets Store CSI driver to mount secrets from Key Vault directly into pods:

• Secrets are mounted as volumes
• Secrets can be automatically refreshed
• Pods access secrets without storing credentials

Secret management architecture

Pod
   |
   v
Workload Identity
   |
   v
Azure Key Vault
   |
   +--> Secrets
   +--> Certificates
   +--> Keys

Best practice: Never store long-lived credentials directly inside container images or source code. Use managed identities and Key Vault for all sensitive configuration.

Storage

Containers are ephemeral—when a pod is deleted, its data is lost. For stateful applications, you need persistent storage.

Persistent volumes and claims

Kubernetes uses PersistentVolumes (PV) and PersistentVolumeClaims (PVC) for storage:

• PersistentVolume – A storage resource provisioned by the administrator
• PersistentVolumeClaim – A request for storage by a user

Storage options in AKS

AKS provides several storage options, each with its own characteristics and use cases:

Storage Option	Access Mode	Use Cases
Azure Disk	ReadWriteOnce (single node)	Databases, applications requiring fast block storage
Azure Files	ReadWriteMany (multiple nodes)	Shared configuration, content management, shared development environments
Azure Blob Storage	ReadWriteMany	Large unstructured datasets, object storage
Azure NetApp Files	ReadWriteMany	High-performance file storage, enterprise workloads

Built-in storage classes

AKS creates several StorageClasses automatically:

• managed-premium – Premium SSD managed disks (default if node has premium storage)
• managed-csi – Standard SSD managed disks
• azurefile-csi – Azure Files Standard
• azurefile-premium-csi – Azure Files Premium

Volume binding modes

AKS supports two volume binding modes:

• Immediate – Volume is provisioned as soon as the PVC is created
• WaitForFirstConsumer – Volume is provisioned only when a pod using the PVC is created

Recommendation: Use WaitForFirstConsumer in multi-zone clusters to ensure volumes are created in the same zone as the pod, avoiding cross-zone data transfer penalties.

Ephemeral OS disks

AKS prefers Ephemeral OS disks when the selected VM SKU supports them. Benefits include:

• OS disk lives on VM cache or temporary storage
• No extra managed disk IOPS cost
• Faster node creation, reimage, and upgrade operations

Scaling

AKS supports multiple scaling dimensions.

Horizontal Pod Autoscaler (HPA)

HPA automatically scales the number of pods in a deployment based on observed CPU utilization, memory utilization, or custom metrics.

How it works:

1. HPA periodically checks metrics (CPU, memory, custom)
2. Calculates desired replica count
3. Updates the deployment's replica count

Custom metrics – HPA can scale based on:

• Queue depth (e.g., Service Bus, Kafka)
• Request rate
• Application-specific metrics

Cluster Autoscaler

Cluster Autoscaler automatically adjusts the number of nodes in a node pool based on pending pods.

How it works:

1. Pods are unscheduled (pending)
2. Cluster Autoscaler detects the resource shortage
3. Adds nodes to the node pool
4. Pods are scheduled on the new nodes

Scaling down – Removes nodes when they're underutilized.

Scaling dimensions

Traffic increases
      |
      v
HPA increases Pods
      |
      v
Not enough node capacity
      |
      v
Cluster Autoscaler adds Nodes

Important considerations

Right-sizing – Start with appropriate pod resource requests and limits. Over-requesting wastes resources; under-requesting leads to instability.

Downstream dependencies – Scaling pods doesn't solve bottlenecks in databases, APIs, or other downstream services.

Scaling policies – Configure appropriate scaling policies to avoid thrashing (rapid scaling up and down).

High Availability

Availability zones

Use availability zones for your AKS clusters as part of your resiliency strategy to increase availability when you deploy to a single region. Many Azure regions provide availability zones. The zones are close enough to have low-latency connections but far enough apart to reduce the likelihood that local outages will affect more than one zone.

Region
 |
 +-- Availability Zone 1
 |      +-- Node
 |      +-- Pods
 |
 +-- Availability Zone 2
 |      +-- Node
 |      +-- Pods
 |
 +-- Availability Zone 3
        +-- Node
        +-- Pods

Multi-region deployments

For critical workloads, deploy multiple clusters across different Azure regions. By geographically distributing AKS clusters, you can achieve higher resiliency and minimize the effects of regional failures.

Internet-facing workloads should use Azure Front Door or Azure Traffic Manager to route traffic across AKS clusters globally.

Pod-level availability

Pod replicas – Run multiple replicas of critical workloads.

Pod anti-affinity – Ensure pods aren't scheduled on the same node or in the same availability zone.

Topology spread constraints – Distribute pods evenly across failure domains.

Pod Disruption Budgets (PDB) – Define minimum availability during voluntary disruptions (node upgrades, cluster scaling).

Uptime SLA

AKS offers an Uptime SLA that provides financially backed SLA for cluster availability. Use it for production workloads requiring high availability.

Azure Backup for AKS

Use Azure Backup to protect AKS cluster and restore to alternate regions during disaster. AKS workloads can leverage Azure Backup for protecting persistent volumes using the Velero backup controller.

Reliability and Self-Healing

Kubernetes provides built-in self-healing capabilities through its reconciliation model.

Desired state reconciliation

Kubernetes controllers continuously watch the state of the cluster and take actions to move the current state toward the desired state. This includes:

• Restarting failed containers
• Rescheduling pods from failed nodes
• Maintaining the desired number of replicas

Probes

Liveness probes – Indicate whether the container is running. If the liveness probe fails, Kubernetes restarts the container.

Readiness probes – Indicate whether the container is ready to serve traffic. If the readiness probe fails, Kubernetes removes the pod from service endpoints.

Startup probes – Indicate whether the application has started. Used for applications with slow startup times.

Key distinction: A pod can be running (liveness probe passes) but not ready to receive traffic (readiness probe fails).

Pod lifecycle

Pending → Running → Succeeded/Failed
                ↓
            CrashLoopBackOff (restarting)

Reliability best practices

• Configure appropriate resource requests and limits
• Use readiness probes to control traffic flow
• Use liveness probes to restart unhealthy containers
• Use Pod Disruption Budgets to maintain availability during maintenance
• Test failure scenarios before production

Deployment Strategies

Rolling deployment (default)

Kubernetes Deployments use rolling updates by default:

• Gradually replaces old pods with new ones
• Zero downtime if properly configured
• Rollback is supported

Blue-green deployment

Run two versions simultaneously:

• Blue – Current version (active)
• Green – New version (staged)

Switch traffic from blue to green when ready. Quick rollback by switching back.

Canary deployment

Gradually roll out new versions:

1. Deploy new version with small traffic percentage (e.g., 5-10%)
2. Monitor for errors and performance
3. Gradually increase traffic percentage
4. Full rollout after validation

Progressive delivery – Use tools like Argo Rollouts or Flagger for automated canary analysis and rollback.

Deployment considerations

• Use maxSurge and maxUnavailable to control update behavior
• Test upgrades in staging before production
• Plan rollback procedures
• Monitor during and after deployment

CI/CD and DevOps

Typical pipeline

Git
 |
 v
CI Pipeline (GitHub Actions / Azure DevOps)
 |
 +--> Build container image
 +--> Run tests
 +--> Security scan (Trivy, Snyk)
 +--> Push to ACR
 |
 v
CD Pipeline
 |
 +--> Validate manifests
 +--> Deploy to AKS
 +--> Verify deployment
 |
 v
AKS

Infrastructure as Code

Use Infrastructure as Code for AKS cluster creation and configuration:

• Terraform – HashiCorp's declarative IaC tool
• Bicep – Azure-native declarative language
• ARM templates – Azure Resource Manager templates
• Helm – Package manager for Kubernetes applications

GitOps

GitOps uses Git as the single source of truth for infrastructure and application configuration:

• Flux – GitOps operator for Kubernetes
• Argo CD – Declarative GitOps continuous delivery

Benefits: Audit trail, rollback capability, consistency across environments.

Helm

Helm is the package manager for Kubernetes:

• Charts – Templated Kubernetes manifests
• Releases – Deployed instances of charts
• Repositories – Chart storage (e.g., ACR supports Helm charts)

Observability

Monitoring dimensions

Production observability must cover both cluster health and application health.

Cluster health:

• Node status, CPU, memory
• Control plane health
• Kubernetes events
• Pod scheduling failures

Application health:

• Application metrics (request rate, latency, errors)
• Container logs
• Pod status and restarts

Azure Monitor and Container Insights

Azure Monitor provides monitoring for AKS:

• Container Insights – Collects metrics, logs, and events from AKS clusters
• Metrics – Node, pod, and container metrics (CPU, memory, disk, network)
• Logs – Container logs, Kubernetes events, and node logs

Application Insights

For application-level monitoring:

• Distributed tracing
• Request rates and latency
• Error rates
• Dependency tracking

Key metrics to monitor

Metric	What It Tells You
Node CPU/Memory	Node resource utilization
Pod CPU/Memory	Workload resource consumption
Pod restarts	Application instability
Scheduling failures	Resource constraints
Network errors	Connectivity issues
Request latency	Application performance

Troubleshooting methodology

Layer 1 – Application:

• Check application logs
• Verify readiness/liveness probe status
• Check for application errors

Layer 2 – Pod:

• Check pod status (CrashLoopBackOff, ImagePullBackOff, Pending)
• Review pod events
• Check pod logs

Layer 3 – Node:

• Check node status (NotReady, memory pressure, disk pressure)
• Review node logs
• Check node resource utilization

Layer 4 – Network:

• Verify service discovery
• Check DNS resolution
• Test network connectivity

Layer 5 – Azure:

• Verify identity permissions
• Check load balancer configuration
• Review networking configuration

AKS for AI and Machine Learning

AKS is a powerful platform for AI and ML workloads, particularly for model inference and GPU-accelerated processing.

GPU workloads in AKS

AKS supports NVIDIA GPU-enabled node pools to run compute-intensive workloads, including AI/ML training, real-time inferencing, and large-scale data analytics.

When to use GPUs:

• Machine learning and deep learning training
• Real-time inference for AI models
• Computer vision and image processing
• Video processing and transcoding
• Data science and analytics workloads

GPU node pools

To run GPU workloads:

1. Create a node pool with GPU-enabled VM SKUs (NC, ND, NV series)
2. Install NVIDIA device plugin for Kubernetes
3. Request GPU resources in pod specifications

AI workload architecture

Client
  |
  v
API Gateway / Ingress
  |
  v
AKS
  |
  +--> RAG API (CPU nodes)
  |
  +--> Retrieval Service (CPU nodes)
  |
  +--> Model Inference (GPU nodes)
  |
  +--> Background Workers (CPU nodes)
  |
  +--> Azure AI Search
  |
  +--> Blob Storage

AI use cases on AKS

Model inference – Deploy trained models as scalable services using frameworks like Triton Inference Server, vLLM, TensorFlow Serving, or PyTorch Serve.

RAG applications – Host RAG (Retrieval-Augmented Generation) APIs that combine vector search with LLM inference.

Batch inference – Process large batches of data through models using background workers.

Training – Run distributed training jobs on GPU node pools.

Data preprocessing – Use CPU nodes for data preparation before GPU-accelerated training.

Important considerations

Cost – GPUs are expensive. Use them only for workloads that benefit from parallel processing. Scale GPU node pools to zero when not in use.

Scheduling – Use node selectors, taints, and tolerations to ensure GPU workloads are scheduled on GPU nodes.

Partitioning – AKS supports NVIDIA GPU node partitioning strategies including Multi-Instance GPU (MIG) for sharing GPU resources.

AKS Architecture Patterns

Pattern 1: Standard web application

Internet
   |
   v
Ingress Controller
   |
   v
AKS
   |
   +--> Web Pods (stateless)
   +--> API Pods (stateless)
   |
   +--> Azure Database (PostgreSQL, SQL)
   +--> Azure Cache for Redis

Characteristics: Stateless web tier, stateful database tier, horizontal scaling.

Pattern 2: Microservices

Internet
   |
   v
API Gateway / Ingress
   |
   v
AKS
   |
   +--> Service A (orders)
   +--> Service B (payments)
   +--> Service C (inventory)
   |
   +--> Service Bus (async communication)
   +--> Databases (per service)

Characteristics: Service decomposition, independent scaling, asynchronous communication.

Pattern 3: Event-driven AKS

Event Source
   |
   v
Event Grid / Service Bus
   |
   v
AKS Workers
   |
   v
Storage / Database

Characteristics: Event-driven processing, decoupled consumers, background processing.

Pattern 4: AI workload

Client
 |
 v
API Gateway
 |
 v
AKS
 |
 +--> Retrieval Service (CPU)
 +--> Inference Service (GPU)
 +--> Background Workers (CPU)
 |
 +--> Azure AI Search
 +--> Blob Storage
 +--> Vector Database

Characteristics: GPU-accelerated inference, RAG patterns, asynchronous processing.

AKS Architecture Decisions

Decision checklist

Area	Considerations
Public vs. private cluster	Public for internet-facing workloads; private for internal-only
Node pool strategy	System + user pools; specialized pools for specific workloads
Networking model	Azure CNI Overlay (recommended); avoid kubenet
Ingress architecture	Ingress controller; Application Gateway Integration; API Management
Identity model	Entra ID + RBAC; Workload Identity for pods
Storage strategy	Azure Disks (RWO), Azure Files (RWX), Azure NetApp Files (high-performance)
Scaling strategy	HPA for pods; Cluster Autoscaler for nodes
Availability zones	Yes for production; spread across zones
Workload isolation	Namespaces; network policies; separate node pools
CI/CD	GitHub Actions, Azure DevOps, Argo CD, Flux
Observability	Container Insights; Application Insights; custom metrics
Security	RBAC; network policies; pod security; Key Vault
Upgrade strategy	Test in staging; control plane first; node pools gradually
Cost model	Right-size nodes; use spot instances; scale down non-production

AKS vs. Other Azure Container Services

AKS vs. Azure Container Apps

Area	AKS	Azure Container Apps
Orchestration	Full Kubernetes API	Managed Kubernetes (abstracted)
Kubernetes API	Full access	Limited abstraction
Operational complexity	Higher (manage Kubernetes)	Lower (managed platform)
Networking control	Full (VNet, CNI, network policies)	Limited (managed networking)
Workload flexibility	High (any Kubernetes workload)	Limited (container apps only)
Autoscaling	HPA + Cluster Autoscaler	Built-in KEDA-based scaling
Microservices	Full support	Good support
Infrastructure control	High (node pools, VM SKUs)	Low (managed infrastructure)
Developer experience	Kubernetes expertise required	Simplified (code-focused)
Best use cases	Complex Kubernetes requirements, full control	Containerized microservices, simplified operations

AKS vs. Azure App Service

Area	AKS	App Service
Abstraction	Infrastructure (Kubernetes)	Platform (PaaS)
Control	Full (nodes, networking, storage)	Limited
Customization	High	Low
Operational complexity	Higher	Lower
Best use cases	Complex microservices, custom networking, AI/GPU workloads	Web applications, APIs, simple microservices

AKS vs. Azure Container Instances (ACI)

Area	AKS	ACI
Orchestration	Full Kubernetes	Single container or container group
Scaling	Horizontal (pods and nodes)	Simple scale
Networking	VNet, network policies	Basic
Best use cases	Complex applications, microservices	Simple containers, burst workloads

AKS vs. AWS EKS vs. Google GKE

Capability	AKS	Amazon EKS	Google GKE
Managed Kubernetes	✅	✅	✅
Control plane pricing	Free	~$0.10/hour	Free (with Autopilot)
Cloud identity integration	Entra ID	IAM	Cloud IAM
Container registry	ACR	ECR	Artifact Registry
Networking integration	Azure VNet, CNI	VPC, CNI	VPC, GKE networking
Autoscaling	HPA, Cluster Autoscaler	HPA, Cluster Autoscaler	HPA, Cluster Autoscaler
Observability	Azure Monitor, Container Insights	CloudWatch	Cloud Monitoring
AI/GPU workloads	GPU node pools	GPU instances (P4, P5)	GPU node pools (A2, G2)
Serverless container alternative	Container Apps	Fargate	Cloud Run

Key differences:

• Kubernetes concepts are portable, while identity, networking, storage, observability, and managed integrations differ significantly across clouds
• AKS offers a free control plane, unlike EKS and GKE which charge approximately $0.10/hour per cluster
• Each cloud provider has different regional availability, VM SKUs, and pricing models

Common Mistakes

Using AKS when Kubernetes isn't required

Kubernetes adds significant complexity. If your application doesn't need its capabilities (multiple microservices, complex scheduling, self-healing), consider Container Apps or App Service.

Treating AKS as completely managed infrastructure

AKS reduces control-plane management but doesn't eliminate cluster operations. You're still responsible for workloads, networking, security, scaling, and application reliability.

Running databases inside AKS without a strong reason

Stateful workloads like databases are more complex to run in Kubernetes. Consider using managed Azure services (Azure SQL, Cosmos DB, PostgreSQL Flexible Server) instead.

Using one node pool for every workload

Different workloads have different requirements. Use separate node pools for system components and user workloads. Consider specialized pools for GPU workloads.

Ignoring availability zones

Without availability zones, a single zone failure can take down your cluster. Use availability zones for production workloads.

Storing secrets in container images

Never hard-code credentials in container images. Use Kubernetes Secrets, Azure Key Vault, or environment variables.

Using static cloud credentials inside pods

Don't store Azure credentials in pods. Use Workload Identity for pod-to-Azure authentication.

Giving pods excessive Azure permissions

Apply least privilege. Grant pods only the permissions they need to function.

Ignoring Kubernetes RBAC

Control plane access should be limited with Kubernetes RBAC. Don't give all users cluster-admin permissions.

Ignoring network policies

Without network policies, any pod can talk to any other pod. Use network policies to enforce zero-trust networking.

Overprovisioning nodes

Large nodes waste money. Right-size nodes and use autoscaling.

Disabling autoscaling

Autoscaling handles variable workloads. Disabling it leads to either over-provisioning (waste) or under-provisioning (performance issues).

Using excessive pod concurrency

Too many pods can overwhelm downstream systems. Set appropriate concurrency limits.

Ignoring downstream dependencies

Your database or API might not handle the load from scaled pods. Design downstream systems for the expected scale.

Treating readiness and liveness probes as interchangeable

Readiness probes control traffic. Liveness probes restart containers. They serve different purposes and should be configured differently.

Performing production upgrades without testing

Kubernetes upgrades can break workloads. Always test upgrades in a staging environment first.

Ignoring deprecated Kubernetes APIs

Kubernetes APIs are deprecated and removed over time. Check for deprecated APIs before upgrades.

Building overly complicated microservices

Not everything needs to be a microservice. Start simple and refactor when necessary.

Using AKS for workloads better suited to Container Apps or App Service

Choose the right tool. Container Apps and App Service are simpler for many workloads.

Treating Kubernetes complexity as automatically beneficial

Kubernetes complexity is a cost. Only pay it when the benefits justify it.

Best Practices

Architecture

• Use AKS only when its capabilities justify Kubernetes complexity
• Separate system and application workloads using appropriate node pools
• Design node pools around workload characteristics (CPU, memory, GPU, storage)
• Use availability zones for production workloads
• Use Azure CNI Overlay networking for new clusters

Identity and security

• Use managed identities and Workload Identity
• Integrate with Microsoft Entra ID for authentication
• Apply least-privilege Azure RBAC and Kubernetes RBAC
• Keep secrets out of source code and container images
• Use Azure Key Vault for sensitive configuration
• Use private networking where required

Workload reliability

• Use multiple replicas for critical workloads
• Spread workloads across failure domains (nodes, zones)
• Configure readiness, liveness, and startup probes appropriately
• Use Pod Disruption Budgets
• Use autoscaling carefully (HPA + Cluster Autoscaler)

Operations

• Use Azure Container Registry securely (managed identity)
• Implement CI/CD with automated testing and image scanning
• Monitor both Kubernetes and application health
• Test Kubernetes upgrades before production
• Keep workloads stateless where practical
• Externalize persistent state into appropriate Azure services
• Maintain separate development, staging, and production environments
• Continuously review cluster utilization and cost

Development

• Use meaningful resource names and labels
• Use ConfigMaps for non-sensitive configuration
• Use Helm for packaging and deploying applications
• Implement health checks (readiness + liveness probes)
• Design for failure and duplicate processing

Practical Learning Path

1. Understand containers – Docker basics, images, container runtimes

2. Learn Kubernetes fundamentals – Pods, Deployments, Services, Namespaces

3. Understand AKS architecture – Control plane, node pools, networking

4. Create a basic AKS cluster – Azure CLI, portal, or Terraform

5. Deploy a containerized application – Kubernetes manifests, kubectl

6. Integrate Azure Container Registry – Push images, pull authentication

7. Configure networking and ingress – Services, Ingress controllers, TLS

8. Learn Azure and Kubernetes RBAC – Entra ID integration, Kubernetes roles

9. Configure Workload Identity – Pod-to-Azure authentication

10. Connect AKS to Azure Storage and databases – PVCs, StorageClasses

11. Implement autoscaling – HPA, Cluster Autoscaler, custom metrics

12. Configure monitoring and logging – Container Insights, Application Insights

13. Practice rolling and canary deployments – Deployment strategies

14. Learn AKS security hardening – Network policies, pod security, Key Vault

15. Practice upgrades and failure recovery – Control plane and node upgrades

16. Build a production-style microservices architecture – Multiple services, ingress, service mesh

17. Experiment with GPU and AI workloads – GPU node pools, model inference

18. Compare AKS with EKS and GKE – Cross-cloud Kubernetes

Key Takeaways

AKS is Azure's managed Kubernetes service that reduces the complexity and operational overhead of managing Kubernetes by offloading much of that responsibility to Azure.

AKS separates the managed control plane from customer-managed nodes. Azure manages the control plane; you manage node pools and workloads.

Node pools are a fundamental AKS architecture decision. Use separate system and user node pools. Specialized pools for GPU and compute-intensive workloads improve isolation and performance.

Networking, identity, security, and observability require deliberate design. These aren't automatically managed by Azure. Use Azure CNI Overlay, Workload Identity, and Azure Monitor.

Workload Identity is preferred over embedded Azure credentials for pod-to-Azure authentication. It eliminates credentials from pods and provides secure, federated identity.

Autoscaling must consider both pods and nodes. HPA scales pods; Cluster Autoscaler scales nodes. They work together.

High availability requires workload distribution across failure domains. Use availability zones, multiple replicas, pod anti-affinity, and Pod Disruption Budgets.

AKS is powerful but operationally more complex than higher-level container platforms. Use it when Kubernetes capabilities justify the complexity.

AKS is particularly useful for complex microservices, highly customized container platforms, and AI/GPU workloads. For simpler workloads, consider Container Apps or App Service.

The best AKS architecture starts with workload requirements, not with Kubernetes features. Design around your application's needs—scalability, state, performance, security, and team expertise—then choose AKS features that support those needs.