The case
Black Friday came early. And each server needed a whole system.
On the eve of Black Friday, Isaac needed more instances of the order system. He launched new VMs, installed the system on each one, and configured the load balancer by hand. When the peak passed, there were still machines running with nothing to do.
The store's containers were already ready. What was missing was a way to run them without looking after each server.
This post builds on Cloud Fundamentals, Cloud Service Models: IaaS, PaaS, SaaS and FaaS and Containers: isolation without a virtual machine.
The problem
Scaling containers on VMs is manual work
A container solves packaging, but it doesn't decide where to run, how many copies to keep, or what to do when a copy fails. If the store runs one VM per instance, every scale-up is a new request, an installation and a configuration.
Without a layer for that, the team spends more time administering servers than improving the store.
The solution
Container services
A container service (Containers as a Service, or CaaS) sits between the VM and the platform: you deliver the image and the configuration, and the provider takes care of the server. There are two main forms.
- Serverless containers: you don't choose or see the server. The provider creates the environment for each container and charges for the CPU and memory time used. Some scale to zero when there is no traffic.
- Managed orchestration: you define how many copies you want, and a control plane (Kubernetes) distributes, restarts and scales the containers over a set of nodes.
Each provider has its own names: Cloud Run on Google Cloud, AWS Fargate on AWS, Azure Container Apps on Azure and OCI Container Instances on Oracle Cloud.
Serverless
You deliver the image and get a URL. No machine to choose and no operating system to update.
At the store: the order API starts in seconds when customers arrive, and goes away when they leave.
Orchestration
Kubernetes keeps the number of copies you defined and replaces the ones that fail.
At the store: if one copy of the order system goes down, another starts without Isaac doing anything.
Automatic scaling
The number of copies follows the load, up and down.
At the store: Black Friday gets extra copies, and they are released after the peak.
Pay for use
You pay for the resources the containers consume while they run.
At the store: idle copies don't generate the cost of a machine standing still.
Don't mix them up
Containers × managed services
Serverless container service
“Do I just want to deliver the container and get a URL, with nothing to manage?”
- Example
- The store's order API, which scales with traffic.
- Focus
- Less administration. You control the image and the configuration.
Orchestration (Kubernetes)
“Do I need many services, with complex network and deployment rules?”
- Example
- Ten store services, each with its replicas and its communication rules.
- Focus
- More control and more complexity. You define the configuration of each piece.
Under the hood
Orchestration with Kubernetes
Kubernetes is the most widely used orchestration system. You describe the desired state, for example "three copies of the order API, with 512 MB each", and it does what is needed to get there: it chooses which node each copy runs on, restarts the ones that fail, and rolls out new versions gradually.
Providers offer the managed control plane: Google Kubernetes Engine (GKE) on Google Cloud, Amazon EKS on AWS, Azure Kubernetes Service (AKS) on Azure and Oracle Kubernetes Engine (OKE) on Oracle Cloud. Even so, you need to understand the pieces: copies, services, configurations and network rules.
Images
Image registry
Every image used in the cloud is stored in a registry. The server (or the service) downloads the image from there when it needs to start a container. Each provider has its own: Artifact Registry on Google Cloud, Amazon ECR on AWS, Azure Container Registry on Azure and OCI Container Registry on Oracle Cloud.
Whoever controls the registry controls what can run in production. That's why access to the registry follows the same identity and permission rules as the rest of the cloud, as we saw in Security and Shared Responsibility in the Cloud.
In practice
Choosing the path
| Situation | Choice |
|---|---|
| An API that scales with traffic, with no infrastructure team | Serverless container service |
| Many services that talk to each other, with their own network rules | Managed orchestration |
| Steady, predictable load, with full control of the machines | VMs, with containers inside if needed |
| An image that must be available in several regions | Registry with replication across regions |
Key terms
Six terms to remember
Tap a card to see the definition.
Quick test
Three questions, exam style
01
The team wants to run an API in containers, without administering servers or Kubernetes. Which option meets that?
02
Where are the images the store uses in production kept?
03
A container service has many instances on Black Friday and few the rest of the year. Which feature reduces the cost outside the peak?
Containers in the cloud take away the most repetitive part of the team's work: choosing servers, installing systems and reacting to every peak. What remains is deciding the size of the copies, the scaling limit and who can publish images. For the store, that means bringing the API up for Black Friday without an afternoon of configuration, and releasing the resources when the peak passes.