Ephemeral App Mindset

Containers are meant to be ephemeral. We as developers should expect our container to be terminated at any time, for reasons such as rolling deployments, downscaling, node termination, or resource usage limits.

This fact becomes even more crucial now that cloud providers offer cost-effective computing through spot instances. With spot instances, your containers will move around nodes, meaning they’ll be provisioned and terminated a lot.

There’s a couple of things to keep in mind to fully embrace the ephemeral app mindset.

Listen for SIGTERM signals. Kubernetes will send a SIGTERM signal to your container to let it know that it’s going to be terminated soon. This is the point where you should finish up any remaining work. Here’s what you should care about:

How much time do you have to wrap things up? By default, 30 seconds. This is determined by terminationGracePeriodSeconds.

The SIGTERM signal is a generic signal used to cause program termination. Unlike SIGKILL, this signal can be blocked, handled, and ignored. It is the normal way to politely ask a program to terminate. Source

Configure your health checks (readiness and liveness probes). For readiness, probe dependencies that your app can’t serve requests without. For liveness, you mostly just need to know when your app is in a deadlock so Kubernetes can restart it, so can be more lenient and probe something simple like a controller.

Horizontal scaling and stateless apps. We’ll spawn and terminate instances on the spot to meet the current demand. Containers should be stateless to be scale horizontally.

See Also