A systemd service crash-looping looks, unmistakably, like a Kubernetes CrashLoopBackOff: the same underlying symptom (a process restarting indefinitely) has existed forever on a plain VM, with its own backoff mechanics and its own circuit breaker, often poorly understood because never explicitly configured.
What Restart= actually triggers
Restart=on-failure automatically relaunches a service exiting with a non-zero code, a configuration that looks sufficient but says nothing about how often attempts happen or the limit past which systemd should give up.
[Service]
ExecStart=/usr/bin/my-app
Restart=on-failure
RestartSec=5
RestartSec sets a fixed delay between each attempt (5 seconds here), a notable difference from Kubernetes’ exponential backoff: systemd, by default, doesn’t automatically increase that delay on each successive failure, absent additional explicit configuration.
The real circuit breaker: StartLimitBurst
Without an explicit limit, a permanently failing service restarts indefinitely, every RestartSec seconds, forever. StartLimitIntervalSec and StartLimitBurst together define a circuit breaker: past a given number of starts (StartLimitBurst) within a given time window (StartLimitIntervalSec), systemd simply stops restarting the service.
[Unit]
StartLimitIntervalSec=60
StartLimitBurst=5
[Service]
Restart=on-failure
RestartSec=5
This configuration tolerates 5 starts in 60 seconds; the sixth failure within that window blocks any further restart, until a manual intervention (systemctl reset-failed). Without this setting, a service in a fast failure loop (a configuration error crashing it within a second) can burn a disproportionate amount of CPU on repeated startup attempts, never stopping on its own.
The trap: a service that looks dead with no clear message
Once StartLimitBurst is hit, systemctl status shows a failed state, but doesn’t always clearly explain that the cause is the circuit breaker rather than a still-active application problem. Confusing the two leads to chasing a bug that may already be fixed, when the service is simply stuck behind the circuit breaker, waiting on an explicit reset-failed.
# Diagnosis: telling a genuinely persistent application
# failure apart from an already-tripped circuit breaker
systemctl status my-app
journalctl -u my-app --since "10 min ago"
# Resets the circuit breaker counter, required after
# a fix, otherwise the service stays stuck even once
# the real bug has been corrected
systemctl reset-failed my-app
systemctl start my-app
What journalctl reveals that systemctl status doesn’t
systemctl status shows a summary, often truncated to the last few lines; journalctl -u <service> gives the full history of start attempts and their associated error messages, the only reliable source for telling a one-off configuration error apart from a problem repeating identically on every attempt.
# The full history, not just the last failure,
# needed to see whether the error changes between attempts
journalctl -u my-app -n 200 --no-pager
Takeaway
Restart=on-failure relaunches a failing service, but never stops it from looping indefinitely without an explicit circuit breaker: StartLimitIntervalSec and StartLimitBurst, together, are that circuit breaker, systemd’s equivalent of the mechanism protecting a Kubernetes cluster from a CrashLoopBackOff that never stops. A service stuck after hitting that limit shows failed without always clearly distinguishing the circuit breaker from a still-active application bug, a confusion resolved by reading the full history via journalctl, never by trusting systemctl status’s summary alone. This setting, often forgotten because it has no visible consequence until the day a service enters a fast failure loop, is part of the Linux operations basics that stay relevant even on a largely containerized infrastructure.