Packer and cloud-init both play a role in a golden-image pipeline, and yet confusing their respective jobs silently breaks that pipeline: a provisioning script placed in the wrong spot (the wrong tool, the wrong moment) either runs too early, never runs, or runs on every instance boot instead of exactly once.

Packer: building an image, once, never in production

Packer boots a temporary machine (a VM, a container), runs the requested provisioning scripts on it, captures the result as an image (AMI, disk image, container image), then destroys the temporary machine: that machine never exists in production, only for the duration of the image build.

# packer.pkr.hcl: provisions a temporary machine,
# never used directly in production
build {
  sources = ["source.amazon-ebs.example"]

  provisioner "shell" {
    script = "install-nginx.sh"
  }
}

Anything that must be identical on every instance derived from this image (installed packages, shared system configuration) belongs at this stage, run exactly once per image version, never rerun when an individual instance boots.

cloud-init: configuring each instance, at its first boot

cloud-init, already documented for the VM-cloning trap, kicks in the moment an individual instance actually boots from the image Packer built: it configures whatever needs to vary from one instance to another (hostname, SSH host keys specific to that instance, injecting secrets or environment-specific configuration via user-data).

# user-data: runs on the first boot
# of THIS specific instance, never during the Packer build
#cloud-config
hostname: web-01
packages:
  - htop

The trap: mixing up the two moments

A package installation script placed in user-data instead of a Packer provisioner runs on every new instance boot, needlessly slowing down every deployment with an operation that should have been baked once into the image. Conversely, instance-specific configuration (a hostname, an environment-specific API key) baked into the image by Packer instead of handled by cloud-init makes that image unusable for more than one instance at a time, defeating the entire point of a reusable image.

The split that actually works

Packer handles everything common to every future instance (installed software, system hardening), producing a versioned, immutable image. cloud-init handles everything that distinguishes one individual instance from another at boot, starting from that same image. The decision rule is simple: if the same value must appear on every instance without exception, it belongs in Packer; if it varies from one instance to another, it belongs in cloud-init.

Takeaway

Packer builds an image exactly once, on a temporary machine later destroyed, never present in production: everything that must be identical across derived instances belongs at this stage. cloud-init configures each individual instance at its actual first boot, starting from that already-built image: everything that varies from one instance to another belongs to it. Mixing up the two moments silently breaks the pipeline, either by slowing down every boot with an operation that should have been baked once, or by producing an image that isn’t reusable because instance-specific configuration wasn’t left to cloud-init.