A sync fails on a CRD, and the error says nothing about the manifest itself:

metadata.annotations: Too long: may not be more than 262144 bytes

The CRD has no annotations. The one overflowing was written by kubectl apply, and this is the point where server-side apply stops being a release-notes curiosity.

The annotation client-side apply drags along

By default, Argo CD applies manifests with a plain client-side kubectl apply, which relies on the kubectl.kubernetes.io/last-applied-configuration annotation to remember the previous state of the resource. That annotation holds the entire serialized manifest. On a thirty-line Deployment, nobody notices. On a CRD whose OpenAPI schema runs to hundreds of kilobytes, it blows past the Kubernetes annotation size limit — 256 KiB total on the object, the 262,144 bytes that show up verbatim in the error.

The three-way merge built on that annotation, and the bug it produces when someone edits an object by hand, is covered separately in the merge nobody actually looks at. What matters here is that it does not scale to a large object, and that the workaround changes the ownership model entirely.

What the server tracks instead

With --server-side, the merge is computed by the API server, and the reference state is no longer an annotation but metadata.managedFields: the list of managers, each with the set of fields it claims.

# managedFields is omitted by default: ask for it explicitly
kubectl get deployment my-app -o yaml --show-managed-fields
managedFields:
  - manager: argocd-controller
    operation: Apply          # "Apply" for server-side apply, "Update" otherwise
    apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:template:
          f:spec:
            f:containers: {}
  - manager: kube-controller-manager
    operation: Update
    fieldsV1:
      f:spec:
        f:replicas: {}

Three rules follow, and they are the real difference with client-side mode:

  • A field owned by another manager with a different value produces a conflict, not a silent overwrite.
  • Removing a field from your manifest deletes it from the object only if no other manager claims it. Otherwise you merely drop your own claim.
  • A write that is not an apply — kubectl edit, kubectl scale, a controller doing an update — also records ownership, with operation: Update, and can never fail on a conflict. Only apply stops.

Two appliers that set the same value on a field share ownership of it. From then on, whichever one wants to change it hits a conflict.

Argo CD never asks permission

The option goes on the Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  syncPolicy:
    syncOptions:
      - ServerSideApply=true

or per resource, with the argocd.argoproj.io/sync-options: ServerSideApply=true annotation. The reverse works too: ServerSideApply=false on a specific resource opts it out while the Application has the option enabled. And Replace=true takes precedence over ServerSideApply=true — combining them does not do what you would expect.

The detail worth stopping on is elsewhere. When the option is on, Argo CD runs kubectl apply --server-side --force-conflicts. The force flag is not configurable: it is the command. The manager is named argocd-controller, and it wins every conflict, on every sync.

That choice matches the Kubernetes recommendation, which tells controllers to force conflicts on objects they own, precisely because a controller has no way to resolve or act on a conflict. But the consequence has to be taken seriously: the conflict mechanism protects nobody from Argo CD. The boundary between Argo CD and another controller is not drawn in managedFields, it is drawn in the repository, by not declaring the field.

The textbook case is spec.replicas facing a HorizontalPodAutoscaler. The Kubernetes docs describe the clean handover: both actors first set the same value, then the one giving up removes the field from its configuration. In GitOps terms, that means deleting replicas from the versioned manifest. An ignoreDifferences alone is not enough: it only affects diff computation, and the desired state is applied as-is during sync. To make it apply during the sync stage too, you need RespectIgnoreDifferences=true — and it only has an effect if the resource already exists.

Applying a manifest that isn’t one

Server-side apply accepts a partial intent: an object containing only the fields you have an opinion about. Argo CD uses this to patch a resource it does not fully manage.

# Not a valid Deployment: no selector, no template.
# Valid as a server-side apply intent.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 3

That file fails client-side schema validation, so validation has to be turned off as well:

spec:
  syncPolicy:
    syncOptions:
      - ServerSideApply=true
      - Validate=false

Switching an object that already exists

This is the question everyone asks before enabling the option on a live estate: what happens to the current owners?

On the Kubernetes side, migration from client-side apply is supported and happens without conflicts — provided the last-applied-configuration annotation is up to date. Fields it does not cover are not considered owned by client-side apply, and those do raise conflicts. A kubectl scale run after the last apply is enough to create that case.

On the Argo CD side, a dedicated mechanism absorbs the transition, and it is enabled by default: during a sync, managedFields entries with operation: Update belonging to the kubectl-client-side-apply manager are transferred to argocd-controller, and the original entry is removed. You can turn it off with ClientSideApplyMigration=false, or point it at a different manager:

metadata:
  annotations:
    argocd.argoproj.io/client-side-apply-migration-manager: 'my-custom-manager'

That last knob is the practically useful one: it reclaims the fields of an operator you removed from the cluster whose claims still linger on the objects. Since Argo CD forces conflicts anyway, the migration is not there to avoid a failure — it is there to avoid leaving a managedFields that lies about who writes what.

The diff switches along with the apply

Enabling server-side apply also changes how Argo CD decides an Application is OutOfSync. The legacy strategy compares live state, desired state and the last-applied-configuration annotation — which is no longer maintained. The strategy that used to take over automatically, structured-merge diff, has been discontinued in favour of Server-Side Diff, stable since Argo CD 3.1.

It runs a server-side apply in dry-run mode for each resource and compares the response with the live state. Enable it globally in the argocd-cmd-params-cm ConfigMap:

data:
  controller.diff.server.side: "true"

or per Application with argocd.argoproj.io/compare-options: ServerSideDiff=true.

Two concrete consequences. The good one: admission webhooks take part in the diff, so a manifest a validating webhook will reject is reported at diff time rather than at sync time. The less obvious one: mutating webhooks are not included by default, you have to add IncludeMutationWebhook=true — otherwise an object mutated at admission can sit in permanent drift. And no dry-run happens when creating a resource that does not exist yet: on a first deployment you get neither benefit.

The result is a more honest model than the annotation, one where you can finally read who wrote what. It does not make cohabitation safe: as long as the controller forces, managedFields is a log, not a guard rail. What actually decides is still what the repository declares — the assumption declarative deployment with Argo CD has been making all along, and one the rest of the CI/CD chain has to honour.