GitOps With ArgoCD: From Hype to Stable Production

ArgoCD^[1] has gone in a few years from an interesting project to the standard deployment practice for Kubernetes. GitOps (where the Git repository is the single source of truth for the cluster’s desired state) has displaced traditional push pipelines in teams already running Kubernetes. This guide covers the principles, what ArgoCD does well, the mistakes seen in production, and the comparison with Flux.

Key takeaways

• GitOps isn’t just "deploy from Git": its essence is continuous reconciliation that corrects drift automatically.

• ArgoCD stands out for its rich UI, mature multi-tenancy, and broad Helm, Kustomize, and plain YAML support.

• The app-of-apps pattern lets you manage the entire cluster declaratively.

• Secrets never go in the config repository: use Sealed Secrets or External Secrets Operator.

• The right sync mode for production at the start is manual; scale toward auto-sync as confidence grows.

The four real GitOps principles

GitOps has four formal principles:

• Declarative: the entire system is described declaratively (Kubernetes YAML, Helm/Kustomize manifests), not as a sequence of commands.

• Versioned and immutable: the desired state lives in Git. Every change is an auditable commit.

• Automatically pulled: agents in the cluster pull from the repository and apply changes. No external push from CI.

• Continuous reconciliation: the agent constantly compares actual vs desired state and corrects drift.

The most important implication is the fourth. If someone edits a resource by hand (kubectl edit), ArgoCD reverts it to the Git state. That difference from traditional pipelines is the core value.

Why ArgoCD has won adoption

ArgoCD is the most popular option because it combines these strengths:

• Excellent UI: visual view of applications, their resources, health state, and diff between Git and cluster.

• Serious multi-tenancy: supports multiple teams with granular permissions via RBAC + projects.

• Flexible sync policies: manual, auto, with prune, with self-heal; each combination serves different scenarios.

• Broad tool support: Helm, Kustomize, plain YAML, and Jsonnet coexist in the same cluster.

• App-of-apps pattern: an ArgoCD app that deploys other ArgoCD apps, useful for managing the entire cluster declaratively.

• Sync waves and hooks: control over deployment order when it matters (CRDs before resources using them).

GitHub Actions logo, one of the CI platforms that feed pipelines updating the ArgoCD config repository (Image: Mohamed Hassan, CC BY-SA 4.0, via Wikimedia Commons)

Typical repository structure

config-repo/
├── applications/
│   ├── prod/
│   │   ├── app-a.yaml      # ArgoCD Application pointing to manifests
│   │   └── app-b.yaml
│   └── staging/
│       └── ...
├── manifests/
│   ├── app-a/
│   │   ├── base/           # Kustomize base
│   │   └── overlays/
│   │       ├── prod/
│   │       └── staging/
│   └── app-b/
│       └── helm/           # Helm chart or values
└── argocd-bootstrap/       # ArgoCD's own configuration

Patterns that work:

• Separate app code from configuration: configuration changes don’t require image rebuild.

• Directories per environment (Kustomize overlays) over branches, making it easier to see differences.

• Image automation with automated commits: ArgoCD Image Updater detects new images and commits to the config repo.

Sync policies: when to use each

• Manual sync: someone presses "Sync" after Git changes. Recommended in production initially.

• Auto-sync without prune: applies changes but doesn’t delete resources disappearing from Git. Conservative.

• Auto-sync with prune: applies all changes and deletes what’s no longer in Git. Dangerous if someone accidentally deletes something.

• Self-heal: in addition to applying Git, reverts any manual change. Real strict GitOps.

Progressive recommendation: manual in production at first, auto-sync without prune in staging, self-heal in dev. Move production to controlled auto-sync as team confidence grows.

Common production mistakes

The mistakes seen most in real projects:

• Self-heal without team discipline: if someone changes something by hand for debugging and ArgoCD reverts it within a minute, frustration follows. Clear policy: manual changes only in documented emergencies.

• Secrets in Git: never. Use Sealed Secrets, External Secrets Operator, or similar.

• Monstrous config repositories: a single repo with 200 apps is slow to sync. Split by domain or team.

• Misused sync waves: too many inter-wave dependencies complicates deploys. Keep the graph simple.

• CRDs as normal apps: CRDs must precede resources using them. Use sync waves or a separate bootstrap app.

• No ArgoCD config backups: ArgoCD itself must be in Git (app-of-apps). If the cluster dies, you recover with argocd app create pointing to the config repo.

• Cluster admin for everyone: configure ArgoCD projects with limited permissions per team and namespace.

Prometheus logo, the metrics tool that complements ArgoCD in observing the deployment stack (Image: Alexander Schwartz (ahus1), Apache License 2.0, via Wikimedia Commons)

Comparison with Flux

Flux^[2] is the other mature GitOps tool. The most relevant practical differences:

• UI: ArgoCD has a rich native UI; Flux’s is more limited (improving with Weave GitOps).

• Multi-tenancy: ArgoCD uses projects; Flux uses Kubernetes native namespaces.

• Philosophy: ArgoCD feels more like an "application"; Flux feels more like "controllers in the cluster".

• Image automation: Flux has it natively; ArgoCD requires the Image Updater component.

• Multi-cluster: ArgoCD supports managing multiple clusters from one instance; Flux assumes one instance per cluster.

Read the full Flux vs ArgoCD comparison for more detail. Both are graduated CNCF projects and solid choices; the difference is which fits your team better.

Conclusion

ArgoCD has consolidated GitOps as a mature deployment practice for Kubernetes. Well implemented (with team discipline, secrets managed separately, and sync policies appropriate to the environment) it improves deploy reliability, audit trail, and recovery speed. Poorly implemented (with secrets in Git, monstrous configuration, and no backups) it just adds another layer of complexity. The difference lies in operational details, not the tool.

Frequently asked questions

Which ArgoCD sync policy should I use in production at the start?

Manual: someone presses Sync after Git changes. The progressive recommendation is manual in production at first and auto-sync without prune in staging, which applies changes but doesn't delete resources that disappear from Git. In dev, self-heal, which also reverts any manual change; auto-sync with prune is dangerous if someone accidentally deletes something. Move production to controlled auto-sync as team confidence grows.

What happens if someone edits a resource by hand with kubectl edit in an ArgoCD-managed cluster?

With self-heal enabled, ArgoCD detects the drift during continuous reconciliation and reverts the resource to the state declared in Git, within about a minute. That is the core value of GitOps over push pipelines, but without team discipline it causes frustration when someone changes something for debugging and it vanishes. The policy must be clear: manual changes only in documented emergencies.

How do I handle secrets if all configuration lives in Git?

Secrets never go in the config repository. Use Sealed Secrets or External Secrets Operator so the cluster gets them encrypted or from an external store. Other common production mistakes: a single monstrous repo with 200 apps, slow to sync and better split by domain or team. Also not keeping ArgoCD itself in Git via the app-of-apps pattern, or giving everyone cluster admin instead of ArgoCD projects with limited permissions per team and namespace.

Sources

1. ArgoCD
2. Flux