GitOps With ArgoCD: From Hype to Stable Production

javi
Pantalla con ramas y commits de Git en un grafo visual

ArgoCD has gone in a few years from being an interesting project to the standard deploy practice for Kubernetes. GitOps — the model where the Git repository is the single source of truth for the cluster’s desired state — has proven to work better than traditional push pipelines for many teams. We cover the principles, what ArgoCD does well, the mistakes seen in production, and the comparison with Flux.

The Real GitOps Principles

GitOps isn’t just “deploy from Git”. It has four formal principles:

  1. Declarative: the entire system is described declaratively (Kubernetes YAML, Helm/Kustomize manifests), not as a sequence of commands.
  2. Versioned and immutable: the desired state lives in Git. Every change is an auditable commit.
  3. Automatically pulled: agents in the cluster (ArgoCD/Flux) pull from the repository and apply changes. No external push from CI.
  4. Continuous reconciliation: the agent constantly compares actual vs desired state and corrects drift.

The most important implication is the fourth: drift is automatically corrected. If someone edits a resource by hand (kubectl edit), ArgoCD reverts it to the Git state. That’s the key difference from traditional pipelines.

Why ArgoCD Has Won Adoption

Some reasons it’s the most popular option in 2023:

  • Excellent UI. Visual view of applications, their resources, health state, diff between Git and cluster.
  • Serious multi-tenancy. Supports multiple teams with granular permissions via RBAC + projects.
  • Flexible sync policies: manual, auto, with prune, with self-heal — every combination useful for different scenarios.
  • Broad tool support: Helm, Kustomize, plain YAML, Jsonnet — coexist in the same cluster.
  • App-of-apps pattern. An ArgoCD app that deploys other ArgoCD apps — useful to manage the entire cluster declaratively.
  • Sync waves and hooks. Control over deployment order when it matters (e.g., CRDs before resources using them).

Typical Repository Structure

A reasonable organisation seen in projects:

config-repo/
├── applications/
│   ├── prod/
│   │   ├── app-a.yaml      # ArgoCD Application pointing to manifests
│   │   └── app-b.yaml
│   └── staging/
│       └── ...
├── manifests/
│   ├── app-a/
│   │   ├── base/           # Kustomize base
│   │   └── overlays/
│   │       ├── prod/
│   │       └── staging/
│   └── app-b/
│       └── helm/           # Helm chart or values
└── argocd-bootstrap/       # ArgoCD's own configuration

Patterns that work:

  • Separate app code from configuration. The app code repo isn’t the config repo. Configuration changes don’t require image rebuild.
  • Branch or directory per environment. I prefer directories (Kustomize overlays) over branches — easier to see differences between environments.
  • Image automation with automated commits. Tools like ArgoCD Image Updater detect new images and commit to the config repo.

Sync Policies: When Each One

ArgoCD offers several auto-sync options:

  • Manual sync: someone must press “Sync” after Git changes. Good for critical production environments where you want human review.
  • Auto-sync without prune: applies changes but doesn’t delete resources that disappear from Git. More conservative.
  • Auto-sync with prune: applies all changes and deletes what’s no longer in Git. More strict but dangerous if someone accidentally deletes something.
  • Self-heal: in addition to applying Git, reverts any manual change. Real strict GitOps.

A recommendation: manual sync in production initially, auto-sync without prune in staging, self-heal in dev. As confidence grows, move production to controlled auto-sync.

Common Production Mistakes

After several projects, the mistakes I see most:

  • Self-heal without team discipline. If a dev changes something by hand for debug and ArgoCD reverts within a minute, frustration. Clear policy: manual changes → understand they get reverted, do them only in documented emergencies.
  • Secrets in Git. Obvious antipattern but happens. Use Sealed Secrets, External Secrets Operator (ESO), or similar — never commit clear secrets.
  • Monstrous configuration repositories. A single repo with 200 apps becomes slow to sync and hard to review. Consider splitting by domain or team.
  • Misused sync waves. Too many dependencies between waves complicates deploys. Keep the graph simple.
  • CRDs as a normal app. CRDs must come before resources using them. Sync waves or separate bootstrap app.
  • No backups of ArgoCD configuration. ArgoCD itself must be in Git (app-of-apps). If your cluster dies, you recover by argocd app create pointing to the config repo.
  • Cluster admin for everyone. Configure ArgoCD projects with limited permissions. Team X can only deploy in namespaces Y.

Comparison With Flux

Flux is the other mature GitOps tool. Practical differences:

  • UI: ArgoCD has a rich native UI; Flux’s is more limited (improving with Weave GitOps).
  • Multi-tenancy: ArgoCD covers it with projects; Flux with namespaces and multi-source.
  • Philosophy: ArgoCD feels more like an “application”, Flux more “controllers in the cluster”.
  • Helm: both support it; integrations slightly different.
  • Image automation: Flux has it natively; ArgoCD requires extra component (Image Updater).
  • Adoption: ArgoCD seems more popular in projects I’ve seen recently.

Either is a reasonable choice. ArgoCD is friendlier for teams valuing UI; Flux aligns more with “everything is a controller in the cluster”.

Conclusion

ArgoCD has consolidated GitOps as a mature deploy practice for Kubernetes. Well implemented — with team discipline, secrets managed separately, and sync policies appropriate to the environment — it improves deploy reliability, audit trail, and recovery speed. Poorly implemented — with secrets in Git, monstrous configuration, no backups — it just adds another layer to maintain. The difference is in the operational details.

Follow us on jacar.es for more on Kubernetes, deployment automation, and modern operation practices.

Entradas relacionadas