Learn GitOps with ArgoCD - Enterprise Patterns & Best Practices

#Introduction

In episode 31 we brought GitOps patterns to three clouds and multi-cloud combinations. But if you've noticed, every episode so far has discussed technology. In the real world, GitOps adoption failures are almost never caused by technology — but by team structure, repo strategy, approval processes, and documentation that didn't change along with it. A GitOps setup that's great on paper can die mid-way because the platform team becomes a bottleneck, or because there's no runbook during an incident.

This episode is the organization and process part. We discuss enterprise patterns: how the platform team works, how repos are structured, how environments are managed, how changes are approved, and how documentation is maintained.

#Organizational Structure

#Platform Team and Application Team Autonomy

The most successful model has two clear roles:

The platform team's goal is to remove itself from the critical path of routine changes. Application teams deploy themselves through self-service (episode 28); the platform team only steps in for platform changes, incidents, and governance. If application teams have to wait for the platform team to deploy, GitOps hasn't succeeded yet.

#Governance Models

Three common governance models:

1. Centralized — all changes go through the platform team; safest, slowest, good for the beginning.
2. Federated — the platform team sets standards and templates; application teams are free within those boundaries. The most balanced model.
3. Autonomous — application teams fully manage their stack; fast, but requires high discipline and tooling that prevents mistakes.

#Center of Excellence (CoE)

A CoE is a small team (2–4 people) that isn't a full platform owner but a standards keeper: putting together templates, the golden path, agreed patterns, and answering other teams' questions. The CoE prevents fragmentation — five teams creating five different ArgoCD patterns. CoE documentation is also a career path: members rotate so knowledge spreads.

#Repository Strategy

#Monorepo vs Polyrepo

Common practice: a per-environment manifest monorepo (one repo containing dev/, staging/, prod/) is the simplest starting point. Move to polyrepo when repo size makes clones slow or access isolation needs grow stronger.

#Shared Configuration Repository

A shared repo contains values used by all teams: base image versions, storage classes, certificates. Referenced via a Kustomize base or Helm values. Changes here affect everyone — so this repo has the strictest review.

#Template Repositories

Template repos (episode 28) are the golden path: an application structure that already meets platform standards. Application teams start from this template, not from scratch. The template is managed by the CoE and becomes an automatic best-practice distribution mechanism.

#Version Control Strategy

• Branch protection — production manifest repos must use branch protection: mandatory review, no direct push.
• Tagged releases — for production, promotion via tags (episode 15) gives a clear trail: release-1.2.3 means a consistent set of manifests.
• Signed commits — for audited environments, commit signatures provide non-repudiation evidence.

#Environment Management

#Environment Parity

Dev, staging, and prod environments should be as close as possible — differing only in values (size, traffic, quota), not behavior. Parity makes staging tests truly meaningful. GitOps helps: the same manifests are promoted upward, not rewritten per environment.

#Configuration Management

Separate manifests and configuration: manifests are identical across environments, configuration (values) differs per environment. In Helm: one values.yaml per environment; in Kustomize: one overlay per environment (episode 8).

#Promotion Pipelines

Promotion = moving manifests from one environment to the next, usually through:

git tag -a release-1.2.3 -m "promote api 1.2.3"
git push origin release-1.2.3
argocd app set api-staging --revision release-1.2.3

After staging proves healthy (health + smoke tests in episode 13), the same is done for prod. Note: what's promoted is the manifest, not the deploy result — the deployment in the cluster is a derivative, not a hand-me-down artifact.

#Environment-Specific Overrides

Per-environment overrides are kept few and explicit: replicaCount, resources, ingress host, storageClass. Every override is a potential drift point — the fewer, the safer.

#Change Management

#Approval Workflows

Git is the best approval mechanism: PR + review = approval workflow. The higher the environment, the stricter the number of reviewers:

branch-protection:
  - branch: prod
    required_pull_request_reviews:
      required_approving_review_count: 2
    required_status_checks:
      - argocd-health

#Change Advisory Board (CAB) and Emergency Procedures

For major changes, keep a CAB (Change Advisory Board) — but limit its scope to high-risk changes (platform migrations, infrastructure changes), not every image bump. The emergency procedure for production incidents must be faster than the normal process: a hotfix mechanism with special access, e.g. a temporary token valid for 60 minutes, or a sync window override (episode 14). A slow emergency process actually pushes people to cut corners — which is more dangerous.

#Rollback Policies

Because Git is the truth, rollback = argocd app rollback api <revision> or reverting the commit. The policy must be written, not implied: when a rollback happens (e.g. error rate > 5 percent for 5 minutes), who's allowed, and how it's reported. A rollback isn't the solution — it returns to the last known good state, then the incident is documented.

#Documentation

Documentation is infrastructure that's rarely deployed but always used in a panic. Four documents that must exist:

• Runbooks — step-by-step procedures for common scenarios: ArgoCD down, sync failure, DR failover. A runbook guides an operator who might be on-call for the first time.
• Architecture diagrams — the map of ArgoCD, clusters, repos, and data flow. As-code diagrams (e.g. Mermaid in the repo) are easier to maintain than static images.
• Onboarding guides — how new teams use the platform: creating applications, using templates, passing approval.
• Troubleshooting guides — episode 27 turned into a runbook: symptom → likely cause → commands.

Store all of it in a repo (e.g. under docs/) so documentation versioned alongside code and reviewable via PR.

#Closing

This episode put the organizational side in order: platform team structure with application team autonomy, governance models from centralized to autonomous, monorepo vs polyrepo repo strategy with shared config and template repos, environment management with parity and promotion pipelines, change management with approval, CAB, and an emergency path, and documentation for runbooks, diagrams, onboarding, and troubleshooting.

The points you should take with you:

• The platform team succeeds if it manages to remove itself from the critical path.
• Git as the approval mechanism (PR + review) beats a separate approval tool.
• What's promoted is the manifest, not the deploy result.
• A fast official emergency path is safer than a slow normal process.
• Documentation is infrastructure — maintain it like code.

The organization is ready. Now let's talk about the thing management always asks about. In the next episode 33 we discuss cost optimization — resource right-sizing, cluster optimization with spot instances and autoscaling, GitOps efficiency, and cost monitoring with cost allocation and anomaly detection. See you in episode 33!