GitOps Multi-Stage Promotion with Kargo — Automating dev to prod with Argo CD Integration
You thought you had Argo CD figured out, and then your team lead asks: "When does the image that went up to dev get into staging?" You're stumped. There's a Jenkins job, but someone has to trigger it manually, and you remember that time you forgot and staging was running a stale version for three days. You adopted GitOps, yet promotion between environments was still a manual process. I was in that exact same situation when I first looked into Kargo, and the first thing I had to understand was whether it overlapped with Argo CD's role.
This post covers Kargo's core concepts — the Warehouse, Freight, Stage, and PromotionTask model — and how to declaratively design an automatic promotion pipeline from dev → staging → prod. By the end, you'll be ready to draft a working 3-stage pipeline by adding Kargo to your existing Argo CD environment. This post targets backend and DevOps engineers who have hands-on experience operating Kubernetes and Argo CD. If Kustomize, Helm, and RBAC are unfamiliar concepts, it's worth skimming the Argo CD official documentation first.
Kargo is a Kubernetes-native promotion orchestration platform open-sourced by Akuity — the company behind Argo CD — in 2023. It hit v1.0 GA in 2024, officially supporting production use, and has rapidly evolved to v1.10 as of 2025. If Argo CD's role is "reconcile what's in Git with the cluster," Kargo's role is "decide what version goes to which environment and when." The two tools are not competitors — they operate at different layers.
Core Concepts
The Missing Piece of GitOps — The Promotion Layer
In a traditional GitOps workflow, Argo CD watches a Git repository and synchronizes cluster state. But who handles "reflecting the image tag from the dev environment into the staging environment's manifest"? Typically it's a CI script committing directly to Git, a team member opening a PR, or at worst, a Slack message asking "is it okay to push to staging?" Kargo fills this gap by introducing three core resources.
| Resource | Role |
|---|---|
| Warehouse | Subscribes to container image registries, Git repositories, and Helm chart repositories to detect new versions |
| Freight | The atomic unit of promotion. A bundle of a specific image tag + Git commit SHA + Helm version |
| Stage | Represents an environment (dev/staging/prod) and connects in a DAG structure to form a pipeline |
What is Freight? It's not just an image tag. It's an immutable deployment unit that bundles a container image, Git commit SHA, and Helm chart version together. "Promotion" is the act of this Freight moving to higher environments along the Stage chain.
Warehouse — The Sentinel That Detects New Versions First
The Warehouse continuously polls sources and automatically creates Freight whenever a new version is detected. When I first saw this YAML, I was confused about discoveryLimit — it's an option that limits the number of recent versions to track at once. Setting it to 10 means only the 10 most recent tags are managed as Freight candidates.
apiVersion: kargo.akuity.io/v1alpha1
kind: Warehouse
metadata:
name: my-app-warehouse
namespace: my-project
spec:
subscriptions:
- image:
repoURL: ghcr.io/my-org/my-app
tagSelectionStrategy: SemVer
allowTags: ^v\d+\.\d+\.\d+$
discoveryLimit: 10The allowTags regex lets you control which tag patterns are tracked, completely preventing feature branch images from accidentally mixing into the production pipeline.
Stage — A Pipeline That Connects Environments as a DAG
A Stage is not simply an "environment" — it also acts as a gate that only accepts Freight verified by the previous Stage. The staging Stage below only accepts Freight that has passed through the dev Stage.
apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
name: staging
namespace: my-project
spec:
requestedFreight:
- origin:
kind: Warehouse
name: my-app-warehouse
sources:
stages:
- devListing multiple Stages in sources.stages defaults to OR semantics — Freight is accepted if verified by any one of them. If you want to allow only Freight verified by all upstream Stages, you can use availabilityStrategy: AllMustBeVerified, introduced in v1.3. Not knowing this distinction when designing complex pipelines can lead to unintended promotions, so be careful.
PromotionTask — Modularization for Eliminating Repetition
If you have 3 Stages (dev, staging, prod), do you need to copy-paste the promotion step sequence three times? I did exactly that at first, and by the time I had 5 Stages, changing a single git-commit message format meant editing five different places. PromotionTask lets you define common promotion logic in one place and reuse it across multiple Stages — just like extracting a function in code.
apiVersion: kargo.akuity.io/v1alpha1
kind: PromotionTask
metadata:
name: standard-gitops-flow
namespace: my-project
spec:
vars:
- name: appName
- name: overlayPath
steps:
- uses: git-clone
config:
repoURL: https://github.com/my-org/gitops-config
branch: main
- uses: kustomize-set-image
config:
path: ${{ vars.overlayPath }}
images:
- image: ghcr.io/my-org/my-app
newTag: ${{ freight.images[0].tag }}
- uses: git-commit
config:
message: "chore: promote ${{ freight.images[0].tag }} to ${{ vars.appName }}"
- uses: git-push
- uses: argocd-update
config:
apps:
- name: ${{ vars.appName }}
sources:
- repoURL: https://github.com/my-org/gitops-config
- uses: argocd-wait
config:
apps:
- name: ${{ vars.appName }}By parameterizing only the values that differ per Stage (app name, overlay path) via vars, any change to the pipeline logic only requires updating the single PromotionTask to apply across all environments.
Practical Application
Example 1: Full dev → staging → prod Pipeline Setup
This example assumes a small team operating a single microservice (my-app). The GitOps config repository (gitops-config) has Kustomize overlays in an overlays/dev, overlays/staging, overlays/prod structure, with corresponding Argo CD apps (my-app-dev, my-app-staging, my-app-prod) already running. When CI builds and pushes an image, Kargo detects it and auto-deploys to dev; once dev verification passes, it auto-promotes to staging; and prod goes through manual approval.
Git credentials setup — this is often the first place people get stuck. To access a private repository, you must first create a Kubernetes Secret in the format Kargo recognizes.
apiVersion: v1
kind: Secret
metadata:
name: gitops-config-credentials
namespace: my-project
labels:
kargo.akuity.io/cred-type: git
stringData:
repoURL: https://github.com/my-org/gitops-config
username: git
password: ghp_YOUR_GITHUB_TOKEN # GitHub Personal Access TokenWithout this Secret, the git-clone step will throw an authentication error. Kargo automatically recognizes Secrets labeled kargo.akuity.io/cred-type: git and uses them to access the corresponding repository.
dev Stage — receives Freight directly from the Warehouse and references the PromotionTask defined earlier.
apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
name: dev
namespace: my-project
spec:
requestedFreight:
- origin:
kind: Warehouse
name: my-app-warehouse
sources:
direct: true
promotionTemplate:
spec:
steps:
- task:
name: standard-gitops-flow
vars:
- name: appName
value: my-app-dev
- name: overlayPath
value: overlays/devstaging Stage — only accepts Freight that has passed dev, and requires a 24-hour soak time before prod promotion is allowed.
What is Soak Time? A waiting period that requires a deployment to remain in an environment for a set duration before promotion to the next Stage is allowed. It's used to enforce rules like "no issues for 24+ hours in staging before allowing prod promotion."
apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
name: staging
namespace: my-project
spec:
requestedFreight:
- origin:
kind: Warehouse
name: my-app-warehouse
sources:
stages:
- dev
verification:
soak:
duration: 24h
promotionTemplate:
spec:
steps:
- task:
name: standard-gitops-flow
vars:
- name: appName
value: my-app-staging
- name: overlayPath
value: overlays/stagingprod Stage — must pass staging, and runs a smoke test using an Argo Rollouts AnalysisTemplate.
What is an AnalysisTemplate? A resource defined in Argo Rollouts that lets you define deployment success criteria as code, based on Prometheus metrics, HTTP endpoints, custom scripts, and more. Kargo's
verification.analysisTemplatesreferences this resource to automatically run verification after a deployment.
apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
name: prod
namespace: my-project
spec:
requestedFreight:
- origin:
kind: Warehouse
name: my-app-warehouse
sources:
stages:
- staging
verification:
analysisTemplates:
- name: smoke-test # Argo Rollouts AnalysisTemplate reference
promotionTemplate:
spec:
steps:
- task:
name: standard-gitops-flow
vars:
- name: appName
value: my-app-prod
- name: overlayPath
value: overlays/prodHow does manual approval work? You can mark approval on a Stage's Freight via the Kargo UI or CLI, or use RBAC (Role-Based Access Control) to grant only specific roles (e.g.,
release-manager) the permission to createpromotionresources. It's recommended to define Stage-level role bindings up front in the KargoProjectresource. In practice, incidents caused by allowing anyone on the team to promote to prod are quite common.
Example 2: Sending a Slack Notification on Deployment Failure with Conditional Steps
Conditional steps (the if: expression), introduced in Kargo v1.3, let you embed Slack webhook calls for promotion failures directly into the pipeline. Honestly, before this feature, we either relied on external monitoring or built separate notification pipelines — now it can be handled inside the Stage itself.
Since putting Slack webhook URLs in plain text YAML creates security issues, it's recommended to manage them as Kubernetes Secrets and inject them via environment variables or ${{ secrets.slackWebhook }}.
promotionTemplate:
spec:
steps:
- uses: git-clone
- uses: kustomize-set-image
- uses: git-commit
- uses: git-push
- uses: argocd-update
- uses: argocd-wait
as: deploy-wait # Name the step for reference in subsequent conditions
- uses: http
if: ${{ steps['deploy-wait'].status == 'Failure' }}
config:
method: POST
url: ${{ secrets.slackWebhookUrl }}
headers:
- name: Content-Type
value: application/json
body: |
{"text": "prod deployment failed: ${{ freight.images[0].tag }}"}
- uses: fail
if: ${{ steps['deploy-wait'].status == 'Failure' }}
config:
message: "Promotion aborted due to argocd-wait failure"| Step | if: Condition |
Role |
|---|---|---|
argocd-wait |
— | Collects Argo CD health check result (named with as: deploy-wait) |
http |
Previous step failed | Sends notification via Slack webhook |
fail |
Previous step failed | Explicitly marks the promotion as failed to make it trackable |
The pattern of naming a step with as: deploy-wait and referencing it via steps['deploy-wait'].status isn't well-highlighted in the official docs, but it's quite useful in practice.
Pros and Cons
Advantages
| Item | Details |
|---|---|
| Declarative pipeline | All promotion policies are version-controlled in Git as YAML, eliminating dependencies on CI scripts or manual work |
| Full traceability | Every record of which Freight was deployed to which environment, when, and with whose approval is captured |
| Atomic promotion | Bundles image + Helm chart + Git commit into a single Freight to guarantee consistency |
| Fine-grained RBAC | Roles can be separated per Stage — e.g., QA team approves staging, SRE team approves prod |
| Coexists with existing tools | Adds only an orchestration layer without replacing Argo CD or Argo Rollouts |
| Freight-level rollback | Instant rollback to a previous Freight on issues, enabling faster incident response |
Disadvantages and Caveats
Learning curve You need to learn new concepts — Warehouse, Freight, Stage, PromotionTask — all at once. Budget at least two days to get your head around the concepts.
- Recommendation: Work through the official Quick Start example hands-on to internalize the concepts
Operational overhead You need to run the Kargo controller separately in the cluster and understand state synchronization with Argo CD.
- Recommendation: Using Akuity Platform (a managed service) can reduce operational burden
Debugging complexity Failures can stem from many sources — conditional steps, verification failures, soak time expirations — making root cause identification difficult.
- Recommendation: Check both the Stage status panel in the Kargo UI and
kubectl describe promotionlogs together
Young ecosystem v1.0 GA was in 2024, so enterprise best practices are still accumulating.
- Recommendation: Supplement with real-world case studies like the JumpCloud Engineering Blog and the ProSiebenSat.1 case
Argo stack dependency The Argo CD + Argo Rollouts combination is effectively required for optimal use. If you're running a Flux-based environment, integration maturity is low — review carefully before proceeding.
Most Common Mistakes in Practice
- Creating Stages without a Warehouse — Without a source for Freight, the pipeline won't work. Always design the Warehouse → Freight → Stage flow first, and verify in the UI that the Warehouse is actually detecting images.
- Omitting
argocd-waitafterargocd-update— Triggering a sync without waiting for the health check means a promotion can be marked successful even when pods are in CrashLoop. This is exactly why the two steps were separated starting in v1.10. - Not configuring per-Stage RBAC from the start — Operating with every team member able to promote the prod Stage is a pattern that leads to incidents. Define Stage-level role bindings in the
Projectresource from day one.
Closing Thoughts
Kargo is the key tool bridging the era where GitOps only solved "synchronization" to the era where "promotion is also managed declaratively." The 48% increase in deployment frequency and 45% reduction in lead time that JumpCloud Engineering achieved after adoption aren't just numbers — they're the result of a paradigm shift that moved promotion decision-making from humans to the pipeline.
Three steps you can start right now:
- Install Kargo on a local cluster — After installing with the command below, follow the official Quick Start's
guestbookexample step by step to see the Warehouse → Freight → Stage flow with your own eyes.bashhelm install kargo oci://ghcr.io/akuity/kargo-charts/kargo \ --namespace kargo --create-namespace - Connect your simplest existing Argo CD app first — Pick one of your currently running Argo CD applications and start by attaching just a Warehouse that subscribes to the image registry and a single dev Stage. Validating incrementally rather than changing everything at once is much safer.
- Extract common steps into a PromotionTask — As soon as you have 2 or more Stages, introduce PromotionTask immediately to eliminate duplication. If you don't do it early, you'll eventually find yourself editing YAML across 10 Stages one by one.
Note that to make full use of verification.analysisTemplates on the prod Stage, it's worth running Argo Rollouts alongside Kargo. Argo Rollouts is an advanced deployment controller that supports canary and Blue-Green deployments on Kubernetes; integrating it with Kargo's verification features lets you automatically evaluate success criteria based on Prometheus metrics or HTTP endpoints after a deployment.
References
Introduction and Quick Start
- What is Kargo? Simplifying Continuous Promotion with GitOps | Akuity
- Kargo Official Documentation | docs.kargo.io
- Kargo Explained: How Warehouses, Freight, and Stages Replace Manual Promotions | Burrell Technology
- Continuous Promotion on Kubernetes with GitOps | Piotr's TechBlog
- From Commit to Production: GitHub Actions + Argo CD + Helm + Kargo | freeCodeCamp
Reference and Official Release Notes
- GitHub - akuity/kargo
- Kargo v1.3 Release Notes — Conditional Steps & Advanced Verification | Akuity Blog
- Kargo v1.10 Release Notes — Custom Steps, HTTP Notifications | Akuity Blog
- Implementing a Modular Kargo Promotion Workflow: PromotionTask | DEV Community
- Change Management with Pulumi Kubernetes Operator and Kargo | Pulumi Blog
Case Studies