Learning Cron Job - Distributed Scheduling: Advanced K8s CronJob

#Introduction

In episode 15 we got to know the basic CronJob: schedule, concurrencyPolicy, and TTL. Now we move up to distributed scheduling — managing dozens of CronJobs that can fail, be delayed, be suspended, and need parallel or chained execution. Kubernetes is a distributed scheduler: it doesn't depend on a single host daemon, tolerates node failures, and provides granular control. This episode dissects advanced CronJob features and their alternatives when your needs go beyond CronJob.

#Advanced CronJob Features

#startingDeadlineSeconds: The Late-Start Limit

If the controller fails to run a job on time (e.g. an overloaded cluster), startingDeadlineSeconds determines how much lateness can still be caught up on:

spec:
  schedule: "30 2 * * *"
  startingDeadlineSeconds: 1800

• Jobs delayed under 1800 seconds still run.
• Delays beyond that are skipped (missed) — not queued up.

#suspend: Pause Without Deleting

Jeda dan lanjutkan CronJob

kubectl patch cronjob backup -p '{"spec":{"suspend":true}}'
kubectl patch cronjob backup -p '{"spec":{"suspend":false}}'

#successfulJobsHistoryLimit: Tight Retention

Limit history so it doesn't pile up — in production, make it part of the manifest. backoffLimit controls how many times a pod is retried before the job is considered failed:

spec:
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      ttlSecondsAfterFinished: 3600
      backoffLimit: 4

#Job Parallelism: Running Many at Once

For jobs that can be split (e.g. processing 100 files), use parallelism:

jobTemplate:
  spec:
    parallelism: 10
    completions: 10
    template:
      spec:
        restartPolicy: OnFailure
        containers:
          - name: worker
            image: myorg/worker:1.0
            args: ["process-chunk", "--index=$(JOB_COMPLETION_INDEX)"]

JOB_COMPLETION_INDEX gives each pod an index from 0-9 — a work queue / fan-out pattern that's useful for batch processing.

#Production Usage Patterns

• An extract → transform → load pipeline, each stage with its own schedule (Argo for dependencies between stages).
• Idempotency (episode 9) still applies: rerun a failed stage without corrupting state.

#Leader Election: Only One Runner

Sometimes you want several app replicas, but only one of them runs periodic jobs. CronJob doesn't pick a leader — your app must use a Kubernetes lease (coordination.k8s.io) via --leader-election so only one instance works. This is the flock equivalent in the distributed world.

#Alternatives for Large Scale

#Argo Workflows: DAG Workflows

Argo Workflows runs workflows with dependencies between steps (DAG), per-step retry, and artifacts:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
spec:
  entrypoint: pipeline
  templates:
    - name: pipeline
      dag:
        tasks:
          - name: transform
            template: transform
            depends: extract
          - name: load
            template: load
            depends: transform

#KEDA Cron Scaler: Scaling on a Schedule

KEDA adds a Cron scaler that scales a deployment up and down following a schedule — for continuous workloads whose volume follows the time of day:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
spec:
  scaleTargetRef:
    name: processor
  triggers:
    - type: cron
      metadata:
        timezone: Asia/Jakarta
        start: "0 2 * * *"
        end: "0 6 * * *"
        desiredReplicas: "10"

#Closing

Key takeaways:

• startingDeadlineSeconds limits how much lateness is caught up on.
• suspend pauses scheduling without deleting the CronJob.
• successfulJobsHistoryLimit + ttlSecondsAfterFinished prevent pile-up.
• parallelism + JOB_COMPLETION_INDEX for batch/fan-out.
• Argo for DAG workflows; KEDA Cron scaler for schedule-based scaling.
• Leader election (lease) for a single runner across many replicas.

In episode 20 we'll cover job monitoring and observability — exporting job duration and status to Prometheus via the node_exporter textfile collector, detecting jobs that never appear with on-miss alerting, plus auditing who changed the crontab and periodic reviews!