Learn Helm Chart - Performance Optimization & Large Scale

#Introduction

After episode 27, where we discussed chart migration and upgrades — from the Helm 2 to Helm 3 migration, major-version upgrades, to facing API deprecations — in this episode we shift from correctness to speed and resilience. Systems that run correctly at small scale can become slow, fragile, and unmanageable at large scale. Helm is not immune to this problem.

Why is this topic important? Consider a real production cluster: not one or two releases, but hundreds of releases — every team's application, every environment, every microservice. At that scale, a few extra seconds per rendering multiplied by hundreds of releases becomes hours of wasted work on every mass deployment. A 50 MB chart full of unused files slows down downloads on every cluster. Hundreds of release state secrets pile up in etcd. And when one of those hundreds of releases fails silently, without proper monitoring you won't know until users report it.

In this episode we dissect optimization across four layers: templates, chart size, installation performance, and large-scale management — then close with Helm monitoring that makes every critical release visible, measurable, and alertable.

#Template Optimization

Template rendering is the heart of every Helm operation. Inefficient templates mean every install, upgrade, template, and lint pays the price.

#Efficient Helpers

Named templates (episode 10) reduce duplication, but there's a hidden cost: every include/template call re-executes that template. Helpers called inside a range loop — for example, {{- include "mychart.labels" . }} inside an iteration of 20 services — get executed 20 times. The solution: cache the include result in a variable outside the loop, then use that variable inside the loop:

{{- $labels := include "mychart.labels" . }}
{{- range .Values.services }}
apiVersion: v1
kind: Service
metadata:
  name: {{ .name }}
  labels:
    {{- $labels | nindent 4 }}
{{- end }}

The difference is invisible in a small chart, but it's very real in a chart with dozens of resources and subcharts.

#Minimizing Lookup

The lookup function (episode 9) makes a real call to the Kubernetes API server. It's useful — for example, to fetch the value of an existing Secret — but it's expensive: every call is a network round-trip. If lookup is called inside a range, the cost multiplies. The golden rule: call lookup once, store it in a variable at the top of the template, then use the variable:

{{- $existing := lookup "v1" "Secret" .Release.Namespace (printf "%s-db" .Release.Name) }}
{{- if $existing }}
# nilai dari secret yang sudah ada dipakai, jangan buat ulang
{{- else }}
# secret baru dibuat
{{- end }}

Also, avoid lookup in charts that are frequently rendered in CI (for example helm template) — in a cluster-less environment, lookup always returns empty, which can change the render output in unexpected ways.

#Reducing Complexity

The most efficient template is the one that never needs to be executed. Practical principles:

• Separate optional resources into their own template file with an if at the top of the file, so when a feature is off (enabled: false), that whole block isn't rendered at all.
• Avoid unnecessary chained pipelines — every | is a function that gets executed.
• Simplify the values structure — deeply nested dicts make accessing .Values.foo.bar.baz.qux long and error-prone; flatten what can be flattened.
• Measure. Use time around the rendering or compare the helm template duration before/after a refactor. Optimization without measurement is just a guess.

#Chart Size Optimization

A chart is a package that gets downloaded, stored, and rendered. The leaner it is, the faster it is.

#Remove Unneeded Files

Many charts store artifacts that aren't needed during rendering: screenshots, demo videos, internal documentation, backups, or build output. All of these get wrapped up in the .tgz. helm package wraps the entire chart directory, so make sure only the files that are truly needed live there.

#Using .helmignore

.helmignore is the .gitignore analog for Helm — it defines the files that are not included in the package. This file is mandatory for a production-grade chart. A practical example:

# Git & CI artifacts
.git
.gitignore
.github
.venv
 
# Artefak lokal & dokumentasi berat
*.md
docs/
screenshots/
*.bak
*.tmp
*.log
*.orig
 
# Hasil package/tes
*.tgz
charts/*.tgz
tests/
coverage/

An important note: .helmignore affects packaging and rendering, but it doesn't remove files from your working directory. And never ignore files that templates genuinely need — for example files read by .Files.Get — because the chart will fail to render on the user's side.

#Dependency Pruning

Every subchart in charts/ gets carried along. Check whether all dependencies are really used: helm dependency list shows the status of each one. Remove unused dependencies, and consider whether some subcharts can be replaced by a library chart (episode 19), which is far leaner. For charts using OCI, a single chart can pull many layers — choose the base chart that fits your needs, not the most "complete" one.

#Reducing Archive Size

The combination of .helmignore + dependency pruning often results in drastic reductions. How to measure it:

helm package ./my-chart
ls -lh my-chart-1.2.3.tgz
 
# Bandingkan setelah optimasi
helm package ./my-chart
ls -lh my-chart-1.2.3.tgz

A healthy chart is usually only a few tens of kilobytes; a "fat" chart can reach hundreds of kilobytes to megabytes — and that gets multiplied by the number of clusters downloading it.

#Installation Performance

Install/upgrade speed is affected by rendering time plus time waiting for resources.

#Parallel Resource Creation

Helm sends manifests to the API server sequentially in one operation. To speed up many mutually independent resources, Helm 3 already sends hook-independent resources in parallel with a certain number of workers. You can't add parallelism directly from the CLI, but you can affect how many can run in parallel by not placing resources that wait on each other behind unnecessary dependencies. If an upgrade feels slow because it's waiting on resources, check which part is actually stuck (see kubectl get events and the status of each resource).

#Tuning --timeout

--timeout determines how long Helm waits before declaring an operation failed. The default of 5 minutes is often too short for charts with heavy migration hooks. A healthy rule:

# Chart biasa
helm upgrade --install api ./api-chart --timeout 10m
 
# Chart dengan hook migration database yang berat
helm upgrade --install api ./api-chart --timeout 30m

Don't use a very large --timeout carelessly — it only extends patience, it doesn't fix the problem. If an upgrade always takes 30 minutes, there's a resource that is genuinely slow; find and fix it.

#The --wait Strategy

--wait makes Helm wait for resources to be ready (Pod Running, resources complete, hooks succeed) before the operation is considered finished. Useful, but it makes every upgrade hang on full readiness — if one Pod is never ready, the whole upgrade hangs until timeout. Consider: use --wait for critical workloads, skip it for workloads that aren't sensitive (let the controller chase readiness), or combine it with --atomic in non-production environments so failures automatically roll back.

#Resource Readiness

The root of a "slow upgrade" is almost always readiness. Deployments waiting on a large image pull, readinessProbes that are too strict, or long minReadySeconds — all of them add time until --wait finishes. Optimizing on the application side (realistic readiness probes, lean images, a startupProbe for applications that need a long initialization) has a bigger impact than tuning Helm flags.

#Large-Scale Deployment

Scale brings organizational problems that don't exist at small scale.

#Many Releases per Cluster

With hundreds of releases, operations like helm list --all-namespaces and large renders become expensive. Practices that help:

• Separate by env and by team into clear namespaces, and get into the habit of filtering: helm list -n api -q.
• Limit release history — helm upgrade --history-max 10 limits the number of stored state secrets per release (default 10; raise it if you need a long audit trail, lower it if storage becomes a problem).
• Don't pile everything into the default namespace — that makes ownership, RBAC, and auditing unclear.

#Namespace Organization

The namespace is a unit of isolation as well as a unit of organization. Use the pattern: one application/team per namespace (api, payment, analytics), with dedicated platform namespaces (argocd, flux-system, monitoring, ingress-nginx). Namespaces enable per-team resource quotas and NetworkPolicies — two mechanisms that are very useful at large scale.

#Resource Management

Hundreds of releases mean hundreds of Deployments competing for resources. Set realistic requests/limits in the chart (never let a release run unbounded), and enforce them with a ResourceQuota per namespace so one "greedy" team can't hurt other teams. This is also the chance to apply the per-environment values pattern we covered in episode 21.

#Storage Considerations

Every release revision is stored as a Secret — and by default never deleted automatically. With a large --history-max and many releases, etcd can fill up with sh.helm.release.v1.* secrets. Note two things: (1) set a reasonable --history-max (10–20 is enough for most cases, raise it only if long-term auditing is needed); (2) on uninstall, history secrets are deleted too unless --keep-history — don't make --keep-history a habit without a reason.

#Monitoring Helm

At large scale, you can't wait for user reports. Releases must be visible and measurable.

#helm-exporter for Prometheus

helm-exporter is an exporter that reads the status of all releases from release secrets and exposes them as Prometheus metrics. Its main metrics:

• helm_release_info — release information (name, namespace, chart, app version, status).
• helm_release_status — release status (deployed, failed, pending-*).
• helm_release_updated_timestamp — when the release was last updated.

Its deployment is simple — here's a ready-to-use example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helm-exporter
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helm-exporter
  template:
    metadata:
      labels:
        app: helm-exporter
    spec:
      serviceAccountName: helm-exporter
      containers:
        - name: helm-exporter
          image: sstarcher/helm-exporter:0.4.11
          args:
            - --config=/tmp/helm-exporter.yaml
          volumeMounts:
            - name: config
              mountPath: /tmp
          ports:
            - containerPort: 9571
              name: metrics
      volumes:
        - name: config
          configMap:
            name: helm-exporter

Because it must read release secrets in all namespaces, helm-exporter needs RBAC get/list/watch on secrets (with a owner=helm label filter) across all namespaces — minimal, not excessive.

#Prometheus Scrape Config

To have Prometheus scrape metrics from helm-exporter, add a Service and a scrape config:

- job_name: helm-exporter
  kubernetes_sd_configs:
    - role: pod
      namespaces:
        names: [monitoring]
  relabel_configs:
    - source_labels: [__meta_kubernetes_pod_label_app]
      action: keep
      regex: helm-exporter
    - source_labels: [__meta_kubernetes_pod_container_port_name]
      action: keep
      regex: metrics

#Grafana Dashboard and Alerting

With those metrics, a Grafana dashboard can show all releases per namespace with their status and chart version. The most valuable alerting rules:

• Releases with failed status — helm_release_status{status="failed"} > 0 should be alerted immediately; it's the earliest signal that something is broken.
• Releases stuck in pending-install/pending-upgrade — a hanging status transition is a sign of an operation that failed halfway.
• Outdated chart versions — helm_release_info makes it possible to compare chart versions with the latest in the repo; prolonged version drift signals a risky upgrade backlog.

This alerting has the biggest impact in large organizations, because the failure of one release among hundreds of others is nearly impossible to find manually.

#Detecting Version Drift

Besides status, what also needs monitoring is configuration drift: releases that should use values X but were manually updated with an unrecorded --set. In a GitOps environment, this drift is a violation of the source of truth. Approaches: compare the actual helm get values with the values in Git (can be automated with a script that generates metrics), or rely on the ArgoCD/Flux self-heal we covered in episode 25 to correct it automatically.

#Conclusion

In this episode 28 we understood that Helm optimization works in layers: efficient templates — by caching include results, minimizing lookup, and streamlining structure; lean charts — via .helmignore, dependency pruning, and measuring archive size; installation performance — by tuning --timeout, --wait strategies, and fixing readiness on the application side; large-scale management — namespace organization, resource quotas, and --history-max control for storage; and monitoring — helm-exporter, Prometheus scrape config, Grafana dashboards, and alerting for failed, stuck, or outdated releases.

The core takeaways:

• Cache template & lookup results in variables — repeated rendering and API costs are hidden enemies.
• .helmignore + dependency pruning shrink charts drastically; measure with helm package.
• A large --timeout doesn't fix problems — it only extends patience; fix the actual readiness.
• Scale means discipline: organized namespaces, per-team quotas, controlled --history-max, no piling up of state secrets.
• Without monitoring, hundreds of releases are a black box — set up helm-exporter and alerting from the start.

All the techniques you've learned from episode 0 through 28 finally converge on one point: building professional systems in an enterprise environment. In episode 29, the final episode, we cover enterprise patterns and production case studies: corporate chart standardization, governance, multi-tenancy, private chart repositories, and an end-to-end case study building a complete chart for a real application. See you in episode 29!