Learn BASH Scripting - Production-Grade Automation Scripts: Practical Scenarios

#Introduction

In episode 25 we covered static analysis & automated testing — how ShellCheck and Bats-core keep script quality in check — and in this episode, the closing one of the 27 episodes (0–26) of the Learn BASH Scripting series, we do the last and most important thing: assemble all the material into real, production-grade scripts.

Over the last 25 episodes you've gathered small pieces: variables and quoting, control flow, functions and arrays, parameter expansion, error handling, heredocs, getopts, sed/awk, curl/jq, logging and color, up to ShellCheck and Bats. Each episode felt like learning one tool in a workshop. Now it's time to build the vehicle — combining all the tools into two machines genuinely used in the working world.

The two scenarios we'll build are not theory:

1. Automatic backup & retention — database dump, compression, remote upload, old file rotation, and notifications. This is the script that saves a company when disaster strikes.
2. Health check & auto-healing — checking a service, restarting the dead one, recording incidents, and reporting. This is the script that lets a system "heal itself" at midnight.

Both scripts are a combination of almost everything you've learned. Reading them line by line, you'll see how getopts, strict mode, logging functions, curl, jq, sed, and error handling work together — not as separate lesson topics, but as one coherent whole. After understanding both, we'll close this series with a journey recap and a final push for your automation career.

#Main Discussion

#Case Study 1: Automatic Backup & Retention

This need is found at almost every company with servers. A database must be dumped periodically, compressed, sent to a safe location (object storage), old files cleaned up so the disk doesn't fill, and the team notified if something fails. Let's build it.

First, define the architecture. The script will accept the flags -d (temporary database dump directory), -k (number of retention days), and -n (database name). Plus environment variables for credentials — the pattern we've emphasized since episode 23.

#!/usr/bin/env bash
set -euo pipefail
 
# --- Konfigurasi (dari environment, bukan hardcode) ---
DB_HOST="${DB_HOST:-localhost}"
DB_USER="${DB_USER:-backup_user}"
DB_PASS="${DB_PASS:?DB_PASS wajib diisi di environment}"
RCLONE_REMOTE="${RCLONE_REMOTE:?misal 's3:mybucket/backups'}"
TELEGRAM_BOT_TOKEN="${TELEGRAM_BOT_TOKEN:-}"
TELEGRAM_CHAT_ID="${TELEGRAM_CHAT_ID:-}"
 
# --- Logging (pola dari episode 24) ---
LOG_FILE="${LOG_FILE:-/var/log/backup/backup.log}"
mkdir -p "$(dirname "$LOG_FILE")"
log() {
    local level=$1; shift
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] [$level] $*" | tee -a "$LOG_FILE"
}
info()    { log INFO    "$@"; }
error()   { log ERROR   "$@"; }
 
notify() {
    [ -z "$TELEGRAM_BOT_TOKEN" ] && return 0
    curl -sf "https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage" \
        --data-urlencode "chat_id=$TELEGRAM_CHAT_ID" \
        --data-urlencode "text=$1" > /dev/null
}
 
usage() {
    echo "Penggunaan: $0 -d DIR -k HARI -n DB_NAME"
    echo "  -d   direktori kerja untuk dump (default: /var/backup)"
    echo "  -k   hari retensi backup lokal"
    echo "  -n   nama database"
    exit 1
}

Notice several design decisions in this part:

• set -euo pipefail — strict mode from episode 13. One failure in the pipeline (for example mysqldump fails but gzip "succeeds") stops the script immediately.
• :? on mandatory variables — the script refuses to run without credentials, with a clear message (episode 23).
• The notify function uses curl -sf and --data-urlencode — the pattern from episode 23; if no token is set, the notification is skipped without error.
• Logging is centralized in the log function — the pattern from episode 24.

Now on to the execution part — the core of getopts, dump, compression, upload, and rotation:

RETENTION_DAYS=7
WORK_DIR=/var/backup
 
while getopts "d:k:n:h" opt; do
    case "$opt" in
        d) WORK_DIR="$OPTARG" ;;
        k) RETENTION_DAYS="$OPTARG" ;;
        n) DB_NAME="$OPTARG" ;;
        h) usage ;;
        *) usage ;;
    esac
done
 
[ -z "${DB_NAME:-}" ] && usage
STAMP=$(date '+%Y%m%d-%H%M%S')
DUMP_FILE="$WORK_DIR/$DB_NAME-$STAMP.sql.gz"
 
info "Memulai backup database '$DB_NAME'"
if ! mysqldump -h "$DB_HOST" -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" | gzip > "$DUMP_FILE"; then
    error "Dump database gagal"
    notify "🚨 BACKUP GAGAL: $DB_NAME di $DB_HOST"
    exit 1
fi
info "Dump selesai: $(du -h "$DUMP_FILE" | cut -f1)"
 
if ! rclone copy "$DUMP_FILE" "$RCLONE_REMOTE" --log-file /dev/null; then
    error "Upload ke remote gagal"
    notify "🚨 UPLOAD GAGAL: $DUMP_FILE"
    exit 1
fi
info "Upload ke remote berhasil: $RCLONE_REMOTE"
 
find "$WORK_DIR" -name "$DB_NAME-*.sql.gz" -mtime +"$RETENTION_DAYS" -delete
info "Rotasi selesai: backup lebih dari $RETENTION_DAYS hari dihapus"
notify "✅ Backup OK: $DB_NAME → $RCLONE_REMOTE"

Discuss the important details:

• getopts from episode 21: this script can be used by other people, not just its owner. -h shows usage, unknown options call usage.
• The mysqldump | gzip > file pipeline — with set -o pipefail, a mysqldump failure is detected. Remember: without pipefail, the script could "succeed" with an empty zip file — that's the most dangerous production bug.
• if ! ... ; then ... exit 1; fi — every critical step has an explicit failure branch, and every failure gets notified. The script never fails silently.
• find ... -mtime +N -delete — rotation (the pattern from episode 22 of the Linux series). -delete removes files older than N days, keeping the disk from ever filling up.
• The Telegram notification at the end becomes a visible confirmation, not just a log.

#Case Study 2: Health Check & Auto-Healing

The second need that's always present on production servers: keeping services alive. Nginx, Docker containers, or apps can die at any time — due to crashes, out-of-memory, or deployment mishaps. A smart health check script will check, restore, record, and report.

#!/usr/bin/env bash
set -euo pipefail
 
SERVICE="${1:-nginx}"
CHECK_URL="${CHECK_URL:-http://localhost/health}"
MAX_RETRY="${MAX_RETRY:-3}"
LOG_FILE="${LOG_FILE:-/var/log/healthcheck/health.log}"
mkdir -p "$(dirname "$LOG_FILE")"
 
log() {
    local level=$1; shift
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] [$level] $*" | tee -a "$LOG_FILE"
}
 
is_service_down() {
    systemctl is-active --quiet "$SERVICE"
    [ $? -ne 0 ]
}
 
restart_service() {
    log WARN "Mencoba me-restart $SERVICE"
    if systemctl restart "$SERVICE"; then
        log INFO "Restart berhasil: $SERVICE"
    else
        log ERROR "Restart GAGAL untuk $SERVICE"
        return 1
    fi
}
 
incident_count=0
 
for i in $(seq 1 "$MAX_RETRY"); do
    if is_service_down; then
        log WARN "Service $SERVICE down (percobaan $i/$MAX_RETRY)"
        restart_service || { incident_count=$((incident_count + 1)); break; }
        sleep 5
        continue
    fi
    log INFO "Service $SERVICE sehat"
    exit 0
done
 
log ERROR "$SERVICE masih down setelah $MAX_RETRY percobaan"
if command -v curl > /dev/null; then
    curl -sf -X POST -H 'Content-type: application/json' \
        -d "{\"text\":\"🚨 $SERVICE down setelah restart berkali-kali di $(hostname)\"}" \
        "$SLACK_WEBHOOK_URL" > /dev/null || true
fi
exit 1

Let's dissect the important patterns:

• systemctl is-active --quiet — this command returns a non-zero status if the service is inactive; quiet suppresses output. Wrapping it in the is_service_down function keeps the main loop easy to read.
• The retry loop with MAX_RETRY — the script doesn't give up on the first failure; it tries a restart, waits, and checks again. Real auto-healing is measured persistence.
• Logging every incident — in health.log there's a complete history: when it went down, when it restarted, whether it succeeded. That's the black box needed for troubleshooting.
• Notifications only for persistent failures — after all attempts are exhausted. This avoids spam notifications that make people ignore alerts (the alert fatigue phenomenon).

#Checklist: The Minimum Standard for Production-Grade Scripts

Here's a summary of the discipline you've built throughout this series — make it a checklist before any script touches a production server:

If your script meets every row of this table, it deserves admiration — not because it's complex, but because every part has a reason and works together seamlessly.

#Journey Recap: Phase 0 to 26

Your journey from episode 0 to this episode isn't just a list of topics — it's a growth curve. Let's see how far you've come.

Phase 1 — Foundation (episodes 0–5): you started from zero: prerequisites and environment setup, shell history, variables & environment variables, quoting & word splitting, globbing, and arithmetic. This is where you learned to speak the BASH language — understanding why "$var" differs from $var, and why globbing is simultaneously the best feature and a hidden danger.

Phase 2 — Program Structure (episodes 6–11): you learned to think like a programmer: conditionals & regex, loops, functions, indexed & associative arrays, parameter expansion, and command/process substitution with heredocs. Your scripts evolved from sequences of commands into programs with branching logic and data structures.

Phase 3 — Resilience (episodes 12–17): you learned to secure code: error handling & strict mode, traps & signals, debugging, up to professional CLIs with getopts. Here your scripts began daring to face failure — and became more reliable precisely because of it.

Phase 4 — Integration & Quality (episodes 18–25): you learned to connect and verify: text processing with sed/awk, interacting with the outside world through curl/jq and notifications, logging & colorizing output, and quality assurance with ShellCheck and Bats-core. Your scripts now talk to APIs, keep their own records, and are tested automatically.

Phase 5 — Production (episode 26, now): you learned to assemble everything into real systems. The two case studies above aren't just examples — they're blueprints of scripts that genuinely run on production servers all over the world.

Notice one thing: each phase is built on the previous one. You can't build auto-healing without understanding strict mode; you can't process JSON without understanding pipelines and quoting. That's why this gradual journey matters — every episode is a brick, and now you stand atop a complete building.

#Conclusion

This is the end of the 27-episode Learn BASH Scripting journey. Let's reflect on what you've accomplished.

You started from the most basic question — "what is a shell and how do I set it up?" — and now you can write scripts that dump a database, compress it, upload it to object storage, rotate old files, and send a Telegram notification if anything fails. You can build systems that check their own health and recover themselves when a service dies. From a user who types commands, you've become someone who builds commands — and builds commands that other people rely on.

This skill isn't merely technical; it's an operational superpower. In a world where servers keep multiplying and teams keep shrinking, the ability to automate is what separates the admin perpetually buried in manual work from the engineer who delegates repetitive tasks to machines. Every script you write is one human hour no longer wasted on work that could be automated. In a few years, you'll look back at these episodes and realize that the foundations you built here — quoting discipline, error handling, logging, testing — are the very same things keeping your production systems standing.

If there's one thing to carry away from this series, it's this: scripting is about trust. Trust that a script will run when needed, trust that it will stop safely when it fails, and trust that it can be understood by others — including yourself six months from now. Build that trust by following the checklist in this episode, and your scripts will survive the test of time.

What's next? The world of automation is far wider than BASH. You now have the right foundation to leap into configuration management like Ansible, infrastructure as code like Terraform, and even container orchestration with Docker and Kubernetes — all of those tools ultimately call a shell behind the scenes. And if one day you face a strange problem on a server, you'll find that a good BASH script is the first tool you reach for.

Thank you for accompanying this series to the end. Now close the textbook, open the terminal, and start automating your world — one tidy script at a time. Happy building, and see you on the next adventure.