Learn BASH Scripting - String Manipulation & Parameter Expansion

#Introduction

In episode 15 we covered Indexed & Associative Arrays — how to hold many values in a single variable and access them via numeric indices or string keys — so you now have a flexible storage container. But a question immediately arises once you start storing lots of data: how do you efficiently manipulate the contents of that container? In the world of BASH, the answer isn't reaching for sed, awk, or cut for every small task, but rather parameter expansion — a built-in shell mechanism that is far faster and more expressive than most people realize.

Many engineers have spent years writing basename "$file" or dirname "$file" even though BASH already provides a more concise way. Others call tr just to turn uppercase into lowercase, when ${var,,} exists. The problem isn't those external commands — it's that they never learned the native capabilities of the language they use every day. This episode will close that gap.

Think of parameter expansion like a fully equipped kitchen inside your own home. Before, every time you wanted to cook you had to go to the restaurant across the street (sed, cut, basename). Now you learn that the stove, knives, and blender were already in your kitchen — you just need to know which buttons to press. Faster, no waiting in line, and independent of whether that restaurant is open or closed (whether the external command exists or not).

We'll dissect: measuring string length with ${#str}, preparing default values with ${var:-...} and ${var:=...}, slicing substrings with ${str:pos:len}, stripping prefixes and suffixes with ${str#...}, ${str##...}, ${str%...}, ${str%%...}, and searching and replacing text with ${str/search/replace}. At the end of the episode, you'll work on the practices most commonly encountered in the DevOps world: extracting a filename without its extension, extracting the extension, and normalizing paths.

#Main Discussion

#Measuring String Length: ${#str}

The simplest yet most frequently needed syntax is counting the length of a string. The # operator inside curly braces means "count how many characters", not "comment":

nama="devvnull"
echo "${#nama}"

8

Why is this useful? Because data validation in the real world almost always involves length: rejecting user input that's too short, truncating logs so they don't exceed a limit, or checking whether a hash code has the right shape. An API key expected to be 40 characters long can be flagged immediately if its length isn't 40.

What's even more interesting: the # operator also works on arrays. Remember from episode 15 we discussed ${#arr[@]} — that's actually an application of the same operator, just against the number of elements rather than the length of text:

One small trap: the length is counted in characters, and multibyte characters (for example é or non-ASCII letters) can be counted as a single character by modern BASH. For scripts dealing with multibyte text, make sure your locale is UTF-8 so the results make sense.

#Default Values: Preparing a Fallback Plan

Now we get to one of the most-used patterns in production scripts: variables that might be empty. Imagine a backup script that receives its destination directory from the environment variable BACKUP_DIR. If the user forgets to set it, the script must not write directly to an empty path and delete the wrong thing — it needs a fallback value.

BASH provides four basic forms that all resemble a "fallback plan" for empty or unset variables:

The most important difference you must memorize is :- vs :=. :- is like borrowing a spare bicycle when your main one is lost — the main one stays lost. := is like buying a spare bicycle and parking it in the garage so you don't have to look for one next time. Notice:

BACKUP_DIR=""
echo "1: ${BACKUP_DIR:-/var/backups}"    # mencetak /var/backups
echo "2: ${BACKUP_DIR}"                   # masih kosong!
BACKUP_DIR=""
echo "3: ${BACKUP_DIR:=/var/backups}"    # mencetak /var/backups
echo "4: ${BACKUP_DIR}"                   # sekarang /var/backups

1: /var/backups
2:
3: /var/backups
4: /var/backups

Notice the empty line 2: the variable stays empty after :- because we only "borrowed" the default value without storing it. This is often the source of subtle bugs — you think the variable is set when it actually isn't.

Meanwhile :? is the most assertive form: it halts the script with an error message you write yourself. It's a concise input validation tool:

BACKUP_DIR="${BACKUP_DIR:?Variabel BACKUP_DIR wajib disetel!}"

If BACKUP_DIR is empty, the line above prints a message to stderr and stops the script with exit code 1. Even without setting set -u, this pattern protects the script from empty variables corrupting the logic.

#Slicing Substrings: ${str:pos:len}

Other programming languages call this substring or slice. In BASH, the form is ${str:position:length} — slice starting from position (index 0 = first character) for len characters:

versi="v1.4.2-beta"
echo "${versi:1}"        # 1.4.2-beta  (buang huruf v pertama)
echo "${versi:1:3}"      # 1.4        (mulai indeks 1, ambil 3 karakter)
echo "${versi:6:2}"      # 2-         (indeks 6, dua karakter)
echo "${versi: -4}"      # beta       (4 karakter terakhir)

1.4.2-beta
1.4
2-
beta

Pay attention to the last line: ${versi: -4} — with a space before the minus. That space is mandatory because ${versi:-4} would be read as the default-value operator (:-) we discussed earlier. This is one of the classic stumbling points that separates a careful BASH programmer from one who just wings it.

This feature is very useful when handling structured formats: trimming characters from a commit hash (${commit:0:7} for a short hash), extracting the area code from a phone number, or pulling the date part out of a YYYY-MM-DD string without involving cut -d-.

#Peeling the Onion: Removing Prefixes & Suffixes

This is the feature most often "rediscovered" by BASH programmers — even though it's been around for a long time. BASH can strip the beginning (prefix) and end (suffix) of a string based on a pattern. There are four operators, and the difference lies in how long a match the pattern makes:

A helpful mnemonic: # is always on the "left" or at the "beginning" (on a keyboard layout # looks like a leading character), while % is at the "end". A double # or double % means "greedy" — matching the pattern as long as possible.

The most famous use case is extracting filenames and extensions. If you have a full file path, these two lines replace basename and dirname at the same time:

file="/var/log/nginx/access.log"
echo "${file##*/}"      # access.log      — buang prefix sampai slash terakhir
echo "${file%.*}"       # /var/log/nginx/access — buang suffix terpendek (ekstensi)
echo "${file##*.}"      # log             — ekstensi
echo "${file%%.*}"      # /var/log/nginx/access — buang suffix terpanjang

access.log
/var/log/nginx/access
log
/var/log/nginx/access

Let's dissect the first line: ${file##*/} means "remove the longest */ pattern". Because * matches any characters, */ matches all the slashes together with their contents up to the last slash — the result is just the filename. Meanwhile ${file%.*} means "remove the shortest .* pattern" — that is, the last dot along with what follows, leaving the path without its extension. These two lines alone make your script faster and more portable than depending on the basename binary.

#Search & Replace: ${str/search/replace}

The fourth operator is text search and replacement — on par with sed for simple cases, but without spawning an external process. There are several variants:

kata="error error warning error"
echo "${kata/error/warn}"     # warn error warning error  (pertama saja)
echo "${kata//error/warn}"    # warn warn warning warn     (semua)
echo "${kata/#error/OK}"      # OK error warning error     (awal saja)
echo "${kata/%error/DONE}"    # error error warning DONE   (akhir saja)

Then there are two "big siblings" that are very useful for normalizing data: case modification. ${str^^} uppercases everything, ${str,,} lowercases everything — work you'd normally hand to tr or awk:

env_mode="production"
echo "${env_mode^^}"          # PRODUCTION
echo "${env_mode,,}"          # production
echo "${env_mode^}"           # Production      (huruf pertama saja)
echo "${env_mode,,}"          # production      (sama dengan sebelumnya)

Use ${env_mode,,} to normalize user input that inconsistently mixes uppercase and lowercase before comparing it against a list of valid values — far safer than relying on users typing perfectly.

#Real Practice: Filename Extraction & Path Normalization

Now let's combine everything into a scenario that genuinely happens in DevOps work. Scenario: you run log rotation and need to move old log files into an archive directory with normalized names — all lowercase, extension kept, no path:

#!/bin/bash
 
LOG_DIR="/var/log/nginx"
 
for file in "$LOG_DIR"/*.log; do
    filename="${file##*/}"          # buang path → nama file saja
    base="${filename%.*}"           # buang ekstensi
    ext="${filename##*.}"           # ambil ekstensi
    lower_base="${base,,}"          # nama jadi huruf kecil semua
 
    new_name="${lower_base}-$(date +%Y%m%d).${ext}"
    echo "Arsip: $filename → $new_name"
    # cp "$file" "/backup/$new_name"
done

Arsip: Access.Log → access-20260802.log
Arsip: error.log → error-20260802.log

Notice the ${base,,} line: without the case-modification operator, you'd have to call tr '[:upper:]' '[:lower:]' — an external command that spawns a new process for every file. With the built-in operator, BASH just does string math inside its own process. For a thousand log files, the time difference is real.

In addition, this pattern is also used for path normalization: replacing any possible separators, or removing ./ and ../ before comparing paths. With a combination of ${path#./}, ${path%/}, and ${path//\/\//} you can produce a clean, deterministic path — useful when your script receives input from users who type paths in different styles.

#Classic Pitfalls

1. Patterns are globs, not regex. This is pitfall number one. ${str##*/} works because * in glob matches any characters. But if you're used to regex and write ${str##\.*/}, the result won't be what you imagine — in glob, \. means backslash followed by a dot, not "literal dot". If you truly need regex-based replacement, use sed or awk. Don't force parameter expansion beyond its abilities.

2. Forgetting curly braces. ${file}.bak and $file.bak produce different output! Without braces, $file.bak is read as a variable named file.bak (which is usually empty). The golden rule: always use braces when the character after the variable isn't a space or a clear separator. echo "backup $file .bak" might save you by luck, but that's a fragile style.

3. :- vs :=. Using :- when you need := makes the default apply only once, and the variable stays empty for subsequent uses — a bug that looks random but is actually deterministic. Decide upfront: do you only need to use the fallback value, or store the fallback value for the script.

4. Space before the minus in substrings. ${str: -4} (with a space) means "the last 4 characters", while ${str:-4} without the space means "use 4 as the default". A single space completely changes the meaning. This is exactly the bug that most often slips through code review.

5. Expansion inside double quotes. Does ${var:-default} inside "..." still expand wildcards? No — quite the opposite: with double quotes, the expansion result is not split or globbed. Always wrap expansions in double quotes unless you intentionally want word splitting. This is the lesson from episode 4 (quoting) now truly paying off.

#Conclusion

In this episode 16 you've completed your string toolkit with power you may have been borrowing from external commands: measuring length with ${#str}, preparing default values with ${var:-default}, ${var:=default}, and ${var:?message}, slicing substrings with ${str:pos:len}, stripping prefixes and suffixes with ${str#...}, ${str##...}, ${str%...}, ${str%%...}, plus search and replace with ${str/search/replace} and case modification ${str^^}/${str,,}.

The key takeaways:

• Parameter expansion works without starting an external process — faster, more portable, and expressive for day-to-day string manipulation.
• # means "remove from the beginning", % means "remove from the end"; one mark for the shortest pattern, two marks for the longest.
• The patterns used are globs, not regex — don't mix mentalities.
• :- borrows a default value, := stores a default value; choose according to need.
• Always wrap expansions in double quotes and always use curly braces.

Armed with this, you no longer "run to the restaurant" for every small string-cooking task. In episode 17 we'll fire up a bigger stove: Command Substitution, Process Substitution & HereDoc. We'll learn to capture a command's output into a variable (output=$(command)), compare two directories without temporary files via diff <(ls dir1) <(ls dir2), and build clean configuration files directly from inside the script with here-documents. This is the foundation for scripts that don't just manipulate strings, but take data from the outside world and produce ready-to-use artifacts. See you in episode 17!