Learn Curl - Configuration, Config File & Environment

#Introduction

In episode 7, you mastered file transfer — from downloads with resume to uploads to FTP and SFTP. In this episode, we'll tidy everything up. Imagine having to write -u user:pass plus --progress-bar plus --connect-timeout 10 on every command, repeatedly, every day. Sooner or later something gets forgotten or mistyped. The solution is configuration: putting default options in one place, then letting curl read them automatically.

There are two mechanisms we'll dissect: config files (for options you want to persist) and environment variables (for settings that depend on the environment, especially proxy). Both work behind the scenes — once understood, you'll see curl change from a finicky tool into a tool that's "pre-configured to your taste".

#Config Files: ~/.curlrc

curl reads the config file before processing command-line arguments. The default file is ~/.curlrc — applying to all curl sessions for the user. The format is simple: one option per line, written without a leading dash. Comments start with #.

# Default options for all curl requests
user-agent = "devvnull-bot/1.0"
connect-timeout = 10
max-time = 60
silent
show-error
follow-redirects
retry = 3
retry-delay = 2
# Don't save any output beyond what's needed
location-trusted

Each line is an option exactly identical to its command-line flag, just without the double dash. silent is equivalent to -s, follow-redirects is equivalent to -L. Once this file exists, every curl command in your terminal automatically uses a 10-second timeout and retries up to three times — without you typing them again. That's the main power of a config file: consistent defaults across all sessions.

The priority is layered and important to remember:

1. Options on the command line — always win.
2. Options in the config file — used if not overridden.
3. curl's built-in defaults — used if neither is present.

That means writing --connect-timeout 30 on the command line overrides connect-timeout = 10 in ~/.curlrc for that request only. The config file isn't a prison — it's just a safety net.

#Per-Project Configuration: --config

For configurations that differ between projects or environments, don't clutter ~/.curlrc — use the --config option to load a specific file. This resembles --env-file on modern tools: one file for staging, another for production, and which file is used is decided when running the command.

# Configuration specific to accessing the production environment
user-agent = "devvnull-cron/1.0"
connect-timeout = 5
max-time = 30
retry = 2
header = "Accept: application/json"
header = "X-Client: deploy-bot"
# Include credentials from the environment
user = "budi"

curl --config .curlrc-prod https://api.example.com/health

Notice two things. First, the header option can be repeated — each line adds one header, so you can send several headers at once. Second, you can combine --config with command-line flags; command-line options still win. For projects that need different authentication, the "one config file per environment" pattern is far cleaner than writing -u in every script.

#Proxy Environment Variables

One reason people are sometimes "unaware" that curl runs through a proxy is because curl reads environment variables automatically — without any flag. The recognized variables:

• http_proxy — proxy for all URLs with the http scheme.
• https_proxy — proxy for URLs with the https scheme.
• all_proxy — proxy for all protocols, used when the scheme-specific variable is absent.
• no_proxy — list of hosts that must be bypassed without a proxy.

export http_proxy="http://proxy.corp:8080"
export https_proxy="http://proxy.corp:8080"
export no_proxy="localhost,127.0.0.1,*.internal.example.com"
 
curl -s https://api.example.com/users

Curl reads the variables in both lowercase and uppercase (for example HTTP_PROXY), giving priority to lowercase. This means exporting https_proxy once in ~/.bashrc is enough to route all curl on your laptop through the corporate proxy — without touching a single command.

To disable or narrow the proxy for a specific request, use --noproxy:

curl --noproxy "*.internal.example.com" -s \
  https://internal.example.com/api/health

The value --noproxy "*" disables all proxies for that request, overriding all proxy environment variables at once — useful when you know the destination host must be reached directly.

#Disabling Config Files: -q

Sometimes ~/.curlrc becomes the problem — for example when mimicking plain curl behavior on another machine, or when default options in the config file make requests behave oddly and you want to know why. For that there's -q (alias --disable):

curl -q https://api.example.com/health

With -q, curl doesn't read any config file at all — neither ~/.curlrc nor a file referenced by --config — and returns to built-in defaults. This is a very useful debugging tool: if a request behaves oddly, run it once with -q. If behavior returns to normal after -q, the cause is definitely in one of the config files; if it's still odd, the problem is in your own command line.

#Closing

Episode 8 equips you with two layers of configuration: config files for persistent options — ~/.curlrc for global defaults and --config for per-project or per-environment configuration — as well as environment variables for settings that depend on the environment, especially http_proxy, https_proxy, all_proxy, and no_proxy with the --noproxy override. Don't forget -q to disable config files when debugging.

What to remember: config files are where options live permanently, the command line is where overrides live. The cleaner you arrange both, the fewer flags you have to memorize every day.

In the next episode 9, we'll go deeper into proxy and networking — how to choose an HTTP, HTTPS, or SOCKS5 proxy, handle proxy authentication, manipulate DNS with --resolve, and troubleshoot connections. See you!