Learn Curl - Proxy & Networking

#Introduction

In episode 8, you got to know proxy environment variables and the --noproxy override. Now we'll bring that knowledge down to real practice: how to choose the right proxy type, how to authenticate against it, and — often overlooked — how curl manipulates the networking layers underneath. These abilities make curl not only useful in the office, but also a scalpel when troubleshooting networks.

#Why Proxies Are Needed

A proxy is an intermediary between curl and the destination server. Imagine a security guard checking everything entering and leaving a building — a proxy does something similar for data packets. It's used for many reasons: corporate security policy, traffic monitoring, hiding the original address, or bypassing geographic restrictions. Understanding how it works means understanding why a request can "disappear" in the middle of a network.

#HTTP Proxy: -x

The -x option (or --proxy) accepts a proxy URL, and curl will route all requests through that proxy:

curl -x http://proxy.corp:8080 https://api.example.com/users

For http URLs, curl sends the full request to the proxy and the proxy forwards it. For https URLs, curl and the proxy establish an encrypted tunnel via the HTTP CONNECT mechanism — the proxy only sees the destination host and cannot read the encrypted data. This is an important distinction: an HTTP proxy can monitor regular http traffic, but is blind to https traffic.

#HTTPS and SOCKS5 Proxies

Besides the HTTP proxy, there are two other common types:

• HTTPS proxy — a proxy that communicates over a TLS connection. Use the https:// scheme in the proxy URL, for example -x https://proxy.corp:8443. Useful when the path to the proxy itself must be encrypted.
• SOCKS5 proxy — a generic proxy working at the TCP level, not HTTP-specific. Suitable for any protocol.

For SOCKS5, there are two variants that are easily confused:

curl --socks5-hostname proxy.corp:1080 https://api.example.com/users

curl --socks5 proxy.corp:1080 https://api.example.com/users

The difference is who performs DNS resolution. --socks5-hostname hands name resolution to the proxy — the domain name never becomes visible on your local network. Meanwhile, --socks5 resolves DNS locally, then sends the IP address to the proxy. For privacy and DNS filtering bypass, --socks5-hostname is almost always the right choice.

#Proxy Authentication: --proxy-user

Many corporate proxies ask for their own credentials, separate from the destination server's. The --proxy-user option handles this:

curl -x http://proxy.corp:8080 \
  --proxy-user "budi:sandirahasia" \
  https://api.example.com/users

Note that --proxy-user differs from -u. -u is used for authentication to the destination server, while --proxy-user is for authentication to the proxy. Mixing them up is a classic mistake that produces a 407 Proxy Authentication Required error even though the credentials are correct — just in the wrong place.

#Disabling Proxy per Host: --noproxy

Not all traffic must go through a proxy. Internal hosts — like image registries, metrics servers, or your team's internal APIs — are better accessed directly for speed and independence from proxy availability. The --noproxy option accepts a comma-separated list of host patterns:

curl -x http://proxy.corp:8080 --noproxy "*.internal.example.com,10.0.0.0/8" \
  https://internal.example.com/api/health

Patterns can be domain names, wildcards, or CIDR ranges. The value "*" disables the proxy for all hosts. The combination of -x for some hosts and --noproxy for others gives you granular control: outbound traffic goes through the proxy, internal traffic stays direct.

#Advanced Networking

#DNS Override: --resolve

Sometimes you need to force curl to connect to a specific address even though public DNS points elsewhere — for example when testing before a DNS cutover, or when a staging server is only reachable through a specific IP.

curl --resolve api.example.com:443:10.0.0.5 \
  https://api.example.com/health

The format is host:port:address. curl still uses the name api.example.com in the URL and in the TLS SNI, but the TCP connection goes to 10.0.0.5. This is very useful for testing an application against another server without editing the hosts file or waiting for DNS propagation.

#Connecting to a Specific Host: --connect-to

If you want to replace the connection destination host entirely, use --connect-to with the format host:port:target_host:target_port:

curl --connect-to api.example.com:443:staging.internal:443 \
  https://api.example.com/health

The difference is subtle but important: --resolve only replaces the IP address being connected to, while --connect-to replaces the connection destination host entirely. --connect-to can even change the scheme — for example connecting https to a host on a different port — making it more flexible in manual failover scenarios.

#Choosing the Source Interface: --interface

For machines with many network interfaces (multiple NICs, VPN, or bonding), curl can choose which interface the connection departs from:

curl --interface eth1 https://api.example.com/users

The --interface value can be an interface name (eth1), an IP address, or a host name. This is useful when you want to force traffic through a particular NIC — for example an interface with dedicated bandwidth or a specific network route.

#Troubleshooting Connections

When a connection misbehaves, don't guess — look at what's happening at the lower layers. Some debugging options you must know:

• -v (verbose) — shows handshake details, request headers, and response headers.
• --trace-ascii - — logs every byte sent and received, in text form.
• --connect-timeout — a time limit for just the connection phase, so scripts don't hang.

curl -v --connect-timeout 10 https://api.example.com/health 2>&1 | head -40

The -v output will show at which stage the failure occurs: DNS failed, TCP timeout, TLS handshake rejected, or an odd HTTP response. Reading this output turns you from "guessing" into "seeing the evidence" — the most valuable skill when networks misbehave.

#Closing

Episode 9 equips you with control over the network: choosing HTTP, HTTPS, and SOCKS5 proxies via -x and --socks5-hostname, proxy authentication with --proxy-user, per-host exclusions with --noproxy, DNS overrides with --resolve, reconnection with --connect-to, source interface selection, and troubleshooting techniques with -v.

The core thing to remember: proxies control where traffic goes, networking flags control how traffic arrives. Both work in layers — and understanding the layers is half of troubleshooting ability.

In the next episode 10, we'll cover authentication and sessions — Basic, Digest, NTLM, Bearer token schemes, and cookie management for complete login sessions. See you!