Learn Scrapling - Fetcher: HTTP Requests & Sessions

#Introduction

In episode 2 you mapped out Scrapling's architecture: fetchers, the Adaptor parser, spiders, the CLI, and the MCP server. Now it's time for your first serious hands-on practice: speaking HTTP with Fetcher.

This episode focuses on HTTP requests — get and post, sessions that persist cookies, browser impersonation, stealth headers, and how to handle status codes and redirects properly. By the end of this episode, you can fetch any web page confidently and handle its response like a pro.

#Fetcher.get: Your First Request

Fetcher is a static class — call its methods directly without instantiation. The simplest request is done with get:

first-request.py

from scrapling import Fetcher
 
page = Fetcher.get("https://example.com")
 
print(page.status)
print(page.text[:200])

Notice the two properties available right away: page.status tells you the status code, and page.text contains the raw HTML. Thanks to built-in TLS impersonation, this request already looks like it was sent from a browser, not from plain Python.

#Impersonation: Disguising Your Fingerprint

Many websites reject requests that don't come from a browser — not just based on the User-Agent, but on the TLS fingerprint. A server can tell which HTTP library sent a request from the pattern of its TLS handshake. Scrapling solves this with the impersonate parameter:

impersonate.py

from scrapling import Fetcher
 
page = Fetcher.get(
    "https://example.com",
    impersonate="chrome",
)
print(page.status)

The value chrome disguises the entire request profile — including the TLS fingerprint, header order, and HTTP/2 settings — so it's identical to real Chrome. Other options like firefox, safari, or specific browser versions are also available. Choose the browser closest to the majority of your target's visitors.

#Sending Data with Fetcher.post

Not every target is fetched with get. Login forms, search, and filters usually require the post method. Use the data or json parameter:

post-request.py

from scrapling import Fetcher
 
page = Fetcher.post(
    "https://example.com/login",
    data={"username": "arman", "password": "rahasia"},
    follow_redirects=True,
)

The follow_redirects parameter ensures the request follows any redirect that happens after login. The result is still the same Adaptor object — selection works exactly as it does after a get.

#FetcherSession: Persisting Cookies

Most serious sites track sessions via cookies. If each request makes a fresh connection without cookies, the server won't remember you — and login won't persist. That's where FetcherSession comes in. It keeps cookies and connection reuse across requests:

session.py

from scrapling import Fetcher
 
session = FetcherSession(impersonate="chrome")
 
session.get("https://example.com/login")
page = session.post(
    "https://example.com/login",
    data={"username": "arman", "password": "rahasia"},
)
profil = session.get("https://example.com/profil")
 
print(profil.status)

Cookies received during login are automatically persisted and re-sent on the profil request. This is the standard pattern for sites that require authentication: log in once with a session, then browse protected pages without sending credentials again.

#Stealthy Headers: Looking More Human

Sometimes impersonate alone isn't enough. Certain websites pay attention to additional headers like Referer or X-Requested-With. Scrapling provides stealthy_headers, which fills popular headers with realistic values:

stealthy-headers.py

from scrapling import Fetcher
 
page = Fetcher.get(
    "https://example.com",
    impersonate="chrome",
    stealthy_headers=True,
)

The impersonate plus stealthy_headers combination makes a request look like a normal browser visit — complete with the headers a real browser would send. For most non-anti-bot sites, this is more than enough.

#Handling Status Codes and Errors

The real world doesn't always return 200. A website might respond 403 (blocked), 404 (not found), or 429 (rate limited). Good code should check the status before processing:

error-handling.py

from scrapling import Fetcher
 
page = Fetcher.get("https://example.com/data")
 
if page.status == 200:
    print("Berhasil, panjang HTML:", len(page.text))
elif page.status == 403:
    print("Diblokir — coba ganti impersonate atau gunakan browser fetcher")
elif page.status == 429:
    print("Kena rate limit — tunggu sebelum request berikutnya")
else:
    print("Status lain:", page.status)

Besides checking the status, also watch for the pattern where the status is 200 but the page contains an error message or challenge. When your selection returns empty data despite a success status, check page.text — you may have received an anti-bot challenge page instead of the content you wanted.

#Reading Response Headers

Sometimes you need to inspect response headers — for example, to see Set-Cookie, Content-Type, or the last update date. Access them via the headers property:

read-headers.py

from scrapling import Fetcher
 
page = Fetcher.get("https://example.com")
 
print(page.headers.get("Content-Type"))
print(page.headers.get("Last-Modified"))

This property is useful for debugging — for instance, when you suspect the server is sending a compressed page or an internal redirect that isn't visible from the status code.

#Closing

Episode 3 is done. You can now fetch a web page with Fetcher.get and Fetcher.post, keep a login alive with FetcherSession, disguise your fingerprint with impersonate, fill in realistic headers with stealthy_headers, and read status codes and response headers for proper error handling.

The key takeaways:

• Fetcher.get and Fetcher.post directly return an Adaptor complete with status and text.
• impersonate="chrome" disguises the TLS fingerprint, header order, and HTTP/2 settings.
• FetcherSession keeps cookies and connections alive so authentication persists across requests.
• stealthy_headers adds realistic headers so requests look more like a browser.
• Always check the status code, and be suspicious of an empty 200 that could be a challenge page.

In episode 4 we learn to read the results: Adaptor and basic parsing — CSS and XPath selection, get vs getall, chaining, and extracting text and attributes. See you there!