Learn Scrapling - Data Extraction & Serialization

#Introduction

So far you've gotten good at fetching pages — synchronous, asynchronous, dynamic, even stealthy — and selecting elements with CSS, XPath, text, all the way to adaptive selectors. But those selection results are still raw: a collection of element objects with no shape yet. In this episode 10 you'll turn them into structured data — dictionaries and JSON ready to save, send to a database, or use for analysis.

This episode's focus: extracting a single element into a dictionary, iterating many elements into a structured dataset, cleaning text of whitespace and junk elements like script and style, handling pagination with the next-link pattern until pages run out, then serializing to JSON and JSONL. After this episode, you can turn a website into a dataset file within minutes.

#Extracting One Item into a Dictionary

The most basic pattern is turning one element into a dictionary. Grab the element with css_first or an index, select each field with a selector, then arrange them into key-value pairs. Take a page containing product cards as an example:

single-item.py

from scrapling import Fetcher
 
page = Fetcher.get("https://quotes.toscrape.com/")
quote = page.css_first(".quote")
 
item = {
    "teks": quote.css(".text::text").get(),
    "penulis": quote.css(".author::text").get(),
    "tags": quote.css(".tag::text").getall(),
}
print(item)

Notice the variation in retrieval: .get(){python} for a single value (or None if absent), .getall(){python} for a list of values. The ::text pseudo-element takes the text content, while ::attr(href) would take an attribute value. The key-value arrangement above is the blueprint item you'll repeat for every element on the page.

For attributes, it works exactly the same way. If you need a link along with its text:

attributes.py

from scrapling import Fetcher
 
page = Fetcher.get("https://quotes.toscrape.com/")
quote = page.css_first(".quote")
 
detail = {
    "url": quote.css_first("a").attrib.get("href"),
    "teks": quote.css_first(".text::text").get(),
}

attrib is the element's attribute dictionary; attrib.get("href") is safe to use because it returns None when the attribute is absent. The combination of ::text for text and .attrib for attributes covers almost all field extraction needs.

#Multiple Items: Building a Dataset

A single page usually contains many items with the same pattern. To build a dataset, iterate over all matching elements, extract each into a dictionary, then collect them into one list. This is the bridge from "saving one item" to "saving the whole page".

dataset.py

from scrapling import Fetcher
 
page = Fetcher.get("https://quotes.toscrape.com/")
dataset = []
 
for quote in page.css(".quote"):
    dataset.append({
        "teks": quote.css(".text::text").get(),
        "penulis": quote.css(".author::text").get(),
        "tags": quote.css(".tag::text").getall(),
    })
 
print("total item:", len(dataset))

This list of dictionaries is a universal data structure — it can be written directly to JSON, used with pandas, or inserted into a relational database. The key point: the selectors inside the loop are consistent for every element, so the entire dataset is built from a single extraction pattern. If one field is empty, .get() returns None and the dataset still assembles without errors.

#Cleaning Text

Raw web data is rarely clean: there's extra whitespace, tabs, blank lines, or text fragments from hidden elements. Before saving, get into the habit of cleaning it. The first trick is strip to remove whitespace at both ends.

clean.py

from scrapling import Fetcher
 
page = Fetcher.get("https://quotes.toscrape.com/")
quote = page.css_first(".quote")
 
teks_kotor = quote.css(".text::text").get()
teks_bersih = teks_kotor.strip()
 
item = {
    "teks": teks_bersih,
    "penulis": quote.css(".author::text").get().strip(),
}

Second, to take all the text of a page or element, Scrapling provides get_all_text with ignore_tags — filtering out tags you don't need like script and style, which commonly smuggle JavaScript or CSS code into extraction results.

all-text.py

from scrapling import Fetcher
 
page = Fetcher.get("https://quotes.toscrape.com/")
teks_hal = page.get_all_text(ignore_tags=("script", "style"))
print(len(teks_hal))

Also get into the habit of normalizing extra whitespace inside sentences — for example, replacing several consecutive spaces with one — because odd whitespace gets saved along and makes later analysis harder. Cleaning text from the start is far cheaper than cleaning an entire dataset afterward.

#Pagination: Exploring All Pages

A single page rarely holds all the data. Most websites split their data across many pages with a "next" link. The most common pagination pattern is the next-link: grab the data on this page, look for the next link, if it exists continue, if not stop. In Scrapling, a simple loop is enough:

pagination.py

from scrapling import Fetcher
 
base = "https://quotes.toscrape.com"
url = base + "/"
dataset = []
 
while url:
    page = Fetcher.get(url)
    for quote in page.css(".quote"):
        dataset.append({
            "teks": quote.css(".text::text").get(),
            "penulis": quote.css(".author::text").get(),
        })
 
    next_link = page.css_first(".next a")
    url = base + next_link.attrib["href"] if next_link else None
 
print("total:", len(dataset))

while url{python} makes the loop stop automatically when url becomes None — that is, when there's no more next link. Note that the links found are usually relative, so you join them with the base URL. For parameter-based pagination like ?page=2, the pattern is the same but the URL is built by incrementing the page number and stops when a page no longer loads new items. Always add a safety limit — for example, a maximum of 50 pages — so the loop doesn't run wild if the last page still has a next link.

#JSON and JSONL Serialization

Once the dataset is collected, it's time to save it. JSON is the most common format for structured data — readable by Python, JavaScript, and other analysis tools. Writing it to a file only needs the standard json module:

save-json.py

import json
 
from scrapling import Fetcher
 
page = Fetcher.get("https://quotes.toscrape.com/")
dataset = []
for quote in page.css(".quote"):
    dataset.append({
        "teks": quote.css(".text::text").get(),
        "penulis": quote.css(".author::text").get(),
    })
 
with open("quotes.json", "w", encoding="utf-8") as fp:
    json.dump(dataset, fp, indent=2, ensure_ascii=False)

indent=2 makes the file human-readable, and ensure_ascii=False ensures non-ASCII characters (like accented letters) are stored as-is. For datasets that keep growing — like daily crawl results — the JSONL format (one JSON object per line) is more practical because you can append new lines without re-reading the whole file. Write a loop and open the file in append mode, or use the following line:

save-jsonl.py

import json
 
with open("quotes.jsonl", "a", encoding="utf-8") as fp:
    for item in dataset:
        fp.write(json.dumps(item, ensure_ascii=False) + "\n")

Each item becomes one line, and new lines are simply appended to the end of the file. JSONL is also easy to read line by line, so it suits streaming pipelines and resuming when a crawl is interrupted mid-way.

#Closing

Episode 10 takes you from raw selectors to a production-ready dataset. You've learned extracting a single item into a dictionary with .get() and .getall(), iterating many elements into a list of dictionaries, cleaning text with strip and get_all_text(ignore_tags=...), handling pagination via the next-link pattern with while url, and serializing to JSON and JSONL with the json module. Websites really can become data now.

The key takeaways:

• .get() for one value, .getall() for a list, .attrib for attributes.
• A dataset = a list of dictionaries, the result of iterating one extraction pattern.
• Clean text with strip and filter out script/style via ignore_tags.
• Next-link pagination uses while url and stops when the next link runs out.
• JSON for a complete snapshot; JSONL for continuously growing, easily resumable data.

In episode 11 we enter real crawl architecture: Spiders — Scrapling's framework for chaining one request into a multi-page crawl with concurrency, multi-session, and depth control. See you there!