Learn Scrapling - CLI, Shell, MCP & Production Tooling

#Introduction

In episode 20 you built a culture of testing and maintenance. Now we go up one level: operating Scrapling in production. Not everyone wants to write Python scripts for small tasks — and not every deployment can rely on a manually installed browser. Episode 21 covers the tooling that bridges both: the CLI, the interactive shell, the MCP server, and the Docker image.

This episode's roadmap: we start with the CLI for extraction from the terminal, then important flags like --css-selector and --impersonate, continue with the interactive shell, and close with the Docker image plus isolated deployment for production.

#CLI: One Command for Extraction

Since v0.3, Scrapling has a powerful CLI. Without writing a single line of Python, you can download a page and extract its contents directly to a file. Start by seeing what's available:

scrapling --help
scrapling extract --help

The core command is scrapling extract. The output file format is determined by its extension: .txt produces clean text, .md produces a Markdown representation, and .html saves the raw HTML. By default, the contents of the body tag are extracted.

#Fetch Modes in the CLI

The extract subcommand has three modes that map to fetchers you already know:

• get — fast HTTP requests with TLS impersonation.
• fetch — JavaScript rendering via a browser.
• stealthy-fetch — a stealth browser for bypassing anti-bot defenses.

scrapling extract get "https://example.com" content.txt

scrapling extract fetch "https://example.com" content.md

scrapling extract stealthy-fetch "https://nopecha.com/demo/cloudflare" captchas.html --solve-cloudflare

All three modes use the same engines as the Python fetchers, so results stay consistent — the difference is that everything is now available without writing code. This is useful for ad hoc tasks, quick debugging, or simple automation via shell scripts.

#Important Flags

The flags you'll use most often:

• --css-selector — restrict extraction to matching elements only, saving output and tokens.
• --impersonate — choose a browser fingerprint, for example --impersonate "chrome".
• --no-headless — show the browser on screen, useful for manual observation.
• --solve-cloudflare — solve Cloudflare/Turnstile challenges automatically.

scrapling extract get "https://shop.example.com" products.html --css-selector ".product-card" --impersonate "chrome"

scrapling extract fetch "https://example.com" content.md --css-selector "#app" --no-headless

With --css-selector, you control exactly which part of the page is taken — the same pattern as the CSS selector narrowing in the MCP server discussed in episode 19. The --impersonate flag relies on the same technology as Fetcher in Python: mimicking the TLS fingerprint and headers of a real browser.

#Interactive Shell for Productivity

For deeper exploration than a one-shot execution, there's an interactive shell based on IPython. It provides shortcuts for fetching and live selector manipulation — the perfect companion for writing and testing selectors before moving them into code.

pip install "scrapling[shell]"
scrapling install
scrapling shell

Inside the shell, variables from fetches and selections can be explored, changed, and re-tested within seconds. The shell and the CLI share the same package (scrapling[shell] enables both), so your workflow flows smoothly from interactive experiments to terminal extraction.

#Docker Image for Production

The hardest part of deploying Scrapling is the browser dependencies. The official Docker image solves this: built automatically on every release, it contains the entire [all] extra, all browsers, and pre-configured system dependencies.

docker pull pyd4vinci/scrapling

This image is ready to use right away:

docker run -it pyd4vinci/scrapling python
docker run -v "$(pwd)":/app pyd4vinci/scrapling python /app/spider.py
docker run -it pyd4vinci/scrapling scrapling shell

For your own deployment, make this image the FROM base:

FROM pyd4vinci/scrapling:latest
 
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
 
CMD ["python", "spider.py"]

Because all browsers are already in the image, StealthyFetcher, PlayWrightFetcher, and scrapling install no longer need to run at runtime — a big saving on cold starts in container environments.

#MCP Server and Isolated Deployment

The MCP tooling you met in episode 19 is also production-ready. The MCP server can be exposed as its own service, isolated from the main scraper:

docker run -p 8000:8000 pyd4vinci/scrapling scrapling mcp --http

The recommended isolated architecture: one container for the main spider, another for the MCP server serving AI agents, and both sharing an external data store. This separation enables independent scaling — heavy load on the crawl side doesn't disturb the MCP service, and vice versa. Don't forget to restrict the MCP server's access to your internal network or protect it with an auth token.

#Closing

Episode 21 completes Phase 6 — Modern Ecosystem & Production Readiness: the CLI for one-shot extraction, the interactive shell for experiments, the Docker image as a deployment foundation, and the MCP server as an isolated service. You now have a path from a single terminal command to a complete production platform.

The key takeaways:

• The scrapling extract CLI extracts to .txt, .md, or .html without writing Python.
• The get, fetch, and stealthy-fetch modes map directly onto Scrapling's core fetchers.
• The --css-selector, --impersonate, and --no-headless flags give full control from the terminal.
• The official pyd4vinci/scrapling image ships all browsers and extras — use it as your production image base.
• Isolated deployment separates the spider, the MCP server, and data so scaling stays independent.

In episode 22 — the final episode — we do a thorough reflection: Ecosystem, Alternatives & Final Reflection, comparing Scrapling with Scrapy, Crawlee, Crawl4AI, and others, plus a full recap of the 0–21 journey along with a production checklist. See you there!