Learning Python - Dependency Management & Reproducible Envs

#Introduction

In episode 7 we started recording dependencies. Episode 8 takes that management to production level: how to make sure your project runs identically on any machine, when to use requirements.txt versus pyproject.toml, and how to lock dependency versions with a lockfile.

We'll also cover modern tools like Poetry, Flit, and pipx. The goal is one thing: reproducible environments — an important foundation before you get to deployment in episode 20.

#requirements.txt vs pyproject.toml

#When to Use requirements.txt

requirements.txt is a simple, long-standing dependency list:

requests==2.32.3
fastapi>=0.110
uvicorn[standard]>=0.29

requirements.txt is easy to understand and often used for applications that aren't libraries. The format package==version pins an exact version, while package>=version gives a range. Install with pip install -r requirements.txt.

#The Advantages of pyproject.toml

pyproject.toml is more structured and has become the standard for modern libraries:

[project]
name = "belajar-web"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "fastapi>=0.110",
    "uvicorn[standard]>=0.29"
]
 
[project.optional-dependencies]
dev = ["pytest>=8.0", "ruff", "mypy"]

[project.optional-dependencies] separates development dependencies from runtime dependencies. Install the main dependencies with pip install -e ., and dev dependencies with pip install -e ".[dev]". This is more expressive than requirements.txt.

#Poetry and Flit

#Poetry: An All-in-One Manager

Poetry combines dependency management, virtual environments, and packaging in one tool:

pip install poetry
poetry new belajar_poetry
cd belajar_poetry
poetry add fastapi

poetry new belajar_poetry creates a new project, and poetry add fastapi adds a dependency along with a lockfile. Poetry creates poetry.lock, which pins the exact versions of all dependencies — the key to reproducible environments.

#Flit: Lightweight Packaging

Flit focuses on making library packaging easy:

pip install flit
flit init
flit publish

flit init creates an interactive pyproject.toml, and flit publish builds and uploads the package to PyPI. Flit suits simple libraries that need a fast build. For complex applications, Poetry or pip-tools are more flexible.

#Lockfiles with pip-tools

#Why You Need a Lockfile

A lockfile records the exact versions of all dependencies and their transitive dependencies, so an environment can be reproduced exactly. pip-tools provides two tools: pip-compile to create the lockfile and pip-sync to install it:

pip install pip-tools
pip-compile --output-file=requirements.lock pyproject.toml

pip-compile reads dependencies from pyproject.toml and produces requirements.lock containing the resolved exact versions. This lockfile is what you commit to Git so all team members use the same versions.

#Installing from the Lockfile

Once the lockfile exists, install its exact versions with pip-sync:

pip-sync requirements.lock

pip-sync requirements.lock aligns your environment exactly with the lockfile's contents — removing packages not in the list and installing the pinned versions. This guarantees consistency between development, staging, and production.

#pipx for CLI Tools

#Installing Tools in Isolation

pipx installs Python CLI tools into isolated environments so they don't pollute project venvs:

pip install pipx
pipx install black
pipx install poetry
pipx install ruff

pipx install black installs black into a dedicated environment and makes its command available globally. Unlike a regular pip install, pipx isolates each tool so dependency conflicts between tools don't happen.

#Managing Tools with pipx

pipx supports full operations for managing tools:

pipx upgrade black
pipx list
pipx uninstall black

pipx list shows all installed tools, pipx upgrade black updates a specific tool, and pipx uninstall black removes it. The rule of thumb: global CLI tools are installed with pipx, while project libraries are installed in the project venv.

#Virtualenv Best Practices

#Reproducible Environment Rules

A few practices that keep environments reproducible:

• Always activate the venv before working.
• Commit the lockfile, not just the dependency list.
• Use the same Python version on all machines.
• Never use sudo pip or global pip.
• Update dependencies regularly, don't let them pile up.

#A Practical Workflow

The recommended workflow per project:

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -e ".[dev]"
pip-compile --output-file=requirements.lock
pip-sync requirements.lock

This sequence builds the venv, installs dev dependencies, then locks and syncs the versions. Run pip-compile whenever there's a new dependency, and commit its lockfile. This guarantees every team member and CI uses an identical environment.

#Closing

Key takeaways:

• requirements.txt is simple; pyproject.toml is more structured and modern.
• Poetry manages dependencies, venv, and packaging all at once.
• Flit focuses on ease of building and publishing libraries.
• pip-tools creates lockfiles with pip-compile and installs with pip-sync.
• pipx installs CLI tools in isolation.
• Commit the lockfile for reproducible environments on all machines.

In the next episode, episode 9, we'll cover persistence and databases — SQLAlchemy Core and ORM, migrations with Alembic, an overview of Redis and MongoDB drivers, and connection pooling and transaction patterns. Your project starts connecting to real data!