Learn DragonFlyBSD - LWKT, Scheduler & Performance

#Introduction

In episode 16 you managed advanced networking with dynamic routing and tunneling. Now we dive into the most distinctive part of DragonFlyBSD: its kernel. Episode 17 covers LWKT — the threading architecture that motivated this fork — plus how you control the scheduler and measure performance.

Remember the story from episode 1? DragonFlyBSD was born because Matthew Dillon considered FreeBSD's kernel threading model insufficiently scalable for multi-core machines. LWKT is the answer: each CPU runs its own kernel thread in parallel, and the scheduler works per-process. This episode explains how that works and how you can take advantage of it.

#LWKT: Lightweight Kernel Threading

#The Core Concept

On most BSDs, the kernel uses a global lock that lets only one CPU enter the kernel at a time — safe but slow on modern machines. LWKT solves this by giving each CPU its own thread structure. Lightweight kernel threads (LWKT threads) can run in parallel across many CPUs, and migration between CPUs is governed by clear rules.

The impact is felt in workloads with many processes and threads: the kernel is no longer the sole bottleneck. This is why DragonFlyBSD excels on machines with many cores and busy server loads.

#The Per-Process Scheduler

Alongside LWKT for the kernel, DragonFlyBSD uses a per-process scheduler — scheduling decisions are made considering the process and the CPU it's on. This scheduler balances load across CPUs with gradual migration, preserves cache affinity, and responds well to I/O load.

Viewing the active scheduler and related parameters:

sysctl kern.sched.name
sysctl kern.sched.slice
sysctl kern.sched.interactive

kern.sched.name shows the active scheduler; the other parameters control scheduling behavior.

#cpuset: CPU Affinity

#Locking a Process to a CPU

cpuset binds a process or thread to specific CPUs — very useful for latency-sensitive applications that shouldn't be migrated:

cpuset -g -p 1234
cpuset -l 0,1 -p 1234

cpuset -g reads the affinity of process 1234; cpuset -l 0,1 binds it to CPUs 0 and 1. For a new process:

cpuset -l 2-3 -- ./heavy-task

This command runs heavy-task only on CPUs 2 and 3 — a common pattern for databases and real-time applications.

#System-Level Affinity

Listing CPUs and the whole system's affinity:

cpuset -g
sysctl hw.ncpu

sysctl hw.ncpu tells you how many cores are available — the starting point before planning process distribution.

#Tuning the Kernel & HAMMER2

#Frequently Tuned sysctl Parameters

Some parameters most often adjusted for particular workloads:

sysctl kern.sched.slice
sysctl vfs.hammer2.double_buffer
sysctl vm.swap_idle_enabled

vfs.hammer2 holds parameters that affect HAMMER2 filesystem behavior — including double buffer usage, which has a big impact on I/O workloads. Every parameter can be viewed and its value tested at runtime before being persisted in /etc/sysctl.conf.

#loader.conf

Parameters that must be available from the very start of boot — before the kernel is fully alive — are set in /boot/loader.conf:

kern.maxvnodes="100000"
vfs.hammer2.double_buffer="1"
hw.igb.num_queues="4"

loader.conf is read at boot, earlier than sysctl.conf. The rule of thumb: set it in loader.conf if the parameter affects kernel structures allocated at boot; set it in sysctl.conf if it's safe to change at runtime.

#Performance Monitoring

#systat

systat is the real-time dashboard:

systat -vmstat 2
systat -ifstat

-vmstat shows CPU, memory, and I/O every 2 seconds; -ifstat shows traffic per interface.

#top

top shows the busiest processes and resource usage:

top
top -b -n 5

-b is batch mode for scripts — useful for capturing load snapshots into logs.

#vmstat

vmstat provides a summary of virtual memory and system activity:

vmstat -w 2

Watch the cs (context switch), in (interrupt), and r (run queue) columns — abnormal spikes in these are early signs of a bottleneck.

#Closing

In this episode 17 you understood the core of the DragonFlyBSD kernel: Lightweight Kernel Threading with per-CPU threads and the per-process scheduler, locking CPU affinity with cpuset, tuning parameters like kern.sched and vfs.hammer2 via sysctl and loader.conf, and monitoring with systat, top, and vmstat.

Key takeaways:

• LWKT lets the kernel run in parallel across many CPUs — DragonFlyBSD's main differentiator.
• cpuset -l 0,1 -p PID locks a process to specific CPUs for latency-sensitive workloads.
• Tune kern.sched.* and vfs.hammer2.* via sysctl; use /boot/loader.conf for early-boot parameters.
• Measure with systat -vmstat, top, vmstat before and after every change.
• Change one parameter per experiment — the measure-change-measure cycle.

In the next episode, episode 18, we run a world inside a world: NVMM & virtualization. You'll get to know the type-2 hypervisor nvmm(4), its integration with QEMU through the qemu package, managing VMs, setting up networking with tap and bridge, and creating VM snapshots.