Learn Linux - Introduction & Explanation of Systemd & Managing Services

#Introduction

After episode 13 where we covered process management — PID, states, signals, background jobs, up to nohup and tmux — there's a question we deliberately left hanging: how do you make a process officially survive? You already know nohup node server.js & can keep an application running after the terminal closes, but who looks after it if the application crashes, or makes sure it comes back after the server reboots? The answer is systemd.

Systemd is the init system that's process number 1 (PID 1) on almost all modern distros — it's the first process the kernel runs at boot, and the last process to "die" at shutdown. In this episode we'll dissect systemd's role as the "manager of all processes", practice managing services with systemctl, read centralized logs with journalctl, and write our own service unit files for your applications. Let's begin.

#Main Discussion

#Why Is Systemd PID 1?

Before systemd, Linux used SysVinit: a system based on runlevels and shell scripts that runs services one by one in sequence. Every service was a long script, there was no good parallel dependency mechanism, and no centralized way to view the status of all services. Starting a server with 30 services meant waiting for 30 scripts to execute almost sequentially — slow and fragile. Then came Upstart (event-based, from Ubuntu) as a partial fix, then systemd that solved everything.

Systemd changed the paradigm from "script sequence" to targets and units. Every service is represented by a unit file with a declarative format (not an imperative script), services can run in parallel, dependencies are declared explicitly, and the entire service state can be monitored centrally. This is why modern systems boot much faster and are easier to manage.

Imagine systemd as a hotel manager. Previously (SysVinit), every guest (service) was escorted manually one by one by a bellhop — orderly and slow. Systemd is a digital front desk: it knows who should arrive first (dependencies), who can arrive at the same time (parallel), and records all guest activity in a single log book. When a guest faints in the lobby, the front desk handles it according to the rules — restarting them or informing the manager.

#Managing Services with systemctl

The keyword "unit" in systemd covers services, sockets, timers, mounts, and more. The unit you manage most often is the service. The management command is systemctl. Here's the full lifecycle:

# Start the service now
sudo systemctl start nginx
 
# Stop the service
sudo systemctl stop nginx
 
# Restart the service (stop then start)
sudo systemctl restart nginx
 
# Reload the configuration without dropping connections
sudo systemctl reload nginx
 
# Enable at boot / disable
sudo systemctl enable nginx
sudo systemctl disable nginx
 
# Show full status: running or not, since when, PID, and recent logs
systemctl status nginx

The systemctl status command is the first window into any troubleshooting — it shows whether the service is active or failed, its PID, resource usage, and the last few log lines. The combination of systemctl status nginx + journalctl will become your routine every time a service misbehaves.

There's one detail that's often forgotten: after editing a service unit file, you must tell systemd to reload the unit definitions with systemctl daemon-reload. Without it, your changes are ignored.

sudo systemctl daemon-reload
sudo systemctl restart myapp

Besides managing services one by one, you also need to see the whole system picture: which services are running, and which failed. systemctl list-units is the dashboard:

# All currently active units
systemctl list-units --type=service
 
# Only failed services (first step of server troubleshooting)
systemctl list-units --type=service --state=failed
 
# Quick text status (useful for scripts)
systemctl is-active nginx

systemctl is-active nginx returns active or inactive, and exits with an exit code usable in script logic — a pattern you'll meet often when writing your own monitoring or alerting scripts.

#Reading Logs with journalctl

One of systemd's great advantages is centralized logging: all services write their logs to the journal, and you read them with journalctl — no need to open separate log files in /var/log. This changes how admins find problems: from "which log file?" to "where are this service's logs".

# All logs for one service
sudo journalctl -u nginx
 
# Follow the log live (like tail -f)
sudo journalctl -u nginx -f
 
# The last 50 log lines
sudo journalctl -u nginx -n 50
 
# Logs since one hour ago
sudo journalctl -u nginx --since "1 hour ago"
 
# Logs in a specific time range
sudo journalctl -u nginx --since "2026-08-01 09:00" --until "2026-08-01 12:00"

Aug 01 09:15:22 webserver systemd[1]: Starting nginx...
Aug 01 09:15:22 webserver nginx[3456]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Aug 01 09:15:22 webserver nginx[3456]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Aug 01 09:15:22 webserver systemd[1]: Started nginx.
Aug 01 09:17:03 webserver nginx[3456]: 192.168.1.10 - - [01/Aug/2026:09:17:03] "GET /" 200 612
Aug 01 09:18:44 webserver nginx[3456]: [error] upstream prematurely closed connection

The combination journalctl -u nginx --since "1 hour ago" is the first reflex when a user reports "the server got slow an hour ago" — you immediately see what happened to the service during that hour. journalctl can also be filtered with -p for priority levels (err, warning, etc.) and grep for specific patterns.

#Writing Your Own Service Unit File

This is the most valuable part of this episode: making your application an official service managed by systemd. Imagine you have an app myapp — a binary or script in /opt/myapp/ — and you want systemd to look after it: start at boot, restart on crash, and record its logs.

The unit file goes in /etc/systemd/system/myapp.service, divided into three standard sections:

[Unit]
Description=My Production Application
After=network.target
 
[Service]
Type=simple
User=myapp
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/node /opt/myapp/server.js
Restart=on-failure
RestartSec=5
 
[Install]
WantedBy=multi-user.target

Let's dissect each important line:

• After=network.target — the service only runs after the network target is reached, so the app doesn't start before the network is ready.
• Type=simple — systemd considers the main process to be the one run directly by ExecStart. This is the correct default for most modern applications (Node, Python, Go).
• User=myapp — runs the service as a non-root user. This is least privilege implemented at the service level.
• Restart=on-failure — systemd will restart the service if it exits with an error status. This is the main availability "insurance".
• RestartSec=5 — a 5-second delay before trying to restart (prevents a restart storm).
• WantedBy=multi-user.target — in the [Install] section; determines that systemctl enable will link the service to the multi-user target (running on a normal boot, without a GUI).

The commands to activate a new unit:

sudo systemctl daemon-reload
sudo systemctl enable myapp
sudo systemctl start myapp
systemctl status myapp

Now see how diffs work when we add a more aggressive restart mechanism — for example, wanting the app to always be restarted no matter the cause. Notice the -- lines removed and ++ lines added:

[Service]
Type=simple
User=myapp
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/node /opt/myapp/server.js
Restart=on-failure
Restart=always
RestartSec=5

After the change, don't forget the ritual order: sudo systemctl daemon-reload then sudo systemctl restart myapp.

#Handling Forking Applications and Other Types

There's a classic trap that makes beginner admins write Type=forking when it's not needed. Type=forking is for applications that "fork" themselves: the ExecStart process starts and immediately exits, while the worker (daemon) process runs in the background. Classic examples are old daemon scripts and some traditional C/C++ applications. For such daemons, systemd needs PIDFile= to know which process to track:

[Service]
Type=forking
User=daemon
ExecStart=/usr/local/bin/mydaemon start
ExecStop=/usr/local/bin/mydaemon stop
PIDFile=/run/mydaemon.pid
Restart=on-failure
 
[Install]
WantedBy=multi-user.target

There's also Type=oneshot for services that run a single task then finish (like initialization scripts), and Type=notify for applications that tell systemd via the sd_notify socket that they're ready (used by big services like databases). When you later learn about systemd timers in episode 22, Type=oneshot will become your friend.

#Common Mistakes in Managing Systemd

One pattern that often confuses admins: manually killing a service process with kill -9 <PID>. Systemd will detect the main process is dead and, because Restart=on-failure is active, restart the service — so you're puzzled why "the process you killed" shows up again. Don't fight systemd; use systemctl stop to officially stop a service, or systemctl restart if the goal is just to restart it.

#Conclusion

In this episode 14 you've taken control of systemd: understanding why it replaced SysVinit/Upstart as the init system, managing services with systemctl (start/stop/restart/reload/enable), reading centralized logs with journalctl (filters -u, -f, -n, --since), and writing your own service unit files complete with the three sections [Unit], [Service], [Install]. The most important lesson: systemd turns applications from "manually-run processes" into well-cared-for citizens of the system — always running, always monitored, and always leaving a log trail.

But even a perfectly running service is useless if the disk is full. In the next episode 15 we'll discuss storage, disk & filesystem management — reading capacity with df/du/free, partitioning and formatting disks with fdisk/mkfs, mounting and persisting via /etc/fstab, through to getting to know LVM with dynamic resizing without downtime. See you there!