systemd Guide for Admins

This is an original planning guide. It is not a vendor manual, an official standard, or legal advice. Example addresses use documentation ranges. Review every command before you paste it.

Units, enablement, and the journal

systemd starts processes from unit files. A service unit says what to run, as which user, after which other units, and what to do when it exits. A timer unit starts a service on a calendar. A socket unit can start a service on a connection. You do not need all three on the first day. You need to read the service you already have.

start runs it now. enable hooks it to a target so the next boot starts it. A unit can be started and not enabled, which is how a test disappears on reboot, or enabled and not started, which is how a reboot surprises you. systemctl status shows both, plus the last journal lines. Read the Active line and the vendor preset before you announce that the service is installed.

Vendor units live under /usr/lib/systemd/system or /lib/systemd/system. Package updates replace them. Your changes belong in /etc/systemd/system. A full copy there masks the vendor file. A drop-in, a file named like /etc/systemd/system/edge.service.d/override.conf, changes only the keys you set. Prefer the drop-in. After either change, daemon-reload, then restart, then status.

After= and Requires= are different. After= is order. Requires= is a dependency that stops this unit if the other fails. Wants= is a weaker dependency. People add Requires=network-online.target and then blame the network for a slow boot. Order the service after the thing it needs. Depend on it only when the service is meaningless without it.

A lab service with a drop-in, not a copied vendor file

The lab unit is edge.service. ExecStart runs a program that stays in the foreground. Type=simple is right for that. Type=forking is for a program that daemonises, and you only use it when the program actually forks. User= is a named account, not root, unless the program must be root. WorkingDirectory= is an absolute path the user can enter.

The vendor file, if a package shipped one, stays where the package put it. You add a drop-in that sets Environment= or a restart limit. systemctl edit edge.service creates the drop-in for you on many hosts. You still read the file it wrote. daemon-reload is the step that makes systemd notice. Restart is the step that makes the process notice. People do one and not the other.

Logs go to the journal. journalctl -u edge.service -n 50 is the first place you look, not a file you hope the program created. If the program also writes a file, that path is in the ticket. The logging guide is the retention decision. This page only insists you can see the last failure without guessing the filename.

The unit generator and the systemctl builder on this site draft a unit and a status command. They do not install it. Compare the draft to the drop-in you meant to write. A draft that sets User=root and Restart=always on a broken command will restart a failure forever. Trim it.

# Review before you paste. Lab unit name: edge.service
# Drop-in path, not a file under /usr:
# /etc/systemd/system/edge.service.d/override.conf
# [Service]
# User=edge
# Then: daemon-reload, restart, status
# Do not enable a unit you have not started once on purpose.

A change you can finish

On the lab host, show the unit, add the drop-in, reload, restart, and keep the status. Reboot only if the ticket says enablement is part of this change. A reboot is a second test, not the first one.

If the new unit fails, the rollback is to remove the drop-in, reload, and start the previous definition. You practised that while the host was still a lab.

systemctl cat the unit and save that output in the ticket.
Write the drop-in under /etc/systemd/system, not under /usr.
daemon-reload, then show the unit again and see the drop-in listed.
Restart and read status, including the Main PID and the last log lines.
Enable only if the next boot should start it, and say so in the ticket.
Remove the drop-in once in the lab and confirm the vendor behaviour returns.
Put the rollback, the reload, and the restart on the card next to the hostname.

Mistakes that outlive the command

systemd fails in the same few ways. Each one looks finished on the day you type it. The list is here so the ticket can forbid them before the window, not so you can recognise them in an incident review.

Editing the unit under /usr and losing the edit on the next package update.
Forgetting daemon-reload, then debugging a file systemd has not read.
enable without a successful start, so the next boot fails a unit you never saw run.
Restart=always on a command that exits immediately, filling the journal.
Requires= where After= was enough, and a slow dependency stalls the boot.
User=root because the directory was not writable by the service account.
Type=forking on a program that stays in the foreground, so systemd thinks it died.
Looking for a log file that was never configured, while the journal has the error.

Decisions to write down before the window

One decision for systemd: Changes live in /etc/systemd/system as a drop-in. The vendor file under /usr is not edited. User= is a named account. Root is a sentence in the ticket, not the default of the generator. Type= matches the process. simple for foreground. forking only if you watched it fork. daemon-reload and restart are both in the window. Status after restart is the proof. enable is a separate decision from start. The ticket says whether the next boot should start the unit. Restart= is limited. A failing command does not loop. StartLimitBurst is left at a number you can explain. After= orders the unit. Requires= is used only when the dependency's failure should stop this unit. Logs are in the journal first. An extra file path is written down if the program has one. The rollback is removing the drop-in and reloading, which you have done once. A generator draft was trimmed. It does not enable every key the form offered. The unit name in the ticket matches the filename. edge.service is not a leftover name from the generator's example if this host's service is called something else. And a package update was considered. Because the vendor file was not edited, the next update keeps your drop-in. You know which package owns the unit.

systemd checks before the unit is enabled

Confirm this before you call the change done: systemctl cat shows the drop-in and the vendor file separately. Nothing you care about was edited under /usr/lib or /lib. daemon-reload was run after the last edit. status shows active and the Main PID you expect. The user of that PID is the account in the ticket, confirmed with the process list. journalctl -u shows a clean start, or the error you already fixed. enable is set only if the ticket asked for boot persistence. Restart= will not loop a failing command. Rollback drop-in removal was practised in the lab. The working directory exists and the user can enter it. A second person read User= and ExecStart aloud. The host's init is systemd. An OpenRC host is not this change. The drop-in keys are only the ones you meant. A commented example was not left uncommented. journalctl still works for this unit after the restart, and you did not point the notes at a file that does not exist. The hostname in the prompt matches the hostname in the ticket for systemd. The result line includes the time and the person, not only the command. You can point at the rollback value without searching a chat history. And done is a sentence a second person can repeat: the proof for systemd happened, and the next host is not started early. If one of those fails, stop.

Tools for this guide

These tools run in the browser on this site. They do not log into your machines.

Guides in this cluster

These supporting pages belong to this pillar.