Linux Logging Guide — MyAdminDesk

This is an original planning guide. It is not a vendor manual, an official standard, or legal advice. Example addresses use documentation ranges. Review every command before you paste it.

A log is evidence with a clock

journald is the default collector on systemd hosts. The journal is binary, on disk or in memory, and journalctl reads it. rsyslog and other syslog daemons read text and can write files or forward lines. A host can run both: journald collects, and a syslog daemon forwards. Know which one you are configuring before you edit a file from the other.

Time comes first. A log from a host whose clock is wrong cannot be lined up with a log from a host whose clock is right. The fix for a lying clock is the time source, not a longer retention. Write the timezone and the NTP or chrony state next to the logging decision.

Retention is a disk decision. A journal that is allowed to fill the root volume will take the host down, which is a performance incident caused by a logging setting. Set a maximum size or a maximum age you can afford, and alert before the volume is full. Forwarding to another host does not remove the need for a local limit. The network can be down on the day you need the lines.

Secrets do not belong in logs. A password, a session token, or a connection string in a debug line becomes a file you now have to protect like a password. If you find one, rotate the secret and stop the program logging it. Do not copy the line into the ticket. The security desk's log guide is the companion for investigations. This page is how the lines get kept.

A lab host that keeps seven days and refuses to fill the disk

The lab decision is seven days of journal on disk, with a size cap that leaves the root volume able to boot. SystemMaxUse and MaxRetentionSec are the kind of keys journald understands. The names on your version are the ones in man journald.conf, not a key you remember from another daemon. The journald generator on this site drafts a drop-in. Read every key. A draft that sets Storage=none will throw away the lines you think you are keeping.

If you also forward with rsyslog, the destination is a host you operate, on a prefix you named, and the forwarding is not the only copy until you have seen a line arrive there. The rsyslog generator drafts that forward. It does not open the collector. Test with one marked line, then search for that mark on the collector.

A service unit's logs are journalctl -u, which the systemd guide already uses. Do not also turn on a debug flag that prints tokens, to see if the journal works. A marked line that says logging-test and the hostname is enough.

After the change, write a line, reboot or restart the daemon as the ticket says, and confirm the line is still there if you asked for persistent storage. A volatile journal looks configured until the next boot.

# Review before you paste. Lab journald drop-in only.
# Confirm keys in man journald.conf on this host.
# Goal: persistent, size-capped, no secrets in the test line.
# logger -t logging-test edge-01
# Then find that line with journalctl, and again after a restart if you required persistence.

A change you can finish

Change retention on the lab host first. Confirm a test line survives the restart you care about, and confirm the cap is visible in the configuration you reloaded. Then copy the same drop-in to the next host, one host at a time.

Rollback is the previous drop-in. Keep it. A logging change that fills the disk is rolled back before you add a second collector.

Confirm the clock and write down the timezone.
Decide journald, syslog, or journald plus a forwarder.
Set a size or age cap that cannot fill the root volume.
Reload the daemon you edited and show the effective config.
Write one marked line with no secret in it.
Find the line locally, and on the collector if you forward.
Restart once if persistence is the requirement, and find the line again.

Mistakes that outlive the command

Linux logging fails in the same few ways. Each one looks finished on the day you type it. The list is here so the ticket can forbid them before the window, not so you can recognise them in an incident review.

Editing rsyslog when the host only has journald, or the reverse.
Storage=volatile when you needed the lines after a reboot.
No size cap, so the journal fills the disk and the host becomes the performance incident.
Forwarding as the only copy, then losing the network and the lines together.
A debug mode that logs passwords or tokens.
A test line that itself contains a real secret.
Clocks that disagree, so two hosts cannot tell one story.
A generator draft applied without deleting the key that disables storage.

Decisions to write down before the window

One decision for Linux logging: The collector is named: journald, syslog, or both, and you know which file you are allowed to edit. The clock is correct enough to compare hosts. The timezone is written down. A size cap or an age cap exists and is smaller than the free space you need to boot. Persistence across reboot is either required and tested, or explicitly not required. Forwarding goes to a host you operate. The lab mark was found there. Secrets are out of scope for log lines. A token in a line is rotated, not filed. The generator draft was read. Storage is not none unless you meant to discard. The service unit logs are in the journal, and any extra file path is in the ticket. Rollback is the previous drop-in, kept beside the ticket. One host is done before the next host gets the same file. And the security companion is for how you investigate. This ticket is only for how you keep the lines.

Logging checks before you rely on the lines

Confirm this before you call the change done: The clock and timezone are recorded. The daemon you reloaded is the daemon you edited. A size or age cap is present in the effective configuration. The cap cannot fill the root volume. A marked test line with no secret was found locally. If persistence matters, the line survived a restart. If you forward, the same mark was found on the collector. The generator draft does not set storage to none. No debug flag that prints secrets is enabled. The previous configuration is saved for rollback. journalctl -u still shows the service you care about. A second person saw the test mark. The next host is not changed until this host's check passed. The hostname in the prompt matches the hostname in the ticket for Linux logging. The result line includes the time and the person, not only the command. You can point at the rollback value without searching a chat history. And done is a sentence a second person can repeat: the proof for Linux logging happened, and the next host is not started early. If one of those fails, stop.

Tools for this guide

These tools run in the browser on this site. They do not log into your machines.

Guides in this cluster

These supporting pages belong to this pillar.

Companion: Security: Log Analysis for Security.