Hardening a fresh Linux install

I’ve been running a small Hetzner VPS for a while now and every time I spin up a new one I find myself re-deriving the same baseline. This is that baseline, written down so I stop re-deriving it and so you can steal it. It assumes Ubuntu 24.04 LTS on a Hetzner VPS, but most things hardly change over the years, or between platforms. None of this is exhaustive, there’s no AppArmor profiling, no AIDE, but it gets you to a point where you won’t be embarrassed if someone runs nmap at you.

Hetzner Cloud lets you upload an SSH key during server creation, which gets installed into /root/.ssh/authorized_keys on the new machine. Good. SSH in as root one time, then make a user for yourself:

adduser atak
usermod -aG sudo atak

mkdir -p /home/atak/.ssh
cp ~/.ssh/authorized_keys /home/atak/.ssh/
chown -R atak:atak /home/atak/.ssh
chmod 700 /home/atak/.ssh
chmod 600 /home/atak/.ssh/authorized_keys

Now try and open a second SSH session as your new user before changing anything else, so you don’t lock yourself out. Once that works, edit /etc/ssh/sshd_config and make sure these are set:

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no

Then sudo systemctl restart ssh. I don’t bother moving SSH to a non-standard port, it doesn’t make you any safer, it just makes your logs quieter. If you want quieter logs, fail2ban (below) does that without the friction of remembering a port number.

Unattended upgrades allow you to automatically install security updates. This is a must-have on a server, unless you like the idea of manually running sudo apt update && sudo apt upgrade every week or so. Install it and configure it with:

sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades

By default this only pulls security updates, which is what you want on a server. You can check that it’s actually doing something with sudo unattended-upgrades --dry-run --debug.

ufw (the “uncomplicated firewall”) is a frontend over iptables / nftables that’s pleasant to use and ships with Ubuntu. The “default deny” posture is three commands:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw enable

That’s it. Open more ports later as you actually need them (sudo ufw allow 443/tcp and so on). sudo ufw status verbose shows you what’s open and blocked. If you mess up and lock yourself out, you can always log in through the Hetzner console and fix it.

By default fail2ban writes ban rules straight into iptables. That works, but now you have two tools writing firewall rules and no single place to look when you want to know what’s actually blocked. Better: tell fail2ban to use ufw as its banning backend, so all your filtering state lives in one tool.

Install it and drop a /etc/fail2ban/jail.local:

sudo apt install fail2ban

# /etc/fail2ban/jail.local
[DEFAULT]
banaction = ufw
banaction_allports = ufw

bantime  = 1h
findtime = 10m
maxretry = 5

# Probably already covered in defaults, but just in case:
[sshd]
enabled = true

Then sudo systemctl enable --now fail2ban. Confirm bans land in ufw with sudo ufw status numbered, banned IPs show up as DENY IN rules at the top of the list. sudo fail2ban-client status sshd tells you who’s currently in the doghouse.

If you stop reading at this point and just install Docker, your firewall is now lying to you. This is the single biggest gotcha on a hardened Ubuntu host and it cost me an evening the first time I hit it.

The problem: when you publish a port with -p 80:80 (or its Compose equivalent), Docker doesn’t ask ufw, it writes its own rules into the DOCKER chain of iptables. Those rules sit upstream of the ufw-user-input chain, so they take effect first. Your ufw deny 80 does nothing. A container bound to 0.0.0.0:80 is reachable from the entire internet whether you “allowed” 80 or not.

You can verify this is happening on your own box with sudo iptables -L DOCKER -n after publishing a port.

Two ways out:

1. Bind containers to localhost only (-p 127.0.0.1:80:80) and put a reverse proxy on the host that’s the only thing actually listening on 0.0.0.0. Clean, but means you can’t just docker run public services casually.
2. Use ufw-docker, a small shim that rewrites the iptables setup so container traffic is actually subject to ufw rules, and gives you a ufw-docker allow command for granting access per container.

I use ufw-docker. Install Docker first via the official repo, then:

sudo wget -O /usr/local/bin/ufw-docker \
  https://github.com/chaifeng/ufw-docker/raw/master/ufw-docker
sudo chmod +x /usr/local/bin/ufw-docker
sudo ufw-docker install
sudo systemctl restart ufw

ufw-docker install edits /etc/ufw/after.rules to add a DOCKER-USER chain that defaults to deny, and routes container traffic through it. After that, granting access to a specific container looks like:

sudo ufw-docker allow nginx 443/tcp

Where nginx is the container name. You can also allow a port for all containers, or scope to a specific source, see ufw-docker help.

The thing to remember: once you’ve done this, removing a container doesn’t automatically remove its ufw-docker allow rules. They’re not tied to the container’s lifecycle. Run sudo ufw status occasionally and clean up stale entries.

A short and incomplete list of things you might want next, in rough order of how much I think they matter for a personal VPS:

• An automatic offsite backup of /etc, your compose files, and whatever Docker volumes hold data you care about. (I wrote about restic and B2 a few years back; still what I use.)
• A reverse proxy with automatic TLS in front of your Docker services. Caddy is the lowest-effort option; nginx + acme.sh if you want more control.
• SSH hardware key auth (yubikey-agent or similar) if you’re feeling fancy.
• AppArmor profiles for the things you actually expose.
• auditd if you want to be able to forensically reconstruct what happened later.

Most of those are individual posts on their own. Get the baseline in first.