SSH Server Hardening

Why Harden SSH?

A default SSH server installation generates multiple host key types (RSA, ECDSA, Ed25519) and accepts a broad range of algorithms. Hardening reduces the attack surface by restricting to only the strongest option: Ed25519.

Host keys identify your server to connecting clients. If an attacker compromises a weak host key, they can impersonate your server (man-in-the-middle attack). Removing unused key types eliminates this risk.

Before you begin: Ensure you have an alternative access method (console, AnyDesk, IPMI) in case SSH becomes inaccessible during hardening. Always keep an existing SSH session open while making changes.

Prerequisites

SSH with 1Password infrastructure

Audit Your SSH Configuration

Start by reviewing what is currently in /etc/ssh/:

ls -la /etc/ssh/

You will likely see multiple host key files:

ssh_host_rsa_key / ssh_host_rsa_key.pub
ssh_host_ecdsa_key / ssh_host_ecdsa_key.pub
ssh_host_ed25519_key / ssh_host_ed25519_key.pub

Check your current sshd_config for host key directives:

grep -i "HostKey" /etc/ssh/sshd_config

On Ubuntu 24.04, also check for drop-in configs that may override the main file:

ls /etc/ssh/sshd_config.d/
grep -r "HostKey" /etc/ssh/sshd_config.d/

The main sshd_config includes Include /etc/ssh/sshd_config.d/*.conf at the top, so drop-in files can override your settings. Remove or edit any conflicting directives in that directory.

Restrict to Ed25519 Only

1. Edit the SSH Server Config

Open /etc/ssh/sshd_config:

sudo nano /etc/ssh/sshd_config

Set a single HostKey directive and remove or comment out the others:

# Host keys — Ed25519 only
HostKey /etc/ssh/ssh_host_ed25519_key

# Disable password authentication (use key-based auth)
PasswordAuthentication no

# Disable root login
PermitRootLogin no

2. Remove Unnecessary Host Keys

Back up and then remove RSA and ECDSA host keys:

# Backup first
sudo mkdir -p /etc/ssh/old_keys
sudo mv /etc/ssh/ssh_host_rsa_key* /etc/ssh/old_keys/
sudo mv /etc/ssh/ssh_host_ecdsa_key* /etc/ssh/old_keys/

3. Regenerate the Ed25519 Host Key (if needed)

If the Ed25519 host key does not exist or you want to start fresh:

sudo rm -f /etc/ssh/ssh_host_ed25519_key*
sudo ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""

4. Validate the Configuration

Always validate before restarting:

sudo sshd -t

If the command exits silently (no output), the configuration is valid.

5. Restart SSH

# Ubuntu / Debian
sudo systemctl restart ssh

# RHEL / Fedora / Arch
sudo systemctl restart sshd

Keep your current session open while restarting. Open a second terminal and verify you can still connect before closing the original session.

6. Change the Default Port (Optional)

Changing the SSH port from 22 to a non-standard port reduces noise from automated scanners:

# In /etc/ssh/sshd_config
Port 2222

This is not a security measure on its own — it only reduces log noise. Always combine with key-based authentication and disabled password login.

When using a non-standard port, known_hosts entries are stored as [hostname]:port, so clearing stale entries requires:

ssh-keygen -R "[your-server]:2222"

Firewall Configuration

UFW Basics

UFW (Uncomplicated Firewall) is the standard firewall on Ubuntu. Allow your SSH port before enabling it:

sudo ufw allow 2222/tcp comment "SSH"
sudo ufw enable
sudo ufw status verbose

Before enabling UFW, always add a rule for your SSH port first. If you skip this step, you will lock yourself out of remote access.

Docker and UFW: The Bypass Problem

Docker modifies iptables directly, bypassing UFW rules entirely. A container publishing a port is accessible from the internet even if UFW blocks that port. This happens because Docker inserts its own chains (DOCKER, DOCKER-USER) that are evaluated before UFW’s chains.

Securing Docker with DOCKER-USER

The DOCKER-USER chain is the correct place to add firewall rules that Docker will respect:

# Allow traffic from your local network to Docker containers
# Replace with your LAN CIDR (e.g., 192.168.1.0/24 or 10.0.0.0/24)
sudo iptables -I DOCKER-USER -s <your-lan-cidr> -j ACCEPT

# Allow established connections (required for outbound traffic)
sudo iptables -I DOCKER-USER -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Drop all other external traffic to Docker containers
sudo iptables -A DOCKER-USER -j DROP

Make the rules persistent across reboots:

sudo apt install iptables-persistent
sudo netfilter-persistent save

Important: Test these rules carefully. Overly restrictive DOCKER-USER rules can break container-to-container communication and outbound traffic from containers. Always verify your services still work after applying rules.

Verify the Hardening

From a client machine, connect and check which host key algorithm is used:

ssh -v user@your-server 2>&1 | grep "Server host key"

You should see:

debug1: Server host key: ssh-ed25519 SHA256:...

Troubleshooting

`sshd -t` fails with “Missing privilege separation directory”

Create the required directory:

sudo mkdir -p /run/sshd

`sshd -t` reports syntax errors

Common causes:

Duplicate directives — ensure each setting appears only once
Trailing whitespace or invalid characters
Include files with conflicting settings — check /etc/ssh/sshd_config.d/

# Check for conflicting drop-in configs
ls /etc/ssh/sshd_config.d/

`client_input_hostkeys: received duplicated ssh-ed25519 host key`

This warning means the same HostKey directive appears more than once in your sshd_config. Check for duplicates:

grep "^HostKey" /etc/ssh/sshd_config

Remove the duplicate line, validate with sshd -t, and restart.

Docker containers accessible despite UFW deny rules

If ufw status shows a port as blocked but the service is still reachable from outside, Docker is bypassing UFW via its own iptables chains. Use the DOCKER-USER chain as described in the Firewall Configuration section above.

Locked out after enabling UFW

If you enabled UFW without first allowing your SSH port, access the server via console, IPMI, or a remote desktop tool and run:

sudo ufw allow <your-ssh-port>/tcp
# Or disable UFW entirely to regain access
sudo ufw disable

Clients get `Host key verification failed`

After changing host keys, clients that previously connected will have the old key fingerprint cached in ~/.ssh/known_hosts. Remove the stale entry:

ssh-keygen -R your-server-hostname

Then reconnect and accept the new fingerprint.

Resources

🔗

sshd_config Manual man.openbsd.org

Complete reference for all SSH server configuration options

🔗

SSH Audit github.com

CLI utility to audit SSH server security configuration