Hardening SSH on a New Server

July 14, 2026

system-administration ssh security

SSH is the front door to every server. A default sshd configuration allows password authentication, permits root login, and accepts connections from anyone. On an internet-facing server, brute force attempts start within minutes of the first boot. This post covers installing OpenSSH, deploying keys, hardening the sshd configuration, and setting up sudo with the wheel group.

Installing OpenSSH

On Gentoo

emerge -av net-misc/openssh
rc-update add sshd default
rc-service sshd start

On Debian

apt install -y ssh
systemctl enable sshd
systemctl start sshd

On Alpine

apk add openssh
rc-update add sshd default
rc-service sshd start

Verify it's listening:

ss -lnt sport = :22
State    Recv-Q   Send-Q   Local Address:Port   Peer Address:Port
LISTEN   0        128      0.0.0.0:22            0.0.0.0:*

Creating the Wheel Group

Before configuring SSH access restrictions, create the wheel group. This group will control both SSH login access (via AllowGroups) and sudo privileges.

groupadd wheel

Add your admin user to the wheel group:

usermod -aG wheel yourusername

On Gentoo, the wheel group typically exists already. On Debian and Alpine, you may need to create it.

Deploying SSH Keys

Key-based authentication is the minimum viable security posture. Password authentication should be disabled entirely once keys are in place.

Generating a Key

If you don't have a keypair yet, generate one on your local machine. Ed25519 is preferred for modern deployments:

ssh-keygen -t ed25519 -C "you@workstation"

For environments that require RSA compatibility:

ssh-keygen -t rsa -b 4096 -C "you@workstation"

Deploying the Public Key

On the server, create the .ssh directory with correct permissions and add your public key:

mkdir -p /root/.ssh
chmod 700 /root/.ssh

Add your public key to the authorized_keys file:

cat >> /root/.ssh/authorized_keys << 'EOF'
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... you@workstation
EOF
chmod 600 /root/.ssh/authorized_keys

For additional users, deploy to their home directories:

mkdir -p /home/yourusername/.ssh
cp /root/.ssh/authorized_keys /home/yourusername/.ssh/
chown -R yourusername:users /home/yourusername/.ssh
chmod 700 /home/yourusername/.ssh
chmod 600 /home/yourusername/.ssh/authorized_keys

Testing Before Locking Down

Open a second terminal and verify you can log in with the key before disabling password authentication. This is critical — if key auth fails and you disable passwords, you're locked out.

ssh -i ~/.ssh/id_ed25519 root@yourserver

Configuring sshd

The sshd configuration controls every aspect of SSH access. The goal is to disable all authentication methods except public key, restrict access to specific groups, and limit the attack surface.

Edit /etc/ssh/sshd_config:

# Protocol and cryptography
Protocol 2
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512

# Authentication
LoginGraceTime 30
PermitRootLogin prohibit-password
StrictModes yes
PubkeyAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no

# Access control
AllowGroups wheel root

# Security
IgnoreRhosts yes
X11Forwarding no

# Misc
PrintMotd no
UsePAM yes
AcceptEnv LANG LC_*
Subsystem sftp internal-sftp

Key Decisions

PermitRootLogin prohibit-password — allows root login with keys but not passwords. Use no if you want to force all access through a regular user with sudo. prohibit-password is a reasonable middle ground for servers where you need direct root access during emergencies.

AllowGroups wheel root — only users in the wheel or root group can log in via SSH. This is the primary access control mechanism. Any user not in these groups is rejected before authentication even starts.

PasswordAuthentication no — disables password login entirely. Combined with ChallengeResponseAuthentication no, this ensures only key-based access works.

Cipher and MAC selection — the listed ciphers and MACs are the strongest available in modern OpenSSH. ChaCha20-Poly1305 is fast on hardware without AES-NI, and the AES-GCM ciphers leverage hardware acceleration where available. The ETM (encrypt-then-MAC) variants are more secure than the non-ETM equivalents.

X11Forwarding no — disabled on headless servers. There's no X11 display to forward to.

Validating the Configuration

Before restarting sshd, validate the configuration file syntax:

sshd -T -f /etc/ssh/sshd_config

This parses the config and dumps the effective settings. Any syntax errors will be reported. If it outputs a wall of configuration values with no errors, the config is valid.

You can also check a specific setting:

sshd -T -f /etc/ssh/sshd_config | grep -i permitrootlogin
permitrootlogin prohibit-password

Restarting sshd

Important: Keep your current SSH session open. Open a new terminal and test the connection after restarting. If something is wrong with the config, you can fix it from the existing session.

On Gentoo and Alpine

rc-service sshd restart

On Debian

systemctl restart sshd

Test from a new terminal:

ssh -i ~/.ssh/id_ed25519 root@yourserver

If that works, try a password login to confirm it's rejected:

ssh -o PubkeyAuthentication=no root@yourserver
# Should see: Permission denied (publickey).

Setting Up sudo

With root login restricted, non-root users need a way to execute privileged commands. sudo provides this with audit logging.

Installing sudo

On Gentoo

emerge -av app-admin/sudo

On Debian

apt install -y sudo

On Alpine

apk add sudo

Configuring Passwordless sudo for Wheel

The simplest and most common configuration is to allow all wheel group members to run any command without a password. This is appropriate for servers where the wheel group membership itself is the access control gate.

Ensure the sudoers.d directory exists:

mkdir -p /etc/sudoers.d
chmod 750 /etc/sudoers.d

Edit the sudoers file using visudo (never edit /etc/sudoers directly):

visudo

Find the wheel group line and set it to:

%wheel ALL=(ALL) NOPASSWD: ALL

Alternatively, use lineinfile or sed to set it non-interactively:

sed -i 's/^# *%wheel ALL=(ALL:ALL) NOPASSWD: ALL/%wheel ALL=(ALL) NOPASSWD: ALL/' /etc/sudoers

Validate the sudoers file:

visudo -cf /etc/sudoers
/etc/sudoers: parsed OK

Testing sudo

Switch to a non-root user in the wheel group and verify:

su - yourusername
sudo whoami
root

If this returns root, sudo is working correctly.

Generating Server-Side SSH Keys

Servers often need SSH keys for automated operations — pulling from git repositories, rsync to backup destinations, or connecting to other servers in your infrastructure.

Create the .ssh directory if it doesn't exist:

mkdir -p /root/.ssh
chmod 700 /root/.ssh

Generate a key for a specific purpose:

ssh-keygen -t rsa -b 4096 -f /root/.ssh/id_rsa_deploy -N "" -C "deploy@$(hostname)"

The -N "" sets an empty passphrase, which is necessary for automated/non-interactive use. The comment includes the hostname for identification when the public key appears in authorized_keys on remote hosts.

Display the public key to copy to the remote host:

cat /root/.ssh/id_rsa_deploy.pub

Test connectivity after deploying the public key to the remote host:

ssh -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new \
  -i /root/.ssh/id_rsa_deploy root@remotehost "echo connected"
connected

The BatchMode=yes flag ensures ssh never prompts for input — it either connects with the key or fails immediately. StrictHostKeyChecking=accept-new accepts the host key on first connection but rejects changes on subsequent connections, which is the right balance between security and automation.

Summary

After completing these steps:

  • SSH only accepts key-based authentication
  • Only users in the wheel group (or root) can log in
  • Weak ciphers and MACs are disabled
  • sudo is available to wheel group members
  • Password authentication is completely disabled

The next step is setting up fail2ban to ban IPs that attempt brute force attacks against the hardened SSH configuration.