Sysadministrivia

Linux, Lagers, and Late Nights

HOWTO: Verify SSH Fingerprints

Posted
Modified
Comments 0

First, if you haven’t hardened your SSH security, you should!

This article is intended to explain the difference in host key issues you’ll see in SSH and how to resolve them.

Verifying Hostkeys

Before digging into resolving both the case of missing hostkeys and changed hostkeys, you need to know how to verify hostkeys.

Identifying Hostkeys

First you need three pieces of information: the key algorithm, hash type, and hash. The hash type and the hash combine to make up the key fingerprint.

Given the following output when trying to SSH into a new server:

The authenticity of host 'foo.bar.baz (203.0.113.2)' can't be established.
ED25519 key fingerprint is SHA256:YofvnAXXKsVSsfLw/g72awhD0KnUCS/f3CSkf7VyEz0.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

or the following when trying to SSH to a server with changed host keys:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:JCR4xEgmwWkCCEIniPLysXX35XmDvF+m8eyar8Df6A4.
Please contact your system administrator.
Add correct host key in /home/YOURUSER/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/YOURUSER/.ssh/known_hosts:SOMELINENUMBER
ED25519 host key for 203.0.113.2 has changed and you have requested strict checking.
Host key verification failed.

The key fingerprints are, respectively, SHA256:YofvnAXXKsVSsfLw/g72awhD0KnUCS/f3CSkf7VyEz0 and SHA256:JCR4xEgmwWkCCEIniPLysXX35XmDvF+m8eyar8Df6A4. In these examples, the hash type is SHA256 (currently, only MD5 and SHA256 are supported). The hash of these keys are YofvnAXXKsVSsfLw/g72awhD0KnUCS/f3CSkf7VyEz0 and JCR4xEgmwWkCCEIniPLysXX35XmDvF+m8eyar8Df6A4 respectively.

The key algorithm for both above examples is ED25519. This more commonly is (SSH-)RSA, but I personally prefer using ED25519.

TIP: You can fetch all advertised host keys of a remote server by using ssh-keyscan. Just be sure to verify them!

Confirming Hostkeys

If you do not control the target machine, contact the administrator of the box and confirm the key fingerprint with them directly.

If you DO control the machine, connect to the target machine via e.g. IP-KVM (do not use SSH!) and compare the fingerprint your SSH client is getting against the fingerprints of the box. You can do this by finding your host public key and doing this: ssh-keygen -l -f <PUBLIC KEY FILE> -E <HASH TYPE> (e.g. ssh-keygen -l -f ssh_host_ed25519_key.pub -E sha256). The second field is the key fingerprint.

TIP: Usually, hostkeys are found in the /etc/ssh/ssh_host_<KEY ALGORITHM>_key.pub file. Your sshd_config file will specify where the private keys are if you’re using an alternative location. Usually the public keys are alongside the private keys and have the same filename plus a “.pub” suffix, but if not you can export public keys from the private by doing ssh-keygen -y -f <PRIVATE KEY FILE>.

Accepting New Hostkeys

The authenticity of host 'foo.bar.baz (203.0.113.2)' can't be established.
ED25519 key fingerprint is SHA256:YofvnAXXKsVSsfLw/g72awhD0KnUCS/f3CSkf7VyEz0.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

After SSHing to a server for the first time (or doing a keyscan as outlined above in the tip) and confirming the fingerprint matches, simply type yes at the prompt and hit enter.

TIP: Alternatively, instead of visually comparing them, you can copy the fingerprint you generated from the public hostkey or received from that server’s administrator and hit enter. If it matches the fingerprint of the key being offered by the server you’re connecting to, it will accept the key automatically and add it to your ~/.ssh/known_hosts file for future use. If it does NOT match, you will be prompted again (Please type 'yes', 'no' or the fingerprint:). If you are certain you have pasted the fingerprint correctly, you can be certain that the hostkey of the server you are trying to SSH to does not match what you expect and there may be a Man-in-the-Middle attack occurring.

TIP: At the cost of a minor loss in security, you can have SSH automatically add all new hostkeys to your known_hosts by adding StrictHostKeyChecking accept-new to your ~/.ssh/config file (or system-wide /etc/ssh/ssh_config).

Accept Changed Hostkeys

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:JCR4xEgmwWkCCEIniPLysXX35XmDvF+m8eyar8Df6A4.
Please contact your system administrator.
Add correct host key in /home/YOURUSER/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/YOURUSER/.ssh/known_hosts:SOMELINENUMBER
ED25519 host key for 203.0.113.2 has changed and you have requested strict checking.
Host key verification failed.

Wow. That’s a terrifying message, isn’t it? Keep in mind that it doesn’t guarantee that a MitM attack is occurring, you’ll get this message on a variety of perfectly acceptable conditions such as new keys being intentionally generated (like what you do while hardening SSH), a server rebuild/reinstall, etc. The IP or host/DNS entry hasn’t changed, but the actual OS has. Unfortunately, from a technological perspective it isn’t really possible for software (such as SSH) to differentiate this from a MitM attack so it errs on the side of caution.

After verifying the remote key fingerprint, the following two commands will (1) remove the cached copy of the hostkey you have locally and (2) add the new key to your local cache. (If you were trying to ssh foo.bar.baz, for instance, the below would be foo.bar.baz. See Verifying Hostkeys above to identify what to use for .)

ssh-keygen -R <REMOTE HOST>
ssh-keyscan -t <KEY ALGORITHM> >> ~/.ssh/known_hosts

You should then be able to SSH in. If you get messages about mismatched duplicate keys in your known_hosts, repeat the process for each duplicate host/IP address it finds.

TIP: It is recommended to back up your SSH hostkeys as part of your backup routines. They shouldn’t change often if ever, but having them in a snapshot will allow you to avoid this issue when rebuilding/restoring.

BONUS: The Stupid Way

This is stupid. Very, very stupid. But this method circumvents the protection that hostkey checking provides.

On the one hand, it means you’ll never be faced with a prompt to accept new keys (a smarter way of doing this is just using StrictHostKeyChecking accept-new) or messages about changed hostkeys.

On the other hand, you will LOSE ANY AND ALL PROTECTION AGAINST SPOOFING/MAN-IN-THE-MIDDLE ATTACKS that SSH hostkeys provide. YOU HAVE BEEN WARNED.

Sometimes you need to do stupid things. #YOLO.

To enable this for a specific user, add these options to ~/.ssh/config as the user.

To enable this system-wide, for extra YOLO, add these to /etc/ssh/ssh_config.

StrictHostKeyChecking no
UserKnownHostsFile /dev/null

Make wise decisions, not easy ones.

Author
Categories ,

Comments

There are currently no comments on this article.

Comment...

Enter your comment below. Fields marked * are required. You must preview your comment before submitting it.