Troubleshooting ssh Connections

Posted by JD 08/17/2015 at 18:02

This is not meant for complete noobs. Hints for tools are provided only. If you don’t know the tool already, RTFM.

Most of these steps really aren’t necessary – they are included just to see if something commonly addressed automatically has been screwed by you or the network guys. It is almost never the network, so be nice before accusing others for your mistake.

Let’s get started.

Steps

  1. Verify the ssh server is running/installed – service ssh status and dpkg -l|grep openssh
  2. See if something simple is preventing the connection by using verbose settings from the client
    ssh -vvv userid@serverIP
  3. Verify the firewall isn’t blocking it – iptables -L
    • server-side firewall – lsof
    • client-side firewall – lsof
    • any edge routers blocking (this can be ingress and egress routers) – have to login to each router and look for blocks.
  4. Verify IP connection is working – ping IP
  5. Verify name resolution is working – ping hostname
  6. Check remote ssh server directory and file permissions are strict.
    • ~/.ssh/ must be 700 and owned by the remote userid
    • ~/.ssh/* – all files should be 600 and owned by the remote userid – especially authorized_keys, authorized_keys2, config, id_rsa, id_ed25519, known_hosts.
    • Public key files can be 644 …id_ed25519.pub & id_rsa.pub – public keys can be less strict.
  7. Test from localhost – ssh localhost – did that work?
  8. Test from another system on the same LANssh remoteUser@LAN-IP – did that work?
  9. Test from another system on the same LAN using the hostname – ssh remoteUser@LAN-hostname – did that work?
  10. Test over the internet – ssh remoteUser@WAN-IP – did that work?

The tests above will isolate the type of issue. If the default configuration files have been modified, put them back to the original settings and try again.

If you are using passwords, try ssh-keys. If you are using keys, try passwords (be certain to put back the keys after!!!). Passwords should NEVER be used over the internet. ssh-keygen, ssh-copy-id are some important tools.

To setup more secure keys, use:

ssh-keygen -t ed25519
ssh-copy-id -i ~/.ssh/id_ed25519.pub userid@remote

Be aware that not all devices (older routers) support ed25519 keys.

All server-side settings belong in /etc/ssh/sshd_config.
Client-side settings belong in either /etc/ssh/ssh_config or ~/.ssh/config Review those settings to see if any would block ssh for the userid, from the subnet or from the IP address the client is trying to connect. The manpages for each file are very good. man sshd_config for details. sshd is very powerful.

Check the tcp-wrappers settings. /etc/hosts_allow and hosts_deny. These aren’t used much anymore, but they were a staple of security settings for 20+ yrs. These days, people tend to use the system firewall and/or the sshd_config for this stuff due to the added flexibility. For example, allow passwords, but only from the LAN and mandate ssh-keys from everywhere else using the the Match Address options in sshd_config. For example:

# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no
Match Address 172.21.21.0/24,172.22.21.0/24
PasswordAuthentication yes

Lots of assumptions above

  • There is an account setup for ssh access AND you have the credentials necessary to access it.
  • No networking issues exist. That is addressed in another article.
  • server has a static IP on the LAN.
  • default port 22/tcp, is used.
  • double NAT adds complexity. If you use this, you need to handle those added complexities.
  • From the LAN means from the same subnet, without any network-based firewalls in the way.
  • Name resolution is working on the LAN – that can be DNS, /etc/hosts. I suppose avahi/zeroconf might work, but I remove that on every system.
  • router WAN ports for ssh forward to internal LAN server IP port 22/tcp. Feel free to use port translation from a different port on the WAN side of the router to the LAN side. Some netgear home routers do not support this feature.
  • ssh security is addressed in another article.
  • ssh capabilities are addressed in another article.

Don’t forget that ssh and sshd have verbose methods. The manpage for each explains. ssh -vvv for sshd – there is a debug option.

Does this cover most things for ssh troubleshooting?
What is missing?