Cannot connect to SSH: Connection closed by remote host

Written by - 0 comments

Published on - Listed in Linux Network


From my monitoring server I received several SSH alerts from another Linux server, running CentOS 6.3.
The alerts were meanwhile flapping between CRITICAL and OK:

[08:48:26] SERVICE ALERT: target-server;SSH;CRITICAL;HARD;3;Server answer:
[08:47:26] SERVICE ALERT: target-server;SSH;CRITICAL;SOFT;2;Server answer:
[08:46:26] SERVICE ALERT: target-server;SSH;CRITICAL;SOFT;1;Server answer:
[08:43:26] SERVICE ALERT: target-server;SSH;OK;HARD;3;SSH OK - OpenSSH_5.3 (protocol 2.0)
[08:37:26] SERVICE ALERT: target-server;SSH;CRITICAL;HARD;3;Server answer:
[08:36:26] SERVICE ALERT: target-server;SSH;CRITICAL;SOFT;2;Server answer:
[08:35:26] SERVICE ALERT: target-server;SSH;CRITICAL;SOFT;1;Server answer:
[08:26:26] SERVICE ALERT: target-server;SSH;OK;HARD;3;SSH OK - OpenSSH_5.3 (protocol 2.0)
[08:05:26] SERVICE ALERT: target-server;SSH;CRITICAL;HARD;3;Server answer:
[08:04:26] SERVICE ALERT: target-server;SSH;CRITICAL;SOFT;2;Server answer:
[08:03:26] SERVICE ALERT: target-server;SSH;CRITICAL;SOFT;1;Server answer:
[08:00:26] SERVICE ALERT: target-server;SSH;OK;SOFT;3;SSH OK - OpenSSH_5.3 (protocol 2.0)

This went on and on...

A manual ssh connection confirmed the alert:

ssh target-server
ssh_exchange_identification: Connection closed by remote host

I ran a tcpdump on the target server and whenever the connection was closed, the output was the following:

tcpdump -n "src host my.own.ip.address and dst port 22"
09:02:05.192513 IP my.own.ip.address.38586 > target-server.ssh: Flags [S], seq 2270934900, win 14600, options [mss 1380,sackOK,TS val 516633443 ecr 0,nop,wscale 7], length 0
09:02:05.193403 IP my.own.ip.address.38586 > target-server.ssh: Flags [.], ack 1633368629, win 115, options [nop,nop,TS val 516633443 ecr 1699421739], length 0
09:02:05.194575 IP my.own.ip.address.38586 > target-server.ssh: Flags [F.], seq 0, ack 2, win 115, options [nop,nop,TS val 516633443 ecr 1699421740], length 0

Interesting is the last entry with the flag F (FIN). So the server definitely sent the closing signal.

To debug SSH, I activated verbose logging in /etc/ssh/sshd_config and restarted /etc/init.d/sshd. I then tailed /var/log/secure and to my big surprise the following entries appeared:

Failed password for root from 144.0.0.34 port 39899 ssh2
Failed password for root from 144.0.0.34 port 39534 ssh2
Failed password for root from 144.0.0.34 port 39439 ssh2
Failed password for root from 144.0.0.34 port 39329 ssh2
Disconnecting: Too many authentication failures for root
[...]

netstat confirmed that there were at least 36 opened ssh connections. So whenever all available tcp slots for ssh were occupied, the server of course immediately closed the connection.
Within four hours, this IP tried 42929 times to log in as root. A typical brute force attack it seems.
After blocking the IP the monitoring (and therefore having all necessary ssh tcp slots available again), checks were OK again.


Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder