Network service troubleshooting and port scanning

At a former workplace, the DB2 team were furious because they couldn’t connect to the database remotely. They have been constantly trying to telnet into a port on the server and they couldn’t. They server wasn’t listening on the port they were trying to telnet into. They have rebooted the server twice and nobody thought of checking the listening ports. The solution was a simple restart of the DB service.

Ping the server. If the server does not respond to ping, access the server locally by way of RDP, IMM , RSA ,CIMC or UCS Manager KVM Console and troubleshoot from there. If the server does reply to ping, scan the port.

TCP port

nc -vz   server_ip port_number

Example of a successful scan

nc -vz server.example.com 443
Connection to server.example.com 443 port [tcp/https] succeeded!

Example of an unsuccessful scan

nc -vz server.example.com 443
nc: connect to server.example.com port 443 (tcp) failed: Connection timed out

UDP port

nc -uvz   server_ip port_number

Example of a successful scan

nc -vzu server.example.com 514
Connection to server.example.com 514 port [udp/syslog] succeeded!

nc is readily available on Linux and ESXi. On Windows, I use MobaXterm which has nc. You have probably been using telnet and you can continue to do so. But I don’t know how to scan UDP port with telnet, if you know please leave a comment.

If the server does reply to ping but port scan fails, access the server and check if the server is listening for incoming connection on the port.

Linux

netstat -ltupn

ESXi

esxcli network ip connection list

Windows

netstat -a

If the server is not listening on the port, check the service/daemon is running. If the service is running and the server is listening on the port, run a packet capture and lookout for incoming connection.

Example 1. Capturing nfs traffic on ESXi

tcpdump-uw -i vmk1 dst nas.example.com | grep nfs

11:54:59.228779 IP truncated-ip - 8450 bytes missing! server.example.com.925 > nas.example.com.nfs: Flags [.], ack 40445, win 512, options [nop,nop,TS val 557074583 ecr 557506100], length 8480
11:54:59.228801 IP truncated-ip - 5762 bytes missing! server.example.com.925 > nas.example.com.nfs: Flags [.], ack 40445, win 512, options [nop,nop,TS val 557074583 ecr 557506100], length 5792
11:54:59.228951 IP truncated-ip - 11658 bytes missing! server.example.com.925 > nas.example.com.nfs: Flags [.], ack 40445, win 512, options [nop,nop,TS val 557074583 ecr 557506100], length 11688
11:54:59.229095 IP truncated-ip - 12794 bytes missing! server.example.com.925 > nas.example.com.nfs: Flags [.], ack 40445, win 512, options [nop,nop,TS val 557074583 ecr 557506100], length 12824
11:54:59.229223 IP truncated-ip - 11554 bytes missing! server.example.com.92

Example 2. Capturing ssh traffic on ESXi

tcpdump-uw -i vmk0  host esxi.example.com and port 22

Example 3. This will capture every traffic on eth0

tcpdump -i eth0

Most likely you will have to use WireShark on Windows.

If the service is not running, start it. If it fails to start, start digging into the log. In most cases, there are clues and hints in the log.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s