A spurious problems may occur if software who needs a fixed TCP port, finds that port in use. This topic explains some methods on how to find it out. More precisely, it describes how it was solved the issue of the Exalead server that stopped to perform searches after a reboot. However, the steps can be applied on any process that requires certain TCP ports to be free.

Troubleshooting steps are of Mauro Antonelli, Tera IST.

Note: the screenshots were taken after the problem was solved so they don't show the problem but simply where to look for it.

From the home of the Exalead admin page we checked which Exalead thread was not running (bottom of the page), and there was one, the searcherserver-ss0 which was looping on the "starting..." status:
 

 
Then we went into the Process logs of Exalead, and we added the log for the searchserver-ss0:
 

 
Scrolling the log we could see that there is an error about a resource in use. This made us think that one of the Exalead port was in use by another application.

So we ran the following command from the Command prompt window:

    netstat -ab > c:\exa.log

Opening the file in Notepad and looking at it, it was clear that port 10012 was in use by java.EXE.
 
[exa.EXE]
  TCP    0.0.0.0:10009          BTDB02:0               LISTENING
 [exa.exe]
  TCP    0.0.0.0:10011          BTDB02:0               LISTENING
 [java.EXE]
  TCP    0.0.0.0:10012          BTDB02:0               LISTENING
 [exa.EXE]
  TCP    0.0.0.0:10013          BTDB02:0               LISTENING
 
We contacted the customer and asked if they installed something new recently: they did. They installed a Symantec backup day before and they restarted the server this morning. We asked them to stop it and in the mean time I also killed the Java.exe process. After that the Exalead searcher could be started correctly.

Still to be investigated if the problem was really caused by the Symantec backup or if the Java was simply a child process of something that got stuck.