SharePoint Search Service Crash
What a way to be greeted by SharePoint on a Monday morning:
cannot connect to the configuration database
Damn. It was working Friday afternoon - something happened over the weekend.
I remoted into the box and checked that SQL Server was running, which it was.
Checked the application event log and noticed an error about space on C drive. Sure enough, 0 bytes free on C drive. This does not bode well!
SequoiaView to the resuce! I used it to scan all of C drive on our MOSS server and determine where the space was being used. It found a huge amount of small(ish) log files in C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG.
So I opened that folder and took a look at one of the log files. It was a stack dump from SQL Server with something about an access violation. Not good. Not good at all.
The access violation, according to the stack dump, started with a call to a stored procedure named proc_MSS_Crawl in our Search database. That clued me in that the problem was somehow rooted in SharePoint's indexer. I stopped the Windows service named "Office SharePoint Server Search" and noted that the little dump files stopped appearing. Of course, a few seconds later the service was restarted by a dependent service and the folder started filling up again.
So I disabled the Office SharePoint Server Search service (phew!) and started searching on Google for phrases from the log file. Eventually I stumbled upon this post on the MSDN forums: Weirdest MOSS Crawl problem ever!
It turns out that applying SQL Server SP2 to that guy's server fixed his problem, and guess what? It fixed mine too.
So anyway - an interesting morning. Things can only pick up from here.
Comments
# Andrew Tobin
3/12/2007 11:21 AM
Wait, your IT Support doesn't have alerts on the system that tell them when hard drives are starting to get to 90% full or something?
Weird ;)