IIS 6 Hangs/Crashes

[xoise]

I'm the server administrator for a small business and I have one app pool that is very problematic. There is one web site in the app pool that receives some pretty heavy usage (40-50k visitors a day). However, the server has specs that I'm sure should be able to handle it, and was handling it well until a couple of months ago. Ever since then, we've had nothing but problems. The app pool hangs frequently - typically around every 20 minutes or so, if that.

As an initial attempt, I changed some settings in the app pool and in the metabase file, but nothing seems to work quite well enough. I tried adding an extra worker to the web garden, increasing thread request limits, etc. all without success. My initial instinct was that the problem is ASP code related, so I've spent quite a few days optimizing and fine tuning code everywhere.

As a temporary "fix", I've enabled a frequent 5 minute worker recycle on the app pool. This helps a lot, but does not FIX the problem.

To diagnose the problem, I've looked in the event viewer and I've run DebugDiag numerous times.

Some common errors/warnings I get in the event viewer are:
- A process serving application pool 'PSX Extreme' exceeded time limits during shut down. The process id was '3892'. (I'm pretty sure this means something went wrong with a recycle?)
- Faulting application w3wp.exe, version 6.0.3790.1830, faulting module unknown, version 0.0.0.0, fault address 0x05c07dc8.

The following are typical errors I get in DebugDiag:
- The following threads in w3wp.exe__PSX Extreme__PID__3892__Date__06_28_2008__Time_10_44_1 2PM__609__IIS_COM+ Hang Dump.dmp are displaying a message box. The call to display the message box originated from user32!NtUserWaitMessage+c.
The text of the message being displayed is:
( 11 )
1.47% of threads blocked
- A COM+ STA Activity Pileup has been detected in w3wp.exe__PSX Extreme__PID__5600__Date__06_28_2008__Time_10_44_1 7PM__609__IIS_COM+ Hang Dump.dmp. There is at least one activity bound to every COM+ STA ThreadPool thread, with the following threads having more than one activity bound:
20 21 22 23 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 44 45 46 47 48 49 50
See the COM+ STA ThreadPool Report for more detail.

Related Information:
-IIS6
-ASP
-MS SQL Server
-2 processors
-4gb RAM

I've already done TONS of reading on issues related to this matter. I've search Google for related information to all of my errors and warnings, how to analyze an IIS hang, read lots of articles by David Wang, but everything I try, I just can't seem to get it to work.

If anyone has ANY words of wisdom or can interpret the latest DebugDiag report on my app pool hang, I would forever be in your debt. Thanks for taking the time to read this.

http://www.xoise.com/CrashHangReport.mht

[kenobewan]

You may want to use IIS State to capture a crash, diagnose it, and fix
that problem -- and repeat if necessary

http://www.iisfaq.com/default.aspx?view=P197

Otherwise you may need to optimise your sql.

[xoise]

Thanks a lot for your reply. I've heard of IIS State before, but I was under the understanding that it was just very similar to DebugDiag. I'll definitely try it out though.

I'm hoping it's not an SQL problem though - because along with optimizing my scripts, it involved optimizing my SQL to make sure only one connection to the database is made and the db is accessed as few times as possible, etc.

[kenobewan]

Thanks a lot for your reply. I've heard of IIS State before, but I was under the understanding that it was just very similar to DebugDiag. I'll definitely try it out though.

I'm hoping it's not an SQL problem though - because along with optimizing my scripts, it involved optimizing my SQL to make sure only one connection to the database is made and the db is accessed as few times as possible, etc.

You are prob right about DebugDiag, but worth a shot.

As far as sql, better to make many small queries. Many connections itself isn't bad and may be better. Of course, if they aren't closed/ disposed there can be probs. Try having the connections only open for executes and reads though.

Try testing your site for speed, I use a good tool in http_watch, but there are others. Is your site slow in general? If your site is fast except, hopefully, one or a few pages are slow, focus on what is on going there. Do you need to use threads so the site doesn't become unusable if a query hangs? I assume you use try catch blocks.