Net runtime crash

[ekz]

Hi guys
I have a very weird bug and will welcome gracefully any help
We developed ASP.Net application that run on production server, Net
Framework 1.1 , IIS 6.0, the application is huge, having hundreds of
user controls and worked under massive requests load : 200-1000
connected sessions and millions of page views per day
The application is stable; the only and urgent problem is that:

About 10 times per day, more frequently at pick hours, we see this
error in windows event log:

Event Type: Error
Event Source: .NET Runtime
Event Category: None
Event ID: 0
Date: 21/12/2005
Time: 18:42:28
User: N/A
Computer: XXX
Description:
The description for Event ID ( 0 ) in Source ( .NET Runtime ) cannot be
found. The local computer may not have the necessary registry
information or message DLL files to display messages from a remote
computer. You may be able to use the /AUXSOURCE= flag to retrieve this
description; see Help and Support for details. The following
information is part of the event: .NET Runtime version 1.1.4322.2300-
Setup Error: Failed to load resources from resource file
Please check your Setup.

After this the worker process that connected to application pool is
recycled and application is extremely slowly till it loads all
resources and updated all cache entries (we use server cache
(HttpContext.Cu rrent.Cache) to save expensive queries results), it is
unacceptable for our client.

The description of event is very general, and all my Google research
can't help me to find helpful information about source of event.

I do not believe that somebody can tell me exactly the source of the
problem,
But any instructions about what I can to do research the problem:
debugging tools, some tools to get more descriptive event source
description, general tips how to research this kind of problem would be
very, very welcome

Thanks a million for any help

[Rahul Soni]

Hi,

As you already it is a very generic error and the 2 (quite often) reasons
why you could see this issue is happening is because of...
1) Security issues -> Try using Filemon and Regmon utility from
www.sysinternals.com

2) Some corrupt files in your .NET framework. Uninstall the Framework and
re-install it to see if that helps...

HTH
--
Thanks,
Rahul Soni
"ek*@hotbox .ru" wrote:

Hi guys
I have a very weird bug and will welcome gracefully any help
We developed ASP.Net application that run on production server, Net
Framework 1.1 , IIS 6.0, the application is huge, having hundreds of
user controls and worked under massive requests load : 200-1000
connected sessions and millions of page views per day
The application is stable; the only and urgent problem is that:

About 10 times per day, more frequently at pick hours, we see this
error in windows event log:

Event Type: Error
Event Source: .NET Runtime
Event Category: None
Event ID: 0
Date: 21/12/2005
Time: 18:42:28
User: N/A
Computer: XXX
Description:
The description for Event ID ( 0 ) in Source ( .NET Runtime ) cannot be
found. The local computer may not have the necessary registry
information or message DLL files to display messages from a remote
computer. You may be able to use the /AUXSOURCE= flag to retrieve this
description; see Help and Support for details. The following
information is part of the event: .NET Runtime version 1.1.4322.2300-
Setup Error: Failed to load resources from resource file
Please check your Setup.

After this the worker process that connected to application pool is
recycled and application is extremely slowly till it loads all
resources and updated all cache entries (we use server cache
(HttpContext.Cu rrent.Cache) to save expensive queries results), it is
unacceptable for our client.

The description of event is very general, and all my Google research
can't help me to find helpful information about source of event.

I do not believe that somebody can tell me exactly the source of the
problem,
But any instructions about what I can to do research the problem:
debugging tools, some tools to get more descriptive event source
description, general tips how to research this kind of problem would be
very, very welcome

Thanks a million for any help

[SharpSmith]

Thanks Rahul
we already used filemon and the outputs looks ok
but reinstalling framework is an interesting idea

many thanks
will keep you updated regarding process

EV

--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[SharpSmith]

hey Rahul,
if i will deploy the application in debug mode
what do you think, will it write more detailed info in event log

Thanks a million

EV
--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[Rahul Soni]

Well, deploying the application in debug mode sounded like an interesting
stuff to me initially. But, it is not a recommended practice for one thing
and more, since it is not related to ASP.NET apps (instead looks more related
to the .NET Runtime in general), I wonder if that would help!!

Are you able to reproduce this error by any means?
How often does it happen?
Did you try running the app in a different app pool and check if the issue
happens in the other pool as well?
Also, did you try isolating the application which you think could be causing
it?

--
Thanks,
Rahul Soni

[A problem well stated is a problem half solved!!]


"SharpSmith " wrote:

hey Rahul,
if i will deploy the application in debug mode
what do you think, will it write more detailed info in event log

Thanks a million

EV
--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[SharpSmith]

hi Rahul
i'm very appriciate your help
agree that deploying in debug isn't good, but at that moment i'm ready
for everything that might help to find the origin of the problem

regarding reproducing the error : i cant catch it and it can take weeks
of work cause the app is realy huge

the error occures 2 or 3 times at night time and
can appear 4 times in a hour during pick hours when we have a thousand
of visitors, so may be it is related to site load

the app runs at dedicated app.pool and i can't separate that app for
smaller units

maybe you are familiar with some debugging tool that i can attach to
process to see the crash source description ?

1000000 of thanks
EV
--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[Rahul Soni]

I would be very glad if I could help you point towards the cause of the error.

As you caught it quite right, it seems to be load related stuff. Let me ask
this...

1) Did you try using Perfmon logs? If not, do the following...

Create a Performance Monitor log file.
1. Expand "Performanc e Logs and Alerts"
2. Right Click on "Counter Logs"
3. Choose "New Log Settings..."
4. Enter a descriptive name
5. Note the log file location for later (or go to the "Log Files" tab and
change the location)
6. Click the "Add" button
7. Click the "All Counters" and "All Instances" radio buttons
8. Select the following from the "Performanc e Object" dropdown, being sure
to "Add" each one as you select it:
• Process
• Thread
• Processor
• Memory
• .NET CLR Data
• .NET CLR Exceptions
• .NET CLR Interop
• .NET CLR Jit
• .NET CLR Loading
• .NET CLR LocksAndThreads
• .NET CLR Memory
• .NET CLR Networking
• .NET CLR Remoting
• .NET CLR Security
• ASP.NET
• ASP.NET Applications
9. Click "Close"
10. Click "OK"

See how these logs behave... For Perfmon analysis, you can use the following
link...
http://msdn.microsoft.com/library/de...sForASPNET.asp

Do keep a tab on the PID of your worker process and see if that changes
whenever the error happens. If it does, then it is a crash and we can proceed
further accordingly. Let me know of the output and about PID stuff or if you
have any questions.

Hope that helps.

--
Thanks,
Rahul Soni

[A problem well stated is a problem half solved!!]


"SharpSmith " wrote:

hi Rahul
i'm very appriciate your help
agree that deploying in debug isn't good, but at that moment i'm ready
for everything that might help to find the origin of the problem

regarding reproducing the error : i cant catch it and it can take weeks
of work cause the app is realy huge

the error occures 2 or 3 times at night time and
can appear 4 times in a hour during pick hours when we have a thousand
of visitors, so may be it is related to site load

the app runs at dedicated app.pool and i can't separate that app for
smaller units

maybe you are familiar with some debugging tool that i can attach to
process to see the crash source description ?

1000000 of thanks
EV
--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[SharpSmith]

Thanks Rahul
i'm out of office now, but i will test your suggestion first thing when
i'll back to work
regarding PID, i sure that it is changed after every new .Net Runtime
entry in event log
and we now seriously checking the option to reinstall Framework, even
that isn't simple for windows server 2003 and will cause down time for
the site
have to solve it

again, thanks a million for your help, i'll take care to update your if
i'll finally fix it

--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[Rahul Soni]

Sure!!

Also if you are sure that PID is getting changed each time, then it must be
a crash and you should be able to catch it using the following method.

1. Go to the link http://beta.microsoft.com.
2. Login in with your passport id.
3. Login using the guest ID "DebugDiag" (it’s case sensitive)
4. Click on the "IIS Debug Diagnostic Tool" link.

Configure DebugDiag
=============== ====
Configure DebugDiag to capture the memory dump when the ExitProcess command
is called. This will ensure that we capture the memory dump when the
faulting thread is terminating the process.

1. Open DebugDiag
2. On the Rules tab, click Add Rule
3. Select Crash and click Next
4. Select "All IIS Processes" and click Next
5. Under Advanced Settings, click Breakpoints
6. Click Add Breakpoint...
7. Select KERNEL32!ExitPr ocess and change Action Type to Full UserDump
8. Click OK
9. Click Save and Close
10. Click Next through the rest of the wizard

Now, let it run for a while and wait for the issue to happen. Once that
happens, most probably you should be having the dump files created in the
respective folders. Let me know if you get any files with .dmp extension
created in the C:\Program Files\IIS Resources\Debug Diag\Logs\<Name of Rule>
folder

--
Thanks,
Rahul Soni

[A problem well stated is a problem half solved!!]


"SharpSmith " wrote:

Thanks Rahul
i'm out of office now, but i will test your suggestion first thing when
i'll back to work
regarding PID, i sure that it is changed after every new .Net Runtime
entry in event log
and we now seriously checking the option to reinstall Framework, even
that isn't simple for windows server 2003 and will cause down time for
the site
have to solve it

again, thanks a million for your help, i'll take care to update your if
i'll finally fix it

--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com

[SharpSmith]

Hey Rahul,

My name is Aviad, im replacing EV for taking care of the issue as he has
to get back into writing code :)

First let me thank you for all the references so far, i appriciate it
alot.

And now down and dirty with business, we have a dump file generated by
DebugDiag, it is huge (around 500mb), but the content in that file
doesnt seem clear to me as it all looks like core function calls...

What do we do from here?

Thanks again,

Aviad

--
Sent via .NET Newsgroups
http://www.dotnetnewsgroups.com