Hi,
I have following type of scenario (also explained here
http://blogs.aspadvice.com/joteke/ar...1/10/2196.aspx )
We have problematic web server (wink2 Standard, 1.5GB of physical memory,
SQL 2000 in same box, framework 1.0 and 1.1 installed, apps use mainly 1.0)
whose aspnet_wp.exe's memory consumption increases slowly but surely, leading
to process restart eventually and then again recollecting memory etc etc
Size of aspnet_wp raises to somewhere 700-750MB, SQL stays somewhere 500MB,
CPU util rarely over 15%
Server has 32 web sites and I'd want to know which one of them is taking
most of resources (remember, every web site is one AppDomain inside process
memory)
There's no significant difference when checking basic ASP.NET Perf counters
of these sites, which would tell right away which site consumes the
resources. All sites have quite steady traffic, and one of them is such that
very big files (even over 200MB) can be up/downloaded but they don't still
indicate anything we could use as argument for sure.
However counters for aspnet_wp looked like following at one point:
Object: .NET CLR Loading
================
aspnet_wp
Assembly Search Length 0.000
Bytes in Loader Heap 46673920.000
Current appdomains 33.000
Current Assemblies 7120.000
Current Classes Loaded 25832.000
Rate of appdomains 0.000
Rate of Assemblies 0.000
Rate of Classes Loaded 0.000
Object: .NET CLR Memory
aspnet_wp
========
# Bytes in all Heaps 238191776.000
# GC Handles 115407.000
# Gen 0 Collections 20428.000
# Gen 1 Collections 7607.000
# Gen 2 Collections 897.000
# of Pinned Objects 0.000
# Total committed Bytes 275623600.000
# Total reserved Bytes 469761712.000
Finalization Survivors 4.000
Gen 0 heap size 28400000.000
Gen 1 heap size 1674480.000
Gen 2 heap size 95579928.000
Large Object Heap size 112537368.000
Promoted Finalization-Memory from Gen 0 56.000
Promoted Finalization-Memory from Gen 1 0.000
Promoted Memory from Gen 0 1404796.000
Promoted Memory from Gen 1 0.000
Object: Process
=========
aspnet_wp
Private Bytes 633970688.000
Virtual Bytes 1851805696
Working Set 648499200.000
What bothers me is the size of large object heap and Gen 2 Heap as well as
why the process size is so major when bytes in all heaps is somewhere 213MB?
Also could the count of assemblies and classes just be the reason?
When I run vadump on server as is demonstrated
(http://www.theserverside.net/blogs/s...ngMemoryLeaks), it
complains about too big size of the working set (truncating buffers which go
above 65535), so it shows me only 0's as result. I have to admit that I suck
with this sort of performance solving/digging type of stuff, so any pointers
would be greatly appreciated.
Is there something clear I haven't seen (my memory leaks)? Problems in code
are just as possible as anything else, but trying to dig 30 apps isn't very
tempting task.
Here's what I get with vadump
====================
C:\Documents and Settings\Teemu>vadump -sop 4912
Too many working set buffer entries (158507) - truncating to 65535
Catagory Total Private Shareable
Pages KBytes KBytes KBytes
Page Table Pages 0 0 0 0
Other System 0 0 0 0
Code/StaticData 0 0 0 0
Heap 0 0 0 0
Stack 0 0 0 0
Teb 0 0 0 0
Mapped Data 0 0 0 0
Other Data 65535 262140 262140 0
Total Modules 0 0 0 0
Total Dynamic Data 65535 262140 262140 0
Total System 0 0 0 0
Grand Total Working Set 65535 262140 262140 0
Module Working Set Contributions in pages
Total Private Shareable Module
Heap Working Set Contributions
0 pages from Process Heap (class 0x00000000)
0x00130000 - 0x00230000 0 pages
0x17630000 - 0x17730000 0 pages
0x1A190000 - 0x1A390000 0 pages
0x24300000 - 0x24700000 0 pages
0x39930000 - 0x3A130000 0 pages
0x45F80000 - 0x46F80000 0 pages
0x6E2A0000 - 0x702A0000 0 pages
0 pages from UNKNOWN Heap 0 (class 0x00008000)
0x00230000 - 0x00240000 0 pages
....
....
I just put perf counters on for those sites we still suspect that might have
something to do with the problem (two most frequently used ones) plus then
all these aspnet_wp level counters, and I run them for a day or two to get
more accurate data on server's behaviour.
Any pointers or hints how I could take the memory footprint on appdomain
basis plus anything else you might have, would be greatly appreciated.
--
Teemu Keiski
ASP.NET MVP, AspInsider
Finland, EU
