473,325 Members | 2,771 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,325 software developers and data experts.

Memory usage issues

SDS
I am writing an ASP.NET application (in C#) that, as part of a
particular response, populates a MemoryStream object with binary data
that is being collected from a Process object's StandardOutput. This
data can be between 60MB and 100MB under normal circumstances. I write
this data to the response stream.

I'm noticing that calls to this particular page are causing W3WP to use
upwards of 60MB. When W3WP starts up, its using around 24MB or so.
Calls to simpler pages do not cause the memory usage to increase
significantly at all. So, it makes sense to me that this memory usage
is directly related to those stream objects.

In order to get memory usage back down, I've simplified code and tried
everything I can think of. I'm calling .Close() on the stream, I'm
calling .Close() on the BinaryWriter, I've tried casting the stream to
IDisposable in order to call .Dispose() on it. I've even made a call
to GC.Collect() as a last resort to see if it had any effect. Nothing
is working. I have a high level of confidence that my code is
releasing resources as it should (unless, of course, there is something
I can do outside of the methods I described above).

The desired result is to see the W3WP drop back down to (or somewhere
close at least) the 24MB it initially started up with. Are my
expectations unreasonable? What am I missing here? Is there something
I don't understand about the way garbage collection works?

Incidentally, subsequent calls to this same page do not cause the
memory usage to double/triple/etc. It seems to be holding tight at the
60MB mark or so (I can see the memory usage increase briefly and then
drop back down). So, for some reason, after the initial call to the
page in question, there is always this 40MB or so chunk that just
refuses to go away.

Thanks in advance! =)

Nov 17 '05 #1
5 3127
Inline

Willy.

"SDS" <ss*******@gmail.com> wrote in message
news:11********************@g14g2000cwa.googlegrou ps.com...
I am writing an ASP.NET application (in C#) that, as part of a
particular response, populates a MemoryStream object with binary data
that is being collected from a Process object's StandardOutput. This
data can be between 60MB and 100MB under normal circumstances. I write
this data to the response stream.

I'm noticing that calls to this particular page are causing W3WP to use
upwards of 60MB. When W3WP starts up, its using around 24MB or so.
Calls to simpler pages do not cause the memory usage to increase
significantly at all. So, it makes sense to me that this memory usage
is directly related to those stream objects.

In order to get memory usage back down, I've simplified code and tried
everything I can think of. I'm calling .Close() on the stream, I'm
calling .Close() on the BinaryWriter, I've tried casting the stream to
IDisposable in order to call .Dispose() on it. I've even made a call
to GC.Collect() as a last resort to see if it had any effect. Nothing
is working. I have a high level of confidence that my code is
releasing resources as it should (unless, of course, there is something
I can do outside of the methods I described above).

The desired result is to see the W3WP drop back down to (or somewhere
close at least) the 24MB it initially started up with. Are my
expectations unreasonable? What am I missing here? Is there something
I don't understand about the way garbage collection works?

No, this is the normal behavior and the result of how the CLR manages it's
memory pool (the managed heaps), whenever you create such large object (or a
large number of smaller objects), the CLR needs grab additional memory from
the process heap in order to store the object data in the large object heap.
After the object is released and collected by the GC, that extra allocated
memory is not returned to the OS as long as the OS is not requesting the CLR
to do so. The task of the GC is only to remove the garbage from the managed
heaps and to compact the objects inside the heaps, not to manage the heaps
themself, this is the task of the CLR in cooperation with the OS.
Incidentally, subsequent calls to this same page do not cause the
memory usage to double/triple/etc. It seems to be holding tight at the
60MB mark or so (I can see the memory usage increase briefly and then
drop back down). So, for some reason, after the initial call to the
page in question, there is always this 40MB or so chunk that just
refuses to go away.

Thanks in advance! =)

Nov 17 '05 #2
SDS
Very cool, I was expecting an answer along these lines. =)

So, my dev box has 2GB of RAM I believe. Obviously 60MB is not a
concern to me personally, but my concerns were more along the lines of
this being installed by others on boxes with substantially less RAM
(like 256MB for example, where 60MB *is* a big deal). Would it be safe
to say that the behavior I am observing may vary from system to system,
depending on the hardware configuration of that system?

Also, if watching the process memory utilization is not an indication
of whether or not I am cleaning up resources properly, what would be a
good way to monitor such things?

Thanks!

Nov 17 '05 #3
SDS,

You might want to check the performance counters for .NET. It will give
you information such as number of GC's, bytes retrieved per GC, and a whole
slew of other information (there are 12 categories alone for .NET, 2 of
which are specific to data providers, the other 10 related to the CLR, with
individual counters in each category).

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"SDS" <ss*******@gmail.com> wrote in message
news:11*********************@z14g2000cwz.googlegro ups.com...
Very cool, I was expecting an answer along these lines. =)

So, my dev box has 2GB of RAM I believe. Obviously 60MB is not a
concern to me personally, but my concerns were more along the lines of
this being installed by others on boxes with substantially less RAM
(like 256MB for example, where 60MB *is* a big deal). Would it be safe
to say that the behavior I am observing may vary from system to system,
depending on the hardware configuration of that system?

Also, if watching the process memory utilization is not an indication
of whether or not I am cleaning up resources properly, what would be a
good way to monitor such things?

Thanks!

Nov 17 '05 #4
Hi,
"SDS" <ss*******@gmail.com> wrote in message
news:11*********************@z14g2000cwz.googlegro ups.com...
Very cool, I was expecting an answer along these lines. =)

So, my dev box has 2GB of RAM I believe. Obviously 60MB is not a
concern to me personally, but my concerns were more along the lines of
this being installed by others on boxes with substantially less RAM
(like 256MB for example, where 60MB *is* a big deal). Would it be safe
to say that the behavior I am observing may vary from system to system,
depending on the hardware configuration of that system?


It will vary of course, if you have less memory the OS may reclaim the
memory sooner. What you can be sure is that if you have a memorystream of
60MB you will have that consuption no matter where you run it so the OS
would need to page to disk part of the memory used by other programs.
Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation

Nov 17 '05 #5

"SDS" <ss*******@gmail.com> wrote in message
news:11*********************@z14g2000cwz.googlegro ups.com...
Very cool, I was expecting an answer along these lines. =)

So, my dev box has 2GB of RAM I believe. Obviously 60MB is not a
concern to me personally, but my concerns were more along the lines of
this being installed by others on boxes with substantially less RAM
(like 256MB for example, where 60MB *is* a big deal). Would it be safe
to say that the behavior I am observing may vary from system to system,
depending on the hardware configuration of that system?

Also, if watching the process memory utilization is not an indication
of whether or not I am cleaning up resources properly, what would be a
good way to monitor such things?

Thanks!


Well, honestly, running asp.net on a 256MB box isn't exactly what I would
suggest anyway.
Another thing you shouldn't do is use a webserver as a means of transferring
huge chunk's of data like this, and in general in .NET you should try NOT to
allocate such huge "objects" at all, the reason for this is that they end on
the LOH which is never compacted by the GC, so the danger exists that you'll
end with a LOH that is so badly fragmented that you'll end with memory
allocation failures when there is still sufficient (total) memory available
but not as one contiguous block.
Also note that asp.net has a protection mecahnism to prevent excessive
memory consumption by bad behaving web applications, the memory limit can be
set to a certain percentage of the total memory in the machine config file.
Monitoring memory resources can best be done using permon, herewith you can
watch the OS counters for process memory and the CLR counters for GC memory
(hot normal and large heaps).

Willy.


Nov 17 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: George Gre | last post by:
Hi, I wrote a c# programme that listens to incoming TCP requests and services them. This programme is meant to be running as long as the server its installed on is running. So we assume for...
4
by: Douglas | last post by:
Hi, Any suggestions for estimating the amount of memory a program will use in advance, before it is run. Rules of thumb etc.? Is it then possible to warn a user that a program is likely to...
7
by: Rich Denis | last post by:
Hello, I have been trying to solve a mysterious memory leak problem and was hoping that you could help me out on my stuck point. First a bit of background. We have two app servers in an app...
16
by: JCauble | last post by:
We have a large Asp.net application that is currently crashing our production servers. What we are seeing is the aspnet_wp eat up a bunch of memory and then stop unexpectedly. Does not recycle. ...
2
by: Jenniflower | last post by:
Hi Gurus, Our system is using SqlServer 2005 on XP.( On my machine,only this application access SQLServer.) The sqlserver memory is configured to 128MB (Min)~512 MB(Max) After our system get...
5
by: Frank Rizzo | last post by:
Hello, I have a very frustrating issue I've been working on. I have a routine that does the following: 1. Load a large (declared local to the function) DataSet from the database. It contains...
1
by: Daniel | last post by:
i seem to be having memory issues, will deplying a release build instead of a debug build help with memory usage of my .net process?
10
by: Daniel Peterson | last post by:
I'm responsible for a pair of IIS 6 webservers that run our production ASP.Net application. We push code on a monthly basis, and about two weeks after our October code push, we started to run into...
22
by: Peter | last post by:
I am using VS2008. I have a Windows Service application which creates Crystal Reports. This is a multi theaded application which can run several reports at one time. My problem - there is a...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.