473,326 Members | 2,102 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Why is OutOfMemoryException thrown?

I can't figure out why once or twice per week, my C# server application
throws OutOfMemoryExceptions. There is plenty of memory in the machine. As
far as I can tell, when the exception is thrown, only 400 megs of 1gig total
is in use.

The exception is thrown while trying to update List<>. Here's a bit of the
stack trace:

===
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException'
was thrown.
at System.Collections.Generic.List`1.set_Capacity(Int 32 value)
at System.Collections.Generic.List`1.EnsureCapacity(I nt32 min)
at System.Collections.Generic.List`1.Add(T item)
at Mosaic.Core.StatUtils.ExtractRORs(MiniPerfItem[] perfItems, Int32
startIdx, Int32 endIdx, Boolean useNAOnNonPosBal) in
C:\Development\Mosaic\Mosaic-AM\Mosaic.Core.Shared\Code\Utilities\Statistical\S tatUtils.cs:line
38
===

I have a lot of small objects in memory at the time of the exception but I'm
nowhere close to exhausting available RAM. Any ideas?

Oct 11 '07 #1
19 2008
Lamont Sanford <ya********@sanford.sonwrote:
I can't figure out why once or twice per week, my C# server application
throws OutOfMemoryExceptions. There is plenty of memory in the machine. As
far as I can tell, when the exception is thrown, only 400 megs of 1gig total
is in use.

The exception is thrown while trying to update List<>. Here's a bit of the
stack trace:

===
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException'
was thrown.
at System.Collections.Generic.List`1.set_Capacity(Int 32 value)
at System.Collections.Generic.List`1.EnsureCapacity(I nt32 min)
at System.Collections.Generic.List`1.Add(T item)
at Mosaic.Core.StatUtils.ExtractRORs(MiniPerfItem[] perfItems, Int32
startIdx, Int32 endIdx, Boolean useNAOnNonPosBal) in
C:\Development\Mosaic\Mosaic-AM\Mosaic.Core.Shared\Code\Utilities\Statistical\S tatUtils.cs:line
38
===

I have a lot of small objects in memory at the time of the exception but I'm
nowhere close to exhausting available RAM. Any ideas?
Do you have any idea how big the list is at the point of failure?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Oct 11 '07 #2
OutOfMemoryException really means "out of resources". Are you opening files
and not disposing them or something?

http://mrpmorris.blogspot.com/2006/1...maybe-not.html


Oct 11 '07 #3
Do you have any idea how big the list is at the point of failure?
>
At max it could get up to the tens of thousands, but certainly never over
100k...
Oct 11 '07 #4
OutOfMemoryException really means "out of resources". Are you opening
files and not disposing them or something?

http://mrpmorris.blogspot.com/2006/1...maybe-not.html
Interesting. I'll check out the article.

No, I don't make use of files to any major extent, but I do use a lot of
database connections. Maybe I'm not closing them properly?
Oct 11 '07 #5
Most likely - improperly closed / disposed connection objects means you run
out of Connection Pool connections, which can cause exceptions similar to
what you're seeing - especially if combined with other code.

1) use the same connection string througout your app.
2) open a connection just before you need to do database call.
3) close the connection immediately after - this returns it to the pool
4) be very careful with DataReaders as they hold open a connection.
-- Peter
Recursion: see Recursion
site: http://www.eggheadcafe.com
unBlog: http://petesbloggerama.blogspot.com
BlogMetaFinder: http://www.blogmetafinder.com

"Lamont Sanford" wrote:
OutOfMemoryException really means "out of resources". Are you opening
files and not disposing them or something?

http://mrpmorris.blogspot.com/2006/1...maybe-not.html

Interesting. I'll check out the article.

No, I don't make use of files to any major extent, but I do use a lot of
database connections. Maybe I'm not closing them properly?
Oct 11 '07 #6
Lamont Sanford <ya********@sanford.sonwrote:
Do you have any idea how big the list is at the point of failure?

At max it could get up to the tens of thousands, but certainly never over
100k...
That sounds unlikely to be the issue then.

Can I suggest that you use perfmon to look at both the heap and the
handle count of your process?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Oct 11 '07 #7
Lamont Sanford pisze:
I can't figure out why once or twice per week, my C# server application
throws OutOfMemoryExceptions. There is plenty of memory in the machine. As
far as I can tell, when the exception is thrown, only 400 megs of 1gig total
is in use.

The exception is thrown while trying to update List<>. Here's a bit of the
stack trace:

===
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException'
was thrown.
at System.Collections.Generic.List`1.set_Capacity(Int 32 value)
at System.Collections.Generic.List`1.EnsureCapacity(I nt32 min)
at System.Collections.Generic.List`1.Add(T item)
at Mosaic.Core.StatUtils.ExtractRORs(MiniPerfItem[] perfItems, Int32
startIdx, Int32 endIdx, Boolean useNAOnNonPosBal) in
C:\Development\Mosaic\Mosaic-AM\Mosaic.Core.Shared\Code\Utilities\Statistical\S tatUtils.cs:line
38
===

I have a lot of small objects in memory at the time of the exception but I'm
nowhere close to exhausting available RAM. Any ideas?
Maybe due to extensively allocating and freeing memory, your memory is
too much fragmentized ?

Regards
Oct 11 '07 #8
>
That sounds unlikely to be the issue then.

Can I suggest that you use perfmon to look at both the heap and the
handle count of your process?
Sure, I'll check it out. I've never used permon before. What signs/symptoms
should I be looking for?
Oct 12 '07 #9
Maybe due to extensively allocating and freeing memory, your memory is
too much fragmentized ?
How would I go about defragmenting it? Should I manually run a garbage
collection?
Oct 12 '07 #10
On Oct 12, 2:03 pm, "Lamont Sanford" <yaBigDu...@sanford.sonwrote:
Can I suggest that you use perfmon to look at both the heap and the
handle count of your process?

Sure, I'll check it out. I've never used permon before. What signs/symptoms
should I be looking for?
Handles or heap memory rising constantly would be generally bad - but
you can also add performance traces for various .NET aspects, which
can be handy.

Jon

Oct 12 '07 #11
"Lamont Sanford" <ya********@sanford.sonwrote in message
news:MN******************************@giganews.com ...
>
>Maybe due to extensively allocating and freeing memory, your memory is
too much fragmentized ?

How would I go about defragmenting it? Should I manually run a garbage
collection?

No, there is nothing you can do about a fragmented heap (both managed and
process heap), other than restart the server.
Of course it's up to you to prevent possible heap fragmentation as much as
possible.
Therefore, I would like to know what version of the Framework and OS you
are running this on?
How large are the Lists in bytes, if you say up to 100k, do you mean 100kb
or something else, how did you measure this size?
How many of these list do you possibly create simultaneously?
Are you using Sockets connections?
Do you explicitly pin objects, or do you call into unmanaged code?

Willy.

Oct 12 '07 #12
I got the same problem before. Then I rewrited the application by replacing
DataSet by SqlDataReader.

"Lamont Sanford" <ya********@sanford.sonwrote in message
news:b_******************************@giganews.com ...
>I can't figure out why once or twice per week, my C# server application
throws OutOfMemoryExceptions. There is plenty of memory in the machine. As
far as I can tell, when the exception is thrown, only 400 megs of 1gig
total is in use.

The exception is thrown while trying to update List<>. Here's a bit of the
stack trace:

===
System.OutOfMemoryException: Exception of type
'System.OutOfMemoryException' was thrown.
at System.Collections.Generic.List`1.set_Capacity(Int 32 value)
at System.Collections.Generic.List`1.EnsureCapacity(I nt32 min)
at System.Collections.Generic.List`1.Add(T item)
at Mosaic.Core.StatUtils.ExtractRORs(MiniPerfItem[] perfItems, Int32
startIdx, Int32 endIdx, Boolean useNAOnNonPosBal) in
C:\Development\Mosaic\Mosaic-AM\Mosaic.Core.Shared\Code\Utilities\Statistical\S tatUtils.cs:line
38
===

I have a lot of small objects in memory at the time of the exception but
I'm nowhere close to exhausting available RAM. Any ideas?



Oct 12 '07 #13
Try using LinkedList<instead of List<if you can. It won't be as
efficient at stopping memory fragmentation, but it will not call the
EnsureCapacity function, which, in essence, momentarily triples the
amount of memory used so that it can double the amount of memory used.
Okay, so it's not really "memory used" unless you're using structs.
It's memory used for pointers, which on a 32bit system for 100k should
400kilobytes -- not much! (Tripling that should be fine which makes me
wonder if it's a list of structs.)

Oct 12 '07 #14
Therefore, I would like to know what version of the Framework and OS you
are running this on?
I'm running .NET Framework 2.0 (VS 2005) on Win2003.
How large are the Lists in bytes, if you say up to 100k, do you mean 100kb
or something else, how did you measure this size?
I just meant that the size of the list (List<T>.Count) would never exceed
100,000. If type 'T' is a double and double requires 8 bytes, the total
required memory shouldn't exceed 800,000 bytes... and I have hundreds of
gigs free.
How many of these list do you possibly create simultaneously?
Only one can exist at a time, but these lists will be created over and over
again with different values.When a list is no longer needed, it isn't
explicitly Clear()'d, but it does fall out of scope so it should be GCd.
Are you using Sockets connections?
Yes, sockets are used. No more than 30 inbound connections are active at any
given time.
Do you explicitly pin objects, or do you call into unmanaged code?
What does "pin" mean? No, I don't make any calls into unmanaged code.
Oct 15 '07 #15
"Lamont Sanford" <ya********@sanford.sonwrote in message
news:dP******************************@giganews.com ...
>Therefore, I would like to know what version of the Framework and OS you
are running this on?

I'm running .NET Framework 2.0 (VS 2005) on Win2003.
>How large are the Lists in bytes, if you say up to 100k, do you mean
100kb or something else, how did you measure this size?

I just meant that the size of the list (List<T>.Count) would never exceed
100,000. If type 'T' is a double and double requires 8 bytes, the total
required memory shouldn't exceed 800,000 bytes... and I have hundreds of
gigs free.
No you don't? The virtual memory space attributed to your process is only
2GB (supposing W2K3 32 bit). That means that you cannot allocate more than
this space for managed objects minus the space allocated by the application
the runtime and the Framework data and code, this leaves you with less than
1.6GB (non contiguous) in the most optimal case at process creation time.
Now, 800 Kb is not a big deal, but as these objects are larger than 85Kb,
they end on the LOH and as this one is never compacted, you may end with a
highly fragmented heap, if you don't tput an eye on object life time.
>How many of these list do you possibly create simultaneously?

Only one can exist at a time, but these lists will be created over and
over again with different values.When a list is no longer needed, it isn't
explicitly Clear()'d, but it does fall out of scope so it should be GCd.
Yes, but that doesn't mean that only one can exist at a time, the GC doesn't
run deterministically.
>Are you using Sockets connections?

Yes, sockets are used. No more than 30 inbound connections are active at
any given time.
>Do you explicitly pin objects, or do you call into unmanaged code?

What does "pin" mean? No, I don't make any calls into unmanaged code.
Yes, you do (indirectly), there is no single managed application in the
world that don't call in unmanaged code. The socket classes need to pin the
send/reeive buffers when calling into the native Winsock API's. Pinning is
there to prevent the heap allocated object to move in memory when the GC
runs, however, pinning also prevents compactation of the GC heap.
Does it happen that these buffers have something in common with the above
mentioned List's?

Anyway, I would suggest you to inspect your memory allocation pattern by
running your code against a memory profiler.

Willy.
Oct 15 '07 #16
>I just meant that the size of the list (List<T>.Count) would never exceed
>100,000. If type 'T' is a double and double requires 8 bytes, the total
required memory shouldn't exceed 800,000 bytes... and I have hundreds of
gigs free.

No you don't?
Ooops, I meant hundreds of megs, not hundreds of gigs... sorry for the
typeo.

Thanks for the hints. I'll take your advice and hopefully I can get to the
bottom of this.
Oct 15 '07 #17

Hi Lamont,
What is the Database engine that your app is interacting with?
Bob
On Thu, 11 Oct 2007 13:30:16 -0500, "Lamont Sanford"
<ya********@sanford.sonwrote:
>I can't figure out why once or twice per week, my C# server application
throws OutOfMemoryExceptions. There is plenty of memory in the machine. As
far as I can tell, when the exception is thrown, only 400 megs of 1gig total
is in use.

The exception is thrown while trying to update List<>. Here's a bit of the
stack trace:

===
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException'
was thrown.
at System.Collections.Generic.List`1.set_Capacity(Int 32 value)
at System.Collections.Generic.List`1.EnsureCapacity(I nt32 min)
at System.Collections.Generic.List`1.Add(T item)
at Mosaic.Core.StatUtils.ExtractRORs(MiniPerfItem[] perfItems, Int32
startIdx, Int32 endIdx, Boolean useNAOnNonPosBal) in
C:\Development\Mosaic\Mosaic-AM\Mosaic.Core.Shared\Code\Utilities\Statistical\S tatUtils.cs:line
38
===

I have a lot of small objects in memory at the time of the exception but I'm
nowhere close to exhausting available RAM. Any ideas?

Oct 17 '07 #18
>
Hi Lamont,
What is the Database engine that your app is interacting with?
Bob
I'm using SQL Server 2005.
Oct 17 '07 #19
bob
OK.
Not what I thought.
I had weird problem with SQL Anywhere Version 9 that appeared as a
memory problem.
Sorry no other ideas.
Bob

On Wed, 17 Oct 2007 08:20:12 -0500, "Lamont Sanford"
<ya********@sanford.sonwrote:
>>
Hi Lamont,
What is the Database engine that your app is interacting with?
Bob

I'm using SQL Server 2005.
Oct 17 '07 #20

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Peter Aberline | last post by:
Hi all, We have written a C # .NET application and we're encountering memory problems in the form of System.OutOfMemoryException. Our application creates many thousands of objects in a...
0
by: Per Bergland | last post by:
After many woes, I finally managed to get a stack dump of my System Service (written in C#) that insists on crashing when launched at system boot time (see below on how to get this dump - I...
2
by: Fernando Casero | last post by:
Hi, I'm programming on Visual C# Express Beta 2 and I have the following code: class MyClass { int a = new int int b = new int public int SomeMethod() }
1
by: Ripul Handa | last post by:
Hi We are running IIS 5.0 cluster with cisco local director. We are running a website on 2 webservers and I have been observing that from past few days we have are getting this error message of...
1
by: SMG - Idealake | last post by:
Hi all, I am getting following error on my error, what could be the reason? Exception of type System.OutOfMemoryException was thrown. Description: An unhandled exception occurred during the...
2
by: Dave | last post by:
We just started getting this error message in our application today (stack trace below). From the OutOfMemoryException, I'm guessing it could be a memory leak. I'm making sure I'm closing all my...
1
by: Ashkan Daie | last post by:
Hi All, When trying to install a performance counter via InstallUtil I get the following exception: Creating performance counter category Enterprise Library Caching. An exception occurred...
8
by: =?Utf-8?B?UGlnZ3k=?= | last post by:
Hi to all, I am getting this System.OutOfMemoryException calling the Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(<stream>,<Obj>) method. The type of <streamis...
16
by: Lamont Sanford | last post by:
I can't figure out why once or twice per week, my C# server application throws OutOfMemoryExceptions. There is plenty of memory in the machine. As far as I can tell, when the exception is thrown,...
1
by: netwapps | last post by:
I have: Application Type: Web application with VB.net 1.1 WebServer: IIS6 Machine Server: Windows 2003 The User opens 4-5 browsers connecting to the application and clicks on the Autorefresh...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.