Getting an out of memory error

[RD]

I have a VBV.NET application that runs on a timer to do a job at one hour
intervals.
Basically the sequence is as follows
the timer event fires
1- the timer get turned off
2- the code then checks to see if we're just changed hour from when the
previous transfer session took place
IF yes
the code executes an FTP transfer from an FTP site.
When the transfer is completed the file received is parsed (it's a comma
separated ASCII file) line by line and the dat is transferred to SQL server
files.
Remember the hour part of the current transfer session time
end if
Turn the timer on again

All the code can execute faultlessly for a week doing a transfer every hour
on the hour or it can try the first one and stop dead in it's tracks with an
out of memory error message. Never at the same place but always the same
code 80004008. We had logs running tracing the program execution and writing
every procedure to text files and nothing gives us an inkling that it's
going to fail.

All through the code we've been very carefull to close every ADO .Net
connection every time we open one and tried to always set unused objects to
nothing explicitly. All I can come up with as a cause is the new garbage
collector is going out on an illegal strike <GGGG>.

This has been bugging me for over two months in an operational test phase
and I'm wondering if there is a practical way to trace what is happening
with memory useage over time, keeping a record that we can correlate to when
the errors and trying to figure out exactly why an app that runs alone
usoing about 15% of the available RAM on a machine suddenly and unexpectedly
runs out of memory and does so in a totally unpredictable way.

Any help would be immensely appreciated.

MS if you're monitoring this thread, do you think there might be <shudder> a
<NO NO DON'T SAY IT> bug in the garbage collector < SORRY, SORRY MOST ABJECT
APOLOGIES>. What can I do to make this relaible. This is a vital busines
process that must run unattended 24 7 365.

Bob

[Tom Shelton]

On 2004-03-04, RD <no****@nospam.net> wrote:

I have a VBV.NET application that runs on a timer to do a job at one hour
intervals.
Basically the sequence is as follows
the timer event fires
1- the timer get turned off
2- the code then checks to see if we're just changed hour from when the
previous transfer session took place
IF yes
the code executes an FTP transfer from an FTP site.
When the transfer is completed the file received is parsed (it's a comma
separated ASCII file) line by line and the dat is transferred to SQL server
files.
Remember the hour part of the current transfer session time
end if
Turn the timer on again

All the code can execute faultlessly for a week doing a transfer every hour
on the hour or it can try the first one and stop dead in it's tracks with an
out of memory error message. Never at the same place but always the same
code 80004008. We had logs running tracing the program execution and writing
every procedure to text files and nothing gives us an inkling that it's
going to fail.

All through the code we've been very carefull to close every ADO .Net
connection every time we open one and tried to always set unused objects to
nothing explicitly. All I can come up with as a cause is the new garbage
collector is going out on an illegal strike <GGGG>.

This has been bugging me for over two months in an operational test phase
and I'm wondering if there is a practical way to trace what is happening
with memory useage over time, keeping a record that we can correlate to when
the errors and trying to figure out exactly why an app that runs alone
usoing about 15% of the available RAM on a machine suddenly and unexpectedly
runs out of memory and does so in a totally unpredictable way.

Any help would be immensely appreciated.

MS if you're monitoring this thread, do you think there might be <shudder> a
<NO NO DON'T SAY IT> bug in the garbage collector < SORRY, SORRY MOST ABJECT
APOLOGIES>. What can I do to make this relaible. This is a vital busines
process that must run unattended 24 7 365.

Bob

Bob...

There was a bug in the 1.0 GC. It failed to deal with the large object
heap correctly (objects over about 80K in size). So, if you had large
arrays, you could end up running into memory problems. Are you using
1.0 or 1.1? As for a problem with the 1.1 GC, I haven't heard of one
and I have a windows service that has been running flawlessly for quite
some time that talks to an sql server database...

By the way, explicitly setting objects to nothing on the local scope
isn't really going to do anything for you :). Are you sure you are
calling Close/Dispose on all your objects that implement these methods?

Another place to look is if your using any COM components or P/Invoke
calls. Those are good places for memory leaks. You might want to look
into the Performance counter classes. They would probably give you a
much better means of monitoring your applications memory useage...

--
Tom Shelton [MVP]
Powered By Gentoo Linux 1.4
Help me, I'm a prisoner in a Fortune cookie file!

[RD]

Thanks Tom,
I went thru every single procedure that opens a connection and make sure
it's closed before exiting the procedure or function even if an exception
occurs. All my connections are only declared as new in each procedure that
uses them and closed before exiting, so in theory I should not even have to
close them. It should be done automatically as the code leaves the
procedure. Just to make doubly sure I wrote in the code to close them.

I'm not using any com components. It's all managed code.
Do you know of any samples that show how to use those Performance counter
classes?
I'm using the framework 1.1 on windows 2000 workstation with the latest SP
installed.

Thanks again,

Bob

"Tom Shelton" <to*@mtogden.com> wrote in message
news:uA**************@TK2MSFTNGP11.phx.gbl...

On 2004-03-04, RD <no****@nospam.net> wrote:

I have a VBV.NET application that runs on a timer to do a job at one hour intervals.
Basically the sequence is as follows
the timer event fires
1- the timer get turned off
2- the code then checks to see if we're just changed hour from when the
previous transfer session took place
IF yes
the code executes an FTP transfer from an FTP site.
When the transfer is completed the file received is parsed (it's a comma separated ASCII file) line by line and the dat is transferred to SQL server files.
Remember the hour part of the current transfer session time
end if
Turn the timer on again

All the code can execute faultlessly for a week doing a transfer every hour on the hour or it can try the first one and stop dead in it's tracks with an out of memory error message. Never at the same place but always the same
code 80004008. We had logs running tracing the program execution and writing every procedure to text files and nothing gives us an inkling that it's
going to fail.

All through the code we've been very carefull to close every ADO .Net
connection every time we open one and tried to always set unused objects to nothing explicitly. All I can come up with as a cause is the new garbage
collector is going out on an illegal strike <GGGG>.

This has been bugging me for over two months in an operational test phase and I'm wondering if there is a practical way to trace what is happening
with memory useage over time, keeping a record that we can correlate to when the errors and trying to figure out exactly why an app that runs alone
usoing about 15% of the available RAM on a machine suddenly and unexpectedly runs out of memory and does so in a totally unpredictable way.

Any help would be immensely appreciated.

MS if you're monitoring this thread, do you think there might be <shudder> a<NO NO DON'T SAY IT> bug in the garbage collector < SORRY, SORRY MOST ABJECT APOLOGIES>. What can I do to make this relaible. This is a vital busines
process that must run unattended 24 7 365.

Bob

Bob...

There was a bug in the 1.0 GC. It failed to deal with the large object
heap correctly (objects over about 80K in size). So, if you had large
arrays, you could end up running into memory problems. Are you using
1.0 or 1.1? As for a problem with the 1.1 GC, I haven't heard of one
and I have a windows service that has been running flawlessly for quite
some time that talks to an sql server database...

By the way, explicitly setting objects to nothing on the local scope
isn't really going to do anything for you :). Are you sure you are
calling Close/Dispose on all your objects that implement these methods?

Another place to look is if your using any COM components or P/Invoke
calls. Those are good places for memory leaks. You might want to look
into the Performance counter classes. They would probably give you a
much better means of monitoring your applications memory useage...

--
Tom Shelton [MVP]
Powered By Gentoo Linux 1.4
Help me, I'm a prisoner in a Fortune cookie file!

[Tom Shelton]

In article <Oe**************@TK2MSFTNGP10.phx.gbl>, RD wrote:

Thanks Tom,
I went thru every single procedure that opens a connection and make sure
it's closed before exiting the procedure or function even if an exception
occurs. All my connections are only declared as new in each procedure that
uses them and closed before exiting, so in theory I should not even have to
close them. It should be done automatically as the code leaves the
procedure. Just to make doubly sure I wrote in the code to close them.

Good - because leaving a procedure does not close the connection
immediately. The connection will only be closed the next time GC runs.
Remember, finalization is non-deterministic in .NET. Any object that
reqires a close/dispose (sockets, files, etc), should be closed before
exiting the method.
I'm not using any com components. It's all managed code.
Good.

Do you know of any samples that show how to use those Performance counter
classes?

here the documentation for monitoring memory usage:
http://msdn.microsoft.com/library/de...rprocesses.asp

here is the start of the documentation on system monitoring components
in general...

http://msdn.microsoft.com/library/de...dsonserver.asp

--
Tom Shelton [MVP]
OS Name: Microsoft Windows XP Professional
OS Version: 5.1.2600 Service Pack 1 Build 2600
System Up Time: 2 Days, 12 Hours, 21 Minutes, 34 Seconds