Stack Overflow

[Eric Webster]

I have a fairly simple VB.Net program I'm developing that listens to data
coming over the internet and processes it. The data comes in as events - the
app is idle until another chunk of data comes through. When it does, the app
looks to see what kind of data it is, and does something with it, such as
parse the data, write it to a file, ignore it, etc.

The code almost completes when it throws a stack overflow exception. The
code makes no recursive calls. I suspect the data is coming through faster
than the app can process it, and events keep piling up until the stack
overflows. Is that possible? The exception happens at random points - never
the same Sub or Function twice in a row. But it does happen, every time.

The reason I think so is because when I process the data in real time, it
throws an exception. When I just record everything to a file and process it
from a file later instead of live, with the same code, no problem at all. The
problem is, I need the results in more or less real time, not later.

If it could be that the events are piling up too fast, it's not critical
that I process every single piece of data that comes through - there are
natural breakpoints in my code where, if it's possible, I would be willing to
clear all pending events. Can that be done? Is it a reasonable approach?

Thanks!

Eric

[Vadym Stetsyak]

Hello, Eric!

EWThe code almost completes when it throws a stack overflow exception.
EWThe code makes no recursive calls. I suspect the data is coming through
EWfaster than the app can process it, and events keep piling up until the
EWstack overflows. Is that possible?

If I understand correctly data is stored in the buffer.
Then taken from that buffer and processed, right?

Then data is piling up in the heap not in the stack...

Can you provide code sample where you process data?
When exception occured - did you look at callstack?

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com

[Chris Dunaway]

Eric Webster wrote:

The code almost completes when it throws a stack overflow exception. The
code makes no recursive calls. I suspect the data is coming through faster
than the app can process it, and events keep piling up until the stack
overflows. Is that possible? The exception happens at random points - never
the same Sub or Function twice in a row. But it does happen, every time.

Is it possible that your processing code is triggering another event.
For example, if you changed the value of a TextBox within the
TextChanged event of that textbox. Could something like that be
happening?

If the code were in the middle of the event, performing processing, I
don't think the event could be re-entered unless you are calling
DoEvents or something to cause events to be processed.

Perhaps you could, in the event code, take the data and place it into a
queue and then have a dedicated thread that processes the data in the
queue.

[Eric Webster]

Perhaps you could, in the event code, take the data and place it into a

queue and then have a dedicated thread that processes the data in the
queue.

That's a thought - a bit outside my experience ini VB, but worth the stretch
to learn it.

I was calling DoEvents in a number of places, but recently took out all
DoEvents calls to see if that made a difference.

It's also interesting that the code gets slower and slower as it runs. What
it's listening to is stock market data, trades in real time. The first 500
stock tickers happen inside a minute. The next 500 takes 2 minutes, the next
in 2.5, and so on, until we get to 10 minutes between updates. So something
is bogging down the system. When I just dump the stream to a file and process
after hours, I can zip through the whole day in seconds. Live, it takes 45 -
60 minutes and the system eventually crashes.

There are multiple events (I can't change the structure of the events,
that's given to me by the Exchanges) firing - one for timestamp messages, one
for summaries, one for company fundamentals, an one event per successful
trade on the Exchanges (NYSE, NASDAQ, and AMEX), which as you may imagine is
quite a lot.

I can't figure out how to get a dump of the stack to see if I have a zillion
nested calls - when it throws the exception, I basically get nothing back
from VS 2005. Is there an easy way to dump the call stack periodically?

[Eric Webster]

If I understand correctly data is stored in the buffer.

Then taken from that buffer and processed, right?

Then data is piling up in the heap not in the stack...

Well, kind of - events keep firing when data is received and the system just
responds to the "data received" events (there are several). I know it's a
stack exception, unless VS 2005 reports heap boundry errors as stack errors.

[Greg Young]

My guess is that you are tying events multiple times and not untying them ..

So as the day goes on you pick up linear overhead.

Cheers,

Greg Young
MVP - C#
http://codebetter.com/blogs/gregyoung

"Eric Webster" <el*****@nospam.nospamwrote in message
news:D5**********************************@microsof t.com...

>Perhaps you could, in the event code, take the data and place it into a
queue and then have a dedicated thread that processes the data in the
queue.

That's a thought - a bit outside my experience ini VB, but worth the
stretch
to learn it.

I was calling DoEvents in a number of places, but recently took out all
DoEvents calls to see if that made a difference.

It's also interesting that the code gets slower and slower as it runs.
What
it's listening to is stock market data, trades in real time. The first 500
stock tickers happen inside a minute. The next 500 takes 2 minutes, the
next
in 2.5, and so on, until we get to 10 minutes between updates. So
something
is bogging down the system. When I just dump the stream to a file and
process
after hours, I can zip through the whole day in seconds. Live, it takes
45 -
60 minutes and the system eventually crashes.

There are multiple events (I can't change the structure of the events,
that's given to me by the Exchanges) firing - one for timestamp messages,
one
for summaries, one for company fundamentals, an one event per successful
trade on the Exchanges (NYSE, NASDAQ, and AMEX), which as you may imagine
is
quite a lot.

I can't figure out how to get a dump of the stack to see if I have a
zillion
nested calls - when it throws the exception, I basically get nothing back
from VS 2005. Is there an easy way to dump the call stack periodically?

[Jeffrey Tan[MSFT]]

Hi Eric,

I agree with Chris that it is likely that your event handler triggered some
other code which recursively called this event handler again, so an
infinitely calling chain is generated, which causes
StatckOverflowException.

The simplest way to find out the root cause is catching this
StatckOverflowException with a debugger and examine the call stack of this
exception, which will reveal the deep stack root cause clearly.

Normally, when you run your application without debugging, the
StatckOverflowException is thrown, the system will popup a dialog for you
to choose a debugger. You may use a VS2005 instance with source code to
debug this exception. After using the debugger to attach the failure
application, you may open "Call Stack" window from "Debug" menu. Once you
have got the symbol setup correctly, you will get a meaningful call stack
for this StatckOverflowException .

For example, in my test winform application, I used the code snippet below
to simulate the StatckOverflowException:

void RecursionMethod()
{
Console.WriteLine("RecursionMethod");
RecursionMethod();
}

private void button1_Click(object sender, System.EventArgs e)
{
RecursionMethod();
}

When the StatckOverflowException is generated, I choose my source code
debugger in the popup dialog. After the debugger breaks into the
application, I will get the following call stack:

mscorlib.dll!System.IO.TextWriter.WriteLine(string value) Line 448 + 0x3a
bytes C#
mscorlib.dll!SyncTextWriter.WriteLine(string value) Line 711 + 0x1b bytes
C#
mscorlib.dll!System.Console.WriteLine(string value) Line 408 + 0x1e bytes
C#

> StackOverflowTest.exe!StackOverflowTest.Form1.Recu rsionMethod() Line 103

C#
StackOverflowTest.exe!StackOverflowTest.Form1.Recu rsionMethod() Line 104
C#
StackOverflowTest.exe!StackOverflowTest.Form1.Recu rsionMethod() Line 104
C#
StackOverflowTest.exe!StackOverflowTest.Form1.Recu rsionMethod() Line 104
C#
.................................................. ....................

StackOverflowTest.exe!StackOverflowTest.Form1.Recu rsionMethod() Line 104 C#

It is very clear that the root cause of the StatckOverflowException is
generated by recursively calling RecursionMethod.

Hope this helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.

[Jeffrey Tan[MSFT]]

Hi Eric,

Have you reviewed my reply? Have you got the debugger launch for this
StatckOverflowException and got the call stack regarding it?

If you still need any help please feel free to tell me, I will work with
you. Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.

[Eric Webster]

Jeffrey, thanks for the followup. I did get the call stack function up today,
that did the trick. Seems both our theories were true - in a periodic
summarization sub I had left some debugging messages to be displayed in the
app, so to make sure they painted, I called Application.DoEvents. This opened
the floodgate for more events to come flowing through, which were processed -
until the next period summarization event, in which case the cycle started
over again. Tomorrow I'll leave it run taking out the DoEvents call and I'll
guess that the system will function correctly.

Thanks for your help!

Eric

[Jeffrey Tan[MSFT]]

Hi Eric,

Thank you for sharing the information with us!

Best regards,
Jeffrey Tan
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.