We've developed a seismic-data processing app in VB.NET 1.1 which runs on XP
Pro using SQL2K. Like all seismic data apps, it deals with HUGE amounts of
binary data (we use NTFS sparse "flat files") ... a typical user will have a
20 Terabyte raid array.
During a particular operation that reads, formats and exports these files,
we've noticed that sometimes it runs rather slowly... like it might take 12
hours to finish exporting the same amount of data that on another (identical)
export would take only 2 hours or so.
In troubleshooting, we noticed something peculiar: The app writes messages
to a (text) log while it is exporting the data, and when the "slowness"
problem is occuring, the timestamps in those messages are precisely 1 second
apart. Here's a sequence of messages showing two back-to-back files being
exported (of hundreds of thousands in a typical export operation):
09/19/2006 07:44:08.860 PM [InfoMsg]: Reading file 00001102.0.0249.0.3.uad
…
09/19/2006 07:44:09.860 PM [InfoMsg]: Wrote 2050 samples read from file
00001102.0.0249.0.3.uad …
09/19/2006 07:44:09.860 PM [InfoMsg]: Reading file 00001263.0.0249.0.1.uad …
09/19/2006 07:44:10.860 PM [InfoMsg]: Wrote 2050 samples read from file
00001263.0.0249.0.1.uad …
Notice that there is exactly 1 second between the “Reading file” and “Wrote
samples” messages. This same pattern (of messages separated by exactly 1
second) can continue for several hours.
But when the “slowness” problem isn’t occuring, these two messages are
separated by just a few milliseconds, and more significantly, the interval
between them tends to vary a bit, as one would expect.
So, what kind of mechanisms in XP or .NET 1.1 could explain what would
“block” an app for precisely one second? I don’t think that a “resource”
constraint (CPU, memory, network, disk) could explain this precise one second
interval.
My suspicion is that the app is being blocked at the moment I invoke
Date.Now to log the message:
_log.WriteLine(Date.Now.ToString("MM/dd/yyyy hh:mm:ss.fff tt") & " [" &
messageType.ToString & "]: " & message)
…at least that would explain all the symptoms we’re seeing.
But I don’t know how to address it. If I remove the call to Date.Now, I’ll
lose the most critical bit of information in the message log!
Any advice would be greatly appreciated!
Stuck in Houston… 2 1477
Hi David,
(By Date.Now, do you mean DateTime.Now? Maybe that's just a VB thing...)
I don't know of any reason why DateTime.Now would block, but just to be sure you could try replacing DateTime.Now (or Date.Now :)
with Environment.TickCount / 1000 so you'll be able to see the number of seconds elapsed between those tasks in the log file.
HTH
--
Dave Sexton
"David" <Da***@discussions.microsoft.comwrote in message news:E6**********************************@microsof t.com...
We've developed a seismic-data processing app in VB.NET 1.1 which runs on XP
Pro using SQL2K. Like all seismic data apps, it deals with HUGE amounts of
binary data (we use NTFS sparse "flat files") ... a typical user will have a
20 Terabyte raid array.
During a particular operation that reads, formats and exports these files,
we've noticed that sometimes it runs rather slowly... like it might take 12
hours to finish exporting the same amount of data that on another (identical)
export would take only 2 hours or so.
In troubleshooting, we noticed something peculiar: The app writes messages
to a (text) log while it is exporting the data, and when the "slowness"
problem is occuring, the timestamps in those messages are precisely 1 second
apart. Here's a sequence of messages showing two back-to-back files being
exported (of hundreds of thousands in a typical export operation):
09/19/2006 07:44:08.860 PM [InfoMsg]: Reading file 00001102.0.0249.0.3.uad
.
09/19/2006 07:44:09.860 PM [InfoMsg]: Wrote 2050 samples read from file
00001102.0.0249.0.3.uad .
09/19/2006 07:44:09.860 PM [InfoMsg]: Reading file 00001263.0.0249.0.1.uad .
09/19/2006 07:44:10.860 PM [InfoMsg]: Wrote 2050 samples read from file
00001263.0.0249.0.1.uad .
Notice that there is exactly 1 second between the "Reading file" and "Wrote
samples" messages. This same pattern (of messages separated by exactly 1
second) can continue for several hours.
But when the "slowness" problem isn't occuring, these two messages are
separated by just a few milliseconds, and more significantly, the interval
between them tends to vary a bit, as one would expect.
So, what kind of mechanisms in XP or .NET 1.1 could explain what would
"block" an app for precisely one second? I don't think that a "resource"
constraint (CPU, memory, network, disk) could explain this precise one second
interval.
My suspicion is that the app is being blocked at the moment I invoke
Date.Now to log the message:
_log.WriteLine(Date.Now.ToString("MM/dd/yyyy hh:mm:ss.fff tt") & " [" &
messageType.ToString & "]: " & message)
.at least that would explain all the symptoms we're seeing.
But I don't know how to address it. If I remove the call to Date.Now, I'll
lose the most critical bit of information in the message log!
Any advice would be greatly appreciated!
Stuck in Houston.
Could it be the Garbage Collector which starts a separate thread when
necessary and needs some to to process? I would be a good idea to use
perfmon an the .NET / GC related performance counters for more detailed
inspection.
Eventually running in a Virtual Machine (VMWare) can reduce virtual CPU
power in some scenario (although we only saw this when the running (guest)
operation system is Win2k).
"David" <Da***@discussions.microsoft.comwrote in message
news:E6**********************************@microsof t.com...
We've developed a seismic-data processing app in VB.NET 1.1 which runs on
XP
Pro using SQL2K. Like all seismic data apps, it deals with HUGE amounts of
binary data (we use NTFS sparse "flat files") ... a typical user will
have a
20 Terabyte raid array.
During a particular operation that reads, formats and exports these files,
we've noticed that sometimes it runs rather slowly... like it might take
12
hours to finish exporting the same amount of data that on another
(identical)
export would take only 2 hours or so.
In troubleshooting, we noticed something peculiar: The app writes
messages
to a (text) log while it is exporting the data, and when the "slowness"
problem is occuring, the timestamps in those messages are precisely 1
second
apart. Here's a sequence of messages showing two back-to-back files being
exported (of hundreds of thousands in a typical export operation):
09/19/2006 07:44:08.860 PM [InfoMsg]: Reading file 00001102.0.0249.0.3.uad
.
09/19/2006 07:44:09.860 PM [InfoMsg]: Wrote 2050 samples read from file
00001102.0.0249.0.3.uad .
09/19/2006 07:44:09.860 PM [InfoMsg]: Reading file
00001263.0.0249.0.1.uad .
09/19/2006 07:44:10.860 PM [InfoMsg]: Wrote 2050 samples read from file
00001263.0.0249.0.1.uad .
Notice that there is exactly 1 second between the "Reading file" and
"Wrote
samples" messages. This same pattern (of messages separated by exactly 1
second) can continue for several hours.
But when the "slowness" problem isn't occuring, these two messages are
separated by just a few milliseconds, and more significantly, the interval
between them tends to vary a bit, as one would expect.
So, what kind of mechanisms in XP or .NET 1.1 could explain what would
"block" an app for precisely one second? I don't think that a "resource"
constraint (CPU, memory, network, disk) could explain this precise one
second
interval.
My suspicion is that the app is being blocked at the moment I invoke
Date.Now to log the message:
_log.WriteLine(Date.Now.ToString("MM/dd/yyyy hh:mm:ss.fff tt") & " [" &
messageType.ToString & "]: " & message)
.at least that would explain all the symptoms we're seeing.
But I don't know how to address it. If I remove the call to Date.Now, I'll
lose the most critical bit of information in the message log!
Any advice would be greatly appreciated!
Stuck in Houston. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Iain Bishop |
last post by:
I'm trying to model objects for the following problem:
A building site contains assemblies, each of which can contain other
assemblies and/or materials.
I have modelled this using a Site...
|
by: M Wells |
last post by:
Hi All,
I'm trying to track down a mysterious problem we're experiencing in
which updates and inserts to tables in our mssql2k server appear to be
'disappearing.'
To explain our situation:
...
|
by: Pohihihi |
last post by:
I was wondering what is the ill effect of using try catch in the code, both
nested and simple big one.
e.g.
try
{
\\ whole app code goes here
} catch (Exception ee) {}
|
by: Max |
last post by:
Hi There!
I'm having a mysterious error right after I login using Forms Authentication
in my ASP.NET app. Below is the error...
Exception Details: System.NullReferenceException: Object...
|
by: greenb |
last post by:
Our asp.net web app uses a .NET component (DLL) in the bin directory to call
several stored procedures back to back to perform updates. They don't return
any data. Sometimes the total execution...
| |
by: Development - multi.art.studio |
last post by:
Hello everyone,
i just upgraded my old postgres-database from version 7.1 to 7.4.2.
i dumped out my 7.1 database (with pg_dump from 7.1) as an sql-file with
copy-commands and to one file using...
|
by: scarleton |
last post by:
A coworker and I have spent the last day and a half trying to track
down this bug in our code. We found the solution and thought we might
post it to help others out...
The setup: Class...
|
by: sara |
last post by:
I have a report that runs fine with data. If there is no data, I have
its NO Data event sending a MsgBox and cancelling the report.
Then it seems I still get the 2501 message on the Open Report...
|
by: caine |
last post by:
I have a clickable calendar, which user can select the day that they
want to view the news linking to. My calendar has the clickable event
day, but once the user click it, the day selected could...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development projectplanning, coding, testing,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |