DataTable weirdness

[Stephan Steiner]

Hi

I have written a .NET service that uses two DataTables to store runtime
data. One column in one table is periodically updated (every 10 seconds, and
upon user request), another is updated upon user action.

The code works fine most of the time. I've had it running on a W2K server
for over half a year with little issues. Though when running the same code
on a W2K3 server, I regularly get IndexOutofRange exceptions at random
intervals (if I start the service it can run fine for two weeks, or it can
start throwing exceptions right away, and the problem might go away after a
while). Here's an example of such an exception:

03:59:03 Monitor.run for vm Cisco1 catched an exception: Index was out of
range. Must be non-negative and less than the size of the collection.
Parameter name: index source mscorlib stacktrace at
System.Collecti ons.ArrayList.R emoveAt(Int32 index)
at System.Data.Rec ordManager.NewR ecordBase()
at System.Data.Dat aTable.NewRecor d(Int32 sourceRecord)
at System.Data.Dat aRow.BeginEdit( )
at System.Data.Dat aRow.set_Item(D ataColumn column, Object value)
at System.Data.Dat aRow.set_Item(S tring columnName, Object value)
at VRSService.Moni tor.run() in c:\documents and settings\sste\m y
documents\visua l studio projects\vrsser vice\vmmonitor. cs:line 787

The line of code throwing the exception is as follows:

myRow["timestamp"] = timestamp; // timestamp = 64bit integer representing
DateTime.Ticks

Now, in absence of any changes to the table and row in question (I have the
DataTable in one class, and send the appropriate row as constructor
parameter to the class that does the updating). The rows in the datatable
are never deleted/replaced, no new rows are ever inserted after the
application startup, and the first column (unique) is never updated. Also,
only updating the timestamp column (the most regular of any updates because
it happens every 10 seconds, whereas the rest is user triggered and a lot
less frequent) throws that error. During the 24h from Saturday midnight to
Sunday midnight (where no user ever connected), I got this error 5 times,
not for all rows, but for some rows more than once.

Since I'm not touching the table, and since the code can fail without any
apparent reason at any time, I'm getting the lingering suspicion that the
implementation of cell updates (the stacktrace makes me think the entire row
is removed and a new one added), can have a problem. I have never
experienced this problem when running the application on my development
boxes (WinXP), it might have happened once or twice in a half year period on
W2K server but it happens quite frequently on W2K3, which afaik is the only
one to not use the standard .NET framework.

Has anyone else ever experienced something like that? I have since written
my code to catch those exceptions (initially they'd be thrown and kill the
thread to do the update every 10 seconds, which eventually would lead to a
situation where the service had to be restarted), and funny enough it seems
that now once thrown, as the thread runs for the next time (10 seconds
later), things are okay again. Still, I shouldn't get those exceptions as
I'm not referencing any column in the row that is non existing.

Regards
Stephan

[Chris R. Timmons]

Stephan,

Sounds like it might be a threading issue. Are you locking the
DataTable that owns myRow before doing the update?

--
Hope this helps.

Chris.
-------------
C.R. Timmons Consulting, Inc.
http://www.crtimmonsinc.com/

[Stephan Steiner]

Chris

Ahh, now it all comes together. I must've misread the class documentation
because I thought write operations were also thread safe, not only reads.
Hence I'm not locking the datatable prior to the update. What is the most
efficient way to lock and unlock a table that still permits synchronous
operations (so no resorting to using delegates to get a reply from the write
operation) so that my write operations don't interfere with each other?
Regards
Stephan

[Chris R. Timmons]

"Stephan Steiner" <st*****@isuiss e.com> wrote in
news:eP******** ******@TK2MSFTN GP09.phx.gbl:

Chris

Ahh, now it all comes together. I must've misread the class
documentation because I thought write operations were also
thread safe, not only reads. Hence I'm not locking the datatable
prior to the update. What is the most efficient way to lock and
unlock a table that still permits synchronous operations (so no
resorting to using delegates to get a reply from the write
operation) so that my write operations don't interfere with each
other?

Stephan,

If the method updating the data structure is an instance method, then
locking "this" is what is usually done:

lock(this)
{
...
myRow["timestamp"] = timestamp;
...
}

If the method is a static method, there will be no "this" to lock.
In that case, the class needs to create an object to lock on:

public class MyClass
{
private static readonly object countLock = new object();

public static void UpdateMyRow()
{
lock(countLock)
{
...
myRow["timestamp"] = timestamp;
...
}
}
}
MVP Jon Skeet has written an excellent article about threading in C#:

http://www.yoda.arachsys.com/csharp/threads/

--
Hope this helps.

Chris.
-------------
C.R. Timmons Consulting, Inc.
http://www.crtimmonsinc.com/

[Stephan Steiner]

Chris

Thanks for the link to the article (I've seen it on other occasions but
never read the entire thing).

I'm currently implementing locks into my software, but I'm wondering:

If I iterate through all the elements of a table in thread A, and at the
same time I write to a cell in the database from a thread B (the entire
write operation is encapsulated in a lock statement). The doc says DB reads
are thread safe. So, am I correct assuming that the lock in thread B will
block until the iterator is no longer used?

Come to think of it I've experienced similar conditions previously: while
iterating through the rows in a table, I find that I need to delete a row. I
can delete the row just fine inside the iteration but when I try to get the
next element via the foreach iterator, I get an exception because the
collection of lines has changed. So I ended up storing which lines I have to
delete, then after going through all lines, I deleted the lines marked for
deletion. In this case, what would an attempt to delete a record directly
within the iteration (using the lock statement) yield? Will the code in the
lock statement just be skipped because I'm currently interating through the
table rows?

Regards
Stephan

If the method updating the data structure is an instance method, then
locking "this" is what is usually done:

lock(this)
{
...
myRow["timestamp"] = timestamp;
...
}

If the method is a static method, there will be no "this" to lock.
In that case, the class needs to create an object to lock on:

public class MyClass
{
private static readonly object countLock = new object();

public static void UpdateMyRow()
{
lock(countLock)
{
...
myRow["timestamp"] = timestamp;
...
}
}
}
MVP Jon Skeet has written an excellent article about threading in C#:

http://www.yoda.arachsys.com/csharp/threads/

--
Hope this helps.

Chris.
-------------
C.R. Timmons Consulting, Inc.
http://www.crtimmonsinc.com/

[Chris R. Timmons]

"Stephan Steiner" <st*****@isuiss e.com> wrote in
news:ux******** ******@TK2MSFTN GP09.phx.gbl:

Chris

Thanks for the link to the article (I've seen it on other
occasions but never read the entire thing).

I'm currently implementing locks into my software, but I'm
wondering:

If I iterate through all the elements of a table in thread A,
and at the same time I write to a cell in the database from a
thread B (the entire write operation is encapsulated in a lock
statement). The doc says DB reads are thread safe. So, am I
correct assuming that the lock in thread B will block until the
iterator is no longer used?
Stephan,

No. The lock on thread B will block other threads from entering
thread B's locked section until that section is finished executing.
Thread B's lock will not prevent other threads from executing if they
are not trying to enter that same locked section.

In other words, thread A's read operations will run concurrently with
thread B's write operations. This happens because thread A is not
trying to enter thread B's locked section.

It sounds like you might want serialized access to a particular data
structure (atomicity):

http://www.yoda.arachsys.com/csharp/...latility.shtml

To implement atomicity on a collection like an ArrayList, you can
have all read and write operations lock on the same object reference.

Below is a short program that demonstrates "atomicized " multiple read
and write operations on an ArrayList. Both operations lock the
ArrayList to achieve this. To see the non-atomic behavior, simply
uncomment the DoReads() method's lock statement.

using System;
using System.Collecti ons;
using System.Threadin g;

namespace Example
{
public class Test
{
private ArrayList a = new ArrayList();

private void DoWrites()
{
int counter = 0;

while (true)
{
lock (a)
{
if (a.Count >= 15)
a.Clear();

Console.WriteLi ne("===== BEGIN WRITING =====");
for (int i = 0; i < 5; i++)
{
a.Add(++counter );
Console.WriteLi ne("Writing: {0}", counter);
Thread.Sleep(10 0);
}
Console.WriteLi ne("===== END WRITING =====");
}
}
}

private void DoReads()
{
while (true)
{
// Comment out this lock statement to see
// non-atomic behavior.
lock (a)
{
Console.WriteLi ne("===== BEGIN READING =====");
for (int i = 0; i < a.Count; i++)
{
Console.WriteLi ne("Reading: {0}", (int) a[i]);
Thread.Sleep(10 0);
}
Console.WriteLi ne("===== END READING =====");
}
}
}

public void DoReadsAndWrite s()
{
ThreadStart doWrites = new ThreadStart(DoW rites);
Thread doWritesThread = new Thread(doWrites );
doWritesThread. Start();

// Start the thread.
Thread.Sleep(0) ;

ThreadStart doReads = new ThreadStart(DoR eads);
Thread doReadsThread = new Thread(doReads) ;
doReadsThread.S tart();

// Start the thread.
Thread.Sleep(0) ;

// Use Ctrl-C to end the program.
}

public static void Main(string[] args)
{
new Test().DoReadsA ndWrites();
}
}
}
Come to think of it I've experienced similar conditions
previously: while iterating through the rows in a table, I find
that I need to delete a row. I can delete the row just fine
inside the iteration but when I try to get the next element via
the foreach iterator, I get an exception because the collection
of lines has changed. So I ended up storing which lines I have
to delete, then after going through all lines, I deleted the
lines marked for deletion. In this case, what would an attempt
to delete a record directly within the iteration (using the lock
statement) yield? Will the code in the lock statement just be
skipped because I'm currently interating through the table rows?

I don't think a lock would help in that case. The problem arises
within any kind of loop which iterates over a collection of data and
deletes one or more elements from that collection. It has nothing to
do with locking, but with the repositioning of the elements of the
collection when an element is removed.

When an element is removed, the behavior for most ordered collections
is to move the remaining elements "up" one level to fill the gap
created by the removal. For example, if a collection of numbers
consists of:

1
2
3
4
5

and element 3 is removed, the collection will now have a gap between
2 and 4:

1
2

4
5

The underlying collection will typically remove the gap by moving the
4 and 5 elements "up" to fill the gap:

1
2
4
5

This causes problems for any loop that started with the assumption
that the collection held 5 elements. When the loop reaches its fifth
iteration, it will find no element to retrieve, and it will throw an
exception.

The solution is usually to simply reverse the loop so it iterates
over the collection from the end to the beginning. This won't work
with a foreach loop, because those loops always iterate from the
beginning of the collection to the end. But other kinds of looping
constructs, including for and while, can be reversed to iterate over
the collection from the end to the beginning. In these cases,
deleting elements from the collection will not cause problems.

For example, a for loop constructed like this:

for (int i = 0; i < myDataView.Coun t; i++)

can be reversed like this:

for (int i = myDataView.Coun t - 1; i >= 0; i--)

--
Hope this helps.

Chris.
-------------
C.R. Timmons Consulting, Inc.
http://www.crtimmonsinc.com/

[Stephan Steiner]

Chris

Below is a short program that demonstrates "atomicized " multiple read
and write operations on an ArrayList. Both operations lock the
ArrayList to achieve this. To see the non-atomic behavior, simply
uncomment the DoReads() method's lock statement.
Thanks for the example. I went to my program and did something similar
(there's quite a few instances where I potentially could run into problems..
most of them have never ocurred but I figured better safe than sorry and
make this thing work flawlessly no matter what).
Hope this helps.

Yes it helped a lot. Thanks again

Regards
Stephan