How to Automatically Update UI When Data Changes

[Charles Law]

Here's one that should probably have the sub-heading "I'm sure I asked this
once before, but ...".

Two users are both looking at the same data, from a database. One user
changes the data and commits it. How does the other user get the updated
view without polling for changes? Is there some sort of callback mechanism
that can be set up on the dataset or connection?

TIA

Charles

[Charles Law]

Hi Carlos

- A polling mechanism every n seconds
Although not my preferred choice originally, RMT has given me some food for
thought, so I can see me going down this route.

Thanks.

Charles
"Carlos J. Quintero [VB MVP]" <ca*****@NOSPAM sogecable.com> wrote in message
news:eq******** ******@TK2MSFTN GP04.phx.gbl... Hi,

Then you need:

- A polling mechanism every n seconds
- Some specific notification system for updates provided by your database,
if available.

AFAIK, the standard database objects of the APIs (ADO, ODBC, etc.) don't
provide any of this.

Another approach is to prevent data modifications while a user is viewing
it (row-level locking). This is done in Oracle with a SELECT .. FOR UPDATE
statement, for example. Depending on your app, this kind of lock while
viewing data can be suitable or overkill.

All other approaches detect the conflict after the fact, when the second
user tries to modify data that was not the one that he got initially, as I
explained using a timestamp (or using dynamic recordsets, datasets, etc.
which I suppose that do the same internally)
--

Best regards,

Carlos J. Quintero

MZ-Tools: Productivity add-ins for Visual Studio
You can code, design and document much faster:
http://www.mztools.com

"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:%2******** ********@TK2MSF TNGP03.phx.gbl. ..

Hi Carlos

Thanks for the reply. I was thinking more of an automatic way, so that
the second user is notified of the change when it happens, rather than if
and when he tries to make a change. This seems to me like something that
would be quite widely desired. I recollect now that I asked this before
wrt web applications, but I am now looking at a WinForms application, so
perhaps there is a way?

Charles

[RMT]

Well in my case, the implementation details are more complicated, yes - as
an example of implementation detail, which I think is specific to my kind of
application:

(1) Prioritise Thread Requests
(2) Ensure low priority requests don't block high priority ones
(3) Error handling in the thread
(4) Listeners registering only for the events they are interested in
(5) Different types of coherency check
(6) Ensure coherency check "chunks", rather than checks an entire structure
at once
(7) Tree structure - reason about a tree from the flat structure a RDBMS
returns
(8) Enforce pattern on UI components
But in the simplest case - wanting to keep a record set coherent, it isn't
so hard.
Some general principles:

(1) A cache object to store the state you want to persist
(2) A worker thread
(3) A way for the thread to talk to the cache (Invoking methods on an
interface), so
(4) An interface for the Cache object to implement so it can receive
messages from (3)
(5) An abstract class from which your UI components can derive
(6) A list of listeners in your Cache Object, i.e. list of references to (5)
(7) A base "Operation" class, which encapsulates any operation you perform
on the data.
(8) Derived classes to actually perform the operations (ie. add, delete,
update, etc.)
(9) Methods on your data cache that construct objects from (8) and queue
them at (2)

Finally, data will flow around the system:

Either:

User interacts with UI component
UI component executes a Cache method (ie. "InsertReco rd ( aaaabbbb )" )
Cache Object constructs request and adds to worker thread queue

or

Worker thread is in idle time, constructs a coherency check and adds it to
it's own queue

and then.....

Worker Thread iterates it's queue, executing each request in turn
Request Invokes a method on Cache interface (marshall across threads)
Cache resolves changes
Cache fires event on all of it's listeners
UI component changes it's state according to the event fired.


"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:Oa******** ******@TK2MSFTN GP02.phx.gbl...

I see what you mean about non-trivial ;-)

I will read and digest. It is the kind of thing that could fit with my
scenario.

Thanks.

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

The data cache is really the central controller of the whole thing,
created on the UI thread. It basically mediates between the UI
components and the worker thread and the worker thread does all the
talking to the database. The thread maintains queues of operations to
perform. The data cache adds a request to the queue when something needs
to be done, for example:
User clicks ADD RECORD button on a toolbar
Code behind the toolbar click tells the cache to ADD RECORD
The cache formats a request and queues it at the thread,
theThread.AddRe quest (theRequest, CRITICAL)
...then forgets about it.
I have 3 queues: CRITICAL, UI and BORED. Critical is for performing
operations the user requests, UI is for things like fetching thumbnails
(in my application this is a neccessary evil) and BORED is for coherency
checking and other stuff like that.

On each iteration of the worker thread loop ( while not m_bFinished () ),
it takes the single highest priority item from the list of queues and
executes it. If there is nothing in any of the lists, it idles for 100ms
(so it doesn't take up all CPU) - after idling it will choose something
to check coherency for (in my case the Tree Structure, Thumbnails, etc.)
and creates a request, adding it to the BORED queue. Then:

Thread de-queues the single highest priority request
Executes it
Invokes the result back to the cache
The cache resolves changes and fires events to it's listeners.

In terms of frequency, I leave this up to the user. You have to trade
coherency against network traffic and performance - my application is
designed to be able to connect to a webserver via. SOAP or ASP.NET, so
it's important I can throttle the coherency check down if I need to.

Note that even with high coherency, the user can still execute an action
and have it fail - ie. it's still possible to attempt to modify a record
another user has deleted, if you do it before the coherency check works
out that the record was deleted. To handle this case, you simply tell
the user with an error message, ie. "The record could not be modified
because it no longer exists".
It's really a simple pattern but implementation details can be more
tricky. If you need any help, post up :).

"Charles Law" <bl***@nowhere. com> wrote in message
news:ed******** ******@TK2MSFTN GP05.phx.gbl...

Hi

My pattern uses a worker thread for performing operations, I have my
own data cache, ie. I don't use data sets. The thread checks the cache
in idle time to see if anything has changed and fires off Update,
Delete, Insert and Move (it's a tree structure so I need this one)
methods to it's listeners. UI components (user controls) register with
the data cache as listeners for these kind of events and implement an
IDataUpdateThre adNotification interface.

I might have a go at this; I take your point about the danger of missing
a server message. Although it shouldn't happen, I have no doubt that it
could.

Have you determined a suitable frequency for checking in your worker
thread? Where do you create the data caches, on the UI thread or the
worker thread?

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

I am currently implementing a pattern that allows this to happen. It
is actually a non-trivial problem. My pattern uses a worker thread for
performing operations, I have my own data cache, ie. I don't use data
sets. The thread checks the cache in idle time to see if anything has
changed and fires off Update, Delete, Insert and Move (it's a tree
structure so I need this one) methods to it's listeners. UI components
(user controls) register with the data cache as listeners for these
kind of events and implement an IDataUpdateThre adNotification
interface. In actuality, there are over 60 different kinds of event.
Note that the worker thread does not directly alter the cache, it only
reads it, fetches any relevant data and then invokes a method on the
cache object to resolve any changes that were detected.

For example, consider I have a set of records in an array, stored in my
cache. The process would look like this:

(1) Thread is idling, so it decides to do a "coherency check"
(2) Thread creates a data connection and fetches the records
(3) Thread Invokes a method on the cache called "RecordsFetched "
...
(4) Cache compares all of the time stamps in the fetched records with
the current records and:

Any timestamps that are different, fires an Updated event to it's
listeners
Any records that exist in the fetched set but not in the cache,
fires an Inserted event to it's listeners
Any records that exist in the cache by not in the fetched set, fires
a Deleted event to it's listeners
Anyway, this is the basis I am working from with all of my (somewhat
complex) data structures. Perhaps it's easier with a single data set.
I would definatley avoid any server event notification systems - it's
too easy for your client to miss a message and thereby mess up your
state.


"Charles Law" <bl***@nowhere. com> wrote in message
news:uY******** ******@TK2MSFTN GP05.phx.gbl...
> Here's one that should probably have the sub-heading "I'm sure I asked
> this once before, but ...".
>
> Two users are both looking at the same data, from a database. One user
> changes the data and commits it. How does the other user get the
> updated view without polling for changes? Is there some sort of
> callback mechanism that can be set up on the dataset or connection?
>
> TIA
>
> Charles
>
>

[Charles Law]

You have obviously taken a comprehensive approach to developing this
pattern. Have you found that it is efficient in terms of cpu time? How many
rows do you return on an average round-trip? Do you have any metric for the
reaction time of the UI to changes in the data?

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

Well in my case, the implementation details are more complicated, yes - as
an example of implementation detail, which I think is specific to my kind
of application:

(1) Prioritise Thread Requests
(2) Ensure low priority requests don't block high priority ones
(3) Error handling in the thread
(4) Listeners registering only for the events they are interested in
(5) Different types of coherency check
(6) Ensure coherency check "chunks", rather than checks an entire
structure at once
(7) Tree structure - reason about a tree from the flat structure a RDBMS
returns
(8) Enforce pattern on UI components
But in the simplest case - wanting to keep a record set coherent, it isn't
so hard.
Some general principles:

(1) A cache object to store the state you want to persist
(2) A worker thread
(3) A way for the thread to talk to the cache (Invoking methods on an
interface), so
(4) An interface for the Cache object to implement so it can receive
messages from (3)
(5) An abstract class from which your UI components can derive
(6) A list of listeners in your Cache Object, i.e. list of references to
(5)
(7) A base "Operation" class, which encapsulates any operation you perform
on the data.
(8) Derived classes to actually perform the operations (ie. add, delete,
update, etc.)
(9) Methods on your data cache that construct objects from (8) and queue
them at (2)

Finally, data will flow around the system:

Either:

User interacts with UI component
UI component executes a Cache method (ie. "InsertReco rd ( aaaabbbb )" )
Cache Object constructs request and adds to worker thread queue

or

Worker thread is in idle time, constructs a coherency check and adds it to
it's own queue

and then.....

Worker Thread iterates it's queue, executing each request in turn
Request Invokes a method on Cache interface (marshall across threads)
Cache resolves changes
Cache fires event on all of it's listeners
UI component changes it's state according to the event fired.


"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:Oa******** ******@TK2MSFTN GP02.phx.gbl...

I see what you mean about non-trivial ;-)

I will read and digest. It is the kind of thing that could fit with my
scenario.

Thanks.

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

The data cache is really the central controller of the whole thing,
created on the UI thread. It basically mediates between the UI
components and the worker thread and the worker thread does all the
talking to the database. The thread maintains queues of operations to
perform. The data cache adds a request to the queue when something
needs to be done, for example:
User clicks ADD RECORD button on a toolbar
Code behind the toolbar click tells the cache to ADD RECORD
The cache formats a request and queues it at the thread,
theThread.AddRe quest (theRequest, CRITICAL)
...then forgets about it.
I have 3 queues: CRITICAL, UI and BORED. Critical is for performing
operations the user requests, UI is for things like fetching thumbnails
(in my application this is a neccessary evil) and BORED is for coherency
checking and other stuff like that.

On each iteration of the worker thread loop ( while not m_bFinished
() ), it takes the single highest priority item from the list of queues
and executes it. If there is nothing in any of the lists, it idles for
100ms (so it doesn't take up all CPU) - after idling it will choose
something to check coherency for (in my case the Tree Structure,
Thumbnails, etc.) and creates a request, adding it to the BORED queue.
Then:

Thread de-queues the single highest priority request
Executes it
Invokes the result back to the cache
The cache resolves changes and fires events to it's listeners.

In terms of frequency, I leave this up to the user. You have to trade
coherency against network traffic and performance - my application is
designed to be able to connect to a webserver via. SOAP or ASP.NET, so
it's important I can throttle the coherency check down if I need to.

Note that even with high coherency, the user can still execute an action
and have it fail - ie. it's still possible to attempt to modify a record
another user has deleted, if you do it before the coherency check works
out that the record was deleted. To handle this case, you simply tell
the user with an error message, ie. "The record could not be modified
because it no longer exists".
It's really a simple pattern but implementation details can be more
tricky. If you need any help, post up :).

"Charles Law" <bl***@nowhere. com> wrote in message
news:ed******** ******@TK2MSFTN GP05.phx.gbl...
Hi

> My pattern uses a worker thread for performing operations, I have my
> own data cache, ie. I don't use data sets. The thread checks the cache
> in idle time to see if anything has changed and fires off Update,
> Delete, Insert and Move (it's a tree structure so I need this one)
> methods to it's listeners. UI components (user controls) register
> with the data cache as listeners for these kind of events and
> implement an IDataUpdateThre adNotification interface.

I might have a go at this; I take your point about the danger of
missing a server message. Although it shouldn't happen, I have no doubt
that it could.

Have you determined a suitable frequency for checking in your worker
thread? Where do you create the data caches, on the UI thread or the
worker thread?

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .
>
> I am currently implementing a pattern that allows this to happen. It
> is actually a non-trivial problem. My pattern uses a worker thread
> for performing operations, I have my own data cache, ie. I don't use
> data sets. The thread checks the cache in idle time to see if anything
> has changed and fires off Update, Delete, Insert and Move (it's a tree
> structure so I need this one) methods to it's listeners. UI
> components (user controls) register with the data cache as listeners
> for these kind of events and implement an
> IDataUpdateThre adNotification interface. In actuality, there are over
> 60 different kinds of event. Note that the worker thread does not
> directly alter the cache, it only reads it, fetches any relevant data
> and then invokes a method on the cache object to resolve any changes
> that were detected.
>
> For example, consider I have a set of records in an array, stored in
> my cache. The process would look like this:
>
> (1) Thread is idling, so it decides to do a "coherency check"
> (2) Thread creates a data connection and fetches the records
> (3) Thread Invokes a method on the cache called "RecordsFetched "
> ...
> (4) Cache compares all of the time stamps in the fetched records with
> the current records and:
>
> Any timestamps that are different, fires an Updated event to it's
> listeners
> Any records that exist in the fetched set but not in the cache,
> fires an Inserted event to it's listeners
> Any records that exist in the cache by not in the fetched set,
> fires a Deleted event to it's listeners
>
>
> Anyway, this is the basis I am working from with all of my (somewhat
> complex) data structures. Perhaps it's easier with a single data set.
> I would definatley avoid any server event notification systems - it's
> too easy for your client to miss a message and thereby mess up your
> state.
>
>
>
>
> "Charles Law" <bl***@nowhere. com> wrote in message
> news:uY******** ******@TK2MSFTN GP05.phx.gbl...
>> Here's one that should probably have the sub-heading "I'm sure I
>> asked this once before, but ...".
>>
>> Two users are both looking at the same data, from a database. One
>> user changes the data and commits it. How does the other user get the
>> updated view without polling for changes? Is there some sort of
>> callback mechanism that can be set up on the dataset or connection?
>>
>> TIA
>>
>> Charles
>>
>>
>
>

[RMT]

Yes, I've done a lot of research on this, because my first attempt was a
miserable failure (well it worked, but it hardly produced maintainable code,
so version 2.0 is more comprehensive).

I can have well over 1000,000 rows in my database and performance outside of
the pattern (ie. on the server side with the DMBS) therefore depends on
system admin things like maintenance of indexes and hardware. This is
particularly true for me, because I'm doing a lot of work with binary
blobs - and a tree structure (ie. every row in the table has some
ancestor/descendant/sibling relationship with other records).

Inside of the pattern itself, most operations are more or less
simultaneous - ie. well within the timeframe the user might expect. For
operations that take a long time (seconds, a minute, two minutes), I have a
timer on my main form that is started when the cache handles the
Operation_Start ed invocation from the thread and stopped when the cache
handles the Operation_Ended invocation from the thread. After one or two
seconds, I display a progress bar. In my experience users don't mind an
operation that takes a while to complete, as long as they can see more or
less how long they have to go fetch a coffee.

Remember that from UI through Cache into Worker Thread and back to UI is
remarkably quick, ie. milliseconds - the time consuming part is database
access - and you can throw hardware at that problem to improve it. Also
because I'm using a worker thread, the UI will remain responsive.

I'm also using DataReaders/Writers, stored procedures for all data access
and the most important structure in .NET (in my opinion), the Dictionary -
so rather than having to hunt through a list of rows to find the one I'm
interested in, I can simply look the record up using a primary key. It's
worth the maintenance for the speedup you get.


"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:u0******** ******@TK2MSFTN GP05.phx.gbl...

You have obviously taken a comprehensive approach to developing this
pattern. Have you found that it is efficient in terms of cpu time? How
many rows do you return on an average round-trip? Do you have any metric
for the reaction time of the UI to changes in the data?

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

Well in my case, the implementation details are more complicated, yes -
as an example of implementation detail, which I think is specific to my
kind of application:

(1) Prioritise Thread Requests
(2) Ensure low priority requests don't block high priority ones
(3) Error handling in the thread
(4) Listeners registering only for the events they are interested in
(5) Different types of coherency check
(6) Ensure coherency check "chunks", rather than checks an entire
structure at once
(7) Tree structure - reason about a tree from the flat structure a RDBMS
returns
(8) Enforce pattern on UI components
But in the simplest case - wanting to keep a record set coherent, it
isn't so hard.
Some general principles:

(1) A cache object to store the state you want to persist
(2) A worker thread
(3) A way for the thread to talk to the cache (Invoking methods on an
interface), so
(4) An interface for the Cache object to implement so it can receive
messages from (3)
(5) An abstract class from which your UI components can derive
(6) A list of listeners in your Cache Object, i.e. list of references to
(5)
(7) A base "Operation" class, which encapsulates any operation you
perform on the data.
(8) Derived classes to actually perform the operations (ie. add, delete,
update, etc.)
(9) Methods on your data cache that construct objects from (8) and queue
them at (2)

Finally, data will flow around the system:

Either:

User interacts with UI component
UI component executes a Cache method (ie. "InsertReco rd ( aaaabbbb )" )
Cache Object constructs request and adds to worker thread queue

or

Worker thread is in idle time, constructs a coherency check and adds it
to it's own queue

and then.....

Worker Thread iterates it's queue, executing each request in turn
Request Invokes a method on Cache interface (marshall across threads)
Cache resolves changes
Cache fires event on all of it's listeners
UI component changes it's state according to the event fired.


"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:Oa******** ******@TK2MSFTN GP02.phx.gbl...

I see what you mean about non-trivial ;-)

I will read and digest. It is the kind of thing that could fit with my
scenario.

Thanks.

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .
The data cache is really the central controller of the whole thing,
created on the UI thread. It basically mediates between the UI
components and the worker thread and the worker thread does all the
talking to the database. The thread maintains queues of operations to
perform. The data cache adds a request to the queue when something
needs to be done, for example:
User clicks ADD RECORD button on a toolbar
Code behind the toolbar click tells the cache to ADD RECORD
The cache formats a request and queues it at the thread,
theThread.AddRe quest (theRequest, CRITICAL)
...then forgets about it.
I have 3 queues: CRITICAL, UI and BORED. Critical is for performing
operations the user requests, UI is for things like fetching thumbnails
(in my application this is a neccessary evil) and BORED is for
coherency checking and other stuff like that.

On each iteration of the worker thread loop ( while not m_bFinished
() ), it takes the single highest priority item from the list of queues
and executes it. If there is nothing in any of the lists, it idles for
100ms (so it doesn't take up all CPU) - after idling it will choose
something to check coherency for (in my case the Tree Structure,
Thumbnails, etc.) and creates a request, adding it to the BORED queue.
Then:

Thread de-queues the single highest priority request
Executes it
Invokes the result back to the cache
The cache resolves changes and fires events to it's listeners.

In terms of frequency, I leave this up to the user. You have to trade
coherency against network traffic and performance - my application is
designed to be able to connect to a webserver via. SOAP or ASP.NET, so
it's important I can throttle the coherency check down if I need to.

Note that even with high coherency, the user can still execute an
action and have it fail - ie. it's still possible to attempt to modify
a record another user has deleted, if you do it before the coherency
check works out that the record was deleted. To handle this case, you
simply tell the user with an error message, ie. "The record could not
be modified because it no longer exists".
It's really a simple pattern but implementation details can be more
tricky. If you need any help, post up :).

"Charles Law" <bl***@nowhere. com> wrote in message
news:ed******** ******@TK2MSFTN GP05.phx.gbl...
> Hi
>
>> My pattern uses a worker thread for performing operations, I have my
>> own data cache, ie. I don't use data sets. The thread checks the
>> cache in idle time to see if anything has changed and fires off
>> Update, Delete, Insert and Move (it's a tree structure so I need this
>> one) methods to it's listeners. UI components (user controls)
>> register with the data cache as listeners for these kind of events
>> and implement an IDataUpdateThre adNotification interface.
>
> I might have a go at this; I take your point about the danger of
> missing a server message. Although it shouldn't happen, I have no
> doubt that it could.
>
> Have you determined a suitable frequency for checking in your worker
> thread? Where do you create the data caches, on the UI thread or the
> worker thread?
>
> Charles
>
>
> "RMT" <no****@nospam. com> wrote in message
> news:e6******** ***********@new s.demon.co.uk.. .
>>
>> I am currently implementing a pattern that allows this to happen. It
>> is actually a non-trivial problem. My pattern uses a worker thread
>> for performing operations, I have my own data cache, ie. I don't use
>> data sets. The thread checks the cache in idle time to see if
>> anything has changed and fires off Update, Delete, Insert and Move
>> (it's a tree structure so I need this one) methods to it's listeners.
>> UI components (user controls) register with the data cache as
>> listeners for these kind of events and implement an
>> IDataUpdateThre adNotification interface. In actuality, there are
>> over 60 different kinds of event. Note that the worker thread does
>> not directly alter the cache, it only reads it, fetches any relevant
>> data and then invokes a method on the cache object to resolve any
>> changes that were detected.
>>
>> For example, consider I have a set of records in an array, stored in
>> my cache. The process would look like this:
>>
>> (1) Thread is idling, so it decides to do a "coherency check"
>> (2) Thread creates a data connection and fetches the records
>> (3) Thread Invokes a method on the cache called "RecordsFetched "
>> ...
>> (4) Cache compares all of the time stamps in the fetched records with
>> the current records and:
>>
>> Any timestamps that are different, fires an Updated event to it's
>> listeners
>> Any records that exist in the fetched set but not in the cache,
>> fires an Inserted event to it's listeners
>> Any records that exist in the cache by not in the fetched set,
>> fires a Deleted event to it's listeners
>>
>>
>> Anyway, this is the basis I am working from with all of my (somewhat
>> complex) data structures. Perhaps it's easier with a single data
>> set. I would definatley avoid any server event notification systems -
>> it's too easy for your client to miss a message and thereby mess up
>> your state.
>>
>>
>>
>>
>> "Charles Law" <bl***@nowhere. com> wrote in message
>> news:uY******** ******@TK2MSFTN GP05.phx.gbl...
>>> Here's one that should probably have the sub-heading "I'm sure I
>>> asked this once before, but ...".
>>>
>>> Two users are both looking at the same data, from a database. One
>>> user changes the data and commits it. How does the other user get
>>> the updated view without polling for changes? Is there some sort of
>>> callback mechanism that can be set up on the dataset or connection?
>>>
>>> TIA
>>>
>>> Charles
>>>
>>>
>>
>>
>
>

[Charles Law]

Thanks for all the excellent pointers. This gives me a lot to be going on
with. I will start small, to experiment, and build up, I think.

As you say, you are on V2 now, because V1 was a "miserable failure": was
there anything in particular that you learned to avoid, that in V1 made it
so bad?

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

Yes, I've done a lot of research on this, because my first attempt was a
miserable failure (well it worked, but it hardly produced maintainable
code, so version 2.0 is more comprehensive).

I can have well over 1000,000 rows in my database and performance outside
of the pattern (ie. on the server side with the DMBS) therefore depends on
system admin things like maintenance of indexes and hardware. This is
particularly true for me, because I'm doing a lot of work with binary
blobs - and a tree structure (ie. every row in the table has some
ancestor/descendant/sibling relationship with other records).

Inside of the pattern itself, most operations are more or less
simultaneous - ie. well within the timeframe the user might expect. For
operations that take a long time (seconds, a minute, two minutes), I have
a timer on my main form that is started when the cache handles the
Operation_Start ed invocation from the thread and stopped when the cache
handles the Operation_Ended invocation from the thread. After one or two
seconds, I display a progress bar. In my experience users don't mind an
operation that takes a while to complete, as long as they can see more or
less how long they have to go fetch a coffee.

Remember that from UI through Cache into Worker Thread and back to UI is
remarkably quick, ie. milliseconds - the time consuming part is database
access - and you can throw hardware at that problem to improve it. Also
because I'm using a worker thread, the UI will remain responsive.

I'm also using DataReaders/Writers, stored procedures for all data access
and the most important structure in .NET (in my opinion), the Dictionary -
so rather than having to hunt through a list of rows to find the one I'm
interested in, I can simply look the record up using a primary key. It's
worth the maintenance for the speedup you get.


"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:u0******** ******@TK2MSFTN GP05.phx.gbl...

You have obviously taken a comprehensive approach to developing this
pattern. Have you found that it is efficient in terms of cpu time? How
many rows do you return on an average round-trip? Do you have any metric
for the reaction time of the UI to changes in the data?

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .

Well in my case, the implementation details are more complicated, yes -
as an example of implementation detail, which I think is specific to my
kind of application:

(1) Prioritise Thread Requests
(2) Ensure low priority requests don't block high priority ones
(3) Error handling in the thread
(4) Listeners registering only for the events they are interested in
(5) Different types of coherency check
(6) Ensure coherency check "chunks", rather than checks an entire
structure at once
(7) Tree structure - reason about a tree from the flat structure a RDBMS
returns
(8) Enforce pattern on UI components
But in the simplest case - wanting to keep a record set coherent, it
isn't so hard.
Some general principles:

(1) A cache object to store the state you want to persist
(2) A worker thread
(3) A way for the thread to talk to the cache (Invoking methods on an
interface), so
(4) An interface for the Cache object to implement so it can receive
messages from (3)
(5) An abstract class from which your UI components can derive
(6) A list of listeners in your Cache Object, i.e. list of references to
(5)
(7) A base "Operation" class, which encapsulates any operation you
perform on the data.
(8) Derived classes to actually perform the operations (ie. add, delete,
update, etc.)
(9) Methods on your data cache that construct objects from (8) and queue
them at (2)

Finally, data will flow around the system:

Either:

User interacts with UI component
UI component executes a Cache method (ie. "InsertReco rd ( aaaabbbb )" )
Cache Object constructs request and adds to worker thread queue

or

Worker thread is in idle time, constructs a coherency check and adds it
to it's own queue

and then.....

Worker Thread iterates it's queue, executing each request in turn
Request Invokes a method on Cache interface (marshall across threads)
Cache resolves changes
Cache fires event on all of it's listeners
UI component changes it's state according to the event fired.


"Charles Law" <bl***@nowhere. com> escribió en el mensaje
news:Oa******** ******@TK2MSFTN GP02.phx.gbl...
I see what you mean about non-trivial ;-)

I will read and digest. It is the kind of thing that could fit with my
scenario.

Thanks.

Charles
"RMT" <no****@nospam. com> wrote in message
news:e6******** ***********@new s.demon.co.uk.. .
> The data cache is really the central controller of the whole thing,
> created on the UI thread. It basically mediates between the UI
> components and the worker thread and the worker thread does all the
> talking to the database. The thread maintains queues of operations to
> perform. The data cache adds a request to the queue when something
> needs to be done, for example:
>
>
> User clicks ADD RECORD button on a toolbar
> Code behind the toolbar click tells the cache to ADD RECORD
> The cache formats a request and queues it at the thread,
> theThread.AddRe quest (theRequest, CRITICAL)
> ...then forgets about it.
>
>
> I have 3 queues: CRITICAL, UI and BORED. Critical is for performing
> operations the user requests, UI is for things like fetching
> thumbnails (in my application this is a neccessary evil) and BORED is
> for coherency checking and other stuff like that.
>
> On each iteration of the worker thread loop ( while not m_bFinished
> () ), it takes the single highest priority item from the list of
> queues and executes it. If there is nothing in any of the lists, it
> idles for 100ms (so it doesn't take up all CPU) - after idling it will
> choose something to check coherency for (in my case the Tree
> Structure, Thumbnails, etc.) and creates a request, adding it to the
> BORED queue. Then:
>
> Thread de-queues the single highest priority request
> Executes it
> Invokes the result back to the cache
> The cache resolves changes and fires events to it's listeners.
>
>
>
> In terms of frequency, I leave this up to the user. You have to trade
> coherency against network traffic and performance - my application is
> designed to be able to connect to a webserver via. SOAP or ASP.NET, so
> it's important I can throttle the coherency check down if I need to.
>
> Note that even with high coherency, the user can still execute an
> action and have it fail - ie. it's still possible to attempt to modify
> a record another user has deleted, if you do it before the coherency
> check works out that the record was deleted. To handle this case, you
> simply tell the user with an error message, ie. "The record could not
> be modified because it no longer exists".
>
>
> It's really a simple pattern but implementation details can be more
> tricky. If you need any help, post up :).
>
>
>
> "Charles Law" <bl***@nowhere. com> wrote in message
> news:ed******** ******@TK2MSFTN GP05.phx.gbl...
>> Hi
>>
>>> My pattern uses a worker thread for performing operations, I have my
>>> own data cache, ie. I don't use data sets. The thread checks the
>>> cache in idle time to see if anything has changed and fires off
>>> Update, Delete, Insert and Move (it's a tree structure so I need
>>> this one) methods to it's listeners. UI components (user controls)
>>> register with the data cache as listeners for these kind of events
>>> and implement an IDataUpdateThre adNotification interface.
>>
>> I might have a go at this; I take your point about the danger of
>> missing a server message. Although it shouldn't happen, I have no
>> doubt that it could.
>>
>> Have you determined a suitable frequency for checking in your worker
>> thread? Where do you create the data caches, on the UI thread or the
>> worker thread?
>>
>> Charles
>>
>>
>> "RMT" <no****@nospam. com> wrote in message
>> news:e6******** ***********@new s.demon.co.uk.. .
>>>
>>> I am currently implementing a pattern that allows this to happen.
>>> It is actually a non-trivial problem. My pattern uses a worker
>>> thread for performing operations, I have my own data cache, ie. I
>>> don't use data sets. The thread checks the cache in idle time to see
>>> if anything has changed and fires off Update, Delete, Insert and
>>> Move (it's a tree structure so I need this one) methods to it's
>>> listeners. UI components (user controls) register with the data
>>> cache as listeners for these kind of events and implement an
>>> IDataUpdateThre adNotification interface. In actuality, there are
>>> over 60 different kinds of event. Note that the worker thread does
>>> not directly alter the cache, it only reads it, fetches any relevant
>>> data and then invokes a method on the cache object to resolve any
>>> changes that were detected.
>>>
>>> For example, consider I have a set of records in an array, stored in
>>> my cache. The process would look like this:
>>>
>>> (1) Thread is idling, so it decides to do a "coherency check"
>>> (2) Thread creates a data connection and fetches the records
>>> (3) Thread Invokes a method on the cache called "RecordsFetched "
>>> ...
>>> (4) Cache compares all of the time stamps in the fetched records
>>> with the current records and:
>>>
>>> Any timestamps that are different, fires an Updated event to it's
>>> listeners
>>> Any records that exist in the fetched set but not in the cache,
>>> fires an Inserted event to it's listeners
>>> Any records that exist in the cache by not in the fetched set,
>>> fires a Deleted event to it's listeners
>>>
>>>
>>> Anyway, this is the basis I am working from with all of my (somewhat
>>> complex) data structures. Perhaps it's easier with a single data
>>> set. I would definatley avoid any server event notification
>>> systems - it's too easy for your client to miss a message and
>>> thereby mess up your state.
>>>
>>>
>>>
>>>
>>> "Charles Law" <bl***@nowhere. com> wrote in message
>>> news:uY******** ******@TK2MSFTN GP05.phx.gbl...
>>>> Here's one that should probably have the sub-heading "I'm sure I
>>>> asked this once before, but ...".
>>>>
>>>> Two users are both looking at the same data, from a database. One
>>>> user changes the data and commits it. How does the other user get
>>>> the updated view without polling for changes? Is there some sort of
>>>> callback mechanism that can be set up on the dataset or connection?
>>>>
>>>> TIA
>>>>
>>>> Charles
>>>>
>>>>
>>>
>>>
>>
>>
>
>

[Ken Tucker [MVP]]

Charles,

Most database engines support triggers. They are a special type
of stored procedure that runs when you modify data in a database. I would
add a table to the database which contains 2 fields table name and revision
number. I would create a trigger to update the revision number whenever
something is changed. In a timer check the revision number and reload the
data if necessary.

http://msdn.microsoft.com/library/de...thtriggers.asp

Ken
-------------------------

"Charles Law" wrote:

Cor

What if the first was looking while the second was on the toilet and did
the updates after that while the other one was already not looking
anymore. Should there be a message that he/she was on the toilet but
changed it?

On return from the lavatory, the user's screen would be up-to-date, that's
all.

This kind of pseudo accuratesse is in most situations already long time
seen as b*sh*t

By whom? It would seem that Microsoft do not see it that way, as they have
introduced the SqlDependency class in VS2005 to address just this situation.

Anyway, I think I have my answer: it is possible only in VS2005 with
SqlServer. Thanks.

Charles
"Cor Ligthert [MVP]" <no************ @planet.nl> wrote in message
news:Of******** ******@TK2MSFTN GP04.phx.gbl...

Charles,

The described process keeps only track for the updating user to prevent
that there will not wrong changed data because there where already changes
in the meantime.

What if the first was looking while the second was on the toilet and did
the updates after that while the other one was already not looking
anymore. Should there be a message that he/she was on the toilet but
changed it?

This kind of pseudo accuratesse is in most situations already long time
seen as b*sh*t

However not there where there are automatic updates done for by instance
by a stock exchange.

Ken made this sample for this. Problem for me is that it will not work for
me without a 100% load so maybe can you try it as well. Ken has let this
tested by others and told that they had not that problem.

http://www.vb-tips.com/default.aspx?...c-3503c15fd46e

Beside this it is of course advisable not to make processes that keeps to
long the changes data on the client. However if you are on a place where
there is really not any usable connection possible, than it can be that it
takes longer.

I hope this helps,

Cor
"Charles Law" <bl***@nowhere. com> schreef in bericht
news:u7******** ******@TK2MSFTN GP05.phx.gbl...

Hi Cor

Sorry if I am not catching on very quickly, but I don't quite see how
your scenario would work.

A dataset keeps track of the changed rows as long as you did not do
acceptchanges.

Isn't this the dataset of the user who has made the changes? What about
the dataset of the user who is just viewing the data? Does that get some
notification of the changes as well?

In the update process the code checks first row by row of those original
rows are changed in the database. (Created new, deleted or updated).

This sounds like something that would happen when the second user tries
to make changes, in which case I agree that there would be some sort of
error because the data had already changed. But suppose the second user
is just looking at the data? Because datasets are disconnected in .NET it
seems to me that the second user might never see the changes unless he
specifically reloads the data.

Charles
"Cor Ligthert [MVP]" <no************ @planet.nl> wrote in message
news:uu******** ******@TK2MSFTN GP02.phx.gbl...
Charles,

If you are using any kind of dataadapter/tableadapter and
dataset/datatable than it is quiet easy.

A dataset keeps track of the changed rows as long as you did not do
acceptchanges.

In the update process the code checks first row by row of those original
rows are changed in the database. (Created new, deleted or updated).

If that is the fact than you get a concurrency error.
Be aware that accepted rows are handled, so the recovery process is not
easy.
(Although you can set the connection with a begintransactio n and a
commit or non commit of that).

Because of the fact that this situations in most situations is seldom,
it is called optimistic concurrency.
However you have to check it and creates the procedures.
Even if that is telling setting everything back to the first state and
telling the user to do his job again.

Very simple is it not?

Cor

"Charles Law" <bl***@nowhere. com> schreef in bericht
news:uY******** ******@TK2MSFTN GP05.phx.gbl...
> Here's one that should probably have the sub-heading "I'm sure I asked
> this once before, but ...".
>
> Two users are both looking at the same data, from a database. One user
> changes the data and commits it. How does the other user get the
> updated view without polling for changes? Is there some sort of
> callback mechanism that can be set up on the dataset or connection?
>
> TIA
>
> Charles
>
>

[Charles Law]

Hi Ken

Thanks for the suggestion. I understand the principal of what you are saying
but, to me, it seems akin to using files as semaphores. I am not totally
comfortable with this technique as it is generally a workaround, or even a
kludge. For the moment, I am going to pursue RMT's idea, in the absence of a
native solution (not limited to VS2005 and SqlServer).

Regards

Charles
"Ken Tucker [MVP]" <Ke**********@d iscussions.micr osoft.com> wrote in message
news:4E******** *************** ***********@mic rosoft.com...

Charles,

Most database engines support triggers. They are a special type
of stored procedure that runs when you modify data in a database. I would
add a table to the database which contains 2 fields table name and
revision
number. I would create a trigger to update the revision number whenever
something is changed. In a timer check the revision number and reload the
data if necessary.

http://msdn.microsoft.com/library/de...thtriggers.asp

Ken
-------------------------

"Charles Law" wrote:

Cor

> What if the first was looking while the second was on the toilet and
> did
> the updates after that while the other one was already not looking
> anymore. Should there be a message that he/she was on the toilet but
> changed it?

On return from the lavatory, the user's screen would be up-to-date,
that's
all.

> This kind of pseudo accuratesse is in most situations already long time
> seen as b*sh*t

By whom? It would seem that Microsoft do not see it that way, as they
have
introduced the SqlDependency class in VS2005 to address just this
situation.

Anyway, I think I have my answer: it is possible only in VS2005 with
SqlServer. Thanks.

Charles
"Cor Ligthert [MVP]" <no************ @planet.nl> wrote in message
news:Of******** ******@TK2MSFTN GP04.phx.gbl...

> Charles,
>
> The described process keeps only track for the updating user to prevent
> that there will not wrong changed data because there where already
> changes
> in the meantime.
>
> What if the first was looking while the second was on the toilet and
> did
> the updates after that while the other one was already not looking
> anymore. Should there be a message that he/she was on the toilet but
> changed it?
>
> This kind of pseudo accuratesse is in most situations already long time
> seen as b*sh*t
>
> However not there where there are automatic updates done for by
> instance
> by a stock exchange.
>
> Ken made this sample for this. Problem for me is that it will not work
> for
> me without a 100% load so maybe can you try it as well. Ken has let
> this
> tested by others and told that they had not that problem.
>
> http://www.vb-tips.com/default.aspx?...c-3503c15fd46e
>
> Beside this it is of course advisable not to make processes that keeps
> to
> long the changes data on the client. However if you are on a place
> where
> there is really not any usable connection possible, than it can be that
> it
> takes longer.
>
> I hope this helps,
>
> Cor
>
>
> "Charles Law" <bl***@nowhere. com> schreef in bericht
> news:u7******** ******@TK2MSFTN GP05.phx.gbl...
>> Hi Cor
>>
>> Sorry if I am not catching on very quickly, but I don't quite see how
>> your scenario would work.
>>
>>> A dataset keeps track of the changed rows as long as you did not do
>>> acceptchanges.
>>
>> Isn't this the dataset of the user who has made the changes? What
>> about
>> the dataset of the user who is just viewing the data? Does that get
>> some
>> notification of the changes as well?
>>
>>> In the update process the code checks first row by row of those
>>> original
>>> rows are changed in the database. (Created new, deleted or updated).
>>
>> This sounds like something that would happen when the second user
>> tries
>> to make changes, in which case I agree that there would be some sort
>> of
>> error because the data had already changed. But suppose the second
>> user
>> is just looking at the data? Because datasets are disconnected in .NET
>> it
>> seems to me that the second user might never see the changes unless he
>> specifically reloads the data.
>>
>> Charles
>>
>>
>> "Cor Ligthert [MVP]" <no************ @planet.nl> wrote in message
>> news:uu******** ******@TK2MSFTN GP02.phx.gbl...
>>> Charles,
>>>
>>> If you are using any kind of dataadapter/tableadapter and
>>> dataset/datatable than it is quiet easy.
>>>
>>> A dataset keeps track of the changed rows as long as you did not do
>>> acceptchanges.
>>>
>>> In the update process the code checks first row by row of those
>>> original
>>> rows are changed in the database. (Created new, deleted or updated).
>>>
>>> If that is the fact than you get a concurrency error.
>>> Be aware that accepted rows are handled, so the recovery process is
>>> not
>>> easy.
>>> (Although you can set the connection with a begintransactio n and a
>>> commit or non commit of that).
>>>
>>> Because of the fact that this situations in most situations is
>>> seldom,
>>> it is called optimistic concurrency.
>>> However you have to check it and creates the procedures.
>>> Even if that is telling setting everything back to the first state
>>> and
>>> telling the user to do his job again.
>>>
>>> Very simple is it not?
>>>
>>> Cor
>>>
>>> "Charles Law" <bl***@nowhere. com> schreef in bericht
>>> news:uY******** ******@TK2MSFTN GP05.phx.gbl...
>>>> Here's one that should probably have the sub-heading "I'm sure I
>>>> asked
>>>> this once before, but ...".
>>>>
>>>> Two users are both looking at the same data, from a database. One
>>>> user
>>>> changes the data and commits it. How does the other user get the
>>>> updated view without polling for changes? Is there some sort of
>>>> callback mechanism that can be set up on the dataset or connection?
>>>>
>>>> TIA
>>>>
>>>> Charles
>>>>
>>>>
>>>
>>>
>>
>>
>
>

[Dan Kelley]

"Charles Law" wrote:

Here's one that should probably have the sub-heading "I'm sure I asked this
once before, but ...".

Two users are both looking at the same data, from a database. One user
changes the data and commits it. How does the other user get the updated
view without polling for changes? Is there some sort of callback mechanism
that can be set up on the dataset or connection?

TIA

Charles

I don't really know much about your architecture, but you could use UDP
multicast messaging. Assuming you have a server that is performing the
updates, once the update is complete, the server could publish a notification
over a multicast address that the data has changed. Clients would obviously
need to join the multicast group.

Depending on how much sophistication you need, the solution can be as simple
or complex as you like. One thing to note is multicast traffic must be
enabled on your network routers, but there is plenty of documentation around
on how it can be done.

HTH
Dan

[Charles K Law]

Hi Dan

That sounds like a neat idea. Is it something you have used yourself? How
did it work out?

Charles

"Charles Law" wrote:

Here's one that should probably have the sub-heading "I'm sure I
asked this once before, but ...".

Two users are both looking at the same data, from a database. One
user changes the data and commits it. How does the other user get the
updated view without polling for changes? Is there some sort of
callback mechanism that can be set up on the dataset or connection?

TIA

Charles

I don't really know much about your architecture, but you could use
UDP multicast messaging. Assuming you have a server that is performing
the updates, once the update is complete, the server could publish a
notification over a multicast address that the data has changed.
Clients would obviously need to join the multicast group.

Depending on how much sophistication you need, the solution can be as
simple or complex as you like. One thing to note is multicast traffic
must be enabled on your network routers, but there is plenty of
documentation around on how it can be done.

HTH
Dan

[Rob]

Charles,

We are currently doing exactly what you desire. We found our solution
using Genine Channels (www.genuinechannels.com). They have a built in
broadcast engine. Essentially all you do is register MBR's, listener
objects embedded into your business object, into this engine. They
have examples on their site.

Cheers,
Rob Panosh
Advanced Software Designs
Charles Law wrote:

Here's one that should probably have the sub-heading "I'm sure I asked this
once before, but ...".

Two users are both looking at the same data, from a database. One user
changes the data and commits it. How does the other user get the updated
view without polling for changes? Is there some sort of callback mechanism
that can be set up on the dataset or connection?

TIA

Charles