473,320 Members | 1,724 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Multithread wait for empty queue

I have a multithreaded application that manipulates a Queue:

protected Queue outputQueue;

when I add elements to the queue I use a critical section

private void Add(OutputRecord record)
{
lock(outputQueue)
{
// Add the element
outputQueue.Enqueue(record);
}
}

when I remove an element from the queue I do something similar:

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();
}
return record;
}

Now I want to be able to have a process that adds an element and then
waits until the queue is empty. What is the best way to do this? I
thought of creating an additional object called emptyOutputQueue and
creating a WaitAdd function:

private bool WaitAdd(OutputRecord record)
{
lock(emptyOutputQueue)
{
Add(record);
if (! Monitor.Wait(emptyOutputQueue, 20000))
return false;
}
return true;
}

and changing the Remove function to be something like this

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();

if (outputQueue.Count == 0)
{
lock (emptyOutputQueue)
{
Monitor.PulseAll(emptyOutputQueue);
}
}
}
}
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

Thanks

Jeronimo

Jul 21 '05 #1
4 3460
Jeronimo Bertran <je**************@newsgroup.nospam> wrote:
I have a multithreaded application that manipulates a Queue:
<snip>
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?


For a start, I'd try to use a single lock - your current solution
acquires the two locks in different orders, which means you could have
deadlock.

If you have to use two locks (which you may) you should make sure you
always acquire them in the same order. I'd try to avoid calling other
methods which are going to do locking from within a method which
already locks - it'll make it easier to see this kind of problem.

I don't have time to come up with a solution for you right now, but I
can have another look tonight if you haven't solved it by then.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jul 21 '05 #2
Have a producer/consumer blocking queue here.
http://codeproject.com/csharp/BoundedBlockingQueue.asp
http://channel9.msdn.com/ShowPost.aspx?PostID=50960 (same Q with a server
example)

That also sounds a bit like a Reverse Sensing Semaphore, so you can check
this out too for some ideas:
http://codeproject.com/csharp/Dijkst...gSemaphore.asp

Using my queue above, your consumer could just call the TryDequeue() method
in its loop which will return False if the queue is empty. However, if your
producer(s) are still producing, this empty state may last for only a timer
tick or so as a producer may be right behind waiting to write. You may have
also *just missed an Empty state if another consumer grabbed the last
element and a producer just added one - you missed the empty state. A reset
event may be in order. If you describe the problem space in more detail, I
may be able to come up with a better idea. You may find what you need in
the code. Please let us know. Cheers.
--
William Stacey, MVP
http://mvp.support.microsoft.com

"Jeronimo Bertran" <je**************@newsgroup.nospam> wrote in message
news:#5**************@TK2MSFTNGP14.phx.gbl...
I have a multithreaded application that manipulates a Queue:

protected Queue outputQueue;

when I add elements to the queue I use a critical section

private void Add(OutputRecord record)
{
lock(outputQueue)
{
// Add the element
outputQueue.Enqueue(record);
}
}

when I remove an element from the queue I do something similar:

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();
}
return record;
}

Now I want to be able to have a process that adds an element and then
waits until the queue is empty. What is the best way to do this? I
thought of creating an additional object called emptyOutputQueue and
creating a WaitAdd function:

private bool WaitAdd(OutputRecord record)
{
lock(emptyOutputQueue)
{
Add(record);
if (! Monitor.Wait(emptyOutputQueue, 20000))
return false;
}
return true;
}

and changing the Remove function to be something like this

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();

if (outputQueue.Count == 0)
{
lock (emptyOutputQueue)
{
Monitor.PulseAll(emptyOutputQueue);
}
}
}
}
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

Thanks

Jeronimo


Jul 21 '05 #3
William,

Thanks for the article... I could change the behavior of my queue to
look similar to what you have so that instead of blocking until the
queue is empty, block when the queue is full. Here is a more detailed
description of what I was trying to achieve:

Just like in your example, I am using the output Queue to send packets
over the network. For most packet types, I enqueue the data and the
communication thread dequeues and sends the data over the network. I
could use the locking queue in your example when calling Enqueue,
although right now I am simply returning false when the queue is full
(non blocking version).

Now, for some special packet types I wanted to make sure that the packet
was sent before I enqueued the next packet. For this reason, I wanted
to enqueue and then wait until the queue was empty before calling
enqueue for the next element. I understand that it is possible that the
queue never empties if another producer keeps adding elements but my
outputQueue in reality is almost always empty.

I am going to try to change the behavior and use your blocking queue and
simply block when the outputQueue is full. In my implenetation I will
use the nonblocking version of dequeue.

I will let you know if I run into problems.

Thanks again,

Jeronimo
Using my queue above, your consumer could just call the TryDequeue()
method in its loop which will return False if the queue is empty.
However, if your producer(s) are still producing, this empty state may
last for only a timer tick or so as a producer may be right behind
waiting to write. You may have also *just missed an Empty state if
another consumer grabbed the last element and a producer just added
one - you missed the empty state. A reset event may be in order. If
you describe the problem space in more detail, I may be able to come
up with a better idea. You may find what you need in the code.
Please let us know. Cheers.


Jul 21 '05 #4
Are you using UDP? If so, then you still need to check if the packet was
actually received via some ack. If TCP, then the packets will be in order
so not sure why you need to worry about it. I probably miss something in
your needs. Let me know if can help more. Cheers.

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Jeronimo Bertran" <je**************@newsgroup.nospam> wrote in message
news:Xn********************************@207.46.248 .16...
William,

Thanks for the article... I could change the behavior of my queue to
look similar to what you have so that instead of blocking until the
queue is empty, block when the queue is full. Here is a more detailed
description of what I was trying to achieve:

Just like in your example, I am using the output Queue to send packets
over the network. For most packet types, I enqueue the data and the
communication thread dequeues and sends the data over the network. I
could use the locking queue in your example when calling Enqueue,
although right now I am simply returning false when the queue is full
(non blocking version).

Now, for some special packet types I wanted to make sure that the packet
was sent before I enqueued the next packet. For this reason, I wanted
to enqueue and then wait until the queue was empty before calling
enqueue for the next element. I understand that it is possible that the
queue never empties if another producer keeps adding elements but my
outputQueue in reality is almost always empty.

I am going to try to change the behavior and use your blocking queue and
simply block when the outputQueue is full. In my implenetation I will
use the nonblocking version of dequeue.

I will let you know if I run into problems.

Thanks again,

Jeronimo
Using my queue above, your consumer could just call the TryDequeue()
method in its loop which will return False if the queue is empty.
However, if your producer(s) are still producing, this empty state may
last for only a timer tick or so as a producer may be right behind
waiting to write. You may have also *just missed an Empty state if
another consumer grabbed the last element and a producer just added
one - you missed the empty state. A reset event may be in order. If
you describe the problem space in more detail, I may be able to come
up with a better idea. You may find what you need in the code.
Please let us know. Cheers.


Jul 21 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

29
by: Paul L. Du Bois | last post by:
Has anyone written a Queue.Queue replacement that avoids busy-waiting? It doesn't matter if it uses os-specific APIs (eg WaitForMultipleObjects). I did some googling around and haven't found...
8
by: Jun | last post by:
hi, just want to know if std::queue empty() is thread safe (running on 2 or more thread)... so i can do, if( !queue.empty() ) { lock() .... unlock()
4
by: Jeronimo Bertran | last post by:
I have a multithreaded application that manipulates a Queue: protected Queue outputQueue; when I add elements to the queue I use a critical section private void Add(OutputRecord record) {...
5
by: mrkbrndck | last post by:
Please see the code below as I am trying to use multithreading for copying files to a new location in a way that improves performance of the client windows application. The problem occurs when 2...
12
by: Perecli Manole | last post by:
I am having some strange thread synchronization problems that require me to better understand the intricacies of Monitor.Wait/Pulse. I have 3 threads. Thread 1 does a Monitor.Wait in a SyncLock...
0
by: NvrBst | last post by:
I want to send a bunch of keys to an external window using PostMessage. I send a "KEYDOWN" message (then have to wait for the program to produce the CHAR message) then send the "KEYUP" message. ...
2
by: tikcireviva | last post by:
Hi Guys, I've done a mulithread queue implementation on stl<queue>, my developement environment is on VC6 as well as FC3. Let's talks about the win32 side. The suspected memory leak is find...
6
by: gskbond | last post by:
Following is my logic to traverse level by level an mway search tree.... (Yeah I finally used templates to implement a queue... :) ) void mWayTree:: TraverseLevelOrder(Node *root,int lvl) {...
5
by: randysimes | last post by:
I have to make a queue utilizing a linked list for an assignment. When I Push() an item to the queue, and try to Pop() or return the Front() element, it says the queue is empty. When I Push() another...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.