473,378 Members | 1,417 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,378 software developers and data experts.

Multithread wait for empty queue

I have a multithreaded application that manipulates a Queue:

protected Queue outputQueue;

when I add elements to the queue I use a critical section

private void Add(OutputRecord record)
{
lock(outputQueue)
{
// Add the element
outputQueue.Enqueue(record);
}
}

when I remove an element from the queue I do something similar:

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();
}
return record;
}

Now I want to be able to have a process that adds an element and then
waits until the queue is empty. What is the best way to do this? I
thought of creating an additional object called emptyOutputQueue and
creating a WaitAdd function:

private bool WaitAdd(OutputRecord record)
{
lock(emptyOutputQueue)
{
Add(record);
if (! Monitor.Wait(emptyOutputQueue, 20000))
return false;
}
return true;
}

and changing the Remove function to be something like this

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();

if (outputQueue.Count == 0)
{
lock (emptyOutputQueue)
{
Monitor.PulseAll(emptyOutputQueue);
}
}
}
}
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

Thanks

Jeronimo

Jul 21 '05 #1
4 3472
Jeronimo Bertran <je**************@newsgroup.nospam> wrote:
I have a multithreaded application that manipulates a Queue:
<snip>
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?


For a start, I'd try to use a single lock - your current solution
acquires the two locks in different orders, which means you could have
deadlock.

If you have to use two locks (which you may) you should make sure you
always acquire them in the same order. I'd try to avoid calling other
methods which are going to do locking from within a method which
already locks - it'll make it easier to see this kind of problem.

I don't have time to come up with a solution for you right now, but I
can have another look tonight if you haven't solved it by then.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jul 21 '05 #2
Have a producer/consumer blocking queue here.
http://codeproject.com/csharp/BoundedBlockingQueue.asp
http://channel9.msdn.com/ShowPost.aspx?PostID=50960 (same Q with a server
example)

That also sounds a bit like a Reverse Sensing Semaphore, so you can check
this out too for some ideas:
http://codeproject.com/csharp/Dijkst...gSemaphore.asp

Using my queue above, your consumer could just call the TryDequeue() method
in its loop which will return False if the queue is empty. However, if your
producer(s) are still producing, this empty state may last for only a timer
tick or so as a producer may be right behind waiting to write. You may have
also *just missed an Empty state if another consumer grabbed the last
element and a producer just added one - you missed the empty state. A reset
event may be in order. If you describe the problem space in more detail, I
may be able to come up with a better idea. You may find what you need in
the code. Please let us know. Cheers.
--
William Stacey, MVP
http://mvp.support.microsoft.com

"Jeronimo Bertran" <je**************@newsgroup.nospam> wrote in message
news:#5**************@TK2MSFTNGP14.phx.gbl...
I have a multithreaded application that manipulates a Queue:

protected Queue outputQueue;

when I add elements to the queue I use a critical section

private void Add(OutputRecord record)
{
lock(outputQueue)
{
// Add the element
outputQueue.Enqueue(record);
}
}

when I remove an element from the queue I do something similar:

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();
}
return record;
}

Now I want to be able to have a process that adds an element and then
waits until the queue is empty. What is the best way to do this? I
thought of creating an additional object called emptyOutputQueue and
creating a WaitAdd function:

private bool WaitAdd(OutputRecord record)
{
lock(emptyOutputQueue)
{
Add(record);
if (! Monitor.Wait(emptyOutputQueue, 20000))
return false;
}
return true;
}

and changing the Remove function to be something like this

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();

if (outputQueue.Count == 0)
{
lock (emptyOutputQueue)
{
Monitor.PulseAll(emptyOutputQueue);
}
}
}
}
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

Thanks

Jeronimo


Jul 21 '05 #3
William,

Thanks for the article... I could change the behavior of my queue to
look similar to what you have so that instead of blocking until the
queue is empty, block when the queue is full. Here is a more detailed
description of what I was trying to achieve:

Just like in your example, I am using the output Queue to send packets
over the network. For most packet types, I enqueue the data and the
communication thread dequeues and sends the data over the network. I
could use the locking queue in your example when calling Enqueue,
although right now I am simply returning false when the queue is full
(non blocking version).

Now, for some special packet types I wanted to make sure that the packet
was sent before I enqueued the next packet. For this reason, I wanted
to enqueue and then wait until the queue was empty before calling
enqueue for the next element. I understand that it is possible that the
queue never empties if another producer keeps adding elements but my
outputQueue in reality is almost always empty.

I am going to try to change the behavior and use your blocking queue and
simply block when the outputQueue is full. In my implenetation I will
use the nonblocking version of dequeue.

I will let you know if I run into problems.

Thanks again,

Jeronimo
Using my queue above, your consumer could just call the TryDequeue()
method in its loop which will return False if the queue is empty.
However, if your producer(s) are still producing, this empty state may
last for only a timer tick or so as a producer may be right behind
waiting to write. You may have also *just missed an Empty state if
another consumer grabbed the last element and a producer just added
one - you missed the empty state. A reset event may be in order. If
you describe the problem space in more detail, I may be able to come
up with a better idea. You may find what you need in the code.
Please let us know. Cheers.


Jul 21 '05 #4
Are you using UDP? If so, then you still need to check if the packet was
actually received via some ack. If TCP, then the packets will be in order
so not sure why you need to worry about it. I probably miss something in
your needs. Let me know if can help more. Cheers.

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Jeronimo Bertran" <je**************@newsgroup.nospam> wrote in message
news:Xn********************************@207.46.248 .16...
William,

Thanks for the article... I could change the behavior of my queue to
look similar to what you have so that instead of blocking until the
queue is empty, block when the queue is full. Here is a more detailed
description of what I was trying to achieve:

Just like in your example, I am using the output Queue to send packets
over the network. For most packet types, I enqueue the data and the
communication thread dequeues and sends the data over the network. I
could use the locking queue in your example when calling Enqueue,
although right now I am simply returning false when the queue is full
(non blocking version).

Now, for some special packet types I wanted to make sure that the packet
was sent before I enqueued the next packet. For this reason, I wanted
to enqueue and then wait until the queue was empty before calling
enqueue for the next element. I understand that it is possible that the
queue never empties if another producer keeps adding elements but my
outputQueue in reality is almost always empty.

I am going to try to change the behavior and use your blocking queue and
simply block when the outputQueue is full. In my implenetation I will
use the nonblocking version of dequeue.

I will let you know if I run into problems.

Thanks again,

Jeronimo
Using my queue above, your consumer could just call the TryDequeue()
method in its loop which will return False if the queue is empty.
However, if your producer(s) are still producing, this empty state may
last for only a timer tick or so as a producer may be right behind
waiting to write. You may have also *just missed an Empty state if
another consumer grabbed the last element and a producer just added
one - you missed the empty state. A reset event may be in order. If
you describe the problem space in more detail, I may be able to come
up with a better idea. You may find what you need in the code.
Please let us know. Cheers.


Jul 21 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

29
by: Paul L. Du Bois | last post by:
Has anyone written a Queue.Queue replacement that avoids busy-waiting? It doesn't matter if it uses os-specific APIs (eg WaitForMultipleObjects). I did some googling around and haven't found...
8
by: Jun | last post by:
hi, just want to know if std::queue empty() is thread safe (running on 2 or more thread)... so i can do, if( !queue.empty() ) { lock() .... unlock()
4
by: Jeronimo Bertran | last post by:
I have a multithreaded application that manipulates a Queue: protected Queue outputQueue; when I add elements to the queue I use a critical section private void Add(OutputRecord record) {...
5
by: mrkbrndck | last post by:
Please see the code below as I am trying to use multithreading for copying files to a new location in a way that improves performance of the client windows application. The problem occurs when 2...
12
by: Perecli Manole | last post by:
I am having some strange thread synchronization problems that require me to better understand the intricacies of Monitor.Wait/Pulse. I have 3 threads. Thread 1 does a Monitor.Wait in a SyncLock...
0
by: NvrBst | last post by:
I want to send a bunch of keys to an external window using PostMessage. I send a "KEYDOWN" message (then have to wait for the program to produce the CHAR message) then send the "KEYUP" message. ...
2
by: tikcireviva | last post by:
Hi Guys, I've done a mulithread queue implementation on stl<queue>, my developement environment is on VC6 as well as FC3. Let's talks about the win32 side. The suspected memory leak is find...
6
by: gskbond | last post by:
Following is my logic to traverse level by level an mway search tree.... (Yeah I finally used templates to implement a queue... :) ) void mWayTree:: TraverseLevelOrder(Node *root,int lvl) {...
5
by: randysimes | last post by:
I have to make a queue utilizing a linked list for an assignment. When I Push() an item to the queue, and try to Pop() or return the Front() element, it says the queue is empty. When I Push() another...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.