Multithread wait for empty queue

[Jeronimo Bertran]

I have a multithreaded application that manipulates a Queue:

protected Queue outputQueue;

when I add elements to the queue I use a critical section

private void Add(OutputRecord record)
{
lock(outputQueue)
{
// Add the element
outputQueue.Enqueue(record);
}
}

when I remove an element from the queue I do something similar:

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();
}
return record;
}

Now I want to be able to have a process that adds an element and then
waits until the queue is empty. What is the best way to do this? I
thought of creating an additional object called emptyOutputQueue and
creating a WaitAdd function:

private bool WaitAdd(OutputRecord record)
{
lock(emptyOutputQueue)
{
Add(record);
if (! Monitor.Wait(emptyOutputQueue, 20000))
return false;
}
return true;
}

and changing the Remove function to be something like this

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();

if (outputQueue.Count == 0)
{
lock (emptyOutputQueue)
{
Monitor.PulseAll(emptyOutputQueue);
}
}
}
}
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

Thanks

Jeronimo

[Jon Skeet [C# MVP]]

Jeronimo Bertran <je**************@newsgroup.nospam> wrote:

I have a multithreaded application that manipulates a Queue:
<snip>
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

For a start, I'd try to use a single lock - your current solution
acquires the two locks in different orders, which means you could have
deadlock.

If you have to use two locks (which you may) you should make sure you
always acquire them in the same order. I'd try to avoid calling other
methods which are going to do locking from within a method which
already locks - it'll make it easier to see this kind of problem.

I don't have time to come up with a solution for you right now, but I
can have another look tonight if you haven't solved it by then.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[William Stacey [MVP]]

Have a producer/consumer blocking queue here.
http://codeproject.com/csharp/BoundedBlockingQueue.asp
http://channel9.msdn.com/ShowPost.aspx?PostID=50960 (same Q with a server
example)

That also sounds a bit like a Reverse Sensing Semaphore, so you can check
this out too for some ideas:
http://codeproject.com/csharp/Dijkst...gSemaphore.asp

Using my queue above, your consumer could just call the TryDequeue() method
in its loop which will return False if the queue is empty. However, if your
producer(s) are still producing, this empty state may last for only a timer
tick or so as a producer may be right behind waiting to write. You may have
also *just missed an Empty state if another consumer grabbed the last
element and a producer just added one - you missed the empty state. A reset
event may be in order. If you describe the problem space in more detail, I
may be able to come up with a better idea. You may find what you need in
the code. Please let us know. Cheers.
--
William Stacey, MVP
http://mvp.support.microsoft.com

"Jeronimo Bertran" <je**************@newsgroup.nospam> wrote in message
news:#5**************@TK2MSFTNGP14.phx.gbl...

I have a multithreaded application that manipulates a Queue:

protected Queue outputQueue;

when I add elements to the queue I use a critical section

private void Add(OutputRecord record)
{
lock(outputQueue)
{
// Add the element
outputQueue.Enqueue(record);
}
}

when I remove an element from the queue I do something similar:

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();
}
return record;
}

Now I want to be able to have a process that adds an element and then
waits until the queue is empty. What is the best way to do this? I
thought of creating an additional object called emptyOutputQueue and
creating a WaitAdd function:

private bool WaitAdd(OutputRecord record)
{
lock(emptyOutputQueue)
{
Add(record);
if (! Monitor.Wait(emptyOutputQueue, 20000))
return false;
}
return true;
}

and changing the Remove function to be something like this

private OuputRecord Remove()
{
OutputRecord record;

lock (outputQueue)
{
// Remove the element
record = (OutputRecord) outputQueue.Dequeue();

if (outputQueue.Count == 0)
{
lock (emptyOutputQueue)
{
Monitor.PulseAll(emptyOutputQueue);
}
}
}
}
But I am not sure that the WaitAdd function will actually wait if the
queue is not empty. Any suggestions of what is the best solution?

Thanks

Jeronimo

[Jeronimo Bertran]

William,

Thanks for the article... I could change the behavior of my queue to
look similar to what you have so that instead of blocking until the
queue is empty, block when the queue is full. Here is a more detailed
description of what I was trying to achieve:

Just like in your example, I am using the output Queue to send packets
over the network. For most packet types, I enqueue the data and the
communication thread dequeues and sends the data over the network. I
could use the locking queue in your example when calling Enqueue,
although right now I am simply returning false when the queue is full
(non blocking version).

Now, for some special packet types I wanted to make sure that the packet
was sent before I enqueued the next packet. For this reason, I wanted
to enqueue and then wait until the queue was empty before calling
enqueue for the next element. I understand that it is possible that the
queue never empties if another producer keeps adding elements but my
outputQueue in reality is almost always empty.

I am going to try to change the behavior and use your blocking queue and
simply block when the outputQueue is full. In my implenetation I will
use the nonblocking version of dequeue.

I will let you know if I run into problems.

Thanks again,

Jeronimo

Using my queue above, your consumer could just call the TryDequeue()
method in its loop which will return False if the queue is empty.
However, if your producer(s) are still producing, this empty state may
last for only a timer tick or so as a producer may be right behind
waiting to write. You may have also *just missed an Empty state if
another consumer grabbed the last element and a producer just added
one - you missed the empty state. A reset event may be in order. If
you describe the problem space in more detail, I may be able to come
up with a better idea. You may find what you need in the code.
Please let us know. Cheers.

[William Stacey [MVP]]

Are you using UDP? If so, then you still need to check if the packet was
actually received via some ack. If TCP, then the packets will be in order
so not sure why you need to worry about it. I probably miss something in
your needs. Let me know if can help more. Cheers.

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Jeronimo Bertran" <je**************@newsgroup.nospam> wrote in message
news:Xn********************************@207.46.248 .16...

William,

Thanks for the article... I could change the behavior of my queue to
look similar to what you have so that instead of blocking until the
queue is empty, block when the queue is full. Here is a more detailed
description of what I was trying to achieve:

Just like in your example, I am using the output Queue to send packets
over the network. For most packet types, I enqueue the data and the
communication thread dequeues and sends the data over the network. I
could use the locking queue in your example when calling Enqueue,
although right now I am simply returning false when the queue is full
(non blocking version).

Now, for some special packet types I wanted to make sure that the packet
was sent before I enqueued the next packet. For this reason, I wanted
to enqueue and then wait until the queue was empty before calling
enqueue for the next element. I understand that it is possible that the
queue never empties if another producer keeps adding elements but my
outputQueue in reality is almost always empty.

I am going to try to change the behavior and use your blocking queue and
simply block when the outputQueue is full. In my implenetation I will
use the nonblocking version of dequeue.

I will let you know if I run into problems.

Thanks again,

Jeronimo

Using my queue above, your consumer could just call the TryDequeue()
method in its loop which will return False if the queue is empty.
However, if your producer(s) are still producing, this empty state may
last for only a timer tick or so as a producer may be right behind
waiting to write. You may have also *just missed an Empty state if
another consumer grabbed the last element and a producer just added
one - you missed the empty state. A reset event may be in order. If
you describe the problem space in more detail, I may be able to come
up with a better idea. You may find what you need in the code.
Please let us know. Cheers.