470,604 Members | 2,205 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,604 developers. It's quick & easy.

Queue qsize = unreliable?

I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable:

| qsize(self)
| Return the approximate size of the queue (not reliable!).

Any thoughts on why this is unreliable (and more curiously, why it would
be put in there as an unreliable function?) Rather than roll my own
threaded fifo class, it would seem prudent to use Python's built-in
Queue but the warning signs on a rather necessary function seem curious.

Jamie
Jul 18 '05 #1
4 4515
"James R. Saker Jr." <js****@americanrelay.com> writes:
I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable:

| qsize(self)
| Return the approximate size of the queue (not reliable!).

Any thoughts on why this is unreliable (and more curiously, why it would
be put in there as an unreliable function?) Rather than roll my own
threaded fifo class, it would seem prudent to use Python's built-in
Queue but the warning signs on a rather necessary function seem curious.


Well, by the time you examine (or even get!) the answer, it might be
wrong. Kinda hard to avoid this in the setting of multiple threads!

Cheers,
mwh

--
All parts should go together without forcing. You must remember that
the parts you are reassembling were disassembled by you. Therefore,
if you can't get them together again, there must be a reason. By all
means, do not use a hammer. -- IBM maintenance manual, 1925
Jul 18 '05 #2
James R. Saker Jr. wrote:
I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable:

| qsize(self)
| Return the approximate size of the queue (not reliable!).

Any thoughts on why this is unreliable (and more curiously, why it would
be put in there as an unreliable function?) Rather than roll my own
threaded fifo class, it would seem prudent to use Python's built-in
Queue but the warning signs on a rather necessary function seem curious.


(Why do you think this function is necessary? It's probably
rare to really need it, except perhaps during debugging... )

Anyway, the reason it's called "unreliable", though the term
"inaccurate" might be more correct, is because while you are
getting the size of the queue, it might be updated such that
the new size is one or more fewer or larger than the value
that is about to be returned to you. In effect, the value is
guaranteed accurate only for the precise instant in time, now
passed, that it was determined, but by the time the calling
routine actually sees the value the size could be anything.

Note also the latest docs at docs.python.org, which state the
case a little more clearly.

"""Return the approximate size of the queue. Because of
multithreading semantics, this number is not reliable. """

-Peter
Jul 18 '05 #3
On 2004-08-06, Peter Hansen <pe***@engcorp.com> wrote:
I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable:

| qsize(self)
| Return the approximate size of the queue (not reliable!).

Any thoughts on why this is unreliable (and more curiously, why it would
be put in there as an unreliable function?) Rather than roll my own
threaded fifo class, it would seem prudent to use Python's built-in
Queue but the warning signs on a rather necessary function seem curious.


(Why do you think this function is necessary? It's probably
rare to really need it, except perhaps during debugging... )

Anyway, the reason it's called "unreliable", though the term
"inaccurate" might be more correct, is because while you are
getting the size of the queue, it might be updated such that
the new size is one or more fewer or larger than the value
that is about to be returned to you.


I don't think that's any reason to call the function either
unreliable or inaccurate. If you're operating in a
multi-threaded environment, such a statement is trivially true
about anything that accesses shared data.

For example: time.time() needs a disclaimer that it is
unreliable, since the result it returns is incorrect by the
time you get around to using it...

--
Grant Edwards grante Yow! Spreading peanut
at butter reminds me of
visi.com opera!! I wonder why?
Jul 18 '05 #4
In article <ze********************@powergate.ca>,
Peter Hansen <pe***@engcorp.com> wrote:
James R. Saker Jr. wrote:
I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable:

| qsize(self)
| Return the approximate size of the queue (not reliable!).

Any thoughts on why this is unreliable (and more curiously, why it would
be put in there as an unreliable function?) Rather than roll my own
threaded fifo class, it would seem prudent to use Python's built-in
Queue but the warning signs on a rather necessary function seem curious.


(Why do you think this function is necessary? It's probably
rare to really need it, except perhaps during debugging... )

Anyway, the reason it's called "unreliable", though the term
"inaccurate" might be more correct, is because while you are
getting the size of the queue, it might be updated such that
the new size is one or more fewer or larger than the value
that is about to be returned to you. In effect, the value is
guaranteed accurate only for the precise instant in time, now
passed, that it was determined, but by the time the calling
routine actually sees the value the size could be anything.

Note also the latest docs at docs.python.org, which state the
case a little more clearly.

"""Return the approximate size of the queue. Because of
multithreading semantics, this number is not reliable. """


And an inaccurate number is still quite usable. If you are
the only one reading from a queue, you know that the number
is at least as large as indicated. If you read from multiple
queues, you can use the indicated number to pick which queue
to handle first.

This allows you to do pollling and fair scheduling and some other
neat tricks.

The same sort of tricks works for writers as well.

Jacob Hallén

--
Jul 18 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

17 posts views Thread by Bart Nessux | last post: by
9 posts views Thread by phil | last post: by
4 posts views Thread by spinner | last post: by
reply views Thread by Kimmo Laine | last post: by
2 posts views Thread by Arnie | last post: by
1 post views Thread by Jeffrey Barish | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.