473,883 Members | 1,587 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Calling Queue experts

I have a script which is based on the following code. Unfortunately,
it only works on Python 2.3 and not 2.5 because there is no esema or
fsema attribute in the 2.5 Queue. I am hunting through the Queue.py
code now to try to figure out how to make it work in 2.5, but as I am
a beginner, I am having difficulty and would appreciate your help.

Many thanks

Jon

import os
import Queue
import threading
import time
import cPickle

class PickleQueue(Que ue.Queue):
"""A multi-producer, multi-consumer, persistent queue."""
def __init__(self, filename, maxsize=0):
"""Initiali ze a persistent queue with a filename and maximum
size.
The filename is used as a persistent data store for the
queue.
If maxsize <= 0, the queue size is infinite.
"""
self.filename = filename
Queue.Queue.__i nit__(self, maxsize)
if self.queue:
self.esema.rele ase()
if self._full():
self.fsema.acqu ire()
def _init(self, maxsize):
# Implements Queue protocol _init for persistent queue.
# Sets up the pickle files.
self.maxsize = maxsize
try:
self.readfile = file(self.filen ame, 'r')
self.queue = cPickle.load(se lf.readfile)
self.readfile.c lose()
except IOError, err:
if err.errno == 2:
# File doesn't exist, continue ...
self.queue = []
else:
# Some other I/O problem, reraise error
raise err
except EOFError:
# File was null? Continue ...
self.queue = []
# Rewrite file, so it's created if it doesn't exist,
# and raises an exception now if we aren't allowed
self.writefile = file(self.filen ame, 'w')
cPickle.dump(se lf.queue, self.writefile, 1)
def __sync(self):
# Writes the queue to the pickle file.
self.writefile. seek(0)
cPickle.dump(se lf.queue, self.writefile, 1)
self.writefile. flush()
def _put(self, item):
# Implements Queue protocol _put for persistent queue.
self.queue.appe nd(item)
self.__sync()
def _get(self):
# Implements Queue protocol _get for persistent queue.
item = self.queue[0]
del self.queue[0]
self.__sync()
return item

class counterThread(t hreading.Thread ):
numberQueue = PickleQueue('/export/home/jrpf/data.pkl')
exitCounterQueu e = Queue.Queue(1)

def run(self):
command = ''
i = 0
while 1:
self.numberQueu e.put(i)
if i 10:
print "i 10 so attempting to exit"
wt.exit()
self.exit()
print i
try:
command = self.exitCounte rQueue.get(bloc k=False)
except Queue.Empty:
pass
if command == 'exit':
print "Counter thread exited"
break
i = i + 1
time.sleep(1)

def exit(self):
self.exitCounte rQueue.put('exi t')

def main():

ct = counterThread()
ct.setDaemon(Tr ue)
ct.start()
ct.join()

if __name__ == "__main__":
main()

Mar 26 '07 #1
3 2046
Got it. New PickleQueue class should be as follows:

import Queue
import cPickle

class PickleQueue(Que ue.Queue):
"""A multi-producer, multi-consumer, persistent queue."""
def __init__(self, filename, maxsize=0):
"""Initiali ze a persistent queue with a filename and maximum
size.
The filename is used as a persistent data store for the
queue.
If maxsize <= 0, the queue size is infinite.
"""
self.filename = filename
Queue.Queue.__i nit__(self, maxsize)
print self.queue

def _init(self, maxsize):
# Implements Queue protocol _init for persistent queue.
# Sets up the pickle files.
self.maxsize = maxsize
try:
self.readfile = file(self.filen ame, 'r')
self.queue = cPickle.load(se lf.readfile)
self.readfile.c lose()
except IOError, err:
if err.errno == 2:
# File doesn't exist, continue ...
self.queue = Queue.deque()
else:
# Some other I/O problem, reraise error
raise err
except EOFError:
# File was null? Continue ...
self.queue = Queue.deque()
# Rewrite file, so it's created if it doesn't exist,
# and raises an exception now if we aren't allowed
self.writefile = file(self.filen ame, 'w')
cPickle.dump(se lf.queue, self.writefile, 1)
def __sync(self):
# Writes the queue to the pickle file.
self.writefile. seek(0)
cPickle.dump(se lf.queue, self.writefile, 1)
self.writefile. flush()
def _put(self, item):
# Implements Queue protocol _put for persistent queue.
self.queue.appe nd(item)
self.__sync()
def _get(self):
# Implements Queue protocol _get for persistent queue.
item = self.queue.popl eft()
self.__sync()
return item

Mar 26 '07 #2

jrpfinch # Some other I/O problem, reraise error
jrpfinch raise err

I'd just execute a bare raise (without err). That way the caller gets the
stack trace of the actual IOError.

Skip
Mar 26 '07 #3
En Mon, 26 Mar 2007 07:29:32 -0300, jrpfinch <jr******@gmail .comescribió:
Got it. New PickleQueue class should be as follows:
Only a comment:
def _init(self, maxsize):
# Implements Queue protocol _init for persistent queue.
# Sets up the pickle files.
self.maxsize = maxsize
try:
self.readfile = file(self.filen ame, 'r')
self.queue = cPickle.load(se lf.readfile)
self.readfile.c lose()
except IOError, err:
if err.errno == 2:
# File doesn't exist, continue ...
self.queue = Queue.deque()
else:
# Some other I/O problem, reraise error
raise err
except EOFError:
# File was null? Continue ...
self.queue = Queue.deque()
# Rewrite file, so it's created if it doesn't exist,
# and raises an exception now if we aren't allowed
self.writefile = file(self.filen ame, 'w')
cPickle.dump(se lf.queue, self.writefile, 1)
self.readfile may be left open in case of error, I'd use a try/finally.
And since it isn't used anywhere, I'd just use a local variable instead of
an instance attribute.
And the final write is not necesary when you have just read it - and
alters the "last-modified-time" (that may not be relevant for you, of
course, but as a general tool it may confuse other users).

--
Gabriel Genellina

Mar 26 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
2421
by: Matthew Bell | last post by:
Hi, I've got a question about whether there are any issues with directly calling attributes and/or methods of a threaded class instance. I wonder if someone could give me some advice on this. Generally, the documentation suggests that queues or similar constructs should be used for thread inter-process comms. I've had a lot of success in doing that (generally by passing in the queue during the __init__ of the thread) and I can see...
14
3564
by: Mark C. | last post by:
I'm trying to call a batch file that I've built using the FileSystemObject and CreateObject("Wscript.Shell"), oShell.Run... in an asp script. Naturally, I can get the script to work from a command line but not from a browser. The page does not throw an error but the oShell.Run... portion of the script doesn't run. Any help would be appreciated. Thanks.
16
6911
by: William Stacey [MVP] | last post by:
Anyone care to comment on if this non-blocking queue implementation is sound? /// <summary> /// Summary description for NonBlockingQueue. /// Modeled after: http://www.cs.rochester.edu/u/scott/synchronization/pseudocode/queues.html /// </summary> public class NonBlockingQueue {
3
5167
by: Kceiw | last post by:
Dear all, When I use #include "queue.h", I can't link it. The error message follows: Linking... G:\Projects\Datastructure\Queue\Debug\main.o(.text+0x136): In function `main': G:\Projects\Datastructure\Queue\main.cpp:16: undefined reference to `Queue<char>::Queue()' G:\Projects\Datastructure\Queue\Debug\main.o(.text+0x394): In function `Z10do_commandcR5QueueIcE':
4
2027
by: Benny | last post by:
I am creating a thread via "new Thread(new ThreadStart(p.ThreadProc))", is it safe for ThreadProc to use GetMessage/TranslateMessage/DispatchMessage instead of DoEvents? What I would like to do is to have messages posted to ThreadProc and have ThreadProc process those messages (similar to using PostThreadMessage in unmanaged code). Thanks.
7
1659
by: Jeremy Chaney | last post by:
I have an application written in C# that uses objects written in a managed C++ DLL. When I exit my app, my C# classes have their destructors called, but the MC++ objects that those classes hold references to do not get invoked (I can observe this from both breakpoints in the code, and trace output to the console). I was under the impression that when my C# object goes out of scope, it would automatically dispose of all of the references...
12
1309
by: KIRAN | last post by:
hi, the grammer for any programming language says that when a function is called by another function,the callee after executing it's body should return to the point where it left in the caller.. Is there any technique to make the callee to return to some other point(within the current process) other than the callee by changing the call stack in callee... My code runs on 86 processor (If this thread is irrelevent to this group please...
4
4605
by: j_depp_99 | last post by:
Thanks to those guys who helped me out yesterday. I have one more problem; my print function for the queue program doesnt work and goes into an endless loop. Also I am unable to calculate the length of my queue. I started getting compilation errors when I included a length function. <code> template<class ItemType> void Queue<ItemType>::MakeEmpty() {
7
1964
by: jomcfall97 | last post by:
hey wondering if anyone can help me with some work im doing im trying to remove a record from a queue by using a method from a class. class QueueNode { private String document ; private String owner ; private int size ; private QueueNode next ; private QueueNode previous ;
0
9933
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9779
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11114
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10835
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10407
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9563
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5787
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4605
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3230
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.