473,581 Members | 2,220 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Twisted (or for loops ?) madness

Hi,
Probably not the best group to post my question but I'm sure there is
some people here that use Twisted.
First here is the beginning of my source code:

from twisted.interne t import reactor, defer, threads
import time

class CompilerThread( object):
def __init__(self, task, delay):
self.task = task
self.delay = delay

def _processing(sel f, delay):
print 'Start :', self.task
# Simulate delayed result, to fire immediately use
self.d.callback (self.task)
time.sleep(dela y)
return self.task

def compile(self):
print 'Compile :', self.task
print self
# Create Deferred in another thread and add callback
self.d = threads.deferTo Thread(self._pr ocessing,
self.delay).add Callback(self.p rint_result)
# Return the deferred, this way you could add callback later
return self.d

def print_result(se lf, result):
# Print result
print 'Compiler result :', result, self.task
# MUST return result otherwise next callback receive None
return result

# Create Compiler objects
ct1 = CompilerThread( '*OBJECT 1*', 2)
ct2 = CompilerThread( '*OBJECT 2*', 3)
ct3 = CompilerThread( '*OBJECT 3*', 5)

# Use succeed to create a deferred already fired
d = defer.succeed(N one)

Now my problem:
With this code everything work fine:

d.addCallback(l ambda result: ct1.compile())
d.addCallback(l ambda result: ct2.compile())
d.addCallback(l ambda result: ct3.compile())

reactor.callLat er(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.Compi lerThread object at 0x00BAD070>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 2*
<__main__.Compi lerThread object at 0x00BAD050>
Start : *OBJECT 2*
Compiler result : *OBJECT 2* *OBJECT 2*
Compile : *OBJECT 3*
<__main__.Compi lerThread object at 0x00CDA4B0>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
But when I try to replace this code with a for loops, something goes
wrong:

l = [ct1, ct2, ct3]
for c in l:
d.addCallback(l ambda result: c.compile())

reactor.callLat er(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.Compi lerThread object at 0x00BAD030>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 3*
<__main__.Compi lerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
Compile : *OBJECT 3*
<__main__.Compi lerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*

OBJECT 3 run 2 times and OBJECT 2 never ?!?

Any idea ? Maybe something related to Threads ?
Thanks for your help.

Oct 15 '07 #1
6 1434
On Oct 15, 9:46 am, looping <kad...@gmail.c omwrote:
l = [ct1, ct2, ct3]
for c in l:
d.addCallback(l ambda result: c.compile())

reactor.callLat er(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.Compi lerThread object at 0x00BAD030>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 3*
<__main__.Compi lerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
Compile : *OBJECT 3*
<__main__.Compi lerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*

OBJECT 3 run 2 times and OBJECT 2 never ?!?

Any idea ? Maybe something related to Threads ?
Thanks for your help.
After further tests, it look like it is the lambda that cause the
problem:
-Adding a parameter result to compile(self, result)
-Changing d.addCallback(l ambda result: c.compile()) for
d.addCallback(c .compile)

And everything run fine.

Why lambda doesn't work ? (variable scope problem ?)

Oct 15 '07 #2
looping wrote:
On Oct 15, 9:46 am, looping <kad...@gmail.c omwrote:
>l = [ct1, ct2, ct3]
for c in l:
d.addCallback(l ambda result: c.compile())

reactor.callLa ter(20, reactor.stop)
reactor.run( )

Output:

Compile : *OBJECT 1*
<__main__.Comp ilerThread object at 0x00BAD030>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 3*
<__main__.Comp ilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
Compile : *OBJECT 3*
<__main__.Comp ilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*

OBJECT 3 run 2 times and OBJECT 2 never ?!?

Any idea ? Maybe something related to Threads ?
Thanks for your help.

After further tests, it look like it is the lambda that cause the
problem:
-Adding a parameter result to compile(self, result)
-Changing d.addCallback(l ambda result: c.compile()) for
d.addCallback(c .compile)

And everything run fine.

Why lambda doesn't work ? (variable scope problem ?)
Yes. When defined, the lambda inherits the surrounding scope - which
contains the name c, that is bound to the last instance of the iteration.

What you need to do is to introduce a scoped variable inside the lambda at
creation time - e.g. like this:

for a in 1,2,3:
d.addCallback(l ambda arg=a: work(a))

Diez
Oct 15 '07 #3
On Oct 15, 12:10 pm, looping <kad...@gmail.c omwrote:
>
Why lambda doesn't work ? (variable scope problem ?)
This is a well known issue of for loops. Some believe it to be a bug
but Guido says it
is a design decision, in the sense that Python always do late binding.
If you
browse this list you will find many discussions. Basically if you do

funclist = []
for i in 1,2,3:
def f():
print i
funclist.append (f)

you will get funclist[0]() == funclist[1]() == funclist[2]() == 3 (you
get the latest
binding of "i"). As you see, it has nothing to do with lambdas.
Michele Simionato

Oct 15 '07 #4
On Oct 15, 12:33 pm, Michele Simionato <michele.simion ...@gmail.com>
wrote:
is a design decision, in the sense that Python always do late binding.
If you
you will get funclist[0]() == funclist[1]() == funclist[2]() == 3 (you
get the latest
binding of "i"). As you see, it has nothing to do with lambdas.
Thanks Diez, replacing my addCallback with d.addCallback(l ambda
result, comp=c: comp.compile()) do the trick.

So if I understand what Michele wrote (thanks too), when a function is
defined (with def), no scope is saved and every variable value not
passed in parameter is lost ? It means that variable value come from
the outer scope when the function is called ?
Oct 15 '07 #5
On Oct 15, 1:01 pm, looping <kad...@gmail.c omwrote:
So if I understand what Michele wrote (thanks too), when a function is
defined (with def), no scope is saved and every variable value not
passed in parameter is lost ? It means that variable value come from
the outer scope when the function is called ?
Yes, in my example you get the value of "i" at the function *calling*
time,
not at the function definition time. If you want to store the value at
the
definition time, you must use the default argument trick:

funclist = []
for i in 1,2,3:
def f(i=i):
print i
funclist.append (f)

Michele Simionato

Oct 15 '07 #6
On Oct 15, 1:51 pm, Michele Simionato <michele.simion ...@gmail.com>
wrote:
On Oct 15, 1:01 pm, looping <kad...@gmail.c omwrote:
So if I understand what Michele wrote (thanks too), when a function is
defined (with def), no scope is saved and every variable value not
passed in parameter is lost ? It means that variable value come from
the outer scope when the function is called ?

Yes, in my example you get the value of "i" at the function *calling*
time,
not at the function definition time. If you want to store the value at
the
definition time, you must use the default argument trick:

funclist = []
for i in 1,2,3:
def f(i=i):
print i
funclist.append (f)

Michele Simionato
Thanks Michele, now I understand how it works and I learned something
new. Not a bad day...

Oct 15 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
4149
by: Paul Moore | last post by:
I hit a problem yesterday with my mail connection. In a desparate attempt to understand what was going on, I wanted to log the connection traffic. After a bit of searching, I found a post on c.l.p from Andrew Bennetts explaining how to run a port forwarder in 2 lines using Twisted. $ mktap portforward -p 8000 -h remote -d 20 $ twistd -f...
1
2451
by: Fazer | last post by:
Hello, I am very interested in fooling around with Twisted Matrix's HTTP Web Server. I was thinking if .rpy scripts would run much faster than traditional CGI scripts? After looking how asynchronous Twisted's framework really is, I have been interested in it. Thanks,
2
7932
by: Mark Carter | last post by:
I'm trying to create a mail server in Twisted. I either get SMTPSenderRefused or SMTPException: SMTP AUTH extension not supported by server. What do I need to do to get it to work?
11
3586
by: mir nazim | last post by:
hi, i m planning to start writing intranet applications and want ur real cool advices for choosing the correct platform. the choice is between the three: 1. Twisted 2. Medusa 3. Zope (i do not know any of the three).
2
6230
by: Qp | last post by:
Hello. I'm building a simple chat server and client interface, and I've got everything working except for this: While the client's basic.LineReceiver protocol class can sendLine when a connection is made (that is, when the class is called automatically), whenever I try to call sendLine from another class (my ClientFactory class) then I get...
2
1804
by: Taki Jeden | last post by:
Hi Anybody used wxPython with twisted? I started putting together a Twisted-based app with wx GUI, and the widgets just don't work - some controls do not show up etc. - at least on my system. For example, if I just take the wxdemo.py shipped with Twisted, and add to it a static text, it doesn't show - while if I change
13
3512
by: Charlotte | last post by:
I am developing a web application and am looking for the best framework to do this in. I know this is an old question, and I have read the list archives and a couple of comparison pages on the web. I played around with some, and I have reduced my choices to two very different frameworks: cherrypy+cheetah and twisted+nevow. I am writing the...
6
2338
by: Kartic | last post by:
Hello, I downloaded the Win32 installer for Twisted 1.3.0, Python 2.3. The installer, when executed under my login, fails as it requires administrator rights to install (why they have it as a requirement, I don't understand). So I started the installer as the admin user. That too failed because I have a non-admin install of Python 2.3...
2
2370
by: SeSe | last post by:
Hi, I am new to Twisted. I use a Twisted 1.3.0 on MS Windows XP Home Edition, my python version is 2.3 I try the TCP echoserv.py and echoclient.py example. But the client always fail with following message: Unhandled error in Deferred: Traceback (most recent call last): File "D:\PYTHON23\Lib\site-packages\twisted\internet\default.py",...
0
7792
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8149
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7899
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6553
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5674
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
3805
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3827
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2301
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1403
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.