473,287 Members | 1,663 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Twisted (or for loops ?) madness

Hi,
Probably not the best group to post my question but I'm sure there is
some people here that use Twisted.
First here is the beginning of my source code:

from twisted.internet import reactor, defer, threads
import time

class CompilerThread(object):
def __init__(self, task, delay):
self.task = task
self.delay = delay

def _processing(self, delay):
print 'Start :', self.task
# Simulate delayed result, to fire immediately use
self.d.callback(self.task)
time.sleep(delay)
return self.task

def compile(self):
print 'Compile :', self.task
print self
# Create Deferred in another thread and add callback
self.d = threads.deferToThread(self._processing,
self.delay).addCallback(self.print_result)
# Return the deferred, this way you could add callback later
return self.d

def print_result(self, result):
# Print result
print 'Compiler result :', result, self.task
# MUST return result otherwise next callback receive None
return result

# Create Compiler objects
ct1 = CompilerThread('*OBJECT 1*', 2)
ct2 = CompilerThread('*OBJECT 2*', 3)
ct3 = CompilerThread('*OBJECT 3*', 5)

# Use succeed to create a deferred already fired
d = defer.succeed(None)

Now my problem:
With this code everything work fine:

d.addCallback(lambda result: ct1.compile())
d.addCallback(lambda result: ct2.compile())
d.addCallback(lambda result: ct3.compile())

reactor.callLater(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.CompilerThread object at 0x00BAD070>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 2*
<__main__.CompilerThread object at 0x00BAD050>
Start : *OBJECT 2*
Compiler result : *OBJECT 2* *OBJECT 2*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CDA4B0>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
But when I try to replace this code with a for loops, something goes
wrong:

l = [ct1, ct2, ct3]
for c in l:
d.addCallback(lambda result: c.compile())

reactor.callLater(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.CompilerThread object at 0x00BAD030>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*

OBJECT 3 run 2 times and OBJECT 2 never ?!?

Any idea ? Maybe something related to Threads ?
Thanks for your help.

Oct 15 '07 #1
6 1420
On Oct 15, 9:46 am, looping <kad...@gmail.comwrote:
l = [ct1, ct2, ct3]
for c in l:
d.addCallback(lambda result: c.compile())

reactor.callLater(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.CompilerThread object at 0x00BAD030>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*

OBJECT 3 run 2 times and OBJECT 2 never ?!?

Any idea ? Maybe something related to Threads ?
Thanks for your help.
After further tests, it look like it is the lambda that cause the
problem:
-Adding a parameter result to compile(self, result)
-Changing d.addCallback(lambda result: c.compile()) for
d.addCallback(c.compile)

And everything run fine.

Why lambda doesn't work ? (variable scope problem ?)

Oct 15 '07 #2
looping wrote:
On Oct 15, 9:46 am, looping <kad...@gmail.comwrote:
>l = [ct1, ct2, ct3]
for c in l:
d.addCallback(lambda result: c.compile())

reactor.callLater(20, reactor.stop)
reactor.run()

Output:

Compile : *OBJECT 1*
<__main__.CompilerThread object at 0x00BAD030>
Start : *OBJECT 1*
Compiler result : *OBJECT 1* *OBJECT 1*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*
Compile : *OBJECT 3*
<__main__.CompilerThread object at 0x00CD9470>
Start : *OBJECT 3*
Compiler result : *OBJECT 3* *OBJECT 3*

OBJECT 3 run 2 times and OBJECT 2 never ?!?

Any idea ? Maybe something related to Threads ?
Thanks for your help.

After further tests, it look like it is the lambda that cause the
problem:
-Adding a parameter result to compile(self, result)
-Changing d.addCallback(lambda result: c.compile()) for
d.addCallback(c.compile)

And everything run fine.

Why lambda doesn't work ? (variable scope problem ?)
Yes. When defined, the lambda inherits the surrounding scope - which
contains the name c, that is bound to the last instance of the iteration.

What you need to do is to introduce a scoped variable inside the lambda at
creation time - e.g. like this:

for a in 1,2,3:
d.addCallback(lambda arg=a: work(a))

Diez
Oct 15 '07 #3
On Oct 15, 12:10 pm, looping <kad...@gmail.comwrote:
>
Why lambda doesn't work ? (variable scope problem ?)
This is a well known issue of for loops. Some believe it to be a bug
but Guido says it
is a design decision, in the sense that Python always do late binding.
If you
browse this list you will find many discussions. Basically if you do

funclist = []
for i in 1,2,3:
def f():
print i
funclist.append(f)

you will get funclist[0]() == funclist[1]() == funclist[2]() == 3 (you
get the latest
binding of "i"). As you see, it has nothing to do with lambdas.
Michele Simionato

Oct 15 '07 #4
On Oct 15, 12:33 pm, Michele Simionato <michele.simion...@gmail.com>
wrote:
is a design decision, in the sense that Python always do late binding.
If you
you will get funclist[0]() == funclist[1]() == funclist[2]() == 3 (you
get the latest
binding of "i"). As you see, it has nothing to do with lambdas.
Thanks Diez, replacing my addCallback with d.addCallback(lambda
result, comp=c: comp.compile()) do the trick.

So if I understand what Michele wrote (thanks too), when a function is
defined (with def), no scope is saved and every variable value not
passed in parameter is lost ? It means that variable value come from
the outer scope when the function is called ?
Oct 15 '07 #5
On Oct 15, 1:01 pm, looping <kad...@gmail.comwrote:
So if I understand what Michele wrote (thanks too), when a function is
defined (with def), no scope is saved and every variable value not
passed in parameter is lost ? It means that variable value come from
the outer scope when the function is called ?
Yes, in my example you get the value of "i" at the function *calling*
time,
not at the function definition time. If you want to store the value at
the
definition time, you must use the default argument trick:

funclist = []
for i in 1,2,3:
def f(i=i):
print i
funclist.append(f)

Michele Simionato

Oct 15 '07 #6
On Oct 15, 1:51 pm, Michele Simionato <michele.simion...@gmail.com>
wrote:
On Oct 15, 1:01 pm, looping <kad...@gmail.comwrote:
So if I understand what Michele wrote (thanks too), when a function is
defined (with def), no scope is saved and every variable value not
passed in parameter is lost ? It means that variable value come from
the outer scope when the function is called ?

Yes, in my example you get the value of "i" at the function *calling*
time,
not at the function definition time. If you want to store the value at
the
definition time, you must use the default argument trick:

funclist = []
for i in 1,2,3:
def f(i=i):
print i
funclist.append(f)

Michele Simionato
Thanks Michele, now I understand how it works and I learned something
new. Not a bad day...

Oct 15 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Paul Moore | last post by:
I hit a problem yesterday with my mail connection. In a desparate attempt to understand what was going on, I wanted to log the connection traffic. After a bit of searching, I found a post on c.l.p...
1
by: Fazer | last post by:
Hello, I am very interested in fooling around with Twisted Matrix's HTTP Web Server. I was thinking if .rpy scripts would run much faster than traditional CGI scripts? After looking how...
2
by: Mark Carter | last post by:
I'm trying to create a mail server in Twisted. I either get SMTPSenderRefused or SMTPException: SMTP AUTH extension not supported by server. What do I need to do to get it to work?
11
by: mir nazim | last post by:
hi, i m planning to start writing intranet applications and want ur real cool advices for choosing the correct platform. the choice is between the three: 1. Twisted 2. Medusa 3. Zope (i do...
2
by: Qp | last post by:
Hello. I'm building a simple chat server and client interface, and I've got everything working except for this: While the client's basic.LineReceiver protocol class can sendLine when a...
2
by: Taki Jeden | last post by:
Hi Anybody used wxPython with twisted? I started putting together a Twisted-based app with wx GUI, and the widgets just don't work - some controls do not show up etc. - at least on my system....
13
by: Charlotte | last post by:
I am developing a web application and am looking for the best framework to do this in. I know this is an old question, and I have read the list archives and a couple of comparison pages on the web....
6
by: Kartic | last post by:
Hello, I downloaded the Win32 installer for Twisted 1.3.0, Python 2.3. The installer, when executed under my login, fails as it requires administrator rights to install (why they have it as a...
2
by: SeSe | last post by:
Hi, I am new to Twisted. I use a Twisted 1.3.0 on MS Windows XP Home Edition, my python version is 2.3 I try the TCP echoserv.py and echoclient.py example. But the client always fail with...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.