Here is a first draft of a PEP for thunks. Please let me know what you
think. If there is a positive response, I will create a real PEP.
I made a patch that implements thunks as described here. It is available
at: http://staff.washington.edu/sabbey/py_do
Good background on thunks can be found in ref. [1].
Simple Thunks
-------------
Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
A Set of Examples
=================
Thunk statements contain a new keyword, 'do', as in the example below. The
body of the thunk is the suite in the 'do' statement; it gets passed to
the function appearing next to 'do'. The thunk gets inserted as the first
argument to the function, reminiscent of the way 'self' is inserted as the
first argument to methods.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
Thunks can also accept arguments:
def f(thunk):
thunk(6,7)
do x,y in f():
# x==6, y==7
stuff(x,y)
The return value can be captured
def f(thunk):
thunk()
return 8
do t=f():
# t not bound yet
stuff()
print t
==> 8
Thunks blend into their environment
def f(thunk):
thunk(6,7)
a = 20
do x,y in f():
a = 54
print a,x,y
==> 54,6,7
Thunks can return values. Since using 'return' would leave it unclear
whether it is the thunk or the surrounding function that is returning, a
different keyword should be used. By analogy with 'for' and 'while' loops,
the 'continue' keyword is used for this purpose:
def f(thunk):
before()
t = thunk()
# t == 11
after()
do f():
continue 11
Exceptions raised in the thunk pass through the thunk's caller's frame
before returning to the frame in which the thunk is defined:
def catch_everything(thunk):
try:
thunk()
except:
pass # SomeException gets caught here
try:
do catch_everything():
raise SomeException
except:
pass # SomeException doesn't get caught here because it was
already caught
Because thunks blend into their environment, a thunk cannot be used after
its surrounding 'do' statement has finished:
thunk_saver = None
def f(thunk):
global thunk_saver
thunk_saver = thunk
do f():
pass
thunk_saver() # exception, thunk has expired
'break' and 'return' should probably not be allowed in thunks. One could
use exceptions to simulate these, but it would be surprising to have
exceptions occur in what would otherwise be a non-exceptional situation.
One would have to use try/finally blocks in all code that calls thunks
just to deal with normal situations. For example, using code like
def f(thunk):
thunk()
prevent_core_meltdown()
with code like
do f():
p = 1
return p
would have a different effect than using it with
do f():
return 1
This behavior is potentially a cause of bugs since these two examples
might seem identical at first glance.
The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
def f(thunk):
frame = thunk.tk_frame
do f():
pass
Motivation
==========
Thunks can be used to solve most of the problems addressed by PEP 310 [2]
and PEP 288 [3].
PEP 310 deals with the abstraction of acquire/release code. Such code is
needed when one needs to acquire a resource before its use and release it
after. This often requires boilerplate, it is easy to get wrong, and
there is no visual indication that the before and after parts of the code
are related. Thunks solve these problems by allowing the acquire/release
code to be written in a single, re-usable function.
def acquire_release(thunk):
f = acquire()
try:
thunk(f)
finally:
f.release()
do t in acquire_release():
print t
More generally, thunks can be used whenever there is a repeated need for
the same code to appear before and after other code. For example,
do WaitCursor():
compute_for_a_long_time()
is more organized, easier to read and less bug-prone than the code
DoWaitCursor(1)
compute_for_a_long_time()
DoWaitCursor(-1)
PEP 288 tries to overcome some of the limitations of generators. One
limitation is that a 'yield' is not allowed in the 'try' block of a
'try'/'finally' statement.
def get_items():
f = acquire()
try:
for i in f:
yield i # syntax error
finally:
f.release()
for i in get_items():
print i
This code is not allowed because execution might never return after the
'yield' statement and therefore there is no way to ensure that the
'finally' block is executed. A prohibition on such yields lessens the
suitability of generators as a way to produce items from a resource that
needs to be closed. Of course, the generator could be wrapped in a class
that closes the resource, but this is a complication one would like to
avoid, and does not ensure that the resource will be released in a timely
manner. Thunks do not have this limitation because the thunk-accepting
function is in control-- execution cannot break out of the 'do' statement
without first passing through the thunk-accepting function.
def get_items(thunk): # <-- "thunk-accepting function"
f = acquire()
try:
for i in f:
thunk(i) # A-OK
finally:
f.release()
do i in get_items():
print i
Even though thunks can be used in some ways that generators cannot, they
are not nearly a replacement for generators. Importantly, one has no
analogue of the 'next' method of generators when using thunks:
def f():
yield 89
yield 91
g = f()
g.next() # == 89
g.next() # == 91
[1] see the "Extended Function syntax" thread, http://mail.python.org/pipermail/pyt...2003-February/
[2] http://www.python.org/peps/pep-0310.html
[3] http://www.python.org/peps/pep-0288.html 27  2335 
Brian Sabbey wrote: Thunk statements contain a new keyword, 'do', as in the example below.
The body of the thunk is the suite in the 'do' statement; it gets passed
to the function appearing next to 'do'. The thunk gets inserted as the
first argument to the function, reminiscent of the way 'self' is
inserted as the first argument to methods.
It would probably make more sense to pass the thunk as the last
argument, not as the first. That would make it easier to create
functions with optional thunks, as in:
def print_nums(start, end, thunk=None):
for num in xrange(start, end+1):
if thunk is not None:
num = thunk(num)
print num
print_nums(1, 3) # prints 1, 2, 3
do num print_nums(1, 3): # prints 2, 4, 6
continue num * 2
Because thunks blend into their environment, a thunk cannot be used
after its surrounding 'do' statement has finished
Why? Ordinary functions don't have that restriction: def foo():
.... x = 1
.... def bar():
.... return x
.... return bar
.... foo()()
1
I think your proposal is very interesting, I've been missing code blocks
in Python more and more as time goes by.
I'll answer to both the 'thunks" proposal and the "suite-based keywords"
proposal here.
I find the Ruby syntax rather dirty though, because it has a lot of
implicit stuff, treats code blocks as different from normal arguments,
allows passing only one code block, needs a proc keyword, has yield
execute an implicit block... all this, coming from a Python "explicit is
better than implicit" background (which makes a lot of sense) is simply
ugly.
I like your syntax but have a few comments.
I'll give you an unordered list of ideas, up to you to do what you like
with them.
Keep in mind that most of the problems come from the "space is
significant" thing, which is IMHO a very good idea, but prevents us from
putting code in expressions, like :
func( a,b, def callback( x ):
print x
)
or does it ? maybe this syntax could be made to work ?
****************************************
Comments on the thunks.
First of all I view code blocks as essential to a language. They are very
useful for a lot of common programming tasks (like defining callbacks in
an elegant way) :
button = create_button( "Save changes" ):
do
self.save()
However it seems your thunks can't take parameters, which to me is a big
drawback. In ruby a thunk can take parameters. Simple examples :
field = edit_field( "initial value", onsubmit={ |value| if
self.validate(value) then do something else alert( "the value is invalid"
) } )
[1,3,4].each { |x| puts x }
This has the advantage that the interface to the thunk (ie. its
parameters) are right there before your eyes instead of being buried in
the thunk invocation code inside the edit_field.
a more complex random example :
fields['password1'] = edit_field( "Enter Password" )
fields['password2'] = edit_field( "Enter it again", onsubmit = {|value,
other_values| if value != other_values['password_1'] then alert('the two
passwords must be the same !") }
So I think it's essential that thunks take parameters and return a value
(to use them effectively as callbacks).
What shall distinguish them from a simple alteration to def(): which
returns the function as a value, and an optional name ? really I don't
know, but it could be the way they handle closures and share local
variables with the defining scope. Or it could be that there is no need
for two kinds of function/blocks and so we can reuse the keyword def() :
If you wish to modify def(), you could do, without creating any keyword :
# standard form
f = def func( params ):
code
# unnamed code block taking params
func = def (params):
code
Note that the two above are equivalent with regard to the variable
"func", ie. func contains the defined function. Actually I find def
funcname() to be bloat, as funcname = def() has the same functionality,
but is a lot more universal.
# unnamed block taking no params
f = def:
code
************************************************** *
Comments on the suite-based keywords.
Did you notice that this was basically a generalized HEREDOC syntax ?
I'd say that explicit is better than implicit, hence...
Your syntax is :
do f(a,b):
a block
passes block as the last parameter of f.
I don't like it because it hides stuff.
I'd write :
f(a,b,@>,@>):
"""a very
large multi-line
string"""
def (x):
print x
Here the @> is a symbol (use whatever you like) to indicate "placeholder
for something which is on the next line".
Indentation indicates that the following lines are indeed argument for
the function. The : at the end of the function call is there for coherence
with the rest of the syntax.
Notice that, this way, there is no need for a "do" keyword, as the code
block created by an anonymous def() is no different that the other
parameter, in this case a multiline string.
This has many advantages.
It will make big statements more readable :
instead of :
f( a,b, [some very big expression made up of nested class constructors
like a form defintion ], c, d )
write :
f( a, b, @>, c, d ):
[the very big expression goes here]
So, independently of code blocks, this already improves the readability
of big statements.
You could also use named parameters (various proposals):
f( a,b, c=@>, d=@> ):
value of c
value of d
or :
f( a,b, @*> ):
value of c
value of d
f( a,b, @**> ):
c: value of c
d: value of d
Notice how this mimics f( a,b, * ) and f(a,b, ** ) for multiple
arguments, and multiple named arguments. Do you like it ? I do. Especially
the named version where you cant' get lost in the param block because you
see their names !
Now if you say that def returns the defined function as a value, you
don't need a do keyword.
So, for instance :
def withfile( fname, thunk, mode = "r" ):
f = open( fname, mode )
thunk(f)
f.close()
then :
withfile( "afile.txt", @>, "w" ):
def (f):
f.write( something )
Now, you may say that the def on an extra line is ugly, then just write :
withfile( "afile.txt", @>, "w" ): def (f):
f.write( something )
If you really like do you can make it a synonym for def and then :
withfile( "afile.txt", @>, "w" ): do (f):
f.write( something )
The two ":" seem a bit weird but I think they're OK. 'break' and 'return' should probably not be allowed in thunks. One
I do think return should be allowed in a thunk. After all it's a function
block, so why ditch useful functionality ?
yield should also be allowed, after all why can a thunk not be a
generator ? This would be powerful.
def withfile( fname, thunk, mode = "r" ):
f = open( fname, mode )
r = thunk(f)
f.close()
return r
val = withfile( "afile.txt", @>, "w" ):
def (f):
f.write( something )
return something
Well, it seems I have no more ideas for now.
What do you think about all this ?
The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
Hm ?
You mean like a function closure, or that local variables defined inside
the thunk will be visible to the caller ?
This might change a lot of things and it becomes like a continuation, are
you going to recode stackless ?
Brian Sabbey wrote: def get_items(thunk): # <-- "thunk-accepting function"
f = acquire()
try:
for i in f:
thunk(i) # A-OK
finally:
f.release()
do i in get_items():
print i
Seems like You want to solve the addressed generator problem by
manipulating the syntax: "make generators look more function like",
because father compiler won't complain ;-) Sorry, but IMO this is
hackery and has nothing to do with good language design and I consider
this as extremely harmfull. Instead of making things explicit it does
it the other way round and tries to make Python code more obscure.
Moreover I can't notice the superiority of thunks in Your other
examples over more common techniques like decorators for pre- and
postconditions and the GOF command pattern. I thinks the place for such
ideas are Michael Hudsons famous bytecodehacks.
-1 for from me for thunks in Python.
Ciao,
Kay
On Fri, 15 Apr 2005 16:44:58 -0700, Brian Sabbey <sa****@u.washington.edu> wrote: Here is a first draft of a PEP for thunks. Please let me know what you
think. If there is a positive response, I will create a real PEP.
I made a patch that implements thunks as described here. It is available
at:
http://staff.washington.edu/sabbey/py_do
Good background on thunks can be found in ref. [1].
UIAM most of that pre-dates decorators. What is the relation of thunks
to decorators and/or how might they interact?
Simple Thunks
-------------
Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
"blend into their environment" is not very precise ;-)
If you are talking about the code executing in the local namespace
as if part of a suite instead of apparently defined in a separate function,
I think I would prefer a different syntax ;-)
A Set of Examples
=================
Thunk statements contain a new keyword, 'do', as in the example below. The
body of the thunk is the suite in the 'do' statement; it gets passed to
the function appearing next to 'do'. The thunk gets inserted as the first
argument to the function, reminiscent of the way 'self' is inserted as the
first argument to methods.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
Meaning "do" forces the body of f to be exec'd in do's local space? What if there
are assignments in f? I don't think you mean that would get executed in do's local space,
that's what the thunk call is presumably supposed to do...
But let's get on to better examples, because this is probably confusing some, and I think there
are better ways to spell most use cases than we're seeing here so far ;-)
I want to explore using the thunk-accepting function as a decorator, and defining an anonymous
callable suite for it to "decorate" instead of using the do x,y in deco: or do f(27, 28): format.
To define an anonymous callable suite (aka thunk), I suggest the syntax for
do x,y in deco:
suite
should be
@deco
(x, y): # like def foo(x, y): without the def and foo
suite
BTW, just dropping the def makes for a named thunk (aka callable suite), e.g.
foo(x, y):
suite
which you could call like
foo(10, 4)
with the local-where-suite-was-define effect of
x = 10
y = 4
suite
BTW, a callable local suite also makes case switching by calling through locals()[xsuitename]()
able to rebind local variables. Also, since a name is visible in an enclosing scope, it could
conceivably provide a mechanism for rebinding there. E.g.,
def outer():
xsuite(arg):
x = arg
def inner():
xsuite(5)
x = 2
print x # => 2
inner()
print x # => 5
But it would be tricky if outer returned inner as a closure.
Or if it returned xsuite, for that matter. Probably simplest to limit
callable suites to the scope where they're defined.
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
I'm not sure how you intend this to work. Above you implied (ISTM ;-)
that the entire body of f would effectively be executed locally. But is that
true? What if after after() in f, there were a last statment hi='from last statement of f'
Would hi be bound at this point in the flow (i.e., after d f(27, 28): stuff() )?
I'm thinking you didn't really mean that. IOW, by magic at the time of calling thunk from the
ordinary function f, thunk would be discovered to be what I call an executable suite, whose
body is the suite of your do statement.
In that case, f iself should not be a callable suite, since its body is _not_ supposed to be called locally,
and other than the fact that before and after got called, it was not quite exact to say it was _equivalent_ to
before()
stuff() # the do suite
after()
In that case, my version would just not have a do, instead defining the do suite
as a temp executable suite, e.g., if instead
we make an asignment in the suite, to make it clear it's not just a calling thing, e.g.,
do f(27, 28):
x = stuff()
then my version with explict name callable suite would be
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
set_x():
x = stuff() # to make it plain it's not just a calling thing
f(set_x, 27, 28)
# x is now visible here as local binding
but a suitable decorator and an anonymous callable suite (thunk defined my way ;-) would make this
@f(27, 28)
(): x = stuff()
Thunks can also accept arguments:
def f(thunk):
thunk(6,7)
do x,y in f():
# x==6, y==7
stuff(x,y)
IMO
@f
(x, y): stuff(x, y) # like def foo(x, y): stuff(x, y)
is clearer, once you get used to the missing def foo format
The return value can be captured
This is just a fallout of f's being an ordinary function right?
IOW, f doesn't really know thunk is not an ordinary callable too?
I imagine that is for the CALL_FUNCTION byte code implementation to discover?
def f(thunk):
thunk()
return 8
do t=f():
# t not bound yet
stuff()
print t
==> 8
That can't be done very well with a decorator, but you could pass an
explicit named callable suite, e.g.,
thunk(): stuff()
t = f(thunk)
Thunks blend into their environment
ISTM this needs earlier emphasis ;-)
def f(thunk):
thunk(6,7)
a = 20
do x,y in f():
a = 54
print a,x,y
==> 54,6,7
IMO that's more readable as
def f(thunk):
thunk(6, 7)
@f
(x, y): # think def foo(x, y): with "def foo" missing to make it a thunk
a = 54
print a,x,y
IMO we need some real use cases, or we'll never be able to decide what's really useful.
Thunks can return values. Since using 'return' would leave it unclear
whether it is the thunk or the surrounding function that is returning, a
different keyword should be used. By analogy with 'for' and 'while' loops,
the 'continue' keyword is used for this purpose:
Gak ;-/
def f(thunk):
before()
t = thunk()
# t == 11
after()
do f():
continue 11
I wouldn't think return would be a problem if the compiler generated a
RETURN_CS_VALUE instead of RETURN_VALUE when it saw the end of
the callable suite (hence _CS_) (or thunk ;-)
Then it's up to f what to do with the result. It might pass it to after() sometimes.
Exceptions raised in the thunk pass through the thunk's caller's frame
before returning to the frame in which the thunk is defined:
But it should be possible to have try/excepts within the thunk, IWT?
def catch_everything(thunk):
try:
thunk()
except:
pass # SomeException gets caught here
try:
do catch_everything():
raise SomeException
except:
pass # SomeException doesn't get caught here because it was
already caught
Because thunks blend into their environment, a thunk cannot be used after
its surrounding 'do' statement has finished:
thunk_saver = None
def f(thunk):
global thunk_saver
thunk_saver = thunk
do f():
pass
thunk_saver() # exception, thunk has expired
Why? IWT the above line would be equivalent to executing the suite (pass) in its place.
What happens if you defined
def f(thunk):
def inner(it):
it()
inner(thunk)
do f():
x = 123
Of course, I'd spell it
@f
(): x = 123
Is there a rule against that (passing thunk on to inner)?
'break' and 'return' should probably not be allowed in thunks. One could
use exceptions to simulate these, but it would be surprising to have
exceptions occur in what would otherwise be a non-exceptional situation.
One would have to use try/finally blocks in all code that calls thunks
just to deal with normal situations. For example, using code like
def f(thunk):
thunk()
prevent_core_meltdown()
with code like
do f():
p = 1
return p
would have a different effect than using it with
do f():
return 1
This behavior is potentially a cause of bugs since these two examples
might seem identical at first glance.
I think less so with decorator and anonymous callable suite format
@f
(): return 1 # as in def foo(): return 1 -- mnemonically removing "def foo"
The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
def f(thunk):
frame = thunk.tk_frame
# no connection with tkinter, right? Maybe thunk._frame would also say be careful ;-)
assert sys._getframe(1) is frame # ?? when does that fail, if it can?
do f():
pass
Motivation
==========
Thunks can be used to solve most of the problems addressed by PEP 310 [2]
and PEP 288 [3].
PEP 310 deals with the abstraction of acquire/release code. Such code is
needed when one needs to acquire a resource before its use and release it
after. This often requires boilerplate, it is easy to get wrong, and
there is no visual indication that the before and after parts of the code
are related. Thunks solve these problems by allowing the acquire/release
code to be written in a single, re-usable function.
def acquire_release(thunk):
f = acquire()
try:
thunk(f)
finally:
f.release()
do t in acquire_release():
print t
That could be done as a callable suite and decorator
@acquire_release
(t): print t # like def foo(t): print t except that it's a thunk (or anonymous callable suite ;-)
BTW, since this callable suite definition is not named, there is no name binding
involved, so @acquire_release as a decorator doesn't have to return anything.
More generally, thunks can be used whenever there is a repeated need for
the same code to appear before and after other code. For example,
do WaitCursor():
compute_for_a_long_time()
is more organized, easier to read and less bug-prone than the code
DoWaitCursor(1)
compute_for_a_long_time()
DoWaitCursor(-1)
That would reduce to
@WaitCursor
(): compute_for_a_long_time()
PEP 288 tries to overcome some of the limitations of generators. One
limitation is that a 'yield' is not allowed in the 'try' block of a
'try'/'finally' statement.
def get_items():
f = acquire()
try:
for i in f:
yield i # syntax error
finally:
f.release()
for i in get_items():
print i
This code is not allowed because execution might never return after the
'yield' statement and therefore there is no way to ensure that the
'finally' block is executed. A prohibition on such yields lessens the
suitability of generators as a way to produce items from a resource that
needs to be closed. Of course, the generator could be wrapped in a class
that closes the resource, but this is a complication one would like to
avoid, and does not ensure that the resource will be released in a timely
manner. Thunks do not have this limitation because the thunk-accepting
function is in control-- execution cannot break out of the 'do' statement
without first passing through the thunk-accepting function.
def get_items(thunk): # <-- "thunk-accepting function"
f = acquire()
try:
for i in f:
thunk(i) # A-OK
finally:
f.release()
do i in get_items():
print i
@get_items
(i): print i
But no yields in the thunk either, that would presumably not be A-OK ;-)
Even though thunks can be used in some ways that generators cannot, they
are not nearly a replacement for generators. Importantly, one has no
analogue of the 'next' method of generators when using thunks:
def f():
yield 89
yield 91
g = f()
g.next() # == 89
g.next() # == 91
[1] see the "Extended Function syntax" thread,
http://mail.python.org/pipermail/pyt...2003-February/
[2] http://www.python.org/peps/pep-0310.html
[3] http://www.python.org/peps/pep-0288.html
It's interesting, but I think I'd like to explore the decorator/callable suite version in some real
use cases. IMO the easy analogy with def foo(args): ... for specifying the thunk call parameters,
and the already established decorator call mechanism make this attractive. Also, if you allow named
thunks (I don't quite understand why they should "expire" if they can be bound to something. The
"thunk-accepting function" does not appear to "know" that the thunk reference will turn out to
be a real thunk as opposed to any other callable, so at that point it's just a reference,
and should be passable anywhere -- except maybe out of its defining scope, which would necessitate
generating a peculiar closure.
For a named callable suite, the decorator would have to return the callable, to preserve the binding,
or change it to something else useful. But without names, I could envisage a stack of decorators like
(not really a stack, since they are independent, and the first just uses the thunk to store a switch
class instance and a bound method to accumulate the cases ;-)
@make_switch
(switch, case): pass
@case(1,2,3,5)
(v): print 'small prime: %s'% v
@case(*'abc')
(v): print 'early alpha: %r'% v
@case()
(v): print 'default case value: %r'%v
and then being able to call
switch('b')
and see early alpha: 'b'
Not that this example demonstrates the local rebinding capability of thunks, which was the whole
purpose of this kind of switch definition ;-/
Just substitute some interesting suites that bind something, in the place of the prints ;-)
Regards,
Bengt Richter
On Fri, 15 Apr 2005 16:44:58 -0700, Brian Sabbey
<sa****@u.washington.edu> wrote:
Simple Thunks
-------------
Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
I'm not familiar with Ruby or Smalltalk. Could you explain this
without referring to them?
A Set of Examples
=================
Thunk statements contain a new keyword, 'do', as in the example below. The
body of the thunk is the suite in the 'do' statement; it gets passed to
the function appearing next to 'do'. The thunk gets inserted as the first
argument to the function, reminiscent of the way 'self' is inserted as the
first argument to methods.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
You can already do this, this way.
def f(thunk):
.... before()
.... thunk()
.... after()
.... def before():
.... print 'before'
.... def after():
.... print 'after'
.... def stuff():
.... print 'stuff'
.... def morestuff():
.... print 'morestuff'
.... f(stuff)
before
stuff
after f(morestuff)
before
morestuff
after
This works with arguments also.
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
Can you explain what 'do' does better?
Why is the 'do' form better than just the straight function call?
f(stuff, 27, 28)
The main difference I see is the call to stuff is implied in the
thunk, something I dislike in decorators. In decorators, it works
that way do to the way the functions get evaluated. Why is it needed
here?
When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
Cheers,
Ron
Leif K-Brooks wrote: Brian Sabbey wrote: Thunk statements contain a new keyword, 'do', as in the example below. The
body of the thunk is the suite in the 'do' statement; it gets passed to the
function appearing next to 'do'. The thunk gets inserted as the first
argument to the function, reminiscent of the way 'self' is inserted as the
first argument to methods.
It would probably make more sense to pass the thunk as the last argument, not
as the first. That would make it easier to create functions with optional
thunks, as in:
def print_nums(start, end, thunk=None):
for num in xrange(start, end+1):
if thunk is not None:
num = thunk(num)
print num
print_nums(1, 3) # prints 1, 2, 3
do num print_nums(1, 3): # prints 2, 4, 6
continue num * 2
That seems like a good idea to me.
I suppose it also makes sense to have the thunk last because it appears
after all the other arguments in the function call. Because thunks blend into their environment, a thunk cannot be used after
its surrounding 'do' statement has finished
Why? Ordinary functions don't have that restriction:
def foo(): ... x = 1
... def bar():
... return x
... return bar
... foo()()
1
Thunks, as I implemented them, don't create a closure. I believe that
creating a closure will require a performance penalty. Since one use of
thunks is in loops, it seems that their performance may often be
important.
I believe that saving the thunk and calling it a later time is a somewhat
hackish way to use thunks. Explicitly defining a function (perhaps with a
suite-based keyword :) ) seems to me to be a more readable way to go.
But, yes, it is an arbitrary restriction. If it turns out that
performance isn't really affected by creating a closure, or that
performance doesn't matter as much as I think it does, then this
restriction could be lifted.
-Brian
On Sat, 16 Apr 2005, Ron_Adam wrote: Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
I'm not familiar with Ruby or Smalltalk. Could you explain this
without referring to them?
Hopefully my example below is more understandable. I realize now that I
should have provided more motivational examples.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
You can already do this, this way.
def f(thunk): ... before()
... thunk()
... after()
... def before(): ... print 'before'
... def after(): ... print 'after'
... def stuff(): ... print 'stuff'
... def morestuff(): ... print 'morestuff'
... f(stuff) before
stuff
after f(morestuff) before
morestuff
after
This works with arguments also.
Yes, much of what thunks do can also be done by passing a function
argument. But thunks are different because they share the surrounding
function's namespace (which inner functions do not), and because they can
be defined in a more readable way.
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
Can you explain what 'do' does better?
Why is the 'do' form better than just the straight function call?
f(stuff, 27, 28)
The main difference I see is the call to stuff is implied in the
thunk, something I dislike in decorators. In decorators, it works
that way do to the way the functions get evaluated. Why is it needed
here?
You're right that, in this case, it would be better to just write
"f(stuff, 27, 28)". That example was just an attempt at describing the
syntax and semantics rather than to provide any sort of motivation. If
the thunk contained anything more than a call to 'stuff', though, it would
not be as easy as passing 'stuff' to 'f'. For example,
do f(27, 28):
print stuff()
would require one to define and pass a callback function to 'f'. To me,
'do' should be used in any situation in which a callback *could* be used,
but rarely is because doing so would be awkward. Probably the simplest
real-world example is opening and closing a file. Rarely will you see
code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you to
make this abstraction in a way that is more concise and more readable than
the callback example given above:
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they allow
variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more
difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
return f.read()
t = with_file(my_read, 'file.txt')
When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
I used 'do' because that's what ruby uses for something similar. It can
be used in a flow control-like way, or as an item-in-a-list way. For
example, you could replace 'if' with your own version (not that you would
want to):
def my_if(thunk, val):
if val:
thunk()
do my_if(a): # same as "if a:"
assert a
(replacing "else" wouldn't be possible without allowing multiple thunks.)
Or you could create your own 'for' (again, NTYWWT):
def my_for(thunk, vals):
for i in vals:
thunk(i)
do i in my_for(range(10)):
print i
-Brian pe****@free.fr wrote:
Keep in mind that most of the problems come from the "space is
significant" thing, which is IMHO a very good idea, but prevents us from
putting code in expressions, like :
func( a,b, def callback( x ):
print x
)
or does it ? maybe this syntax could be made to work ?
Hmm. I'd like to think that suite-based keywords do make this example
work. One just has to let go of the idea that all arguments to a function
appear inside parentheses.
****************************************
Comments on the thunks.
First of all I view code blocks as essential to a language. They are
very useful for a lot of common programming tasks (like defining callbacks in
an elegant way) :
button = create_button( "Save changes" ):
do
self.save()
However it seems your thunks can't take parameters, which to me is a
big drawback. In ruby a thunk can take parameters. Simple examples :
field = edit_field( "initial value", onsubmit={ |value| if
self.validate(value) then do something else alert( "the value is invalid" ) }
)
[1,3,4].each { |x| puts x }
Thunks can take parameters:
do value in field = edit_field("initial value"):
if self.validate(value):
something
else:
alert("the value is invalid")
But callbacks are better defined with suite-based keywords:
field = edit_field("initial value"):
def onsubmit(value):
if self.validate(value):
something
else:
alert("the value is invalid")
This has the advantage that the interface to the thunk (ie. its
parameters) are right there before your eyes instead of being buried in the
thunk invocation code inside the edit_field.
In the cases in which one is defining a callback, I agree that it would be
preferable to have the thunk parameters not buried next to 'do'.
However, I do not consider thunks a replacement for callbacks. They are a
replacement for code in which callbacks could be used, but normally aren't
because using them would be awkward. Your 'withfile' example below is a
good one. Most people wouldn't bother defining a 'withfile' function,
they would just call 'open' and 'close' individually.
So I think it's essential that thunks take parameters and return a
value (to use them effectively as callbacks).
What shall distinguish them from a simple alteration to def(): which
returns the function as a value, and an optional name ? really I don't know,
but it could be the way they handle closures and share local variables with
the defining scope. Or it could be that there is no need for two kinds of
function/blocks and so we can reuse the keyword def() :
Right, defining a function with 'def' is different than defining a thunk
because thunks share the namespace of the surrounding function, functions
do not:
x = 1
def f():
x = 2 # <- creates a new x
g(f)
print x # ==> 1
do g():
x = 2
print x # ==> 2 ( assuming 'g' calls the thunk at least once)
If you wish to modify def(), you could do, without creating any
keyword :
# standard form
f = def func( params ):
code
# unnamed code block taking params
func = def (params):
code
Note that the two above are equivalent with regard to the variable
"func", ie. func contains the defined function. Actually I find def
funcname() to be bloat, as funcname = def() has the same functionality, but
is a lot more universal.
# unnamed block taking no params
f = def:
code
I'm confused. These examples seem to do something different than what a
'do' statement would do. They create a function with a new namespace and
they do not call any "thunk-accepting" function.
************************************************** *
Comments on the suite-based keywords.
Did you notice that this was basically a generalized HEREDOC syntax ?
I'd say that explicit is better than implicit, hence...
Your syntax is :
do f(a,b):
a block
passes block as the last parameter of f.
I don't like it because it hides stuff.
Yes, it hides stuff. It doesn't seem to me any worse than what is done
with 'self' when calling a method though. Once I got used to 'self'
appearing automatically as the first parameter to a method, it wasn't a
big deal for me.
I'd write :
f(a,b,@>,@>):
"""a very
large multi-line
string"""
def (x):
print x
Here the @> is a symbol (use whatever you like) to indicate
"placeholder for something which is on the next line".
Indentation indicates that the following lines are indeed argument
for the function. The : at the end of the function call is there for
coherence with the rest of the syntax.
Notice that, this way, there is no need for a "do" keyword, as the
code block created by an anonymous def() is no different that the other
parameter, in this case a multiline string.
This has many advantages.
It will make big statements more readable :
instead of :
f( a,b, [some very big expression made up of nested class
constructors like a form defintion ], c, d )
write :
f( a, b, @>, c, d ):
[the very big expression goes here]
So, independently of code blocks, this already improves the
readability of big statements.
It might be useful in some situations to be able to define suite-based
arguments by their order rather than their keyword. But, to me, one of
Python's strengths is that it avoids special syntactic characters. So I
can't say I like "@>" very much. Perhaps there is another way of allowing
this.
You could also use named parameters (various proposals):
f( a,b, c=@>, d=@> ):
value of c
value of d
or :
f( a,b, @*> ):
value of c
value of d
f( a,b, @**> ):
c: value of c
d: value of d
Notice how this mimics f( a,b, * ) and f(a,b, ** ) for multiple
arguments, and multiple named arguments. Do you like it ? I do. Especially
the named version where you cant' get lost in the param block because you see
their names !
I like those ideas, but maybe just leave off the "@>".
f(a,b,c=,d=):
c = 1
d = 2
f(a,b,*)
1
2
f(a,b,**)
c = 1
d = 2
Another problem is that one may create bindings in the suite that should
not be keywords. Explicitly defining the keywords would be useful in this
case too. For example,
f(a,b,c=,d=):
c = [i**2 for i in [1,2]] # 'i' is temporary
d = 2
Here 'i' would not be passed as a keyword argument, because it 'c' and 'd'
are explicitly defined as the only keyword arguments.
Now if you say that def returns the defined function as a value, you
don't need a do keyword.
So, for instance :
def withfile( fname, thunk, mode = "r" ):
f = open( fname, mode )
thunk(f)
f.close()
then :
withfile( "afile.txt", @>, "w" ):
def (f):
f.write( something )
Now, you may say that the def on an extra line is ugly, then just write :
withfile( "afile.txt", @>, "w" ): def (f):
f.write( something )
If you really like do you can make it a synonym for def and then :
withfile( "afile.txt", @>, "w" ): do (f):
f.write( something )
The two ":" seem a bit weird but I think they're OK.
Again, there is the problem of a new namespace being created when using
'def'. Also, it's annoying to have to use 'def' (even though one could
just put it on the same line).
'break' and 'return' should probably not be allowed in thunks. One
I do think return should be allowed in a thunk. After all it's a
function block, so why ditch useful functionality ?
yield should also be allowed, after all why can a thunk not be a
generator ? This would be powerful.
def withfile( fname, thunk, mode = "r" ):
f = open( fname, mode )
r = thunk(f)
f.close()
return r
val = withfile( "afile.txt", @>, "w" ):
def (f):
f.write( something )
return something
Well, it seems I have no more ideas for now.
What do you think about all this ?
Well here you're using a type of suite-based keywords, so 'return' is ok.
Inside thunks I still think 'return' would be confusing. I don't think
one can replace thunks with suite-based keywords. The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
Hm ?
You mean like a function closure, or that local variables defined
inside the thunk will be visible to the caller ?
This might change a lot of things and it becomes like a continuation,
are you going to recode stackless ?
I meant that a thunk is not like a function closure. Variables defined in
the thunk won't be visible to the caller of the thunk (except through
tk_frame), but they will be visible to the function surrounding the thunk.
I made an initial implementation, and it didn't require anything like
stackless.
-Brian
Bengt Richter wrote: Good background on thunks can be found in ref. [1].
UIAM most of that pre-dates decorators. What is the relation of thunks
to decorators and/or how might they interact?
Hmm, I think you answered this below better than I could ;).
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
Meaning "do" forces the body of f to be exec'd in do's local space? What if there
are assignments in f? I don't think you mean that would get executed in do's local space,
that's what the thunk call is presumably supposed to do...
Yes, I see now that there is an ambiguity in this example that I did not
resolve. I meant that the suite of the 'do' statement gets wrapped up as
an anonymous function. This function gets passed to 'f' and can be used
by 'f' in the same way as any other function. Bindings created in 'f' are
not visible from the 'do' statement's suite, and vice-versa. That would
be quite trickier than I intended.
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
I'm not sure how you intend this to work. Above you implied (ISTM ;-)
that the entire body of f would effectively be executed locally. But is that
true? What if after after() in f, there were a last statment hi='from last statement of f'
Would hi be bound at this point in the flow (i.e., after d f(27, 28): stuff() )?
I didn't mean to imply that. The body of 'f' isn't executed any
differently if it were called as one normally calls a function. All of
its bindings are separate from the bindings in the 'do' statement's scope.
I'm thinking you didn't really mean that. IOW, by magic at the time of calling thunk from the
ordinary function f, thunk would be discovered to be what I call an executable suite, whose
body is the suite of your do statement.
yes
In that case, f iself should not be a callable suite, since its body is _not_ supposed to be called locally,
and other than the fact that before and after got called, it was not quite exact to say it was _equivalent_ to
before()
stuff() # the do suite
after()
yes, I said "same effect as" instead of "equivalent" so that too much
wouldn't be read from that example. I see now that I should have been
more clear.
In that case, my version would just not have a do, instead defining the do suite
as a temp executable suite, e.g., if instead
we make an asignment in the suite, to make it clear it's not just a calling thing, e.g.,
do f(27, 28):
x = stuff()
then my version with explict name callable suite would be
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
set_x():
x = stuff() # to make it plain it's not just a calling thing
f(set_x, 27, 28)
# x is now visible here as local binding
but a suitable decorator and an anonymous callable suite (thunk defined my way ;-) would make this
@f(27, 28)
(): x = stuff()
Hmm, but this would require decorators to behave differently than they do
now. Currently, decorators do not automatically insert the decorated
function into the argument list. They require you to define 'f' as:
def f(a, b):
def inner(thunk):
before()
thunk()
after()
return inner
Having to write this type of code every time I want an thunk-accepting
function that takes other arguments would be pretty annoying to me.
Thunks can also accept arguments:
def f(thunk):
thunk(6,7)
do x,y in f():
# x==6, y==7
stuff(x,y)
IMO
@f
(x, y): stuff(x, y) # like def foo(x, y): stuff(x, y)
is clearer, once you get used to the missing def foo format
OK. I prefer a new keyword because it seems confusing to me to re-use
decorators for this purpose. But I see your point that decorators can be
used this way if one allows anonymous functions as you describe, and if
one allows decorators to handle the case in which the function being
decorated is anonymous.
The return value can be captured
This is just a fallout of f's being an ordinary function right?
IOW, f doesn't really know thunk is not an ordinary callable too?
I imagine that is for the CALL_FUNCTION byte code implementation to discover?
yes
def f(thunk):
thunk()
return 8
do t=f():
# t not bound yet
stuff()
print t
==> 8 That can't be done very well with a decorator, but you could pass an
explicit named callable suite, e.g.,
thunk(): stuff()
t = f(thunk)
But not having to do it that way was most of the purpose of thunks.
Thunks blend into their environment
ISTM this needs earlier emphasis ;-)
def f(thunk):
thunk(6,7)
a = 20
do x,y in f():
a = 54
print a,x,y
==> 54,6,7
IMO that's more readable as
def f(thunk):
thunk(6, 7)
@f
(x, y): # think def foo(x, y): with "def foo" missing to make it a thunk
a = 54
print a,x,y
IMO we need some real use cases, or we'll never be able to decide what's really useful.
Thunks can return values. Since using 'return' would leave it unclear
whether it is the thunk or the surrounding function that is returning, a
different keyword should be used. By analogy with 'for' and 'while' loops,
the 'continue' keyword is used for this purpose:
Gak ;-/
def f(thunk):
before()
t = thunk()
# t == 11
after()
do f():
continue 11
I wouldn't think return would be a problem if the compiler generated a
RETURN_CS_VALUE instead of RETURN_VALUE when it saw the end of
the callable suite (hence _CS_) (or thunk ;-)
Then it's up to f what to do with the result. It might pass it to after() sometimes.
It wouldn't be a problem to use 'return' instead of 'continue' if people
so desired, but I find 'return' more confusing because a 'return' in 'for'
suites returns from the function, not from the suite. That is, having
'return' mean different things in these two pieces of code would be
confusing:
for i in [1,2]:
return 10
do i in each([1,2]):
return 10
Exceptions raised in the thunk pass through the thunk's caller's frame
before returning to the frame in which the thunk is defined: But it should be possible to have try/excepts within the thunk, IWT?
yes, it is possible and they are allowed in the example implementation.
def catch_everything(thunk):
try:
thunk()
except:
pass # SomeException gets caught here
try:
do catch_everything():
raise SomeException
except:
pass # SomeException doesn't get caught here because it was
already caught
Because thunks blend into their environment, a thunk cannot be used after
its surrounding 'do' statement has finished:
thunk_saver = None
def f(thunk):
global thunk_saver
thunk_saver = thunk
do f():
pass
thunk_saver() # exception, thunk has expired
Why? IWT the above line would be equivalent to executing the suite (pass) in its place.
The restriction on saving thunks for later is for performance reasons and
because I believe it is hacky to use thunks in that way.
What happens if you defined
def f(thunk):
def inner(it):
it()
inner(thunk)
do f():
x = 123
Of course, I'd spell it
@f
(): x = 123
Is there a rule against that (passing thunk on to inner)?
No, that is fine, as long as execution has not yet left the 'do'
statement. 'break' and 'return' should probably not be allowed in thunks. One could
use exceptions to simulate these, but it would be surprising to have
exceptions occur in what would otherwise be a non-exceptional situation.
One would have to use try/finally blocks in all code that calls thunks
just to deal with normal situations. For example, using code like
def f(thunk):
thunk()
prevent_core_meltdown()
with code like
do f():
p = 1
return p
would have a different effect than using it with
do f():
return 1
This behavior is potentially a cause of bugs since these two examples
might seem identical at first glance.
I think less so with decorator and anonymous callable suite format
@f
(): return 1 # as in def foo(): return 1 -- mnemonically removing "def foo"
In your syntax, 'return' would return from the thunk. With the 'do'
syntax, 'return' would return from the surrounding function. So the issue
does not arise with your syntax.
The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
def f(thunk):
frame = thunk.tk_frame
# no connection with tkinter, right? Maybe thunk._frame would also say be careful ;-)
assert sys._getframe(1) is frame # ?? when does that fail, if it can?
'tk' is supposed to remind one of 'thunk'. I believe that follows a
standard naming convention in python (e.g. see generators).
I don't see how that assert can fail.
I see what you're getting at with decorators and anonymous functions, but
there are a couple of things that, to me, make it worth coming up with a
whole new syntax. First, I don't like how one does not get the thunk
inserted automatically into the argument list of the decorator, as I
described above. Also, I don't like how the return value of the decorator
cannot be captured (and is already used for another purpose). The fact
that decorators require one more line of code is also a little bothersome.
Decorators weren't intended to be used this way, so it seems somewhat
hacky to do so. Why not a new syntax?
-Brian
On Sat, 16 Apr 2005 17:25:00 -0700, Brian Sabbey
<sa****@u.washington.edu> wrote: You can already do this, this way.
> def f(thunk): ... before()
... thunk()
... after()
...> def before():
... print 'before'
...> def after():
... print 'after'
...> def stuff():
... print 'stuff'
...> def morestuff():
... print 'morestuff'
...> f(stuff)
before
stuff
after> f(morestuff)
before
morestuff
after>
This works with arguments also.
Yes, much of what thunks do can also be done by passing a function
argument. But thunks are different because they share the surrounding
function's namespace (which inner functions do not), and because they can
be defined in a more readable way.
Generally my reason for using a function is to group and separate code
from the current name space. I don't see that as a drawback.
Are thunks a way to group and reuse expressions in the current scope?
If so, why even use arguments? Wouldn't it be easier to just declare
what I need right before calling the group? Maybe just informally
declare the calling procedure in a comment.
def thunkit: # no argument list defines a local group.
# set thunk,x and y before calling.
before()
result = thunk(x,y)
after()
def foo(x,y):
x, y = y, x
return x,y
thunk = foo
x,y = 1,2
do thunkit
print result
-> (2,1)
Since everything is in local name space, you really don't need to
pass arguments or return values.
The 'do' keyword says to evaluate the group. Sort of like eval() or
exec would, but in a much more controlled way. And the group as a
whole can be passed around by not using the 'do'. But then it starts
to look and act like a class with limits on it. But maybe a good
replacement for lambas?
I sort of wonder if this is one of those things that looks like it
could be useful at first, but it turns out that using functions and
class's in the proper way, is also the best way. (?)
You're right that, in this case, it would be better to just write
"f(stuff, 27, 28)". That example was just an attempt at describing the
syntax and semantics rather than to provide any sort of motivation. If
the thunk contained anything more than a call to 'stuff', though, it would
not be as easy as passing 'stuff' to 'f'. For example,
do f(27, 28):
print stuff()
would require one to define and pass a callback function to 'f'. To me,
'do' should be used in any situation in which a callback *could* be used,
but rarely is because doing so would be awkward. Probably the simplest
real-world example is opening and closing a file. Rarely will you see
code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you to
make this abstraction in a way that is more concise and more readable than
the callback example given above:
How would abstracting open/close help reduce bugs?
I'm really used to using function calls, so anything that does things
differently tend to be less readable to me. But this is my own
preference. What is most readable to people tends to be what they use
most. IMHO
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they allow
variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more
difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
return f.read()
t = with_file(my_read, 'file.txt')
Wouldn't your with_file thunk def look pretty much the same as the
callback?
I wouldn't use either of these examples. To me the open/read/close
example you gave as the normal case would work fine, with some basic
error checking of course. Since Python's return statement can handle
multiple values, it's no problem to put everything in a single
function and return both the status with an error code if any, and the
result. I would keep the open, read/write, and close statements in
the same function and not split them up. When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
I used 'do' because that's what ruby uses for something similar.
I could see using do as an inverse 'for' operator. Where 'do' would
give values to items in a list, verses taking them from a list. I'm
not exactly sure how that would work. Maybe...
def fa(a,b):
return a+b
def fb(c,d):
return c*b
def fc(e,f):
return e**f
fgroup:
fa(a,b)
fb(c,d)
fc(e,f)
results = do 2,3 in flist
print results
-> (5, 6, 8)
But this is something else, and not what your thunk is trying to do.
So it looks to me you have two basic concepts here.
(1.) Grouping code in local name space.
I can see where this could be useful. It would be cool if the group
inherited the name space it was called with.
(2.) A way to pass values to items in the group.
Since you are using local space, that would be assignment statements
in place of arguments.
Would this work ok for what you want to do?
def with_file: # no argument list, local group.
f = open(filename)
t = callback(f)
f.close
def my_read(f):
return f.read()
callback = my_read
filename = 'filename'
do with_file
Something I've noticed is that there seems to be a trend to want to
put things behind other things as with decorators.
do with_file:
filename = 'filename'
callback = my_read
This in my opinion is yet a third item. Could be that 'do' could also
relate to that as in do 'a' after 'b'.
do: # do this after next group
do with_file # do group named with_file
after: # don't like 'after', but what else?
filename = 'filename'
callback = my_read
Any way, this if just food for thought, I'm really undecided on most
of this.
Cheers,
Ron
On Sat, 16 Apr 2005 18:46:28 -0700, Brian Sabbey <sa****@u.washington.edu> wrote:
[...] In that case, my version would just not have a do, instead defining the do suite
as a temp executable suite, e.g., if instead
we make an asignment in the suite, to make it clear it's not just a calling thing, e.g.,
do f(27, 28):
x = stuff()
then my version with explict name callable suite would be
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
set_x():
x = stuff() # to make it plain it's not just a calling thing
f(set_x, 27, 28)
# x is now visible here as local binding
but a suitable decorator and an anonymous callable suite (thunk defined my way ;-) would make this
@f(27, 28)
(): x = stuff()
Hmm, but this would require decorators to behave differently than they do
now. Currently, decorators do not automatically insert the decorated
function into the argument list. They require you to define 'f' as:
def f(a, b):
def inner(thunk):
before()
thunk()
after()
return inner
Having to write this type of code every time I want an thunk-accepting
function that takes other arguments would be pretty annoying to me.
Yes, I agree. That's the way it is with the decorator expression.
Maybe decorator syntax is not the way to pass thunks to a function ;-)
My latest thinking is in terms of suite expressions, ::<suite> being one,
and actually (<arglist>):<suite> is also a suite expression, yielding a thunk.
So the call to f could just be written explicitly with the expression in line:
f((():x=stuff()), 27, 28)
or
f(():
x = stuff()
done = True
,27, 28) # letting the dedented ',' terminate the ():<suite> rather than parenthesizing
or
safe_open((f):
for line in f:
print f[:20]
,'datafile.txt', 'rb')
That's not too bad IMO ;-)
Thunks can also accept arguments:
def f(thunk):
thunk(6,7)
do x,y in f():
# x==6, y==7
stuff(x,y)
IMO
@f
(x, y): stuff(x, y) # like def foo(x, y): stuff(x, y)
is clearer, once you get used to the missing def foo format
OK. I prefer a new keyword because it seems confusing to me to re-use
decorators for this purpose. But I see your point that decorators can be
used this way if one allows anonymous functions as you describe, and if
one allows decorators to handle the case in which the function being
decorated is anonymous.
I tend to agree now about using decorators for this.
With thunk calling parameter and extra f calling parameters, in line would look like
f((x,y):
# x==6, y==7
stuff(x, y)
,'other' ,'f' ,args)
I guess you could make a bound method to keep the thunk
dothunk_n_all = f.__get__((x, y):
stuff(x,y)
,type(():pass))
and then call that with whatever other parameter you specified for f
do_thunk_n_all(whatever)
if that seemed useful ;-)
The return value can be captured
This is just a fallout of f's being an ordinary function right?
IOW, f doesn't really know thunk is not an ordinary callable too?
I imagine that is for the CALL_FUNCTION byte code implementation to discover?
yes
def f(thunk):
thunk()
return 8
do t=f():
# t not bound yet
stuff()
print t
==> 8
That can't be done very well with a decorator, but you could pass an
explicit named callable suite, e.g.,
thunk(): stuff()
t = f(thunk)
But not having to do it that way was most of the purpose of thunks.
I forgot that ():<suite> is an expression
t = f(():stuff()) # minimal version
or
final_status = safe_open((f):
for line in f:
print f[:20]
,'datafile.txt', 'rb')
<snip>
It wouldn't be a problem to use 'return' instead of 'continue' if people
so desired, but I find 'return' more confusing because a 'return' in 'for'
suites returns from the function, not from the suite. That is, having
'return' mean different things in these two pieces of code would be
confusing:
for i in [1,2]:
return 10
do i in each([1,2]):
return 10
But in my syntax,
each((i):
return 10
,[1,2])
Um, well, I guess one has to think about it ;-/
The thunk-accepter could pass the thunk a mutable arg to
put a return value in, or even a returnvalue verse thunk?
def accepter(thk, seq):
acquire()
for i in seq:
thk(i, (retval):pass)
if retval: break
release()
accepter((i, rvt):
print i
rvt(i==7) # is this legal?
, xrange(10))
Hm, one thing my syntax does, I just noticed, is allow you
to pass several thunks to a thunk-accepter, if desired, e.g.,
(parenthesizing this time, rather than ending ():<suite> with
dedented comma)
each(
((i): # normal thunk
print i),
((j): # alternative thunk
rejectlist.append(j)),
[1,2])
<snip>
I see what you're getting at with decorators and anonymous functions, but
there are a couple of things that, to me, make it worth coming up with a
whole new syntax. First, I don't like how one does not get the thunk
inserted automatically into the argument list of the decorator, as I
described above. Also, I don't like how the return value of the decorator
cannot be captured (and is already used for another purpose). The fact
that decorators require one more line of code is also a little bothersome.
Decorators weren't intended to be used this way, so it seems somewhat
hacky to do so. Why not a new syntax?
Ok, I agree this does not fit decorators well. And I do agree that a nice
syntax that sugars over the creation and passing of thunks makes for clean
simple cases, but I would like access to the non-sugar primitives too, which for
do <opt assignment> <thunk arg list> in <callable>(<arglist>):
<thunk-defining suite>
IIUC translates to my
< opt assignment> <callable>(((<thunk arg list>):
<thunk-defining-suite>),<arglist>)
With possible paren dropping and white space insertion if you want to arrange things.
With the primitives, I can pass multiple thunks to an accepter, and put them in
specific arg positions. I also can also bind it and use it to get a side effect
out of a generator expression that now has its own scope, e.g.,
count = 0
counter(count): count +=1
list(i for i in xrange(20) if counter() or True)
Interestingly, this ought to work, right?
stopper(i): if i>5: raise StopIteration
list(stopper(i) or i for i in xrange(20))
All untested hadwaving, of course ;-)
Regards,
Bengt Richter
Ron_Adam wrote: I sort of wonder if this is one of those things that looks like it
could be useful at first, but it turns out that using functions and
class's in the proper way, is also the best way. (?)
I think Your block is more low level. It is like copying and pasting
code-fragments together but in reversed direction: if You copy and
paste a code fragment and wants to change the behaviour You probable
have to change the fragment the same way in every place of occurence.
This is a well known anti-pattern in software construction. If You
change Your thunk somehow e.g. from
def yield_thunk:
yield i
to
def yield_thunk:
yield j
You have to change all the environments that use the thunk e.g.
renaming variables. It is the opposite direction of creating
abstractions i.e. a method to deabstract functions: introduce less
modularity and more direct dependencies. This is the reason why those
macros are harmfull and should be abandoned from high level languages (
using them in C is reasonable because code expansion can be time
efficient and it is also a way to deal with parametric polymorphism but
Python won't benefit from either of this issues ).
Ciao,
Kay
On 17 Apr 2005 01:46:14 -0700, "Kay Schluehr" <ka**********@gmx.net>
wrote: Ron_Adam wrote:
I sort of wonder if this is one of those things that looks like it
could be useful at first, but it turns out that using functions and
class's in the proper way, is also the best way. (?)
I think Your block is more low level.
Yes, that's my thinking too. I'm sort of looking to see if there is a
basic building blocks here that can be used in different situations
but in very consistent ways. And questioning the use of it as well.
It is like copying and pasting
code-fragments together but in reversed direction: ...
Yes, in a function call, you send values to a remote code block and
receive back a value.
The reverse is to get a remote code block, then use it.
In this case the inserted code blocks variables become local, So my
point is you don't need to use arguments to pass values. But you do
need to be very consistent in how you use the code block. (I'm not
suggesting we do this BTW)
The advantage to argument passing in this case would be that it puts a
control on the block that certain arguments get assigned before it
gets executed.
Is it possible to have a tuple argument translation independently of a
function call? This would also be a somewhat lower level operation,
but might be useful, for example at a certain point in a program you
want to facilitate that certain values are set, you could use a tuple
argument parser to do so. It could act the same way as a function
call argument parser but could be used in more places and it would
raise an error as expected. Basically it would be the same as:
def argset(x,y,z=1):
return x,y,z
But done in a inline way.
a,b,c = (x,y,z=1) # looks familiar doesn't it. ;-)
As an inline expression it could use '%' like the string methods,
something like this?
(x,y,z=1)%a,b,c # point x,y,z to a,b,c (?)
And combined with code chunks like this.
def chunk: # code chunk, ie.. no arguments.
# set (x,y) # informal arguments commented
return x+y # return a value for inline use
value = (x,y)%a,b: chunk # use local code chunk as body
I think this might resemble some of the suggested lambda replacements.
The altenative might be just to have functions name space imported as
an option.
def f(x,y):
return x+y
z = dolocal f(1,2) #But why would I want to do this?
I think these pre-peps are really about doing more with less typing
and don't really add anything to Python. I also feel that the
additional abstraction when used only to compress code, will just make
programs harder to understand.
You have to change all the environments that use the thunk e.g.
renaming variables. It is the opposite direction of creating
abstractions i.e. a method to deabstract functions: introduce less
modularity and more direct dependencies. This is the reason why those
macros are harmfull and should be abandoned from high level languages (
using them in C is reasonable because code expansion can be time
efficient and it is also a way to deal with parametric polymorphism but
Python won't benefit from either of this issues ).
Ciao,
Kay
I agree. If a code block of this type is used it should be limited to
within the function it's defined in. The advantage, if it's used in a
bare bones low level way with no argument passing, would be some
performance benefits over function calls in certain situations. That
is, a small reusable code block without the function call overhead.
But only use it in the local name space it's defined in. Otherwise
use a function or a class.
Cheers,
Ron
Ron_Adam wrote: On Sat, 16 Apr 2005 17:25:00 -0700, Brian Sabbey Yes, much of what thunks do can also be done by passing a function
argument. But thunks are different because they share the surrounding
function's namespace (which inner functions do not), and because they can
be defined in a more readable way.
Generally my reason for using a function is to group and separate code
from the current name space. I don't see that as a drawback.
I agree that one almost always wants separate namespaces when defining a
function, but the same is not true when considering only callback
functions. Thunks are a type of anonymous callback functions, and so a
separate namespace usually isn't required or desired.
Are thunks a way to group and reuse expressions in the current scope?
If so, why even use arguments? Wouldn't it be easier to just declare
what I need right before calling the group? Maybe just informally
declare the calling procedure in a comment.
def thunkit: # no argument list defines a local group.
# set thunk,x and y before calling.
before()
result = thunk(x,y)
after()
def foo(x,y):
x, y = y, x
return x,y
thunk = foo
x,y = 1,2
do thunkit
print result
-> (2,1)
Since everything is in local name space, you really don't need to
pass arguments or return values.
I'm kicking myself for the first example I gave in my original post in
this thread because, looking at it again, I see now that it really gives
the wrong impression about what I want thunks to be in python. The
'thunkit' function above shouldn't be in the same namespace as the thunk.
It is supposed to be a re-usable function, for example, to acquire and
release a resource. On the other hand, the 'foo' function is supposed to
be in the namespace of the surrounding code; it's not re-usable. So your
example above is pretty much the opposite of what I was trying to get
across.
The 'do' keyword says to evaluate the group. Sort of like eval() or
exec would, but in a much more controlled way. And the group as a
whole can be passed around by not using the 'do'. But then it starts
to look and act like a class with limits on it. But maybe a good
replacement for lambas?
I sort of wonder if this is one of those things that looks like it
could be useful at first, but it turns out that using functions and
class's in the proper way, is also the best way. (?)
I don't think so. My pickled_file example below can't be done as cleanly
with a class. If I were to want to ensure the closing of the pickled
file, the required try/finally could not be encapsulated in a class or
function.
You're right that, in this case, it would be better to just write
"f(stuff, 27, 28)". That example was just an attempt at describing the
syntax and semantics rather than to provide any sort of motivation. If
the thunk contained anything more than a call to 'stuff', though, it would
not be as easy as passing 'stuff' to 'f'. For example,
do f(27, 28):
print stuff()
would require one to define and pass a callback function to 'f'. To me,
'do' should be used in any situation in which a callback *could* be used,
but rarely is because doing so would be awkward. Probably the simplest
real-world example is opening and closing a file. Rarely will you see
code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you to
make this abstraction in a way that is more concise and more readable than
the callback example given above:
How would abstracting open/close help reduce bugs?
I gave two examples of bugs that one can encounter when using open/close.
Personally, I have run into the first one at least once.
I'm really used to using function calls, so anything that does things
differently tend to be less readable to me. But this is my own
preference. What is most readable to people tends to be what they use
most. IMHO
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they allow
variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more
difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
return f.read()
t = with_file(my_read, 'file.txt')
Wouldn't your with_file thunk def look pretty much the same as the
callback?
It would look exactly the same. You would be able to use the same
'with_file' function in both situations.
I wouldn't use either of these examples. To me the open/read/close
example you gave as the normal case would work fine, with some basic
error checking of course.
But worrying about the error checking is what one wants to avoid. Even if
it is trivial to remember to close a file, it's annoying to have to think
about this every time one wants to use a file. It would be nice to be
able to worry about closing the file exactly once.
It's also annoying to have to use try/finally over and over again as one
would in many real-life situations. It would be nice to be able to think
about the try/finally code once and put it in a re-usable function.
The open/close file is just the simplest example. Instead of a file, it
maybe be a database or something more complex.
Since Python's return statement can handle
multiple values, it's no problem to put everything in a single
function and return both the status with an error code if any, and the
result. I would keep the open, read/write, and close statements in
the same function and not split them up.
What about try/finally? What if it is more complex than just opening and
closing a file? The example that got me annoyed enough to write this
pre-PEP is pickling and unpickling. I want to unpickle a file, modify it,
and immediately pickle it again. This is a pretty easy thing to do, but
if you're doing it over and over again, there gets to be a lot of
boilerplate. One can of course create a class to handle the boilerplate,
but instantiating a class is still more complicated than it has to be.
Here is an example of using thunks to do this:
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
Now I can re-use pickled_file whenever I have to modify a pickled file:
do data in pickled_file('pickled.txt'):
data.append('more data')
data.append('even more data')
In my opinion, that is easier and faster to write, more readable, and less
bug-prone than any non-thunk alternative. When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
I used 'do' because that's what ruby uses for something similar.
I could see using do as an inverse 'for' operator. Where 'do' would
give values to items in a list, verses taking them from a list. I'm
not exactly sure how that would work. Maybe...
def fa(a,b):
return a+b
def fb(c,d):
return c*b
def fc(e,f):
return e**f
fgroup:
fa(a,b)
fb(c,d)
fc(e,f)
results = do 2,3 in flist
print results
-> (5, 6, 8)
But this is something else, and not what your thunk is trying to do.
So it looks to me you have two basic concepts here.
(1.) Grouping code in local name space.
I can see where this could be useful. It would be cool if the group
inherited the name space it was called with.
(2.) A way to pass values to items in the group.
Since you are using local space, that would be assignment statements
in place of arguments.
Would this work ok for what you want to do?
def with_file: # no argument list, local group.
f = open(filename)
t = callback(f)
f.close
def my_read(f):
return f.read()
callback = my_read
filename = 'filename'
do with_file
This wouldn't work since with_file wouldn't be re-usable. It also doesn't
get rid of the awkwardness of defining a callback.
-Brian
Bengt Richter wrote: Hm, one thing my syntax does, I just noticed, is allow you
to pass several thunks to a thunk-accepter, if desired, e.g.,
(parenthesizing this time, rather than ending ():<suite> with
dedented comma)
each(
((i): # normal thunk
print i),
((j): # alternative thunk
rejectlist.append(j)),
[1,2])
<snip>
I see that it might be nice to be able to use multiple thunks, and to be
able to specify the positions in the argument list of thunks, but I think
allowing a suite inside parentheses is pretty ugly. One reason is that it
is difficult to see where the suite ends and where the argument list
begins again. I'm not sure even what the syntax would be exactly. I
suppose the suite would always have to be inside its own parentheses?
Also, you wind up with these closing parentheses far away from their
corresponding open parentheses, which is also not pretty. It's getting
too Lisp-like for my tastes.
-Brian
On Sun, 17 Apr 2005 15:02:12 -0700, Brian Sabbey
<sa****@u.washington.edu> wrote:
Brian Sabbey wrote: I'm kicking myself for the first example I gave in my original post in
this thread because, looking at it again, I see now that it really gives
the wrong impression about what I want thunks to be in python. The
'thunkit' function above shouldn't be in the same namespace as the thunk.
It is supposed to be a re-usable function, for example, to acquire and
release a resource. On the other hand, the 'foo' function is supposed to
be in the namespace of the surrounding code; it's not re-usable. So your
example above is pretty much the opposite of what I was trying to get
across.
This would explain why I'm having trouble seeing it then.
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
Now I can re-use pickled_file whenever I have to modify a pickled file:
do data in pickled_file('pickled.txt'):
data.append('more data')
data.append('even more data')
In my opinion, that is easier and faster to write, more readable, and less
bug-prone than any non-thunk alternative.
The above looks like it's missing something to me. How does 'data'
interact with 'thunk(l)'? What parts are in who's local space?
This might be the non-thunk version of the above.
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
def data_append(L):
L.append('more data')
L.append('still more data')
pickled_file(data_append, name)
I don't think I would do it this way. I would put the data
list in a class and add a method to it to update the pickle file. Then
call that from any methods that update the data list. def with_file: # no argument list, local group.
f = open(filename)
t = callback(f)
f.close
def my_read(f):
return f.read()
callback = my_read
filename = 'filename'
do with_file
This wouldn't work since with_file wouldn't be re-usable. It also doesn't
get rid of the awkwardness of defining a callback.
As long as the name with_file isn't rebound to something else it could
be used as often as needed. I admit there are better ways to do it
though.
Cheers,
Ron
Brian Sabbey wrote: used, but rarely is because doing so would be awkward. Probably the
simplest real-world example is opening and closing a file. Rarely will
you see code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you
to make this abstraction in a way that is more concise and more readable
than the callback example given above:
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they
allow variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
return f.read()
t = with_file(my_read, 'file.txt')
Definitely put this example into the PEP. I didn't really understand
what you were suggesting until I saw this example. All the other ones
you gave just confused me more. When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
I used 'do' because that's what ruby uses for something similar. It can
be used in a flow control-like way, or as an item-in-a-list way.
Please spend some time in the PEP explaining why you chose the keywords
you chose. They gave me all the wrong intuitions about what was
supposed to be going on, and really confused me. I also got mixed up in
when you were talking about parameters to the thunk, and when you were
talking about parameters to the function that is called with the thunk
as a parameter.
I'd also like to see you start with the full example syntax, e.g.:
do <unpack_list> in <returnval> = <callable>(<params>):
<code>
And then explain what each piece does more carefully. Something like:
"""
When a do-statement is executed, first <callable> is called with the
parameters <params>, augmented by the thunk object, e.g.
do func(4, b=2):
...
would call
func(thunk_obj, 4, b=2)
Next, the body of the function is executed. If the thunk object is
called, then <code> will be executed with the names in <unpack_list>
bound to the objects with which the thunk was called, e.g.
def func(thunk):
thunk(1, y=2)
do x, y, z=4 in func():
print x, y, z
would call:
func(thunk_obj)
thunk(1, y=2)
and thus x, y and z would be bound to 1, 2 and 4 and the body of the
thunk would be executed, printing "1 2 4".
The code in <callable> is then resumed, and the process is repeated
until <callable> returns. Note that this means that each additional
call to the thunk object will cause another execution of <code>, with
potentially different bindings for the names in <unpack_list>.
When the function finally returns, the return value will be bound to
<returnval>, e.g.:
def func(thunk):
thunk()
thunk()
return True
do r = func():
print "thunk called"
print r
would print "thunk called" twice as the body of the thunk is executed
for each call to thunk() in func, and then would print "True" in the
code following the do-statement.
"""
Not sure if I actually understood everything right, but you definitely
need a much more throrough walkthrough of what happens with a thunk --
it's not clear at all from the current pre-PEP.
STeVe
Ron_Adam wrote: def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
Now I can re-use pickled_file whenever I have to modify a pickled file:
do data in pickled_file('pickled.txt'):
data.append('more data')
data.append('even more data')
In my opinion, that is easier and faster to write, more readable, and less
bug-prone than any non-thunk alternative.
The above looks like it's missing something to me. How does 'data'
interact with 'thunk(l)'? What parts are in who's local space?
Your example below explains it well, with 'data' renamed as 'L'. The
scope of bindings are the same in both examples, with the exception that
'data' is in the outermost namespace in the above example, and 'L' is
local to the function 'data_append' in the below example.
This might be the non-thunk version of the above.
yes
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
def data_append(L):
L.append('more data')
L.append('still more data')
pickled_file(data_append, name)
I don't think I would do it this way. I would put the data
list in a class and add a method to it to update the pickle file. Then
call that from any methods that update the data list.
I also wouldn't do it that way. I don't see a class as being much better,
though. If I understand you correctly, with classes you would have
something like:
p = Pickled('pickled.txt')
p.load()
p.data.append('more data')
p.data.append('even more data')
p.dump()
This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.
The motivation for thunks is similar to the motivation for generators--
yes, a class could be used instead, but in many cases it's more work than
should be necessary.
-Brian
On Sun, 17 Apr 2005 15:32:56 -0700, Brian Sabbey <sa****@u.washington.edu> wrote: Bengt Richter wrote: Hm, one thing my syntax does, I just noticed, is allow you
to pass several thunks to a thunk-accepter, if desired, e.g.,
(parenthesizing this time, rather than ending ():<suite> with
dedented comma)
each(
((i): # normal thunk
print i),
((j): # alternative thunk
rejectlist.append(j)),
[1,2])
<snip>
I see that it might be nice to be able to use multiple thunks, and to be
able to specify the positions in the argument list of thunks, but I think
allowing a suite inside parentheses is pretty ugly. One reason is that it
is difficult to see where the suite ends and where the argument list
begins again. I'm not sure even what the syntax would be exactly. I
suppose the suite would always have to be inside its own parentheses?
Also, you wind up with these closing parentheses far away from their
corresponding open parentheses, which is also not pretty. It's getting
too Lisp-like for my tastes.
Having had a past love affair (or at least fling) with scheme,
that doesn't bother me so much ;-)
The "where" specifier syntax might help, e.g.,
each(thk1, thk2, [1, 2]) where:
thk1 = (i): # normal thunk
print i
thk2 = (j): # alternative thunk
rejectlist.append(j)
This still uses my (<arglist>):<suite> expression for thunks
but they don't need to be parenthesized, because their suites
terminate with normal dedenting under the where: suite
I.e., the 'p' of 'print i' is the left edge of the (i): suite
and thus the 't' of 'thk2 = ...' ends the (i): suite. The (j): suite
left edge is at the 'r' of 'rejectlist'... so anything to the left
of that (excluding comments) will end the (j): suite, like normal
indentation.
I could have used dedent termination in the previous example too
by moving the commas around to let them trigger dedent level out of
the preceding suite, e.g., with args evaluated in place again:
each((i): # normal thunk
print i
,(j): # alternative thunk
rejectlist.append(j)
,[1,2])
Of course, if you want lispy, the above simple thunks can be done in a oneliner:
each(((i):print i), ((j):rejectlist.append(j)), [1,2])
I personally like the power of being able to write that, but given a clean sugar
alternative, I would use it. But if it's an exclusive-or choice, I'll take primitives
over sugar, because sugar never covers all the useful combinations of primitives that
will turn up later.
Regards,
Bengt Richter
Brian Sabbey wrote: do f in with_file('file.txt'):
print f.read()
def with_file(filename):
f = open(filename)
yield f
f.close()
for f in with_file('file.txt'):
print f.read()
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
t = "no file read yet"
for f in with_file('file.txt'):
t = f.read()
--
Serhiy Storchaka
On Sun, 17 Apr 2005 19:56:10 -0700, Brian Sabbey
<sa****@u.washington.edu> wrote: I also wouldn't do it that way. I don't see a class as being much better,
though. If I understand you correctly, with classes you would have
something like:
p = Pickled('pickled.txt')
p.load()
p.data.append('more data')
p.data.append('even more data')
p.dump()
The load and dump would be private to the data class object. Here's a
more complete example.
import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...
pdata = PickledData('filename')
pdata.append('more data')
pdata.append('even more data')
print pdata.L
['more data', 'even more data']
This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.
A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.
Cheers,
Ron
The motivation for thunks is similar to the motivation for generators--
yes, a class could be used instead, but in many cases it's more work than
should be necessary.
-Brian
Ron_Adam wrote: The load and dump would be private to the data class object. Here's a
more complete example.
import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...
pdata = PickledData('filename')
pdata.append('more data')
pdata.append('even more data')
print pdata.L
['more data', 'even more data']
This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.
A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.
This class isn't reusable in the case that one wants to pickle something
other than an array. Every type of object that one would wish to pickle
would require its own class.
Also, this implementation behaves differently because the object is
re-pickled after every modification. This could be a problem when writing
over a network, or to a shared resource.
-Brian
On Mon, 18 Apr 2005 21:11:52 -0700, Brian Sabbey
<sa****@u.washington.edu> wrote: Ron_Adam wrote: The load and dump would be private to the data class object. Here's a
more complete example.
import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...
pdata = PickledData('filename')
pdata.append('more data')
pdata.append('even more data')
print pdata.L
['more data', 'even more data']
This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.
A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.
This class isn't reusable in the case that one wants to pickle something
other than an array. Every type of object that one would wish to pickle
would require its own class.
....Or in a function, or the 3 to 6 lines of pickle code someplace.
Many programs would load data when they start, and then save it when
the user requests it to be saved. So there is no one method fits all
situations. Your thunk example does handle some things better.
Here's yet another way to do it, but it has some limitations as well.
import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()
file = 'filename'
L = []
opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)
Also, this implementation behaves differently because the object is
re-pickled after every modification. This could be a problem when writing
over a network, or to a shared resource.
-Brian
In some cases writing to the file after ever modification would be
desired. A way around that would be to use a buffer of some sort. But
then again, that adds another level of complexity and you would have
to insure it's flushed at some point which get's back to the issue of
not closing a file.
Thanks for explaining how thunks works. I'm still undecided on
whether it should be built in feature or not.
I would rather have a way to store a block of code and pass it to a
function, then execute it at the desired time. That would solve both
the issue where you would use a thunk, and replace lambdas as well.
But I understand there's a lot of resistance to that because of the
potential abuse.
Cheers,
Ron
Brian Sabbey wrote: do f in with_file('file.txt'):
print f.read()
I don't like this syntax. Try to read it as an English sentence:
"Do f in with file 'file.txt'". Say what???
To sound right it would have to be something like
with_file('file.txt') as f do:
print f.read()
But, while that works with this particular function
name, and others of the form "with_xxx", there are
bound to be other use cases which would require
different words or word orders in order not to sound
contrived.
It's very difficult to come up with a good syntax for
this that isn't skewed towards one kind of use case.
That's probably a large part of the reason why nothing
like it has so far been seriously considered for
adoption.
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg
Greg Ewing wrote: Brian Sabbey wrote:
do f in with_file('file.txt'):
print f.read()
I don't like this syntax. Try to read it as an English sentence:
"Do f in with file 'file.txt'". Say what???
To sound right it would have to be something like
with_file('file.txt') as f do:
print f.read()
This is still strange since f is the arguments the thunk was called
with, e.g. the current syntax is basically:
do <unpack_list> in <returnval> = <callable>(<params>):
<code>
I don't really know a more readable sequence of keywords, though someone
suggested 'with' and 'from', which might read something like:
with <unpack_list> from <callable>(<params>):
<code>
which looks okay to me, though I'm not sure that 'with' makes it clear
that this is not a normal block... I also find readability problems
when I try to stick <returnval> back in.
One of the other issues is that, with the current proposal, the thunk
can be called multiple times within a function, so the keywords have to
make sense both with a single iteration interpretation and a multiple
iteration interpretation... Makes it even harder...
STeVe
Ron_Adam <radam2_@_tampabay.rr.com> writes: Here's yet another way to do it, but it has some limitations as well.
import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()
file = 'filename'
L = []
opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)
Um - it doesn't look like this will work. You pass L in as the "obj"
paremeter, and then it gets changed to the results of a
pickle.load. However, you call L.append in the for loop, so the data
will be appended to L, not obj. You'll lose an list elements that were
in the pickle.
<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Mike Meyer wrote: Ron_Adam <radam2_@_tampabay.rr.com> writes:
Here's yet another way to do it, but it has some limitations as well.
import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()
file = 'filename'
L = []
opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)
Um - it doesn't look like this will work. You pass L in as the "obj"
paremeter, and then it gets changed to the results of a
pickle.load. However, you call L.append in the for loop, so the data
will be appended to L, not obj. You'll lose an list elements that were
in the pickle.
<mike
Hmmm, you are correct. :-/
I was trying to not use eval(). This could be made to work, but it will
get messy, which is another reason not to do it.
Cheers,
Ron This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Headless |
last post by:
I've marked up song lyrics with the <pre> tag because it seems the most
appropriate type of markup for the type of data. This results in
inefficient use of horizontal space due to UA's default...
|
by: Neal |
last post by:
A few questions about pre...
When presenting preformatted text using the white-space: pre;
property/value, Opera renders long lines at small viewport widths as
exiting the borders of the...
|
by: Alan Illeman |
last post by:
How do I set several different properties for PRE in
a CSS stylesheet, rather than resorting to this:
<BODY>
<PRE STYLE="font-family:monospace;
font-size:0.95em;
width:40%;
border:red 2px...
|
by: Buck Turgidson |
last post by:
I want to have a css with 2 PRE styles, one bold with large font, and
another non-bold and smaller font.
I am new to CSS (and not exactly an expert in HTML, for that matter). Is
there a way to...
|
by: Michael Shell |
last post by:
Greetings,
Consider the XHTML document attached at the end of this post.
When viewed under Firefox 1.0.5 on Linux, highlighting
and pasting (into a text editor) the <pre> tag listing will...
|
by: Jarno Suni not |
last post by:
It seems to be invalid in HTML 4.01, but valid in XHTML 1.0. Why is there the difference? Can that pose a problem when such a XHTML document is served as text/html?
|
by: Eric Lindsay |
last post by:
I can't figure how to best display little snippets of shell script using
<pre>. I just got around to organising to bulk validate some of my web
pages, and one of the problems occurs with Bash...
|
by: Xah Lee |
last post by:
The Concepts and Confusions of Pre-fix, In-fix, Post-fix and Fully
Functional Notations
Xah Lee, 2006-03-15
Let me summarize: The LISP notation, is a functional notation, and is
not a...
|
by: Schraalhans Keukenmeester |
last post by:
I am building a default sheet for my linux-related pages.
Since many linux users still rely on/prefer viewing textmode and
unstyled content I try to stick to the correct html tags to pertain good...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
| |
