Python 2.4: Why only assignments to None are forbiden?

[Josef Meile]

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and
variables should be also forbiden. It's a common mistake to do things
like this:

def getFileName(fil e):

.... parts=file.spli t('/')
.... return parts('/')[-1]

Specially if you come from python 2.1.x where "file" didn't exist.
Instead, there was "open"

On the example's context, file is a string and won't cause any damage
because it is inside a function, so, the scope is local and it will be
deleted after the function call. But think what would happen if somebody
defines "file" as a global variable and other people use that code? For
the author won't be any consequences at all, because if s/he does this,
it means that probably s/he isn't working with files, but if somebody
else takes that code and doesn't see this, then s/he will invest some
time trying to find the bug.

Doing this validation for python 2.4.x will break some things, like the
validation with "None" does. But I think it's better when you now that
you are trying to use a "reserved word", you could as well use the "str"
as an example, which I think is also common on newies.

Regards,
Josef

[James Stroud]

Forbidding assignment to arbitrary things seems like a bad idea.

I remember some learning manual recommended this (or the equivalent):

def do_stuff_to_bob (bob=None):
if not bob:
bob = ["carol","ted"," alice"]
#do stuff to bob
return bob

otherwise bob would already be defined from the last call (because it is a
mutable type? I can't remember.). How else to handle this in 2.4?

On Friday 12 November 2004 01:11 pm, Josef Meile wrote:

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and
variables should be also forbiden.

James

[David M. Cooke]

Josef Meile <jm****@hotmail .com> writes:

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and
variables should be also forbiden. It's a common mistake to do things
like this:

>>> def getFileName(fil e):

... parts=file.spli t('/')
... return parts('/')[-1]

Specially if you come from python 2.1.x where "file" didn't exist.
Instead, there was "open"

On the example's context, file is a string and won't cause any damage
because it is inside a function, so, the scope is local and it will be
deleted after the function call. But think what would happen if
somebody defines "file" as a global variable and other people use that
code?
That may not be great style, but it's not a mistake: file is used as a
local variable, so the global binding doesn't enter into it. And with
this short of function, I wouldn't worry too much.

Longer functions, yes, you might get bit, especially with other
builtins, like str and dict.
Doing this validation for python 2.4.x will break some things, like
the validation with "None" does. But I think it's better when you now
that you are trying to use a "reserved word", you could as well use
the "str" as an example, which I think is also common on newies.

I believe the idea behind making None non-assignable is one of
optimization: it's used a lot, and each time, the runtime has to do a
lookup to find the current value.

But making other builtins unassignable, while nice, removes some of
the dynamism that makes Python nice.

Recently, I needed to check what files my program was opening (and not
closing; I was running out of file descriptors). The easy way to do
that was to override the builtin open() function, returning a proxy
object that would annouce it's been opened, and when it's been closed.
This isn't quite the same thing: I had to assign to __builtin__.ope n
so that all modules would get my new open. However, if open were
declared unassignable, I'll bet you that it would be bound at
*compile* time instead of runtime, and this trick wouldn't work anyways.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)phy sics(dot)mcmast er(dot)ca

[Ian Bicking]

Josef Meile wrote:

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and
variables should be also forbiden. It's a common mistake to do things
like this:

>>> def getFileName(fil e):

... parts=file.spli t('/')
... return parts('/')[-1]

Specially if you come from python 2.1.x where "file" didn't exist.
Instead, there was "open"

Well, it's not really a mistake, because it works just fine and causes
no problems. I use builtin names often, as they are annoyingly very
convenient local variables. I try not to, but it still happens
(especially file, though I make sure to distinguish between files and
filenames).

Anyway, it's a big backward compatibility issue. No one ever reassigns
None, so it's not a problem. Maybe True and False will go this way too
(though that *will* cause backward compatible problems with code that
preceded the True/False builtins, and so created its own True/False
objects).
On the example's context, file is a string and won't cause any damage
because it is inside a function, so, the scope is local and it will be
deleted after the function call. But think what would happen if somebody
defines "file" as a global variable and other people use that code? For
the author won't be any consequences at all, because if s/he does this,
it means that probably s/he isn't working with files, but if somebody
else takes that code and doesn't see this, then s/he will invest some
time trying to find the bug.

There are proposed optimizations that depend on there being no
global-level redefinition of builtins, and there's a couple PEPs about
this. I think the ideas have been viewed favorably, but the
implementations haven't been fully fleshed out, and kind of fell by the
wayside. They could still be picked up. There were a few people who
were concerned; I think boost particularly defines some public symbols
like boost.int, which this would disallow.

Anyway, the much more conservative proposal that disallows global
variables that share a name with a builtins seems reasonable and largely
backward compatible.

--
Ian Bicking / ia**@colorstudy .com / http://blog.ianbicking.org

[Bengt Richter]

On Fri, 12 Nov 2004 13:08:30 -0800, James Stroud <js*****@mbi.uc la.edu> wrote:

Forbidding assignment to arbitrary things seems like a bad idea.

I remember some learning manual recommended this (or the equivalent):
Two things wrong in the following, as noted:def do_stuff_to_bob (bob=None): #XXX# > if not bob:
if bob is None: # this is the usual gerneral idiom, so that e.g., one could
# distinguish from a caller passing a legitimate empty list []
# (though of course you could use if not bob if that's what you meant ;-) bob = ["carol","ted"," alice"] # ^^^^^ -- this does not assign to None, it rebinds bob,
# which has no effect on the previously bound object (in this case None)
# That's a serious misunderstandin g of python's assignment semantics, which
# your momentary lapse should not be allowed to promote to newbies ;-)
#do stuff to bob
return bob

otherwise bob would already be defined from the last call (because it is a
mutable type? I can't remember.). How else to handle this in 2.4?

On Friday 12 November 2004 01:11 pm, Josef Meile wrote:

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and
variables should be also forbiden.

James

Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32
Type "help", "copyright" , "credits" or "license" for more information.

None = 'foo' SyntaxError: assignment to None 1234 = 'foo' SyntaxError: can't assign to literal bob = None
bob
repr(bob) 'None' bob = 1234
bob

1234

Regards,
Bengt Richter

[Peter Hansen]

Josef Meile wrote:

I think in general assignments to built-in types, functions, and
variables should be also forbiden.

Very little in Python should be *forbidden*. Certainly not
something like this! Maybe make it hard to do inadvertently.
Maybe require it be an explicit call to a special function
or something. But forbid it? Please no!

One perfectly good reason to assign to builtins (and there are others,
I'm sure) is to provide a "mock" function to replace a standard
one, for use during automated testing. See, for example, the
reference to a "mock filesystem" that I posted a few days ago,
where you can insert your own "open" function in the builtin
namespace, providing a fake filesystem that intercepts all
file creation and reading and simulates the behaviour.

You can do this easily on a single module, or several, just
by inserting a global into the module(s) in question, but if
you want to do it through an application it becomes a real
burden (and a way to let bugs leak through) to have to do it
that way. A single assignment override to the standard open
is much more effective and safe.

Every time this issue comes up I feel compelled to point out
this perfectly useful and valid use of assigning to builtins...
I really hope the message gets through.

-Peter

[Dan Bishop]

Ian Bicking <ia**@colorstud y.com> wrote in message news:<ma******* *************** *************** *@python.org>.. .

Josef Meile wrote:

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and
variables should be also forbiden. It's a common mistake to do things
like this:

>>> def getFileName(fil e):

... parts=file.spli t('/')
... return parts('/')[-1]

Specially if you come from python 2.1.x where "file" didn't exist.
Instead, there was "open"

Well, it's not really a mistake, because it works just fine and causes
no problems. I use builtin names often, as they are annoyingly very
convenient local variables. I try not to, but it still happens
(especially file, though I make sure to distinguish between files and
filenames).

I have the same problem.

Perhaps it would have better if "file" were named "File" instead.

[John Roth]

"Josef Meile" <jm****@hotmail .com> wrote in message
news:41******@p faff2.ethz.ch.. .

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and variables
should be also forbiden.

As Ian points out in another response, there's been some
discussion about making a fairly large subset of the builtins
name space immutable, and also to disallow shadowing
of builtins with identical names in module scope.

The main reason for this is performance optimization.
Right now, the len() function, for example, requires
a lookup and two (!) function evaluations. If the
compiler could trust that len() was always the function
in the builtins, then it could compile a direct call to the
__len__() function on the object, at a savings of the
lookup and one function call. There is a considerable savings
projected over the entire name builtin name space.

There's always a tension between protecting people
against common errors and keeping the language
as flexible and dynamic as possible. I'd prefer that
the editors and IDEs take up the slack. For example,
I found that JEdit colorized builtins differently from
ordinary names. This pointed out a few bad habits
I'd fallen into, without taking away the option of
overriding something when I really wanted it.

In any case, it won't happen in 2.4.x. That kind
of change will wait for a full point release: 2.5
or later.

John Roth

[Andrew Clover]

Ian Bicking <ia**@colorstud y.com> wrote:

Anyway, the much more conservative proposal that disallows global
variables that share a name with a builtins seems reasonable and largely
backward compatible.

This still sounds undesirable to me.

- it requires new authors to know and avoid all the builtins, even
those they have no intention of using;

- afterwards, adding any new builtins becomes a potentially
app-breaking proposition.

Protecting None, False and True seems fine to me; they're extremely
common and part of the syntax itself in most languages. But banning a
global variable called 'license', for example, or a class 'Warning',
would be going too far IMO.

--
Andrew Clover
mailto:an*@doxd esk.com
http://www.doxdesk.com/

[Bengt Richter]

On Sat, 13 Nov 2004 08:04:28 -0600, "John Roth" <ne********@jhr othjr.com> wrote:

"Josef Meile" <jm****@hotmail .com> wrote in message
news:41******@ pfaff2.ethz.ch. ..

Hi,

Textually from the highlights of python 2.4:

"Assigning to None - the compiler now treats assigning to None as a
SyntaxError."

I think in general assignments to built-in types, functions, and variables
should be also forbiden.

As Ian points out in another response, there's been some
discussion about making a fairly large subset of the builtins
name space immutable, and also to disallow shadowing
of builtins with identical names in module scope.

The main reason for this is performance optimization.
Right now, the len() function, for example, requires
a lookup and two (!) function evaluations. If the
compiler could trust that len() was always the function
in the builtins, then it could compile a direct call to the
__len__() function on the object, at a savings of the
lookup and one function call. There is a considerable savings
projected over the entire name builtin name space.

There's always a tension between protecting people
against common errors and keeping the language
as flexible and dynamic as possible. I'd prefer that
the editors and IDEs take up the slack. For example,
I found that JEdit colorized builtins differently from
ordinary names. This pointed out a few bad habits
I'd fallen into, without taking away the option of
overriding something when I really wanted it.

In any case, it won't happen in 2.4.x. That kind
of change will wait for a full point release: 2.5
or later.

John Roth

I don't mind things being locked or unlocked by sensible
default if I have a key. I.e., I'd like some way of telling
the compiler to generate current-style lookup code for a particular
builtin that might be optimized in the future.

Since this is a compile-time issue , perhaps it is time to consider
a pragma statement as a framework for telling python pre-runtime things
like optimization info? E.g., pragma could send in-context meta-info
to various python infrastructure components using python method call syntax
which the compiler could recognize in the token stream and compile and
execute in its working environment right then. So e.g.,

pragma compiler.overri de(len=True, cmp=False)

would say "after this statement in the token stream, generate code for
old-style lookup of len (allowing overriding) and cease to do that for cmp."

Right now we can do this:

def calledat(level= 2): # default = caller's caller ... f = sys._getframe()
... for _ in xrange(level): f = f and f.f_back
... return f and '%s line %s'%(f.f_code.c o_name, f.f_lineno) or '???'
... def showlencall(seq ): ... print '---> len(%r) from %s'%(seq, calledat(2))
... __builtins__.le n <built-in function len> def showlencall(seq ): ... print '---> len(%r) from %s'%(seq, calledat(2))
... return __builtins__.le n(seq)
...

Here (or at the top of this scope) we would in the future need:

pragma compiler.overri de(len=True) # be able to specify scope other than local??
len = showlencall
def foo(x): ... return len(x)
... foo(range(4))

---> len([0, 1, 2, 3]) from foo line 2
4

Not that this shows implicit calls to __len__ in various places...
Hm, that might make an interesting pragma

pragma compiler.method hook(__len__= my_len_tracer)

I better stop here ;-)

Regards,
Bengt Richter