Deprecating reload() ???

[Ellinghaus, Lance]

> > Other surprises: Deprecating reload()

Reload doesn't work the way most people think
it does: if you've got any references to the old module,
they stay around. They aren't replaced. It was a good idea, but the implementation simply
doesn't do what the idea promises.

I agree that it does not really work as most people think it does, but how
would you perform the same task as reload() without the reload()?

Would this be done by "del sys.modules['modulename']" and then perform an
'import'?

I use reload() for many purposes, knowing how it works/does not work.

Lance Ellinghaus

[John Roth]

"Ellinghaus , Lance" <la************ **@eds.com> wrote in message
news:ma******** *************** *************** @python.org...

Other surprises: Deprecating reload()

Reload doesn't work the way most people think
it does: if you've got any references to the old module,
they stay around. They aren't replaced.

It was a good idea, but the implementation simply
doesn't do what the idea promises.

I agree that it does not really work as most people think it does, but how
would you perform the same task as reload() without the reload()?

Would this be done by "del sys.modules['modulename']" and then perform an
'import'?

I use reload() for many purposes, knowing how it works/does not work.

I usually figure out another way to do it - mostly reload the entire
program from scratch. In any case, it's not a real issue for me -
it's just enough of an irritation that the PythonWin editor does it that
I keep a command line open rather than use what is (to me) a
really broken implementation of run.

There are a lot of things in life that I wished worked the way they
are advertised. They don't, so you cope.

John Roth
Lance Ellinghaus

[David MacQuigg]

On Thu, 11 Mar 2004 15:10:59 -0500, "Ellinghaus , Lance"
<la************ **@eds.com> wrote:

> Other surprises: Deprecating reload()

Reload doesn't work the way most people think
it does: if you've got any references to the old module,
they stay around. They aren't replaced.

It was a good idea, but the implementation simply
doesn't do what the idea promises.

I agree that it does not really work as most people think it does, but how
would you perform the same task as reload() without the reload()?

Seems like reload() could be *made* to work by scanning currently
loaded modules for all references to objects in the reloaded module,
and resetting them to the new objects. If an object is missing from
the new module, throw an exception.

GvR suggests using 'exec' as an alternative.
http://python.org/doc/essays/ppt/regrets/6
I don't see how that solves the problem of eliminating references to
old objects.

-- Dave

[user]

Ellinghaus, Lance wrote:

I agree that it does not really work as most people think it does, but how
would you perform the same task as reload() without the reload()?

Would this be done by "del sys.modules['modulename']" and then perform an
'import'?

Ah, interesting! I've been doing this:

del modulename; import modulename

but it doesn't pick up any recent changes to the
modulename.py file.

That's a better solution than exiting Python and
starting it up again. Thanks!

--
Steven D'Aprano

[Peter Hansen]

us**@domain.inv alid wrote:

Ellinghaus, Lance wrote:

I agree that it does not really work as most people think it does, but
how
would you perform the same task as reload() without the reload()?
Would this be done by "del sys.modules['modulename']" and then perform an
'import'?

Ah, interesting! I've been doing this:

del modulename; import modulename

but it doesn't pick up any recent changes to the modulename.py file.

That's a better solution than exiting Python and starting it up again.

Just to clarify, *neither* of the above solutions does anything useful,
as far as I know. Only reload() will really reload a module. Certainly
the del mod/import mod approach is practically a no-op...

-Peter

[David MacQuigg]

On Fri, 12 Mar 2004 16:27:36 +1100, us**@domain.inv alid wrote:

Ellinghaus, Lance wrote:

I agree that it does not really work as most people think it does, but how
would you perform the same task as reload() without the reload()?

Would this be done by "del sys.modules['modulename']" and then perform an
'import'?

Ah, interesting! I've been doing this:

del modulename; import modulename

but it doesn't pick up any recent changes to the
modulename.p y file.

That's a better solution than exiting Python and
starting it up again. Thanks!

This doesn't work. It doesn't matter if you first "del modulename".
All that does is remove the reference to "modulename " from the current
namespace.

testimport.py
a = 8
print 'a =', a

import testimport a = 8 ## ==> change a to 9 in testimport.py
testimport.a 8 dir() ['__builtins__', '__doc__', '__name__', 'testimport'] del testimport
dir() ['__builtins__', '__doc__', '__name__'] import testimport
dir() ['__builtins__', '__doc__', '__name__', 'testimport'] testimport.a 8 <== The old module is still in memory !!! a = testimport.a
dir() ['__builtins__', '__doc__', '__name__', 'a', 'testimport'] reload(testimpo rt) a = 9 testimport.a 9 <== reload() changes new fully-qualified references a 8 <== but not any other references that were set up before.

I know this is the *intended* behavior of reload(), but it has always
seemed to me like a bug. Why would you *ever* want to keep pieces of
an old module when that module is reloaded?

Seems to me we should *fix* reload, not deprecate it.

-- Dave

[Peter Hansen]

David MacQuigg wrote:

I know this is the *intended* behavior of reload(), but it has always
seemed to me like a bug. Why would you *ever* want to keep pieces of
an old module when that module is reloaded?
If you wanted a dynamic module, where changes would be picked up by the
application from time to time, but didn't want to terminate existing
instances of the old version. There's nothing wrong with the idea that
both old and new versions of something could exist in memory, at least
for a while until the old ones are finished whatever they are doing.

Basically, envision an application update mechanism for long-running
applications, and its special needs.
Seems to me we should *fix* reload, not deprecate it.

Reload is not broken, and certainly shouldn't be deprecated at least
until there's a better solution that won't suffer from reload's one
problem, IMHO, which is that it surprises some people by its behaviour.
I think that when you consider Python's namespace mechanism, you can't
avoid the possibility of situations like the ones reload can now lead to.

-Peter

[Michael Hudson]

"Ellinghaus , Lance" <la************ **@eds.com> writes:

Other surprises: Deprecating reload()

Reload doesn't work the way most people think
it does: if you've got any references to the old module,
they stay around. They aren't replaced.

It was a good idea, but the implementation simply
doesn't do what the idea promises.

I missed these mails...
I agree that it does not really work as most people think it does, but how
would you perform the same task as reload() without the reload()?

Would this be done by "del sys.modules['modulename']" and then perform an
'import'?

I use reload() for many purposes, knowing how it works/does not work.

To paraphrase Tim Peters, you can have reload() when I'm dead. Just
because it doesn't and never has done what some people expect is no
argument at all for deprecating it.

Cheers,
mwh

--
But maybe I've just programmed in enough different languages to
assume that they are, in fact, different.
-- Tony J Ibbs explains why Python isn't Java on comp.lang.pytho n

[Skip Montanaro]

Regarding

del sys.modules['modulename']; import modulename
vs.

del modulename ; import modulename

Peter sez:

Peter> Just to clarify, *neither* of the above solutions does anything
Peter> useful, as far as I know. Only reload() will really reload a
Peter> module. Certainly the del mod/import mod approach is practically
Peter> a no-op...

Not so. del sys.modules['mod']/import mod is effectively what reload()
does. Given foo.py with this content:

a = 1
print a

here's a session which shows that it is outwardly the same as reload(...).
That is, the module-level code gets reexecuted (precisely because there's
not already a reference in sys.modules).

import foo 1 reload(foo) 1
<module 'foo' from 'foo.pyc'> import foo
import sys
del sys.modules['foo']
import foo

1

David MacQuigg's contention that it doesn't work is missing the fact that
when he executed

a = testimport.a

he simply created another reference to the original object which was outside
the scope of the names in testimport. Reloading testimport created a new
object and bound "testimport .a" to it. That has nothing to do with the
object to which "a" is bound. This is the problem reload() doesn't solve.

Given Python's current object model it would be an interesting challenge to
write a "super reload" which could identify all the objects created as a
side effect of importing a module and for those with multiple references,
locate those other references by traversing the known object spaces, then
perform the import and finally rebind the references found in the first
step. I thought I saw something like this posted in a previous thread on
this subject.

In case you're interested in messing around with this, there are three
object spaces to be careful of, builtins (which probably won't hold any
references), all the imported module globals (which you can find in
sys.modules) and all the currently active local Python functions (this is
where it gets interesting ;-). There's a fourth set of references
- those held by currently active functions which were defined in
extension modules - but I don't think you can get to them from pure Python
code. Whether or not that's a serious enough problem to derail the quest
for super_reload() function is another matter. Most of the time it probably
won't matter. Every once in a great while it will probably bite you in the
ass. Accordingly, if super_reload() can't track down all the references
which need rebinding it should probably raise a warning/exception or return
a special value to indicate that.

Skip

[David MacQuigg]

On Fri, 12 Mar 2004 08:45:24 -0500, Peter Hansen <pe***@engcorp. com>
wrote:

David MacQuigg wrote:

I know this is the *intended* behavior of reload(), but it has always
seemed to me like a bug. Why would you *ever* want to keep pieces of
an old module when that module is reloaded?
If you wanted a dynamic module, where changes would be picked up by the
application from time to time, but didn't want to terminate existing
instances of the old version. There's nothing wrong with the idea that
both old and new versions of something could exist in memory, at least
for a while until the old ones are finished whatever they are doing.

OK, I agree this is a valid mode of operation.
Basically, envision an application update mechanism for long-running
applications , and its special needs.
I think a good example might be some objects that need to retain the
state they had before the reload.

Seems to me we should *fix* reload, not deprecate it.

Reload is not broken, and certainly shouldn't be deprecated at least
until there's a better solution that won't suffer from reload's one
problem, IMHO, which is that it surprises some people by its behaviour.

It's worse than just a surprise. It's a serious problem when what you
need to do is what most people are expecting -- replace every
reference to objects in the old module with references to the new
objects. The problem becomes a near impossibility when those
references are scattered throughout a multi-module program.

For the *exceptional* case where we want to pick and chose which
objects to update, seems like the best solution would be to add some
options to the reload function.

def reload(<module> , objects = '*'):

The default second argument '*' updates all references. '?' prompts
for each object. A list of objects updates references to just those
objects automatically. 'None' would replicate the current behavior,
updating only the name of the module itself.

The '?' mode would normally ask one question for each object in the
module to be reloaded, and then update all references to the selected
objects. We could even have a '??' mode that would prompt for each
*reference* to each object.

In a typical debug session, there is only one object that you have
updated, so I think there would seldom be a need to enter more than
one name as the second argument.
I think that when you consider Python's namespace mechanism, you can't
avoid the possibility of situations like the ones reload can now lead to.

I don't understand. My assumption is you would normally update all
references to the selected objects in all namespaces.

-- Dave