unexpected from/import statement behaviour

[nisp]

Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

.....

print >stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

.....

print >sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

Thanks in advance !

[Diez B. Roggisch]

nisp schrieb:

Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

....

print >stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

....

print >sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

http://effbot.org/zone/import-confusion.htm

Diez

[alex23]

nisp <[e-mail]:

I've always been convinced of the equivalence of the two ways of using
the import statement but it's clear I'm wrong :-(

Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>from sys import stderr
import sys
sys.stderr is stderr

True

Behaviourly, I'm finding no difference between the two either.

Could you cut & paste a minimal example that isn't producing the
correct behaviour, and perhaps mention what type of OS you're using?

[Diez B. Roggisch]

alex23 schrieb:

nisp <[e-mail]:

>I've always been convinced of the equivalence of the two ways of using
the import statement but it's clear I'm wrong :-(

Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>>from sys import stderr
import sys
sys.stderr is stderr

True

Behaviourly, I'm finding no difference between the two either.

Could you cut & paste a minimal example that isn't producing the
correct behaviour, and perhaps mention what type of OS you're using?

You did not read this part of the post:

"""
I'm not able to capture its stderr and of course I would like not to
do this kind of change.
"""

He tries to set sys.stderr to a new stream and capture the print
statements, but fails to do so because he created a local alias.
Diez

[Peter Otten]

nisp wrote:

Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

....

print >stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

....

print >sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

Thanks in advance !

A practical approach to complement Diez' link to the explanation:

Instead of modifying the external module you can either redirect stderr
before you import the external module

import sys
sys.stderr = whatever
import external

or monkey-patch:

import sys
import external

sys.stderr = external.sterr = whatever

Peter

[nisp]

On Aug 27, 9:56 am, Peter Otten <[e-mail]:

nisp wrote:

Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

....

print >stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

....

print >sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

Thanks in advance !

A practical approach to complement Diez' link to the explanation:

Instead of modifying the external module you can either redirect stderr
before you import the external module

import sys
sys.stderr = whatever
import external

or monkey-patch:

import sys
import external

sys.stderr = external.sterr = whatever

Peter

Hi all !

Thanks first of all ! I read the interesting Diez's link but something
still remains to me unclear, on the other hand it's clear the my
problem is concentrated there and on symbols.
Here is what I'm trying to do

HelloWorld.py: this is a real simplification of my external module
though still reflecting its structure (commented out is the version
that, let's say, works)

from sys import stderr
#import sys

class Cheers:
def __init__(self):
self.cheersMsg = 'Hello World !!'
print "Cheers stderr %s" % stderr
#print "Cheers stderr %s" % sys.stderr
def _putCheers(self):
print>>stderr, 'Here is my msg:', self.cheersMsg
print>>stderr, 'This is a nice day today !!'
#print>>sys.stderr, 'Here is my msg:', self.cheersMsg
#print>>sys.stderr, 'This is a nice day today !!'
def doSomeStuff(self):
self._putCheers()

And below there is the module that uses the above one (mymodule.py):

#!/usr/bin/python

import sys
from HelloWorld import *

class StderrCatcher:
def __init__(self):
self.data = ''
def write(self,stuff):
self.data = self.data + "\t" + stuff
def main():

print "mymodule stderr: %s" % sys.stderr

sys.stderr = stderr = StderrCatcher()
m = Cheers()
m.doSomeStuff()
print "stderr: \n%s" % sys.stderr.data

if __name__ == '__main__':
main()
Below there is the output when it doesn't work:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7d160b0>
Cheers stderr <open file '<stderr>', mode 'w' at 0xb7d160b0>
Here is my msg: Hello World !!
This is a nice day today !!
stderr:

And here when it works:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7dc40b0>
Cheers stderr <__main__.StderrCatcher instance at 0xb7d8bd4c>
stderr:
Here is my msg: Hello World !!
This is a nice day today !!
Thanks again!

PS Sorry for having probably replied to somone of you directly :-(

[Diez B. Roggisch]

nisp wrote:

On Aug 27, 9:56 am, Peter Otten <[e-mail]:

>nisp wrote:

Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

....

print >stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

....

print >sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

Thanks in advance !

A practical approach to complement Diez' link to the explanation:

Instead of modifying the external module you can either redirect stderr
before you import the external module

import sys
sys.stderr = whatever
import external

or monkey-patch:

import sys
import external

sys.stderr = external.sterr = whatever

Peter

Hi all !

Thanks first of all ! I read the interesting Diez's link but something
still remains to me unclear, on the other hand it's clear the my
problem is concentrated there and on symbols.
Here is what I'm trying to do

HelloWorld.py: this is a real simplification of my external module
though still reflecting its structure (commented out is the version
that, let's say, works)

from sys import stderr

This is your problem. You create a HelloWorld.stderr-alias to the object
bound to sys.stderr. Rebinding the latter won't affect the former. That is
precisely what the link I gave you explains.

The short answer to the whole issue is: dont' use the from-import syntax
until you really know what you are doing. Or not at all.

Diez

[nisp]

On Aug 27, 2:43 pm, "Diez B. Roggisch" <[e-mail]:

nisp wrote:

On Aug 27, 9:56 am, Peter Otten <[e-mail]:

nisp wrote:
Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

....

print >stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

....

print >sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

Thanks in advance !

A practical approach to complement Diez' link to the explanation:

Instead of modifying the external module you can either redirect stderr
before you import the external module

import sys
sys.stderr = whatever
import external

or monkey-patch:

import sys
import external

sys.stderr = external.sterr = whatever

Peter

Hi all !

Thanks first of all ! I read the interesting Diez's link but something
still remains to me unclear, on the other hand it's clear the my
problem is concentrated there and on symbols.
Here is what I'm trying to do

HelloWorld.py: this is a real simplification of my external module
though still reflecting its structure (commented out is the version
that, let's say, works)

from sys import stderr

This is your problem. You create a HelloWorld.stderr-alias to the object
bound to sys.stderr. Rebinding the latter won't affect the former. That is
precisely what the link I gave you explains.

The short answer to the whole issue is: dont' use the from-import syntax
until you really know what you are doing. Or not at all.

Diez

Hi Diez! I well understand the problem now, the fact is that
unfortunately the external module is not mine and i doubt i can change
it :-( Anyway as far as i'm concerned i've learned the lesson ! Thanks
again and greetings,
Nisp

[Peter Otten]

nisp wrote:

Thanks first of all ! I read the interesting Diez's link but something
still remains to me unclear, on the other hand it's clear the my
problem is concentrated there and on symbols.

Read it again. If you have two modules

module1.py
from sys import stderr

module2.py
from module1 import stderr

you get three names 'stderr' in three different namespaces, and each name
may bind a different object.

Here is what I'm trying to do

HelloWorld.py: this is a real simplification of my external module
though still reflecting its structure (commented out is the version
that, let's say, works)

from sys import stderr
#import sys

class Cheers:
Â*Â*Â*Â*Â*Â*Â*Â*def __init__(self):
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self.cheersMsg = 'Hello World !!'
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*print "Cheers stderr %s" % stderr
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*#print "Cheers stderr %s" % sys.stderr
Â*Â*Â*Â*Â*Â*Â*Â*def _putCheers(self):
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*print>>stderr, 'Here is my msg:', self.cheersMsg
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*print>>stderr, 'This is a nice day today !!'
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*#print>>sys.stderr , 'Here is my msg:', self.cheersMsg
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*#print>>sys.stderr , 'This is a nice day today !!'
Â*Â*Â*Â*Â*Â*Â*Â*def doSomeStuff(self):
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self._putCheers()

And below there is the module that uses the above one (mymodule.py):

#!/usr/bin/python

import sys

This imports HelloWorld.stderr:

from HelloWorld import *

You now have a global variable 'stderr' in module __main__, initialized to
the same value as HelloWorld.stderr.

class StderrCatcher:
Â*Â*Â*Â*Â*Â*Â*Â*def __init__(self):
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self.data = ''
Â*Â*Â*Â*Â*Â*Â*Â*def write(self,stuff):
Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self.data = self.data + "\t" + stuff
def main():

Â*Â*Â*Â*Â*Â*Â*Â*print "mymodule stderr: %s" % sys.stderr

This rebinds sys.stderr and a local 'stderr' in main()

Â*Â*Â*Â*Â*Â*Â*Â*sys.stderr = stderr = StderrCatcher()

but both HelloWorld.stderr and __main__.stderr are unaffected. What you need
is
import HelloWorld
sys.stderr = HelloWorld.stderr = StderrCatcher()

Â*Â*Â*Â*Â*Â*Â*Â*m = Cheers()
Â*Â*Â*Â*Â*Â*Â*Â*m.doSomeStuff()
Â*Â*Â*Â*Â*Â*Â*Â*print "stderr: \n%s" % sys.stderr.data

if __name__ == '__main__':
Â* Â* main()
Below there is the output when it doesn't work:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7d160b0>
Cheers stderr <open file '<stderr>', mode 'w' at 0xb7d160b0>
Here is my msg: Hello World !!
This is a nice day today !!
stderr:

And here when it works:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7dc40b0>
Cheers stderr <__main__.StderrCatcher instance at 0xb7d8bd4c>
stderr:
Â* Â* Â* Â* Here is my msg: Â* Â* Â* Â* Hello World !!
Â* Â* Â* Â* This is a nice day today !!
Thanks again!

PS Sorry for having probably replied to somone of you directly :-(

[alex23]

On Aug 27, 5:42*pm, "Diez B. Roggisch" <[e-mail]:

alex23 schrieb:

nisp <[e-mail]:

I've always been convinced of the equivalence of the two ways of using
the import statement but it's clear I'm wrong :-(

Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>from sys import stderr
import sys
sys.stderr is stderr

True

Behaviourly, I'm finding no difference between the two either.

Could you cut & paste a minimal example that isn't producing the
correct behaviour, and perhaps mention what type of OS you're using?

You did not read this part of the post:

"""
I'm not able to capture its stderr and of course I would like not to
do this kind of change.
"""

He tries to set sys.stderr to a new stream and capture the print
statements, but fails to do so because he created a local alias.

No, I read it, I just totally didn't parse it that way :)

Sorry for the confusion, nisp.

[nisp]

On Aug 27, 3:35 pm, Peter Otten <[e-mail]:

nisp wrote:

Thanks first of all ! I read the interesting Diez's link but something
still remains to me unclear, on the other hand it's clear the my
problem is concentrated there and on symbols.

Read it again. If you have two modules

module1.py
from sys import stderr

module2.py
from module1 import stderr

you get three names 'stderr' in three different namespaces, and each name
may bind a different object.

Here is what I'm trying to do

HelloWorld.py: this is a real simplification of my external module
though still reflecting its structure (commented out is the version
that, let's say, works)

from sys import stderr
#import sys

class Cheers:
def __init__(self):
self.cheersMsg = 'Hello World !!'
print "Cheers stderr %s" % stderr
#print "Cheers stderr %s" % sys.stderr
def _putCheers(self):
print>>stderr, 'Here is my msg:', self.cheersMsg
print>>stderr, 'This is a nice day today !!'
#print>>sys.stderr, 'Here is my msg:', self.cheersMsg
#print>>sys.stderr, 'This is a nice day today !!'
def doSomeStuff(self):
self._putCheers()

And below there is the module that uses the above one (mymodule.py):

#!/usr/bin/python

import sys

This imports HelloWorld.stderr:

from HelloWorld import *

You now have a global variable 'stderr' in module __main__, initialized to
the same value as HelloWorld.stderr.

class StderrCatcher:
def __init__(self):
self.data = ''
def write(self,stuff):
self.data = self.data + "\t" + stuff

def main():

print "mymodule stderr: %s" % sys.stderr

This rebinds sys.stderr and a local 'stderr' in main()

sys.stderr = stderr = StderrCatcher()

but both HelloWorld.stderr and __main__.stderr are unaffected. What you need
is
import HelloWorld
sys.stderr = HelloWorld.stderr = StderrCatcher()

m = Cheers()
m.doSomeStuff()
print "stderr: \n%s" % sys.stderr.data

if __name__ == '__main__':
main()

Below there is the output when it doesn't work:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7d160b0>
Cheers stderr <open file '<stderr>', mode 'w' at 0xb7d160b0>
Here is my msg: Hello World !!
This is a nice day today !!
stderr:

And here when it works:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7dc40b0>
Cheers stderr <__main__.StderrCatcher instance at 0xb7d8bd4c>
stderr:
Here is my msg: Hello World !!
This is a nice day today !!

Thanks again!

PS Sorry for having probably replied to somone of you directly :-(

Hi Peter!

Definitely you are right ! May be this time I understood completely
the topic (or may be not ?... gosh !) ... I didn't considered well the
"from HelloWolrd import *" in mymodule.py.
Now my module is:

import sys
import HelloWorld

class StderrCatcher:
def __init__(self):
self.data = ''
def write(self,stuff):
self.data = self.data + "\t" + stuff
def main():

print "mymodule stderr: %s" % sys.stderr

mystderr = sys.stderr = HelloWorld.stderr = StderrCatcher()
m = HelloWorld.Cheers()
m.doSomeStuff()
print >>sys.stderr, "mymodule prints something"
print "stderr: \n%s" % mystderr.data

if __name__ == '__main__':
main()

and its output is:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7d2d0b0>
Cheers stderr <__main__.StderrCatcher instance at 0xb7cf4e4c>
stderr:
Here is my msg: Hello World !!
This is a nice day today !!
mymodule prints something

now it works perfectly !

Great ! Thanks to all of you !

Nisp