473,325 Members | 2,342 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,325 software developers and data experts.

module import search path strangeness

tow
I have a python script (part of a django application, if it makes any
difference) which is exhibiting the following behaviour:

import my_module # succeeds
imp.find_module("my_module") # fails, raising ImportError

which is completely baffling me. According to sys.path, both should
fail; the directory containing my_module is not in sys.path (though
the my_module directory itself is). More puzzlingly, printing out
my_module.__file__ gives:

/home/tow/test/my_module/../my_module/__init__.pyc

I don't really understand what the ".." is doing in there.

Can someone explain what I'm missing here, it's got me stumped.

Toby
Aug 11 '08 #1
8 1537
En Mon, 11 Aug 2008 19:19:19 -0300, tow <to************@googlemail.com>
escribi�:
I have a python script (part of a django application, if it makes any
difference) which is exhibiting the following behaviour:

import my_module # succeeds
imp.find_module("my_module") # fails, raising ImportError

which is completely baffling me. According to sys.path, both should
fail; the directory containing my_module is not in sys.path (though
the my_module directory itself is).
my_module is not a module but a package, right? Else I don't understand
the above statement.
More puzzlingly, printing out
my_module.__file__ gives:

/home/tow/test/my_module/../my_module/__init__.pyc

I don't really understand what the ".." is doing in there.

Can someone explain what I'm missing here, it's got me stumped.
Perhaps you have ".." in sys.path? And the current directory happens to be
/home/tow/test/my_module?

--
Gabriel Genellina

Aug 12 '08 #2
tow
On Aug 12, 4:59*am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
En Mon, 11 Aug 2008 19:19:19 -0300, tow <toby.o.h.wh...@googlemail.com*
escribi :
I have a python script (part of a django application, if it makes any
difference) which is exhibiting the following behaviour:
import my_module # succeeds
imp.find_module("my_module") # fails, raising ImportError
which is completely baffling me. According to sys.path, both should
fail; the directory containing my_module is not in sys.path (though
the my_module directory itself is).

my_module is not a module but a package, right? Else I don't understand *
the above statement.
Sorry, sloppy terminology on my part, yes, my_module is a package.
More puzzlingly, printing out
my_module.__file__ gives:
/home/tow/test/my_module/../my_module/__init__.pyc
I don't really understand what the ".." is doing in there.
Can someone explain what I'm missing here, it's got me stumped.

Perhaps you have ".." in sys.path? And the current directory happens to be *
/home/tow/test/my_module?
The current directory is actually a subdirectory of /home/tow/test/
my_module,
but ".." is not in sys.path (nor is anything containing "..") In any
case, if
it were a matter of sys.path, surely the imp.find_module call above
should succeed?

Basically, I had thought that import and imp.find_module used exactly
the same
search path, but the above example shows that at least in this
circumstance they
don't; import is picking up additional search paths from somewhere -
what am I missing?

Toby
Aug 12 '08 #3
tow wrote:
>I have a python script (part of a django application, if it makes any
difference) which is exhibiting the following behaviour:
import my_module # succeeds
imp.find_module("my_module") # fails, raising ImportError
which is completely baffling me. According to sys.path, both should
fail; the directory containing my_module is not in sys.path (though
the my_module directory itself is).
The current directory is actually a subdirectory of /home/tow/test/
my_module,
but ".." is not in sys.path (nor is anything containing "..") In any
Basically, I had thought that import and imp.find_module used exactly
the same
search path, but the above example shows that at least in this
circumstance they
don't; import is picking up additional search paths from somewhere -
what am I missing?
Grepping through the django source finds

../trunk/django/core/management/__init__.py:
sys.path.append(os.path.join(project_directory, os.pardir))

sys.path could be changed as a side effect of an import, so I ask you to
verify (again) that import and imp.find_module() see the same sys.path.

assert all(".." not in p for p in sys.path)
import my_module

assert all(".." not in p for p in sys.path)
imp.find_module("my_module")
Peter
Aug 12 '08 #4
tow
On Aug 12, 9:56*am, Peter Otten <__pete...@web.dewrote:
tow wrote:
Basically, I had thought that import and imp.find_module used exactly
the same
search path, but the above example shows that at least in this
circumstance they
don't; import is picking up additional search paths from somewhere -
what am I missing?

Grepping through the django source finds

./trunk/django/core/management/__init__.py: *
sys.path.append(os.path.join(project_directory, os.pardir))
Hmm. It turns out that that is indeed the issue, but in a way that
wasn't immediately obvious to me. Looking at it in more context:

sys.path.append(os.path.join(project_directory, os.pardir))
project_module = __import__(project_name, {}, {}, [''])
sys.path.pop()

sys.path is extended, the project module is imported, then the
additional path is dropped from sys.path.
And if I comment out those sys.path manipulations, I get the result I
expect later on.

What I think is happening is that in that stanza, project_module
(which is my_module in my case) is
imported from the altered sys.path. sys.path is then put back, but
python now knows about my_module.

As a result when my_module is imported again elsewhere, python already
knows about it, so no searching
of sys.path is done (which is good, because it wouldn't be found on
the current sys.path), and the
import apparently succeeds, though in fact it's effectively
(actually?) a no-op.

However, imp.find_module("my_module") forces a search of the sys.path,
and thus fails.

Or at least, that is what I surmise, given that I see

import my_module # succeeds
imp.find_module("my_module") # fails, raising ImportError

So, to answer my original question, the difference in search behaviour
between "import" and "imp.find_module" is that the former might not
look at sys.path at all if the module has already been loaded, while
the latter will only search on the current sys.path.

Am I right?

Toby
Aug 12 '08 #5
tow wrote:
On Aug 12, 9:56Â*am, Peter Otten <__pete...@web.dewrote:
>tow wrote:
Basically, I had thought that import and imp.find_module used exactly
the same
search path, but the above example shows that at least in this
circumstance they
don't; import is picking up additional search paths from somewhere -
what am I missing?

Grepping through the django source finds

./trunk/django/core/management/__init__.py:
sys.path.append(os.path.join(project_directory, os.pardir))

Hmm. It turns out that that is indeed the issue, but in a way that
wasn't immediately obvious to me. Looking at it in more context:

sys.path.append(os.path.join(project_directory, os.pardir))
project_module = __import__(project_name, {}, {}, [''])
sys.path.pop()
Ouch.
So, to answer my original question, the difference in search behaviour
between "import" and "imp.find_module" is that the former might not
look at sys.path at all if the module has already been loaded, while
the latter will only search on the current sys.path.

Am I right?
Yes. 'import' looks up the file in a cache, the sys.modules dictionary,
before it falls back to the more costly alternatives.

Peter
Aug 12 '08 #6
Peter Otten <__*******@web.dewrote:
>tow wrote:
> sys.path.append(os.path.join(project_directory, os.pardir))
project_module = __import__(project_name, {}, {}, [''])
sys.path.pop()
Ouch.
I presume that "Ouch" is in consideration of what might happen if
the subject of the __import__ also calls sys.path.append ...?

--
\S -- si***@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
"Frankly I have no feelings towards penguins one way or the other"
-- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
Aug 12 '08 #7
tow
On Aug 12, 4:59*pm, Peter Otten <__pete...@web.dewrote:
tow wrote:
On Aug 12, 9:56*am, Peter Otten <__pete...@web.dewrote:
tow wrote:
Basically, I had thought that import and imp.find_module used exactly
the same
search path, but the above example shows that at least in this
circumstance they
don't; import is picking up additional search paths from somewhere -
what am I missing?
Grepping through the django source finds
./trunk/django/core/management/__init__.py:
sys.path.append(os.path.join(project_directory, os.pardir))
Hmm. It turns out that that is indeed the issue, but in a way that
wasn't immediately obvious to me. Looking at it in more context:
* * sys.path.append(os.path.join(project_directory, os.pardir))
* * project_module = __import__(project_name, {}, {}, [''])
* * sys.path.pop()

Ouch.
So, to answer my original question, the difference in search behaviour
between "import" and "imp.find_module" is that the former might not
look at sys.path at all if the module has already been loaded, while
the latter will only search on the current sys.path.
Am I right?

Yes. 'import' looks up the file in a cache, the sys.modules dictionary,
before it falls back to the more costly alternatives.

Peter
Thanks, at least I understand what's going on now.

Toby
Aug 12 '08 #8
Sion Arrowsmith wrote:
Peter Otten <__*******@web.dewrote:
>>tow wrote:
>> sys.path.append(os.path.join(project_directory, os.pardir))
project_module = __import__(project_name, {}, {}, [''])
sys.path.pop()
Ouch.

I presume that "Ouch" is in consideration of what might happen if
the subject of the __import__ also calls sys.path.append ...?
Yes ;)

Peter
Aug 12 '08 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Csaba Henk | last post by:
Hello! I use Python 2.2.2 on Linux. I met the following issue: Say I have the module mymod and the script myprog.py in /mydir. myprog.py imports mymod. If I create a symlink /foo/myprog2.py...
3
by: Stephen Ferg | last post by:
I need a little help here. I'm developing some introductory material on Python for non-programmers. The first draft includes this statement. Is this correct? ...
4
by: Jesse B. | last post by:
I have just begun to learn python using o'reilly's Learning Python, and in chapter 3 it talks about importing modules. It has me make modules to practice using the 'import' and 'from' statements...
6
by: kimes | last post by:
I've just started digging into how python works.. I found that other mudules are clearly declared like one file per a module.. But the only os.path doesn't have their own file.. ye I know is...
6
by: Theo v. Werkhoven | last post by:
Goodday, Something strange going on here. A piece of code I wrote bombs out in one of de directories under $HOME, but not in others. Here's a snipped: #v+ def bin2asc(c): s=''
3
by: kwatch | last post by:
What is the condition of module name which is available in 'from .. import ..' statement ? ---------------------------------------- import os print os.path # <module 'posixpath'...
1
by: mhearne808[insert-at-sign-here]gmail[insert-dot-he | last post by:
I think I don't understand how the module search path works... Let's say I have a folders called 'test'. Underneath it, I create two more folders called 'foo' and 'bar'. In 'foo', I create an...
5
by: koara | last post by:
Hello, is there a way to access a module that is hidden because another module (of the same name) is found first? More specifically, i have my own logging.py module, and inside this module,...
5
by: sawilla | last post by:
First, I'm new to Python. I'm getting and error when I run Python 2.5.2 as a regular user in Vista but not when I run Python as an administrator. For example, if I type "import numpy" after I...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.