473,727 Members | 2,018 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Import module with non-standard file name

Howdy all,

Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?

Background:

On Unix, I write programs intended to be run as commands to a file
with no extension. This allows other programs to use the command as an
interface, and I can re-write the program in some other language
without obsoleting the commandline interface.

e.g., I might write 'frobnicate-foo' as a shell program so that other
programs can 'frobnicate-foo --bar baz'. If I later decide to
re-implement 'frobnicate-foo' in Python, I'll save the top level
module to the same file name since it implements the same command-line
interface.

Now that I've got it written as a Python module, I'd like to write
unit tests for that module, which of course will need to import the
program module to test it. The unit test can explicitly add the
directory where the program module lives to 'sys.path' for the purpose
of importing that module.

However, the Python reference tells me that 'import' (specifically,
'__import__()') needs modules to live in files named a particular way:
with a '.py' suffix. But my module is in a file called
'frobnicate-foo', with no suffix, and that's part of the definition of
the program interface.

I don't want symbolic links, or anything else that presents two
filenames for the same module, because there's no need for that except
for Python's apparent insistence on a particular naming
convention. Also, avoiding symbolic links inside the source code tree
makes version control smoother.
What are my options to import a module from a file whose name can't
change?

--
\ "[W]e are still the first generation of users, and for all that |
`\ we may have invented the net, we still don't really get it." |
_o__) -- Douglas Adams |
Ben Finney

Aug 8 '06 #1
10 3764

Ben Finney wrote:
Howdy all,

Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?

Background:

On Unix, I write programs intended to be run as commands to a file
with no extension. This allows other programs to use the command as an
interface, and I can re-write the program in some other language
without obsoleting the commandline interface.

e.g., I might write 'frobnicate-foo' as a shell program so that other
programs can 'frobnicate-foo --bar baz'. If I later decide to
re-implement 'frobnicate-foo' in Python, I'll save the top level
module to the same file name since it implements the same command-line
interface.

Now that I've got it written as a Python module, I'd like to write
unit tests for that module, which of course will need to import the
program module to test it. The unit test can explicitly add the
directory where the program module lives to 'sys.path' for the purpose
of importing that module.
If it can do that, it can copy the MUT to some temp directory, adding
..py to the end of the name of the new file, and put the temp directory
in sys.path .... can't it?

Cheers,
John

Aug 8 '06 #2
"John Machin" <sj******@lexic on.netwrites:
Ben Finney wrote:
Now that I've got it written as a Python module, I'd like to write
unit tests for that module, which of course will need to import
the program module to test it. The unit test can explicitly add
the directory where the program module lives to 'sys.path' for the
purpose of importing that module.

If it can do that, it can copy the MUT to some temp directory,
adding .py to the end of the name of the new file, and put the temp
directory in sys.path .... can't it?
Sounds like a nasty hack (not that fiddling sys.path isn't a hack, but
at least that one's addressed in Python 2.5 with relative and absolute
imports).

The problem with importing the program module from a file in a
different directory is that the program won't be able to find its own
relative modules. That leads to either *more* sys.path hackery, or
importing from a temporary file in the *same* directory.

Besides which, that's still two file names for the same module
code. The whole point of testing is to know that I'm testing the same
module; with a two-file shim, that's one step further away from that
ideal.

What you describe is possible, but leads to very smelly hacks. I'd
like to see what other options there are.

--
\ "It's not what you pay a man, but what he costs you that |
`\ counts." -- Will Rogers |
_o__) |
Ben Finney

Aug 8 '06 #3

Ben Finney wrote:
"John Machin" <sj******@lexic on.netwrites:
Ben Finney wrote:
Now that I've got it written as a Python module, I'd like to write
unit tests for that module, which of course will need to import
the program module to test it. The unit test can explicitly add
the directory where the program module lives to 'sys.path' for the
purpose of importing that module.
If it can do that, it can copy the MUT to some temp directory,
adding .py to the end of the name of the new file, and put the temp
directory in sys.path .... can't it?

Sounds like a nasty hack (not that fiddling sys.path isn't a hack, but
at least that one's addressed in Python 2.5 with relative and absolute
imports).

The problem with importing the program module from a file in a
different directory is that the program won't be able to find its own
relative modules. That leads to either *more* sys.path hackery, or
importing from a temporary file in the *same* directory.
Please explain both the "own" and "relative" in "its own relative
modules". Do these modules not have names that end in ".py"?
>
Besides which, that's still two file names for the same module
code. The whole point of testing is to know that I'm testing the same
module; with a two-file shim, that's one step further away from that
ideal.
The two-file caper exists only for the duration of the test. You'll
have to trust yourselt to write and test a file copying gadget. :-)

>
What you describe is possible, but leads to very smelly hacks. I'd
like to see what other options there are.
Probably smellier ones ...:<)

Cheers,
John

Aug 8 '06 #4
"John Machin" <sj******@lexic on.netwrites:
Ben Finney wrote:
"John Machin" <sj******@lexic on.netwrites:
If it can [modify sys.path], it can copy the MUT to some temp
directory, adding .py to the end of the name of the new file,
and put the temp directory in sys.path .... can't it?
The problem with importing the program module from a file in a
different directory is that the program won't be able to find its
own relative modules. That leads to either *more* sys.path
hackery, or importing from a temporary file in the *same*
directory.

Please explain both the "own" and "relative" in "its own relative
modules". Do these modules not have names that end in ".py"?
The program can import modules with relative paths, because it can
expect its position in the directory tree to remain the same relative
to those modules. If the program module suddenly exists in a different
directory, that assumption no longer holds and the relative imports
performed by the program will fail.

Thus to avoid that problem, testing needs to be done on the program
module with its position relative to all other modules the same as
when that program runs.

--
\ "It may be that our role on this planet is not to worship God |
`\ -- but to create him." -- Arthur C. Clarke |
_o__) |
Ben Finney

Aug 8 '06 #5

Ben Finney wrote:
Howdy all,

Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?
This is a piece of cake in Python.
>>from types import ModuleType
x = ModuleType('myM odName')
data = open('myfilenam e').read()
exec data in x.__dict__
Your output here...

This won't save a .pyc, but as your message later explains, this is for
unittesting, so this could probably be considered a feature for this
usage.

Regards,
Pat

Aug 8 '06 #6
Ben Finney wrote:
Howdy all,

Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?

Background:

On Unix, I write programs intended to be run as commands to a file
with no extension. This allows other programs to use the command as an
interface, and I can re-write the program in some other language
without obsoleting the commandline interface.

e.g., I might write 'frobnicate-foo' as a shell program so that other
programs can 'frobnicate-foo --bar baz'. If I later decide to
re-implement 'frobnicate-foo' in Python, I'll save the top level
module to the same file name since it implements the same command-line
interface.

Now that I've got it written as a Python module, I'd like to write
unit tests for that module, which of course will need to import the
program module to test it. The unit test can explicitly add the
directory where the program module lives to 'sys.path' for the purpose
of importing that module.

However, the Python reference tells me that 'import' (specifically,
'__import__()') needs modules to live in files named a particular way:
with a '.py' suffix. But my module is in a file called
'frobnicate-foo', with no suffix, and that's part of the definition of
the program interface.

I don't want symbolic links, or anything else that presents two
filenames for the same module, because there's no need for that except
for Python's apparent insistence on a particular naming
convention. Also, avoiding symbolic links inside the source code tree
makes version control smoother.
What are my options to import a module from a file whose name can't
change?

--
\ "[W]e are still the first generation of users, and for all that |
`\ we may have invented the net, we still don't really get it." |
_o__) -- Douglas Adams |
Ben Finney
Leave your python module with the .py extension and create a small
python script without the .py extension to import and run your code
from the command line.

For example, on my [linux] system /usr/local/bin/idle contains this:

#!/usr/bin/python

from idlelib.PyShell import main
if __name__ == '__main__':
main()

You also get a modest performance boost because the interpreter will
only process the text of this small script but will use the precompiled
byte-code .pyc files (when available) of your main module, rather than
re-parsing its text.

HTH,
~Simon

Aug 8 '06 #7
"Patrick Maupin" <pm*****@gmail. comwrites:
Ben Finney wrote:
Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?

This is a piece of cake in Python.
>from types import ModuleType
x = ModuleType('myM odName')
data = open('myfilenam e').read()
exec data in x.__dict__
Your output here...

This won't save a .pyc, but as your message later explains, this is for
unittesting, so this could probably be considered a feature for this
usage.
Very nice. Okay, my unit testing scaffold module now has a new function:

def make_module_fro m_file(module_n ame, file_name):
""" Make a new module object from the code in specified file """

from types import ModuleType
module = ModuleType(modu le_name)

module_file = open(file_name, 'r')
exec module_file in module.__dict__

return module

The unit test now just imports that functionality, and then makes the
module object via that function:

import scaffold
module_name = 'frobnicate_foo '
module_file_und er_test = os.path.join(sc affold.code_dir , 'frobnicate-foo')
frobnicate_foo = scaffold.make_m odule_from_file (
module_name, module_file_und er_test)

The rest of the unit test then has 'frobnicate_foo ' as a module to test.

It's working fine. Does anyone foresee any problems with doing it this way?

--
\ "Injustice is relatively easy to bear; what stings is justice." |
`\ -- Henry L. Mencken |
_o__) |
Ben Finney

Aug 8 '06 #8
Hello Ben,
Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?
http://docs.python.org/lib/module-imp.html (hint: load_source :)

HTH,
Miki
http://pythonwise.blogspot.com/

Aug 8 '06 #9
Ben Finney schrieb:
Question: I have Python modules named without '.py' as the extension,
and I'd like to be able to import them. How can I do that?
I recommend to use imp.load_module .

Regards,
Martin
Aug 8 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
2242
by: Matthew Wilson | last post by:
Hi- I'm writing a bunch of classes, several of which need functions and variables defined in the math module. In some instances, I'm going to import my module like this: import myshapes and then in others, I'll do:
9
2940
by: Paul Rubin | last post by:
That's what the Python style guides advise. They don't seem to like def frob(x): import re if re.search('sdfxyz', x): ... instead preferring that you pollute your module's global namespace with the names of all your imports. What's the point of that? It gets worse when you want to do something like
4
7277
by: Peter L. Buschman | last post by:
I have a package tree laid out like this foo/bar/module/submodule foo/bar/module/submodule/test foo/bar/module/submodule/test/test1.py foo/bar/module/submodule/test/test2.py .... What I want is for test1.py and test2.py to be able to import their parent "submodule" as part of a regression
8
2080
by: Grant D. Watson | last post by:
If this has been answered before, or if my terminology is off, please bear with me; my Python experience is limited to use in one class and to personal projects. I'd like to do something rather silly: I'd like to run a particular piece of code from a given module every time that the module is imported, and not just at the time that the module is originally loaded. So, every time a module says import foo or something analogous, I want...
5
2477
by: Steve Holden | last post by:
This is even stranger: it makes it if I import the module a second time: import dbimp as dbimp import sys if __name__ == "__main__": dbimp.install() #k = sys.modules.keys() #k.sort() #for kk in k:
3
1282
by: Jim | last post by:
I would be very grateful for help on the following. I have the following modules in a program. Names changed to protect the innocent. 1.Simulation 2.Branches 3.MyFiles I import Branches, Myfiles and the publicly available module Numeric
9
382
by: fegge | last post by:
i have written script save as hello.py. i can run it. but why cant i import it as a modular in other programs?
5
5336
by: Roland Hedberg | last post by:
Hi! I'm having a bit of a problem with import. I'm writing a marshalling system that based on a specification will create one or more files containing mostly class definitions. If there are more than one file created (and there are reasons for creating more than one file in some instances) then they will import each other since there may be or is interdependencies in between them.
8
3755
by: dmitrey | last post by:
howto check does module 'asdf' exist (is available for import) or no? (without try/cache of course) Thx in advance, D.
1
1835
by: John Boy | last post by:
First post and very much a newbie to Python. Have 2.5 on Linux (GG). I think I have set up PYTHONPATH correctly in that I can import a module apply_bp and it complains about line 20 in apply_bp which is: import sys, aipy, numpy, os At the interpreter prompt, however, I can import sys, numpy etc. and can do dir() and see the entry points so I think my overall installation is OK.
0
8891
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9406
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9260
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9185
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
1
6703
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6011
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4521
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4786
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3228
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.