473,722 Members | 2,147 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Hey, get this! [was: import from database]

This is even stranger: it makes it if I import the module a second time:

import dbimp as dbimp
import sys

if __name__ == "__main__":
dbimp.install()
#k = sys.modules.key s()
#k.sort()
#for kk in k:
#print kk
#import bsddb.db
import a.b.c.d
import smtplib
import ftplib
import fileinput
try:
print "first import"
import bsddb
except:
print "second import"
import bsddb
print "Done!"

$ python -i test.py
dbimporter: item: *db* args: () keywords: {}
Accepted *db*
dbimporter: item: /c/steve/Projects/Python/dbimp args: () keywords: {}
dbimporter: item: /c/steve/Projects/Python/dbimp/c args: () keywords: {}
dbimporter: item: /c/steve/Projects/Python/dbimp/\code args: () keywords: {}
dbimporter: item: /usr/lib/python24.zip args: () keywords: {}
dbimporter: item: /usr/lib/python2.4 args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/plat-cygwin args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/lib-tk args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/lib-dynload args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/site-packages args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/site-packages/a args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/site-packages/a/b args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/site-packages/a/b/c args: ()
keywords: {}
found smtplib in db
load_module: smtplib
found socket in db
load_module: socket
socket loaded: <module 'socket' from 'db:socket'> pkg: 0
found rfc822 in db
load_module: rfc822
rfc822 loaded: <module 'rfc822' from 'db:rfc822'> pkg: 0
found base64 in db
load_module: base64
base64 loaded: <module 'base64' from 'db:base64'> pkg: 0
found hmac in db
load_module: hmac
hmac loaded: <module 'hmac' from 'db:hmac'> pkg: 0
dbimporter: item: /usr/lib/python2.4/email args: () keywords: {}
found random in db
load_module: random
random loaded: <module 'random' from 'db:random'> pkg: 0
found quopri in db
load_module: quopri
quopri loaded: <module 'quopri' from 'db:quopri'> pkg: 0
smtplib loaded: <module 'smtplib' from 'db:smtplib'> pkg: 0
found ftplib in db
load_module: ftplib
dbimporter: item: /usr/lib/python2.4/site-packages/PIL args: () keywords: {}
dbimporter: item: /usr/lib/python2.4/site-packages/piddle args: ()
keywords: {}
ftplib loaded: <module 'ftplib' from 'db:ftplib'> pkg: 0
found fileinput in db
load_module: fileinput
fileinput loaded: <module 'fileinput' from 'db:fileinput'> pkg: 0
first import
found bsddb in db
load_module: bsddb
found weakref in db
load_module: weakref
weakref loaded: <module 'weakref' from 'db:weakref'> pkg: 0
second import
Done!


So it's clearly something pretty funky. It now "works" (for some value
of "work" wiht both MySQL and sqlite. I hope I have this sorted out
before PyCon ... I'm currently a bit confused!

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC +1 703 861 4237 +1 800 494 3119

Jul 18 '05 #1
5 2476
Steve Holden wrote:
This is even stranger: it makes it if I import the module a second time:


[second import seems to succeed]

Maybe you are experiencing some version confusion? What you describe looks
much like the normal Python 2.3 behaviour (with no import hook involved)
whereas you seem to operate on the 2.4 library.
A partially initialized module object is left behind in sys.modules and seen
by further import attempts.

$ cat arbitrary.py

import not_there

def f():
print "you ain't gonna see that"

$ python
Python 2.3.3 (#1, Apr 6 2004, 01:47:39)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
import arbitrary Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 2, in ?
import not_there
ImportError: No module named not_there import arbitrary
arbitrary.f() Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'module' object has no attribute 'f'
I have no experience with import hooks, but for normal imports that has been
fixed in Python 2.4:

$ py24
Python 2.4 (#5, Jan 4 2005, 10:14:01)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright" , "credits" or "license" for more information. import arbitrary Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 2, in ?
import not_there
ImportError: No module named not_there import arbitrary Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 2, in ?
import not_there
ImportError: No module named not_there


Peter

Jul 18 '05 #2
Peter Otten wrote:
Steve Holden wrote:

This is even stranger: it makes it if I import the module a second time:

[second import seems to succeed]

Maybe you are experiencing some version confusion? What you describe looks
much like the normal Python 2.3 behaviour (with no import hook involved)
whereas you seem to operate on the 2.4 library.
A partially initialized module object is left behind in sys.modules and seen
by further import attempts.

I agree that this is 2.3-like behavior, but Python cannot lie ...

sholden@dellboy ~/Projects/Python/dbimp
$ python
Python 2.4 (#1, Dec 4 2004, 20:10:33)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright" , "credits" or "license" for more information.
$ cat arbitrary.py

import not_there

def f():
print "you ain't gonna see that"

$ python
Python 2.3.3 (#1, Apr 6 2004, 01:47:39)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
import arbitrary
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 2, in ?
import not_there
ImportError: No module named not_there
import arbitrary
arbitrary.f ()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'module' object has no attribute 'f'
I have no experience with import hooks, but for normal imports that has been
fixed in Python 2.4:

$ py24
Python 2.4 (#5, Jan 4 2005, 10:14:01)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
import arbitrary
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 2, in ?
import not_there
ImportError: No module named not_there
import arbitrary


Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 2, in ?
import not_there
ImportError: No module named not_there
Peter

$ cat arbitrary.py
import not_there

def f():
print "you ain't gonna see that"
sholden@dellboy ~/Projects/Python/dbimp
$ python
Python 2.4 (#1, Dec 4 2004, 20:10:33)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright" , "credits" or "license" for more information. import arbitrary Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 1, in ?
import not_there
ImportError: No module named not_there import arbitrary Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "arbitrary. py", line 1, in ?
import not_there
ImportError: No module named not_there


Yup, looks like 2.4 (despite this funny cygwin stuff, could that make a
difference).

Let me try it under Windows [ferkle, ferkle ...]

Does the same thing there.

This problem also seems to depend what's already loaded. I wrote a
program to write a test program that looks like this:

import dbimp
dbimp.install()

print "Trying aifc"
try:
import aifc
except:
print "%Failed: aifc"
print "Trying anydbm"
try:
import anydbm
except:
print "%Failed: anydbm"
print "Trying asynchat"
try:
import asynchat
except:
print "%Failed: asynchat"

...

import dbimp
dbimp.install()

print "Trying aifc"
try:
import aifc
except:
print "%Failed: aifc"
print "Trying anydbm"
try:
import anydbm
except:
print "%Failed: anydbm"
print "Trying asynchat"
try:
import asynchat
except:
print "%Failed: asynchat"

The two platforms give expectedly close results. I'm storing compiled
code, so a version incompatibility would be a problem, I agree, but I
have checked the program that loaded the database, and loaded it again
from the Windows source rather than the CygWin source, just to see
whether there were any unnoticed platform dependencies. The results were
exactly the same using either library, and Windows and Cygwin showed
only minor variations.

exhaustCyg24.tx t:%Failed: bsddb.dbtables
exhaustCyg24.tx t:%Failed: bsddb.test.test _all
exhaustCyg24.tx t:%Failed: bsddb.test.test _associate
exhaustCyg24.tx t:%Failed: bsddb.test.test _basics
exhaustCyg24.tx t:%Failed: bsddb.test.test _compat
exhaustCyg24.tx t:%Failed: bsddb.test.test _dbobj
exhaustCyg24.tx t:%Failed: bsddb.test.test _dbshelve
exhaustCyg24.tx t:%Failed: bsddb.test.test _dbtables
exhaustCyg24.tx t:%Failed: bsddb.test.test _env_close
exhaustCyg24.tx t:%Failed: bsddb.test.test _get_none
exhaustCyg24.tx t:%Failed: bsddb.test.test _join
exhaustCyg24.tx t:%Failed: bsddb.test.test _lock
exhaustCyg24.tx t:%Failed: bsddb.test.test _misc
exhaustCyg24.tx t:%Failed: bsddb.test.test _queue
exhaustCyg24.tx t:%Failed: bsddb.test.test _recno
exhaustCyg24.tx t:%Failed: bsddb.test.test _thread
exhaustCyg24.tx t:%Failed: tzparse
exhaustWin24.tx t:%Failed: bsddb.dbtables
exhaustWin24.tx t:%Failed: bsddb.test.test _all
exhaustWin24.tx t:%Failed: bsddb.test.test _associate
exhaustWin24.tx t:%Failed: bsddb.test.test _basics
exhaustWin24.tx t:%Failed: bsddb.test.test _compat
exhaustWin24.tx t:%Failed: bsddb.test.test _dbobj
exhaustWin24.tx t:%Failed: bsddb.test.test _dbshelve
exhaustWin24.tx t:%Failed: bsddb.test.test _dbtables
exhaustWin24.tx t:%Failed: bsddb.test.test _env_close
exhaustWin24.tx t:%Failed: bsddb.test.test _get_none
exhaustWin24.tx t:%Failed: bsddb.test.test _join
exhaustWin24.tx t:%Failed: bsddb.test.test _lock
exhaustWin24.tx t:%Failed: bsddb.test.test _misc
exhaustWin24.tx t:%Failed: bsddb.test.test _queue
exhaustWin24.tx t:%Failed: bsddb.test.test _recno
exhaustWin24.tx t:%Failed: bsddb.test.test _thread
exhaustWin24.tx t:%Failed: pty
exhaustWin24.tx t:%Failed: rlcompleter
exhaustWin24.tx t:%Failed: tzparse

As a workaround I can for the moment just omit everything that show the
least suspicion of not working from the database, but I'd really rather
know what's failing.

If you have any clues I'd be grateful.

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC +1 703 861 4237 +1 800 494 3119
Jul 18 '05 #3
Steve Holden wrote:
Peter Otten wrote:
Steve Holden wrote:

This is even stranger: it makes it if I import the module a second time:


[second import seems to succeed]

Maybe you are experiencing some version confusion? What you describe
looks
much like the normal Python 2.3 behaviour (with no import hook involved)
whereas you seem to operate on the 2.4 library.
A partially initialized module object is left behind in sys.modules
and seen
by further import attempts.

I agree that this is 2.3-like behavior, but Python cannot lie ...

sholden@dellboy ~/Projects/Python/dbimp
$ python
Python 2.4 (#1, Dec 4 2004, 20:10:33)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright" , "credits" or "license" for more information.
>>>

Just to make things simpler, and (;-) to appeal to a wider audience,
here is a program that doesn't use database at all (it loads the entire
standard library into a dict) and still shows the error.

What *I* would like to know is: who is allowing the import of bsddb.os,
thereby somehow causing the code of the os library module to be run a
second time.

#
# Establish standard library in dict
#
import os
import glob
import sys
import marshal
import new

def importpy(dct, path, modname, package):
c = compile(file(pa th).read(), path, "exec")
dct[modname] = (marshal.dumps( c), package, path)
if package:
print "Package", modname, path
else:
print "Module", modname, path

def importall(dct, path, modlist):
os.chdir(path)
for f in glob.glob("*"):
if os.path.isdir(f ):
fn = os.path.join(pa th, f, "__init__.p y")
if os.path.exists( fn):
ml = modlist + [f]
importpy(dct, fn, ".".join(ml ), 1)
importall(dct, os.path.join(pa th, f), ml)
elif f.endswith('.py ') and '.' not in f[:-3] and f !=
"__init__.p y":
importpy(dct, os.path.join(pa th, f),
".".join(modlis t+[f[:-3]]), 0)

class dbimporter(obje ct):

def __init__(self, item, *args, **kw):
##print "dbimporter : item:", item, "args:", args, "keywords:" , kw
if item != "*db*":
raise ImportError
print "Accepted", item

def find_module(sel f, fullname, path=None):
print "find_modul e:", fullname, "from", path
if fullname not in impdict:
#print "Bailed on", fullname
return None
else:
print "found", fullname, "in db"
return self

def load_module(sel f, modname):
print "load_modul e:", modname
if modname in sys.modules:
return sys.modules[modname]
try:
row = impdict[modname]
except KeyError:
#print modname, "not found in db"
raise ImportError, "DB module %s not found in modules"
code, package, path = row
code = marshal.loads(c ode)
module = new.module(modn ame)
sys.modules[modname] = module
module.__name__ = modname
module.__file__ = path # "db:%s" % modname
module.__loader __ = dbimporter
if package:
module.__path__ = sys.path
exec code in module.__dict__
print modname, "loaded:", repr(module), "pkg:", package
return module

def install():
sys.path_hooks. append(dbimport er)
sys.path_import er_cache.clear( ) # probably not necessary
sys.path.insert (0, "*db*") # probably not needed with a metea-path
hook?

if __name__ == "__main__":
impdict = {}
for path in sys.argv[1:]:
importall(impdi ct, path, [])
install()
import bsddb

Running this against a copy of the Python 2.4 standard library in C:\Lib
gives me

[...]
Module _strptime C:\Lib\_strptim e.py
Module _threading_loca l C:\Lib\_threadi ng_local.py
Module __future__ C:\Lib\__future __.py
Accepted *db*
find_module: bsddb from None
found bsddb in db
load_module: bsddb
find_module: bsddb._bsddb from None
find_module: bsddb.sys from None
find_module: bsddb.os from None
find_module: bsddb.nt from None
find_module: bsddb.ntpath from None
find_module: bsddb.stat from None
Traceback (most recent call last):
File "C:\Steve\Proje cts\Python\dbim p\dictload.py", line 79, in ?
import bsddb
File "C:\Steve\Proje cts\Python\dbim p\dictload.py", line 65, in
load_module
exec code in module.__dict__
File "C:\Lib\bsddb\_ _init__.py", line 62, in ?
import sys, os
File "C:\Python24\li b\os.py", line 133, in ?
from os.path import (curdir, pardir, sep, pathsep, defpath, extsep,
altsep,
ImportError: No module named path

The 2.3 bsddb library doesn't cause the same problems (and even loads
into 2.4 quite nicely). Lots of modules *will* import, and most packages
don't seem to cause problems. Anyone give me a pointer here?

regards
Steve
--
Meet the Python developers and your c.l.py favorites March 23-25
Come to PyCon DC 2005 http://www.python.org/pycon/2005/
Steve Holden http://www.holdenweb.com/
Jul 18 '05 #4
Steve Holden <st***@holdenwe b.com> writes:
What *I* would like to know is: who is allowing the import of bsddb.os,
thereby somehow causing the code of the os library module to be run a
second time.
I would guess (without actually running the code) that this part is
responsible:
if package:
module.__path__ = sys.path


You usually should initialize a package's __path__ to an empty list.
The __init__ module will take care of modifying it if necessary.

Bernhard

--
Intevation GmbH http://intevation.de/
Skencil http://skencil.org/
Thuban http://thuban.intevation.org/
Jul 18 '05 #5
Bernhard Herzog wrote:
Steve Holden <st***@holdenwe b.com> writes:

What *I* would like to know is: who is allowing the import of bsddb.os,
thereby somehow causing the code of the os library module to be run a
second time.

I would guess (without actually running the code) that this part is
responsible:

if package:
module.__path__ = sys.path

You usually should initialize a package's __path__ to an empty list.
The __init__ module will take care of modifying it if necessary.

Bernhard

Thanks, Bernhard, that appeared to fix the problem. The error certainly
went away, anyway ... now I can do more experimentation .

regards
Steve
--
Meet the Python developers and your c.l.py favorites March 23-25
Come to PyCon DC 2005 http://www.python.org/pycon/2005/
Steve Holden http://www.holdenweb.com/
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
3362
by: James | last post by:
I have a from with 2 fields: Company & Name Depening which is completed, one of the following queries will be run: if($Company){ $query = "Select C* From tblsample Where ID = $Company Order By Company ASC";
5
2755
by: Scott D | last post by:
I am trying to check and see if a field is posted or not, if not posted then assign $location which is a session variable to $location_other. If it is posted then just assign it to $location_other I keep getting "Notice: Undefined index: location_other" referring to (!($_POST)) { $location_other = $location; } else
2
2729
by: Nick | last post by:
Can someone please tell me how to access elements from a multiple selection list? From what ive read on other posts, this is correct. I keep getting an "Undefined variable" error though... Form page************************************************************ <form action="/process.php" method="get" name="formOne" id="formOne"> <select name="owner" size="6" multiple id="owner"> <option value="one">one</option> <option...
2
2549
by: Alexander Ross | last post by:
I have a variable ($x) that can have 50 different (string) values. I want to check for 7 of those values and do something based on it ... as I see it I have 2 options: 1) if (($x=="one") || ($x=="two") || ... || ($x=="seven")) ... or 2) switch ($x){ case("one"):
0
3280
by: Dan Foley | last post by:
This script runs fine, but I'd like to know why it's so slow.. Thanks for any help out there on how i can make it faster (it might take up to 5 min to write these 3 export files whith 15 records each!!!) Dan ==================== <style> body, table, tr, td { font-family: 'verdana'; font-size: 12px;
5
3230
by: Lee Redeem | last post by:
Hi there I've created abd uploaded this basic PHP script: <html> <head> <title>PHP Test</title> </head> <body> <H1 align="center">
5
10053
by: christopher vogt | last post by:
Hi, i'm wondering if there is something like $this-> to call a method inside another method of the same class without using the classname in front. I actually use class TEST { function func1()
6
2667
by: Phil Powell | last post by:
Ok guys, here we go again! SELECT s.nnet_produkt_storrelse_navn FROM nnet_produkt_storrelse s, nnet_produkt_varegruppe v, nnet_storrelse_varegruppe_assoc sv, nnet_produkt p WHERE s.nnet_produkt_storrelse.id = sv.nnet_produkt_storrelse_id AND sv.nnet_produkt_varegruppe_id = v.nnet_produkt_varegruppe_id AND sv.nnet_produkt_varegruppe_id IN ( SELECT nnet_produkt_varegruppe_id FROM nnet_produkt_varegruppe
1
2195
by: Michel | last post by:
a site like this http://www.dvdzone2.com/dvd Can you make it in PHP and MySQL within 6 weeks? If so, send me your price 2 a r a (at) p a n d o r a . b e
1
1509
by: Acorn Tutors | last post by:
What Im trying to do is select a single entry from a table called HEADPIC where the current date and time are after a datetime value in a field called POSTED and before a datetime value in a field called APOSTED. Then I want to take the matching text (which is the name of a picture) in the PICNAME field and put it in a link so that, depending on the date, a different picture will show up (for holidays and such). I have two ways that I...
0
8863
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9384
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
9157
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9088
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8052
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
4502
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4762
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3207
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2147
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.