looking for way to include many times some .py code from anotherpython code

[Martin MOKREJ©]

Hi,
I'm looking for some easy way to do something like include in c or PHP.
Imagine I would like to have:
cat somefile.py
a = 222
b = 111
c = 9
cat somefile2.py
self.xxx = a
self.zzz = b
self.c = c
self.d = d
cat anotherfile.py

def a():
include somefile
postprocess(a)

def b():
include somefile
postprocess(a, b, c)

class klass():
def __init__(self, a, b, c, d):
include somefile2

I know about module imports and reloads, but am not sure if this is the right
way to go. Mainly, I want to assign to multiple object instances some self bound
variables. Their values will be different, so I can't use global variables.

Martin

[Scott David Daniels]

Martin MOKREJ© wrote:

.... If I put them into a module, it get's executed only once unless I
do reload. And I'd have to use: "from some import *",
because mainly I'm interrested in assigning to self:
self.x = "blah"
self.y = "uhm"
OK, somewhere in here I think I get what you want to do. Essentially
you want to set a lot of attributes on some object which is almost
always named "self", and the "set a lot of attributes" varies as
separate chunks. The perfect application for a function (not a _pure_
function in the functional programming sense). So, it is my opinion
that you want to define a function to call, not include code from some
other file.

How about:

Here are some "whole lot of variables" functions, put them in 'code.py':

def do_a_bunch(obj) :
obj.a = 123
obj.b = 3.141529
obj.c = 'what on earth?'
obj.author = u'Charles Dickens'
...

def do_other_stuff( obj):
obj.a = 123456789
obj.b2 = 3.141529 ** .5
obj.c = u'Where in Jupiter?'
obj.author = u'Martin MOKREJ©'
...

And here is how you use them:

from code import do_a_bunch, do_other_stuff

class SomethingOrOthe r(SomeSuperClas s):
def __init__(self, stuff, nonsense):
SomeSuperClass. __init__(self, stuff)
self.fiddle(non sense)
do_a_bunch(self )

def some_other_meth od(self):
...
do_a_bunch(self )
def mangle(self):
...
do_other_stuff( self)

I'm newbie, sure.

That is why I was trying to figure out your original requirement,
not how to accomplish your original plan. I was trying to see if
there was a good reason you needed to use "#include" - like behavior.
Does something like this address your problem?

--Scott David Daniels
Sc***********@A cm.Org

[Martin MOKREJŠ]

Hi to everyone who has repsonded. I'll try to clarify my problem in more detail.
I believe I have the answers how to assign to self. in superclasses. In case
you would know of yet another way, let me know. ;)

Steve Holden wrote:

Martin MOKREJŠ wrote:

Diez B. Roggisch wrote:

See my post on Mar 2 about "automating assignment of class variables".
I got no answers, maybe I wasn't clear enough ... :( class Foo:
def __init__(self, a, b, .....):
self.a = a
self.b = b
....
If that is what you want, then this might help you: Put all the
values in a

The data passed to an instance come from sql tables. I have the idea
every table will be represented by an object. At the most upper level
of abstraction, I work with, partly overlapping set of tables.

imagine two objects, X and Y. X refers to table T1, T2, T3 while
Y refers to T1, T3, T4, T5.

let's say:
T1 has columns T1C1, T1C2, T1C3
T2 has columns T2C1, T2C2, T2C3, ...
and so on

object t1 is something like:

class t1:
def __init__(self, T1C1, T1C2, T1C3=None):
self.T1C1 = T1C1
self.T1C2 = T1C2
self.T1C3 = T1C3

similarly in case of t2 or any other table. The column names/types are different,
also requirements for non-NULL values differ (T1C3 above is allowed to be empty).
T1C2 is ENUM type, for example which must be equal ether to "a" or "b" or "d" ... etc.
object X is something like:

class x:
def __init__(self, t1object, t2object, t3object):
self.t1object = t1object
self.t2object = t2object
self.t3object = t3object
self.xml_root = cElementTree.El ement("xml root")

# and now the boring stuff comes
self.xml_t1 = cElementTree.Su bElement(self.x ml_root, "table 1")
self.xml_t1_T1C 1 = cElementTree.Su bElement(self.x ml_T1C1, "name of T1C1 comlumn")
self.xml_t1_T1C 1.text = self.t1object.T 1C1
self.xml_t1_T1C 2 = cElementTree.Su bElement(self.x ml_T1C2, "name of T1C2 comlumn")
self.xml_t1_T1C 2.text = self.t1object.T 1C2
# ... and more or less the same for tables t2 and t3

def set_more_data_f rom_optional_ta bles(self, t6object, t7object):
# I'd do (self, anyobject) on the line above, but the objects represent different
# sql tables, so different variable names apply. I believe the best I could do
# is to walk their __dict__ so I could assign the data under self.xml_root ...
# can any of teh XML: libraries represent an object in XML?
self.xml_t6_T6C 1 = cElementTree.Su bElement(self.x ml_T6C1, "name of T6C1 comlumn")
self.xml_t6_T6C 1.text = self.t6object.T 1C1
self.xml_t6_T6C 2 = cElementTree.Su bElement(self.x ml_T6C2, "name of T6C2 comlumn")
self.xml_t6_T6C 2.text = self.t6object.T 1C2
# ... and more or less the same for tables t7
class y:
def __init__(self, t1object, t3object, t4object, t5object=None):
self.t1object = t1object
self.t3object = t3object
self.t4object = t4object
if t5object:
self.t5object = t5object

self.xml_root = cElementTree.El ement("xml root")

# now the code from x.__init__() to table t1 and t3 would be cut&pasted,
# or now I can say any of the two or three approaches suggested in this
# thread will be used
Good guess! ;)

dictionary - like this:

my_vals = {"a": 1, "b" : 2, ....}

There are plenty of other ways to create such a dictionary, but I won't
digress on that here.

Now in your class, you pass than dict to your constructor and then
simply
update the instance's __dict__ so that the keys-value-pairs in my_vals
become attributes:

class Foo:

def __init__(self, my_vals):
self.__dict__.u pdate(my_vals)

foo = Foo(my_vals)

print foo.a

-> 1

Hope this helps,
Sure. Thanks! Would you prefer exactly for this method over the method
posted by Kent Johnson
few minutes ago?

Am I so deperately fighting the language? No-one here on the list
needs to set hundreds
variables at once somewhere in their code? I still don't get why:

Well, consider that you haven't actually made any kind of a case for
using variables!

OK, imagine I want to read the data from sql into memory and check the
values (there're some conditions they have to fullfill). Therefore, once
I get the classes defined, I'll implement methods to check every table
t1 to tx for it's contents. ^H^H^H^H^H^H^H^ H^H^H^H^H Actually, the checks
have to be based on X or Y instance. The check differs between X.t1
and Y.t1. :(

If you ask me why do I have such a mess, my answer is that I don't want
to store same datatype into two different tables. So the location is only
one, but the value has to be filled in only when X and is unused when Y.
Similarly it happens when some foreign keys are/aren't defined.

If the names of these things are dynamic then why would you want to put
them in an object's namespace, when you could just as easily include

self.insDict = {}

in your __init__() method and then simply update self.insDict whenever
you want.

If you "set hundreds variables", and their names aren't predictable,
then presumably you will have to go through similar contortions to
access them.

So it is generally simpler just to use a dictionary to hold the values
whose names aren't known in advance. Techniques for inserting names into
namespaces are known (as you have discovered), but when a beginner wants
to know about them it's usually considered a sign of "fighting the
language".
I can't smash everything into dictionary, I'd have to keep about 3 or 4-levels
of dictionaries withing dictionaries. Imagine, t10 refers to rows in tables
t8 and t9 but also t20 refers to other rows in t8 and t9. I may not loose
the information what refers to what.

The easiest is to use variables. Anyway, I need them to test their content.
Actually, also to assign to them new data to write into the database.
I have about 20 tables and about 250 columns in total. And I use foreign keys
a lot.
So, think about this a while and then tell us exactly *why* it's so
important that these values are stored in variables.

"include somefile.py" would be that non-pythonic so that it's not
available, but
I have already two choices how to proceed anyway. Thanks. ;)

Now have to figure out how to assign them easily into the XML tree.

martin

You do know there are lots of Python libraries that support XML, right?

Well, see baove. ;)

M.

[Steven Bethard]

Martin MOKREJŠ wrote:

The data passed to an instance come from sql tables. I have the idea
every table will be represented by an object.

Have you looked at SQLObject? I've never used it, but it does seem like
this is the sort of thing it was designed for:

http://sqlobject.org/

STeVe

[Peter Hansen]

Martin MOKREJŠ wrote:

Am I so deperately fighting the language? No-one here on the list needs
to set hundreds variables at once somewhere in their code?
Nobody needs to do that. As others have pointed out, creating variables
implies wanting to access them distinctly, not as a whole group. If
you are just going to access them as a group, use contain objects such
as lists or dicts or a custom class, not individual variables.
Now have to figure out how to assign them easily into the XML tree.

This might be the hint that others were hoping for, about your
real requirements. Do you mean to say that the whole reason
you have for assigning hundreds of variables is to go and
shove the values right back into another data structure such
as an XML document? If so, trust us, you very likely don't
want to do it by assigning and then referencing hundreds of
variables.

-Peter

[Martin MOKREJŠ]

Peter Hansen wrote:

Martin MOKREJŠ wrote:

Am I so deperately fighting the language? No-one here on the list
needs to set hundreds variables at once somewhere in their code?

Nobody needs to do that. As others have pointed out, creating variables
implies wanting to access them distinctly, not as a whole group. If
you are just going to access them as a group, use contain objects such
as lists or dicts or a custom class, not individual variables.

I understand, but this is unfortunately not my case now. It's really
about assigning data from mysql to some variables and getting them later
checked. The checks will be different for most variables and will even
differ if I read from sql or alternatively I instantiate first new data
obtained from web interface and write subsequently into sql (for example,
all row ID's won't be known while instantiating the objects).

Now have to figure out how to assign them easily into the XML tree.

This might be the hint that others were hoping for, about your
real requirements. Do you mean to say that the whole reason
you have for assigning hundreds of variables is to go and
shove the values right back into another data structure such
as an XML document? If so, trust us, you very likely don't

No, the xml is another reason why I wanted to walk over the __dict__
of some object and let something magically constrcut the XML tree for me.
But this is really another, distinct problem from teh one I posted originally.
want to do it by assigning and then referencing hundreds of
variables.

I need to test almost every for it's content. Some tests
are just that the value is non-empty (few cases), but in most cases
a lot more checks (only certain values allowed, or only int type allowed,
or only \w is allowed ...).

FYI: The program/database runs at the moment under php + mysql.

[Steve Holden]

Martin MOKREJŠ wrote:

Peter Hansen wrote:

Martin MOKREJŠ wrote:

Am I so deperately fighting the language? No-one here on the list
needs to set hundreds variables at once somewhere in their code?

Nobody needs to do that. As others have pointed out, creating variables
implies wanting to access them distinctly, not as a whole group. If
you are just going to access them as a group, use contain objects such
as lists or dicts or a custom class, not individual variables.

I understand, but this is unfortunately not my case now. It's really
about assigning data from mysql to some variables and getting them later
checked. The checks will be different for most variables and will even
differ if I read from sql or alternatively I instantiate first new data
obtained from web interface and write subsequently into sql (for example,
all row ID's won't be known while instantiating the objects).

Now have to figure out how to assign them easily into the XML tree.

This might be the hint that others were hoping for, about your
real requirements. Do you mean to say that the whole reason
you have for assigning hundreds of variables is to go and
shove the values right back into another data structure such
as an XML document? If so, trust us, you very likely don't

No, the xml is another reason why I wanted to walk over the __dict__
of some object and let something magically constrcut the XML tree for me.
But this is really another, distinct problem from teh one I posted
originally.

want to do it by assigning and then referencing hundreds of
variables.

I need to test almost every for it's content. Some tests
are just that the value is non-empty (few cases), but in most cases
a lot more checks (only certain values allowed, or only int type allowed,
or only \w is allowed ...).

FYI: The program/database runs at the moment under php + mysql.

I will be *very* surprised if you can't get a much better (i.e. easier
and more efficient) solution by stepping back from the programming
details for a moment and explaining what it is you are actually trying
to achieve in user-space.

Can you describe the problem you are trying to solve, rather than the
solution you are hoping to adopt?

regards
Steve

[Martin MOKREJŠ]

Steve Holden wrote:

Martin MOKREJŠ wrote:

Peter Hansen wrote:

Martin MOKREJŠ wrote:

Am I so deperately fighting the language? No-one here on the list
needs to set hundreds variables at once somewhere in their code?


Nobody needs to do that. As others have pointed out, creating variables
implies wanting to access them distinctly, not as a whole group. If
you are just going to access them as a group, use contain objects such
as lists or dicts or a custom class, not individual variables.
I understand, but this is unfortunately not my case now. It's really
about assigning data from mysql to some variables and getting them later
checked. The checks will be different for most variables and will even
differ if I read from sql or alternatively I instantiate first new data
obtained from web interface and write subsequently into sql (for example,
all row ID's won't be known while instantiating the objects).

Now have to figure out how to assign them easily into the XML tree.


This might be the hint that others were hoping for, about your
real requirements. Do you mean to say that the whole reason
you have for assigning hundreds of variables is to go and
shove the values right back into another data structure such
as an XML document? If so, trust us, you very likely don't

No, the xml is another reason why I wanted to walk over the __dict__
of some object and let something magically constrcut the XML tree for me.
But this is really another, distinct problem from teh one I posted
originally.

want to do it by assigning and then referencing hundreds of
variables.

I need to test almost every for it's content. Some tests
are just that the value is non-empty (few cases), but in most cases
a lot more checks (only certain values allowed, or only int type allowed,
or only \w is allowed ...).

FYI: The program/database runs at the moment under php + mysql.

I will be *very* surprised if you can't get a much better (i.e. easier
and more efficient) solution by stepping back from the programming

Hmm, I'm not convinced, but I'll put few more words here then. ;)
details for a moment and explaining what it is you are actually trying
to achieve in user-space.

Can you describe the problem you are trying to solve, rather than the
solution you are hoping to adopt?

User inputs data through html forms, data has to be quality-checked
and strored into mysql. Later, data is read from mysql and presented
through web. There's nothing special in it, just that the tables
describe a lot of specific experimental parameters. Tables do reflect
type of informations, so they group the data into logical units - so
tablename reflects the data contained.

In general, there are two types of data, hence my X and Y objects.
The underlaying data at some time point go into sql tables. Before that
happens, the data == variable contents are checked that they contain
expected values (in some cases enumerated values, in some cases integers,
sometime chars and only about 6 blobs). I spent a year developing the
database schema and php code, the schema is nearly optimal. I got bored
by the php code, as it was partly developed by a lazy guy (lazier than I'm).
I went fot python - to have better error handling, have not only web app,
but reusable code for standalone application (html forms can be replaced
by any tcl/tk widget for M$ Windows). Sql transaction I have added to
the php code, but anyway it sucks to work with it further.

My idea is to check some of the values while instantiating, as I get it for
free (assigning either to a default value or raising an exception when
variable is empty). In most cases this is not enough, and I have to type in
the allowed values.
1. In case of enumerated types, I hope to find a tool
able to read sql files and able to extract column definitions. In this
particular case, the program would dynamically read allowed ENUM values,
so whenever sql table is altered, the program will recognize new value
allowed.
2. In most other cases, the values are simply some kind of string, and
..find() et al. will suffice.
3. In case data was read from mysql, I can verify that foreign keys refer
to what they should refer.

OK, I get the data written to mysql. I can fetch it back, and want to dump
it into xml and present on web/(local gui).

I have the claases corresponding to all tables as superclasses of X and Y
as necessary. I went to ask on this list how to assign the variables easily
because parts of the code are almost identical. I believe this has been
answered quite well.

I believe the approach using classes corresponding to every single table
is right, when using them as superclasses for those two, practically
used objects: X and Y.

To print the output on web or any gui, I think I'll use the xml output
and just parse it. I need xml anyway for testing, and definitely want
to be able to construct the html/GUI output from the xml input - again,
for testing. So the objects will more or less exist only to get the
necessary checks done for those myriads of variables, which must be
evaluated in current context. I'd get crazy if I'd store things into
bsbdb -- I'm not going to remember that a[0] is table1, a[1] is table2,
a[0][0] is the primary key called blah, a[0][22] is allowed to be
equal only to "foo" or "bar" ... and that if a[2][4] is defined (actually number),
the number is the key to search in c[key]. Simply, that's for what I use mysql
I don't want to invent the database schema in bsddb in python. ;)
It's simply data, it must be read into variables in some objects, those
object are groupped into just two superobjects. The superobjects define
check-methods, define how to dump the it's data into xml, how
to write (in which order) the values into mysql.

I'm sorry not to send in the sql schema + the code, but this is my phd thesis. ;)

I'm very glad there's so many people interrested to help - not only - me.
Thanks! Now I'm really looking forward how would you rework this thing.
It's simple, easy, it's just sometime tedious as having 250 columns in 20 tables
simply makes you bored to type the code, after while.

The only think where I think I need help is, how to dump easily into xml say object
X, having variables a, b, c, where c is a ref. to object B, containing variables p, q, r.

B = obj()
setattr(B, p, 44)
setattr(B, q, "sdjahd")
setattr(B, r, "qew")
X = obj()
setattr(X, a, 1)
setattr(X, a, 2)
setattr(X, a, B) print do_magick(X) <X>
<a>1</a>
<b>2</b>
<B>
<p>44</p>
<q>sdjahd</q>
<r>qew</r>
</B>
</X>

Martin

[Martin MOKREJ©]

Scott David Daniels wrote:

Martin MOKREJ© wrote:

.... If I put them into a module, it get's executed only once unless I

> do reload. And I'd have to use: "from some import *",

because mainly I'm interrested in assigning to self:
self.x = "blah"
self.y = "uhm"

OK, somewhere in here I think I get what you want to do. Essentially
you want to set a lot of attributes on some object which is almost
always named "self", and the "set a lot of attributes" varies as
separate chunks. The perfect application for a function (not a _pure_
function in the functional programming sense). So, it is my opinion
that you want to define a function to call, not include code from some
other file.

Basically, doing in a class method
def(self, a, b, c):
self.a = a
self.b = b
self.c = c
sounds stupid. With next instance I'll loose a, b, c, so I have to save
then to a variable, "self." prefix is generally proposed way. But
it's not surprising a gets to self.a, right? Actually, I thought about
the *args tuple and even **kwargs, but I thought this will make the core
even less readable. Thinking of it now, you'd probably do
self.__dict__.u pdate(kwargs), right? Hmm, but does it assign to self or not?
I mean, does it equivalent to `a = 1' or `self.a = 1' ? The latter seem to
be true, right?


How about:

Here are some "whole lot of variables" functions, put them in 'code.py':

def do_a_bunch(obj) :
obj.a = 123
obj.b = 3.141529
obj.c = 'what on earth?'
obj.author = u'Charles Dickens'
...

def do_other_stuff( obj):
obj.a = 123456789
obj.b2 = 3.141529 ** .5
obj.c = u'Where in Jupiter?'
obj.author = u'Martin MOKREJ©'
...

And here is how you use them:

from code import do_a_bunch, do_other_stuff

class SomethingOrOthe r(SomeSuperClas s):
def __init__(self, stuff, nonsense):
SomeSuperClass. __init__(self, stuff)
self.fiddle(non sense)
do_a_bunch(self )

def some_other_meth od(self):
...
do_a_bunch(self )
def mangle(self):
...
do_other_stuff( self)

I'm newbie, sure.

That is why I was trying to figure out your original requirement,
not how to accomplish your original plan. I was trying to see if
there was a good reason you needed to use "#include" - like behavior.
Does something like this address your problem?

This doesn't help me. ;)

[Steven Bethard]

Martin MOKREJ© wrote:

Basically, doing in a class method
def(self, a, b, c):
self.a = a
self.b = b
self.c = c
sounds stupid. With next instance I'll loose a, b, c, so I have to save
then to a variable, "self." prefix is generally proposed way. But
it's not surprising a gets to self.a, right? Actually, I thought about
the *args tuple and even **kwargs, but I thought this will make the core
even less readable. Thinking of it now, you'd probably do
self.__dict__.u pdate(kwargs), right? Hmm, but does it assign to self or
not?
I mean, does it equivalent to `a = 1' or `self.a = 1' ? The latter seem to
be true, right?

Try it and see:

py> def update1(obj, **kwargs):
.... obj.__dict__.up date(kwargs)
....
py> def update2(obj, a, b, c):
.... obj.a, obj.b, obj.c = a, b, c
....
py> class C(object):
.... pass
....
py> c = C()
py> update1(c, a=1, b=2, c=3)
py> c.a, c.b, c.c
(1, 2, 3)
py> c = C()
py> update2(c, a=1, b=2, c=3)
py> c.a, c.b, c.c
(1, 2, 3)
py> c = C()
py> update1(c, 1, 2, 3)
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: update1() takes exactly 1 argument (4 given)
py> c = C()
py> update2(c, 1, 2, 3)
py> c.a, c.b, c.c
(1, 2, 3)

Note however that unless you do something like

def update(obj, **kwargs):
assert list(kwargs) == 'a b c'.split()
...

then the update with the **kwargs will take any parameters (not just a,
b and c). Don't know if that matters to you.

STeVe

[Peter Hansen]

Martin MOKREJŠ wrote:

I need to test almost every for it's content. Some tests
are just that the value is non-empty (few cases), but in most cases
a lot more checks (only certain values allowed, or only int type allowed,
or only \w is allowed ...).

This sounds very much like an opportunity for some code
that is "data-driven". Rather than dealing with individual
variables, you define data structures (or even just input
files specialized for your situation, if that seems best)
which describe the sorts of tests you talk about above.

You would include the names of the various fields, in
order to match them up with their respective descriptions,
and then have generic code that dealt with each field in
the appropriate manner, driven by what was in the above
data structures.

As trivialized example, to show what I mean:

import types
NonEmpty = object()
Unique = object()

info = {
'field1': (NonEmpty, r'\w+', types.int),
'field2': (None, r'[a-zA-Z_][a-za-Z0-9]*', types.str),
'foo': (Unique, r'\d+', types.float),
}

# inside some routine that loads the data:
for name, text in readMyData():
try:
flag, pattern, type = info[name]
except KeyError:
print 'ignoring unrecognized field', name
else:
if flag == NonEmpty:
if text == '':
raise ValueError('fie ld %s cannot be empty' % name)
value = convertToType(t ext, type)
storeMyData(nam e, value)
and so forth....

I think that gives the idea. Generally if I had
hundreds of fields, I wouldn't even bother with
the hardcoded dict as above but would just define my
own file format to describe all this, and parse it
first into some internal representation before trying
to read the real data.

-Peter