By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
429,087 Members | 1,508 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 429,087 IT Pros & Developers. It's quick & easy.

Loading a Python collection from an text-file

P: n/a
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age

..

--
http://lazaridis.com
Jan 23 '06 #1
Share this Question
Share on Google+
14 Replies


P: n/a
Ilias Lazaridis wrote:
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age

.


This is specific for the text above. You will have to re-craft a regex
if the actual file is different.

import re

def get_names(afile):
regex = re.compile(r'{([^,]*),\s*([^}]*)}')
names = []
for aline in afile:
m = regex.search(aline)
names.append(m.groups())
return names

def test():
import cStringIO
afile = cStringIO.StringIO("{peter, 16},\n{anton, 21}\n")
print get_names(afile)

test()
Jan 23 '06 #2

P: n/a
Ilias Lazaridis wrote:
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age

.

"""
What I do for this kind of work is to use a gnumeric spreadsheet
which saves the data in a simple xml format. xml is much less
error-prone than plain text.
Google for, and study 'The gnumeric file format' by David Gilbert.
You need to know how to unzip the file, and how to write a SAX parser.
If you want to use a plain text format, keep it simple. I would
separate the two fields with tab (thus permit a comma within a field)
and allow 'comment' lines that start with a hash.
You don't need the braces, or the end-of-line comma you included.

# snip 'text-file.txt'
# name and age on one line separated by tab
Jonny 8
Mary 87
Moses 449
# end-snip 'text-file.txt'
Then:
"""

import string

class user:
def __init__(self,name,age):
self.name=name
self.age=int(age) # or a float, or a time-interval, or date-of-birth

def show(self):
print "%s is aged %s" % (self.name, self.age)

if __name__=="__main__":
users=[]
filename="text-file.txt"
fieldsep="\t"
F=open(filename,"r")
Lines=F.readlines()
for L0 in Lines:
L1=string.strip(L0)
if not L1.startswith("#"):
Record=string.split(L1,fieldsep)
# insert error handling/validation here
users.append(user(Record[0],Record[1]))

F.close()
for user in users:
user.show()
Jan 23 '06 #3

P: n/a
another approach (probably frowned upon, but it has worked for me) is
to use python syntax (a dictionary, say, or a list) and just import (or
reload) the file

Jan 24 '06 #4

P: n/a
Take a look at ConfigParser module. The format of the file would be
something like:

[members]
peter=16
anton=21

People are accustomed to this format file (windows .ini format).

-Larry
Ilias Lazaridis wrote:
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age

.

Jan 24 '06 #5

P: n/a
js*****@gmail.com wrote:
another approach (probably frowned upon, but it has worked for me) is
to use python syntax (a dictionary, say, or a list) and just import (or
reload) the file


this sounds good.

can I import a whole collection of instances this way?

-

(thanks for all the other answers within this thread).

..

--
http://lazaridis.com
Jan 27 '06 #6

P: n/a
perhapse consider using the pickle module?
http://docs.python.org/lib/module-pickle.html

Jan 27 '06 #7

P: n/a
On Mon, 23 Jan 2006 21:00:55 +0200, Ilias Lazaridis <il***@lazaridis.com> wrote:
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age

.

--
http://lazaridis.com


I'd use a CSV text file, maybe something like (only tested as far as you see!):

----< for_ilias_lazaridis.py >----------------------------------------------
import csv, types

class Fields(object):
def __init__(self, kvpairs): self.__dict__.update(kvpairs)

class Users(object):
def __init__(self):
self.userlist=[]
def load(self, lineiter):
if isinstance(lineiter, basestring):
lineiter = open(lineiter) # assume it's a file path
csvit = csv.reader(lineiter)
self.colnames = colnames = csvit.next()
typenames = csvit.next()
self.coltypes =coltypes = [getattr(types, name.capitalize()+'Type')
for name in typenames]
for row in csvit:
self.userlist.append(Fields(zip(colnames, (t(s) for t,s in zip(coltypes, row)))))
def __iter__(self): return iter(self.userlist)

def test():
import StringIO
f = StringIO.StringIO("""\
name,age
String,Int
peter,16
anton,21
""")
users = Users()
users.load(f)
for user in users:
print user.name, user.age
for user in users:
for name in users.colnames:
print '%s=%s,'%(name, getattr(user, name)),
print

if __name__ == '__main__': test()
-----------------------------------------------------------------------

Output:

[ 4:47] C:\pywk\clp>py24 for_ilias_lazaridis.py
peter 16
anton 21
name=peter, age=16,
name=anton, age=21,

(the first for user in users loop presumes knowledge of the field names name and age.
The second gets them automatically from the names loaded in the load method from
the first line of the text file. The second line expects type names as you see
in the types module, except without the "Type" suffix.

Perhaps you can adapt for your purposes.

Regards,
Bengt Richter
Jan 27 '06 #8

P: n/a
Seeing as we're suggesting alternatives, ConfigObj is great for hand
readable/writable data persistence.

You can use validate and ConfigPersist for automatic type conversion.

You can persist (basically) all the standard datatypes using this. The
syntax is usually more 'familiar' than Yaml, but it's not as flexible.

http://www.voidspace.org.uk/python/configobj.html

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Jan 27 '06 #9

P: n/a
Ilias Lazaridis wrote:
js*****@gmail.com wrote:
another approach (probably frowned upon, but it has worked for me) is
to use python syntax (a dictionary, say, or a list) and just import (or
reload) the file


this sounds good.

can I import a whole collection of instances this way?


Sure, it's just a Python module with variables in it.

I wouldn't try to teach my users Python syntax though.

If you really need this kind of data structure freedom,
I'd lean towards YAML or possibly XML. (XML isn't too
bad if you provide good tools. It's not a good idea
with just a text editor.)

If a spreadsheet like layout is enough, I (still)
recommend csv.
Jan 27 '06 #10

P: n/a
Ilias Lazaridis wrote:
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

If a spreadsheet like layout fits, use the csv module and
a plain comma separated file.

Then the end user can also use e.g. Excel to edit the data.
Jan 27 '06 #11

P: n/a
Ido Yehieli wrote:
perhapse consider using the pickle module?
http://docs.python.org/lib/module-pickle.html


User editable? We should be kind to our users!
d = {'peter':14, 'paul':23}
pickle.dumps(d)

"(dp0\nS'paul'\np1\nI23\nsS'peter'\np2\nI14\ns ."
Jan 27 '06 #12

P: n/a
>>Sure, it's just a Python module with variables in it.

I wouldn't try to teach my users Python syntax though.


not to mention the security risks

Jan 27 '06 #13

P: n/a
Ido Yehieli wrote:
Sure, it's just a Python module with variables in it.

I wouldn't try to teach my users Python syntax though.


not to mention the security risks


you mean all the things they can do from inside python that they
cannot do from the command line ?

</F>

Jan 28 '06 #14

P: n/a
Ken Starks wrote:
Ilias Lazaridis wrote:
within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age
[...]

the solutions below seems to be the most compact one.

this, or the suggested CSV module within the other messages.

thank's to everyone for the feedback.

[...] If you want to use a plain text format, keep it simple. I would
separate the two fields with tab (thus permit a comma within a field)
and allow 'comment' lines that start with a hash.
You don't need the braces, or the end-of-line comma you included.

# snip 'text-file.txt'
# name and age on one line separated by tab
Jonny 8
Mary 87
Moses 449
# end-snip 'text-file.txt'
Then:
"""

import string

class user:
def __init__(self,name,age):
self.name=name
self.age=int(age) # or a float, or a time-interval, or date-of-birth

def show(self):
print "%s is aged %s" % (self.name, self.age)

if __name__=="__main__":
users=[]
filename="text-file.txt"
fieldsep="\t"
F=open(filename,"r")
Lines=F.readlines()
for L0 in Lines:
L1=string.strip(L0)
if not L1.startswith("#"):
Record=string.split(L1,fieldsep)
# insert error handling/validation here
users.append(user(Record[0],Record[1]))

F.close()
for user in users:
user.show()


..

--
http://lazaridis.com
Jan 28 '06 #15

This discussion thread is closed

Replies have been disabled for this discussion.