input record seperator (equivalent of "$|" of perl)

[les_ander]

Hi,
I know that i can do readline() from a file object.
However, how can I read till a specific seperator?
for exmple,
if my files are

name
profession
id
#
name2
profession3
id2

I would like to read this file as a record.
I can do this in perl by defining a record seperator;
is there an equivalent in python?
thanks

[Fredrik Lundh]

le*******@yahoo .com wrote:

I know that i can do readline() from a file object.
However, how can I read till a specific seperator?

for exmple,
if my files are

name
profession
id
#
name2
profession3
id2

I would like to read this file as a record.
I can do this in perl by defining a record seperator;
is there an equivalent in python?

not really; you have to do it manually.

if the file isn't too large, consider reading all of it, and splitting on the
separator:

for record in file.read().spl it(separator):
print record # process record

if you're using a line-oriented separator, like in your example, tools like
itertools.group by can be quite handy:

from itertools import groupby

def is_separator(li ne):
return line[:1] == "#"

for sep, record in groupby(file, is_separator):
if not sep:
print list(record) # process record

or you could just spell things out:

record = []
for line in file:
if line[0] == "#":
if record:
print record # process record
record = []
else:
record.append(l ine)
if record:
print record # process the last record, if any

</F>

[Keith Dart]

le*******@yahoo .com wrote:

Hi,
I know that i can do readline() from a file object.
However, how can I read till a specific seperator?
for exmple,
if my files are

name
profession
id
#
name2
profession3
id2

I would like to read this file as a record.
I can do this in perl by defining a record seperator;
is there an equivalent in python?
thanks

I don't think so. But in the pyNMS package
(http://sourceforge/net/projects/pynms) there is a module called
"expect", and a class "Expect". With that you can wrap a file object and
use the Expect.read_unt il() method to do what you want.

--
-- ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~
Keith Dart <kd***@kdart.co m>
public key: ID: F3D288E4
=============== =============== =============== =============== =========

[Doug Holton]

le*******@yahoo .com wrote:

Hi,
I know that i can do readline() from a file object.
However, how can I read till a specific seperator?
for exmple,
if my files are

name
profession
id
#
name2
profession3
id2

I would like to read this file as a record.
I can do this in perl by defining a record seperator;
is there an equivalent in python?
thanks

To actually answer your question, there is no equivalent to $| in python.

You need to hand code your own record parser, or else read in the whole
contents of the file and use the string split method to chop it up into
fields.

[M.E.Farmer]

What about a generator and xreadlines for those really large files:

py>def recordbreaker(r ecordpath, seperator='#'):
.... rec = open(recordpath ,'r')
.... xrecord = rec.xreadlines( )
.... a =[]
.... for line in xrecord:
.... sep = line.find(seper ator)
.... if sep != -1:
.... a.append(line[:sep])
.... out = ''.join(a)
.... a =[]
.... a.append(line[sep+1:])
.... yield out
.... else:
.... a.append(line)
.... if a:
.... yield ''.join(a)
.... rec.close()
....
py>records = recordbreaker('/tmp/myrecords.txt')
py>for item in records:
.... print item

M.E.Farmer

le*******@yahoo .com wrote:

Hi,
I know that i can do readline() from a file object.
However, how can I read till a specific seperator?
for exmple,
if my files are

name
profession
id
#
name2
profession3
id2

I would like to read this file as a record.
I can do this in perl by defining a record seperator;
is there an equivalent in python?
thanks

[Fredrik Lundh]

"M.E.Farmer " wrote:

What about a generator and xreadlines for those really large files:

when you loop over a file object, Python uses a generator and a xreadlines-
style buffering system to read data as you go. (if you check the on-line help,
you'll notice that xreadlines itself is only provided for compatibility reasons).

or in other words, the examples I posted a couple of hours ago uses no more
memory than your version.

</F>

[M.E.Farmer]

Fredrik,
Thanks didn't realize that about reading a file on a for loop. Slick!
By the way the code I posted was an attempt at not building a
monolithic memory eating list like you did to return the values in your
second example.
Kinda thought it would be nice to read them as needed instead of all at
once.
I dont have itertools yet. That module looks like it rocks.
thanks for the pointers,
M.E.Farmer

[Scott David Daniels]

M.E.Farmer wrote:

I dont have itertools yet. That module looks like it rocks.
thanks for the pointers,
M.E.Farmer

If you have python 2.3 or 2.4, you have itertools.
--Scott David Daniels
Sc***********@A cm.Org

[M.E.Farmer]

Yea I should have mentioned I am running python 2.2.2.
Can it be ported to python 2.2.2?
Till they get python 2.4 all up and running....I'll wait a bit.
Thanks for the info,
M.E.Farmer

Scott David Daniels wrote:

M.E.Farmer wrote:

I dont have itertools yet. That module looks like it rocks.
thanks for the pointers,
M.E.Farmer

If you have python 2.3 or 2.4, you have itertools.
--Scott David Danies
Sc***********@A cm.Org

[Gábor Farkas]

Scott David Daniels wrote:

M.E.Farmer wrote:

I dont have itertools yet. That module looks like it rocks.
thanks for the pointers,
M.E.Farmer

If you have python 2.3 or 2.4, you have itertools.

for me it seems that 2.3 does not have itertools.group by.
it has itertools, but not itertools.group by.

activepython-2.4:(win)

import itertools
dir(itertools) ['__doc__', '__name__', 'chain', 'count', 'cycle', 'dropwhile',
'groupby', 'ifilter', 'ifilterfalse', 'imap', 'islice', 'izip',
'repeat', 'starmap', 'takewhile', 'tee']
python-2.3:(linux) import itertools
dir(itertools)

['__doc__', '__file__', '__name__', 'chain', 'count', 'cycle',
'dropwhile', 'ifilter', 'ifilterfalse', 'imap', 'islice', 'izip',
'repeat', 'starmap', 'takewhile']
gabor