473,320 Members | 1,900 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

readlines() doesn't read entire file

I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file. I have the following
syntax:

# some initial stuff
XS = xsdir(Datapath + '/xsdir', options.debug)
# some more stuff

class xsdir(object): #{{{1
"""This class handles all of the data and methods for reading
the xsdir file."""

def __init__(self, Datapath, debug=False):
self.xsdir = file(Datapath, 'r') # File object
self.lines = self.xsdir.readlines()
if debug:
print self.lines
# and then other stuff as well
I can see all the lines in the list self.lines, but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines, but Python cannot. Can someone help me with this one?
Thanks,
Jeremy

Jul 21 '05 #1
3 3210
Jeremy wrote:
I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file. I have the following
syntax:
.... self.xsdir = file(Datapath, 'r') # File object

I can see all the lines in the list self.lines, but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines, but Python cannot. Can someone help me with this one?


What platform? What version of Python?

You're opening the file in "text" mode. If you are on Windows and the
file actually contains a ^Z (byte 26) it is treated as EOF. Is that the
problem?

If not, have you tried cutting parts out of the file, to produce the
smallest file that still shows the problem? At that point you will
likely resolve the issue on your own. Also, does the same thing happen
if you use the interactive interpreter to read the file "manually"?

These are all basic troubleshooting techniques you can use at any time
on any problem...

-Peter
Jul 21 '05 #2
Peter Hansen wrote:
Jeremy wrote:
I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file. I have the following
syntax:


...
self.xsdir = file(Datapath, 'r') # File object

I can see all the lines in the list self.lines, but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines, but Python cannot. Can someone help me with this one?

What platform? What version of Python?

You're opening the file in "text" mode. If you are on Windows and the
file actually contains a ^Z (byte 26) it is treated as EOF. Is that the
problem?

If not, have you tried cutting parts out of the file, to produce the
smallest file that still shows the problem? At that point you will
likely resolve the issue on your own. Also, does the same thing happen
if you use the interactive interpreter to read the file "manually"?

These are all basic troubleshooting techniques you can use at any time
on any problem...

-Peter

Well, now I have to make a humbling retraction. I realized I wasn't
reading the file I thought I was, but one similar to it. Now that I am
reading the correct file, I am getting all the lines I expected to.
Thanks,
Jeremy

Jul 21 '05 #3
Jeremy wrote:
I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file.
Answer all of Peter Hansen's questions, then read on ...

You are on platform X; did you get the file from platform Y where Y != X?

Where did you get the file from? How was it constructed? What is the
encoding: ascii/utf-8/iso-something/cp125x/dunno/what's-an-encoding ...
I have the following
syntax:

# some initial stuff
XS = xsdir(Datapath + '/xsdir', options.debug)
# some more stuff

class xsdir(object): #{{{1
"""This class handles all of the data and methods for reading
the xsdir file."""

def __init__(self, Datapath, debug=False):
self.xsdir = file(Datapath, 'r') # File object
If you replace 'r' by 'rU', does the behaviour change?
self.lines = self.xsdir.readlines()
if debug:
print self.lines
Add this: print len(self.lines)
# and then other stuff as well
I can see all the lines in the list self.lines,
These are *not* pedantic questions: How many can you "see"? What does
"see" mean? What is your definition of "line"?
but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines,
Repeat above questions.

AND where are the extra lines that you can "see" with vim? In one block
at the end/beginning/middle? Randomly interspersed? Are you sure you are
not being confused by a screen-wraparound effect?

but Python cannot. Can someone help me with this one? Thanks,
Jeremy


What does "wc -l yourfile" tell you?
[If you are on Windows, get a copy of wc.exe from the gnuwin32 site.]
Jul 21 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Leeds, Mark | last post by:
I did more investigation into my previous problem and what happens is that my text file has \r for representing a new line instead of a \n. is there a way to tell the readlines function that the...
5
by: Richard | last post by:
Hi, Can anyone tell me what the difference is between for line in file.readlines( ): and for line in file:
2
by: Yong Wang | last post by:
Hi, I use readlines() to read one data file. Python automatically parses the read contents into a list of lines. When I used list to print out the 1st line, it is ok. When I use the list index 2...
9
by: wordsender | last post by:
Hey guys, I can't figure this one out, why is this simple script giving me problems? logfile=file(r'test.txt','w') logfile.write('datetime') test=logfile.readlines() When I run it I get...
2
by: vch | last post by:
Does a call to file.readlines() reads all lines at once in the memory? Are the any reasons, from the performance point of view, to prefer *while* loop with readline() to *for* loop with readlines()?
34
by: Ross Reyes | last post by:
HI - Sorry for maybe a too simple a question but I googled and also checked my reference O'Reilly Learning Python book and I did not find a satisfactory answer. When I use readlines, what...
4
by: wscrsurfdude | last post by:
Ik have an uml file I want to read with readlines. I have the following code: infile = open("out2.txt","r") for line in infile.readlines(): print line The print statement just gives the data,...
7
by: Wojciech Gryc | last post by:
Hi, I'm currently using Python to deal with a fairly large text file (800 MB), which I know has about 85,000 lines of text. I can confirm this because (1) I built the file myself, and (2)...
0
by: Jean-Paul Calderone | last post by:
On Mon, 12 May 2008 08:05:39 -0700 (PDT), cshirky <cshirky@gmail.comwrote: file.readlines reads the entire file into a list in memory. You may not want to do this. You could try, instead,...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.