473,322 Members | 1,523 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Reading a file and then writing something back

Hi All -

I'm not sure, but I'm wondering if this is a bug, or maybe (more
likely) I'm misunderstanding something...see below:
f = open('testfile', 'w')
f.write('kevin\n')
f.write('dan\n')
f.write('pat\n')
f.close()
f = open('testfile', 'r+')
f.readline() 'kevin\n' f.readline() 'dan\n' f.write('chris\n')

Traceback (most recent call last):
File "<stdin>", line 1, in ?
IOError: (0, 'Error')

I've figured out that I can do an open('testfile', 'r+') and then seek
and write something (without an error), but it just seems odd that I
would get an IOError for what I was trying to do. Oh, and I also
tried to do "f.flush()" before the write operation with no luck.
I've searched google, but can't seem to find much. Any thoughts???
TIA,

Kevin
Jul 18 '05 #1
6 1755
Kevin T. Ryan wrote:
I'm not sure, but I'm wondering if this is a bug, or maybe (more
likely) I'm misunderstanding something...see below:

f = open('testfile', 'w')
f.write('kevin\n')
f.write('dan\n')
f.write('pat\n')
f.close()
f = open('testfile', 'r+')
f.readline()
'kevin\n'
f.readline()
'dan\n'
f.write('chris\n')

Traceback (most recent call last):
File "<stdin>", line 1, in ?
IOError: (0, 'Error')


This is just a guess, I don't know the inner workings of files in
Python, but here we go:

I think that readline() doesn't read one character at a time from the
file, until it finds a newline, but reads a whole block of characters,
looks for the first newline and returns that string (for efficiency
reasons). Due to this buffering, the file pointer position is undefined
after a readline(), and so a write() afterwards doesn't make sense.
Python tries to help you not to fall into this trap.
I've figured out that I can do an open('testfile', 'r+') and then seek
and write something (without an error), but it just seems odd that I
would get an IOError for what I was trying to do.


When you do a seek(), the file pointer position is clearly defined, so a
write() makes sense.

A tentative solution could be:

pos = f.tell()
s = f.readline() # Reads 'dan\n'
f.seek(pos + len(s))
f.write('chris\n')

However, I'm not sure you want to do that, as the string written will
just overwrite the previous content, and will probably not be aligned
with the next newline in the file. Except if you don't care about the
data following your write.

Hope this helps.
-- Remy
Remove underscore and anti-spam suffix in reply address for a timely
response.
Jul 18 '05 #2
Python's file object is based on ISO C's file I/O primitives
(fopen, fread, etc) and inherits both the requirements of the standard
and any quirks of your OS's C implementation.

According to this document
http://www.lysator.liu.se/c/rat/d9.html#4-9-5-3
a direction change is only permitted after a "flushing" operation
(fsetpos, fseek, rewind, fflush). file.flush calls C's fflush.

I believe that this C program is equivalent to your Python program:

#include <stdio.h>

int main(void) {
char line[21];
FILE *f = fopen("testfile", "w");
fputs("kevin\n", f);
fputs("dan\n", f);
fputs("pat\n", f);
fclose(f);

f = fopen("testfile", "r+");
fgets(line, 20, f); printf("%s", line);
fgets(line, 20, f); printf("%s", line);

fflush(f);

if(fputs("chris\n", f) == EOF) { perror("fputs"); }
fclose(f);

return 0;
}

On my Linux machine, it prints
kevin
pat
and testfile's third and final line is "chris".

On a windows machine nearby (compiled with mingw, but using msvcrt.dll)
it prints
kevin
dan
fputs: No error
and testfile's third and final line is "pat".

If I add fseek(f, 0, SEEK_CUR) after fflush(f), I don't get a failure
but I do get the curious contents
kevin
dan
pat
chris

If I use just fseek(f, 0, SEEK_CUR) I get no error and correct
contents in testfile.

I don't have a copy of the actual C standard, but even Microsoft's
documentation says
When the "r+", "w+", or "a+" access type is specified, both reading
and writing are allowed (the file is said to be open for “update”).
However, when you switch between reading and writing, there must be
an intervening fflush, fsetpos, fseek, or rewind operation. The
current position can be specified for the fsetpos or fseek
operation, if desired.
http://msdn.microsoft.com/library/de...c_._wfopen.asp
so it smells like a bug to me. Do you happen to be using Windows? I
guess you didn't actually say.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFA/ov9Jd01MZaTXX0RAp+CAJ9/rH5C+G1fNhCCFqbCp88tTx2rtgCeNqkw
JPdi2MVpPT81hWiMYjADddE=
=ax80
-----END PGP SIGNATURE-----

Jul 18 '05 #3
Hi Kevin,

Even though I am fairly new to Python, it appears that you might of
found a bug with 'r+' writing / reading mode.

Here's a couple of suggestions which you might find helpful:

1) To make your programs faster and less 'error'-prone, you might want
to read the text file into memory (a list) first, like this:

f = open("c:/test.txt", "r")
names = f.readlines() # Read all lines in file and store data in a list.
for name in names: # Display a listing of all lines in the file.
print name
f.close() # Close the text file (we'll change it later).
2) When you want to add new names to the "text file" (in memory), you
can easily do so by doing this:

names = names + ["William\n"] # Adds William to the list (text file
in memory).
names = names + ["Steven\n"] # Adds Steven to the list.
names = names + ["Tony\n"] # Adds Tony to the list also.
3) If you wish to sort the list in memory, you can do this:

names.sort() # Places the names in the list now in ascending
order (A - Z).
4) Finally, to re-write the text file on the disk, you can do this:

f = open("c:/test.txt", "w") # Re-write the file from scratch with
revised info.
for name in names: # For each name that is in the list (names)
f.write(name) # Write it to the file.
f.close() # Finally, since the file has now been 100%
rewritten with new data, close it.
--------------

Why does this have advantages? Several reasons, which are:

1) It does the processing in the memory, which is much quicker. Faster
programs are always a nice feature!
2) It allows for additional processes to occur, such as sorting, etc.
3) It reduces the chances of "having a disk problem." One simple read &
one simple write.
Hope this helps,

Byron
---
Jul 18 '05 #4
Opps, forgot to add one extra thing:
--------------------------------------------

If you would like to see all of the names from your "names" list, you
can do the following:

for name in names:
print name
This provides you with the results:

Dan
Kevin
Pat
Steven
Tony
William
---

If you would like to see the first and fourth items in the list, you can
do the following:

print names[0] # Display the first item in the list. First item
always starts with zero.
print names[3] # Display the fourth item in the list.

Result is:

Dan
Steven

---

Finally, if you would like to remove an item from the list:

del names[3]

---

Hope this helps!

Byron
----------------------------
Jul 18 '05 #5
Remy Blank wrote:
Kevin T. Ryan wrote:
I'm not sure, but I'm wondering if this is a bug, or maybe (more
likely) I'm misunderstanding something...see below:

>f = open('testfile', 'w')
>f.write('kevin\n')
>f.write('dan\n')
>f.write('pat\n')
>f.close()
>f = open('testfile', 'r+')
>f.readline()


'kevin\n'
>f.readline()


'dan\n'
>f.write('chris\n')


Traceback (most recent call last):
File "<stdin>", line 1, in ?
IOError: (0, 'Error')


This is just a guess, I don't know the inner workings of files in
Python, but here we go:

I think that readline() doesn't read one character at a time from the
file, until it finds a newline, but reads a whole block of characters,
looks for the first newline and returns that string (for efficiency
reasons). Due to this buffering, the file pointer position is undefined
after a readline(), and so a write() afterwards doesn't make sense.
Python tries to help you not to fall into this trap.
I've figured out that I can do an open('testfile', 'r+') and then seek
and write something (without an error), but it just seems odd that I
would get an IOError for what I was trying to do.


When you do a seek(), the file pointer position is clearly defined, so a
write() makes sense.

A tentative solution could be:

pos = f.tell()
s = f.readline() # Reads 'dan\n'
f.seek(pos + len(s))
f.write('chris\n')

However, I'm not sure you want to do that, as the string written will
just overwrite the previous content, and will probably not be aligned
with the next newline in the file. Except if you don't care about the
data following your write.

Hope this helps.
-- Remy
Remove underscore and anti-spam suffix in reply address for a timely
response.

Thanks all for the suggestions. I'm guessing that what Remy stated seems to
be about right. Jeff: I AM using windows (or at least, was today while i
was writing that script)...if Remy was wrong, then it still might be a bug
though - I tried to do the f.flush(), but the error still occurred.

Byron - thanks for the advice. For my simple example, you're totally
correct, but I was thinking along the lines of a much bigger file w/ tons
of records - and therefore didn't want to slurp everything in to memory in
case the file got to be too big. Maybe I'm wrong though - I don't know how
much a "normal" computer could hold in memory (maybe 100's of 1,000's of
lines?).

Oh well, thanks again :)
Jul 18 '05 #6
IEEE Std 1003.1 says that a "file positioning function" (fseek, fsetpos,
rewind) must be called when a stream's direction changes from input
to output.

http://www.opengroup.org/onlinepubs/...ons/fopen.html

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFA/pmWJd01MZaTXX0RAs7EAJ9nZCXeYK0bIr+zu7oQw7VRLZvxLwC dGlrD
eEVIhhgHe/2N4DAqKWg2Jkk=
=P/SR
-----END PGP SIGNATURE-----

Jul 18 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Marc | last post by:
Hello, I have a question about how PHP handles multiple file reads/ writes. I made a page containing a self-submitting form where the user can type his name, topic and a text. When he submits the...
4
by: john smith | last post by:
Hi, I have a file format that is going to contain some parts in ascii, and some parts with raw binary data. Should I open this file with ios::bin or no? For example: filename: a.bin number of...
5
by: Benjamin de Waal | last post by:
Hey all, I'm trying to figure out how to directly write to a device in Windows. Basically, what I'm wanting to do is create an image of a device (specifically, a CompactFlash card that uses a...
12
by: jcrouse | last post by:
I am using the following code to write to an XML file myXmlTextWriter.Formatting = System.Xml.Formatting.Indente myXmlTextWriter.WriteStartElement("CPViewer"...
6
by: arne.muller | last post by:
Hello, I've come across some problems reading strucutres from binary files. Basically I've some strutures typedef struct { int i; double x; int n; double *mz;
12
by: SAL | last post by:
Hello, Is it possible to read a CSV from the Client, and bind my Datagrid to the data in the CSV file without uploading the file to the Server first? I have tried and in Debug mode on my...
1
by: ChrisFrohlich | last post by:
ASP.NET 2.0 with Text DataTypes: I've got a similar question going in the SQL group, but I was wondering if anyone has successfully implemented reading/writing character data from a Text datatype...
12
by: glennanthonyb | last post by:
Hi The company I work for has finally woken up to data security on our field laptops. I'm writing something in C# that will allow remote deletion of sensitive data and I don't believe...
3
by: =?Utf-8?B?UmF5IE1pdGNoZWxs?= | last post by:
Hello, I'm trying to write an array of structures named myStructArray to a binary file and later on read it back. Although I could complete the entire project in C in about 2 minutes, I...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shllpp 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.