Hello,
I've often found that I am writing little scripts at the interpretor to
read a text file, perform some conversion, and then write the converted
data back out to a file. I normally accomplish the above task by
reading the lines of the entire file into a list, preforming some
function to that list and then writing the list back out. I want to
write a generic function/module that will free me from repeatedly
typing the same thing, perhaps convertFile(filename, covertFunc) and I
was wondering what would be a more optimal solution for writing the
data,
fout = open('somefile','w')
for line in convertedData:
fout.write("%s\n" % line)
fout.close()
-- or --
fout = open('somefile','w')
fout.write("%s" % '\n'.join(convertedData))
fout.close()
.... or maybe some hybrid of the two which writes chunks of the
convertedData list out in one shot ...
An issue that I'm probably most concerned with is scalabitiy, what if
the file was huge, like some sort of log file. As well, I know from
'import this' that there should only be one obvious way to do something
.... however to me it's not so obvious :(
Any suggestions,
Colin 6 1274
cdecarlo wrote:
>
fout = open('somefile','w')
for line in convertedData:
fout.write("%s\n" % line)
fout.close()
-- or --
fout = open('somefile','w')
fout.write("%s" % '\n'.join(convertedData))
fout.close()
I'd go for something like...
fout = open('somefile','w')
fout.writelines( line+"\n" for line in convertedData )
fout.close()
Although your first solution should perform about the same.
If you have 2.5, you may prefer this...
with open('somefile','w') as fout:
fout.writelines( line+"\n" for line in convertedData )
... or maybe some hybrid of the two which writes chunks of the
convertedData list out in one shot ...
The OS should buffer it for you.
Will McGugan
fout = open('somefile','w')
for line in convertedData:
fout.write("%s\n" % line)
fout.close()
-- or --
fout = open('somefile','w')
fout.write("%s" % '\n'.join(convertedData))
fout.close()
I shouldn't think it matters too much which of these you use - time
them and see what happens.
An issue that I'm probably most concerned with is scalabitiy, what if
the file was huge, like some sort of log file.
Sucking in the entire file into a list won't scale well, as a huge log
file could quickly eat all of your available memory. You'd be better
off processing each line as you go, and writing it to a temp file,
renaming the temp file once you have finished. Something like:
in_f = "access.log"
out_f = "access.log.tmp"
infile = open(in_f)
outfile = open(out_f)
for line in infile:
outfile.write(process(line))
infile.close()
outfile.close()
os.remove(in_f)
os.rename(out_f, in_f)
(Not tested, but you get the idea...)
Whoops:
outfile = open(out_f)
outfile = open(out_f, 'w')
may be better ;-)
>>>>cdecarlo <cd******@gmail.comwrites:
fout = open('somefile','w')
for line in convertedData:
fout.write("%s\n" % line)
fout.close()
-- or --
fout = open('somefile','w')
fout.write("%s" % '\n'.join(convertedData))
fout.close()
... or maybe some hybrid of the two which writes chunks of the
convertedData list out in one shot ...
The second option would be definitely faster.
An issue that I'm probably most concerned with is scalabitiy, what if
the file was huge, like some sort of log file.
Considering that you've already read in the whole file into a list, it's too
late to worry about scalability when writing out :-). Have you considered
the fileinput module?
Ganesan
--
Ganesan Rajagopal
See also this recent discussion on techniques for making sure you get
the file closed, even if there is an exception. http://tinyurl.com/fryrv
rd
cdecarlo wrote:
Hello,
I've often found that I am writing little scripts at the interpretor to
read a text file, perform some conversion, and then write the converted
data back out to a file. I normally accomplish the above task by
<snip>
>
Any suggestions,
Colin
You should check out the fileinput module.
HTH,
~Simon This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Rob Somers |
last post by:
Hey all
I am writing a program to keep track of expenses and so on - it is not
a school project, I am learning C as a hobby - At any rate, I am new
to structs and reading and writing to files,...
|
by: Greg |
last post by:
I am continuing writing my simple file system and I have run across
the following problem.
I am writing a superblock to a certain frame in the file as follows:
(&sb is the address of a...
|
by: Paul Kuebler |
last post by:
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????...
|
by: serge calderara |
last post by:
Dear all,
I need to build a web application which will contains articles (long or short)
I was wondering on what is the correct way to retrive those article on web
page.
In orther words, when...
|
by: Tomás |
last post by:
Up until lately I've been writing all mad kinds of code for accomplishing
things, but lately I've decided to lean more toward the whole readability,
etc. side of things.
I have a command line...
|
by: true911m |
last post by:
This is a simple walkthrough to get PyInstaller up and running.
I decided to give PI a try, because it claims to be more selective about what it bundles into its executable files by default, and...
|
by: its me Rock |
last post by:
Hi All,
How to load bulk of records from one file to another file.Is there any memory allocation techiques available.Need suggestion in that....
Eg:
just reading the data from the...
|
by: theronnightstar |
last post by:
I am having a problem with a small bit of code. I can't figure out why
it won't work. Here:
//code:
#include <iostream>
#include <fstream>
#include <ctype.h>
#include <string>
|
by: jerry |
last post by:
i have written a simple phonebook program,i'll show you some of the
codes,the program's head file is member.h . i suppose the head file
works well.so i don't post it. here's the clips of main...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |