473,394 Members | 1,932 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Need advice on reading contents of a file into memory

f = open(someFilePath, "rb")
content = []
for data in content.read()
content.append(data)
fullContent = "".join(content)

Is there a more efficient way of doing this? I'll be running this
operation on 10,000+ files where each file is an image file with size
50k-100k

Mar 15 '06 #1
3 1163
Em Qua, 2006-03-15 Ã*s 13:49 -0800, vinjvinj escreveu:
f = open(someFilePath, "rb")
content = []
for data in content.read()
content.append(data)
fullContent = "".join(content)

Is there a more efficient way of doing this? I'll be running this
operation on 10,000+ files where each file is an image file with size
50k-100k


If you really need everything in memory, why not just...

fullContent = open(someFilePath, "rb").read()

....?

Mar 15 '06 #2
"vinjvinj" wrote:
f = open(someFilePath, "rb")
content = []
for data in content.read()
content.append(data)
fullContent = "".join(content)

Is there a more efficient way of doing this?


read reads until end of file, so unless the source is something unusual,
a plain

fullContent = content.read()

should be good enough (not that it matters much; that join will be in
no-op, and the list/append overhead is marginal compared to the time
required to get the data from disk)

</F>

Mar 15 '06 #3
Thanks. read() did not work when I opened the file with:

f = open(someFilePath)

But after changing to f = open(someFilePath, "rb") the read() works
fine.

VJ

Mar 15 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Markus Hofmann | last post by:
Hi @ all hope someone can help me as my PC is ready to go through the window...here it is: I have a text file with approximately 150,000 lines of the following format: ...
20
by: sahukar praveen | last post by:
Hello, I have a question. I try to print a ascii file in reverse order( bottom-top). Here is the logic. 1. Go to the botton of the file fseek(). move one character back to avoid the EOF. 2....
50
by: Michael Mair | last post by:
Cheerio, I would appreciate opinions on the following: Given the task to read a _complete_ text file into a string: What is the "best" way to do it? Handling the buffer is not the problem...
8
by: Andrew Robert | last post by:
Hi Everyone. I tried the following to get input into optionparser from either a file or command line. The code below detects the passed file argument and prints the file contents but the...
66
by: genestarwing | last post by:
QUESTION: Write a program that opens and read a text file and records how many times each word occurs in the file. Use a binary search tree modified to store both a word and the number of times it...
15
by: Gan Quan | last post by:
I'm writing a c++ program that has many (100+) threads read/write files simultaneously. It works well if not considering the efficiency. The file i/o seems to be the bottleneck. This is my code...
16
by: Jm.GlezdeRueda | last post by:
Hi all, Im trying to read a 24bit bmp with fread, and i have some problems.. I want to read the whole structure in one time, but i dont know why, it only reads the first member well.. I have...
7
by: jvdb | last post by:
Hi all, I need some help on the following issue. I can't seem to solve it. I have a binary (pcl) file. In this file i want to search for specific codes (like <0C>). I have tried to solve it...
3
by: =?Utf-8?B?SmF5IFZpbnRvbg==?= | last post by:
I see general messages about how to learn .NET but I have an immediate requirement to ramp up my old skills very quickly. Can anyone recommend the FASTEST way for me to get almost-competent in...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.