473,395 Members | 1,949 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,395 software developers and data experts.

performance, boost mmap files, fstream

8
Hi,

MSVC 9
Boost 1.38

I have an existing program that reads text files a line at a time using the std::ifstream and istream& getline ( istream& is, string& str );

Typical file size is ~40mb

To increase the file reading performance i used boost::iostreams::mapped_file_source and tokenized the memory buffer using boost::algorithm::split with "\n" as delimiter

Before implementing this change in the main program i wrote this logic in a small test program and i found that using the boost memory mapped file is almost 50% faster than using std::fstream

However when i implemented the same logic in my main program i did not get the same performance improvement. [Note: I have taken care to ensure that no other logic in my main program is negating any performance gain achieved by using the memory mapped files]

/O2 and /GL are enabled

Size of small test program: 180K
Size of main program: 680K

I am using the /MT option to link to the boost libs

Can someone thow some light onto this behaviour please.

Thanks

Regards,
Ameya
Jun 17 '09 #1
1 6223
Banfa
9,065 Expert Mod 8TB
Not really except to say you rarely get best case potential improvement in a real life program.
Jun 17 '09 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

7
by: Lee | last post by:
Hi, I'm a stream virgin and am attempting to output strings to a file. My approach is to write the string initially to a 'stringstream' and only when complete write the stringstream to the file...
4
by: Fabiano Sidler | last post by:
Hi folks! I created an mmap object like so: --- snip --- from mmap import mmap,MAP_ANONYMOUS,MAP_PRIVATE fl = file('/dev/zero','rw') mm = mmap(fl.fileno(), 1, MAP_PRIVATE|MAP_ANONYMOUS) ---...
2
by: smith4894 | last post by:
{ not sure you're aware of that but there are the newsgroups for all major operating systems. you might want to try asking in the forum 'comp.os.linux.development.apps', since memory-mapped files...
26
by: myeates | last post by:
Hi Anyone ever done this? It looks like Python2.4 won't take a length arg Mathew
1
by: Yahooooooooo | last post by:
Just practicing BOOST regular expressions....giving errors... -- wanted to replace SPACE with NULL. #include <iostream> #include <fstream> #include <sstream> #include <string> #include...
1
by: sam_cit | last post by:
Hi Everyone, I searched for mmap() and i found the following in wikipedia, 'Anonymous mappings are mappings of physical RAM to virtual memory. This is similar to malloc, and is used in some...
1
by: James T. Dennis | last post by:
I've been thinking about the Python mmap module quite a bit during the last couple of days. Sadly most of it has just been thinking ... and reading pages from Google searches ... and very little...
1
by: koara | last post by:
Hello all, i am using the mmap module (python2.4) to access contents of a file. My question regards the relative performance of mmap.seek() vs mmap.tell(). I have a generator that returns...
6
by: George2 | last post by:
Hello everyone, I have several physical file and I want to use file map (MapViewOfFileEx) to map the file into memory to improve performance. Each file is about several hundred M bytes. All...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.