473,385 Members | 1,409 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Map and memory issues

Hi all,

I am writing to a map in a while loop and running into some issues.
Here is a pseudo code --

/* Standard headers included from namespace std*/

int main () {

string line;
size_t lines_written = 0;
map<Myobj, stringmymap;

try {

while (input file is being read) {

Myobj *obj;

...(read line)..
/* Construct obj on dynamic store based on the line read (if not
already created, else reuse)*/
itr = mymap.find(obj);
if (itr == mymap.end()) {
mymap.insert(std::make_pair(obj, line));
lines_written += line.length();
} else {
/* Append current record to existing value */
mymap[obj] += line;
lines_written += mymap[obj].length();
}
if (lines_written BIG_VALUE) {
/* Write the map using the obj objects to some files (I get files
sorted by obj) */
...
/* Clear the map */
mymap.clear();
lines_written = 0;
}

} /* while loop */

} catch(...) {

/* Code reaching here */
}
}

I see that my code goes into the exception block after a while when I
execute it for big files with BIG_VALUE set to something like
(1024*1024). I am trying to write the files when maps have grown
sufficiently large. I then try to erase everything for the next
iteration. It works fine for small files.

It seems that map insertions are failing and map.clear is not helping
me here. I am probably running out of memory. Is there any bit that I
am missing or that I could improve?

Thanks ..
P.S. - I feel that I could go the C way and handle memory management
on my own by continually writing to a buffer on stack and later
sorting it based on obj (the not so recommended way).


Dec 13 '07 #1
2 1302
On Wed, 12 Dec 2007 18:20:25 -0800 (PST) in comp.lang.c++, Kelvin Moss <km**********@yahoo.comwrote,
>It seems that map insertions are failing and map.clear is not helping
me here. I am probably running out of memory. Is there any bit that I
am missing or that I could improve?
Did you call delete for every object you allocated with new?
Dec 13 '07 #2
Kelvin Moss <km**********@yahoo.comwrote:
Hi all,

I am writing to a map in a while loop and running into some issues.
Here is a pseudo code --

/* Standard headers included from namespace std*/

int main () {

string line;
size_t lines_written = 0;
map<Myobj, stringmymap;

try {

while (input file is being read) {

Myobj *obj;

..(read line)..
/* Construct obj on dynamic store based on the line read (if not
already created, else reuse)*/
itr = mymap.find(obj);
if (itr == mymap.end()) {
mymap.insert(std::make_pair(obj, line));
lines_written += line.length();
} else {
/* Append current record to existing value */
mymap[obj] += line;
lines_written += mymap[obj].length();
}
if (lines_written BIG_VALUE) {
/* Write the map using the obj objects to some files (I get files
sorted by obj) */
...
/* Clear the map */
mymap.clear();
lines_written = 0;
}

} /* while loop */

} catch(...) {

/* Code reaching here */
}
}

I see that my code goes into the exception block after a while when I
execute it for big files with BIG_VALUE set to something like
(1024*1024). I am trying to write the files when maps have grown
sufficiently large. I then try to erase everything for the next
iteration. It works fine for small files.

It seems that map insertions are failing and map.clear is not helping
me here. I am probably running out of memory. Is there any bit that I
am missing or that I could improve?
I agree with David Harmon, are probably newing objects without deleting
them.

Frankly though, we are all just guessing because your code as written is
inconsistent as to what is supposed to be stored in the map. (parts of
the above are written as if map stores Myobjs, while other parts are
written as if it stores Myobj*s.
Dec 13 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

18
by: Tron Thomas | last post by:
Given the following information about memory management in C++: ----- The c-runtime dynamic memory manager (and most other commercial memory managers) has issues with fragmentation similar to a...
29
by: keredil | last post by:
Hi, Will the memory allocated by malloc get released when program exits? I guess it will since when the program exits, the OS will free all the memory (global, stack, heap) used by this...
25
by: Zeng | last post by:
I finally narrowed down my code to this situation, quite a few (not all) of my CMyClass objects got hold up after each run of this function via the simple webpage that shows NumberEd editbox. My...
4
by: Simon Johnson | last post by:
Dear All, I have decided to take the big plunge and drop the Microsoft platform and use Mod_Python and Apache in it's place. I've never used Linux before this project so it's a really big...
14
by: Joe | last post by:
I'm trying to track down where objects are referenced because I want to clean up the memory when I'm done with them. I thought I found all references but it doesn't seem that way because calling...
2
by: ruca | last post by:
Hi gurus, Can anyone tell me please when I really must setup the value of memory that aspnet_wp or w3wp must use in IIS server machine? What I mean is that we have clients that have diferent...
5
by: Frank Rizzo | last post by:
Hello, I have a very frustrating issue I've been working on. I have a routine that does the following: 1. Load a large (declared local to the function) DataSet from the database. It contains...
3
by: Sam Samson | last post by:
Can any-one recommend some good sites detailing how to code a simple memory profiler? How do they "look under the hood"? I built an uber simple service that tracks the process' memory but I need...
5
by: Max2006 | last post by:
Hi, What is the limit for memory that a .NET process or AppDomain can use? Thank you, Max
22
by: Peter | last post by:
I am using VS2008. I have a Windows Service application which creates Crystal Reports. This is a multi theaded application which can run several reports at one time. My problem - there is a...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.