468,512 Members | 971 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,512 developers. It's quick & easy.

hash_map in stl c++ memory issues

2
Hi!

I have written code using STL hash_map in c++ and am running into memory issues. I have two kinds of hash_maps:
A <int,int> which stores ~5million key-value pairs, and needs to be accessed often to store in memory, its not updated during the program.
B <int,hash_map<int,float>> which is also ~5m key-value pairs, but I can read this in parts from the disk. The inner hash_map has about 20 elements.

In my program, I read in A and B1(first 100K key-values), I perform the required computation, and say B1.clear(), and read the next set of 100k elements.

1) The program exits with an std::bad_alloc error. How can I fix this?

2) What hash_fns would be good for this large data?

3) It seems that clear does not free the memory. What can I do? When I say B1.clear(), will it free the inner hash_map also?

4) How can I find out what is using up the memory?

I am a novice to stl programing, any help would be appreciated.

Thanks!
Mar 18 '08 #1
1 4170
weaknessforcats
9,207 Expert Mod 8TB
1) The program exits with an std::bad_alloc error. How can I fix this?

2) What hash_fns would be good for this large data?

3) It seems that clear does not free the memory. What can I do? When I say B1.clear(), will it free the inner hash_map also?

4) How can I find out what is using up the memory?
1) You have run out of heap memory. Check your system and see what the heap limits are. That is, your program is in a process address space and you have to stay within that range of addresses. Becuae you have a 2GB machine does not mean you cna have a 2GB heap.

2) I have no idea. You need to research here. Check out The Art of Computer Programming by Donald Knuth Volume 2 - Sorting and Searching.

3) None of the STL containers gives up memory once it it acquired. This is by design. Allocating memory is slow operation and the STL is optimized for speed so once your container gloms onto memory, it keeps it. The B1.clear() just erases all of the elements in the container but does not reduce the memory size of the container. You can force the container to give up this memory by swapping it with a container that was created empty.

4) See (3) above.
Mar 18 '08 #2

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

5 posts views Thread by Charles Herman | last post: by
2 posts views Thread by gibffe | last post: by
4 posts views Thread by Dymus | last post: by
4 posts views Thread by lokki | last post: by
4 posts views Thread by newbie | last post: by
6 posts views Thread by SimpleCode | last post: by
4 posts views Thread by James Kanze | last post: by
reply views Thread by NPC403 | last post: by
1 post views Thread by fmendoza | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.