hash_map in stl c++ memory issues

[mono12]

Hi!

I have written code using STL hash_map in c++ and am running into memory issues. I have two kinds of hash_maps:
A <int,int> which stores ~5million key-value pairs, and needs to be accessed often to store in memory, its not updated during the program.
B <int,hash_map<int,float>> which is also ~5m key-value pairs, but I can read this in parts from the disk. The inner hash_map has about 20 elements.

In my program, I read in A and B1(first 100K key-values), I perform the required computation, and say B1.clear(), and read the next set of 100k elements.

1) The program exits with an std::bad_alloc error. How can I fix this?

2) What hash_fns would be good for this large data?

3) It seems that clear does not free the memory. What can I do? When I say B1.clear(), will it free the inner hash_map also?

4) How can I find out what is using up the memory?

I am a novice to stl programing, any help would be appreciated.

Thanks!

[weaknessforcats]

1) The program exits with an std::bad_alloc error. How can I fix this?

2) What hash_fns would be good for this large data?

3) It seems that clear does not free the memory. What can I do? When I say B1.clear(), will it free the inner hash_map also?

4) How can I find out what is using up the memory?

1) You have run out of heap memory. Check your system and see what the heap limits are. That is, your program is in a process address space and you have to stay within that range of addresses. Becuae you have a 2GB machine does not mean you cna have a 2GB heap.

2) I have no idea. You need to research here. Check out The Art of Computer Programming by Donald Knuth Volume 2 - Sorting and Searching.

3) None of the STL containers gives up memory once it it acquired. This is by design. Allocating memory is slow operation and the STL is optimized for speed so once your container gloms onto memory, it keeps it. The B1.clear() just erases all of the elements in the container but does not reduce the memory size of the container. You can force the container to give up this memory by swapping it with a container that was created empty.

4) See (3) above.