By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,629 Members | 1,222 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,629 IT Pros & Developers. It's quick & easy.

Controlling memory allocation

P: n/a
Let's say I have an OS that can't limit the amount of memory a process
is allowed to consume, but I want to run some Python code in a
simulated low memory environment...

Is there a way to tell Python to limit the size of it's heap? Can you
ask it to use a function other than malloc through the C API? Any
stunts you might recommend to achieve the same results?

Chad

Sep 4 '05 #1
Share this Question
Share on Google+
3 Replies


P: n/a
Hello.
I have done the thing you are requesting. You can easily create your
own memory pool and have Python to use it.

I added some rows to Pyconfig.h:

#ifdef malloc
#undef malloc
#endif /*malloc*/
#define malloc Python_malloc

Then I have some overhead in my own Python_malloc(size_t nBytes)

Sep 5 '05 #2

P: n/a
Good to know that works :)

Did you also do the same for free and realloc to keep up with your
accounting?

The more I thought about it, the more I realized that calls to other
library functions could cause address space expansion if that code
internally called malloc or mmap.

See, I can use the OS to limit stack and data segment growth, but on OS
X/Darwin, it seems malloc uses mmap instead of brk to get more memory,
so setting a ulimit on the data segment doesn't work, because
technically the data segment doesn't grow during a malloc. Setting a
ulimit on the RSS (resident set size) doesn't work either as that limit
seems to only limit the number of pages the process is allowed to touch
(keep resident in memory) and the address range is still allowed to
expand.

Chad

To**********@gmail.com wrote:
Hello.
I have done the thing you are requesting. You can easily create your
own memory pool and have Python to use it.

I added some rows to Pyconfig.h:

#ifdef malloc
#undef malloc
#endif /*malloc*/
#define malloc Python_malloc

Then I have some overhead in my own Python_malloc(size_t nBytes)


Sep 5 '05 #3

P: n/a
Yep!
I redirect Free, Realloc and Malloc. (calloc is never used by Python).

"The more I thoug..."
That could be a possible problem. I will look in to it today and try
and find out if that occurs and in that case how often.
//T

Sep 6 '05 #4

This discussion thread is closed

Replies have been disabled for this discussion.