Allocating heap to unfreeable function static variables.

[charlie]

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
static int *temp_stuff = NULL;
static int ntemp_stuffs = 0;

if(nstuffs ntemp_stuffs) {
ntemp_stuffs = nstuffs;
temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
}

/* Use the temp stuffs for something */
memcpy(temp_stuff, stuff, nstuffs * sizeof(int));
}

Is this a horrible thing to do?

Cheers,
Charlie

[Richard]

charlie <ch*************@gmail.comwrites:

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
static int *temp_stuff = NULL;
static int ntemp_stuffs = 0;

if(nstuffs ntemp_stuffs) {
ntemp_stuffs = nstuffs;
temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
}

/* Use the temp stuffs for something */
memcpy(temp_stuff, stuff, nstuffs * sizeof(int));
}

Is this a horrible thing to do?

It is certainly horrible to name static variables "temp" anything.

But to answer your question this is simply horrible. You have no concept
of ownership and this could and will lead to horrific bugs later down
the line where someone else calls fn() without paying proper heed to
already "owns" the contents it pretends to guard.

[s0suk3]

On Sep 12, 10:58*am, charlie <ch*************@gmail.comwrote:

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
* * *static int *temp_stuff = NULL;
* * *static int ntemp_stuffs = 0;

* * *if(nstuffs ntemp_stuffs) {
* * * * *ntemp_stuffs = nstuffs;
* * * * *temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
* * *}

* * */* Use the temp stuffs for something */
* * *memcpy(temp_stuff, stuff, nstuffs * sizeof(int));

}

Is this a horrible thing to do?

It's rather inconvenient and unintuitive, not to mention it isn't
thread-safe. Also, you didn't check whether realloc() succeeded, and
if you do check, and if it fails, you can only return without doing
anything because your function returns void. The solution is to use a
VLA (variable-length array).

Sebastian

[jacob navia]

charlie wrote:

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
static int *temp_stuff = NULL;
static int ntemp_stuffs = 0;

if(nstuffs ntemp_stuffs) {
ntemp_stuffs = nstuffs;
temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
}

/* Use the temp stuffs for something */
memcpy(temp_stuff, stuff, nstuffs * sizeof(int));
}

Is this a horrible thing to do?

Cheers,
Charlie

It depends on your discipline when using it.

I had a similar problem in my IDE for C programming
when I was programming under windows 3.1 (16 bits).

To avoid mallocing each time I needed some buffer
and to save stack space that was at a premimum
I allocated at startup a global "buf" of 35K

I used it for temporary strings before I stuffed
them into list boxes, to pass data between functions
(saving stack), and many othere terribly "clever"
uses.

Then, windows 95 came along, and memory and stack weren't
THAT important, even though still important. The IDE gew,
I added a debugger, and "buf" started producing strange
problems. A procedure would use buf to pass data to another
but some routine in that call tree would ALSO use buf

This produced a lot of bugs, and led me to a big rewrite
eliminating buf from the software (as far as I could)

The rule is:

"Never assume buf is alive through a function call"
"Only use buf in a SINGLE THREAD", maybe the main thread.
It is a horrible thing to do?

Under Vista 64 YES.
Under windows 3.1 NO
Under a small embedded system? Maybe, it depends the security
requirements of the system.
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

[jameskuyper]

charlie wrote:

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

If you're worried about it being unfreeable, just provide a mechanism
to free it:

void fn(int *stuff, int nstuffs)
{
static int *temp_stuff = NULL;
static int ntemp_stuffs = 0;

if(stuff == NULL)
{ // you should be checking for this anyway
free(temp_stuff);
temp_stuff = NULL;
ntemp_stuffs = 0;
}
else

if(nstuffs ntemp_stuffs) {
ntemp_stuffs = nstuffs;
temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
}

/* Use the temp stuffs for something */
memcpy(temp_stuff, stuff, nstuffs * sizeof(int));
}

[charlie]

On Sep 12, 12:20*pm, [e-mail] wrote:

On Sep 12, 10:58*am, charlie <[e-mail]:

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
* * *static int *temp_stuff = NULL;
* * *static int ntemp_stuffs = 0;

* * *if(nstuffs ntemp_stuffs) {
* * * * *ntemp_stuffs = nstuffs;
* * * * *temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
* * *}

* * */* Use the temp stuffs for something */
* * *memcpy(temp_stuff, stuff, nstuffs * sizeof(int));

}

Is this a horrible thing to do?

It's rather inconvenient and unintuitive, not to mention it isn't
thread-safe. Also, you didn't check whether realloc() succeeded, and
if you do check, and if it fails, you can only return without doing
anything because your function returns void. The solution is to use a
VLA (variable-length array).

Sebastian

Thank-you, you make great arguments. Thankfully I can mandate c99 and
just use VLAs and yes, the code would have to be thread safe.

C

[Keith Thompson]

charlie <ch*************@gmail.comwrites:

On Sep 12, 12:20*pm, [e-mail] wrote:

>On Sep 12, 10:58*am, charlie <[e-mail]:

I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
* * *static int *temp_stuff = NULL;
* * *static int ntemp_stuffs = 0;

* * *if(nstuffs ntemp_stuffs) {
* * * * *ntemp_stuffs = nstuffs;
* * * * *temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));

The value of temp_stuff at this point is guaranteed to be NULL. Why
use realloc rather than malloc? (Perhaps realloc makes sense in your
actual code.)

* * *}

* * */* Use the temp stuffs for something */
* * *memcpy(temp_stuff, stuff, nstuffs * sizeof(int));

}

Is this a horrible thing to do?

It's rather inconvenient and unintuitive, not to mention it isn't
thread-safe. Also, you didn't check whether realloc() succeeded, and
if you do check, and if it fails, you can only return without doing
anything because your function returns void. The solution is to use a
VLA (variable-length array).

Sebastian

Thank-you, you make great arguments. Thankfully I can mandate c99 and
just use VLAs and yes, the code would have to be thread safe.

A few problems with VLAs:

There's no standard way to detect an allocation failure; if there's
not enough memory to create a VLA, the behavior is undefined. (The
same problem occurs with ordinary fixed-size arrays, but it's probably
more likely to show up with VLAs.)

There's no equivalent of realloc for VLAs. The array is allocated
once when it's created, and deallocated when it leaves its scope. The
size can't change during its lifetime.

At least one major implementation, gcc, has a "broken" implementation
of VLAs. I don't know just what the problem is (probably some
misbehavior in some relatively obscure cases), but if you're using gcc
you'll need to watch out for that.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[s0suk3]

On Sep 12, 2:58*pm, Keith Thompson <ks***@mib.orgwrote:

charlie <ch*************@gmail.comwrites:

>On Sep 12, 12:20*pm, [e-mail] wrote:

>>On Sep 12, 10:58*am, charlie <[e-mail]:
I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

>void fn(int *stuff, int nstuffs)
{
* * *static int *temp_stuff = NULL;
* * *static int ntemp_stuffs = 0;

>* * *if(nstuffs ntemp_stuffs) {
* * * * *ntemp_stuffs = nstuffs;
* * * * *temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));

The value of temp_stuff at this point is guaranteed to be NULL. *Why
use realloc rather than malloc? *(Perhaps realloc makes sense in your
actual code.)

>* * *}

>* * */* Use the temp stuffs for something */
* * *memcpy(temp_stuff, stuff, nstuffs * sizeof(int));

>}

>Is this a horrible thing to do?

>>It's rather inconvenient and unintuitive, not to mention it isn't
thread-safe. Also, you didn't check whether realloc() succeeded, and
if you do check, and if it fails, you can only return without doing
anything because your function returns void. The solution is to use a
VLA (variable-length array).

>>Sebastian

>Thank-you, you make great arguments. Thankfully I can mandate c99 and
just use VLAs and yes, the code would have to be thread safe.

A few problems with VLAs:

There's no standard way to detect an allocation failure; if there's
not enough memory to create a VLA, the behavior is undefined. *(The
same problem occurs with ordinary fixed-size arrays, but it's probably
more likely to show up with VLAs.)

Why? The compiler generates instructions to allocate the given size. I
would expect that the probability of failure depends on the size of
the array, but you can check that before allocating the VLA anyway.

There's no equivalent of realloc for VLAs. *The array is allocated
once when it's created, and deallocated when it leaves its scope. *The
size can't change during its lifetime.

....which is precisely what the OP needed. Hence the "temp" variables.
malloc() and realloc() are only inconvenient here.

Sebastian

[Richard Harter]

On Fri, 12 Sep 2008 08:58:29 -0700 (PDT), charlie
<ch*************@gmail.comwrote:

>I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
static int *temp_stuff = NULL;
static int ntemp_stuffs = 0;

if(nstuffs ntemp_stuffs) {
ntemp_stuffs = nstuffs;
temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));
}

/* Use the temp stuffs for something */
memcpy(temp_stuff, stuff, nstuffs * sizeof(int));
}

Is this a horrible thing to do?

The answer is, Maybe.

There are some faults that others have noted, e.g., your code
doesn't handle allocation failures and it doesn't provide for
freeing the space. The faults are easily fixed; fix them.

The up side is that in some applications you can get a
significant performance boost with this device. I have seen
applications where 40% of the execution time was spent in
allocating and freeing scratch space.

There are some caveats that others have noted, basically that it
requires usage discipline and that it is not thread safe. Also
it doesn't work with recursion.

Thread safety, recursion don't mix very well with static storage.

Your usage is okay because it is providing scratch space that is
not used outside the function -- if fn does not implicitly or
explicitly call itself.

I have used that device extensively in some code in the past. I
wouldn't do it now. Rather, I use special purpose allocators
that supply temporary space without the overhead of malloc/free.
Calling a special purpose allocator isn't quite as time efficient
as inline code but it is safer and cleaner. YMMV.

Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[Richard Harter]

On Fri, 12 Sep 2008 12:58:17 -0700, Keith Thompson
<ks***@mib.orgwrote:

>charlie <ch*************@gmail.comwrites:

>On Sep 12, 12:20*pm, [e-mail] wrote:

>>On Sep 12, 10:58*am, charlie <[e-mail]:
I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
* * *static int *temp_stuff = NULL;
* * *static int ntemp_stuffs = 0;

* * *if(nstuffs ntemp_stuffs) {
* * * * *ntemp_stuffs = nstuffs;
* * * * *temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));

The value of temp_stuff at this point is guaranteed to be NULL. Why
use realloc rather than malloc? (Perhaps realloc makes sense in your
actual code.)

???? temp_stuff is static; it will only be NULL on the first
call; thereafter it will be whatever it was at the end of the
previous invocation. That's the point of this, er, hack - the
space only has to be reallocated if the amount of space used last
time isn't enough.

>

>* * *}

* * */* Use the temp stuffs for something */
* * *memcpy(temp_stuff, stuff, nstuffs * sizeof(int));

}

Is this a horrible thing to do?

It's rather inconvenient and unintuitive, not to mention it isn't
thread-safe. Also, you didn't check whether realloc() succeeded, and
if you do check, and if it fails, you can only return without doing
anything because your function returns void. The solution is to use a
VLA (variable-length array).

Sebastian

Thank-you, you make great arguments. Thankfully I can mandate c99 and
just use VLAs and yes, the code would have to be thread safe.

A few problems with VLAs:

There's no standard way to detect an allocation failure; if there's
not enough memory to create a VLA, the behavior is undefined. (The
same problem occurs with ordinary fixed-size arrays, but it's probably
more likely to show up with VLAs.)

There's no equivalent of realloc for VLAs. The array is allocated
once when it's created, and deallocated when it leaves its scope. The
size can't change during its lifetime.

At least one major implementation, gcc, has a "broken" implementation
of VLAs. I don't know just what the problem is (probably some
misbehavior in some relatively obscure cases), but if you're using gcc
you'll need to watch out for that.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[Keith Thompson]

s0****@gmail.com writes:

On Sep 12, 2:58*pm, Keith Thompson <ks***@mib.orgwrote:

[...]

>A few problems with VLAs:

There's no standard way to detect an allocation failure; if there's
not enough memory to create a VLA, the behavior is undefined. *(The
same problem occurs with ordinary fixed-size arrays, but it's probably
more likely to show up with VLAs.)

Why? The compiler generates instructions to allocate the given size. I
would expect that the probability of failure depends on the size of
the array, but you can check that before allocating the VLA anyway.

For fixed-size arrays, in a typical stack-based implementation, the
compiler just figures out the (constant) size of the function's local
data and generates code to move the stack pointer by that amount on
function entry. (For other schemes, something similar happens.)

For a VLA, the memory typically has to be allocated in a separate
step, using a non-constant size.

Also, it seems likely that VLAs are more likely to be very large than
fixed-size arrays. I have no firm basis for that assumption; it's
just a hunch.

When you say you can check the size before allocating the VLA, do you
mean that the compiler-generated code can do this, or that the
programmer can do it? If you're talking about the compiler, it can
certainly check whether the allocation will succeed, but the language
says nothing about what it should do if the check fails, and provides
no way for the program to handle such a failure. If you're talking
about the programmer, there's no portable way to determine whehter the
allocation is going to succeed.

By contrast, malloc() returns NULL on failure; the program can detect
this and take some alternative action.

>There's no equivalent of realloc for VLAs. *The array is allocated
once when it's created, and deallocated when it leaves its scope. *The
size can't change during its lifetime.

...which is precisely what the OP needed. Hence the "temp" variables.
malloc() and realloc() are only inconvenient here.

Perhaps you understood his code better than I did.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[James Kuyper]

s0****@gmail.com wrote:

On Sep 12, 2:58 pm, Keith Thompson <ks***@mib.orgwrote:

....

>There's no standard way to detect an allocation failure; if there's
not enough memory to create a VLA, the behavior is undefined. (The
same problem occurs with ordinary fixed-size arrays, but it's probably
more likely to show up with VLAs.)

Why? The compiler generates instructions to allocate the given size. I
would expect that the probability of failure depends on the size of
the array, but you can check that before allocating the VLA anyway.

As a practical matter, VLAs are likely to be very long than fixed-sized
arrays. The availability of VLAs encourages you to define automatic
arrays of unknown length, that in particular cases might be much longer
than you would ordinarily be willing to allocate automatically.

[Keith Thompson]

cr*@tiac.net (Richard Harter) writes:

On Fri, 12 Sep 2008 12:58:17 -0700, Keith Thompson
<ks***@mib.orgwrote:

>>charlie <ch*************@gmail.comwrites:

>>On Sep 12, 12:20*pm, [e-mail] wrote:
On Sep 12, 10:58*am, charlie <[e-mail]:
I came up with this idiom for cases where a function needs a variable
amount of memory for it's temporary storage so as to avoid constantly
mallocing. It makes me feel a little uncomfortable to have unfreeable
(but still technically reachable) memory hanging around but it is, I
think still a useful thing to do:

void fn(int *stuff, int nstuffs)
{
* * *static int *temp_stuff = NULL;
* * *static int ntemp_stuffs = 0;

* * *if(nstuffs ntemp_stuffs) {
* * * * *ntemp_stuffs = nstuffs;
* * * * *temp_stuff = realloc(temp_stuff, ntemp_stuffs * sizeof(int));

The value of temp_stuff at this point is guaranteed to be NULL. Why
use realloc rather than malloc? (Perhaps realloc makes sense in your
actual code.)

???? temp_stuff is static; it will only be NULL on the first
call; thereafter it will be whatever it was at the end of the
previous invocation. That's the point of this, er, hack - the
space only has to be reallocated if the amount of space used last
time isn't enough.

You're quite right; I wasn't paying attention. (I noticed my mistake
about 10 seconds before reading your article.)

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Ron Ford]

On Fri, 12 Sep 2008 14:47:35 -0700, Keith Thompson posted:

>???? temp_stuff is static; it will only be NULL on the first
call; thereafter it will be whatever it was at the end of the
previous invocation. That's the point of this, er, hack - the
space only has to be reallocated if the amount of space used last
time isn't enough.

You're quite right; I wasn't paying attention. (I noticed my mistake
about 10 seconds before reading your article.)

I think memory management is the place where I've seen more mistakes from
usually careful persons than any other in C. It's a little disheartening
for the rest of us. It's almost like you need to have Heathfield on
speed-dial if you want to realloc.
--
Don't overestimate the decency of the human race.
H. L. Mencken