473,836 Members | 1,586 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Can I Trust Pointer Arithmetic In Re-Allocated Memory?

Bear with me, as I am not a "profession al" programmer, but I was
working on part of program that reads parts of four text files into
a buffer which I re-allocate the size as I read each file. I read some
of the items from the bottom up of the buffer, and some from the
top down, moving the bottom items back to the new re-allocated
bottom on every file read.

Then when I've read all four files, I sort the top and bottom items
separately using qsort(), which takes a pointer to a list of items, and
write the two sorted lists to two new files.

Problem is, I worry that if I just supply a pointer to the first item
in the bottom list to qsort(), it might point out to bozo-land during
the sort because I thought that dynamically re-allocated memory
is not necessarily contiguous. So I've done a little two step where
I write the bottom list to another buffer to do the sorting and writing,
and everything works great, but I'm wondering if I'm wasting time
and worrying about nothing...after all, if I can't trust a pointer to an
arbitrary point in the list, how can I trust a pointer to the start of
the list?

Any light you can shed on how pointers are handled in dynamically
allocated memory would be interesting and helpful...thank s.

---
William Ernest Reid

Aug 11 '06
26 3071

Herbert Rosenau <os****@pc-rosenau.dewrote in message
news:wm******** *************** ****@JUPITER1.P C-ROSENAU.DE...
On Wed, 16 Aug 2006 01:01:06 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:

Anybody blames her/himself as s/he can. :-) You will learn that
mostenly when you tries to blame the compiler you blames yourself
instead.
I blame society.

As to your code, I'm not sure why you do the two-step
with "*p" and "*temp"...i t seems like all is required is to set
*p=NULL when declared, at least that's what I did, and
it worked just fine. Am I (again!) missing something?

p is the pointer holding the address of the memory block. As realloc
an fail (returning NULL) you needs another pointer to assign the
result of realloc until you knows that realloc returns (new) memory
address.

When realloc fails you needs to either work with the memory (p) you
have already or to cleanup (free(p). Overwriting p with NULL gives you
a memory leak as you lost the address of the memory you have laready
allocated.
Hmmm...if this is the case this should be in the NON-documentation.
Actually, I guess it kind of is, but after a quick read of this I always
thought realloc() freed the original memory block and returned NULL
if it couldn't reallocate the new block:

....

Syntax

#include <stdlib.h>
void *realloc(void *block, size_t size);

....

If the block cannot be reallocated, realloc returns NULL.

If the value of size is 0, the memory block is freed and realloc returns
NULL.

---end of NON-documentation

I guess I got the last two conditions conflated in my mind; it just seemed
logical to me that if realloc() failed it would free the previous block.
It SEEMS like it should.

As an orthogonal point, what horrible things happen if you try
to free() a NULL pointer?
Another tip:

You should initialise any variable when defining it to be sure to fail
on that because you've not already assigned a known valid value. So
initialise a 0 (or a value you can easyly identify as invalid to data
and NULL to pointer. Then learn how to use a debugger and set a
breakpoint immediately before the fail occures, analyse the date found
there and then, when anything seems ok step a single step forward and
analyse again until the failture occures.
I'm not quite sure how this is functionally different from SOP using
a debugger; initialized or not, I can "mouse over" ALL the variables
after an exeption point or break point, so I don't see garbage at an
exception point I step back and put in a break, run it again, check
the values, step forward, etc.

---
William Ernest Reid

Aug 17 '06 #21
"Bill Reid" <ho********@hap pyhealthy.netwr ites:
[...]
As an orthogonal point, what horrible things happen if you try
to free() a NULL pointer?
Nothing; free(NULL) is guaranteed to be a no-op.

Either your system's documentation or any decent C textbook should
tell you this. Failing that, you can download a copy of the latest
draft of the C standard; search for "n1124.pdf" .

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 18 '06 #22
On Thu, 17 Aug 2006 23:54:41 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:
When realloc fails you needs to either work with the memory (p)
you
have already or to cleanup (free(p). Overwriting p with NULL gives you
a memory leak as you lost the address of the memory you have laready
allocated.
Hmmm...if this is the case this should be in the NON-documentation.
Actually, I guess it kind of is, but after a quick read of this I always
thought realloc() freed the original memory block and returned NULL
if it couldn't reallocate the new block:
Yes, but it does NOT free() the old block then. That block leaves
unchanged.
...

Syntax

#include <stdlib.h>
void *realloc(void *block, size_t size);

...

If the block cannot be reallocated, realloc returns NULL.

If the value of size is 0, the memory block is freed and realloc returns
NULL.

---end of NON-documentation

I guess I got the last two conditions conflated in my mind; it just seemed
logical to me that if realloc() failed it would free the previous block.
It SEEMS like it should.
But it does not so because you may need to continue your work with the
old block.
When you have no need for the old block you have to free() that
yourself, else you should free() it. In any case you needs its
address.
As an orthogonal point, what horrible things happen if you try
to free() a NULL pointer?
Nothing. free(NULL); works like a noop. Nothing occures. That is
guaranteed.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2 Deutsch ist da!
Aug 18 '06 #23

Herbert Rosenau <os****@pc-rosenau.dewrote in message
news:wm******** *************** ****@JUPITER1.P C-ROSENAU.DE...
On Thu, 17 Aug 2006 23:54:41 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:

I guess I got the last two conditions conflated in my mind; it just
seemed
logical to me that if realloc() failed it would free the previous block.
It SEEMS like it should.

But it does not so because you may need to continue your work with the
old block.
Sure, but right off-hand I can't think of single case where I would
actually want to do anything with "half a loaf". When memory
allocation fails in whole or in part, I just want to get out of there
as quickly as possible, maybe just break out of that particular
module operation, quite often just to exit the whole program.

I suspect that is true of the vast majority of programs out there.
So not automatically freeing the block seems to be a case of "the
needs of the few outweighing the needs of the many".
When you have no need for the old block you have to free() that
yourself, else you should free() it. In any case you needs its
address.
In this imperfect world, I guess so...what a friggin' hassle...
As an orthogonal point, what horrible things happen if you try
to free() a NULL pointer?

Nothing. free(NULL); works like a noop. Nothing occures. That is
guaranteed.
So I've kind of been wasting a little time with these types of
generalized pre-exit memory cleanup routines:

void free_time_serie s_mem(void) {
unsigned ts_idx;

for(ts_idx=0;ts _idx<TS_MAX;ts_ idx++) {
if(time_series[ts_idx]!=NULL) {
free(time_serie s[ts_idx]);
time_series[ts_idx]=NULL;
}
}

num_series=0;
}

Not only don't I need:
if(time_series[ts_idx]!=NULL)

What the hell is the point of:
time_series[ts_idx]=NULL;

??? Who knows...what the hell was I thinking...oh, wait, I WASN'T
thinking...

However, note the clever use of the "TS_MAX" define; NO "MAGIC
NUMBER" HERE FOR THIS BOY!!!

---
William Ernest Reid

Aug 18 '06 #24
"Bill Reid" <ho********@hap pyhealthy.netwr ites:
Herbert Rosenau <os****@pc-rosenau.dewrote in message
news:wm******** *************** ****@JUPITER1.P C-ROSENAU.DE...
>On Thu, 17 Aug 2006 23:54:41 UTC, "Bill Reid"
<ho********@ha ppyhealthy.netw rote:
>
I guess I got the last two conditions conflated in my mind; it just
seemed
logical to me that if realloc() failed it would free the previous block.
It SEEMS like it should.

But it does not so because you may need to continue your work with the
old block.

Sure, but right off-hand I can't think of single case where I would
actually want to do anything with "half a loaf". When memory
allocation fails in whole or in part, I just want to get out of there
as quickly as possible, maybe just break out of that particular
module operation, quite often just to exit the whole program.

I suspect that is true of the vast majority of programs out there.
So not automatically freeing the block seems to be a case of "the
needs of the few outweighing the needs of the many".
You can write a wrapper function for the standard realloc that
does what you want. It's not possible to wrap a function that
has your desired behavior to do what the standard realloc does.
--
"Your correction is 100% correct and 0% helpful. Well done!"
--Richard Heathfield
Aug 18 '06 #25
On Fri, 18 Aug 2006 23:27:03 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:
>
Herbert Rosenau <os****@pc-rosenau.dewrote in message
news:wm******** *************** ****@JUPITER1.P C-ROSENAU.DE...
On Thu, 17 Aug 2006 23:54:41 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:
>
I guess I got the last two conditions conflated in my mind; it just
seemed
logical to me that if realloc() failed it would free the previous block.
It SEEMS like it should.
But it does not so because you may need to continue your work with the
old block.

Sure, but right off-hand I can't think of single case where I would
actually want to do anything with "half a loaf". When memory
allocation fails in whole or in part, I just want to get out of there
as quickly as possible, maybe just break out of that particular
module operation, quite often just to exit the whole program.
.... and leaving the database in an undefined state. Maybe you needs a
lot of single steps on the incloplete data to do to unwind any action
you've already done with. So the data is needed by your program. Most
programs are more complex in data handling as you currently aware of.

So the best realloc can do for you to flag the error that it is out of
memory and leave data it arrived unchanged. Sometimes you gets a
chance to continue when you free()s some other data area.
Sometimes you'll split the aready occupied block into one or more
smaller ones, write older, smaller blocks out and reuse them.

There are lots of possibilities to continue even without - or with -
asking the user. Exit is seldom the choice in real programs. Often you
will have a solution for "out of memory" on a higher level.

exit() is good for ingenious programs but the real world is more
complex. So realloc gives you the chance for a real cleanup of any
kind you may need. It is simply a bug to overwrite a data area you
have already allocated with a null pointer. You have to cleanup.
I suspect that is true of the vast majority of programs out there.
So not automatically freeing the block seems to be a case of "the
needs of the few outweighing the needs of the many".
You not written a reals program using realloc. You will then quickly
revise your standpoint.
When you have no need for the old block you have to free() that
yourself, else you should free() it. In any case you needs its
address.
In this imperfect world, I guess so...what a friggin' hassle...
As an orthogonal point, what horrible things happen if you try
to free() a NULL pointer?
Nothing. free(NULL); works like a noop. Nothing occures. That is
guaranteed.
So I've kind of been wasting a little time with these types of
generalized pre-exit memory cleanup routines:

void free_time_serie s_mem(void) {
unsigned ts_idx;

for(ts_idx=0;ts _idx<TS_MAX;ts_ idx++) {
if(time_series[ts_idx]!=NULL) {
free(time_serie s[ts_idx]);
time_series[ts_idx]=NULL;
}
}

num_series=0;
}

Not only don't I need:
if(time_series[ts_idx]!=NULL)

What the hell is the point of:
time_series[ts_idx]=NULL;
That is needed to save yourself from accessing data you've given back
to the system. The only point where you would savely NOT set the
free()'d pointer to NULL is when the pointer and the data it points to
is only alive only inside the function you calls free() AND one of the
next statements is return, In any other case setting it to NULL will
give you a clean "access of pointer to 0 instead of undefined
behavior.
--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2 Deutsch ist da!
Aug 19 '06 #26

Herbert Rosenau <os****@pc-rosenau.dewrote in message
news:wm******** *************** ****@JUPITER1.P C-ROSENAU.DE...
On Fri, 18 Aug 2006 23:27:03 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:
Herbert Rosenau <os****@pc-rosenau.dewrote in message
news:wm******** *************** ****@JUPITER1.P C-ROSENAU.DE...
On Thu, 17 Aug 2006 23:54:41 UTC, "Bill Reid"
<ho********@hap pyhealthy.netwr ote:

I guess I got the last two conditions conflated in my mind; it just
seemed
logical to me that if realloc() failed it would free the previous
block.
It SEEMS like it should.
>
But it does not so because you may need to continue your work with the
old block.
Sure, but right off-hand I can't think of single case where I would
actually want to do anything with "half a loaf". When memory
allocation fails in whole or in part, I just want to get out of there
as quickly as possible, maybe just break out of that particular
module operation, quite often just to exit the whole program.

... and leaving the database in an undefined state.
Maybe YOUR database. I started this whole thread while looking
at a specific operation in one specific module. Each operation in the
module downloads several raw data files from the net, parses out the
data items, and writes the data to a database. Unless I get ALL of
the data, ALL of the files downloaded properly in the correct format
and with the correct data, NOTHING, none of the about 100,000
data items in this specific operation gets written to the database.

I specifically DESIGNED these download modules to work
this way, even though the database can never be "undefined" (if data
is written to the database, attempts to "over-write" it can't occur and
don't matter anyway), out of force of habit writing the routines that
analyze the data. In those modules, it is a complete waste of time to
in any way analyze partial data, so if I can't download it all from the
database, I just give up (except that's actually never happened,
probably because I run that stuff at night with no other processes
running).
Maybe you needs a
lot of single steps on the incloplete data to do to unwind any action
you've already done with.
Well, maybe YOU do. It's not like I'm not aware of how I could
corrupt my results or my database, it's just that I design the various
parts of the program to avoid these types of situations.
So the data is needed by your program. Most
programs are more complex in data handling as you currently aware of.
Possibly...and maybe not. I've seen some REALLY "complex"
programs in my time, and on a LOC basis alone (or any other basis)
I can't really classify what I do as "simple"... but I'm definitely not
multi-threading or some other things that might dramatically increase
the "complexity "...
So the best realloc can do for you to flag the error that it is out of
memory and leave data it arrived unchanged. Sometimes you gets a
chance to continue when you free()s some other data area.
Sometimes you'll split the aready occupied block into one or more
smaller ones, write older, smaller blocks out and reuse them.
Hmmmm, yeah, I was afraid of this type of response...yeah , there's
about a million things I COULD do, I just DON'T...

You may have missed the part where I said I only care that my
stuff WORKS, works flawlessly, and blindingly-fast for the amount
of data being processed. I'm not TRYING to generate bugs,
cuz I don't get paid to fix them, I just LOSE money...
There are lots of possibilities to continue even without - or with -
asking the user.
I thought the general drill was to ask the user (in this case, I'm the
only user) to close out some other applications. But I guess another
general strategy for the "commercial market" is to let the thing sit
there and grind away pointlessly; at least it keeps the room warm
on a cold day...
Exit is seldom the choice in real programs.
Yeah, I think I'm getting your point here, it's just that it doesn't
really have anything to do with "complexity ", but something else...
Often you
will have a solution for "out of memory" on a higher level.
Yeah, but I would suspect these would all slow you down to
a crawl...for my purposes, I'll pass, but again, for the "commercial "
market it might make sense...and since most software is written for
the "commercial " market, maybe it DOES make "statistica l" sense
for realloc() to work that way...
exit() is good for ingenious programs but the real world is more
complex.
Well, I don't generally break all the way out to exit(), except in the case
of the database initialization stuff whenever I start up, but I probably
could
since each function tends to clean up after itself on error and return
success or failure in some way...
So realloc gives you the chance for a real cleanup of any
kind you may need. It is simply a bug to overwrite a data area you
have already allocated with a null pointer. You have to cleanup.
SOME people have to clean up in that fashion. I'm already clean,
I just have to close some open files, release some related memory,
I'm good...
I suspect that is true of the vast majority of programs out there.
So not automatically freeing the block seems to be a case of "the
needs of the few outweighing the needs of the many".

You not written a reals program using realloc. You will then quickly
revise your standpoint.
If I'm lucky I never WILL write a "real" program. I would hate
to be "corrupted" ...
When you have no need for the old block you have to free() that
yourself, else you should free() it. In any case you needs its
address.
>
In this imperfect world, I guess so...what a friggin' hassle...
As an orthogonal point, what horrible things happen if you try
to free() a NULL pointer?
>
Nothing. free(NULL); works like a noop. Nothing occures. That is
guaranteed.
>
So I've kind of been wasting a little time with these types of
generalized pre-exit memory cleanup routines:

void free_time_serie s_mem(void) {
unsigned ts_idx;

for(ts_idx=0;ts _idx<TS_MAX;ts_ idx++) {
if(time_series[ts_idx]!=NULL) {
free(time_serie s[ts_idx]);
time_series[ts_idx]=NULL;
}
}

num_series=0;
}

Not only don't I need:
if(time_series[ts_idx]!=NULL)

What the hell is the point of:
time_series[ts_idx]=NULL;

That is needed to save yourself from accessing data you've given back
to the system.
Again, you are correct sir! Well, sort of...

I mean, that's a good reason TO do it; it's just that I incorrectly
described that particular function as a "pre-exit" memory clean-up
routine when I quickly looked at it. If was truly "pre-exit", then
there would be no need worry about an actual address in
the pointer, the next call is to exit() (or return to main(), exit())...
The only point where you would savely NOT set the
free()'d pointer to NULL is when the pointer and the data it points to
is only alive only inside the function you calls free() AND one of the
next statements is return, In any other case setting it to NULL will
give you a clean "access of pointer to 0 instead of undefined
behavior.
....but I realized after I posted that I also call it when "the user"
leaves one particular module for another. Then I could POSSIBLY
come back later and have the old pointer laying around to screw
me up; it would specifically mess up the purpose of the function
I use to populate the array on an "as-needed" basis as any module
requires those particular structures:

TIME_SERIES *create_time_se ries(void) {
unsigned ts_idx;

for(ts_idx=0;ts _idx<TS_MAX;ts_ idx++) {
if(time_series[ts_idx]==NULL) {
if((time_series[ts_idx]=
(TIME_SERIES *)malloc(sizeof (TIME_SERIES))) ==NULL) {
#ifdef __CONSOLE__
printf("\nNot enough memory to create time series");
#endif
#ifdef __WINGUI__
if(MessageDlg(" Not enough memory to create time series",
mtConfirmation, TMsgDlgButtons( )<<mbOK,0)==mrO k) ;
#endif
break;
}

time_series[ts_idx]->ts_vars=init_t sv;
time_series[ts_idx]->ds_vars=init_d sv;
num_series++;
break;
}
}

return time_series[ts_idx];
}

Of course, that's all screwed up in the first place because of the
ridiculous cast of malloc()...

---
William Ernest Reid

Aug 20 '06 #27

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
2194
by: ceo | last post by:
Hi, Following is a program that doesn't give the expected output, not sure what's wrong here. I'm adding the size of derived class to the base class pointer to access the next element in the array. Why does bp point to some address that is not &d? Please clarify. Thanks,
22
12757
by: Alex Fraser | last post by:
From searching Google Groups, I understand that void pointer arithmetic is a constraint violation, which is understandable. However, generic functions like qsort() and bsearch() must in essence do exactly this, and similarly generic functions seem to have useful applications. In a few posts I read, it was suggested to avoid void pointer arithmetic by casting to a char pointer, and performing arithmetic on that (in multiples of...
6
2297
by: Francois Grieu | last post by:
Are these programs correct ? #include <stdio.h> unsigned char a = {1,2}; int main(void) { unsigned char j; for(j=1; j<=2; ++j) printf("%u\n", *( a+j-1 )); return 0; }
3
2213
by: randomtalk | last post by:
hello everyone! Well, recently i've been trying to pick up c and see what is pointer all about (been programming in lisp/python for the better part of my last two years).. mmm.. I'm currently reading The C Programming Language Second Edition.. When i hit on pointer arithmetic example, i have no idea what's happening, hopefully some of you could alleviate my confusion, so here goes:
7
528
by: barikat | last post by:
int a; int *Ptr1, *Ptr2; Ptr1 = a; Ptr1++; Ptr2 = a; printf("Ptr1 : %p\n", Ptr1); printf("Ptr2 : %p\n\n", Ptr2);
5
3183
by: jerry | last post by:
I need to modify the code of a command-line tool. The source code starts out like this: int main(int argc, char *argv) { int ch, A, B ; while ((ch = getopt(argc, argv, "AB")) != -1) switch (ch) { case 'A':
41
3701
by: Summercool | last post by:
Can we confirm the following? also someone said, Java also has "reference" like in C++, which is an "implicit pointer": Pointer and Reference --------------------- I am starting to see what pointer and reference are and how they relate to each other.
3
1740
by: =?Utf-8?B?RGlwZXNoX1NoYXJtYQ==?= | last post by:
Hi all, I am porting my code in VC++ to VC.net i.e managed. I have seen somewhere that i need to convert my char* to String*, but String * doesnt perform pointer arithmetic like String* p; *p++=' '; So please suggest as to what i have to do? Am i doing wrong by converting char* to String* in managed VC.net. Thanks, Dipesh.
19
3913
by: =?iso-8859-1?b?VG9t4XMg0yBoyWlsaWRoZQ==?= | last post by:
Coming originally from C++, I used to do the likes of the following, using a pointer in a conditional: void Func(int *p) { if (p) { *p++ = 7; *p++ = 8;
22
2480
by: sophia | last post by:
Dear all, can any one explain what is meant by pointer normalization given here:- http://c-faq.com/ansi/norml.html
0
9820
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, weíll explore What is ONU, What Is Router, ONU & Routerís main usage, and What is the difference between ONU and Router. Letís take a closer look ! Part I. Meaning of...
0
10846
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10254
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7793
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Duprť who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5650
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5828
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4458
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4020
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3116
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.