By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
435,131 Members | 1,749 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 435,131 IT Pros & Developers. It's quick & easy.

Need to Allocate more space than size_t

P: n/a
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
Plz consider that all the physical hardware req for such an operation
is available... i know this sounds crazy... but just help me out here
....
Thanks in Advance
Rahul

Nov 14 '06 #1
Share this Question
Share on Google+
20 Replies


P: n/a

i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to

consequtive bytes and dynamic allocation ???

Nov 14 '06 #2

P: n/a

ra*******************@gmail.com wrote:
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
This is probably wandering into "Off-topic for comp.lang.c" - there's
no way in the standard language.

You'd have to delve into the O/S to do this, if it is possible at all.
On un*x-like systems the "brk()" or "sbrk()" calls are probably
relevant, or the memory-mapping calls (mmap()) but...

Are you actually able to deal with more memory than size_t can denote?
You will be limited, one way or another by issues such as
addressability.. If you are working on a 32-bit O/S, then you cannot
(AFAIK) have more than 4Gb of address space for a process, for example.
Plz consider that all the physical hardware req for such an operation
is available...
But if the O/S can't access more than size_t can denote, that's no help
at all..
i know this sounds crazy... but just help me out here
You probably need to give us more information, but we'll probably point
you at a newsgroup relating to your platform...

Nov 14 '06 #3

P: n/a
"ra*******************@gmail.com" <ra*******************@gmail.com>
wrote:
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
There isn't any. If you need more memory than can be specified in a
size_t, ISO C gives you no way to do so. Ideally, this shouldn't even be
possible; your implementation should make size_t large enough to cater
for every possible memory block size.

Richard
Nov 14 '06 #4

P: n/a
ra*******************@gmail.com wrote:
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
Plz consider that all the physical hardware req for such an operation
is available... i know this sounds crazy... but just help me out here
...
This is not possible using strictly standard C. The type size_t is
specifically meant to store the sizes of objects in a C program and it
should be possible to contain within a size_t object, the size of the
largest possible object guaranteed under standard C.

Generally, one would expect size_t to be equivalent to the largest
unsigned type, under most implementations, though you cannot assume
this. For example, on my implementation here, size_t is a typedef for
unsigned int, even though unsigned long long int, a much larger
unsigned type, is also available.

Anyway, returning to your question, the short answer is that you'll
have to use implementation specific extensions or, more likely, OS
specific system calls and their corresponding types. Asking in a group
for your system might get more helpful responses.

Nov 14 '06 #5

P: n/a
sa*****@yahoo.co.in wrote:
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to


consequtive bytes and dynamic allocation ???
Why not? As far as a conforming C program is concerned, the component
bytes of a single dynamically allocated object are consecutive, though
that may not be the case as far as the layout of the object on physical
memory is concerned.

Nov 14 '06 #6

P: n/a
Why not? As far as a conforming C program is concerned, the component
bytes of a single dynamically allocated object are consecutive, though
that may not be the case as far as the layout of the object on physical
memory is concerned.
I have never seen a requirement to allocate memory dynamically and
to expect that to be in consecutive locations... may be the OP should
explain the actual reason as to why he needs this, if he can...

Nov 14 '06 #7

P: n/a
sa*****@yahoo.co.in wrote:
>Why not? As far as a conforming C program is concerned, the component
bytes of a single dynamically allocated object are consecutive, though
that may not be the case as far as the layout of the object on physical
memory is concerned.

I have never seen a requirement to allocate memory dynamically and
to expect that to be in consecutive locations... may be the OP should
explain the actual reason as to why he needs this, if he can...
The OP seems to be talking about resizing, (enlarging), the allocation
for a single object with a call to realloc(). The bytes of a single
dynamically allocated object in C are consecutive to the program.

If you're talking about the layout of two or more dynamic objects then
yes, they need not be consecutive.

Nov 14 '06 #8

P: n/a
<ra*******************@gmail.comwrote in message
news:11*********************@m73g2000cwd.googlegro ups.com...
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
size_t is able to hold the size of the largest object you can allocate
with malloc() or realloc() by definition. That means that, within the
realm of Standard C, there is no way to allocate an object larger than
SIZE_MAX.

Your OS may have mechanisms to allocate larger objects; you'll need to
ask about those in an OS-specific newsgroup.
Plz consider that all the physical hardware req for such an operation
is available... i know this sounds crazy... but just help me out here
Note that any sane implementation will define size_t such that it can
handle anything the system is capable of. It is unlikely that your OS
does, in fact, have something better, because that's what the C
implementors would have used if it were available.

For example, on x86 with PAE, your machine may have 36 bits of physical
memory, but it's impossible to access more than 32 bits of address space
from a single program, so size_t will be 32 bits. Ditto for an 32-bit
program running inside a 64-bit OS; the OS can hand out a different 4GB
address space to each program, but no single program can use more than
4GB on its own. A 64-bit program on the same OS would have a 64-bit
size_t.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking

--
Posted via a free Usenet account from http://www.teranews.com

Nov 14 '06 #9

P: n/a

ra*******************@gmail.com wrote:
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
As I and others have pointed out, you can't do this in Standard C and
it's very likely that you can't do it using non-standard techniques
either.

Why not describe what you are trying to achieve with this huge memory
allocation and let us suggest alternative ways of meeting your goal?

Nov 14 '06 #10

P: n/a
ra*******************@gmail.com wrote:
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
Plz consider that all the physical hardware req for such an operation
is available... i know this sounds crazy... but just help me out here
I suggest getting a C compiler that support your hardware. So, if you
have a 64-bit system, then you need a compiler which support for e.g.
LP64 model. You cannot expect a ILP32 compiler to give you some magic
way to address an object with greater size than 32-bit.

--
Tor <torust AT online DOT no>

Nov 14 '06 #11

P: n/a
ra*******************@gmail.com:
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
Plz consider that all the physical hardware req for such an operation
is available... i know this sounds crazy... but just help me out here
...

Either:

(1) Your particular implementation intends such large chunks of memory to
be allocated via implementation-specific means (e.g. System API function).
(2) Your particular implementation intends for you to be able to allocate
such large chunks of memory using Standard C, but has neglected to make
"size_t" big enough.

Solutions:

(1) Unfortunately, you'll have to resort to implementation-specific means.
(2) You'll have to contact your compiler manufacturer and ask for a fix, or
perhaps try fix it yourself (e.g. altering the size_t typedef)
That's as much as I can tell you without going Off-Topic here.

--

Frederick Gotham
Nov 14 '06 #12

P: n/a
Frederick Gotham wrote:
Solutions:

(1) Unfortunately, you'll have to resort to implementation-specific means.
The problem is that OP's implementation doesn't support his HW
archtecture.
(2) You'll have to contact your compiler manufacturer and ask for a fix, or
perhaps try fix it yourself (e.g. altering the size_t typedef)
How is changing size_t typedef gonna help?

--
Tor <torust AT online DOT no>

Nov 14 '06 #13

P: n/a
Tor Rustad:
Frederick Gotham wrote:
>Solutions:

(1) Unfortunately, you'll have to resort to implementation-specific
means.

The problem is that OP's implementation doesn't support his HW
archtecture.

If you're supposed to be able to allocate a terrabyte of memory without
resorting to implementation-specific means, then "size_t" should be able to
hold a large enough value.

If this is indeed intended, then the implementation is flawed in that
size_t isn't big enough.

If this isn't intended, then you'll have to use implementation-specific
means to achieve your goal.

>(2) You'll have to contact your compiler manufacturer and ask for a
fix, or perhaps try fix it yourself (e.g. altering the size_t typedef)

How is changing size_t typedef gonna help?

Perhaps it's defined(Do you "declare" or do you "define" a typedef?) as
follows in his/her system headers:

typedef long unsigned size_t;

, and perhaps he/she can change it to something platform-specific like:

typedef __int128 unsigned size_t;

Of course, this might mess up his/her entire system.

--

Frederick Gotham
Nov 14 '06 #14

P: n/a
2006-11-14 <wO*******************@news.indigo.ie>,
Frederick Gotham wrote:
Perhaps it's defined(Do you "declare" or do you "define" a typedef?) as
follows in his/her system headers:

typedef long unsigned size_t;

, and perhaps he/she can change it to something platform-specific like:

typedef __int128 unsigned size_t;

Of course, this might mess up his/her entire system.
You misspelled "will".

He'd have to, at the _very_ least, modify and recompile all his libraries.
Nov 14 '06 #15

P: n/a
ra*******************@gmail.com wrote:
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..
You could use calloc( ), it can allocate more memory than
SIZE_MAX. Then copy all the data from your previous allocation,
and free it.

Nov 14 '06 #16

P: n/a
Old Wolf wrote:
ra*******************@gmail.com wrote:
>>hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..


You could use calloc( ), it can allocate more memory than
SIZE_MAX. Then copy all the data from your previous allocation,
and free it.
Doing the copy could be a bit of a challenge if you have more memory
than the size of a pointer.

--
Ian Collins.
Nov 14 '06 #17

P: n/a
Frederick Gotham <fg*******@SPAM.comwrites:
[...]
Either:

(1) Your particular implementation intends such large chunks of memory to
be allocated via implementation-specific means (e.g. System API function).
(2) Your particular implementation intends for you to be able to allocate
such large chunks of memory using Standard C, but has neglected to make
"size_t" big enough.

Solutions:

(1) Unfortunately, you'll have to resort to implementation-specific means.
(2) You'll have to contact your compiler manufacturer and ask for a fix, or
perhaps try fix it yourself (e.g. altering the size_t typedef)
That's as much as I can tell you without going Off-Topic here.
No, altering the size_t typedef in your implementation's <stddef.h>
file (assuming it's a file) will not help. It's not absolutely
guaranteed to break the implementation (it's barely conceivable that
everything would magically work correctly with the modified typedef),
but I'd be astonished if it actually worked.

Don't try modify your implementation's system header files unless
you're absolutely sure you know what you're doing. And if you *think*
you're absolutely sure you know what you're doing, you're probably
mistaken.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Nov 14 '06 #18

P: n/a
"Old Wolf" <ol*****@inspire.net.nzwrites:
ra*******************@gmail.com wrote:
>hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system . after i do a malloc .. i am unable to
do a realloc because it takes size_t as a new size and not as an
incremental size..
can you tell me which library/system call to use..

You could use calloc( ), it can allocate more memory than
SIZE_MAX. Then copy all the data from your previous allocation,
and free it.
Not necessarily. calloc() takes two arguments, and the number of
bytes allocated is their product, but it's likely that an attempt to
allocate more than SIZE_MAX bytes will fail. Also, calloc()
zero-initializes the allocated space; with the amount of memory we're
talking about, that could take a significant amount of time.

If you really need to deal with more than SIZE_MAX bytes, your best
bet is probably to use a file (or to use a system with a larger
SIZE_MAX).

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Nov 14 '06 #19

P: n/a
ra*******************@gmail.com wrote
(in article
<11*********************@m73g2000cwd.googlegroups. com>):
hi folks
i have a peculiar problem. i have to allocate more than size_t
consequtive bytes on a system .
Then you are using a compiler not properly matched to the
system, otherwise size_t would be plenty big.
can you tell me which library/system call to use..
The existing ones work fine, provided your implementation is
suited to the host architecture.
Plz consider that all the physical hardware req for such an operation
is available... i know this sounds crazy... but just help me out here
Using a 32-bit compiler on a 64-bit system by chance?

--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw

Nov 14 '06 #20

P: n/a
Frederick Gotham wrote:
Tor Rustad:
Frederick Gotham wrote:
Solutions:

(1) Unfortunately, you'll have to resort to
implementation-specific means.
The problem is that OP's implementation doesn't
support his HW archtecture.


If you're supposed to be able to allocate a terrabyte of
memory without resorting to implementation-specific means,
then "size_t" should be able to hold a large enough value.
FYI, *size_t* is NOT likely to be the main problem.

If this is indeed intended, then the implementation is
flawed in that size_t isn't big enough.
Why even consider the implementation is flawed?

Even if that was the case... is there really a way to fix
it, without having the compiler sources?

(2) You'll have to contact your compiler manufacturer
and ask for a fix, or perhaps try fix it yourself (e.g.
altering the size_t typedef)
How is changing size_t typedef gonna help?


Perhaps it's defined(Do you "declare" or do you "define" a
typedef?) as follows in his/her system headers:

typedef long unsigned size_t;
And I ask once more, what is the purpose of doing this?

What are you gonna use this new size_t for?

Of course, this might mess up his/her entire system.
Yes, invoking undefined behavior might do that.

--
Tor <torust AT online DOT no>

Nov 15 '06 #21

This discussion thread is closed

Replies have been disabled for this discussion.