how to malloc 2G ram? in freebsd, win2k

hi
i try to malloc 2G size mem . so i wirte code like that.
it failt in freebsd 5.3 and win2k

what's bug in my code ? how can i change it ?

thank all :)
benjiam
///////////////////////////////////////////////
#include <iostream>
using namespace std;

#include <cstdlib>
const int len = 2000;
int main()
{
char* p1[len];
static int size = int (1<<20);
cout << size << endl;
for (int i =0 ; i < len ;i++)
{
p1[i] = new char [size];
cout << size << endl;
char *tp = p1[i];
for ( int x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = char( (x%26)+ 48);

}
*(tp+size-1) =0;
cout << "p [" << i << "]ok" << endl;
}

for (int i =0 ; i < len ;i++)
{
delete [] p1[i];
}
return 0;
}
////////////////////////////////////////////

[Martin Ambuhl]

ch***********@hotmail.com wrote:

hi
i try to malloc 2G size mem . so i wirte code like that.
You have no guarantee that you can do so. You need to check in a
newsgroup, mailing list, or tech support for your implementation.
it failt in freebsd 5.3 and win2k
what's bug in my code ? how can i change it ?
We really can't help, partially because you posted to comp.lang.c, where
we use C, and your code is not C. comp.lang.c++ is a different newsgroup
for a different language.

Examples of C++isms are: #include <iostream> This is not a C header. using namespace std; This is a compilation error. #include <cstdlib> This is not a C header. cout << size << endl; 'cout' and 'endl' are undeclared identifiers, and '<<' is a
left-shift operator that probably don't do what you want. p1[i] = new char [size];

'new' is an undeclared identifier, and the above is a compilation error.

There are more instances, but I don't care to be tiresome.

[Walter Roberson]

In article <d1**********@news.yaako.com>, <ch***********@hotmail.com> wrote:
: i try to malloc 2G size mem . so i wirte code like that.
:it failt in freebsd 5.3 and win2k

:what's bug in my code ? how can i change it ?

And how are you compiling? It is fairly common for there to be a 2 GB
process size limit unless you specifically compile to use 64 bit
pointers. Yes, in theory with 32 bits you -could- get to 4 GB instead
of 2 GB, but it is not uncommon for half of the memory to be reserved
for system pointers.

Even within the 2 GB limit, it is common for there to be address space
layout issues: it is common for your code not to be placed at address 0
in virtual memory, and it is common for there to be an area reserved
for the stack, and it is common for there to be an area reserved for
the heap, and it is common for there to be one or more areas reserved
for DLLs or equivilent. You may have to give special linking options or
run special post-processors on the executable image in order to move
these reserved areas to give yourself the maximum amount of room.

On Unix systems it is also common for there to be resource limits that
prevent you from using gobs of memory unless you override them or get
the systems administrator to override them. See the 'ulimit' man page
to get yourself started, and the getrlimit() functions. The mechanisms
used by the systems administrator to control the limit would very with
unix version; I do not know the details for freebsd.

--
Entropy is the logarithm of probability -- Boltzmann

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed
///////////////////////////////////////////
#include <stdio.h>
#include <stdlib.h>
const int len = 2000;
int main()
{
char* p1[len];
int i =0;
int x =0;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1[i] = (char*) malloc (size);
printf( "%d \n", size );
char *tp = p1[i];
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

for ( i =0 ; i < len ;i++)
{
free(p1[i]);
}
return 0;
}

//////////////////////////////////////////

"Martin Ambuhl" <ma*****@earthlink.net>
??????:u6*************@newsread2.news.atl.earthlin k.net...

ch***********@hotmail.com wrote:

hi
i try to malloc 2G size mem . so i wirte code like that.
You have no guarantee that you can do so. You need to check in a
newsgroup, mailing list, or tech support for your implementation.

it failt in freebsd 5.3 and win2k
what's bug in my code ? how can i change it ?

We really can't help, partially because you posted to comp.lang.c, where
we use C, and your code is not C. comp.lang.c++ is a different newsgroup
for a different language.

Examples of C++isms are:

#include <iostream>

This is not a C header.

using namespace std;

This is a compilation error.

#include <cstdlib>

This is not a C header.

cout << size << endl;

'cout' and 'endl' are undeclared identifiers, and '<<' is a
left-shift operator that probably don't do what you want.

p1[i] = new char [size];

'new' is an undeclared identifier, and the above is a compilation

error.
There are more instances, but I don't care to be tiresome.

[CBFalconer]

ch***********@hotmail.com wrote:

.... snip ... #include <iostream>
using namespace std;

comp.lang.c++ is down the hall to the right.

--
"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare

[Raymond Martineau]

On Tue, 22 Mar 2005 16:35:15 +0800, <ch***********@hotmail.com> wrote:

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed
/////////////////////////////////////////// [...]
p1[i] = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check the return
value.

[Jonathan Bartlett]

ch***********@hotmail.com wrote:

hi
i try to malloc 2G size mem . so i wirte code like that.
it failt in freebsd 5.3 and win2k

what's bug in my code ? how can i change it ?

I don't know about FreeBSD, but on Windows a process cannot be bigger
than 2G, even if you have more RAM than that. Each process is limitted
to at most 2G. So, if you add the space for your code, libraries, and
existing variables, malloc'ing 2G on a Windows box is guaranteed to fail.

I _think_ Linux gives you 3G, but I could be wrong. I don't know if
these limits persist to AMD-64 or not. However, I would try it on a
64-bit PowerPC -- it should work there.

Jon
----
Learn to program using Linux assembly language
http://www.cafeshops.com/bartlettpublish.8640017

[CBFalconer]

Raymond Martineau wrote:

<ch***********@hotmail.com> wrote:
.... snip ...

p1[i] = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check
the return value.

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

However, there may be a later problem when using p[i].

--
"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare

[Christopher Benson-Manica]

CBFalconer <cb********@yahoo.com> spoke thus:

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided. However, there may be a later problem when using p[i].

I think this is what he was actually referring to, although his
quoting could have been better.

--
Christopher Benson-Manica | I *should* know what I'm talking about - if I
ataru(at)cyberspace.org | don't, I need to know. Flames welcome.

#include <stdio.h>

#include <stdlib.h>
const int len = 2000;
int main()
{
char* p1[len];
int i =0;
int x =0;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1[i] = (char*) malloc (size);
printf( "%d \n", size );
char *tp = p1[i];
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

for ( i =0 ; i < len ;i++)
{
free(p1[i]);
}
return 0;
}

hi
I _think_ Linux gives you 3G, but I could be wrong. I don't know if
these limits persist to AMD-64 or not. However, I would try it on a
64-bit PowerPC -- it should work there.

u run my code? :)

#include <stdio.h>

#include <stdlib.h>
const int len = 2000;
int main()
{
char* p1[len];
int i =0;
int x =0;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1[i] = (char*) malloc (size);
printf( "%d \n", size );
char *tp = p1[i];
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

for ( i =0 ; i < len ;i++)
{
free(p1[i]);
}
return 0;
}







"Jonathan Bartlett" <jo*****@eskimo.com>
??????:42********@news.tulsaconnect.com... ch***********@hotmail.com wrote:

hi
i try to malloc 2G size mem . so i wirte code like that.
it failt in freebsd 5.3 and win2k

what's bug in my code ? how can i change it ?

I don't know about FreeBSD, but on Windows a process cannot be bigger
than 2G, even if you have more RAM than that. Each process is limitted
to at most 2G. So, if you add the space for your code, libraries, and
existing variables, malloc'ing 2G on a Windows box is guaranteed to fail.

I _think_ Linux gives you 3G, but I could be wrong. I don't know if
these limits persist to AMD-64 or not. However, I would try it on a
64-bit PowerPC -- it should work there.

Jon
----
Learn to program using Linux assembly language
http://www.cafeshops.com/bartlettpublish.8640017

[Raymond Martineau]

On Tue, 22 Mar 2005 18:57:26 GMT, CBFalconer <cb********@yahoo.com> wrote:

Raymond Martineau wrote:

<ch***********@hotmail.com> wrote:

... snip ...

p1[i] = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check
the return value.

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

Actually, I was referring to the fact that there wasn't any checking on the
return value (which was later blindly used.)

In this case, I didn'tpoint out the line at which it would fail.

[Walter Roberson]

In article <d1**********@news.yaako.com>, <ch***********@hotmail.com> wrote:
:I _think_ Linux gives you 3G, but I could be wrong.

Something I read awhile ago suggested that it depended on the kernel
version and how you compiled.

:u run my code? :)

After some cleanup and optimization, yes. Compiled as a 32 bit
executable, it runs on one machine until it pretty much reaches the
end of the storage space "underneath" the virtual address that libc
gets placed at. Compiled as a 64 bit executable and run on a different
machine, it run to completion -- the object layout is different for
the 64 bit executables.
--
Any sufficiently advanced bug is indistinguishable from a feature.
-- Rich Kulawiec

[Stephen Sprunk]

<ch***********@hotmail.com> wrote in message
news:d1**********@news.yaako.com...

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed
There is no guarantee that a program can allocate arbitrarily large amounts
of memory. malloc() is allowed to return NULL at any time.

<OT>
Virtual memory doesn't mean that you can allocate 2-4GB in each application;
there is almost certainly a limit that your OS or configuration imposes
below that. Apparently your system's limit is 294MB for this program.
</OT>
for ( i =0 ; i < len ;i++)
{
p1[i] = (char*) malloc (size);
The cast is unnecessary (and not idiomatic).
printf( "%d \n", size );
char *tp = p1[i];
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);
You forgot to check if p1[i], and thus tp, is NULL; dereferencing that'll
ruin your day.

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

S

--
Stephen Sprunk "Stupid people surround themselves with smart
CCIE #3723 people. Smart people surround themselves with
K5SSS smart people who disagree with them." --Aaron Sorkin

[Jonathan Burd]

ch***********@hotmail.com wrote:

hi

<mowed>

Please do not top-post in c.l.c.

Regards,
Jonathan.

--
Email: "jonathan [period] burd [commercial-at] gmail [period] com" sans-WSP

"I usually swear at C++, but so far it has never sworn back."
- Ben "Noir"

[Randy Howard]

In article <d1**********@canopus.cc.umanitoba.ca>, ro******@ibd.nrc-
cnrc.gc.ca says...

In article <d1**********@news.yaako.com>, <ch***********@hotmail.com> wrote:
:I _think_ Linux gives you 3G, but I could be wrong.

Something I read awhile ago suggested that it depended on the kernel
version and how you compiled.
OT: This can be done under Windows also, provided you have less than
4GB of RAM total and do not have /PAE turned on. Google for boot.ini
options.
After some cleanup and optimization, yes. Compiled as a 32 bit
executable, it runs on one machine until it pretty much reaches the
end of the storage space "underneath" the virtual address that libc
gets placed at.
I think that you will discover that for a system running 32-bit
Windows or Linux with 2GB of RAM or more installed, and using default
kernel options (I.e. no 3GB address space hack), that a reasonably
sized command line app will be able to malloc around 1600-1700MB of
RAM prior to it failing. People that claim that malloc() never fails
should try this for themselves.
Compiled as a 64 bit executable and run on a different machine, it
run to completion -- the object layout is different for the 64 bit
executables.

Malloc will also fail (eventually) on Linux/glibc 64-bit
combinations. The point it fails varies. I tried this a while back
on an x86_64 w/64GB (not a typo) of RAM. I forgot where it failed
in a single process (16GB?), but recall that with pthreads (NPTL),
it would fail at just under 8GB/thread, but you could effectively
chew up the entire 64GB by starting 8 threads and having each ask
for 8GB.

I haven't tried this on a Win64 x86_64 system (yet).

--
Randy Howard (2reply remove FOOBAR)
"Making it hard to do stupid things often makes it hard
to do smart ones too." -- Andrew Koenig

[aurgathor]

For one thing, you need a systen capable
of allocating over 2 Gigs of memory, and
that could be a bit challenging to an average
PC.

Next, ever heard of a NULL pointer?

BTW, I run it on my sytem with a target changed to
3 Gig, and with some extra diag code put in there,
and it did go to 2.1 Gig.

(Said PC has 2 Gigs of physical memory, BTW)

<ch***********@hotmail.com> wrote in message
news:d1**********@news.yaako.com...

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed

hi
thanks for your help , and i change it again.
code i fixed
///////////////////////////////////////
#include <stdio.h>
#include <stdlib.h>

const int len = 800;
int main()
{
char* p1[len];
int i =0;
int x =0;
char *tp;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1[i] = (char*) malloc (size);

tp = p1[i];
if (tp == NULL)
continue;
printf( "%d \n", size );

for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);

}

for ( i =0 ; i < len ;i++
)
{
if (p1[i] == NULL)
continue;
free(p1[i]);
printf("i clean %d \n", i);
}
return 0;
}

///////////////////////

in win2k it can run success,
but in linux and freebsd it fail. killed by os

i wish if malloc() fail, it can continue, in win2k it success
in linux and freebsd it fail.

benjiam







"Raymond Martineau" <bk***@ncf.ca>
??????:kn********************************@4ax.com. ..

On Tue, 22 Mar 2005 18:57:26 GMT, CBFalconer <cb********@yahoo.com> wrote:

Raymond Martineau wrote:

<ch***********@hotmail.com> wrote:
... snip ...

p1[i] = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check
the return value.

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

Actually, I was referring to the fact that there wasn't any checking on

the return value (which was later blindly used.)

In this case, I didn'tpoint out the line at which it would fail.

[Walter Roberson]

In article <d1**********@news.yaako.com>, <ch***********@hotmail.com> wrote:

thanks for your help , and i change it again. code i fixed
p1[i] = (char*) malloc (size);
tp = p1[i];
if (tp == NULL)
continue;

You do not want a 'continue' there, you want a 'break'.
Otherwise it will loop around again, and fail the malloc
again.

The version below is a bit more efficient.

#include <stdio.h>
#include <stdlib.h>
const int len = 2000;
const int size = 1<<20;
int main()
{
char* p1[len];
char p2[size];
int i =0;
int x =0;
for ( x = 0; x < (size-1); x++ ) p2[x] = (char)((x%26) + 48 );
p2[size-1] = 0;

printf( "%d \n", size);
for ( i =0 ; i < len ;i++) {
p1[i] = malloc (size);
if ( !p1[i] ) break;
strncpy( p1[i], p2, size );
printf( "p [ %d ] ok\n" , i);
}

while (--i >= 0) free( p1[i] );

return 0;
}
I haven't had a chance yet to try it out compiled for 64 bit pointers
on a system with 1.5 Gb physical memory.
--
'The short version of what Walter said is "You have asked a question
which has no useful answer, please reconsider the nature of the
problem you wish to solve".' -- Tony Mantler

i suppose
in case of i =90
p1[i] = (char*) malloc (size);
tp = p1[i];
if (tp == NULL)
continue;
it fail
loop
i= 91

it fail
loop
i =92
at this time ,the other process free a lot of memory
and this time it will malloc success.(maybe it is wrong idea,resource
cortroled by os)

You do not want a 'continue' there, you want a 'break'.
Otherwise it will loop around again, and fail the malloc
again.
"Walter Roberson" <ro******@ibd.nrc-cnrc.gc.ca> дÈëÏûÏ¢ÐÂÎÅ
:d1**********@canopus.cc.umanitoba.ca...

In article <d1**********@news.yaako.com>, <ch***********@hotmail.com>

wrote:

thanks for your help , and i change it again.

code i fixed
p1[i] = (char*) malloc (size);
tp = p1[i];
if (tp == NULL)
continue;

You do not want a 'continue' there, you want a 'break'.
Otherwise it will loop around again, and fail the malloc
again.

The version below is a bit more efficient.

#include <stdio.h>
#include <stdlib.h>
const int len = 2000;
const int size = 1<<20;
int main()
{
char* p1[len];
char p2[size];
int i =0;
int x =0;
for ( x = 0; x < (size-1); x++ ) p2[x] = (char)((x%26) + 48 );
p2[size-1] = 0;

printf( "%d \n", size);
for ( i =0 ; i < len ;i++) {
p1[i] = malloc (size);
if ( !p1[i] ) break;
strncpy( p1[i], p2, size );
printf( "p [ %d ] ok\n" , i);
}

while (--i >= 0) free( p1[i] );

return 0;
}
I haven't had a chance yet to try it out compiled for 64 bit pointers
on a system with 1.5 Gb physical memory.
--
'The short version of what Walter said is "You have asked a question
which has no useful answer, please reconsider the nature of the
problem you wish to solve".' -- Tony Mantler

[Keith Thompson]

<ch***********@hotmail.com> writes:
[snip]

Once again, "chong19782002", please stop changing the subject header
when you post a followup. The subject header is not a place to make
short comments, it's the place to describe what the discussion is
about, and it should almost always remain exactly the same (apart from
the "Re: " that your newsreader will add) for every article in a
thread.

And please don't top-post. Any new text you write belongs *after* any
quoted material, or interspersed with it if you're responding to
individual points. See almost every article in this newsgroup for
examples of how to do this.

These are not arbitrary rules that we impose for the fun of it;
they're conventions that make it possible to carry on a coherent
conversation.

If you want help, please do us all the courtesy of posting properly.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

o! i see, i will stop change subject header.

please don't top-post,
i did not do it. i always post a followup. it is a followup in
my outlook

thanks for your advice.

"Keith Thompson" <ks***@mib.org> ??????:ln************@nuthaus.mib.org...

<ch***********@hotmail.com> writes:
[snip]

Once again, "chong19782002", please stop changing the subject header
when you post a followup. The subject header is not a place to make
short comments, it's the place to describe what the discussion is
about, and it should almost always remain exactly the same (apart from
the "Re: " that your newsreader will add) for every article in a
thread.

And please don't top-post. Any new text you write belongs *after* any
quoted material, or interspersed with it if you're responding to
individual points. See almost every article in this newsgroup for
examples of how to do this.

These are not arbitrary rules that we impose for the fun of it;
they're conventions that make it possible to carry on a coherent
conversation.

If you want help, please do us all the courtesy of posting properly.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst> San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst> We must do something. This is something. Therefore, we must do this.

[Keith Thompson]

<ch***********@hotmail.com> writes:

o! i see, i will stop change subject header.

please don't top-post,
i did not do it. i always post a followup. it is a followup in
my outlook

thanks for your advice.

"Keith Thompson" <ks***@mib.org> ??????:ln************@nuthaus.mib.org...

<ch***********@hotmail.com> writes:
[snip]

Once again, "chong19782002", please stop changing the subject header
when you post a followup. The subject header is not a place to make
short comments, it's the place to describe what the discussion is
about, and it should almost always remain exactly the same (apart from
the "Re: " that your newsreader will add) for every article in a
thread.

And please don't top-post. Any new text you write belongs *after* any
quoted material, or interspersed with it if you're responding to
individual points. See almost every article in this newsgroup for
examples of how to do this.

These are not arbitrary rules that we impose for the fun of it;
they're conventions that make it possible to carry on a coherent
conversation.

If you want help, please do us all the courtesy of posting properly.

You just top-posted again. As you can see from the above (your
previous article, quoted in full), you posted your new comments
*above* the text quoted from the previous article; that's called
top-posting. Any new text you write should be *below* any quoted
text, and you should delete anything that's not relevant to your
followup.

I think Outlook encourages top-posting by placing the cursor above the
quoted text. You still need to write below the quoted text.

Please read the following:

<http://www.caliburn.nl/topposting.html>
<http://www.netmeister.org/news/learn2quote.html>

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Mark McIntyre]

On Fri, 25 Mar 2005 10:00:15 +0800, in comp.lang.c ,
<ch***********@hotmail.com> wrote:

o! i see, i will stop change subject header.

please don't top-post,
i did not do it. i always post a followup. it is a followup in
my outlook

top posting is when you write your reply at the top of the screen
above any text you are replying to. In CLC, we very much prefer you to
put your BELOW the text of the message you're replying to.

You should also remove any text that is not relevant to your reply,
like I have done here by removing everything below this point except
my sig block....

--
Mark McIntyre
CLC FAQ <http://www.eskimo.com/~scs/C-faq/top.html>
CLC readme: <http://www.ungerhu.com/jxh/clc.welcome.txt>

----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----