Memory limit?

Hi, I need to use a lot of memory in an aplication. But I get
OutOfMemoryException whenever I try to allocate more than around 1.2GB.
Ex:
byte [] c=new byte[1300000000];
My computer (win xp) have 4GB of RAM memory and it looks like 3GB is free
(Performance Monitor->Available MB).
Is there some way to use more memory for my application?
Greatefull for answers!
Karl Daggfeldt

re: Memory limit?

KalleD wrote:[color=blue]
> Hi, I need to use a lot of memory in an aplication. But I get
> OutOfMemoryException whenever I try to allocate more than around 1.2GB.
> Ex:
> byte [] c=new byte[1300000000];
> My computer (win xp) have 4GB of RAM memory and it looks like 3GB is free
> (Performance Monitor->Available MB).
> Is there some way to use more memory for my application?
> Greatefull for answers!
> Karl Daggfeldt[/color]

a guess: as you want to allocate one array, the system might want
to use a single contiguous block, which doesn't appear to be there.
So the total number of bytes free could be 3GB, but the largest
single block is less than 1.2GB, therefore you get the exception.

just curious: why do you need this space?

--
Hans Kesting

re: Memory limit?

Thanks for your suggestion!
I have tried to divide the memory in smaller parts (3 parts) , but the
maximal limit do not seem to change. The data is consisting of an image
volume (densly spaced images) of selected parts of the human body (visible
human data). I need the data in memory in order to quickly be able to move
trough the data and turn to arbitrary viewing planes.
Karl Daggfeldt


"Hans Kesting" wrote:
[color=blue]
> KalleD wrote:[color=green]
> > Hi, I need to use a lot of memory in an aplication. But I get
> > OutOfMemoryException whenever I try to allocate more than around 1.2GB.
> > Ex:
> > byte [] c=new byte[1300000000];
> > My computer (win xp) have 4GB of RAM memory and it looks like 3GB is free
> > (Performance Monitor->Available MB).
> > Is there some way to use more memory for my application?
> > Greatefull for answers!
> > Karl Daggfeldt[/color]
>
> a guess: as you want to allocate one array, the system might want
> to use a single contiguous block, which doesn't appear to be there.
> So the total number of bytes free could be 3GB, but the largest
> single block is less than 1.2GB, therefore you get the exception.
>
> just curious: why do you need this space?
>
> --
> Hans Kesting
>[/color]

re: Memory limit?

Apparently it is some sort of .NET limit. Here's a link to another .NET app
which has the same problem. See if their solution will help you:
http://www.runuo.com/forum/showthread.php?t=38360

--
Adam Clauss
cabadam@tamu.edu

"KalleD" <KalleD@discussions.microsoft.com> wrote in message
news:D05F7A58-F4F2-4BB4-A43F-0C3B9681DD2B@microsoft.com...[color=blue]
> Hi, I need to use a lot of memory in an aplication. But I get
> OutOfMemoryException whenever I try to allocate more than around 1.2GB.
> Ex:
> byte [] c=new byte[1300000000];
> My computer (win xp) have 4GB of RAM memory and it looks like 3GB is free
> (Performance Monitor->Available MB).
> Is there some way to use more memory for my application?
> Greatefull for answers!
> Karl Daggfeldt[/color]

re: Memory limit?

"KalleD" <KalleD@discussions.microsoft.com> wrote in message
news:D05F7A58-F4F2-4BB4-A43F-0C3B9681DD2B@microsoft.com...[color=blue]
> Hi, I need to use a lot of memory in an aplication. But I get
> OutOfMemoryException whenever I try to allocate more than around 1.2GB.
> Ex:
> byte [] c=new byte[1300000000];
> My computer (win xp) have 4GB of RAM memory and it looks like 3GB is free
> (Performance Monitor->Available MB).
> Is there some way to use more memory for my application?
> Greatefull for answers!
> Karl Daggfeldt[/color]

The address space for your process is limited to 2GB, chance are that no
contiguous block of 1.3 GB is available in RAM due to fragmentation.
When running .NET v1.1, you can take advantage of the 4GT Ram tuning by
adding the /3GB switch to the Boot.ini file. Doing this gives you the
possibility to use 3GB of Ram for the process, of course this is no
guarantee for a successful allocation of such a big chunk, but at least
you'll have more memory that can be allocated.

Willy.

re: Memory limit?

Thank you all!
I think Adam and Will have the answer. I just have not been able to set the
linker setting LARGEADDRESSAWARE on my application. I find nowhere in visual
studio to do this. And I get an error (mspdb71.dll not found) when I try to
use the Link.exe program to make the setting (as suggested in Adams link). I
will look for what component I have to install to get Link working. Unless
anybody know how to make the setting in visual studio?
Karl Daggfeldt

"KalleD" wrote:
[color=blue]
> Hi, I need to use a lot of memory in an aplication. But I get
> OutOfMemoryException whenever I try to allocate more than around 1.2GB.
> Ex:
> byte [] c=new byte[1300000000];
> My computer (win xp) have 4GB of RAM memory and it looks like 3GB is free
> (Performance Monitor->Available MB).
> Is there some way to use more memory for my application?
> Greatefull for answers!
> Karl Daggfeldt[/color]

re: Memory limit?

"KalleD" <KalleD@discussions.microsoft.com> wrote in message
news:CBC8CD7D-4C77-4E83-BD0C-5F790243F22E@microsoft.com...[color=blue]
> Thank you all!
> I think Adam and Will have the answer. I just have not been able to set
> the
> linker setting LARGEADDRESSAWARE on my application. I find nowhere in
> visual
> studio to do this. And I get an error (mspdb71.dll not found) when I try
> to
> use the Link.exe program to make the setting (as suggested in Adams link).
> I
> will look for what component I have to install to get Link working. Unless
> anybody know how to make the setting in visual studio?
> Karl Daggfeldt
>[/color]
You can't and don't have to set LARGEADDRESSAWARE for C# applications. All
you need is Net v1.1 and the boot.ini file /3GB switch.
But again this won't garantee you to get a contiguous range of memory of
that size (1.3 GB).
All it brings you is a larger process space, but don't expect you can
allocate more than 1.5 - 1.6 GB in one chunk.

Willy.

re: Memory limit?

Hi Karl, Willy


| You can't and don't have to set LARGEADDRESSAWARE for C# applications.
All
| you need is Net v1.1 and the boot.ini file /3GB switch.


Actually, you can set this, and must if you want to take advantage of > 2GB
of user space:
http://msdn.microsoft.com/library/de...us/vcext/html/
vxlrfVCProjectEngineLibraryVCLinkerToolLargeAddres sAware.asp

| But again this won't garantee you to get a contiguous range of memory of
| that size (1.3 GB).
| All it brings you is a larger process space, but don't expect you can
| allocate more than 1.5 - 1.6 GB in one chunk.

Willy is exactly right. There may not be a chunk in memory of that size
available.

Karl, have you considered a more dynamic data structure, like an ArrayList?
That way the GC can allocate it in as many chunks as it needs.

-Chris

re: Memory limit?

Thanks Chris!
I will try using another memory configuration (but it will have to wait a
while because it means a lot of rewriting). For the moment I gave up on
seting LARGEADDRESSAWARE in visual studio (as instructed from your link). I
got "specified cast not valid" when running my aplication from the Macro
Explorer. Maybe I need c++ installed in order to make it work?

Anyway, after some more reading and trying I was able to set the
LARGEADRESSAWARE from the comand line. For the benifit of anybody
experiencing the same problem I give your the steps I used to make it work on
my system (probably not the most elegant way);

4GB ram needed

1. Add /3GB switch to boot.ini (Start->Control
Panel->System->Advanced->System and recovery Settings->Edit)

Example (2 options at startup):
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Micro soft Windows XP
Professional" /fastdetect /NoExecute=OptIn
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Micro soft Windows XP
Professional 3GB" /fastdetect /NoExecute=OptIn /3GB

2.In order to be able to build from the comand line, run VSVARS32.BAT (in
the visual studio subdirectory: Common7\Tools)

3. Link the program with the LARGEADDRESSAWARE option using the editbin
program;
editbin /LARGEADDRESSAWARE myApp.exe
(first move myApp.exe to the Common7\Tools subdirectory and run the comand
from there)

Thanks again everybody for contributing!
Karl



""Chris Lyon [MSFT]"" wrote:
[color=blue]
> Hi Karl, Willy
>
>
> | You can't and don't have to set LARGEADDRESSAWARE for C# applications.
> All
> | you need is Net v1.1 and the boot.ini file /3GB switch.
>
>
> Actually, you can set this, and must if you want to take advantage of > 2GB
> of user space:
> http://msdn.microsoft.com/library/de...us/vcext/html/
> vxlrfVCProjectEngineLibraryVCLinkerToolLargeAddres sAware.asp
>
> | But again this won't garantee you to get a contiguous range of memory of
> | that size (1.3 GB).
> | All it brings you is a larger process space, but don't expect you can
> | allocate more than 1.5 - 1.6 GB in one chunk.
>
> Willy is exactly right. There may not be a chunk in memory of that size
> available.
>
> Karl, have you considered a more dynamic data structure, like an ArrayList?
> That way the GC can allocate it in as many chunks as it needs.
>
> -Chris
>
>[/color]

re: Memory limit?

""Chris Lyon [MSFT]"" <clyon@online.microsoft.com> wrote in message
news:1XrwsAiNFHA.2632@TK2MSFTNGXA03.phx.gbl...[color=blue]
> Hi Karl, Willy
>
>
> | You can't and don't have to set LARGEADDRESSAWARE for C# applications.
> All
> | you need is Net v1.1 and the boot.ini file /3GB switch.
>
>
> Actually, you can set this, and must if you want to take advantage of >
> 2GB
> of user space:
> http://msdn.microsoft.com/library/de...us/vcext/html/
> vxlrfVCProjectEngineLibraryVCLinkerToolLargeAddres sAware.asp
>[/color]
Karl, Chris,

My bad, I thought all managed executable images were largeaddress aware
since v1.1.

Willy.

re: Memory limit?

Thanks for the info Karl.

I have some question.
I extended the memory to 4GB and to apply all the setting as you described
(to the Boot.ini and to my application etc. ).

But the Windows XP Pro OS report only on 3.25 BG instead of the 4GB (the
BIOS does see the 4GB), and I think that this is why my app fail to get more
then 1.2 GB of memory.

Can you tell me in your case what is the amount of memory the Windows OS
reports on and what version of windows OS are you using?


--------
Thanks
Sharon

re: Memory limit?

"Sharon" <SharonG@newsgroups.nospam> wrote in message
news:2163C355-8ADD-49BC-92AC-C73A121AE969@microsoft.com...[color=blue]
> Thanks for the info Karl.
>
> I have some question.
> I extended the memory to 4GB and to apply all the setting as you described
> (to the Boot.ini and to my application etc. ).
>
> But the Windows XP Pro OS report only on 3.25 BG instead of the 4GB (the
> BIOS does see the 4GB), and I think that this is why my app fail to get
> more
> then 1.2 GB of memory.
>
> Can you tell me in your case what is the amount of memory the Windows OS
> reports on and what version of windows OS are you using?
>
>
> --------
> Thanks
> Sharon[/color]

As I told you before, Windows XP can NEVER see the whole 4GB of RAM, the
reason for this is that the BIOS maps the hardware devices memory into the
upper range of the 4GB addressable range.
Start "Computer management" , select the Device Manager and watch your
device resources used like Mother boards and Video adapters, you will see
the memory ranges are mapped into the 4GB address range.
For instance if you have a graphics card with 128MB memory, it will probably
be mapped at D8000000-DFFFFFFF, if you have video adapter with 256 RAM, it
will probably be mapped at D0000000-DFFFFFFF.
Other devices like PCI bus and System boards have similar memory mappings,
that means that a part of the 4GB memory space is not usable (not seen) by
the software on a 32 bit OS, in your case its the range above 3.25 GB
(D0000000-FFFFFFFF).

2. The reason that you can't use more than 1.2 GB of virtual memory has
nothing to do with (1). The reason is virtual memory space fragmentation.
Even with /3GB enabled, your VM is fragmented such that, while there might
be 2GB or more free memory , it's not guaranteed that this will be in a
single block.
In general, for a small .NET program, the largest free CONTIGIOUS memory
block (at program start), is something like ~1.4GB, the second largest being
~275 MB. For a /3GB enabled process, the above figures are the same except
that you have an extra free block of ~1GB (above 2GB). As I told you before,
on a 32 bit OS you should never assume to have such large blocks of free
memory, that is, you should never create array's of that size, it will fail.

Willy.