Memeory limitation problems ???

[Sharon]

I Have a PC with dual XEON CPU’s and 4 Giga Byte RAM win Windows XP Pro.
I have 2 problems with it:
(1) Windows is showing only 3.25 Giga byte on the System Properties ?
General tab.
While the bios to see all of the 4 GBytes.
I some article I read that the WinXP Pro can only handle 4GBytes of memory
including the virtual memory, so I set it to not use any page file, but it
did not help.
(2) I want my .NET application to use more then 1.2 GByte of memory, so I
insert at the Boot.ini file the /3GB flag and did reboot. Now I also need to
add to my application linker the /LARGEADDRESSAWA RE flag for it to be
IMAGE_FILE_LARG E_ADDRESS_AWARE (see
http://www.microsoft.com/whdc/system...E/PAEmem.mspx). But I
find no where to put the flag in the project properties.

Can anybody tell me how solve this 2 issues?
--------
Thanks
Sharon

[Sharon]

Hi Willy,

Thanks for your replay.

(1) I found a discussions about the same issue:
http://msdn.microsoft.com/newsgroups...f-0c3b9681dd2b
and it seems to succeed with the 4Mbytes RAM. So I very much need to do it.
Also, the Windows Xp 32 bit Pro documentation indicate that it can consume
up to 4MB.
The BIOS does recognize the extended RAM as 4,192,768 bytes. so I really
need this.
(2) About the drawback - Don't you think that 1GB of RAM should be enough
for the kernel?
---------
Thanks
Sharon

[Willy Denoyette [MVP]]

"Sharon" <Sh*****@newsgr oups.nospam> wrote in message
news:F3******** *************** ***********@mic rosoft.com...

Hi Willy,

Thanks for your replay.

(1) I found a discussions about the same issue:
http://msdn.microsoft.com/newsgroups...f-0c3b9681dd2b
and it seems to succeed with the 4Mbytes RAM. So I very much need to do
it.
Also, the Windows Xp 32 bit Pro documentation indicate that it can consume
up to 4MB.
The BIOS does recognize the extended RAM as 4,192,768 bytes. so I really
need this.
(2) About the drawback - Don't you think that 1GB of RAM should be enough
for the kernel?
---------
Thanks
Sharon

1) A windows application is limitted to 2GB or 3GB of virtual address space
depending on the set-up.
If you think you need 3GB you have to set the /3GB switch in boot.ini and
you need to link your application with the /LARGEADDRESSAWA RE switch set.
..NET managed application not linked, here you have to pass the exe file to
editbin.exe with the /LARGEADDRESSAWA RE switch set (editbin.exe will set the
IMAGE_FILE_LARG E_ADDRESS_AWARE bit in the PE header).

Not sure what you mean with "I really need this".
- Your program will never be able to use 4GB on a 32 bit windows OS. And you
will never be able to use 4GB RAM unless you run a 64 bit windows version
with "memory remapping" enabled (in the BIOS or software).
- Your application will never be able to allocate a single array larger than
~1.5GB, if this is what you want forget this whole /3GB thing, your only
option is Windows XP64 or W2K3 64.

2) I don't know it depends on so many things, it depends on the number and
type of HW devices, it depends on the software running, the number of
processes number of threads etc.
Why do you think 1GB is more than enough for the kernel, while 2GB is not
enough for your single application?

Some KB articles you should read carefully before deciding to go down that
road.

http://support.microsoft.com/default...b;en-us;319043
http://support.microsoft.com/kb/316739/EN-US/

Willy.

[Sharon]

Hi Willy,

Thanks a lot for your prompt replay.

(1) I already did all that is necessary for my application to use more then
2GB and it's working fine now.

When I say "I really need this", I mean that I need the Windows operation
system to recognize all the 4GB installed.

(2) I think that the Windows kernel will run nicely on a 1GB RAM because my
PC is a dedicated PC and I can tell what will run on it, so as I see it 1GB
seems to be fine, after all; Windows XP Pro can run on a 256 MB PC's as well.

After reading the MSDN KB you mentioned, I can see that is for Windows 2003
servers (one of the KB) and the other is saying that it may happen, but it is
not happening in my case as everything is running OK after I applied the /3GB
switch.
So now the only trouble I have is how to make the Windows XP Pro 32 bit with
SP2 to see all of the 4GB installed and not only 3.25 GB ?
--------
Thanks
Sharon

[Willy Denoyette [MVP]]

"Sharon" <Sh*****@newsgr oups.nospam> wrote in message
news:22******** *************** ***********@mic rosoft.com...

Hi Willy,

Thanks a lot for your prompt replay.

(1) I already did all that is necessary for my application to use more
then
2GB and it's working fine now.

When I say "I really need this", I mean that I need the Windows operation
system to recognize all the 4GB installed.

(2) I think that the Windows kernel will run nicely on a 1GB RAM because
my
PC is a dedicated PC and I can tell what will run on it, so as I see it
1GB
seems to be fine, after all; Windows XP Pro can run on a 256 MB PC's as
well.

After reading the MSDN KB you mentioned, I can see that is for Windows
2003
servers (one of the KB) and the other is saying that it may happen, but it
is
not happening in my case as everything is running OK after I applied the
/3GB
switch.
So now the only trouble I have is how to make the Windows XP Pro 32 bit
with
SP2 to see all of the 4GB installed and not only 3.25 GB ?

As I told you previously, the Windows OS CAN see/access most of the 4GB RAM
space, but 'you' can't see all the 4GB RAM, part of this (above 3.25GB in
your case) is taken by the hardware mapping done by the system BIOS, for
instance your video RAM (depending on the size) must be mapped into that 4GB
address space to be addressable by a 32 bit OS. A 64 bit system running a 64
bit OS can remap this area (some chipsets do not !!!, so please check your
systems BIOS to see whether this is possible) to some address outside the
installed RAM space, but this same system running a 32 bit OS cannot, simply
because a 32 bit OS cannot address anything above 4GB.

You remark on point 2 make little sense, you need to make a clear
destinction between RAM space and virtual memory (that is RAM + pagefile
space). A 256 MB RAM system will have a hard time to run anything
significant without a pagefile, don't even try to run a DirectX or OpenGL
application on such system with a 128/256MB RAM video card installed, it
wont work (even not with a 500MB pagefile).
And don't forget that a system with 4GB installed RAM will have a 16 times
larger page table (kernel space) than a 256 MB system.

Willy.

[Sharon]

I must say it looks a too much memory for only mapping. It's also mean that
the Windows XP OS can not really use 4GB but less. But how much less? 3.25GB
looks like a very much less.

Furthermore, I extended the memory to 4GB in order to allow to my
application to use up to 3GB. But if the OS sees only 3.25GB it will not let
my application use the amount of memory I want it to use.

So my final question is which is very important to me:

In the case I described (4GB RAM that shown as a 3.25GB), does my
application can use up to 3GB (after applying the /3GB in the Boot.ini and
adding the IMAGE_FILE_LARG E_ADDRESS_AWARE flag to my app image header)?
--------
Thanks
Sharon

[Willy Denoyette [MVP]]

"Sharon" <Sh*****@newsgr oups.nospam> wrote in message
news:BD******** *************** ***********@mic rosoft.com...

I must say it looks a too much memory for only mapping. It's also mean that
the Windows XP OS can not really use 4GB but less. But how much less?
3.25GB
looks like a very much less.
No, Windows XP 32 cannot use the full 4GB RAM space, because the hardware
components memory must be mapped in this address space, the amount of mapped
memory space depends on the type and number of controllers, expecially video
controllers can take a large amount of this (there are Nvidia boards with
512MB memory on board) and they have to be aligned on a 256MB boundary. In
your case it looks like you have a 256MB video board or at least a board
that is mapped at that address (0XD0000000).
Furthermore, I extended the memory to 4GB in order to allow to my
application to use up to 3GB. But if the OS sees only 3.25GB it will not
let
my application use the amount of memory I want it to use.
Why? 3.25 GB is more than 3GB isn't it? Plus, you don't need 3GB RAM for an
application to consume 3GB of memory, please read my other reply where I
explain the difference between virtual and physical memory.
So my final question is which is very important to me:

In the case I described (4GB RAM that shown as a 3.25GB), does my
application can use up to 3GB (after applying the /3GB in the Boot.ini and
adding the IMAGE_FILE_LARG E_ADDRESS_AWARE flag to my app image header)?

Well you have to be more specific, 3GB is the max. space on a system that
has /3GB enabled and IMAGE_FILE_LARG E_ADDRESS_AWARE set for the application,
BUT that doesn't mean your program code can allocate all 3GB, first you have
your code and the runtimes mapped in this address space, you have the space
reserved by the stacks (1MB per thread) and you have holes in this address
space that can't be used - a 64 KB hole at the start (address range 0-FFFF)
and a hole before the end of the 2GB range (7FFF0000-7FFF0000). And you have
to consider the mapping of the DLL's (code) into that address space (some
are mapped at a very unfortunate location). All these mappings and holes
result in a fragmented virtual memory space,so you don't have one single
large chunk of free memory, no you have several chunks from which the
largest is ~1.4 GB for a minimal .NET application.
That means that at best, the largest array that can be allocated is ~1.4GB
(but that's never guaranteed!!!), allocating four arrays of 0.5 GB plus a
couple of 100MB arrays on a /3GB system, should mostly succeed, but again
this is never guaranteed, VM can become framgmented over time.

One last suggestion, if you absolutely need to allocate a single array
that's larger than say 1GB, you should consider moving to Windows XP 64,
here you can remap the HW to addresses above 4GB and the VM address space is
4GB for 32 bit applications.
Note that whatever OS you are using, the max. size of an array in .NET is
2GB (the max size of whatever CLR object).
Willy.

[Sharon]

OK, I see what you mean.

Allow to me to explain the real problem and requirements that led to the
trouble at hand;

I'm developing a project that need to take large image at a time, load it
all into memory and do some image processing.
The images are of type TIF file 62992 x 113386 Pixels, Huffman RLE
compression, 3200 x 3200 DPI resolution, binary colored (1 Bit Per Pixel),
file on disk size 43.08 MB (45,169,042 Bytes).
This kind of image should consume 851.66 MB (893,028,184 Bytes) of memory
when loaded (this is what it takes using the IrfanView
http://www.irfanview.com/) and a project at
http://www.codeproject.com/bitmap/graphicsuite.asp), and this is exactly what
I want it to consume. Otherwise; the representation per pixel will be 1 byte
(8 Bits), which means that this kind of image require 6.652 GB (7,142,410,912
Bytes) that I absolutely can not have in any possible way.

I found that the Bitmap object can hold 1 bit per pixel, but for some
reason, it can not handle this kind of picture and throws ArgumentExcepti on
saying
"Invalid parameter used". But if I load the image using
Image.FromFile( imageFilePath) it throw en OutOfMemoryExce ption saying "Out Of
Memory" and it's because it consume twice the memory it should (not 8 times
the memory).

So, in order to meet with this requirements we took a dual XEON PC with 4GB
memory and set the Boot.ini with the /3GB and linked our application with the
IMAGE_FILE_LARG E_ADDRESS_AWARE flag to my application image header. Yet it
still fails.

So, we are doing the image processing algorithms using unmanaged C++, but
yet, I need to load a high resolution sample of the image I mentioned above
by loading it to a Bitmap object and reduce it size to loaded to a PictureBox
control.
I already have a very good code for resizing the image, but it required a
Bitmap object, and it fails to load this image as I described above.

Now you have the bigger picture to understand the actual problem.

In this stage, using Windows 64 bit, is not an option, but it's the only
solution, we may use it.

Do I need to change something in my code on in my assembly to use a 64 bit
Windows system?
Can I still work with the same PC hardware (CPU's, devices etc.)?
What do you think?
--------
Thanks
Sharon

[Sharon]

I have some more info:

At the PC device driver a saw that the PC has ASUS Extreme graphics adapter
with 128 MB and another ASUS graphics adapter with 128 MB as Secondary
(memory range D8000000-DFFFFFFF, D0800000-D080FFFF, 000A0000-000BFFFF). The
PC also installed with LAN and GLAN, and a Ceroco Imaging device X64-CL_iPro
Board (memory range D0400000-D07FFFFF).

I have posted another question on the win32.programme r.kernel Discussions, I
the answer I gat is very confusing
http://msdn.microsoft.com/newsgroups...5-ad91520ac866
I actually say that it does see the installed 4GB.

What do you think?

--------
Thanks
Sharon

[Willy Denoyette [MVP]]

See inline

Willy.

"Sharon" <Sh*****@newsgr oups.nospam> wrote in message
news:DB******** *************** ***********@mic rosoft.com...

OK, I see what you mean.

Allow to me to explain the real problem and requirements that led to the
trouble at hand;

I'm developing a project that need to take large image at a time, load it
all into memory and do some image processing.
The images are of type TIF file 62992 x 113386 Pixels, Huffman RLE
compression, 3200 x 3200 DPI resolution, binary colored (1 Bit Per Pixel),
file on disk size 43.08 MB (45,169,042 Bytes).
This kind of image should consume 851.66 MB (893,028,184 Bytes) of memory
when loaded (this is what it takes using the IrfanView
http://www.irfanview.com/) and a project at
http://www.codeproject.com/bitmap/graphicsuite.asp), and this is exactly
what
I want it to consume. Otherwise; the representation per pixel will be 1
byte
(8 Bits), which means that this kind of image require 6.652 GB
(7,142,410,912
Bytes) that I absolutely can not have in any possible way.

I found that the Bitmap object can hold 1 bit per pixel, but for some
reason, it can not handle this kind of picture and throws
ArgumentExcepti on
saying
"Invalid parameter used". But if I load the image using
Image.FromFile( imageFilePath) it throw en OutOfMemoryExce ption saying "Out
Of
Memory" and it's because it consume twice the memory it should (not 8
times
the memory).

This is how GDI+ works (wrapped by the System.Drawing) , Image.FromFile
instructs GDI+ to load the file as a memory mapped object and from that it
builds an imageobject (fi. a Bitmap). That means at some point in time you
have the file data and the bitmap data allocated in unmanaged memory (yes,
they are unmanaged resource), this roughly represents two times the file
data. The same thing is done using IrfanView or any other unmanaged tool,
what's different is that once the Bitmap is built, the file is closed and
the mapped memory released (you can check this with perfmon), while when
using .NET the resources are released when you call Dispose on the image.
So, if it works with the tools you mentioned, it should work with .NET also.
Did you try by simply loading the tiff file using a small console program.

public static int Main()
{
Image i = Image.FromFile( @"YourtTiffFile ");
}

So, in order to meet with this requirements we took a dual XEON PC with
4GB
memory and set the Boot.ini with the /3GB and linked our application with
the
IMAGE_FILE_LARG E_ADDRESS_AWARE flag to my application image header. Yet it
still fails.

So, we are doing the image processing algorithms using unmanaged C++, but
yet, I need to load a high resolution sample of the image I mentioned
above
by loading it to a Bitmap object and reduce it size to loaded to a
PictureBox
control.
Why don't you load and reduce it's size using C++ as well?
I already have a very good code for resizing the image, but it required a
Bitmap object, and it fails to load this image as I described above.

Now you have the bigger picture to understand the actual problem.

In this stage, using Windows 64 bit, is not an option, but it's the only
solution, we may use it.

Do I need to change something in my code on in my assembly to use a 64 bit
Windows system?
Can I still work with the same PC hardware (CPU's, devices etc.)?

Don't know for sure, it depends on the exact CPU type (should be 64 bit
capable) and other devices. Anyway, you should try this to be sure. I guess
you can obtain an evaluation versions of W2K3 64bit or XP64 from MS.