How to use a huge memory in C#?

[fAnSKyer/C# newbie]

My system has 4GB memory and My program in C# is really memory
consuming. and I noticed that when the memory I used is more than 2GB I
will get an exception.

How to solve this? thanks a lot

[Greg Young]

That is a windows issue not a C# issue ..There is a 2gb process limit in
windows ..

see
http://msdn.microsoft.com/library/de...netchapt17.asp
search for /3 gb for instructions for how to make this larger.

Cheers,

Greg Young
MVP - C#
http://codebetter.com/blogs/gregyoung

"fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
news:11******** **************@ m7g2000cwm.goog legroups.com...

My system has 4GB memory and My program in C# is really memory
consuming. and I noticed that when the memory I used is more than 2GB I
will get an exception.

How to solve this? thanks a lot

[Willy Denoyette [MVP]]

"fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
news:11******** **************@ m7g2000cwm.goog legroups.com...
| My system has 4GB memory and My program in C# is really memory
| consuming. and I noticed that when the memory I used is more than 2GB I
| will get an exception.
|
| How to solve this? thanks a lot
|

Why does your program uses that much memory? Is it by design or are you
chasing a bug?
If it's the first, you'll have to change your design or move to a 64 bit OS,
if it's a bug, you will have to profile your application.

Willy.

[Chris Mullins]

I'm pretty sure .Net doesn't respects the /3gb switch.

I haven't looked in a while, but I know we did quite a bit of performance
tuning in 32-bit land a while back and were very crabby about this.

Fortunatly 64-bit hardware came along, and life was good. I love allocating
20+gb of memory! :)

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins

"Greg Young" <dr************ *******@hotmail .comwrote in message
news:eV******** ******@TK2MSFTN GP05.phx.gbl...

That is a windows issue not a C# issue ..There is a 2gb process limit in
windows ..

see
http://msdn.microsoft.com/library/de...netchapt17.asp
search for /3 gb for instructions for how to make this larger.

Cheers,

Greg Young
MVP - C#
http://codebetter.com/blogs/gregyoung

"fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
news:11******** **************@ m7g2000cwm.goog legroups.com...

>My system has 4GB memory and My program in C# is really memory
consuming. and I noticed that when the memory I used is more than 2GB I
will get an exception.

How to solve this? thanks a lot

[Chris Mullins]

You're not hitting .Net limits, you're hitting 32-bit Windows Process
limits. In 32-bit land, you're pretty much stuck.

You're going to need to go buy 64-bit hardware, install a 64-bit O/S (XP,
Win2k3, Vista, Longhorn), and run your application as a 64-bit app.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins

"fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
news:11******** **************@ m7g2000cwm.goog legroups.com...

My system has 4GB memory and My program in C# is really memory
consuming. and I noticed that when the memory I used is more than 2GB I
will get an exception.

How to solve this? thanks a lot

[Willy Denoyette [MVP]]

Sure it does, .NET (the CLR) is LARGEADRESSAWAR E since v1.1. The only
problem is that the C# compiler doesn't have an option to set the
largeaddressawa re flag, you have to edit the PE header using editbin.

editbin /largeaddressawa re <some.exe>
Another option (framework V2)is to compile all modules as netmodules using
the /t:module compiler flag and use the linker (link.exe) to build the PE
file.
here is a sample of such linker command...

link /entry:someNames pace.someClass. Main /subsystem:conso le
/largeaddressawa re some.netmodule someother.netmo dule

in both cases you should see something like following when running dumpbin

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
3 number of sections
452C1EAA time date stamp Wed Oct 11 00:28:58 2006
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
122 characteristics
Executable
Application can handle large (>2GB) addresses
32 bit word machine
Willy.

"Chris Mullins" <cm******@yahoo .comwrote in message
news:uv******** ******@TK2MSFTN GP02.phx.gbl...
| I'm pretty sure .Net doesn't respects the /3gb switch.
|
| I haven't looked in a while, but I know we did quite a bit of performance
| tuning in 32-bit land a while back and were very crabby about this.
|
| Fortunatly 64-bit hardware came along, and life was good. I love
allocating
| 20+gb of memory! :)
|
| --
| Chris Mullins, MCSD.NET, MCPD:Enterprise
| http://www.coversant.net/blogs/cmullins
|
|
|
| "Greg Young" <dr************ *******@hotmail .comwrote in message
| news:eV******** ******@TK2MSFTN GP05.phx.gbl...
| That is a windows issue not a C# issue ..There is a 2gb process limit in
| windows ..
| >
| see
| >
http://msdn.microsoft.com/library/de...netchapt17.asp
| search for /3 gb for instructions for how to make this larger.
| >
| Cheers,
| >
| Greg Young
| MVP - C#
| http://codebetter.com/blogs/gregyoung
| >
| "fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
| news:11******** **************@ m7g2000cwm.goog legroups.com...
| >My system has 4GB memory and My program in C# is really memory
| >consuming. and I noticed that when the memory I used is more than 2GB I
| >will get an exception.
| >>
| >How to solve this? thanks a lot
| >>
| >
| >
|
|

[Chris Mullins]

Sure enough. A quick Google Search agrees with you.

I wonder if the last time I tried was v1.0? I don't think it's been that
long.

I do remember at that time (we were doing extensive profiling), we wanted to
host our Server using the Server CLR, not the Workstation CLR. I needed to
write a C++ hosting exe to do this. If I remember right, that means it must
have been 1.0, since in 1.1 CLR selection was a config file entry.

Ah well. I'm probably just confused. We've been 64-bit for a while now, so
this just doesn't come up for me anymore. Anytime a customer asks "how big
can it scale?" we just give out the 64-bit numbers.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins
"Willy Denoyette [MVP]" <wi************ *@telenet.bewro te in message
news:eP******** ******@TK2MSFTN GP03.phx.gbl...

Sure it does, .NET (the CLR) is LARGEADRESSAWAR E since v1.1. The only
problem is that the C# compiler doesn't have an option to set the
largeaddressawa re flag, you have to edit the PE header using editbin.

editbin /largeaddressawa re <some.exe>
Another option (framework V2)is to compile all modules as netmodules using
the /t:module compiler flag and use the linker (link.exe) to build the PE
file.
here is a sample of such linker command...

link /entry:someNames pace.someClass. Main /subsystem:conso le
/largeaddressawa re some.netmodule someother.netmo dule

in both cases you should see something like following when running dumpbin

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
3 number of sections
452C1EAA time date stamp Wed Oct 11 00:28:58 2006
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
122 characteristics
Executable
Application can handle large (>2GB) addresses
32 bit word machine
Willy.

"Chris Mullins" <cm******@yahoo .comwrote in message
news:uv******** ******@TK2MSFTN GP02.phx.gbl...
| I'm pretty sure .Net doesn't respects the /3gb switch.
|
| I haven't looked in a while, but I know we did quite a bit of
performance
| tuning in 32-bit land a while back and were very crabby about this.
|
| Fortunatly 64-bit hardware came along, and life was good. I love
allocating
| 20+gb of memory! :)
|
| --
| Chris Mullins, MCSD.NET, MCPD:Enterprise
| http://www.coversant.net/blogs/cmullins
|
|
|
| "Greg Young" <dr************ *******@hotmail .comwrote in message
| news:eV******** ******@TK2MSFTN GP05.phx.gbl...
| That is a windows issue not a C# issue ..There is a 2gb process limit
in
| windows ..
| >
| see
| >
http://msdn.microsoft.com/library/de...netchapt17.asp
| search for /3 gb for instructions for how to make this larger.
| >
| Cheers,
| >
| Greg Young
| MVP - C#
| http://codebetter.com/blogs/gregyoung
| >
| "fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
| news:11******** **************@ m7g2000cwm.goog legroups.com...
| >My system has 4GB memory and My program in C# is really memory
| >consuming. and I noticed that when the memory I used is more than 2GB
I
| >will get an exception.
| >>
| >How to solve this? thanks a lot
| >>
| >
| >
|
|

[fAnSKyer/C# newbie]

Thanks a lot for give me great help.
I will try the /3GB to see if it solve the problem

But I am quite curious that a 32-bit can search a 4GB memory, why the
bottleneck is 2GB?

This is not a bug, but I am wonder if I can do it better because I
write 2 memory consuming thread in one program. I may separate them to
two program and using .NET remoting to control these two to solve this?

Thanks again
fAnSKyer

Chris Mullins wrote:

Sure enough. A quick Google Search agrees with you.

I wonder if the last time I tried was v1.0? I don't think it's been that
long.

I do remember at that time (we were doing extensive profiling), we wanted to
host our Server using the Server CLR, not the Workstation CLR. I needed to
write a C++ hosting exe to do this. If I remember right, that means it must
have been 1.0, since in 1.1 CLR selection was a config file entry.

Ah well. I'm probably just confused. We've been 64-bit for a while now, so
this just doesn't come up for me anymore. Anytime a customer asks "how big
can it scale?" we just give out the 64-bit numbers.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins
"Willy Denoyette [MVP]" <wi************ *@telenet.bewro te in message
news:eP******** ******@TK2MSFTN GP03.phx.gbl...

Sure it does, .NET (the CLR) is LARGEADRESSAWAR E since v1.1. The only
problem is that the C# compiler doesn't have an option to set the
largeaddressawa re flag, you have to edit the PE header using editbin.

editbin /largeaddressawa re <some.exe>
Another option (framework V2)is to compile all modules as netmodules using
the /t:module compiler flag and use the linker (link.exe) to build the PE
file.
here is a sample of such linker command...

link /entry:someNames pace.someClass. Main /subsystem:conso le
/largeaddressawa re some.netmodule someother.netmo dule

in both cases you should see something like following when running dumpbin

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
3 number of sections
452C1EAA time date stamp Wed Oct 11 00:28:58 2006
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
122 characteristics
Executable
Application can handle large (>2GB) addresses
32 bit word machine
Willy.

"Chris Mullins" <cm******@yahoo .comwrote in message
news:uv******** ******@TK2MSFTN GP02.phx.gbl...
| I'm pretty sure .Net doesn't respects the /3gb switch.
|
| I haven't looked in a while, but I know we did quite a bit of
performance
| tuning in 32-bit land a while back and were very crabby about this.
|
| Fortunatly 64-bit hardware came along, and life was good. I love
allocating
| 20+gb of memory! :)
|
| --
| Chris Mullins, MCSD.NET, MCPD:Enterprise
| http://www.coversant.net/blogs/cmullins
|
|
|
| "Greg Young" <dr************ *******@hotmail .comwrote in message
| news:eV******** ******@TK2MSFTN GP05.phx.gbl...
| That is a windows issue not a C# issue ..There is a 2gb process limit
in
| windows ..
| >
| see
| >
http://msdn.microsoft.com/library/de...netchapt17.asp
| search for /3 gb for instructions for how to make this larger.
| >
| Cheers,
| >
| Greg Young
| MVP - C#
| http://codebetter.com/blogs/gregyoung
| >
| "fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
| news:11******** **************@ m7g2000cwm.goog legroups.com...
| >My system has 4GB memory and My program in C# is really memory
| >consuming. and I noticed that when the memory I used is more than 2GB
I
| >will get an exception.
| >>
| >How to solve this? thanks a lot
| >>
| >
| >
|
|

[Chris Mullins]

That's just the way Win32 works. This is why the move to 64 bit computing
has been so welcomed by database providers and other server application
authors.

For more detail:
http://msdn.microsoft.com/msdnmag/issues/0700/hood/
http://msdn.microsoft.com/library/de...sdn_virtmm.asp

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins

"fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
news:11******** *************@i 42g2000cwa.goog legroups.com...

Thanks a lot for give me great help.
I will try the /3GB to see if it solve the problem

But I am quite curious that a 32-bit can search a 4GB memory, why the
bottleneck is 2GB?

This is not a bug, but I am wonder if I can do it better because I
write 2 memory consuming thread in one program. I may separate them to
two program and using .NET remoting to control these two to solve this?

Thanks again
fAnSKyer

Chris Mullins wrote:

>Sure enough. A quick Google Search agrees with you.

I wonder if the last time I tried was v1.0? I don't think it's been that
long.

I do remember at that time (we were doing extensive profiling), we wanted
to
host our Server using the Server CLR, not the Workstation CLR. I needed
to
write a C++ hosting exe to do this. If I remember right, that means it
must
have been 1.0, since in 1.1 CLR selection was a config file entry.

Ah well. I'm probably just confused. We've been 64-bit for a while now,
so
this just doesn't come up for me anymore. Anytime a customer asks "how
big
can it scale?" we just give out the 64-bit numbers.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins
"Willy Denoyette [MVP]" <wi************ *@telenet.bewro te in message
news:eP******* *******@TK2MSFT NGP03.phx.gbl.. .

Sure it does, .NET (the CLR) is LARGEADRESSAWAR E since v1.1. The only
problem is that the C# compiler doesn't have an option to set the
largeaddressawa re flag, you have to edit the PE header using editbin.

editbin /largeaddressawa re <some.exe>
Another option (framework V2)is to compile all modules as netmodules
using
the /t:module compiler flag and use the linker (link.exe) to build the
PE
file.
here is a sample of such linker command...

link /entry:someNames pace.someClass. Main /subsystem:conso le
/largeaddressawa re some.netmodule someother.netmo dule

in both cases you should see something like following when running
dumpbin

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
3 number of sections
452C1EAA time date stamp Wed Oct 11 00:28:58 2006
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
122 characteristics
Executable
Application can handle large (>2GB) addresses
32 bit word machine
Willy.

"Chris Mullins" <cm******@yahoo .comwrote in message
news:uv******** ******@TK2MSFTN GP02.phx.gbl...
| I'm pretty sure .Net doesn't respects the /3gb switch.
|
| I haven't looked in a while, but I know we did quite a bit of
performance
| tuning in 32-bit land a while back and were very crabby about this.
|
| Fortunatly 64-bit hardware came along, and life was good. I love
allocating
| 20+gb of memory! :)
|
| --
| Chris Mullins, MCSD.NET, MCPD:Enterprise
| http://www.coversant.net/blogs/cmullins
|
|
|
| "Greg Young" <dr************ *******@hotmail .comwrote in message
| news:eV******** ******@TK2MSFTN GP05.phx.gbl...
| That is a windows issue not a C# issue ..There is a 2gb process
limit
in
| windows ..
| >
| see
| >
http://msdn.microsoft.com/library/de...netchapt17.asp
| search for /3 gb for instructions for how to make this larger.
| >
| Cheers,
| >
| Greg Young
| MVP - C#
| http://codebetter.com/blogs/gregyoung
| >
| "fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
| news:11******** **************@ m7g2000cwm.goog legroups.com...
| >My system has 4GB memory and My program in C# is really memory
| >consuming. and I noticed that when the memory I used is more than
2GB
I
| >will get an exception.
| >>
| >How to solve this? thanks a lot
| >>
| >
| >
|
|

[Greg Young]

This is a prime example of why multiple process is preferred in many cases
over multi-thread :)

Cheers,

Greg
"fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
news:11******** *************@i 42g2000cwa.goog legroups.com...

Thanks a lot for give me great help.
I will try the /3GB to see if it solve the problem

But I am quite curious that a 32-bit can search a 4GB memory, why the
bottleneck is 2GB?

This is not a bug, but I am wonder if I can do it better because I
write 2 memory consuming thread in one program. I may separate them to
two program and using .NET remoting to control these two to solve this?

Thanks again
fAnSKyer

Chris Mullins wrote:

>Sure enough. A quick Google Search agrees with you.

I wonder if the last time I tried was v1.0? I don't think it's been that
long.

I do remember at that time (we were doing extensive profiling), we wanted
to
host our Server using the Server CLR, not the Workstation CLR. I needed
to
write a C++ hosting exe to do this. If I remember right, that means it
must
have been 1.0, since in 1.1 CLR selection was a config file entry.

Ah well. I'm probably just confused. We've been 64-bit for a while now,
so
this just doesn't come up for me anymore. Anytime a customer asks "how
big
can it scale?" we just give out the 64-bit numbers.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins
"Willy Denoyette [MVP]" <wi************ *@telenet.bewro te in message
news:eP******* *******@TK2MSFT NGP03.phx.gbl.. .

Sure it does, .NET (the CLR) is LARGEADRESSAWAR E since v1.1. The only
problem is that the C# compiler doesn't have an option to set the
largeaddressawa re flag, you have to edit the PE header using editbin.

editbin /largeaddressawa re <some.exe>
Another option (framework V2)is to compile all modules as netmodules
using
the /t:module compiler flag and use the linker (link.exe) to build the
PE
file.
here is a sample of such linker command...

link /entry:someNames pace.someClass. Main /subsystem:conso le
/largeaddressawa re some.netmodule someother.netmo dule

in both cases you should see something like following when running
dumpbin

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
3 number of sections
452C1EAA time date stamp Wed Oct 11 00:28:58 2006
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
122 characteristics
Executable
Application can handle large (>2GB) addresses
32 bit word machine
Willy.

"Chris Mullins" <cm******@yahoo .comwrote in message
news:uv******** ******@TK2MSFTN GP02.phx.gbl...
| I'm pretty sure .Net doesn't respects the /3gb switch.
|
| I haven't looked in a while, but I know we did quite a bit of
performance
| tuning in 32-bit land a while back and were very crabby about this.
|
| Fortunatly 64-bit hardware came along, and life was good. I love
allocating
| 20+gb of memory! :)
|
| --
| Chris Mullins, MCSD.NET, MCPD:Enterprise
| http://www.coversant.net/blogs/cmullins
|
|
|
| "Greg Young" <dr************ *******@hotmail .comwrote in message
| news:eV******** ******@TK2MSFTN GP05.phx.gbl...
| That is a windows issue not a C# issue ..There is a 2gb process
limit
in
| windows ..
| >
| see
| >
http://msdn.microsoft.com/library/de...netchapt17.asp
| search for /3 gb for instructions for how to make this larger.
| >
| Cheers,
| >
| Greg Young
| MVP - C#
| http://codebetter.com/blogs/gregyoung
| >
| "fAnSKyer/C# newbie" <fa******@gmail .comwrote in message
| news:11******** **************@ m7g2000cwm.goog legroups.com...
| >My system has 4GB memory and My program in C# is really memory
| >consuming. and I noticed that when the memory I used is more than
2GB
I
| >will get an exception.
| >>
| >How to solve this? thanks a lot
| >>
| >
| >
|
|