c# definitive explantion of dotnet JIT compilation please and how to speed up performace

[Peted]

Hi,

im hoping someone cane provide or point to a definitive accurate
explantion of dotnet compilation when run and the best way to optimise
peformace when dotnet code is run first time and execution speed after
compliation.

Ive searched the net and seem to find various opinions on what
actually happens when you compile and run a c# dotnet application.
What im looking for is basicly how to make dotnet dlls load as fast as
possible and run as fast as possible

Eg

what exactly can ngen.exe do for speed and loading improvments

ive read with dotnet code is only compiled when methods are actually
called which would imply that everyone you start and app you can can
take a performance hit everytime a new piece of code is executed for
the first time.

In addition to that this all revolves around improving the performace
of a dotnet c# module hosted under a delphi win32 frame, with messages
passing between the two and the delphi host trapping some key events
in the c# module it is hosting.

The c# module is a sql database accessing module
Any advice or thoughts would be appreciated

Peted

[Peter Duniho]

On Thu, 24 Jul 2008 20:56:40 -0700, <Petedwrote:

[...]
what exactly can ngen.exe do for speed and loading improvments

ive read with dotnet code is only compiled when methods are actually
called which would imply that everyone you start and app you can can
take a performance hit everytime a new piece of code is executed for
the first time.

In Java, the run-time will interpret infrequently used code, and compile
in the background code that it detects is used frequently. When the
compiler is done compiling a given piece of code, then the run-time uses
the compiled version instead of interpreting it. This is sort of like the
"when methods are actually called" that you describe.

But as far as I know, in C# all of the code in an assembly is compiled
when the assembly is first loaded. The "global assembly cache" is, if I
recall correctly, where the compiled code is then stored for future
executions. You can pre-compile using ngen.exe and then somehow have your
compiled code added to the GAC (never done it myself, so don't know the
specifics, though looking at the docs it seems that perhaps ngen.exe
itself will do this) so that it's already there when you run.

Reading the MSDN discussion of ngen.exe, it looks to me as though
pre-compiling your code is a win mainly when you have code that's shared
among several different executables running at the same time. When this
happens, each process can use the same native code; otherwise, each
executable would wind up with its own copy of the native code. There is
also a memory usage reduction when the JIT compiler doesn't have to be
loaded.

There's also a theoretical improvement in loading times when the code
doesn't have to be compiled, as well as if the code is being shared among
processes and is already loaded. But I would expect those to be
relatively uncommon scenarios (of course, if you're one of the people in
those uncommon scenarios, they wind up being more important :) ).

Personally, I wouldn't bother unless I started to run into a measurable
performance or memory consumption issue that could be definitively fixed
by using ngen.exe. After all, just how often does your code get executed
for the first time? :)

Pete

[Peter Duniho]

Okay, did some reading, I got at least two things wrong... :(

On Thu, 24 Jul 2008 21:30:17 -0700, Peter Duniho
<Np*********@nnowslpianmk.comwrote:

[...]
But as far as I know, in C# all of the code in an assembly is compiled
when the assembly is first loaded. The "global assembly cache" is, if I
recall correctly, where the compiled code is then stored for future
executions.

As near as I can tell, the GAC is only for ngen-ed code. I saw a
reference to other JIT-compiled code being cached, but it implied that
unless you specifically install the code in the GAC, it winds up in a
non-global cache.

Also, it looks to me as though the JIT compiler only compiles code when
it's first called, not when it's first loaded. So for a given assembly,
if a method in the assembly is never called, it's never compiled. So this
is in fact similar (but not identical) to the Java paradigm, in the sense
that code is left uncompiled until it's been used some number of times
(where for .NET that number is 1).

All that said, I still don't see anything in your original question that
suggests that pre-compiling your assemblies would solve your performance
problems. For the benefit of more knowledgeable people who might be able
to offer practical advice (i.e. someone other than me :) ), you might want
to elaborate on that part of your question.

Pete

[Marc Gravell]

Points to improve initial load time:

* note that 3.5 SP1 (later this year) includes some runtime-level
tricks for cold-start performance
* look carefully at what code actually executes during startup; you
might be able to "async" some of it and display something (incomplete)
sooner, and fill in the gaps as other code completes; in reality it
will take *longer*, but perception is everything...
* consider, where appropriate, tools like ngen to reduce JIT; I've
never used it myself... JIT performance has never been a significant
problem
* consider, where appropriate, tools like ilmerge to reduce the amount
of "fusion" time; "fusion" is resolving and loading the various
assemblies needed for your code as it runs

Points to improve execution performance:

* profile; find out what the code is spending most of the time on
* treat external data (network [web / database]) as suspect (high
latency)
* can any data access / data processing be done async?
* look carefully at your list etc data-structuers; sometimes swapping
a list for a dictionary, or finding a smart way to remove a nested
loop, can be the life saver; note that for short lists (say, <150, but
do your own measures), a simple list or array can be *more* efficient
than a dictionary : in one recent example swapping a dictionary for a
flat sorted array increased performance by a factor of 10 [but you
need to *measure* first]

And tons more... performance is a complex beast...

Marc

[Pavel Minaev]

On Jul 25, 8:30*am, "Peter Duniho" <[e-mail]>
wrote:

But as far as I know, in C# all of the code in an assembly is compiled *
when the assembly is first loaded.

Nope, it's still JITted, though unlike Java which tries to postpone
this (i.e. it will first run the method through interpreter several
times before attempting the more expensive JITting), CLR will JIT
immediately on the first call. But not on assembly load.

Here's an article that covers it in more detail (among other things):

http://msdn.microsoft.com/en-us/magazine/cc163791.aspx

It actually explains the detailed layout of object vtables and format
JITter thunks. The latter is mostly applicable to 2.0 runtime as well
- I've successfully used the information in the article to patch those
thunks at runtime.

>*The "global assembly cache" is, if I recall correctly, where the compiled code is then stored for future *
executions.

Not really. NGen does indeed store precompiled assemblies in GAC, but
GAC also stored plain assemblies as well. It's whatever you put there
using gacutil.

[Peter Duniho]

Among all the smart things Marc had to say on the topic, I'd like to
emphasize one above all others...

On Thu, 24 Jul 2008 23:15:33 -0700, Marc Gravell <ma**********@gmail.com>
wrote:

[...] but perception is everything...

He includes this only as an effective footnote in a point under "initial
load time", but IMHO it's one of the most important performance points to
understand in all situations.

In extreme cases, any random user is going to be able to tell the
difference. For example, if you've got an implementation that takes an
hour, and one that takes five minutes, obviously the five minute run-time
is more desirable. So it's not that actually improving throughput isn't
important. It is, very much.

But in many cases, the overall performance difference is one of relatively
small percentage differences. Often less than 10% but I'd say a less than
20% difference describes a very large proportion of scenarios, once you've
eliminated all the really obvious speed-ups (i.e. no dumb code).

A user very often will not be able to notice the difference, and an
application that is responsive will always _feel_ faster than one that's
not, even if it takes longer to do what it's doing.

Marc already gave a good example in his notes, but I'll offer another from
the game development world: game frame rate. Lots of so-called
"hard-core" gamers fixate on the FPS meter, but in truth, unless you're
actually looking at that speed measurement, smoothness of frame rate has a
lot more to do with perception than the actual frame rate.

A game that's doing 100 fps most of the time, but bogs down to 60 fps
occasionally, is going to seem worse than a game that's just always doing
60 fps. Obviously there's a point at which running the game at the lowest
guaranteed frame rate doesn't work; if you're struggling just to get 20
fps, and occasionally you drop to 5 fps, you definitely don't want to be
doing 5 fps all the time. But at faster speeds, smoothness takes priority.

Some of the most important "performance improvements" I've seen over the
years had nothing to do with making the total execution time faster. :)

Pete

[Peter Morris]

Also, it looks to me as though the JIT compiler only compiles code when

it's first called, not when it's first loaded.

You saved me from correcting you :-) If it were done when loaded it would
probably be called the "dont worry I did it ages ago compiler" rather than
the "just in time compiler" :-)

Pete

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Pavel Minaev wrote:

On Jul 25, 8:30 am, "Peter Duniho" <[e-mail]>
wrote:

>But as far as I know, in C# all of the code in an assembly is compiled
when the assembly is first loaded.

Nope, it's still JITted, though unlike Java which tries to postpone
this (i.e. it will first run the method through interpreter several
times before attempting the more expensive JITting), CLR will JIT
immediately on the first call. But not on assembly load.

And the granularity is method not assembly.

Arne