XSLT - MemoryStream - Performance

[Trevor Oakley]

I am writing thousands of html pages from an MS SQL source using a
DataSet and then an XslTransform. I have an interest in making the
code run faster as it takes several minutes (sometimes ten minutes) to
generate a batch of pages (say two thousand).

If anyone has URL resources which performance information germane to
XSL Transforms, Streams, and XML I am interested in hearing about
them. I have written some notes below of what I am doing.
My Trace.Warns have shown that the time used is used on the
XslTransform (and associated classes).

Methods
=======

First Method: All XML docs were written to disk and the read in
locating via MapPath.

Second: All XML docs were written to MemoryStream and then the streams
are read directly into Xpath and then the transform. Traces show
internally this is much quicker (about five times) but the elapsed
time of the whole process is still slow. The DB access is very quick
(100 ms) and so is DataSet handling.

[Oleg Tkachenko [MVP]]

Trevor Oakley wrote:

I am writing thousands of html pages from an MS SQL source using a
DataSet and then an XslTransform. I have an interest in making the
code run faster as it takes several minutes (sometimes ten minutes) to
generate a batch of pages (say two thousand).

If anyone has URL resources which performance information germane to
XSL Transforms, Streams, and XML I am interested in hearing about
them.

Take a look at recently published "Improving XML Performance" chapter of
the "Improving .NET Application Performance and Scalability" guide -
http://msdn.microsoft.com/library/de...netchapt09.asp

I'd try to optimize XSLT first.

--
Oleg Tkachenko [XML MVP]
http://blog.tkachenko.com

[Morantex]

Hi Trevor

I have just read your post about performance concerns.

I want to draw your attention to a new technology we have recently made available in beta.

Persistore is the first managed class library for .NET 2 that provides access to persistent shared memory.

The technology lets you drag a load of files into a repository (a bit like a database) which then immediately become memory resident SharedStream objects; this has implications for XM and XSL processing.

By keeping this data in-memory (actually mapped into the address space, even the web server) one avoids the overhead of going through the NTFS system or to an external database.

SharedStreams may be used anywhere that a stream is used, and the data is actually in memory, so it accessed very rapidly indeed, in C# or VB.NET this is relatively simple code.

I won’t dwell on these details here; you are encouraged to download the beta class library from our website: http://www.morantex.com/Persistore.aspx

Do not hesitate to contact me with questions, feature requests or bugs, we will try to include any suggestions in our next beta.

On a 64-bit system one can in principle, store over 8,000 G bytes of XML and other data in a shared memory repository, this promises improved performance over a more conventional storage medium.

Regards
Hugh

I am writing thousands of html pages from an MS SQL source using a
DataSet and then an XslTransform. I have an interest in making the
code run faster as it takes several minutes (sometimes ten minutes) to
generate a batch of pages (say two thousand).

If anyone has URL resources which performance information germane to
XSL Transforms, Streams, and XML I am interested in hearing about
them. I have written some notes below of what I am doing.


My Trace.Warns have shown that the time used is used on the
XslTransform (and associated classes).

Methods
=======

First Method: All XML docs were written to disk and the read in
locating via MapPath.

Second: All XML docs were written to MemoryStream and then the streams
are read directly into Xpath and then the transform. Traces show
internally this is much quicker (about five times) but the elapsed
time of the whole process is still slow. The DB access is very quick
(100 ms) and so is DataSet handling.