Connecting Tech Pros Worldwide Forums | Help | Site Map

Using SQl to store aspx pages and memory problems

matvdl
Guest
 
Posts: n/a
#1: Nov 19 '05
I have a system that was originally developed in asp - the pages are saved in
SQL (there are over 10,000 pages) and saved to a temp directory in the server
when requested by a client.

I have updated this system and changed the pages that are saved to the
server as aspx - everything works fine and pages can be served - but

Its not impossible for a single client to request 100 plus pages in one
session - as each page is requested it is retrieved from SQL saved to the
temp directory and compiled - problems are.
- Performance is well down on the original asp system - I believe that
this due to the compiling of the pages in asp.net
- The memory usage of server also goes through the roof - there is a
considerable increase in memory as each page is loaded

Are there any solutions to these problems??

--
matthew

TJS
Guest
 
Posts: n/a
#2: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


what exactly does this mean ?

"as each page is requested it is retrieved from SQL saved to the
temp directory and compiled "

why are you saving it twice ?

if it 's saved in sql why not just write it back to the client?



"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=blue]
>I have a system that was originally developed in asp - the pages are saved
>in
> SQL (there are over 10,000 pages) and saved to a temp directory in the
> server
> when requested by a client.
>
> I have updated this system and changed the pages that are saved to the
> server as aspx - everything works fine and pages can be served - but
>
> Its not impossible for a single client to request 100 plus pages in one
> session - as each page is requested it is retrieved from SQL saved to the
> temp directory and compiled - problems are.
> - Performance is well down on the original asp system - I believe that
> this due to the compiling of the pages in asp.net
> - The memory usage of server also goes through the roof - there is a
> considerable increase in memory as each page is loaded
>
> Are there any solutions to these problems??
>
> --
> matthew[/color]


matvdl
Guest
 
Posts: n/a
#3: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Many of the pages have server side code - they are aspx files - not html
files. e.g some of the files may need to get information such as current
date the file was displayed - or they are customized based on the browser
that views them - so they get modifed at run-time. Many have server side
controls that display charts. (chartfx control)

The way I manage this is to write the saved aspx file to a temp directory on
the server and redirect the request to this file.

I also have html files that are retrieved from SQL but these are simply
written back to the client - the problem is with aspx files.

Any help would be appreciated.

Matthew
--
matthew


"TJS" wrote:
[color=blue]
> what exactly does this mean ?
>
> "as each page is requested it is retrieved from SQL saved to the
> temp directory and compiled "
>
> why are you saving it twice ?
>
> if it 's saved in sql why not just write it back to the client?
>
>
>
> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=green]
> >I have a system that was originally developed in asp - the pages are saved
> >in
> > SQL (there are over 10,000 pages) and saved to a temp directory in the
> > server
> > when requested by a client.
> >
> > I have updated this system and changed the pages that are saved to the
> > server as aspx - everything works fine and pages can be served - but
> >
> > Its not impossible for a single client to request 100 plus pages in one
> > session - as each page is requested it is retrieved from SQL saved to the
> > temp directory and compiled - problems are.
> > - Performance is well down on the original asp system - I believe that
> > this due to the compiling of the pages in asp.net
> > - The memory usage of server also goes through the roof - there is a
> > considerable increase in memory as each page is loaded
> >
> > Are there any solutions to these problems??
> >
> > --
> > matthew[/color]
>
>
>[/color]
Bob Lehmann
Guest
 
Posts: n/a
#4: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


This is just such a dumb way to do things. You should just start over, and
brain the idiot who did this to you.

Bob Lehmann

"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=blue]
> I have a system that was originally developed in asp - the pages are saved[/color]
in[color=blue]
> SQL (there are over 10,000 pages) and saved to a temp directory in the[/color]
server[color=blue]
> when requested by a client.
>
> I have updated this system and changed the pages that are saved to the
> server as aspx - everything works fine and pages can be served - but
>
> Its not impossible for a single client to request 100 plus pages in one
> session - as each page is requested it is retrieved from SQL saved to the
> temp directory and compiled - problems are.
> - Performance is well down on the original asp system - I believe that
> this due to the compiling of the pages in asp.net
> - The memory usage of server also goes through the roof - there is a
> considerable increase in memory as each page is loaded
>
> Are there any solutions to these problems??
>
> --
> matthew[/color]


TJS
Guest
 
Posts: n/a
#5: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


why are are you doing this ? This is not a good design. I'm surprised itr
works at all .

Just use use regular aspx files and be done with it.




"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:AAA1B336-8604-4D0B-B50C-2BAD27ACF9E5@microsoft.com...[color=blue]
> Many of the pages have server side code - they are aspx files - not html
> files. e.g some of the files may need to get information such as current
> date the file was displayed - or they are customized based on the browser
> that views them - so they get modifed at run-time. Many have server side
> controls that display charts. (chartfx control)
>
> The way I manage this is to write the saved aspx file to a temp directory
> on
> the server and redirect the request to this file.
>
> I also have html files that are retrieved from SQL but these are simply
> written back to the client - the problem is with aspx files.
>
> Any help would be appreciated.
>
> Matthew
> --
> matthew
>
>
> "TJS" wrote:
>[color=green]
>> what exactly does this mean ?
>>
>> "as each page is requested it is retrieved from SQL saved to the
>> temp directory and compiled "
>>
>> why are you saving it twice ?
>>
>> if it 's saved in sql why not just write it back to the client?
>>
>>
>>
>> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
>> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=darkred]
>> >I have a system that was originally developed in asp - the pages are
>> >saved
>> >in
>> > SQL (there are over 10,000 pages) and saved to a temp directory in the
>> > server
>> > when requested by a client.
>> >
>> > I have updated this system and changed the pages that are saved to the
>> > server as aspx - everything works fine and pages can be served - but
>> >
>> > Its not impossible for a single client to request 100 plus pages in one
>> > session - as each page is requested it is retrieved from SQL saved to
>> > the
>> > temp directory and compiled - problems are.
>> > - Performance is well down on the original asp system - I believe
>> > that
>> > this due to the compiling of the pages in asp.net
>> > - The memory usage of server also goes through the roof - there is a
>> > considerable increase in memory as each page is loaded
>> >
>> > Are there any solutions to these problems??
>> >
>> > --
>> > matthew[/color]
>>
>>
>>[/color][/color]


matvdl
Guest
 
Posts: n/a
#6: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Bob,

I will respond to your response - as the designer I won’t feel as bad as TJS
who basically called me an idiot.

There are many reason why I am doing it this way - and I don't believe that
they are all stupid.

1. Firstly the system was inherited from asp - in asp this idea caused no
problems at all and worked very well - other than the overhead of obtaining
the text from SQL server and writing it to disk - it all worked very well.

2. I'm not sure - but the idea of having 10,000 aspx files sitting on the
web server does not make sense - and it would not solve my problem anyway -
the compiling time would still be there and so would the memory issues.

3. It provides an excellent mix of creating a file that has a combination
of data that was only available at the time the file was created and
combining it with functionality available at the time it was displayed.

So - the ability of me to re-design is not possible or desired – wat I have
works very well – except for the problems of memory and speed.

This was a good idea in asp - so to say that it is simply stupid when I move
to aspx is not an helpful answer and sounds more like a cop-out - what I need
is someone willing to understand the reasons for designing it this way and
for someone to attempt to answer my questions.

For example - I have saved the copy of each invoice I send out to clients as
text within a database - this text could have been saved as plain html and
forwarded to the client - but I have included a chart within these invoices
and this chart is a active x control (chart fx - fully interactive) - for the
chart fx control to display correctly it must be served from a server -
either aspx or asp page - I also need to load the control with data - so the
only way to do this was through an aspx file.

NOW - it is not possible to re-create the invoice each time someone wants to
view it - much of the data used to create the invoice is not available.

Also - the fact that SQL is involved is not the problem - the problem comes
about as a result of having 10,000 aspx files and the memory and speed that
it takes to display them.

So - can anyone help? - can anyone offer some insight into how this can be
made more efficient - is it possible to save the compiled code into the
server and load this up with the file when it is displayed. This would solve
the speed issue.

Is it possible to get a page to unload once it has been compiled - this
would solve the memory issue.

Some constructive answers would be helpful.
--
matthew


"Bob Lehmann" wrote:
[color=blue]
> This is just such a dumb way to do things. You should just start over, and
> brain the idiot who did this to you.
>
> Bob Lehmann
>
> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=green]
> > I have a system that was originally developed in asp - the pages are saved[/color]
> in[color=green]
> > SQL (there are over 10,000 pages) and saved to a temp directory in the[/color]
> server[color=green]
> > when requested by a client.
> >
> > I have updated this system and changed the pages that are saved to the
> > server as aspx - everything works fine and pages can be served - but
> >
> > Its not impossible for a single client to request 100 plus pages in one
> > session - as each page is requested it is retrieved from SQL saved to the
> > temp directory and compiled - problems are.
> > - Performance is well down on the original asp system - I believe that
> > this due to the compiling of the pages in asp.net
> > - The memory usage of server also goes through the roof - there is a
> > considerable increase in memory as each page is loaded
> >
> > Are there any solutions to these problems??
> >
> > --
> > matthew[/color]
>
>
>[/color]
matvdl
Guest
 
Posts: n/a
#7: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Bob

I made yousund like the good guy - I am sorry - but thats not true.

It was TJS who was a little less unreasonable


--
matthew


"matvdl" wrote:
[color=blue]
> Bob,
>
> I will respond to your response - as the designer I won’t feel as bad as TJS
> who basically called me an idiot.
>
> There are many reason why I am doing it this way - and I don't believe that
> they are all stupid.
>
> 1. Firstly the system was inherited from asp - in asp this idea caused no
> problems at all and worked very well - other than the overhead of obtaining
> the text from SQL server and writing it to disk - it all worked very well.
>
> 2. I'm not sure - but the idea of having 10,000 aspx files sitting on the
> web server does not make sense - and it would not solve my problem anyway -
> the compiling time would still be there and so would the memory issues.
>
> 3. It provides an excellent mix of creating a file that has a combination
> of data that was only available at the time the file was created and
> combining it with functionality available at the time it was displayed.
>
> So - the ability of me to re-design is not possible or desired – wat I have
> works very well – except for the problems of memory and speed.
>
> This was a good idea in asp - so to say that it is simply stupid when I move
> to aspx is not an helpful answer and sounds more like a cop-out - what I need
> is someone willing to understand the reasons for designing it this way and
> for someone to attempt to answer my questions.
>
> For example - I have saved the copy of each invoice I send out to clients as
> text within a database - this text could have been saved as plain html and
> forwarded to the client - but I have included a chart within these invoices
> and this chart is a active x control (chart fx - fully interactive) - for the
> chart fx control to display correctly it must be served from a server -
> either aspx or asp page - I also need to load the control with data - so the
> only way to do this was through an aspx file.
>
> NOW - it is not possible to re-create the invoice each time someone wants to
> view it - much of the data used to create the invoice is not available.
>
> Also - the fact that SQL is involved is not the problem - the problem comes
> about as a result of having 10,000 aspx files and the memory and speed that
> it takes to display them.
>
> So - can anyone help? - can anyone offer some insight into how this can be
> made more efficient - is it possible to save the compiled code into the
> server and load this up with the file when it is displayed. This would solve
> the speed issue.
>
> Is it possible to get a page to unload once it has been compiled - this
> would solve the memory issue.
>
> Some constructive answers would be helpful.
> --
> matthew
>
>
> "Bob Lehmann" wrote:
>[color=green]
> > This is just such a dumb way to do things. You should just start over, and
> > brain the idiot who did this to you.
> >
> > Bob Lehmann
> >
> > "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> > news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=darkred]
> > > I have a system that was originally developed in asp - the pages are saved[/color]
> > in[color=darkred]
> > > SQL (there are over 10,000 pages) and saved to a temp directory in the[/color]
> > server[color=darkred]
> > > when requested by a client.
> > >
> > > I have updated this system and changed the pages that are saved to the
> > > server as aspx - everything works fine and pages can be served - but
> > >
> > > Its not impossible for a single client to request 100 plus pages in one
> > > session - as each page is requested it is retrieved from SQL saved to the
> > > temp directory and compiled - problems are.
> > > - Performance is well down on the original asp system - I believe that
> > > this due to the compiling of the pages in asp.net
> > > - The memory usage of server also goes through the roof - there is a
> > > considerable increase in memory as each page is loaded
> > >
> > > Are there any solutions to these problems??
> > >
> > > --
> > > matthew[/color]
> >
> >
> >[/color][/color]
Bob Lehmann
Guest
 
Posts: n/a
#8: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:5A0DAAF2-6BA7-4F1F-A28D-DEEC9A3475B7@microsoft.com...[color=blue]
> I made yousund like the good guy - I am sorry - but thats not true.[/color]

You're forgiven. Thanks for clearing that up.

Bob Lehmann


TJS
Guest
 
Posts: n/a
#9: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


first, I didn't call you an idiot, Bob Lehmann did...........


second, you can't take a bad approach and make it efficient....


third, if the classic asp system works, you could leave it in classic asp
for that piece. classic and .net can run side by side.





"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:77EF1A4E-63FD-457A-966E-0C82968F2242@microsoft.com...[color=blue]
> Bob,
>
> I will respond to your response - as the designer I won't feel as bad as
> TJS
> who basically called me an idiot.
>
> There are many reason why I am doing it this way - and I don't believe
> that
> they are all stupid.
>
> 1. Firstly the system was inherited from asp - in asp this idea caused no
> problems at all and worked very well - other than the overhead of
> obtaining
> the text from SQL server and writing it to disk - it all worked very well.
>
> 2. I'm not sure - but the idea of having 10,000 aspx files sitting on the
> web server does not make sense - and it would not solve my problem
> anyway -
> the compiling time would still be there and so would the memory issues.
>
> 3. It provides an excellent mix of creating a file that has a combination
> of data that was only available at the time the file was created and
> combining it with functionality available at the time it was displayed.
>
> So - the ability of me to re-design is not possible or desired - wat I
> have
> works very well - except for the problems of memory and speed.
>
> This was a good idea in asp - so to say that it is simply stupid when I
> move
> to aspx is not an helpful answer and sounds more like a cop-out - what I
> need
> is someone willing to understand the reasons for designing it this way and
> for someone to attempt to answer my questions.
>
> For example - I have saved the copy of each invoice I send out to clients
> as
> text within a database - this text could have been saved as plain html and
> forwarded to the client - but I have included a chart within these
> invoices
> and this chart is a active x control (chart fx - fully interactive) - for
> the
> chart fx control to display correctly it must be served from a server -
> either aspx or asp page - I also need to load the control with data - so
> the
> only way to do this was through an aspx file.
>
> NOW - it is not possible to re-create the invoice each time someone wants
> to
> view it - much of the data used to create the invoice is not available.
>
> Also - the fact that SQL is involved is not the problem - the problem
> comes
> about as a result of having 10,000 aspx files and the memory and speed
> that
> it takes to display them.
>
> So - can anyone help? - can anyone offer some insight into how this can be
> made more efficient - is it possible to save the compiled code into the
> server and load this up with the file when it is displayed. This would
> solve
> the speed issue.
>
> Is it possible to get a page to unload once it has been compiled - this
> would solve the memory issue.
>
> Some constructive answers would be helpful.
> --
> matthew
>
>
> "Bob Lehmann" wrote:
>[color=green]
>> This is just such a dumb way to do things. You should just start over,
>> and
>> brain the idiot who did this to you.
>>
>> Bob Lehmann
>>
>> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
>> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...[color=darkred]
>> > I have a system that was originally developed in asp - the pages are
>> > saved[/color]
>> in[color=darkred]
>> > SQL (there are over 10,000 pages) and saved to a temp directory in the[/color]
>> server[color=darkred]
>> > when requested by a client.
>> >
>> > I have updated this system and changed the pages that are saved to the
>> > server as aspx - everything works fine and pages can be served - but
>> >
>> > Its not impossible for a single client to request 100 plus pages in one
>> > session - as each page is requested it is retrieved from SQL saved to
>> > the
>> > temp directory and compiled - problems are.
>> > - Performance is well down on the original asp system - I believe
>> > that
>> > this due to the compiling of the pages in asp.net
>> > - The memory usage of server also goes through the roof - there is a
>> > considerable increase in memory as each page is loaded
>> >
>> > Are there any solutions to these problems??
>> >
>> > --
>> > matthew[/color]
>>
>>
>>[/color][/color]


matvdl
Guest
 
Posts: n/a
#10: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


TJS,

Sorry - I made a mistake on my first response I did try to correct the issue.

Thanks for providing some constructive comments - going back to ASP for
these pages is something that I have considered.

By looking at this from another direction - maybe a solution can be provided.

What if I use an external program to return a portion of the HTML text that
is returned the user - I am sure that this configuration is not unusual.
Basically when the aspx page is called it goes and calls an external function
and this returns text formatted in HTML - this can simply sent back to the
client using the write statement.

Just say though that the external program decided that it wanted to embed
some server specific information into its html text - say the server name -
to do this you could embed a <%= %> statement into the HTML. Is there not
anyway of parsing the returned HTML and embedded <%= %> statements to result
in pure html text that is then written to the browser?

This does not sound like an unreasonable thing to do - and from the
perspective of the external program - if this was possible - would add
significantly to the flexibility of the system.

Hopefully this example has made some sense - does this sound possible?

--
matthew


"TJS" wrote:
[color=blue]
> first, I didn't call you an idiot, Bob Lehmann did...........
>
>
> second, you can't take a bad approach and make it efficient....
>
>
> third, if the classic asp system works, you could leave it in classic asp
> for that piece. classic and .net can run side by side.
>
>
>
>
>
> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> news:77EF1A4E-63FD-457A-966E-0C82968F2242@microsoft.com...[color=green]
> > Bob,
> >
> > I will respond to your response - as the designer I won't feel as bad as
> > TJS
> > who basically called me an idiot.
> >
> > There are many reason why I am doing it this way - and I don't believe
> > that
> > they are all stupid.
> >
> > 1. Firstly the system was inherited from asp - in asp this idea caused no
> > problems at all and worked very well - other than the overhead of
> > obtaining
> > the text from SQL server and writing it to disk - it all worked very well.
> >
> > 2. I'm not sure - but the idea of having 10,000 aspx files sitting on the
> > web server does not make sense - and it would not solve my problem
> > anyway -
> > the compiling time would still be there and so would the memory issues.
> >
> > 3. It provides an excellent mix of creating a file that has a combination
> > of data that was only available at the time the file was created and
> > combining it with functionality available at the time it was displayed.
> >
> > So - the ability of me to re-design is not possible or desired - wat I
> > have
> > works very well - except for the problems of memory and speed.
> >
> > This was a good idea in asp - so to say that it is simply stupid when I
> > move
> > to aspx is not an helpful answer and sounds more like a cop-out - what I
> > need
> > is someone willing to understand the reasons for designing it this way and
> > for someone to attempt to answer my questions.
> >
> > For example - I have saved the copy of each invoice I send out to clients
> > as
> > text within a database - this text could have been saved as plain html and
> > forwarded to the client - but I have included a chart within these
> > invoices
> > and this chart is a active x control (chart fx - fully interactive) - for
> > the
> > chart fx control to display correctly it must be served from a server -
> > either aspx or asp page - I also need to load the control with data - so
> > the
> > only way to do this was through an aspx file.
> >
> > NOW - it is not possible to re-create the invoice each time someone wants
> > to
> > view it - much of the data used to create the invoice is not available.
> >
> > Also - the fact that SQL is involved is not the problem - the problem
> > comes
> > about as a result of having 10,000 aspx files and the memory and speed
> > that
> > it takes to display them.
> >
> > So - can anyone help? - can anyone offer some insight into how this can be
> > made more efficient - is it possible to save the compiled code into the
> > server and load this up with the file when it is displayed. This would
> > solve
> > the speed issue.
> >
> > Is it possible to get a page to unload once it has been compiled - this
> > would solve the memory issue.
> >
> > Some constructive answers would be helpful.
> > --
> > matthew
> >
> >
> > "Bob Lehmann" wrote:
> >[color=darkred]
> >> This is just such a dumb way to do things. You should just start over,
> >> and
> >> brain the idiot who did this to you.
> >>
> >> Bob Lehmann
> >>
> >> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> >> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...
> >> > I have a system that was originally developed in asp - the pages are
> >> > saved
> >> in
> >> > SQL (there are over 10,000 pages) and saved to a temp directory in the
> >> server
> >> > when requested by a client.
> >> >
> >> > I have updated this system and changed the pages that are saved to the
> >> > server as aspx - everything works fine and pages can be served - but
> >> >
> >> > Its not impossible for a single client to request 100 plus pages in one
> >> > session - as each page is requested it is retrieved from SQL saved to
> >> > the
> >> > temp directory and compiled - problems are.
> >> > - Performance is well down on the original asp system - I believe
> >> > that
> >> > this due to the compiling of the pages in asp.net
> >> > - The memory usage of server also goes through the roof - there is a
> >> > considerable increase in memory as each page is loaded
> >> >
> >> > Are there any solutions to these problems??
> >> >
> >> > --
> >> > matthew
> >>
> >>
> >>[/color][/color]
>
>
>[/color]
DLM
Guest
 
Posts: n/a
#11: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Go to this link:
http://www.theserverside.net/books/i...ngPerf/pag.tss

We all know that there is an initial performance 'hit' the first time a ASP
..NET Page is requested, and this is true with the page sitting in it's
virtual directory on IIS. What you seem to be doing is increasing the round
trip time for any given request/response for a page to include the time it
takes to get the page from SQL Server. And this added time is database access
time which is probably longer than any HTTP request.

I have never seen this kind of architecture before. The whole idea is to
serve pages from a highly scalable architecture such as IIS from the very
start, and not make the ASP .NET worker process wait for the files before it
can dynamically compile pages. In a sense you are using SQL Server as a web
server to your web server, which is highly unusual. You need to find a way to
place all your pages on the web server and use SQL Server only for data
retrieval/update.

TAKE A LOOK AT MICROSOFT PATTERNS & PRACTICES


Bob Lehmann
Guest
 
Posts: n/a
#12: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


First, I didn't call matvdl an idiot.

Second, I did say that the methodology was dumb, and the person
_who_designed_it was an idiot.

Third, is your <shift > key broken?

Fourth, I agree with everything else you've said.

Bob Lehmann


"TJS" <nospam@here.com> wrote in message
news:erhiC8AgFHA.2548@TK2MSFTNGP10.phx.gbl...[color=blue]
> first, I didn't call you an idiot, Bob Lehmann did...........
>
>
> second, you can't take a bad approach and make it efficient....
>
>
> third, if the classic asp system works, you could leave it in classic asp
> for that piece. classic and .net can run side by side.
>
>
>
>
>
> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> news:77EF1A4E-63FD-457A-966E-0C82968F2242@microsoft.com...[color=green]
> > Bob,
> >
> > I will respond to your response - as the designer I won't feel as bad as
> > TJS
> > who basically called me an idiot.
> >
> > There are many reason why I am doing it this way - and I don't believe
> > that
> > they are all stupid.
> >
> > 1. Firstly the system was inherited from asp - in asp this idea caused[/color][/color]
no[color=blue][color=green]
> > problems at all and worked very well - other than the overhead of
> > obtaining
> > the text from SQL server and writing it to disk - it all worked very[/color][/color]
well.[color=blue][color=green]
> >
> > 2. I'm not sure - but the idea of having 10,000 aspx files sitting on[/color][/color]
the[color=blue][color=green]
> > web server does not make sense - and it would not solve my problem
> > anyway -
> > the compiling time would still be there and so would the memory issues.
> >
> > 3. It provides an excellent mix of creating a file that has a[/color][/color]
combination[color=blue][color=green]
> > of data that was only available at the time the file was created and
> > combining it with functionality available at the time it was displayed.
> >
> > So - the ability of me to re-design is not possible or desired - wat I
> > have
> > works very well - except for the problems of memory and speed.
> >
> > This was a good idea in asp - so to say that it is simply stupid when I
> > move
> > to aspx is not an helpful answer and sounds more like a cop-out - what I
> > need
> > is someone willing to understand the reasons for designing it this way[/color][/color]
and[color=blue][color=green]
> > for someone to attempt to answer my questions.
> >
> > For example - I have saved the copy of each invoice I send out to[/color][/color]
clients[color=blue][color=green]
> > as
> > text within a database - this text could have been saved as plain html[/color][/color]
and[color=blue][color=green]
> > forwarded to the client - but I have included a chart within these
> > invoices
> > and this chart is a active x control (chart fx - fully interactive) -[/color][/color]
for[color=blue][color=green]
> > the
> > chart fx control to display correctly it must be served from a server -
> > either aspx or asp page - I also need to load the control with data - so
> > the
> > only way to do this was through an aspx file.
> >
> > NOW - it is not possible to re-create the invoice each time someone[/color][/color]
wants[color=blue][color=green]
> > to
> > view it - much of the data used to create the invoice is not available.
> >
> > Also - the fact that SQL is involved is not the problem - the problem
> > comes
> > about as a result of having 10,000 aspx files and the memory and speed
> > that
> > it takes to display them.
> >
> > So - can anyone help? - can anyone offer some insight into how this can[/color][/color]
be[color=blue][color=green]
> > made more efficient - is it possible to save the compiled code into the
> > server and load this up with the file when it is displayed. This would
> > solve
> > the speed issue.
> >
> > Is it possible to get a page to unload once it has been compiled - this
> > would solve the memory issue.
> >
> > Some constructive answers would be helpful.
> > --
> > matthew
> >
> >
> > "Bob Lehmann" wrote:
> >[color=darkred]
> >> This is just such a dumb way to do things. You should just start over,
> >> and
> >> brain the idiot who did this to you.
> >>
> >> Bob Lehmann
> >>
> >> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> >> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...
> >> > I have a system that was originally developed in asp - the pages are
> >> > saved
> >> in
> >> > SQL (there are over 10,000 pages) and saved to a temp directory in[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> >> server
> >> > when requested by a client.
> >> >
> >> > I have updated this system and changed the pages that are saved to[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> >> > server as aspx - everything works fine and pages can be served - but
> >> >
> >> > Its not impossible for a single client to request 100 plus pages in[/color][/color][/color]
one[color=blue][color=green][color=darkred]
> >> > session - as each page is requested it is retrieved from SQL saved[/color][/color][/color]
to[color=blue][color=green][color=darkred]
> >> > the
> >> > temp directory and compiled - problems are.
> >> > - Performance is well down on the original asp system - I believe
> >> > that
> >> > this due to the compiling of the pages in asp.net
> >> > - The memory usage of server also goes through the roof - there is[/color][/color][/color]
a[color=blue][color=green][color=darkred]
> >> > considerable increase in memory as each page is loaded
> >> >
> >> > Are there any solutions to these problems??
> >> >
> >> > --
> >> > matthew
> >>
> >>
> >>[/color][/color]
>
>[/color]


Bob Lehmann
Guest
 
Posts: n/a
#13: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


> What if I use an external program to return a portion of the HTML text

Yes, adding another layer of complexity to a poorly designed system will
surely be beneficial.

Bob Lehmann

"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:D876A691-4C2B-461D-BD75-70D2DBEBEFAB@microsoft.com...[color=blue]
> TJS,
>
> Sorry - I made a mistake on my first response I did try to correct the[/color]
issue.[color=blue]
>
> Thanks for providing some constructive comments - going back to ASP for
> these pages is something that I have considered.
>
> By looking at this from another direction - maybe a solution can be[/color]
provided.[color=blue]
>
> What if I use an external program to return a portion of the HTML text[/color]
that[color=blue]
> is returned the user - I am sure that this configuration is not unusual.
> Basically when the aspx page is called it goes and calls an external[/color]
function[color=blue]
> and this returns text formatted in HTML - this can simply sent back to the
> client using the write statement.
>
> Just say though that the external program decided that it wanted to embed
> some server specific information into its html text - say the server[/color]
name -[color=blue]
> to do this you could embed a <%= %> statement into the HTML. Is there[/color]
not[color=blue]
> anyway of parsing the returned HTML and embedded <%= %> statements to[/color]
result[color=blue]
> in pure html text that is then written to the browser?
>
> This does not sound like an unreasonable thing to do - and from the
> perspective of the external program - if this was possible - would add
> significantly to the flexibility of the system.
>
> Hopefully this example has made some sense - does this sound possible?
>
> --
> matthew
>
>
> "TJS" wrote:
>[color=green]
> > first, I didn't call you an idiot, Bob Lehmann did...........
> >
> >
> > second, you can't take a bad approach and make it efficient....
> >
> >
> > third, if the classic asp system works, you could leave it in classic[/color][/color]
asp[color=blue][color=green]
> > for that piece. classic and .net can run side by side.
> >
> >
> >
> >
> >
> > "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> > news:77EF1A4E-63FD-457A-966E-0C82968F2242@microsoft.com...[color=darkred]
> > > Bob,
> > >
> > > I will respond to your response - as the designer I won't feel as bad[/color][/color][/color]
as[color=blue][color=green][color=darkred]
> > > TJS
> > > who basically called me an idiot.
> > >
> > > There are many reason why I am doing it this way - and I don't believe
> > > that
> > > they are all stupid.
> > >
> > > 1. Firstly the system was inherited from asp - in asp this idea[/color][/color][/color]
caused no[color=blue][color=green][color=darkred]
> > > problems at all and worked very well - other than the overhead of
> > > obtaining
> > > the text from SQL server and writing it to disk - it all worked very[/color][/color][/color]
well.[color=blue][color=green][color=darkred]
> > >
> > > 2. I'm not sure - but the idea of having 10,000 aspx files sitting on[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> > > web server does not make sense - and it would not solve my problem
> > > anyway -
> > > the compiling time would still be there and so would the memory[/color][/color][/color]
issues.[color=blue][color=green][color=darkred]
> > >
> > > 3. It provides an excellent mix of creating a file that has a[/color][/color][/color]
combination[color=blue][color=green][color=darkred]
> > > of data that was only available at the time the file was created and
> > > combining it with functionality available at the time it was[/color][/color][/color]
displayed.[color=blue][color=green][color=darkred]
> > >
> > > So - the ability of me to re-design is not possible or desired - wat I
> > > have
> > > works very well - except for the problems of memory and speed.
> > >
> > > This was a good idea in asp - so to say that it is simply stupid when[/color][/color][/color]
I[color=blue][color=green][color=darkred]
> > > move
> > > to aspx is not an helpful answer and sounds more like a cop-out - what[/color][/color][/color]
I[color=blue][color=green][color=darkred]
> > > need
> > > is someone willing to understand the reasons for designing it this way[/color][/color][/color]
and[color=blue][color=green][color=darkred]
> > > for someone to attempt to answer my questions.
> > >
> > > For example - I have saved the copy of each invoice I send out to[/color][/color][/color]
clients[color=blue][color=green][color=darkred]
> > > as
> > > text within a database - this text could have been saved as plain html[/color][/color][/color]
and[color=blue][color=green][color=darkred]
> > > forwarded to the client - but I have included a chart within these
> > > invoices
> > > and this chart is a active x control (chart fx - fully interactive) -[/color][/color][/color]
for[color=blue][color=green][color=darkred]
> > > the
> > > chart fx control to display correctly it must be served from a[/color][/color][/color]
server -[color=blue][color=green][color=darkred]
> > > either aspx or asp page - I also need to load the control with data -[/color][/color][/color]
so[color=blue][color=green][color=darkred]
> > > the
> > > only way to do this was through an aspx file.
> > >
> > > NOW - it is not possible to re-create the invoice each time someone[/color][/color][/color]
wants[color=blue][color=green][color=darkred]
> > > to
> > > view it - much of the data used to create the invoice is not[/color][/color][/color]
available.[color=blue][color=green][color=darkred]
> > >
> > > Also - the fact that SQL is involved is not the problem - the problem
> > > comes
> > > about as a result of having 10,000 aspx files and the memory and speed
> > > that
> > > it takes to display them.
> > >
> > > So - can anyone help? - can anyone offer some insight into how this[/color][/color][/color]
can be[color=blue][color=green][color=darkred]
> > > made more efficient - is it possible to save the compiled code into[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> > > server and load this up with the file when it is displayed. This[/color][/color][/color]
would[color=blue][color=green][color=darkred]
> > > solve
> > > the speed issue.
> > >
> > > Is it possible to get a page to unload once it has been compiled -[/color][/color][/color]
this[color=blue][color=green][color=darkred]
> > > would solve the memory issue.
> > >
> > > Some constructive answers would be helpful.
> > > --
> > > matthew
> > >
> > >
> > > "Bob Lehmann" wrote:
> > >
> > >> This is just such a dumb way to do things. You should just start[/color][/color][/color]
over,[color=blue][color=green][color=darkred]
> > >> and
> > >> brain the idiot who did this to you.
> > >>
> > >> Bob Lehmann
> > >>
> > >> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> > >> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...
> > >> > I have a system that was originally developed in asp - the pages[/color][/color][/color]
are[color=blue][color=green][color=darkred]
> > >> > saved
> > >> in
> > >> > SQL (there are over 10,000 pages) and saved to a temp directory in[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> > >> server
> > >> > when requested by a client.
> > >> >
> > >> > I have updated this system and changed the pages that are saved to[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> > >> > server as aspx - everything works fine and pages can be served -[/color][/color][/color]
but[color=blue][color=green][color=darkred]
> > >> >
> > >> > Its not impossible for a single client to request 100 plus pages in[/color][/color][/color]
one[color=blue][color=green][color=darkred]
> > >> > session - as each page is requested it is retrieved from SQL saved[/color][/color][/color]
to[color=blue][color=green][color=darkred]
> > >> > the
> > >> > temp directory and compiled - problems are.
> > >> > - Performance is well down on the original asp system - I[/color][/color][/color]
believe[color=blue][color=green][color=darkred]
> > >> > that
> > >> > this due to the compiling of the pages in asp.net
> > >> > - The memory usage of server also goes through the roof - there[/color][/color][/color]
is a[color=blue][color=green][color=darkred]
> > >> > considerable increase in memory as each page is loaded
> > >> >
> > >> > Are there any solutions to these problems??
> > >> >
> > >> > --
> > >> > matthew
> > >>
> > >>
> > >>[/color]
> >
> >
> >[/color][/color]


TJS
Guest
 
Posts: n/a
#14: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Quit complaining , I capitalized your name. And yes, you basically did
call him an idiot and he felt the same way and said as much.
Unfortunately, he thought it was me but later did clear that up .

At least we agree about his approach.


"Bob Lehmann" <nospam@dontbotherme.zzz> wrote in message
news:epXSIIEgFHA.1048@tk2msftngp13.phx.gbl...[color=blue]
> First, I didn't call matvdl an idiot.
>
> Second, I did say that the methodology was dumb, and the person
> _who_designed_it was an idiot.
>
> Third, is your <shift > key broken?
>
> Fourth, I agree with everything else you've said.
>
> Bob Lehmann
>
>
> "TJS" <nospam@here.com> wrote in message
> news:erhiC8AgFHA.2548@TK2MSFTNGP10.phx.gbl...[color=green]
>> first, I didn't call you an idiot, Bob Lehmann did...........
>>
>>
>> second, you can't take a bad approach and make it efficient....
>>
>>
>> third, if the classic asp system works, you could leave it in classic
>> asp
>> for that piece. classic and .net can run side by side.
>>
>>
>>
>>
>>
>> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
>> news:77EF1A4E-63FD-457A-966E-0C82968F2242@microsoft.com...[color=darkred]
>> > Bob,
>> >
>> > I will respond to your response - as the designer I won't feel as bad
>> > as
>> > TJS
>> > who basically called me an idiot.
>> >
>> > There are many reason why I am doing it this way - and I don't believe
>> > that
>> > they are all stupid.
>> >
>> > 1. Firstly the system was inherited from asp - in asp this idea caused[/color][/color]
> no[color=green][color=darkred]
>> > problems at all and worked very well - other than the overhead of
>> > obtaining
>> > the text from SQL server and writing it to disk - it all worked very[/color][/color]
> well.[color=green][color=darkred]
>> >
>> > 2. I'm not sure - but the idea of having 10,000 aspx files sitting on[/color][/color]
> the[color=green][color=darkred]
>> > web server does not make sense - and it would not solve my problem
>> > anyway -
>> > the compiling time would still be there and so would the memory issues.
>> >
>> > 3. It provides an excellent mix of creating a file that has a[/color][/color]
> combination[color=green][color=darkred]
>> > of data that was only available at the time the file was created and
>> > combining it with functionality available at the time it was displayed.
>> >
>> > So - the ability of me to re-design is not possible or desired - wat I
>> > have
>> > works very well - except for the problems of memory and speed.
>> >
>> > This was a good idea in asp - so to say that it is simply stupid when I
>> > move
>> > to aspx is not an helpful answer and sounds more like a cop-out - what
>> > I
>> > need
>> > is someone willing to understand the reasons for designing it this way[/color][/color]
> and[color=green][color=darkred]
>> > for someone to attempt to answer my questions.
>> >
>> > For example - I have saved the copy of each invoice I send out to[/color][/color]
> clients[color=green][color=darkred]
>> > as
>> > text within a database - this text could have been saved as plain html[/color][/color]
> and[color=green][color=darkred]
>> > forwarded to the client - but I have included a chart within these
>> > invoices
>> > and this chart is a active x control (chart fx - fully interactive) -[/color][/color]
> for[color=green][color=darkred]
>> > the
>> > chart fx control to display correctly it must be served from a server -
>> > either aspx or asp page - I also need to load the control with data -
>> > so
>> > the
>> > only way to do this was through an aspx file.
>> >
>> > NOW - it is not possible to re-create the invoice each time someone[/color][/color]
> wants[color=green][color=darkred]
>> > to
>> > view it - much of the data used to create the invoice is not available.
>> >
>> > Also - the fact that SQL is involved is not the problem - the problem
>> > comes
>> > about as a result of having 10,000 aspx files and the memory and speed
>> > that
>> > it takes to display them.
>> >
>> > So - can anyone help? - can anyone offer some insight into how this can[/color][/color]
> be[color=green][color=darkred]
>> > made more efficient - is it possible to save the compiled code into the
>> > server and load this up with the file when it is displayed. This would
>> > solve
>> > the speed issue.
>> >
>> > Is it possible to get a page to unload once it has been compiled - this
>> > would solve the memory issue.
>> >
>> > Some constructive answers would be helpful.
>> > --
>> > matthew
>> >
>> >
>> > "Bob Lehmann" wrote:
>> >
>> >> This is just such a dumb way to do things. You should just start over,
>> >> and
>> >> brain the idiot who did this to you.
>> >>
>> >> Bob Lehmann
>> >>
>> >> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
>> >> news:753E0555-B034-40F7-BD3A-E3EE61600EA0@microsoft.com...
>> >> > I have a system that was originally developed in asp - the pages are
>> >> > saved
>> >> in
>> >> > SQL (there are over 10,000 pages) and saved to a temp directory in[/color][/color]
> the[color=green][color=darkred]
>> >> server
>> >> > when requested by a client.
>> >> >
>> >> > I have updated this system and changed the pages that are saved to[/color][/color]
> the[color=green][color=darkred]
>> >> > server as aspx - everything works fine and pages can be served - but
>> >> >
>> >> > Its not impossible for a single client to request 100 plus pages in[/color][/color]
> one[color=green][color=darkred]
>> >> > session - as each page is requested it is retrieved from SQL saved[/color][/color]
> to[color=green][color=darkred]
>> >> > the
>> >> > temp directory and compiled - problems are.
>> >> > - Performance is well down on the original asp system - I believe
>> >> > that
>> >> > this due to the compiling of the pages in asp.net
>> >> > - The memory usage of server also goes through the roof - there
>> >> > is[/color][/color]
> a[color=green][color=darkred]
>> >> > considerable increase in memory as each page is loaded
>> >> >
>> >> > Are there any solutions to these problems??
>> >> >
>> >> > --
>> >> > matthew
>> >>
>> >>
>> >>[/color]
>>
>>[/color]
>
>[/color]


matvdl
Guest
 
Posts: n/a
#15: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


DLM,

Thanks for your response.

I don't believe that SQL is my problem - the time it takes to do the remote
calls to the external program has not presented any real limitations or
caused any significant time-delays. The problem that I have is trying to
deal with the quanitity of files that I have and the combiling of those pages
once they have been returned. Any time issues that I have had are the result
of parsing and compilling the page.

I understand your point - that asp.net was designed to have the pages
pre-compiled before requests where made for the pages - I guess this is one
of the fundamental differances between asp and asp.net. From experiance the
design that has been implemented had no significant disadvantages when
working with asp - in fact I believe that it had some very significant
advantages - mainly due to the unlimited number of files it could manage.

Question - is it practicale to have 10,000 pages in a single asp.net
application. I wouldn't have thought so?

I have no alternative but to deal with this many files as it is a legal
requirment of the industry I work in is to be able to provide the original
copies of invoices to clients - I can't go back to the original data and
attempt to re-produce the same invoice just in case the data has changed.

So - considering this is what I am stuck with - my problem mainly relates to
the memory increases - the slowness of the request could be dealt with by the
more grunt - but the ever increasing memory is more difficult. I believe
that this is the result of the creation of a new class each time a new page
is displayed - although this page gets un-loaded once the info is returned -
are the increases in memory the result of the new class definition in the
asp.net application?

If this is the case - is it possible to remove this class definition once it
has been created? I have tried to delete the file - but this does not appear
to fix the problem.

Does any of this make sense.


--
matthew


"DLM" wrote:
[color=blue]
> Go to this link:
> http://www.theserverside.net/books/i...ngPerf/pag.tss
>
> We all know that there is an initial performance 'hit' the first time a ASP
> .NET Page is requested, and this is true with the page sitting in it's
> virtual directory on IIS. What you seem to be doing is increasing the round
> trip time for any given request/response for a page to include the time it
> takes to get the page from SQL Server. And this added time is database access
> time which is probably longer than any HTTP request.
>
> I have never seen this kind of architecture before. The whole idea is to
> serve pages from a highly scalable architecture such as IIS from the very
> start, and not make the ASP .NET worker process wait for the files before it
> can dynamically compile pages. In a sense you are using SQL Server as a web
> server to your web server, which is highly unusual. You need to find a way to
> place all your pages on the web server and use SQL Server only for data
> retrieval/update.
>
> TAKE A LOOK AT MICROSOFT PATTERNS & PRACTICES
>
>[/color]
Klaus H. Probst
Guest
 
Posts: n/a
#16: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


The following suggestion may or may not work for you, but I'll chime in.

First off, to the folks that called the approach your existing app uses
"idiotic", I've seen classic ASP applications design this way that work
extremely well, are scalable and extremely manageable. However, the devil is
in the details. Just putting "pages" in IMAGE or TEXT fields on a SQL
database isn't some sort of magic pixie dust that solves all problems. It
takes a lot of thought and experimentation to get just right, and the
rewards in most cases offset the negative aspects of these designs.

Having said that, your approach is, as you've figured out by now, wrong.
Others have already mentioned the perf hit you're taking by forcing ASP.NET
to function in a way it wasn't designed to. But there are ways to get around
the problem.

First off, I'll assume that you have more content than code. That is, *most*
of your existing ASP/ASPX pages in the database contain stuff, like invoices
and whatnot, that do not require intervention by the parser to load and
display. If this is not the case then stop reading now =)

There are a lot of web applications nowadays that use their databases as
primary content storage repositories. Blogs are a good example of this.
However, they do not store code in the database, they store content. If you
can adapt to this type of scenario then all you have to do is create a sort
of "master page" or maybe an IHttpHandler implementation that looks at the
request (even a path), pulls the *content* from the database and then
displays it on a sort of templated page container. Think of a site like
MSDN. Lots of articles, yet all the pages look the same. The content is what
varies. The trick is to simply treat your "pages" as opaque chunks of
content, and "stream" them into a placeholder page that is rendered the same
way every time. You can pretty much do anything you want once you're hooked
into the ASP.NET processing pipeline.

If you do this you'll get rid of your perf problem and you'll still have the
flexibility and manageability of the database storage. This is web
application design 101 - always separate content from function and
structure.

Of course getting from point A to point B could be tricky, but maybe this
gives you some ideas =)

--
Klaus H. Probst, MVP
http://www.simulplex.net/


"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:DF375EB0-8373-4D91-B7F3-0400A8AE22E9@microsoft.com...[color=blue]
> DLM,
>
> Thanks for your response.
>
> I don't believe that SQL is my problem - the time it takes to do the[/color]
remote[color=blue]
> calls to the external program has not presented any real limitations or
> caused any significant time-delays. The problem that I have is trying to
> deal with the quanitity of files that I have and the combiling of those[/color]
pages[color=blue]
> once they have been returned. Any time issues that I have had are the[/color]
result[color=blue]
> of parsing and compilling the page.
>
> I understand your point - that asp.net was designed to have the pages
> pre-compiled before requests where made for the pages - I guess this is[/color]
one[color=blue]
> of the fundamental differances between asp and asp.net. From experiance[/color]
the[color=blue]
> design that has been implemented had no significant disadvantages when
> working with asp - in fact I believe that it had some very significant
> advantages - mainly due to the unlimited number of files it could manage.
>
> Question - is it practicale to have 10,000 pages in a single asp.net
> application. I wouldn't have thought so?
>
> I have no alternative but to deal with this many files as it is a legal
> requirment of the industry I work in is to be able to provide the original
> copies of invoices to clients - I can't go back to the original data and
> attempt to re-produce the same invoice just in case the data has changed.
>
> So - considering this is what I am stuck with - my problem mainly relates[/color]
to[color=blue]
> the memory increases - the slowness of the request could be dealt with by[/color]
the[color=blue]
> more grunt - but the ever increasing memory is more difficult. I believe
> that this is the result of the creation of a new class each time a new[/color]
page[color=blue]
> is displayed - although this page gets un-loaded once the info is[/color]
returned -[color=blue]
> are the increases in memory the result of the new class definition in the
> asp.net application?
>
> If this is the case - is it possible to remove this class definition once[/color]
it[color=blue]
> has been created? I have tried to delete the file - but this does not[/color]
appear[color=blue]
> to fix the problem.
>
> Does any of this make sense.
>
>
> --
> matthew
>
>
> "DLM" wrote:
>[color=green]
> > Go to this link:
> > http://www.theserverside.net/books/i...ngPerf/pag.tss
> >
> > We all know that there is an initial performance 'hit' the first time a[/color][/color]
ASP[color=blue][color=green]
> > .NET Page is requested, and this is true with the page sitting in it's
> > virtual directory on IIS. What you seem to be doing is increasing the[/color][/color]
round[color=blue][color=green]
> > trip time for any given request/response for a page to include the time[/color][/color]
it[color=blue][color=green]
> > takes to get the page from SQL Server. And this added time is database[/color][/color]
access[color=blue][color=green]
> > time which is probably longer than any HTTP request.
> >
> > I have never seen this kind of architecture before. The whole idea is to
> > serve pages from a highly scalable architecture such as IIS from the[/color][/color]
very[color=blue][color=green]
> > start, and not make the ASP .NET worker process wait for the files[/color][/color]
before it[color=blue][color=green]
> > can dynamically compile pages. In a sense you are using SQL Server as a[/color][/color]
web[color=blue][color=green]
> > server to your web server, which is highly unusual. You need to find a[/color][/color]
way to[color=blue][color=green]
> > place all your pages on the web server and use SQL Server only for data
> > retrieval/update.
> >
> > TAKE A LOOK AT MICROSOFT PATTERNS & PRACTICES
> >
> >[/color][/color]


DancesWithBamboo
Guest
 
Posts: n/a
#17: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


To understand why the solution you had working in clasic asp doesn't perform
in .net, you have to understand the internals of how these systems work.

In asp, every page was intrepreted runtime. It looked for your <% %> tags
and intrepeted the code between them real time as it ran each page. If your
page didn't caontain much server-side code and was mostly content then there
wasn't much to do. Basically the system was designed to look at each page
each time it was hit as necessary. This is why almost all systems (except
your) are faster in .net.

..net on the other hand "compiles" every page into IL code when the page
changes. This IL code is then intrepeted real time by the runtime engine.
Compiling is a lot of work. It outputs new code, checks syntax, checks type
casts, checks variable declarations, function calls, and another million
things. asp would just crap out if it didn't like what it saw. The key here
is that .aspx files are compiled once for every folder and every folder every
time a file changes in it. On a normal system, you take a hit once for each
folder that has a page in it requested. Then nothing changes for a long time
so the compiler may not do anything for months.

In your system, every time you push a page out from SQL to disk you are
instructing the compiler that there is a new page that it must compile (not
to mention the file system write which could possibly be the slowest thing
you could do on a Windows box). Now if you start loading all 10000 pages
into the same folder, the work it has to do will get worse as the more pages
are requested since there will be more pages to compile.

The key to making your system work is to do some major rearchitecting or
leave it in clasic asp. Why do you want to change? If you rearchitect, you
need to only store "data" in the database and keep code in files on the
server compiled. Then you load the pages with data from the db. This is how
content management systems work.

An example would be your invoice pages. I assume that many of your "pages"
in the database are invoices. You need to just store them as text along with
whatever data you need to make the page in SQL Server (table called Invoice?)
and then have a page called invoice.aspx on the server that uses code to grab
the data and format it for display(including your graph control).

It really has never been a "good" design to put code in a database. It just
happens that classic asp was stupid enough that you could get away with it.
You need to leverage the systems for what they are good at; IIS/.net for page
serving and SQL server for data storage.
matvdl
Guest
 
Posts: n/a
#18: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Klaus,

Thankyou for your help - I must admit that at the beginning I was a little
taken back by people's negative responses to the problem - I believe that
people were far to quick to make judgement without bothering to understand
the problem - I mentioned - I think in almost every response I mentioned that
I have around 10,000 pages - and no one bothered to provide much help with
the specific problem - I think the method I have used is not too bad - just
needs to be modified to work efficiently with asp.net.

Yes - most of the pages have mostly content and there is not allot of code
in each page. So I am trying to work out how to implement something similar
to your suggestion - but still provide the flexibility of enabling server
side code where needed.

What can make this is a little difficult is my lack of experience in asp.net
- but hopefully that will improve.

The solution I was looking at would be to create some specific server side
asp.net controls for some of the more specific tasks that I have - for
instance - I need to load charts at the server and therefore need to be able
to provide the data to these charts - sometimes this could be allot of data.
I want the data to be saved in the page so I thought that a asp.net control
could be the way to go - I could pass the data to the control and this
control would already be defined on the server.

So - if I have a string of HTML text with a embedded asp.net server control
and I use the response.write function will asp.net recognize that there is a
control with in the document and load it accordingly??

So there wouldn't be any <% %> within the html - just the server controls.

If this don't work that only way that I can image doing it would be to
implement my own method of tags - search from them at run-time and extract
the code between them and use the Eval function to evaluate the code. Does
not seem like a good solution - but I think it would work.

Part of my aim is to limit the dependence of the need to store data - once
the invoice or file is produced ideally I need future requests to only look
at this one file for it to display correctly - therefore the reasoning on
imbedding the data within the file - but that does not mean that the code
could be pulled out of the file and placed elsewhere. Having as few
dependencies as possible on what is required to display the page was part of
the original design and one that ensures that future system changes will not
inadvertently make viewing older invoices fail. Something that has worked
well over the past 5 years.

Are these the sorts of ideas you were thinking off??


--
matthew


"Klaus H. Probst" wrote:
[color=blue]
> The following suggestion may or may not work for you, but I'll chime in.
>
> First off, to the folks that called the approach your existing app uses
> "idiotic", I've seen classic ASP applications design this way that work
> extremely well, are scalable and extremely manageable. However, the devil is
> in the details. Just putting "pages" in IMAGE or TEXT fields on a SQL
> database isn't some sort of magic pixie dust that solves all problems. It
> takes a lot of thought and experimentation to get just right, and the
> rewards in most cases offset the negative aspects of these designs.
>
> Having said that, your approach is, as you've figured out by now, wrong.
> Others have already mentioned the perf hit you're taking by forcing ASP.NET
> to function in a way it wasn't designed to. But there are ways to get around
> the problem.
>
> First off, I'll assume that you have more content than code. That is, *most*
> of your existing ASP/ASPX pages in the database contain stuff, like invoices
> and whatnot, that do not require intervention by the parser to load and
> display. If this is not the case then stop reading now =)
>
> There are a lot of web applications nowadays that use their databases as
> primary content storage repositories. Blogs are a good example of this.
> However, they do not store code in the database, they store content. If you
> can adapt to this type of scenario then all you have to do is create a sort
> of "master page" or maybe an IHttpHandler implementation that looks at the
> request (even a path), pulls the *content* from the database and then
> displays it on a sort of templated page container. Think of a site like
> MSDN. Lots of articles, yet all the pages look the same. The content is what
> varies. The trick is to simply treat your "pages" as opaque chunks of
> content, and "stream" them into a placeholder page that is rendered the same
> way every time. You can pretty much do anything you want once you're hooked
> into the ASP.NET processing pipeline.
>
> If you do this you'll get rid of your perf problem and you'll still have the
> flexibility and manageability of the database storage. This is web
> application design 101 - always separate content from function and
> structure.
>
> Of course getting from point A to point B could be tricky, but maybe this
> gives you some ideas =)
>
> --
> Klaus H. Probst, MVP
> http://www.simulplex.net/
>
>
> "matvdl" <matvdl@discussions.microsoft.com> wrote in message
> news:DF375EB0-8373-4D91-B7F3-0400A8AE22E9@microsoft.com...[color=green]
> > DLM,
> >
> > Thanks for your response.
> >
> > I don't believe that SQL is my problem - the time it takes to do the[/color]
> remote[color=green]
> > calls to the external program has not presented any real limitations or
> > caused any significant time-delays. The problem that I have is trying to
> > deal with the quanitity of files that I have and the combiling of those[/color]
> pages[color=green]
> > once they have been returned. Any time issues that I have had are the[/color]
> result[color=green]
> > of parsing and compilling the page.
> >
> > I understand your point - that asp.net was designed to have the pages
> > pre-compiled before requests where made for the pages - I guess this is[/color]
> one[color=green]
> > of the fundamental differances between asp and asp.net. From experiance[/color]
> the[color=green]
> > design that has been implemented had no significant disadvantages when
> > working with asp - in fact I believe that it had some very significant
> > advantages - mainly due to the unlimited number of files it could manage.
> >
> > Question - is it practicale to have 10,000 pages in a single asp.net
> > application. I wouldn't have thought so?
> >
> > I have no alternative but to deal with this many files as it is a legal
> > requirment of the industry I work in is to be able to provide the original
> > copies of invoices to clients - I can't go back to the original data and
> > attempt to re-produce the same invoice just in case the data has changed.
> >
> > So - considering this is what I am stuck with - my problem mainly relates[/color]
> to[color=green]
> > the memory increases - the slowness of the request could be dealt with by[/color]
> the[color=green]
> > more grunt - but the ever increasing memory is more difficult. I believe
> > that this is the result of the creation of a new class each time a new[/color]
> page[color=green]
> > is displayed - although this page gets un-loaded once the info is[/color]
> returned -[color=green]
> > are the increases in memory the result of the new class definition in the
> > asp.net application?
> >
> > If this is the case - is it possible to remove this class definition once[/color]
> it[color=green]
> > has been created? I have tried to delete the file - but this does not[/color]
> appear[color=green]
> > to fix the problem.
> >
> > Does any of this make sense.
> >
> >
> > --
> > matthew
> >
> >
> > "DLM" wrote:
> >[color=darkred]
> > > Go to this link:
> > > http://www.theserverside.net/books/i...ngPerf/pag.tss
> > >
> > > We all know that there is an initial performance 'hit' the first time a[/color][/color]
> ASP[color=green][color=darkred]
> > > .NET Page is requested, and this is true with the page sitting in it's
> > > virtual directory on IIS. What you seem to be doing is increasing the[/color][/color]
> round[color=green][color=darkred]
> > > trip time for any given request/response for a page to include the time[/color][/color]
> it[color=green][color=darkred]
> > > takes to get the page from SQL Server. And this added time is database[/color][/color]
> access[color=green][color=darkred]
> > > time which is probably longer than any HTTP request.
> > >
> > > I have never seen this kind of architecture before. The whole idea is to
> > > serve pages from a highly scalable architecture such as IIS from the[/color][/color]
> very[color=green][color=darkred]
> > > start, and not make the ASP .NET worker process wait for the files[/color][/color]
> before it[color=green][color=darkred]
> > > can dynamically compile pages. In a sense you are using SQL Server as a[/color][/color]
> web[color=green][color=darkred]
> > > server to your web server, which is highly unusual. You need to find a[/color][/color]
> way to[color=green][color=darkred]
> > > place all your pages on the web server and use SQL Server only for data
> > > retrieval/update.
> > >
> > > TAKE A LOOK AT MICROSOFT PATTERNS & PRACTICES
> > >
> > >[/color][/color]
>
>
>[/color]
matvdl
Guest
 
Posts: n/a
#19: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


DancesWithBamboo,

Thankyou for your help - your response I believe hits the nail on the head.

I am trying to look at my options on pulling the code out of the pages and
placing it in asp.net - not sure exactly how to do it - but are looking at
different options. If you look at one of my recent replies to Klaus I have
outlined a few of my ideas.

I found the asp solution very flexible and it provided a good solution to
the issues I was having - but I was forced to move to asp.net due to
reliability issues and a need to use webservices. Asp pages still get served
occasionally from the server as they were created 3-4 years ago and the
problems I have been having with asp.net have not really caused a great
problem until recently. I guess its time to do some more up-grading.

Thanks for your help.

--
matthew


"DancesWithBamboo" wrote:
[color=blue]
> To understand why the solution you had working in clasic asp doesn't perform
> in .net, you have to understand the internals of how these systems work.
>
> In asp, every page was intrepreted runtime. It looked for your <% %> tags
> and intrepeted the code between them real time as it ran each page. If your
> page didn't caontain much server-side code and was mostly content then there
> wasn't much to do. Basically the system was designed to look at each page
> each time it was hit as necessary. This is why almost all systems (except
> your) are faster in .net.
>
> .net on the other hand "compiles" every page into IL code when the page
> changes. This IL code is then intrepeted real time by the runtime engine.
> Compiling is a lot of work. It outputs new code, checks syntax, checks type
> casts, checks variable declarations, function calls, and another million
> things. asp would just crap out if it didn't like what it saw. The key here
> is that .aspx files are compiled once for every folder and every folder every
> time a file changes in it. On a normal system, you take a hit once for each
> folder that has a page in it requested. Then nothing changes for a long time
> so the compiler may not do anything for months.
>
> In your system, every time you push a page out from SQL to disk you are
> instructing the compiler that there is a new page that it must compile (not
> to mention the file system write which could possibly be the slowest thing
> you could do on a Windows box). Now if you start loading all 10000 pages
> into the same folder, the work it has to do will get worse as the more pages
> are requested since there will be more pages to compile.
>
> The key to making your system work is to do some major rearchitecting or
> leave it in clasic asp. Why do you want to change? If you rearchitect, you
> need to only store "data" in the database and keep code in files on the
> server compiled. Then you load the pages with data from the db. This is how
> content management systems work.
>
> An example would be your invoice pages. I assume that many of your "pages"
> in the database are invoices. You need to just store them as text along with
> whatever data you need to make the page in SQL Server (table called Invoice?)
> and then have a page called invoice.aspx on the server that uses code to grab
> the data and format it for display(including your graph control).
>
> It really has never been a "good" design to put code in a database. It just
> happens that classic asp was stupid enough that you could get away with it.
> You need to leverage the systems for what they are good at; IIS/.net for page
> serving and SQL server for data storage.[/color]
Bruce
Guest
 
Posts: n/a
#20: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Hi,
Convert the aspx pages to controls and use Loadcontrol from a single aspx by
looking up the ID of the SQL stored control! As well you could cache the
controls to improve performance.

regards
Brucek

"matvdl" wrote:
[color=blue]
> I have a system that was originally developed in asp - the pages are saved in
> SQL (there are over 10,000 pages) and saved to a temp directory in the server
> when requested by a client.
>
> I have updated this system and changed the pages that are saved to the
> server as aspx - everything works fine and pages can be served - but
>
> Its not impossible for a single client to request 100 plus pages in one
> session - as each page is requested it is retrieved from SQL saved to the
> temp directory and compiled - problems are.
> - Performance is well down on the original asp system - I believe that
> this due to the compiling of the pages in asp.net
> - The memory usage of server also goes through the roof - there is a
> considerable increase in memory as each page is loaded
>
> Are there any solutions to these problems??
>
> --
> matthew[/color]
Bruce
Guest
 
Posts: n/a
#21: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


You could also use ParseControl to pass them as a string from the database,
Loadcontrol would load from a file.
Basically in pseudocode:
1. Put a placeholder in a aspx page
2. Get id of stored control from calling page
3. Get Text of ascx in database
4. New Control = Parsecontrol(text)
5. Placeholder.Controls.Add(Control)

Place 2-5 in loop if more than 1 control per page

This way you get one aspx, with 1 or more content controls without redirects
etc.

Regards
Bruce


"Bruce" wrote:
[color=blue]
> Hi,
> Convert the aspx pages to controls and use Loadcontrol from a single aspx by
> looking up the ID of the SQL stored control! As well you could cache the
> controls to improve performance.
>
> regards
> Brucek
>
> "matvdl" wrote:
>[color=green]
> > I have a system that was originally developed in asp - the pages are saved in
> > SQL (there are over 10,000 pages) and saved to a temp directory in the server
> > when requested by a client.
> >
> > I have updated this system and changed the pages that are saved to the
> > server as aspx - everything works fine and pages can be served - but
> >
> > Its not impossible for a single client to request 100 plus pages in one
> > session - as each page is requested it is retrieved from SQL saved to the
> > temp directory and compiled - problems are.
> > - Performance is well down on the original asp system - I believe that
> > this due to the compiling of the pages in asp.net
> > - The memory usage of server also goes through the roof - there is a
> > considerable increase in memory as each page is loaded
> >
> > Are there any solutions to these problems??
> >
> > --
> > matthew[/color][/color]
Klaus H. Probst
Guest
 
Posts: n/a
#22: Nov 19 '05

re: Using SQl to store aspx pages and memory problems


Matthew,

"matvdl" <matvdl@discussions.microsoft.com> wrote in message
news:EFDE3AA2-B54E-4E62-A11F-9A523E4C8067@microsoft.com...
[color=blue]
> So - if I have a string of HTML text with a embedded asp.net server[/color]
control[color=blue]
> and I use the response.write function will asp.net recognize that there is[/color]
a[color=blue]
> control with in the document and load it accordingly??[/color]

No, the template parser is long done after you get to write to the HTTP
output stream.
[color=blue]
> If this don't work that only way that I can image doing it would be to
> implement my own method of tags - search from them at run-time and extract
> the code between them and use the Eval function to evaluate the code.[/color]
Does[color=blue]
> not seem like a good solution - but I think it would work.[/color]

You can embed type names or something like that, and then use some sort of
class factory implementation to instantiate the "controls" that need to
render themselves. Pass the HttpContext reference to them or some such so
they can play. Remember that a "control" in ASP.NET is something that just
renders some HTML. They're nothing special and they don't have to be loaded
from disk or have an ASCX extension (although that has its advantages).
Still, avoid that sort of thing as much as you can.
[color=blue]
> Part of my aim is to limit the dependence of the need to store data - once
> the invoice or file is produced ideally I need future requests to only[/color]
look[color=blue]
> at this one file for it to display correctly - therefore the reasoning on
> imbedding the data within the file - but that does not mean that the code
> could be pulled out of the file and placed elsewhere. Having as few
> dependencies as possible on what is required to display the page was part[/color]
of[color=blue]
> the original design and one that ensures that future system changes will[/color]
not[color=blue]
> inadvertently make viewing older invoices fail. Something that has worked
> well over the past 5 years.[/color]

Ideally you never store data with presentation metadata (your markup), but I
will recognize that in some cases it makes sense. But storing the data with
the markup is not a good idea just because you think the application will
change someday - you design applications to be resilient and adaptable to
most foreseeable changes.

--
Klaus H. Probst, MVP
http://www.simulplex.net/


Closed Thread