Tomcat 4.1.x Performance advice - 6 servers for 4000 users?

[nmac]

Hi all, hopefully someone can offer some sagely advice regarding
Production use of Jakarta's Tomcat.
First, some brief background.
My company have a servlet application that connects to a MySQL
database.
The servlet is deployed on two seperate win2k servers (Access to the
tomcat servers is via DNS round robin load balancing).

The database is on a another win2k server.

These three servers are used for no other purpose than the servlet
application.

Three or four times a year, 4000 employees are required to access the
servlet application across a time period of 14 days, each user may
access the servlet many times during this period - placing a lot of
load on the network, and the servers.

Historically response time has been a big issue.
Assume Network Capacity cannot be improved...:-(

The tomcat/servlet solution was provided by a third party vendor.

The vendor is now suggesting we increase the webserver count from two
Windows2K Server machines to a total of SIX, all running Tomcat.
Personally, I think this is overkill, and the vendor is simply pushing
boxes at the problem as they do not really know how to tune tomcat.

I don't want to get into a debate about the various merits or not of
Linux/Windows, but I do want to ask, if anyone knows of any links to
benchmarks that show some comparison between tomcat on Win2K and
Linux? and even some ballpark figures of typical users/servers.

This application is actually quite simple, there is no heavy
processing on the servlet side.

I also recall some time back, that tomcat on Windows was not
recommended for Production use, but I can't recall where I read that,
or even if I read it at all!!
Now, while I know that it runs fine on Win2K for many, I am simply
interested if anyone has any links to articles that may offer a more
informed opinion than mine.

Any advice is really welcomed, thanks in advance.

Nmac
:-)

[Dave Ockwell-Jenner]

"nmac" <nm*****@hotmail.com> wrote in message
news:44**************************@posting.google.c om...
[snip]

Three or four times a year, 4000 employees are required to access the
servlet application across a time period of 14 days, each user may
access the servlet many times during this period - placing a lot of
load on the network, and the servers.
What does the "hits/day" profile look like? Do the majority of those 4,000
employees interact with the application at the start of the 14 days, near
the end, or semi-uniform across the time span?
The tomcat/servlet solution was provided by a third party vendor.
Since nearly all performance problems (IMHO) are application-related and not
infrastructure-related, is the vendor interested in profiling and tuning
their application. I have seen a number of bottlenecks that arise due to
design decisions in the application.
The vendor is now suggesting we increase the webserver count from two
Windows2K Server machines to a total of SIX, all running Tomcat.
Personally, I think this is overkill, and the vendor is simply pushing
boxes at the problem as they do not really know how to tune tomcat.
This *sounds* like the vendor is just throwing horsepower at the problem.
What kind of hardware are they suggesting for these machines?
I don't want to get into a debate about the various merits or not of
Linux/Windows, but I do want to ask, if anyone knows of any links to
benchmarks that show some comparison between tomcat on Win2K and
Linux? and even some ballpark figures of typical users/servers.
Personally, I have supported a web application for 15,000 (internal) and
1,000 (external) users running on a single instance of Tomcat. Yes, this
was on Solaris, but not on anything you could consider 'extreme' hardware.
This application is actually quite simple, there is no heavy
processing on the servlet side.
I'm assuming that you have tested this theory out with network traces, etc.?
Sometimes, even simple operations are implemented with excessive waste in
terms of transactions.
I also recall some time back, that tomcat on Windows was not
recommended for Production use, but I can't recall where I read that,
or even if I read it at all!!
Now, while I know that it runs fine on Win2K for many, I am simply
interested if anyone has any links to articles that may offer a more
informed opinion than mine.

It's true that Tomcat is not the fastest cat on the block. BUT, it's easy
to use and relatively stable, so I think the trade-off is not all that bad
in reality, unless you have some super-performance critical requirements. I
don't have any articles I can point to that would provide any solid
information, just my own experience.
--
Dave Ockwell-Jenner
Solar Nexus Solutions
http://www.solar-nexus.com/

[Nathan Zumwalt]

Tomcat is the official JSP/Servlet reference implementation, and as
such (I believe) isn't recommended for production use on any platform.
I've had good luck running Tomcat, though, and most people just
ignore this.

I've heard that under heavy load, Tomcat doesn't perform very well
(this doesn't surprise me because Apache is concerned with correctly
implementing the features of the spec, not optimizing performance). I
would check out Resin or Jetty, both of which are supposed to be solid
performers.

I would also do some testing of your application and determine the
bottle neck. If the application logic in Tomcat is so simple, is the
database bogging down?

-Nathan

nm*****@hotmail.com (nmac) wrote in message news:<44**************************@posting.google. com>...

Hi all, hopefully someone can offer some sagely advice regarding
Production use of Jakarta's Tomcat.
First, some brief background.
My company have a servlet application that connects to a MySQL
database.
The servlet is deployed on two seperate win2k servers (Access to the
tomcat servers is via DNS round robin load balancing).

The database is on a another win2k server.

These three servers are used for no other purpose than the servlet
application.

Three or four times a year, 4000 employees are required to access the
servlet application across a time period of 14 days, each user may
access the servlet many times during this period - placing a lot of
load on the network, and the servers.

Historically response time has been a big issue.
Assume Network Capacity cannot be improved...:-(

The tomcat/servlet solution was provided by a third party vendor.

The vendor is now suggesting we increase the webserver count from two
Windows2K Server machines to a total of SIX, all running Tomcat.
Personally, I think this is overkill, and the vendor is simply pushing
boxes at the problem as they do not really know how to tune tomcat.

I don't want to get into a debate about the various merits or not of
Linux/Windows, but I do want to ask, if anyone knows of any links to
benchmarks that show some comparison between tomcat on Win2K and
Linux? and even some ballpark figures of typical users/servers.

This application is actually quite simple, there is no heavy
processing on the servlet side.

I also recall some time back, that tomcat on Windows was not
recommended for Production use, but I can't recall where I read that,
or even if I read it at all!!
Now, while I know that it runs fine on Win2K for many, I am simply
interested if anyone has any links to articles that may offer a more
informed opinion than mine.

Any advice is really welcomed, thanks in advance.

Nmac
:-)

[Steve Jasper]

Here's some interesting data that may be of use to you:

http://www.webperformanceinc.com/library/ServletReport/

From my experience, I believe based on your load and load balancing
scheme that Tomcat will not be your bottleneck, especially since the
Servlet you are using does not do any major processing. These days,
using Tomcat for a production environment isn't an absolute no-no, in
fact, quite a few folks find it performs quite well.

What I would pay more attention to is the database, specifically the
manner by which the connections to the database are made. MySQL is
pretty good....Oracle or MS SQL 2000 would perform better, but
realistically that ship has likely sailed at this point. Outside of
using a better performing database, I would make sure that connections
to the database are pooled, since creating a new connection to the
database for every query is likely to be the most memory intensive
process of your application. Moving from creating connections on
demand to pulling connections from a pool of existing connections will
result in a dramatic performance enhancement.

Good luck, tomcat has come a long way in the past couple years and I
rely on it exclusively....

nm*****@hotmail.com (nmac) wrote in message news:<44**************************@posting.google. com>...

Hi all, hopefully someone can offer some sagely advice regarding
Production use of Jakarta's Tomcat.
First, some brief background.
My company have a servlet application that connects to a MySQL
database.
The servlet is deployed on two seperate win2k servers (Access to the
tomcat servers is via DNS round robin load balancing).

The database is on a another win2k server.

These three servers are used for no other purpose than the servlet
application.

Three or four times a year, 4000 employees are required to access the
servlet application across a time period of 14 days, each user may
access the servlet many times during this period - placing a lot of
load on the network, and the servers.

Historically response time has been a big issue.
Assume Network Capacity cannot be improved...:-(

The tomcat/servlet solution was provided by a third party vendor.

The vendor is now suggesting we increase the webserver count from two
Windows2K Server machines to a total of SIX, all running Tomcat.
Personally, I think this is overkill, and the vendor is simply pushing
boxes at the problem as they do not really know how to tune tomcat.

I don't want to get into a debate about the various merits or not of
Linux/Windows, but I do want to ask, if anyone knows of any links to
benchmarks that show some comparison between tomcat on Win2K and
Linux? and even some ballpark figures of typical users/servers.

This application is actually quite simple, there is no heavy
processing on the servlet side.

I also recall some time back, that tomcat on Windows was not
recommended for Production use, but I can't recall where I read that,
or even if I read it at all!!
Now, while I know that it runs fine on Win2K for many, I am simply
interested if anyone has any links to articles that may offer a more
informed opinion than mine.

Any advice is really welcomed, thanks in advance.

Nmac
:-)

[nmac]

Dave Thanks for taking the time to respond to my post.
In answer to your queries:
[my snip]
Three or four times a year, 4000 employees are required to access the
servlet application across a time period of 14 days, each user may
access the servlet many times during this period - placing a lot of
load on the network, and the servers.
[your snip]
What does the "hits/day" profile look like? Do the majority of those
4,000
employees interact with the application at the start of the 14 days,
near
the end, or semi-uniform across the time span?

[nmac says:]
The hits/day is relatively uniform, peaking towards the end of each
period.

[my snip]
The tomcat/servlet solution was provided by a third party vendor.
[your snip]
Since nearly all performance problems (IMHO) are application-related
and not
infrastructure-related, is the vendor interested in profiling and
tuning
their application. I have seen a number of bottlenecks that arise due
to
design decisions in the application.

[nmac says:]
Personally, I don't think the vendor is capable of tuning the
application. I have begun to look into the config on the tomcat
servers, and will look into connection pooling for the DB connect if
none is currently used.
I tend to agree that it is application/config related. Throwing boxes
at the problem seems to be a blatant attempt to get more sales.
[my snip]
The vendor is now suggesting we increase the webserver count from two
Windows2K Server machines to a total of SIX.
[your snip]
This *sounds* like the vendor is just throwing horsepower at the
problem.
What kind of hardware are they suggesting for these machines?

[nmac says:]
Agreed! Hardware are Dual P4 servers, 1GBram. SCSI RAID 5 (I would use
0+1 personally!)

[my snip:]
I don't want to get into a debate about the various merits or not of
Linux/Windows...
[your snip:]
Personally, I have supported a web application for 15,000 (internal)
and
1,000 (external) users running on a single instance of Tomcat...

[nmac says:]
Well, this is where I am suspicous that the vendor has their sales
targets at heart. I too have run applications on tomcat on both
Solaris and Linux, with massive hits/sec requirements, and user
audiences in their thousands, with no real performance problems.
[your snip]
I'm assuming that you have tested this theory out with network traces,
etc.?
Sometimes, even simple operations are implemented with excessive waste
in
terms of transactions.

[nmac says:]
I am working on getting more detail on the architecture, as I have
been pulled into this issue, and yes, traces are in my plans, unless
of course the architecture is obviously a mess....which I think it may
be...e.g no private segment between the WS and DB Servers....sheesh.

[my snip:]
I also recall some time back, that tomcat on Windows was not
recommended for Production use...

[your snip:]
I don't have any articles I can point to that would provide any solid
information, just my own experience.

[nmac says:]
No worries, really grateful for your insight, suggestions and
thoughts, thanks again.
Watch this space!!
Cheers, Nmac.
:-)

[nmac]

Cheers Nathan and Steve, really grateful for the ideas, insight and
the link!
I'll keep digging, and if anything rears its head will update the
thread. Very grateful.
nmac:-)