473,320 Members | 1,857 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Concurrent Logging: How?

Hi all,

I have an enterprise application. I am using Apache Log4J for the logging
purposes. WHen a request is received by the application, it goes throug
servlets and many classes, and the necessary details are logged to files,
which has autorate option. I mean when certain file exceeds the size,
automatically another file is generated for a maximum of 10 files and then
again starts from the first.

So far is so good. Now assume that say 100 users are concurrently making a
request. Since servlets are mutithreaded, the logs generated does not show
each request sequentially. All the logs of all the requests are mixed
together making it difficult and highly spagettic to read and understand.

How can I make the logging strategy such that each user request is written
in a sequential manner so that there are no overlaps of the logs? Please
advise, thanks
Best regards,
Ravi

Jul 17 '05 #1
5 7974
"Ravi Shankar" <su*********@pacific.net.sg> wrote in message
news:ck**********@nobel.pacific.net.sg...

So far is so good. Now assume that say 100 users are concurrently making a
request. Since servlets are mutithreaded, the logs generated does not show
each request sequentially. All the logs of all the requests are mixed
together making it difficult and highly spagettic to read and understand.

How can I make the logging strategy such that each user request is written
in a sequential manner so that there are no overlaps of the logs? Please
advise, thanks

Isn't this more an issue of having an appropriate viewer for the log?
Assuming you create log entries that identify the user, by session id or
whatever, you just need a viewer that will group them appropriately. If you
take this strategy, you will of course need to format the log entries
consistently. The alternative strategy, of grouping the messages before
they are written to the log, would be bad because generally you want to
write the messages to a physical medium as soon as possible, for obvious
reasons. You could write the messages to a database, in which case queries
on user would be easier, but there's an obvious downside to that as well.
So my advice: format your messages consistently, include a user field, and
implement an appropriate viewer.

Regards,
Daniel Parker
Jul 17 '05 #2
Daniel Parker wrote:
"Ravi Shankar" <su*********@pacific.net.sg> wrote in message
news:ck**********@nobel.pacific.net.sg...
So far is so good. Now assume that say 100 users are concurrently making a
request. Since servlets are mutithreaded, the logs generated does not show
each request sequentially. All the logs of all the requests are mixed
together making it difficult and highly spagettic to read and understand.

How can I make the logging strategy such that each user request is written
in a sequential manner so that there are no overlaps of the logs? Please
advise, thanks


Isn't this more an issue of having an appropriate viewer for the log?
Assuming you create log entries that identify the user, by session id or
whatever, you just need a viewer that will group them appropriately. If you
take this strategy, you will of course need to format the log entries
consistently. The alternative strategy, of grouping the messages before
they are written to the log, would be bad because generally you want to
write the messages to a physical medium as soon as possible, for obvious
reasons. You could write the messages to a database, in which case queries
on user would be easier, but there's an obvious downside to that as well.
So my advice: format your messages consistently, include a user field, and
implement an appropriate viewer.


log4j has a few tools to help you to this end. You can use NDC or MDC
to associate the user to the thread; that way you do not need to change
all of your existing log statements. Also, if you write the log in XML
format (there is a formatter in log4j for this) it can be parsed by
Chainsaw (a GUI tool from log4j) giving you a powerful viewer. You can
also configure log4j to send messages directly to Chainsaw via a socket.

For more details and assistance I suggest you refer to the log4j-users
mailing list from apache.org.

HTH,
Ray

--
XML is the programmer's duct tape.
Jul 17 '05 #3
Ravi Shankar wrote:
Hi all,

I have an enterprise application. I am using Apache Log4J for the logging
purposes. WHen a request is received by the application, it goes throug
servlets and many classes, and the necessary details are logged to files,
which has autorate option. I mean when certain file exceeds the size,
automatically another file is generated for a maximum of 10 files and then
again starts from the first.

So far is so good. Now assume that say 100 users are concurrently making a
request. Since servlets are mutithreaded, the logs generated does not show
each request sequentially. All the logs of all the requests are mixed
together making it difficult and highly spagettic to read and understand.

How can I make the logging strategy such that each user request is written
in a sequential manner so that there are no overlaps of the logs? Please
advise, thanks


You'd need to write a unique request-id in each line of the log. Then
you can apply filters (e.g. grep, perl) to extract useful information
from the log. You could write (or get someone else to write) a script
to process these log-files and collate requests.

You can also use SQL for logging, so then it's slightly easier to apply
queries and filters, for example to order by (request-id, time) instead
of just time.

Calum
Jul 17 '05 #4
The latest version of Chainsaw V2 can help you once you've added
unique identifiers to each event (via MDC) to slice & dice logs - you
can even combine client and server logs together.

Chainsaw V2 is available via WebStart here:
http://logging.apache.org/log4j/docs/chainsaw.html

Chainsaw can display and tail a log file generated by any logging
framework using LogFilePatternReceiver.

Here's a link to the LogFilePatternReceiver javadoc, describing the
'logFormat' keywords:
http://cvs.apache.org/viewcvs.cgi/lo...1.19&view=auto

If you want to tail your log file, modify the below-provided xml as
needed and save as chainsaw.xml.

Inside Chainsaw, select:
view-show application wide preferences
- enter the URL of this chainsaw.xml in the 'automatic configuration
URL' field
- restart chainsaw
chainsaw.xml:
------------------------------
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"
debug="true">

<plugin name="logFileReceiver"
class="org.apache.log4j.varia.LogFilePatternReceiv er">
<param name="fileURL" value="file:///c:/downloads/log4j/sample.log" />
<param name="timestampFormat" value="yyyyMMddHHmmssSSS"/>
<param name="logFormat" value="TIMESTAMP LEVEL(FILE:LINE)#CLASS,LOGGER
THREAD!MESSAGE"/>
<param name="name" value="logfileReceiver" />
<param name="tailing" value="true" />
</plugin>
<root>
<level value="debug"/>
</root>
</log4j:configuration>
------------------------------
Hope this helps - there is a tutorial available from the 'Welcome' tab
that explains how to use it.

Feel free to send an email to the log4j user mailing list with
questions:
http://logging.apache.org/site/mailing-lists.html

Scott
Calum Grant <ca****@no.onetel.spam.com> wrote in message news:<_P**************@newsfe6-gui.ntli.net>...
Ravi Shankar wrote:
Hi all,

I have an enterprise application. I am using Apache Log4J for the logging
purposes. WHen a request is received by the application, it goes throug
servlets and many classes, and the necessary details are logged to files,
which has autorate option. I mean when certain file exceeds the size,
automatically another file is generated for a maximum of 10 files and then
again starts from the first.

So far is so good. Now assume that say 100 users are concurrently making a
request. Since servlets are mutithreaded, the logs generated does not show
each request sequentially. All the logs of all the requests are mixed
together making it difficult and highly spagettic to read and understand.

How can I make the logging strategy such that each user request is written
in a sequential manner so that there are no overlaps of the logs? Please
advise, thanks


You'd need to write a unique request-id in each line of the log. Then
you can apply filters (e.g. grep, perl) to extract useful information
from the log. You could write (or get someone else to write) a script
to process these log-files and collate requests.

You can also use SQL for logging, so then it's slightly easier to apply
queries and filters, for example to order by (request-id, time) instead
of just time.

Calum

Jul 17 '05 #5
HI all,

Thanks a lot to all for the wonderful help. I will be using Chainsaw with
NDC/MDC.

Thanks and regards,
Ravi

"Ravi Shankar" <su*********@pacific.net.sg> wrote in message
news:ck**********@nobel.pacific.net.sg...
Hi all,

I have an enterprise application. I am using Apache Log4J for the logging
purposes. WHen a request is received by the application, it goes throug
servlets and many classes, and the necessary details are logged to files,
which has autorate option. I mean when certain file exceeds the size,
automatically another file is generated for a maximum of 10 files and then
again starts from the first.

So far is so good. Now assume that say 100 users are concurrently making a
request. Since servlets are mutithreaded, the logs generated does not show
each request sequentially. All the logs of all the requests are mixed
together making it difficult and highly spagettic to read and understand.

How can I make the logging strategy such that each user request is written
in a sequential manner so that there are no overlaps of the logs? Please
advise, thanks
Best regards,
Ravi

Jul 17 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Huzefa | last post by:
I am working on a amll project in Java that includes many classes. Each of the classes has a Logger object. I have associated a FileHandler with each of these Logger objects. The file is the same...
6
by: JT | last post by:
i have a text file that is used as a logfile for easier debugging. i am using the OpenTextFile method to write to this file. this method exists inside a logging function that can be called from...
2
by: BenM | last post by:
Description: I would like to prevent a user from logging in with their user/password combination on a different computer or even a different browser window, if they are already logged in. I have...
1
by: Roman Ziak | last post by:
I switched to Windows server and logs generated by my ISP are pathetic comparing to those from Apache. I would like to do logging via PHP and use the same log for visits and for PHP tracing. That...
2
by: runner7 | last post by:
Can anyone tell me if there is a way in PHP to determine when a session times out on the server or how many concurrent sessions there are in your application?
1
by: Jay Douglas | last post by:
Sorry for the cross post but I was hoping between developers and systems ppl I can get answer to this question. I've searched; found issues similar to mine but the solutions are not working. My...
2
by: mktselvan | last post by:
Hi, Existing running oracle application 11i (11.5.8) Database version is 8.1.7.4 There is any command / way to know the number of concurrent users for this application. ...
1
by: sanjupommen | last post by:
I am in the process of exploring the possibility of providing our products on databases other than Oracle.I am able to migrate the data, procedures etc without too much effort (latest version of DB2...
0
amitpatel66
by: amitpatel66 | last post by:
There is always a requirement that in Oracle Applications, the Concurrent Program need to be execute programatically based on certain conditions/validations: Concurrent programs can be executed...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.