473,550 Members | 2,452 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Building a Syslog Server

Hello.

I am needing some advice. I need to build a syslog server that writes
to a sql backend.
I am concerned about the server not being able to handle multiple
requests. For example on an enterprise network, I may need to handle a
large number of requests. I assume a thread pool will be needed but I'm
wondering what other advice some of you might have?

Thanks in advance.

-A

Jan 21 '06 #1
8 11315
Guy,

You wouldn't need to do this at all. SQL Server is built from the
ground up to handle multiple requests from multiple sources and not worry
about contention.

The only thing you have to worry about is if you are going to issue
multiple statements at the same time, and want them treated as an atomic
action (using a transaction), but that logic is up to you.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m
"Guy Noir" <ah******@gmail .com> wrote in message
news:11******** **************@ z14g2000cwz.goo glegroups.com.. .
Hello.

I am needing some advice. I need to build a syslog server that writes
to a sql backend.
I am concerned about the server not being able to handle multiple
requests. For example on an enterprise network, I may need to handle a
large number of requests. I assume a thread pool will be needed but I'm
wondering what other advice some of you might have?

Thanks in advance.

-A

Jan 21 '06 #2
Nicholas, thanks for the reply.

What I am worried about is when the server is handling the inbound udp
message will it miss other incoming messages?

Thanks again
-A

Jan 21 '06 #3
Guy,

You shouldn't be worried about SQL Server when it handles incoming
messages. It handles concurrent messages fine on it's own.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"Guy Noir" <ah******@gmail .com> wrote in message
news:11******** **************@ g49g2000cwa.goo glegroups.com.. .
Nicholas, thanks for the reply.

What I am worried about is when the server is handling the inbound udp
message will it miss other incoming messages?

Thanks again
-A

Jan 21 '06 #4
Guy,
you would need to build a multithreaded UDPClient Server (probably a windows
service) to receive the SysLog messages.

I just finished one and I used Ami Bar's "Smart Threadpool" to handle the
thread pooling needs. I'd happily share some code, but I can't under my NDA.
Hope that helps.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Guy Noir" wrote:
Hello.

I am needing some advice. I need to build a syslog server that writes
to a sql backend.
I am concerned about the server not being able to handle multiple
requests. For example on an enterprise network, I may need to handle a
large number of requests. I assume a thread pool will be needed but I'm
wondering what other advice some of you might have?

Thanks in advance.

-A

Jan 21 '06 #5

"Guy Noir" <ah******@gmail .com> wrote in message
news:11******** **************@ z14g2000cwz.goo glegroups.com.. .
Hello.

I am needing some advice. I need to build a syslog server that writes
to a sql backend.
I am concerned about the server not being able to handle multiple
requests. For example on an enterprise network, I may need to handle a
large number of requests. I assume a thread pool will be needed but I'm
wondering what other advice some of you might have?

Thanks in advance.

-A


Rather than reinventing the wheel, you may consider looking at Kiwi Syslog
Daemon. It runs as a Windows service and will log to number of different
targets, such a text files or ODBC data sources. We're using it on our
network to capture Syslog output from a number of network devices and Unix
servers. We have it configured to log to a SQL Server database. It works
quite well and the prices isn't too bad either, about $100, I believe.

http://www.kiwisyslog.com

CVD
Jan 21 '06 #6
Excellent idea, I never even thought to mention it.

Although in my case we need to be able to do message / priority / facility
parsing with a rules engine that can forward to another SysLog server, send
an SMS or email message, forward to MSMQ, and log to a SQL Server backend all
at the same time.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Cletus Van Damme" wrote:

"Guy Noir" <ah******@gmail .com> wrote in message
news:11******** **************@ z14g2000cwz.goo glegroups.com.. .
Hello.

I am needing some advice. I need to build a syslog server that writes
to a sql backend.
I am concerned about the server not being able to handle multiple
requests. For example on an enterprise network, I may need to handle a
large number of requests. I assume a thread pool will be needed but I'm
wondering what other advice some of you might have?

Thanks in advance.

-A


Rather than reinventing the wheel, you may consider looking at Kiwi Syslog
Daemon. It runs as a Windows service and will log to number of different
targets, such a text files or ODBC data sources. We're using it on our
network to capture Syslog output from a number of network devices and Unix
servers. We have it configured to log to a SQL Server database. It works
quite well and the prices isn't too bad either, about $100, I believe.

http://www.kiwisyslog.com

CVD

Jan 21 '06 #7
>Excellent idea, I never even thought to mention it.

Although in my case we need to be able to do message / priority / facility
parsing with a rules engine that can forward to another SysLog server, send
an SMS or email message, forward to MSMQ, and log to a SQL Server backend all
at the same time.


You can do that with Kiwi Syslog and NotePage's software - they have some info
about it here: http://www.notepage.net/kiwi-syslog/kiwi-syslog.htm

HTH
R. Crane

Jan 22 '06 #8
Peter, thanks for the link to Ami Bar's "Smart Threadpool".

I have used kiwi in the past as some have suggested, however I'm trying
to develop a turnkey package. (I have the client piece/service already
written.)

Do you have a suggestion as to the number of threads in the pool vs the
number of simultaneous udp syslog messages being fired at the server?

tia!!

-Aaron

Jan 23 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
2944
by: Erich Trowbridge | last post by:
Has anybody seen this tool? It is awesome. check out http://vw.vermeer.org/ . It's a php front end for large-scale syslog deployments. It makes managing syslog in large networks a snap. The idea is to fifo pipe network syslog into a MySQL database backend. The php scripts reference the database, and print stuff to the screen. My Info ->...
5
3383
by: Micha³ Wo¼niak | last post by:
Hi again The Question of Logs, part II. :) What are the advantages/disadvantages of using syslog() to do the logging rather than writing the logs with fopen(), fwrite() and fclose()? What should be kept in mind while using either technique? And, of course, which is considered to be the better one? TIA
0
4624
by: Stephen Touset | last post by:
--=-vqiBeaRpc7WKjfgJmDzK Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Is there now a way within MySQL to log to syslog? I was reading an archived email conversation between Sinisa Milivojevic and Vincent Stoessel in which they seemed to indicate that there was no need to log to syslog. As Vincent only had a request...
1
3710
by: Gregory Starck | last post by:
Hello, I'm having what seems to be a deadlock in syslog with the code (wich comes from a project I'm working on, but I've reduced the test case to its minimum) I put at the end of this messsage. It must be compiled with -lpthread. (so the fork could be also a possible cause of what I'm mentionning). The "bug" does not occur every times....
1
2516
by: liujixuan | last post by:
I am a new bie of Python. I knew there is module for syslog of Linux, but I want to create a syslog server on the windows server like windows 2k or 2k3, to collect the log message sent from our firewall. Any one has any idea about this? Many thanks. Jason Liu
1
5981
by: pconrad | last post by:
I've got a syslog server posting to MS SQL. It works great as far as posting all the data. However, it puts all the useful information into one big text field called message. How can I parse that data & place into separate data fields? Sample text: 3625: Jun 9 13:35:54.392: %CRYPTO-MESSAGE_UP: (Server) Mode=CLIENTMODE Client_type=UNKNOWN...
3
6137
by: Troels Arvin | last post by:
Hello, On a server with DB2 v. 8.2, db2diag.log ran full, it seems (i.e. reached a size of 2GB). And it seems that in this case, DB2 started using syslog instead. I would actually like DB2 to use syslog instead of / in addition to the local log files, not only in case of a problem with the local log files. Is that possible?
1
2855
by: anjaliraman2 | last post by:
Hi All, I am facing a problem of writing into and reading from syslog in Linux Enviornment using C++. The CPU utilization is extremely high of about 55% and is creating problems for other processes. ideally the CPU utilization has to be only max of 10% not more. I have used getline() for reading. I hope u will please provide me a suitable...
1
6683
by: Lincoln Yeoh | last post by:
Hi, I've just started to learn python (I've been using perl for some years). How do I redirect ALL stderr stuff to syslog, even stderr from external programs that don't explicitly change their own stderr? Say I have a program called foo: #!/usr/bin/python
0
7490
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7765
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7846
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6104
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5404
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5129
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
1
1980
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1096
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
802
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.