[logs] naming multiple output files with syslog-ng

[Tom Le]

On Dec 17, 2007 6:20 AM, Christian Folini <christian.folini at post.ch> wrote:
> I managed to get quite close by configuring ErrorLog
> in apache as follows:
> ErrorLog "| /usr/bin/logger -t www.example.com_port80 -u
/tmp/logger.socket"
> ErrorLog "| /usr/bin/logger -t www.example.com_port443 -u
/tmp/logger.socket"
>
> And then configure syslog-ng along the lines of
> destination df_file { file(".../$PROGRAM/$HOST/apache-error.log"); };

Logger is really expensive used in this way unless your log files are
relatively small.  Can you use native syslog forwarding capability by
defining a custom facility/priority in Apache config?  (Also, technically,
the service name should be 32 or fewer characters per RFC3164... so it's
possible using this format could violate that for long hostnames.)

You may want to consider not using the service name in this way as it can
limit your ability to do other types of analysis later using tools that rely
on the service name.
If you want to perform any customized parsing of the error logs, you can
pipe to Perl (it's not as bad as people perceive) or build a small compiled
application to do the custom formatting for you.  You don't have to build it
-- say a C application -- from scratch (though not that bad given a small
scope).  You can use the traditional Lex/Yac/Bison approach and just add
syslog forwarding (lots of source code floating around for syslog
forwarding).

There is also Snare for Apache (
http://www.intersectalliance.com/projects/SnareApache/index.html) which you
can modify source code for your custom format and add parsing rules.  The
Snare Text agent is pretty straightforward and you can use that as an idea
for how to build your own parser + forwarder.

Finally, I'd add one final question to consider.  How critical are your
logs?  Most centralized syslog logging solutions, even those using TCP only
delivery, endure some small amount of message loss (for a variety of
differnet reasons).  So does your logging have to ensure delivery with
queueing/store & forward mechanisms?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.loganalysis.org/pipermail/loganalysis/attachments/20071217/8172bd03/attachment.html