[logs] regexless parsing, again?

[Tom Le]

On 9/17/07, Daniel Cid <danielcid at yahoo.com.br> wrote:
| First of all, I think most projects do log analysis
| wrong. They confuse log decoding with rule matching
| and end up with hundred of regexes that are checked on
| every log. Regexes can be used to extract some bits of
| patterns from the logs, but not as the main method to
| do the log analysis...

I think scale is an important factor to discuss when we talk about
performance.  With advances in CPU speeds, even the worse case scenario of
looping through N regex rules is not that bad for most applications.  You
mentioned OSSEC with 500 rules, which isn't that bad a problem even if you
had to try all of them.

Now if you had 5k, 50k, or 500k rules... then scale becomes a factor.
Similarly, you have to look at throughput as well... do you need to parse
100 messages per second or 10,000?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.loganalysis.org/pipermail/loganalysis/attachments/20070917/5b614d25/attachment.html