[logs] regexless parsing, again?

[Raffael Marty]

Anton,

I am not sure if you are asking the right question. I don't think  
regexes are necessarily bad. However, they are bad if you apply them  
on the incoming data stream... I list a lot of those disadvantages on  
my blog:

http://raffy.ch/blog/2007/08/25/event-processing-normalization/

I am not quite sure what you are after with your question about  
regexless parsing. You need to parse for certain uses of your data.  
The question is just _when_ do you parse. It's not really _how_. I  
couldn't care less, as long as the performance is decent and I get  
the desired results quickly.

Cheers

   -raffy

Disclaimer: I work for Splunk>

--
   Raffael Marty
   Chief Security Strategist @ Splunk>
   Security Visualization: http://secviz.org       raffy.ch/blog


On Sep 13, 2007, at 11:56 AM, Anton Chuvakin wrote:

> All,
>
> I think it is a good time to revisit this fun subject that we
> _revisited_ back in 2005: regexless log message processing. (e.g. see
> my post "regex-less parsing of messages" and the prolonged discussion
> that followed here:
> http://lists.jammed.com/loganalysis/2005/12/index.html)
>
> So, has the world changed since that glorious time? :-) I think it
> did, but only a little. We do have a lot more weird logs to analyze,
> log indexing got much better (but the quality and presentation of
> parsed data still beats the indexed data) and more people want to do
> the log management right (there is also this compliance thing, but I
> digress..)
>
> Anybody care to restart the discussion and see what the collective
> wisdom of loganalysis can produce?
>
> As a semi-humorous warning, please don't suggest the following - we've
> seen these before:
>
> - wait until all logs are in a common XML schema (we know how this one
> ends: MJR emerges out of the darkest part of the woods and kicks
> everybody's ass :-))
> - use our award-losing UI to "easily" create the regexes
> - be happy with keyword searching
> - just write the darn regexes
> (also see http://lists.jammed.com/loganalysis/2005/12/0025.html)
>
> Ready, set, GO!!!
>
> Best,
> -- 
> Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA
>       http://www.chuvakin.org
>   http://chuvakin.blogspot.com
>     http://www.info-secure.org
> _______________________________________________
> LogAnalysis mailing list
> LogAnalysis at loganalysis.org
> http://www.loganalysis.org/mailman/listinfo/loganalysis
>