[logs] regexless parsing, again?

[Andrew Hay]

Hey Ashish,

This might be a bit off-topic but I thought I'd respond to your post.

Although offloading this work to another card provides an immediate
short-term benefit, the underlying issue remains that your Regex's
might not be efficient enough to handle larger loads down the road.
Reducing the complexity of your regular expressions may make them
faster so a regular review of what you've written (or even a second
set of eyes) might alleviate the need for an offloading mechanism
altogether...depending on core hardware of course ;)

On 18/09/2007, Desai, Ashish <Ashish.Desai at fmr.com> wrote:
>
>
> There have been some amazing advances in hardware to do PCRE.
> Once you max out a regular CPU, you can consider to offload this to a card.
> Check out Tarari
> http://www.tarari.com/PDF/Tarari-T9000-CP_PB.pdf
> They have a API
> 1.Allows you to load up the chip with all the regexs you desire.
> 2. Then blast the content you want to want to test and out comes a list of
> all the matches
> The speeds are pretty incredible (at least on paper),  that even writting
> the the crappiest regexs you
> would have a hard time hitting the system maximum.
>
> Ashish
>
>
>
>  ________________________________
>  From: loganalysis-bounces at loganalysis.org
> [mailto:loganalysis-bounces at loganalysis.org] On Behalf Of
> Tom Le
>
> Heh.  Just making sure we didn't trivialize the fact that one can still
> maintain the more traditional ways of building regex rules and still achieve
> significant performance gains.  Scale should be mentioned here.  If you go
> from parsing 1000 msgs/sec => 10,000 msgs/sec that might be great for you,
> but insignificant for others.  YMMV.
>
> More like: "Marcus, you should separate discussion of regexes vs. other
> parsing approaches into separate categories: performance, initial ruleset
> development cost, and on-going maintenance."
>
> Each discussion has it's pros and cons with different cost(x) *
> complexity(y) functions depending on the what you're doing and size of your
> rulesets.  I was just trying to explore a deeper level of discussion than
> the usual 'regexes suck' or 'PCRE performance sucks' or 'maintaining 100,000
> rules is ugly' type discussions.
>
> Note, however, that I will reserve the right to use parts of your above
> quote in the future. :)
>
> _______________________________________________
> LogAnalysis mailing list
> LogAnalysis at loganalysis.org
> http://www.loganalysis.org/mailman/listinfo/loganalysis
>


-- 
Andrew Hay
blog: https://www.andrewhay.ca
email: andrewsmhay || at || gmail.com