[logs] open source artificial ignorance-like systems

[Marcus J. Ranum]

I take it as a matter of doctrine, now, that it's not possible to have a
"one size fits all" (i.e.: productizable) log analysis system in the
first place. Every damn one that has been useful has had to
have extensive tailoring to collect appropriate information, ignore
inappropriate information, etc - all on a case-by-case basis
that is specific to the customer. Because no two customers'
networks look exactly alike or are used in exactly the same
way.

Every time I've done a logging project, I have found that sitting
down and looking at the logs - seeing what's there and
what's interesting, and THINKING about them - has paid off
vastly more than any alternative. The approach of buying a
"one size fits all" solution only makes sense if you're too
damn stupid to understand your logs - in which case you're
also too stupid to understand the output of a SIM (other than
"ooooo pretty pie chart! colors!")   If you're not too stupid
to understand your logs, then a couple days reading them
and writing triage-flows in your favorite language is vastly
superior except no pie charts. Unless you want to invest a
day learning GNUplot or rrdtool and then you can out-chart the
"one size fits all" solutions, too.

An earlier poster expressed the sentiment (paraphrased) that
lots of UNIXy tools are bad because as you integrate them into
your pipeline it's tricky  And I've heard a lot of customers
say stuff like "we can't do any development or customization
so building our own analysis routines is too much work and
hard to maintain."  Oh, really?? What about writing all the
"correlation" rules for a commercial SIM isn't "work"? What
about some unwieldy CPU-hungry monster kludge SIM
product isn't "hard to maintain"? etc. Unless your environment
is absolute plain vanilla you're going to spend more time
writing rules for a $200,000 SIM deployment than you'd
spend writing a couple dozen pages of code in whatever
language you code in, and wrapping it with a cgi-bin
script that calls gnuplot, that's for sure.

But, hey, I know I'm a radical "do it yourselfer" and that's
just not how the industry is going. Security is now becoming
the "letting people appear to be doing something with data
that they appear to understand, without their having to actually
learn, understand, or know anything."

mjr.