[logs] History of Log Analysis and Modern Search Engine

[Tom Le]

On 7/6/07, Anton Chuvakin <anton at chuvakin.org> wrote:
> Since I am feeling pretty warlike now :-) I would like to explore this
> further: why do you think that there is a link between log analysis
> and search engines (apart from whatever historical one)? Searching
> logs, while necessary sometimes, is certainly not a fun thing and,
> just as certainly, not the most effective way to make sense of logs.

> Maybe, just maybe, search the web is the best way to make sense of the
web.
> However, I can't say the same about logs.

Depends what you mean by "searching".  At some point in the cycle you have
to do some kind of a "search" to product useful information from log data -
whether you are just browsing, grep'ing, searching indexed, sorted,
meta-data tagged or otherwise sliced data.

> Just today I was helping investigate this fun incident where logs from
> a compromised server  was the only evidence available. Figuring out
> "what to search for" is pretty much a non-starter...
I disagree.   There are many security professionals that do nothing but
forensic and incident analysis and the very first step in the analysis is
knowing what/where/how to search.  Obviously how useful and reliable the log
data you have to work with depends on a lot of factors.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.loganalysis.org/pipermail/loganalysis/attachments/20070706/5d3041d3/attachment.html