[logs] regexless parsing, again?

[Mike Heisler]

Mordechai T. Abzug wrote:
> 
> [Cue a bunch of starry-eyed innocents talking up CEE, CEF, and other
> "standards" that are supposed to make unstructured logs go away Real
> Soon Now.]

Which is to say, as others have, we need solutions yesterday and 
official standards aren't going to solve the problem.

Log files aren't generally written for us, they are written for the 
programmer or some auditor, individually, in a vacuum for each 
application. If I were to write an application today and wanted to log 
transactions of some sort why would I think to look for standards? Where 
would I start to look? Who cares about my log file any way?

I'd like to suggest we can do something proactive

- Share some of the good methodologies that have already been developed 
for parsing, categorizing or otherwise making sense of these files

- Share log formats we are aware of (Splunk base is an example)

- Come up with 2 or 3 or (N < 10) log file format recommendations. Maybe 
those in standards process.

- Collect this information in 2 or 3 places.

- Most importantly, get the info and recommendations into the faces of 
developers everywhere we can.

Programmers coming out of school and those who've never seen a syslog 
have no clue that they there is any standard that could or should be 
followed. They roll their own, just like those needing to read the logs.

-- 
Mike Heisler        607-255-3058    cell: 607-227-6791
Systems & Operations,     Cornell Information Technologies
mgh4 at cornell.edu    703 Rhodes Hall    Ithaca, NY 14853