[logs] Syslog and Windows

[Eric Fitzgerald]

FYI:

Windows event log events may be up to 64k in length.  In the security
event log events are constrained to 55k in length for various reasons
(and if I caught someone calling my APIs with more than 1K or so of data
I would go have a talk with them, it IS a log).

In my experience with the Windows security log, the average length of
raw events in that log is ~500 bytes unformatted (depending on audit
policy avg length might be as high as 700 bytes unformatted).

By unformatted, I mean that we are storing timestamp as a 64-bit integer
FileTime quantity, event ID as a 16-bit integer, success/failure in our
16 bits of flags, category as an integer, etc.  These all have to be
text formatted for syslog (min 2:1-3:1 expansion of event header) and
for our own Event Viewer application.

In Windows events, it's common to embed invariants rather than strings-
for instance instead of storing "Account Enabled" we store "%%2048";
which Event Viewer looks up as "Account Enabled" in the locale of the
viewer.  Likewise we store security IDs and AD object GUIDs rather than
the actual names of the objects; the names have to be looked up before
presenting to the user; in SEM this is typically done at the agent prior
to transmission to the SEM server.

Finally, in Windows, we like to present formatted event messages to
users, but we only store raw data in the logs.  We also like to localize
the messages, so we look them up at view time.  Again, in SEM this is
typically done at the agent prior to transmission to the SEM server;
there's no guarantee that the message will be available at the server.

If you have a solution which does all these lookups and translations and
combines the event message text with the raw event data prior to
transmission, the average event length will likely increase up to 4x, to
the vicinity of 2k-3k per event record in the security event log.

Syslog only supports 1k per message per RFC 3164.

Any syslog-based solution for gathering Windows logs is likely either
truncating a large percentage of Windows events, or not collecting
Windows events in a way that they can be analyzed by human beings (in
that case don't blame Windows; blame your SEM).

In summary, syslog is probably a poor solution for Windows security
events for the reasons described above.  Other logs on Windows typically
have shorter events but you might still have many of the same
shortcomings with syslog.


-----Original Message-----
From: loganalysis-bounces at loganalysis.org
[mailto:loganalysis-bounces at loganalysis.org] On Behalf Of Bill Scherr IV
Sent: Thursday, June 21, 2007 9:35 PM
To: loganalysis
Subject: [logs] Syslog and Windows

All...

   What do you suggest for sending windows logs to syslog

B.

On 18 Jun 2007, a message purporting to be from Chris Brenton appeared:

Subject:        	Re: [logs] Facility 101 (was: Syslog and
facilities)
From:           	Chris Brenton <cbrenton at chrisbrenton.org>
To:             	loganalysis <loganalysis at loganalysis.org>
Date sent:      	Mon, 18 Jun 2007 09:04:41 -0400

> The other problem is some of the facilities are a bit dated. For
example
> there is a facility for FTP (11) but not HTTP. UUCP even has its own
> facility (8) but of course no one uses it anymore (I use it for my
Windows
> stuff. Keeps it from getting mixed in with other log entries ;-)
> 

Bill Scherr IV, GSEC, GCIA
Principal Security Engineer
EWA Information and Infrastructure Technologies
bscherr at iit-tek.com
bscherr at ewa.com
703-478-7608

_______________________________________________
LogAnalysis mailing list
LogAnalysis at loganalysis.org
http://www.loganalysis.org/mailman/listinfo/loganalysis