[logs] Re: Syslog and Windows

[Rainer Gerhards]

Hi Gord,
 
> Windows CLIENT agents (e.g. the ones that convert from .EVT 
> to syslog format) are also a problem here since the only one 
> that allows for multiple syslog destinations, is the old 

I am not sure if I understood you correctly, but MonitorWare (and
EventReporter that is based on it) allows as many destinations as you
like. The event is passed to the rule engine just as a syslog message is
- so the full set of rule engine processing is available. Most customers
use two destinations, but you can add many more.

> I'd add to this that most of the eventlog to syslog agents 
> have a periodic problem since the eventlog field data is 
> store separatly from the message in Windows. When you look at 
> the eventlog and see "Logon Successful: taylorgo", the "Logon 
> Successful:" part is stored in a language file (to allow the 
> eventlogs to be read in any language) while the "taylorgo" 
> (along with eventid, date/time, etc) are in the .EVT file. 
> The problem happens when the eventlog language file is locked 
> by another process, the syslog agents only collect the 
> field-level data. 
>  
> The result ends up being that you'll see the same event id 
> represented two different ways in syslog: one with the 
> full-text formatting, and another with just the field level 
> data. Makes parsing infinitely more complex (you really have 
> to expect to parse each event twice). Lasso (and I think 
> MonitorWare) get around this by keeping a locally cached copy 
> of the language files and refreshing them every X minutes 
> (configurable). It's really the only way to do it reliably. 

It's not just a question of caching - it's also a question of how you
open the ressouce files. An application should not lock them exlusively.
But, yes, we do caching. And yes, x32/x64 is also a very interesting
issue that breaks many tools.

Another caveat is that "taylorgo" is not stored in that format. It is
stored as a SID or GUID. The eventlog-to-syslog collector needs to
resolve that identifier in order to get something meaningful. Hint:
tools vary greatlyin the way how (if) they handle that ;)
  
> Vista/Longhorn are supposed to be better because they store 
> everything in XML format, so at least shouldn't be anymore 
> messing around with the language files.

Vista does not have this problem, at least hopefully. The OS itself
formats the complete message here - I guess it has some ways to work
around. As I wrote, we have implemented a special monitor for Vista logs
(proud dad says: "again, world's first - as far as I know" ;)). So I
really know what I am talking about. It just needs to be seen how all of
this works under heavy load, probably to be explored when Server 2008 is
out.

Rainer