[logs] CEE - a new logging standard

[Raffael Marty]

I was quiet for too long... [and sorry for the long post!]

First, here are a couple of more pieces of information about CEE:

http://raffy.ch/blog/2007/04/19/standard-logging-format-common-event-exchange-cee/

Second, let me address some of the points you mention:

> Cost - For the vast majority of vendors (especially the big ones), logging
> is one of the least important aspects of their development process.   There
> are different groups responsible for different components.  If normalizing
> logging internally adds cost and time to development, I don't see vendors
> fixing their existing logging issues and on top of that compiling with a new
> standard.  

This is only partly true. The entire regulatory landscape is changing
things QUITE a bit! You need accountability and more and more
applications HAVE to log and log review has to be in place.

And contrary to what you claim, I am dealing with approximately 20
vendors who are implementing a standard logging format at this very
moment. [http://www.arcsight.com/solutions_cef.htm]

> One could argue that by providing the standard, we give vendors a
> road map for fixing internal logging issues.  True, but vendors must first
> commit to the process which is expensive.  It's like watching Oprah and Dr.
> Phil and having the roadmap to fixing your life.  If it's too expensive, or
> too much effort, you'll never get started. 

Great point. And you know what? The existing standards (almost all of
them) are too expensive to implement. They require you to implement
complicated transports, etc. However, if you provide products with an
easy standard, it is quite compelling to implement it! [I am talking
from experience!]

> Legacy issues - Vendors (especially the big ones) have a lot of logging
> "history" based on nuance & convention.  To change their logging format,
> content, or transport will not only involve new product development efforts,
> but downstream effects to any part of the business that the logs
> touch: support issues, training courses, certifications, books,

I agree.

> vendors/partners, etc.  The only way to introduce a drastic change in
> logging and maintain legacy support is to do both in parallel and allow
> users to choose either logging format (i.e. this is how SNMPv2 and v3
> adoption happened).  

SNMP is not a format. It's a transport! You could argue it's a format
too, but at most a really ugly one ;)

> This means added cost.  There are also legacy issues in
> the open source community that will have resistance to a new logging format
> .  E.g. Unix logs stand to benefit the most from a new standard, but that
> community may be the most likely to resist.  Not to mention the many
> different components with diverse origins and contributors.  

I couldn't agree more. And I am realistic enough to understand that
there cannot be a big "bang". A standard cannot be mandated from day
one. People should start migrating things over. I am the first one to
scream if you change the syslog output as all of my scripts will not
work anymore.

> (Note: I've
> been talking for awhile about an opportunity today for a mid-tier
> application to take existing logs and transform them to some standard, but
> this only addresses what content is available today what is logged today,
> not what should be logged but isn't.)

There are products which do this! It outputs CEF (Common Event Format,
see link above).

> Competitive advantages - Paul Melson accurately wrote in his blog (in
> commenting on CEE) that "It costs SIM vendors a decent chunk of change in
> development and support cost to spin up support for new products.  Standards
> make this proposition cheaper: one set of code to maintain, potentially
> hundreds of products supported."  This is precisely the reason why you WON'T
> see the largest vendors evangelizing any new log standard.  It's a huge
> barrier to entry (in their favor).  

Okay. Here I COMPLETELY disagree. ArcSight is pushing a standard called
CEF and we want as many people as possible to adopt it. It is NOT a
competitive advantage to support more devices. We rather have 500 more
supported products that we can sell solutions, appliances, etc. It's all
about solving customer use-cases and supporting a wide variety of
use-case. 

> I'm not just talking SIM vendors but any
> log management company or managed security services provider who has already
> invested in normalizing and building content around log messages -- they 
> have
> a huge competitive advantage.  

Well, a good standard will make content development scalable. If this is
not compelling enough, I don't know what is.

> Even Anton's company and it's competitors
> would be inviting competition if a new log standard was adopted overnight.

Why? The SIMs and Log aggregators all have unique capabilities that they
market. You build better platforms, you are a player in the field. But
believe me, it's not _that_ easy.

> Imagine how easy it would be to build a new log-centric product or service 
> if
> you didn't have to spend as much time building support for a hundred vendor
> devices (e.g. an MSSP that support a million log events).

MSSPs are more and more using COTS products and not going with their own
solutions anymore. It's too expensive to maintain a product like this
(i.e., a ESM tool). There is a reason why some of the SIM vendors employ
over 200 people. These things don't come to live over night.

> What is the incentive for
> existing vendors (especially the big ones) to change their logging format?

Interoperability. Supporting existing log collection frameworks, SIMs,
enabling solutions (e.g., a SOX solution from a SIM vendor). You are
suddenly helping solve the SOX problem without doing much.

> There is no incentive or mandate for existing device vendors to change their
> logging format.  

Absolutely there is! And it's not just the syntax or transport. We are
also talking recommendations (see my blog post)!

> IMHO, the only way you'll see the big device vendors jumping on a new log
> standard is if you see the big log derivative players drawing a line in the
> sand.  But that's like saying the vultures can tell the lions what to do.
> The log derivative industry exists because they live off the logs produced
> by the device industry.  Will there ever be a log derivative player large
> enough to push around the device players?  Perhaps HP Openview, but they
> will be pushing a narrow framework that supports their own best interests.
> Besides, Openview is the king of modules and plugins - a huge competitive
> advantage to deal with a heterogeneous logging universe.

You don't call OpenView a log correlation tool, do you? They are a
network management tool. SNMP is about as far as they really support log
feeds. I consider the ESM products of the world and I know for a fact
that they are already supporting these types of efforts!

> Another issue regarding the log derivate industry - by the definition of
> what they do, and in order to maintain their competitive advantage, these
> vendors will always be supporting existing & legacy formats.  The reality is
> by adding a new standard you require these vendors to do more work, not
> less, as they have to support both until the existing logging formats are
> truly EOL'ed.

Sure. You are saying it yourself. There will never be a world where
everyone will support the standard logging format, so there will ALWAYS
be space for the log management / correlation / SIM /ESM tools to build
connectors and agents and have better support for some feeds than their
competition. The game will change though! Instead of supporting 300
products, you can scale much better and support 3000 products, because
2700 of them are using a standard format!

> Considering that the new logging standard is still in-process, and the
> issues mentioned above, I just don't see adoption for this happening any
> time soon.  In fact, I speculate that you are are more likely to see IPV6
> widely deployed before a new log standard such as CEE.

I hope so, but CEE is very promising. Again, I speak from experience and
I have seen that we already benefit a lot from device vendors supporting
a common logging format. 

> Having said that, you have to start somewhere.  So I welcome the effort, but
> am also a realist. :)

Me too, an optimistic one ;)

  -raffy

-- 

Raffael Marty, GCIA, CISSP                    raffael.marty at arcsight.com
Manager                                  Strategic Application Solutions
ArcSight, Inc.                                         +1 (408) 864 2662
Security Data Visualization:                           http://secviz.org