[logs] Analyzing tons of logs

[Chetan Gupta]

Dear List Members,

I am looking for opinion from the experts for a particluar problem.

How do we go about log analysis if we have tons (maybe in trillions) of 
logs from lets say tcpdump (raw logs) or some firewall (like netscreen or 
pix)?
What would be the best way to normalize and analyze these logs in the 
shortest possible time?
Import them into a database? Use a commercial application like arcsight? 
loglogic? simple text editor like editplus?
Any suggestions/comments would be appreciated.

Regards,

Thanks and Regards,
ERNST & YOUNG ®
Ernst & Young Pvt. Ltd

Chetan Gupta
Consultant
Risk and Business Solutions
FIDS 
_______________________________________________________

          
Mobile:      +91 - 9810718489
Fax:          +91 - 11 - 2661 1012           
URL:          http://www.ey.com/in
_______________________________________________________



----------------------------------------------------------
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it.   It may contain confidential or legally privileged information.   If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. Ernst & Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.loganalysis.org/pipermail/loganalysis/attachments/20070328/d9ee208e/attachment.html