Application Specific Log Parsing

  • Firewalls
    • Firewall Logging -- A generic introduction to logging firewall devices, with specifics on ipchains and FireWall-1, compiled by tbird
    • cislog [.tar.gz]: A rudimentary tool for reporting on Cisco-based syslog data, created as part of the COSI project. For more information contact John Kristoff.
    • FireGen for PIX: analyzes syslog output from Cisco PIX firewalls
    • FireGen for Symantec Enterprise Firewall: analyzes syslog output from the Symantec Enterprise Firewall (formerly Raptor)
    • firewall1.6: a script that configures and manages IPtables firewalls. Includes a variety of logging options, and enables detection of some port scans and probes based on the log data.
    • A simple perl script that does "artificial ignorance"
    • Fire-Waller 1.2: Compares syslog firewall data to packet filter configurations and produces an HTML document showing what connections were allowed and denied according to rule.
    • FW-1-loggrabber: a log export client written for the Checkpoint FireWall-1 Log Export API, for free, written by Torstein Fellhauer
    • fwanalog: Balazs Barany's shell script to parse and summarize firewall logfiles. Currently (version 0.6.4pre3) understands logs from ipf (tested with OpenBSD 2.8's and 2.9's ipf, also FreeBSD, NetBSD and Solaris 8 with ipf), OpenBSD 3.x pf, Linux 2.2 ipchains, Linux 2.4 iptables, some ZyXEL/NetGear routers and (experimentally) Cisco PIX, Watchguard Firebox and Firewall-One firewalls
    • fwlogsum: A Perl script that produces digests of Checkpoint FireWall-1 logs. It parses other logfiles via converters.
    • fwlogwatch: An open source firewall log analyzer and realtime attack detection and response tool that can parse several types of log formats and output text and html summaries.
    • icewatch: A small efficient program that monitors a given file (usually the log file produced by the NetworkICE PC firewall product) for changes in size. NetworkICE monitors common probe attempts coming in from the Internet and creates a log file with details of the attempted access. Icewatch monitors changes in the log file size and plays an alarm sound when changes are detected.
    • Ken McKinlay's FW-1 Parsing Tools
    • loggrep: a tool for scanning and manipulating packet log entries from ipchains and iptables.
    • Pix2ss.pl is a Perl script that reads in PIX syslog denied connection logs and parses them into a file that can be read and processed by SnortSnarf. It's available at ActiveWorx and is supported by jdell@seisint.com.
    • pixlog: a tool for summarizing PIX firewall traffic and keeping track of failed logins and attempts to access the PIX enable function.
    • pix-summarize -- Perl-based Cisco PIX log summarizer.
    • Pixie - open source Cisco Pix log analysis tool (uses MySQL and PHP)
    • ReportGen Log Reporter: frequently used in conjunction with Kiwi's syslog Daemon, this product parses and summarizes log data from SonicWall, GNATBox and NetScreen firewalls. Contact Ralph Murray for more information.
    • ScanAlert: A utility for analyzing iptables logs and alerting system administrators when port scans are detected
    • CheckScan 0.2: Processes syslog data created by ipchains. Includes rudimentary port scan detection based on multiple denied network connections.
    • wflogs: a firewall log analyzer that can parse netfilter, ipchains, ipfilter, cisco, or snort log formats. It can output text, html or XML summaries, or monitor logs in realtime. It's particularly fast when asynchronous DNS resolution is enabled.

     

  • ftpd
    • flog: An ftpd log analysis tool. Produces usage statistics rather than looking for anomalies.

     

  • multilog
    • multilog-watch: log parser and alerter tuned to qmail and multilog formats

    •  

  • named
    • named-report v1.4 - BIND 9 log summary and report tool. It is written in Perl and is essentially a bunch of regex's comparing log messages fed to it and and those known by the program in order to create the various "Top 10 lists" in the report output. Contact John Kristoff for more information.

     

  • postfix
    • pflogsumm: provides an overview of postfix activity, with just enough detail to give the administrator a "heads up" for potential trouble spots.

     

  • sendmail
    • anteater: A sendmail parsing tool written in C++. It's very fast, allows users to write their own parsing modules, and would be appropriate for handling data from large mail hubs.
    • bms: generates statistics from sendmail logs of rejected mail.
    • fromto-1.5.pl: Yet another Perl-based sendmail statistics summarizer.
    • mreport: A sendmail parsing tool suitable for small mail servers; it stores all log entries in memory for its processing.
    • sm.logger.pl : A Perl script that produces different details from a sendmail log, useful for determining the amount of mail usage per user.
    • smtpstats: Brian Beecher's shell script that produces SMTP summaries based on sendmail logs.

     

  • system logs
    • Analog: Shows the usage patterns on a Web server.
    • mod_log_forensic: Ben Laurie's improvement to Apache logging. The module writes each request (including headers) to a log file before request processing begins, including a unique request ID. After request processing is completed, the unique ID is again logged to the log file. If a security issue is exploited on a server running mod_log_forensic, crashing a child process, the log can then be used to discover exactly what request was used in the exploit, allowing further investigation.
    • Nimda detection script reads Apache logs, and produces a count by hour of probes for Code Red and Nimda.
    • Webalizer The Webalizer is a fast, free Web server logfile analysis program. It produces highly detailed, easily configurable usage reports in HTML format for viewing with a standard Web browser. Contact Brad Barrett.

     

  • windows

NEW!Learn
Log
Analysis!

(Bonus: Now You can download Tbird's SANS Webcast Slides)