Log Pseudo Indexer (logpi)
Logpi is a very small and fast indexing utility that creates indexes
for a subset of data including IPv4/v6 and MAC addresses in arbitrary
text data. The pseudo indexes can be used to reference and recall
log data quickly. This is useful for security practitioners that
need to search through text log data more than once.
Wirespy Daemon (wsd)
Wirespy is a lightweight packet sniffer and metadata logger. It supports
live network monitoring and off-line PCAP file processing including
TCP flow reassembly. This tool can give you firewall log like fidelity
on any point on your network where you choose to tap. I use it to
extend the duration of network visibility on my network recorders.
Directory Tree Differ (difftree)
dt is short for difftree and it is a fast directory comparison tool.
It is a command-line utility to compare two or more
directories. It detects changes in file metadata including
size, ownership, permissions, etc. This is very handy during a security incident when time is of the essence.
Converts logs to templates for anomaly detection (tmpltr)
Templater is a small and fast log processor that provides simple artificial
ignorance capabilities. You use the tool to process past log data and store
templates that represent normal log line structures. You then run the tool
against current or target logs and all normal patterns are automatically ignored.
I got tired of getting ssh bruteforced without any idea of what the bad guys
were trying to do. So I lifted Pete Morris's sshpot code
and enhanced to meet my needs. A couple of interesting features that
sshcaray has include the ability to extract additional interesting info
from the ssh connection and allows you to tell the client that the
authentication was successful (just to see what happens next).
PDF Forensic Parser (pdfcarve)
|A buddy of mine came to me with a problem one day. He had a PDF that he
knew was malicious, but had no way of reviewing. When he
attempted to open it in a VM, the viewer just crashed without any
errors. On a victim machine, the Adobe Reader would execute the
malware and the system would get compromised. I spend a bit a
time searching the Internet looking for a tool that could parse the
file and found nothing that would handle to clearly malformed and
dangerous file. So, I downloaded the PDF specification and opened
the file in a hex editor and started carving it up. After a few
He was elated when I sent him what I had found and promptly sent
me ten more malicious PDF files. Being the lazy UNIX
administrator that I am, I codified my manual process in a tool called
pdfcarve. It makes as few assumptions about the format of the PDF
as possible and verbosely displays all of the objects.
Passive Proxy Daemon:
Reads web traffic and generates squid proxy logs (pproxyd)
This tool reads pcap format files or reads packets directly from the network, assembles web based traffic and
generates squid proxy style logs. Logs are sent to standard out while in interactive mode and via syslog when
running as a daemon. The log format is similar to native squid v1.1/2.x format.
Converts Netscreen/Juniper firewall logs from syslog to psuedo-XML (quickparser)
A co-conspirator and I
have been building some log analysis tools. To feed data to the tools,
we needed to convert log files into a psuedo XML format. I built a
couple of regex log parsers in perl but they were very slow. We have
spent some time talking about other ways of parsing logs and I canabalized a log
compression tool that I had built using regexless parsing and found out
that it is pretty fast. By fast I am talking 2.6M lines per
minute on a single processor 2.5GHz PC running linux and 950K lines per minute on
a SunBlade 2000 with 1GHz processors.
BaraCUDA: Multiple GPU password cracker
|Also known as Nightingale,
this CUDA based password cracker supports mutliple GPUs and currently
supports NTLM, MD4, MD5 and WINCRYPT hashes. Crypt(3) MD5 and RAR
v3 are in the works.
Log Store: Fast and efficient text log storage tool (logstore)
|One of the offshoots of my regexless log parser (quickparser)
is a tool that provides high speed compression that is I/O vs. CPU
bound. It also allows for fast keyword searching due to the
parsing method used. It is much faster than bzip2 and faster than gzip
while providing considerably better compression ratios. The down
side is that it is tailored for compressing text based log data.
I use it on logs that are generated by syslog compatible software.
DNS Spy: The passive DNS monitor (pdnsd)
|In my never ending quest to find a better way to detect bad programs
and people, I am working with a small group of guys to build a system
that monitors DNS traffic for anomalous and known to be evil patterns.
As long as the bad guys use the network and stay away from static IP
addresses, they must leave tracks in DNS.
The Wiretappers Toolkit (twt)
|A friend of mine and I are working on a web based tool that records all
network traffic and allows an investigator to slice out network traffic
based on search criteria like source/destination IP address, e-mail
address and regex string searches. It is very simple, but
gets the job done.
Response & Assesment Tool (PiRAT)
|I just upgraded from a Nokia
N800 to a N810
and it is time to rebuild my tool chain again. I was very
happy with the way that PIRAT v0.1 turned out, I will be opening up PiRAT
v0.2 for the N810 in the near future. PiRAT still offers a
port of all of my InfoSec related tools to this
ultra-small computing platform. There is a pretty well put
together development environment and a strong open source development
community branded as Maemo.
|I wrongly assumed that it would
be simple to find an intellectual property (IP) leakage monitoring tool
for removable media on OS/X. It is unfortunate that the
abundance of this type of tool on Windows has not rubbed off on Mac.
That said, I hacked together a volume monitoring daemon (vmd)
to provide logging of file activity on removable devices on systems
running OS/X. It is the poor mans solution to the problem.
It uses syslog, so it is possible to have all of your Mac's
point to a central syslog server and gain arms length auditing of removable
media across all of your OS/X based systems with this little daemon.
A word of caution, I used the fsevents functionality of
10.4 and 10.5 which is not a standard hook for applications.
So don't be surprised if this stops working in a future
The following is a quick install guide for non-*nix folks.
configuration will have the removable media activity logged to
/var/log/secure.log on the local system. It is a simple thing
change the configuration to point to a centralized logging server and
have all systems log to it.
To install (as root):
1. Get the tools
Download the vmd binary for
10.5 (MD5 = ddc147333246d13c08ffa4c2bc9de1ca)
Download the vmd.sh startup script
Download the StartupParameters.plist file
2. Update syslog
NOTE: The <tab>s (\t) are important.
echo "daemon.*\t\t\t/var/log/secure.log" >>
kill -HUP `cat /var/run/syslog.pid`
3. Copy vmd
mkdir -p /usr/local/bin
cp vmd /usr/local/bin
chmod 755 /usr/local/bin/vmd
4. Setup vmd to start automatically
mkdir -p /usr/local/bin
cp StartupParameters.plist /Library/StartupItems/vmd
chmod 644 /Library/StartupItems/vmd/StartupParameters.plist
cp vmd.sh /Library/StartupItems/vmd/vmd
chmod 755 /Library/StartupItems/vmd/vmd
Passive Syslog Monitoring Daemon (psmd)
|This tool allows you to monitor syslog traffic on a segment of the
network without having any control over the source or the syslog server
that receives the data. This is very hand in a DMZ
You can use psmd as a syslog replacement allowing all of your
clients to forward their messages to a non-existent IP address in the
DMZ. You can black-hole the syslog data and still record it
all without exposing where your syslog server is or punching a hole in your
firewalls. Lastly, you can use psmd to relay traffic.
This works really well when trying to split in a commercial log analysis
appliance that expects all devices to forward traffic directly to the
appliance. You can keep your syslog infrastructure just the
way it is, add psmd and forward all sniffed logs to the appliance.
Psmd will even rewrite the source addresses of the traffic to
make the log analysis appliance happy.
The DARPA Grand Challenge
I was lucky enough to get the opportunity to join SciAutonics for
the 2005 DARPA
Grand Challenge. Our
was approved and we were off to the races (after a couple of cullings).
It was quite an adventure and provided me an opportunity to
with some very smart people. I worked on the computer
software architecture and the lidar drivers. We received a bit of press
while we worked on the 2005 challenge including a piece in one of the local
2007 Urban Grand Challenge is coming up quick, wish us all
luck as we try once again to get into the race. We have
managed it twice before!
And for the third time, we have made it into the semi-finals!
Wire Spy Daemon (wsd)
This tool is one of my current "under construction" projects.
tool allows you to monitor a network segment over long periods of time
(like 90 days+) and while it is running, it builds ACLs for the traffic
that is seen. This is a great way to impliment ACLs on a wire
where you don't know what is passing across it and you don't want to
take the chance of breaking some important traffic while you move from
ANY->ANY Allow to something less.
SMB Crawler (smbcd)
Ever want to know what files were accessible across your entire network
but had no idea how to find them? Try using a search engine
only to find out that there was not enough disk space on your box to store
all the search date? I had that exact problem, so I build a
crawler that stores limited meta data about files while searching huge
amounts of data. All searches happen as the data is crawled
removing the need to large amounts of disk space to store search data.
TCP Stream Carver (tcpcarve)
While bolting together The Wiretappers
Toolkit it became obvious that tcpflow
was not cutting it. TWT
needed a sniffer/TCP flow reassembler that was easy to graft into the TWT framework. That said, I
started to convert wsd into a
TCP flow assembler and archiver.