Please donate if you find the tools, scripts and info useful.BTC: 1GwYToq2AuUWUfJJ7NeCpksfjMth7bw7Tu
2013-12-31 Log Templater (tmpltr) v0.6 is now available
I have been laying some ground work for implimenting log chain templates in the next version of Log Templater.
Part of that work includes adding support for more field types (IPv4, IPv6, MAC Address and HEX numbers).
These changes clean up the templates and make it simpler and faster to find some key data when responding to
The webcast is available and who knows what may come next.
The webcast has been posted along with some takeaways.
Special thanks to Dragos for the use of the secure WiFi gear.
In addition, I have added a greedy mode where the templater will ignore double quotes. This is very useful when processing web server logs. In non-greedy mode, tmpltr will assume most things in between a pair of double quotes are a single string. In greedy mode, all data will be handled the same.
2011-10-01 Log Templater (tmpltr) v0.3 is now available
During the last security incident that I worked on, I needed to grind through 20gb of log files looking for any odd log lines that would indicate the point where the bad guys got in. If I had done it manually, I would still be looking at log data. Instead, I built a tool that converted logs into pattern templates and looked for templates that I had never seen before. This allowed me to zero in on just a few hundred log lines out of all the data.
2011-07-29 Quick Parser (quickparser) v0.5 is now available
After re-implementing some parser code in difftree with bug fixes and improved bounds checking, I have ported those changes back to this tool. So, I now present the new and improved quick parser with less bugs and more bounds checking. Re-energized after touching the code, I am working on the template based version, which will not be limited to syslog style, key=value formatted logs.
2011-07-27 Directory Tree Differ (difftree) v0.5 is now available
I have released a new version of difftree with more features and less memory leaks. This new version adds the ability to save directory tree state and to compare both directories and archives of directory trees that were previously saved. In addition, you can hash and compare files if accuracy is more important the speed.
2011-07-16 Directory Tree Differ (difftree) v0.4 is now available
2011-07-03 SSD Redux
After fighting with the RAID chassis that I picked up to house my new clutch of SSD drives I returned them all in a fit of nerdrage and downgraded from a RAID enclosure to a simple JBOD and larger SSD drives. I have not been disappointed:
Using the very handy HD TunePro utility, I was able to get a bit more imperical data about the difference in performance between my old configuration and the new SSD smoke-fest. Here is the read throughput and access time graph for the 4x120Gb OCZ Vertex II SSD (Raid-0):
That may not look too impressive, but if you compare it to the same read throughput and access time graph for my old 2x1Tb 7200RPM SATA II HDD (Raid-0) configuration you will see a bit of a difference and why my grin is from ear to ear:
2011-05-21 Power supply failure leads to faster boot times
In the spirit of tricking people into reading what I write, I have given in to temptation and deliberately misled you. Yes, my old power supply on my Windows development system died and I was forced to replace it today. In addition, I have improved my boot times considerably. But the only correlation between the two is that I figured I would do a bit of upgrading while I had my case open to replace the power supply. So, out with the old 850W and in with the new 1200W Thermaltake. Additionally, out with the old 2 x 1TB Raid-0 boot drives and in with the new 4 x 64GB Raid-0 SSD. All this "high speed, low drag" boot hotness in a fancy 5 1/4" chassis by Patriot. Boot speed times are as follows:.
2010-11-24 Generate password lists using Markov chains
In preparation for adding additional password plaintext generators to my GPU password cracker, I prototyped a simple Markov chain plaintext generator (pwMarkov.pl). I added it to my John scripts for good measure and am testing to see if there is a measurable improvement in cracking efforts now.
2010-05-18 Say goodbye to my liquid cooling system
After my second coolant leak, it was time to remove my liquid cooling system. It worked really well at keeping my system cool. It was not as effective at keeping it dry. It would not have been as annoying if I did not have an air cooled system running two dual GPU graphics cards and a Tesla super computer sitting under my desk at the same time. My GPU password cracking box just hummed along without any overheating issues while my game system learned how to swim in low conductivity cooling solution. So, I upgraded my case to an Antec 1200 to match my GPU cracker and replaced my pair of ATI 4895's with two ATI 5970's and I am back to the races. The 3DMark numbers speak wonders about both the video cards and the difference between running Windows XP and Windows 7.
2010-05-12 LogReporter v1.5 is available
In response to some feedback from Santiago Zapata, I have updated the LogReporter.c code to enumerate and report on all active MAC and IP addresses. Irrespective of the size of your Windows environment, you probably struggle with mapping actions in your firewalls or IDS logs back to users. Stick this program in your login scripts and it will provide syslog records for each login including NETBIOS computer name, username as well as all active MAC and IP addresses.
2010-03-19 Mining A/V logs continues to extract gold
After updating my simple Symantec A/V log parser I used it to convert syslog Symantec A/V logs to CSV files and loaded the data into Advizor Analyst. This type of graph shows interesting re-infection patterns for individual hosts (horizontal lines), signature updates following malware blooms (vertical patterns with the same colors) as well as others.
The y-axis is a list of all infected machines so each horizontal line is a single system (Hostnames clipped for anonymity). The color coding is based on the name of the malware with similar names having similar colors on the pallet. Analysis of the four solid verticals outside of the weekly schedule scan pattern (the consistent vertical pattern across all the plots) were four different outbreaks of a network scanner where signature updates were loaded across all systems and the IPS started detecting and blocking immediately. The analysis was done to evaluate the effectiveness of an A/V deployment. The initial impact was a clear determination that the A/V system needed some attention and over time, the analysis was used to re-enforce the value of enabling additional options in the A/V system. Ongoing analysis shows the tempo of malware detects and changes to the A/V environment.
2009-10-13 LogReporter v1.4 is available
I needed to add support for unicode in my simple login tracker. While poking around in the source, I added a few more features to make is a bit easier to use. If you have a small, medium or large Windows environment, you probably struggle with mapping actions in your firewalls or IDS logs back to users. Stick this program in your login scripts and it will provide syslog records for each login including NETBIOS computer name, username, MAC address and IP. The new version does a better job with long and unicode netbios names and allows you to specify your loghost by name or IP address.
2009-09-21 A/V logs can help you find troubled systems
I was culling through the logs on one of my systems the other day and realized that I was getting a fair amount of alerts from my Symantec A/V servers. At first, I was not interested in what malware was being detected and cleaned but it got me thinking about what interesting patterns existed. I suspected that the majority of malware infections were caused by a minority of users as most malware these days require some user action. To test this theory I wrote a simple parser to convert the logs to something that I could push into a visualizer and started looking for interesting patterns.