DirBuster -> Burp, the Missing Link

Everyone loves Burp Suite. It’s the interception proxy of choice for the majority of web app pentesters, and with built-in tools such as Intruder and Repeater, it’s a must-have weapon in any web app hacker’s arsenal. One of the only weak points in Burp is its inability to thoroughly brute force unlinked web content. Don’t get me wrong, Burp Intruder can brute force unknown directories and files, but it doesn’t have the ability to recursively brute force and scrape newly discovered content at runtime.
DirBuster, a popular OWASP project, is a long standing and suitable replacement for this lack of functionality in Burp. DirBuster was designed solely for the discovery of unlinked web content and has the ability to recursively brute force directories and file names at runtime and crawl newly found links as they are discovered. It is threaded, which allows for speed/reliability tuning, and written in Java, so it is pretty much platform independent. It comes prepackaged with some decent word lists, but they haven’t been updated for about 3 years. However, there are plenty of resources for quality word lists, so this is no big deal. The newly released RAFT project has a few lists which I highly recommend. The downside to DirBuster is a less than user friendly scan report, which is not conducive to importing results into other tools.
Seeing as DirBuster is my brute forcer of choice, and Burp is my interception proxy of choice, bridging the gap between these 2 tools and getting the output from DirBuster into Burp for further analysis is crucial. As you can see below, one bash command, about 140 characters long, does the trick. It takes the report file from DirBuster and plays it back against your interception proxy. In my case, Burp.
cat report.txt | grep '^/' | grep -v ':' | while read line; do curl -s http://[target of scan]$line --proxy 127.0.0.1:8080 -o /dev/null; done
Here’s a breakdown of the command:
1. pass the report file to stdout
2. grep out all of the directory, file, and internal error results
3. un-grep all of the internal errors
4. loop through all of the results
5. use curl to craft web requests to each iteration of the results
6. configure curl to use a proxy
7. dump the curl output to /dev/null to suppress stdout (optional)
All of your DirBuster results are now available for analysis in your interception proxy and tools like Burp will have passively scanned and spidered the results in the process. There’s nothing like some command line kung fu goodness to solve a common problem with such simplicity and elegance. Enjoy!
Look for my SANS 560 Mentor sessions coming to Northwestern Ohio beginning November 2011! In the mean time, join Mark Baggett for SANS 560 Network Penetration Testing and Ethical Hacking vLIve! Class begins September 12, 2011. For a limited time attendees will receive an IPAD2! Register today for a FREE IPAD2!!