I first heard about “fuzzing” about 15 years ago when it started circulating around the security community, and basically blew it off at the time. I said to myself something like “Eh, that’s just throwing random data at a program to see if it crashes. That may be useful, but isn’t really interesting.”, and proceeded to not pay much attention to it.
As a hardware validator that’s what I did – I generated tests with constrained random outputs to drive the hardware inputs and then watched to see if it did something unexpected. Fuzzing seemed pretty basic, and I figured it was one of those security fads that was hyped up with a cool-sounding buzzword that would vanish back into the ether. Some friends even created a fuzzing tool, and I was too blase` (and to be fair, distracted with a full-time job and kids) to pay a lot of attention to it at the time.
In my recent research efforts I attended a number of fuzzing-related presentations and started actually paying attention, and am now here to eat crow. My initial assessment wasn’t wrong exactly, “fuzzing” really is just a slang term for constrained-random testing – throwing random data at a program to see if it crashes. I pretty much entirely missed the point though, and the fact that all the stuff around that process is super interesting.
On the input side there are all sorts of algorithms for either generating new random data or taking example data and morphing it in random ways. On the output side you not only detect that your target program has crashed, but there are all sort of cool ways to get more information on how it crashed. You can then feed that output knowledge back into your input algorithm to get more targeted test data and find more crashes faster. Researchers are currently busy figuring out how to tie in machine learning algorithms to improve pattern matching. The problem space is so insanely huge that the art is in trimming it down – figuring out how to craft targeted input data to minimize your effort.
Then there’s all the cool stuff around creating harnesses to run your target software in the first place. You may be fuzzing a software library, so you may have to create a program around it to feed in data. Or you may be fuzzing firmware for an embedded system, so you have to either instrument a physical platform or emulate the whole thing in software. If you go the emulation route you may want to speed things up by replacing hardware function calls with faster software which is good, but creates a less realistic model. There’s an art to getting software from whatever program or device and instrumenting it so you can run fuzzing tools.
There’s the code coverage tools – disassembling your program and watching how it runs to see if you’ve exercised all the code. Particularly because you may not have access to the original source code. If you only execute half of the program you’re potentially missing a lot of bugs, so analyzing inputs that cause crashes on one code path can help find inputs that exercise other code paths. There’s an art to figuring out how to exercise the whole program.
Once your target program crashes, indicating a potential hole, how do you determine if that is a real vulnerability and not a problem with your software? If it is a real vulnerability how do you craft an exploit that you can use? There’s an art to going from crash to exploit.
What if you’re not fuzzing a program, but a web page or service or endpoint remotely? There’s a whole ‘nother set of arts and tools around that.
Each one of these things is an entire discipline in and of itself, and researchers are busy creating new and cool tools to do all of them in faster and better ways. I’m just scratching the surface here, but it is obvious that fuzzing works, and will continue to improve – getting more and more use and better and better tools. Companies are setting up ci/cd flows where they run fuzzers against their software continually in huge server farms, trying to catch bugs in each new version before release. Researchers are busy working their way through all the open-source software that exists and finding vulnerabilities that have been undetected for years. The security race never stops, and fuzzing is driving a huge part of it now.
I’m definitely arriving late to the party – it’s simply amazing what everyone has been busy doing in this space. Coming from a hardware validation perspective though the core random-constrained testing methodology is the same, so it’s like someone took my car, mounted a jet engine on the back, and pushed the button. It’s cool looking ahead and trying to figure out where the rocket is going while I run my fastest to catch up.
As I continue to learn this I’ll do my best to create digestible summaries and tutorials, so stay tuned!
zerosquared.io 