Fuzzing all crates

Today Manishearth blogged about mitigating underhandedness by fuzzing your code. Underhanded code contains a backdoor hidden by the syntax or semantics of the programming language and should not even be visible to the cautious reviewer. Fuzzing tests a program for random (or not so random) inputs and reports found bugs like memory access errors or panics. Simple fuzzing uses random inputs but modern tools inspect the executable and modify the input to try out more branches and check obscure edge cases. Like all kinds of testing fuzzing is a good way to find underhanded code, but also many kinds of other bugs. Unlike with unit tests no one needs to write all kinds of test cases with a given input and an expected output and will still miss some bugs because of misconceptions about the topic and the impossibility to test every path. Of course fuzzing can’t find other kinds of errors, like all errors not causing illegal memory access or panics but wrong behavior. The article contains a tutorial on how to use cargo fuzz a new command-line tool to do simple fuzzing with rust crates. If you haven’t already read the article now and try it with one of your own crates (or one from crates.io).

The crate I selected for fuzzing is httpdate. I blogged about it before. It is very simple and parses a date found in a HTTP header to a SystemTime and is also able to format a time inside a header.

With the command line utility a fuzzing folder is created. A fuzzer needs a function to test. It has to take a slice of bytes as an input and test it. The function I wrote is very simple: I tries to read the bytes as an UTF-8 string and if this fails it does just nothing. Otherwise it feeds the input to parse_http_date. If this function panics we have found a bug! In case the fuzzer actually produced a valid timestamp it also formats the time again (and checks if a non-empty string is produced).

#[export_name="rust_fuzzer_test_input"]
pub extern fn go(data: &[u8]) {
    if let Ok(s) = str::from_utf8(data) {
        if let Ok(d) = parse_http_date(s) {
            let o = fmt_http_date(d);
            assert!(!o.is_empty());
        }
    }
}

The whole logic is really simple and I had a few tests so I did not really expect many crashes. But I was proven wrong and the fuzzer immediately tried to parse some Ùníçôde characters as a date. This crashed the parser which works with string slices. Take the string "foobar". Slicing it with s[3..5] gives "ba". But if you try "später" it will panic with a message that 3 is no character boundary. This is the case because 'ä' takes up two bytes instead of one like ASCII symbols and rusts slice indices work on bytes and not codepoints. The fix was simple: Just check before parsing whether s.is_ascii() is true. I recompiled the code and started the fuzzer again (cargo fuzz stops after the first bug found). This turned up another bug: Dates which contain some thing like “May 35” will be accepted because I did never check if the individual values where in a valid range! This caused some arithmetic under- and overflows in the conversion from a date with year-month-day-hour-minute-second to seconds since the epoch. Now I check if all numbers are in a valid range. This is a quite obvious bug, I should have been able to spot it earlier. After this the fuzzer found no further bugs. I just added some checks if all required whitespace is present in the date strings and then published a bugfix release 0.2.1 to crates.io.

Why the title? While many Rustaceans say “If it compiles it works” and this mostly true the code will still contain errors. Since fuzzing in Rust is now as easy as writing a single function every library that works with inputs should be fuzzed. In the next days or weeks I will try to find bugs in other crates from the rust ecosystem and write about my findings.

If you also want to fuzz a crate but don’t know which one the ones listed for the keyword “parser” are usually simple. If they were already fuzzed in the past for example with afl.rs you most likely not find anything but otherwise I am certain there will be crashes in many crates.