A “Fuzzy” Method, Clarified

by A.J. Kleber

For Sadullah Canakci (PhD’22), the future looks fuzzy – and he’s pretty pleased about it! He was the recipient of the Best Paper Award at May’s IEEE International Symposium on Hardware Oriented Security and Trust (HOST) for his work on hardware fuzzing.

“Fuzzing” is a testing and verification strategy used for  both software and hardware, though the latter is a fairly novel and recent application developed in response to the increasing complexity of today’s processors. Fuzzing  involves the intentional introduction of a diverse set of  inputs into a system to test its responses, which can help developers to identify bugs in a prototype prior to manufacture. However, the metrics used to assess software testing coverage–whether a test case or set of cases adequately represents the full range of functions which need to be tested–does not transfer well to hardware, and according to Canakci, early attempts to create hardware-appropriate metrics have fallen short on accuracy, among other issues.

In their award-winning paper, Canakci and his fellow researchers, including his former faculty advisors Professor Ajay Joshi and Professor Manuel Egele, present their work on  ProcessorFuzz, which uses several novel features utilizing CSR transitions and ISA simulations to improve the hardware fuzzing process. CSRs, or Control Status Registers, store information about the state of the processor , while the ISA, or Instruction Set Architecture, defines the interactions between software and hardware. Using CSR transitions and ISA simulations to guide the fuzzing process yields better coverage; it allows ProcessorFuzz to quickly identify whether a test session is “interesting,” or representative of a state that has not yet been tested, allowing it to move on quickly and identify a larger number of “interesting” tests overall.

The team evaluated ProcessorFuzz’s performance with three open-source processors, Rocket, BOOM, and BlackParrot, and were able to identify bugs 1.23 times faster, on average, than the best pre-existing hardware fuzzer. Their results included the exposure of 9 new bugs across the three test subjects, which were then confirmed by their respective developers. Such improved testing performance bodes well for all kinds of processor-based technology.

According to Canakci and Professor Joshi, the next step will be to find a way to make the technique broadly applicable, moving beyond the focus on processors to full systems on-chip. Expanding to leverage field-programmable circuits (FPGAs), instead of operating in simulation, would also make the testing process faster.

Sadullah Canakci received his Ph.D. in Spring 2022, advised by Professors Manuel Egele and Ajay Joshi. His dissertation was based on both hardware and software fuzzing. He is currently employed as a Silicon Design Engineer with AMD.