Fuzzy Hashing Aided Enhanced YARA Rules for Malware Triaging

Abstract

Cybercriminals are becoming more sophisticated wearing a mask of anonymity and unleashing more destructive malware on a daily basis. The biggest challenge is coping with the abundance of malware created and filtering targeted samples of destructive malware for further investigation and analysis whilst discarding any inert samples, thus optimising the analysis by saving time, effort and resources. The most common technique is malware triaging to separate likely malware and unlikely malware samples. One such triaging technique is YARA rules, commonly used to detect and classify malware based on string and pattern matching, rules are triggered and alerted when their condition is satisfied. This pattern matching technique used by YARA rules and its detection rate can be improved in several ways, however, it can lead to bulky and complex rules that affect the performance of YARA rules. This paper proposes a fuzzy hashing aided enhanced YARA rules to improve the detection rate of YARA rules without significantly increasing the complexity and overheads inherent in the process. This proposed approach only uses an additional fuzzy hashing alongside basic YARA rules to complement each other, so that when one method cannot detect a match, then the other technique can. This work employs three triaging methods fuzzy hashing, import hashing and YARA rules to perform extensive experiments on the collected malware samples. The detection rate of enhanced YARA rules is compared against the detection rate of the employed triaging methods to demonstrate the improvement in the overall triaging results.