A compression-based technique to classify metamorphic malware

Metamorphic malware are able to change their appearance to evade detection by traditional anti-malware software. One of the ways to help mitigate the threat of new metamorphic malware is to determine their origins, i.e., the families to which they belong. This type of metamorphic malware analysis is not typically handled by commercial software. Moreover, existing works rely on analyzing the op-code sequences extracted from the Assembly files of the malware. Very few papers have tried to perform analysis on the binary files of the malware. However, they focused on the simple binary problem of differentiating between a certain malware family and benign files. In this work, we address the more difficult problem of determining the origin of a new metamorphic malware by measuring its similarity to hundreds of variants taken from 13 families of real malware. To address this problem, we use a compression-based classification approach. We experiment with two such approaches: AMDL and BCN. The results showed that AMDL performed no better than a random guess (11% accuracy for AMDL and 18% for the random baseline). On the other hand, BCN performed really well with 67% accuracy.