Deobfuscation of virtualization-obfuscated software

Kevin Coogan, Gen Lu, Saumya Debray
2011 Proceedings of the 18th ACM conference on Computer and communications security - CCS '11  
When new malware are discovered, it is important for researchers to analyze and understand them as quickly as possible. This task has been made more difficult in recent years as researchers have seen an increasing use of virtualization-obfuscated malware code. These programs are difficult to comprehend and reverse engineer, since they are resistant to both static and dynamic analysis techniques. Current approaches to dealing with such code first reverseengineer the byte code interpreter, then
more » ... e this to work out the logic of the byte code program. This outside-in approach produces good results when the structure of the interpreter is known, but cannot be applied to all cases. This paper proposes a different approach to the problem that focuses on identifying instructions that affect the observable behavior of the obfuscated code. This inside-out approach requires fewer assumptions, and aims to complement existing techniques by broadening the domain of obfuscated programs eligible for automated analysis. Results from a prototype tool on real-world malicious code are encouraging.
doi:10.1145/2046707.2046739 dblp:conf/ccs/CooganLD11 fatcat:g5a56ojclfbj3o7v3hd567bmk4