Abstract:
Malware is the major cause of data breaches, resulting in financial losses in excess of $400 billion in 2017. The key ones, eluding malware scanners, are metamorphic. Malware metamorphic engines use varying obfuscating techniques to evade virus scanners. Current detection solutions are not effective against metamorphic malware. This study investigated metamorphic malware intrusions and developed a detection mechanism that combats them. A taxonomy of current malware detection mechanisms was created through extensive review of extant literature. Cosine similarity index, used to compare two files was added to dynamic link library, a feature derived from the disassembly process of a portable executable, to effectively determine if a file is a malware or not. A prototype of the system was developed using the java programming language. The virustotal website, which contains about 66 antimalware engines and scanners, was used to scan benign and malicious files. Experiments were conducted to prove that certain concealing techniques could aid malware evade existing antivirus scanners. A prototype of the detection system was evaluated against malware obfuscated using register re-assignment and dead
code insertion techniques. Dead code insertion, register reassignment and instruction substitution were the three beclouding techniques used by malware metamorphic engines. The use of cosine similarity index together with linked libraries approach to detect metamorphic malware prototype was developed. A portable executable file is classified as a malware when its similarity index is high, 0.6 – 1, and it uses suspicious dynamic linked libraries. It was discovered that the most difficult obfuscating technique to implement by malware
metamorphic engine is instruction substitution because non-availability of a line of code that is syntactically synonymous is probable. It was also observed that the register re-assignment technique on a malware made it evade every antimalware scanner on the virustotal website. Results showed that the prototype was 100% accurate as long as the right threshold was used, and as long as the parent malware was known. It was concluded that financial losses through malware invasion would be avoided, by adding the developed detection system to complement existing detection systems, in order to capture metamorphic malware effectively. This will benefit the general public, as the adoption of
the proposed detection system by antimalware companies such as Symantec and McAfee, would lead to more efficacious antimalware systems. This would contribute to a more secure computing environment.
Keywords: Code substitution, Cosine similarity index, Dead code insertion, Linked library, Malware, Metamorphic engine, Register
reassignment.
