Detection of In-memory indicators of compromise for stealthy Windows backdoors

The significant increase in harmful software applications, or malware, in recent years has made it increasingly difficult for anti-virus programs to detect and remove the malware before any harm could be done.  One variant is called backdoor malware, a type of malicious software that allows an attacker to gain unauthorised access to a computer system through unconventional methods.

Backdoor malware excels in gaining control of computer systems by using the system’s vulnerabilities. This type of malware is used to steal information and install additional malware. Backdoor malware is a serious threat to computer systems. X-Force, a threat intelligence and research team at IBM, observed that 21% of the top threat actors in 2022 were caused by backdoor malware (see Figure 1) [1].

Backdoor malware developers are constantly using new methods to ensure that their backdoor applications would be harder to detect. One of these is dynamic code-loading, which allows attackers to load harmful code into a computer’s memory/RAM, making it more challenging for anti-malware tools to detect it. The stealthier and more advanced the dynamic code-loading technique is, the harder it would be for anti-virus software to detect malware. There are three main types of tools that anti-virus programs use to find malware, namely: those that look for it before it runs, those that check while it is running, and those that search for it after it finishes running.

The aim of this project was to develop a detector for Windows against backdoor activity that would be resilient to stealth techniques.  These indicators could provide other anti-virus tools with valuable information to help them catch backdoor malware using dynamic code-loading. The tool would be designed to scan files in real-time while running, where it would search for any suspicious behaviour in the memory/RAM that could indicate the presence of malicious code. It would identify and gather the instructions sent by the attacker/hacker. It would also locate the malicious code that was stored in memory/RAM for stealth and the main actions that the file made in this period.

The core work underpinning the detector’s design involves how the various backdoor activity to be collected directly from the computer’s memory could be identified and understood. This analysis played a significant role in the tool’s design, as it helped solidify the understanding of how it should be designed and built, providing the steps the tool would need to take. 

This project contributes to increased protection for computer systems against backdoor malware that uses dynamic code loading, thus helping to ensure better security of personal information and systems.

Figure 1. Top threat actor impacts for 2022 by IBM Security X-Force

Student: Kyle Borg
Supervisor: Dr Mark Vella