Fileless Malware and its evolving ML based detection, classification and prevention

By InnoSec May 28, 2024

The main approach for detection and prevention of malware is based upon the concept of file signatures. More specifically, the anti-virus (AV) solutions are scanning the system’s storage devices and hash files at process/creation time. They then compare each of these hashes with their respective curated database which contains signatures of known malware samples. If there is a match then the file is flagged as malicious and subsequent actions take place to contain it. But what could happen if there was some kind of malware that did not rely on a file to accomplish its mission?

During the last decade the Cybersecurity field witnessed the rise and domination of fileless malware (Advanced Volatile Threat – AVT). It debuted in 2001 (Code Red worm) and according to Cybersecurity enterprises and institutes, by the end of 2022 around 71% of the attacks employed fileless techniques. The dawn of the present decade came with a surging increase of fileless malware detections by a whooping 900% as threat actors eventually came to realise its potential and efficiency. Indicative of the former is the fact that APT groups and threat actors’ operations utilise fileless malware.

In contrast to traditional malware, fileless malware do not download any files into target system’s storage. Instead, the malicious code is injected directly into memory. The aforementioned is usually accomplished by taking advantage of trusted applications already present in the system (aka LOLBins and LOLScripts i.e. Living-of-the-Land Binaries and Scripts) like PowerShell, WMI and PsExec in Windows OS. Since there is no file to generate its signature and compare it against a database of signatures of known malware, AV solutions are of little to no use against fileless malware. Machine Learning (ML) can support the detection of fileless malware, thus making it a valuable component for an effective, contemporary AV solution.

The challenge isn’t just detecting the threat but also classifying it accurately. Not all unusual activity is malicious — sometimes it’s just a legitimate program acting differently. ML-based classification helps security tools separate real threats from false alarms, making sure teams don’t waste time chasing harmless anomalies. This approach also evolves continuously: the more data the system sees, the better it gets at distinguishing normal from dangerous behaviour.

Prevention is the final piece of the puzzle. With ML-powered tools, organisations can proactively block suspicious activity before it causes damage, rather than waiting for a breach to happen. This might mean automatically stopping a process that looks risky, isolating affected devices, or alerting IT teams instantly. For everyday users, these advances mean better protection behind the scenes, by keeping devices safe even against threats that can’t be spotted with traditional antivirus software. In a world where cyberattacks are getting smarter, fileless malware is a tough opponent, but machine learning is proving to be a powerful defense.