Latest / Cybersecurity Business with Fexingo: Security Companies, Breaches, and Enterprise Defense / How Security Teams Use Memory Scanning to Stop Fileless Malware
Transcript
- Lucas: If these conversations are useful for what you're building or running, that's exactly why we do them. We deliberately don't run ads on these shows. If you want to support that choice, the link is buy me a coffee dot com slash fexingo. Now, onto today's topic. Luna: And it's a good one. We're talking about something that keeps security teams up at night: fileless malware. Lucas: Right. Fileless malware is not really a single technique. It's a category of attacks that don't write a malicious executable to disk. Instead, they operate entirely in memory, using legitimate system tools like PowerShell, Windows Management Instrumentation, or even Microsoft Office macros that load code directly into RAM. Luna: And that's what makes it so hard to detect. Traditional antivirus scans files on disk. If there's no file, there's nothing to scan. Lucas: Exactly. So security teams have had to shift their approach. Instead of looking for known malware signatures, they're now focusing on behavior. Specifically, they're scanning memory in real time to spot anomalous process activity. This is called in-memory scanning or memory forensics at runtime. Luna: But scanning memory is computationally expensive, right? You can't just dump every process's memory and analyze it without slowing down the system. Lucas: That's the challenge. The key is that you don't scan all memory all the time. Modern endpoint detection and response platforms use a combination of hooks — they monitor API calls, process creation events, and memory allocation patterns. When something looks suspicious, they trigger a deeper scan of that specific process's memory space. Luna: So it's more like a triage system than a full scan. What are they looking for specifically? Lucas: Common indicators include processes that inject code into other processes, which is a classic sign of something like Cobalt Strike or Meterpreter. They also look for PowerShell that decodes and executes base64-encoded scripts, or WMI that spawns child processes with network connections. Basically, any behavior that deviates from the baseline of how a tool is normally used. Luna: And attackers have caught on. They're now using things like reflective DLL loading or process hollowing to try to evade even those behavioral detections. Lucas: Right. Reflective DLL loading means the malware maps a DLL into memory from its own copy, bypassing the standard Windows loader. That avoids creating a file on disk and also avoids some API monitoring hooks. So defenders have had to go deeper. Luna: What's the state of the art right now, mid-2026? Are most large enterprises deploying memory scanning? Lucas: Most large enterprises now have some form of endpoint detection and response that includes memory scanning. But the sophistication varies. A lot of organizations rely on the built-in capabilities from CrowdStrike, SentinelOne, or Microsoft Defender for Endpoint. Those platforms have kernel-level drivers that can inspect memory without being easily detected by user-mode malware. Luna: But there's a performance trade-off. I've heard of cases where aggressive memory scanning caused blue screens or application crashes. Lucas: Absolutely. That's why tuning is critical. You don't want to block legitimate software. For example, some enterprise applications use process injection for legitimate purposes — like performance monitoring tools. So security teams have to build exception lists and test thoroughly before rolling out new detection rules. Luna: Let's talk about a real example. I read about a bank that caught a fileless ransomware attack using memory scanning. Can you walk through that? Lucas: Yeah, this was a major European bank in late 2025. The attack started with a phishing email that contained a Word document with a malicious macro. When the user enabled macros, the macro launched PowerShell to download a payload from a remote server — but the payload was never saved to disk. Luna: Classic fileless chain. So what happened next? Lucas: The PowerShell script used.NET reflection to load an assembly directly into memory. That assembly then enumerated network shares and attempted to encrypt files. But the bank's EDR detected the unusual PowerShell process spawning child processes that were writing to file shares — a behavior that didn't match any approved administrative task. Within thirty seconds, the EDR triggered a memory scan of that PowerShell process, found the malicious assembly, and terminated the process. Luna: So detection came from behavioral monitoring, not from the memory scan itself. The memory scan confirmed the suspicion. Lucas: Exactly. The behavioral alert triggered the memory scan, which provided the forensic evidence. That's the typical workflow. And it's effective because attackers often try to hide their code in memory, but once you look, it's usually obvious — like a.NET assembly that's not signed and has no legitimate purpose. Luna: Now, what about attackers who use encryption to obfuscate their in-memory payload? I've seen reports of malware that encrypts its code until it's executed, so even memory scanning sees only garbage. Lucas: That's a growing arms race. Some advanced malware uses what's called 'sleeping' or 'decoy' code — it appears benign until triggered. But even then, behavioral patterns give it away. For example, if a process allocates memory with read write execute permissions, that's a huge red flag. Most legitimate software doesn't need that. So memory scanning doesn't just look for known signatures; it looks for suspicious memory permissions and structures. Luna: What about Linux environments? Fileless malware isn't just a Windows problem anymore. Lucas: True. Linux has its own fileless techniques — using Python scripts, bash, or even direct system calls via tools like memfd_create, which creates an anonymous file in memory. Security teams are now applying similar EDR capabilities to Linux servers, especially in cloud environments where containers can be infected without ever writing to disk. Luna: So the trend is clear: memory scanning is becoming a standard part of the security stack. But what about smaller organizations that can't afford a full EDR suite? Are there any open-source tools that help? Lucas: There are tools like Volatility for memory forensics, but that's more for post-incident analysis, not real-time. For real-time protection, you really need an EDR. However, Microsoft's Defender for Endpoint is included with some enterprise licenses, and there are more affordable options now. The key is that even basic behavioral monitoring — like logging PowerShell execution and alerting on suspicious patterns — can catch a lot of fileless attacks without a huge investment. Luna: Let's look ahead. How do you see this evolving over the next year or two? Lucas: I think we'll see more use of AI to analyze memory patterns in real time. Instead of just looking for known bad behaviors, machine learning models can detect novel anomalies — like a process that suddenly allocates a huge chunk of memory in a way that doesn't match its historical profile. Also, expect more integration with cloud workload protection platforms to cover containers and serverless functions. Luna: And attackers will respond. They always do. Lucas: Absolutely. That's the nature of the game. But the core principle remains: if you can see what's running in memory, you can catch attacks that leave no file on disk. And that visibility is only getting better. Luna: Good episode. Next time, we'll look at how attackers are using AI to generate polymorphic malware that changes its in-memory signature every execution.