Examining Evasive Malware Techniques: A Memory-Based and Behavioral Study of AgentTesla

Authors

DOI:

https://doi.org/10.32996/jcsts.2025.7.10.28

Keywords:

Behavioral Analysis, Anti-VM/Sandbox, SMTP Exfiltration, Memory Injection, Obfuscation, Firewalls

Abstract

One of the largest evasive malware programs, AgentTesla, circumvents conventional detection methods by taking advantage of cutting-edge techniques like memory injection, sandbox evasion, and obfuscation. In this work, 35 AgentTesla samples gathered from open malware repositories under the name MalwareBazaar are analyzed behaviorally and memory based. A thorough description of evasion techniques is provided throughout the study to show how AgentTesla successfully overcomes defenses including signature-based and heuristic ones, such as anti-VM checks, SMTP-based data exfiltration, and hollowing. The study's conclusions emphasize the limitations of continuous analytic techniques and the need for behavioral, memory-focused, adaptive detection models to avoid these dangers. In order to enhance the future, this research also suggests a consolidated detection framework that combines memory forensics, machine learning training, and behavioral recording. In order to enhance the malware detection process going forward, this article also suggests a consolidated detection framework that combines memory forensics, machine learning training, and behavioral logging.

Author Biographies

  • Tabassum Sheikh Atkia, Master of Science in Cybersecurity, Computer Science & Engineering, Washington University of Science & Technology, Virginia, USA

    Master of Science in Cybersecurity, Computer Science & Engineering

  • Mahmud Rafsan, Independent Researcher, Virginia, USA

    Independent Researcher, Bachelor of Technology in Automotive

  • Sheikh Said Evna Jahidul Hoque, Independent Researcher, Federation University, Victoria, Australia

    Independent Researcher, Bachelor of Engineering (Mechanical), Federation University, Australia

  • Ashfaqur Rahman Jaigirdar, Master of Science in Cybersecurity, Computer Science & Engineering, Washington University of Science & Technology, Virginia, USA

    Master of Science in Cybersecurity, Computer Science & Engineering

Downloads

Published

2025-10-08

Issue

Section

Research Article

How to Cite

Sheikh Atkia, T., Rafsan, M., Sheikh Said Evna Jahidul Hoque, & Ashfaqur Rahman Jaigirdar. (2025). Examining Evasive Malware Techniques: A Memory-Based and Behavioral Study of AgentTesla. Journal of Computer Science and Technology Studies, 7(10), 240-249. https://doi.org/10.32996/jcsts.2025.7.10.28