Botacin's Lab¶
Malware analysis & detection · Texas A&M University
Malware research that holds up in the real world.
We study how malware works, how it evades defenses, and how to detect it. Our work spans large-scale analyses of threats in the wild, machine-learning detection and its limits, antivirus operations, and detectors built into the hardware itself.
Principal investigator Prof. Marcus Botacin
Department Computer Science & Engineering
Institution Texas A&M University
Contact botacin@tamu.edu
Principal investigator
Director¶
Prof. Marcus Botacin¶
Assistant Professor · Computer Science & Engineering
Marcus Botacin is an Assistant Professor in the Department of Computer Science & Engineering at Texas A&M University, where he directs Botacin's Lab. His research focuses on malware analysis, evasion and detection, sandbox development, antivirus operations, hardware-assisted security and reverse engineering.
He joined Texas A&M in 2022 as a Visiting Assistant Professor and became an Assistant Professor in 2024. He earned his PhD from the Federal University of Paraná (UFPR), Brazil, in 2021, and previously taught at UFPR as a lecturer and external professor.
Teaching: CSCE 413 Software Security (2025), CSCE 704 Data Analytics for Cybersecurity (2024), CSCE 689 Machine Learning-Based CyberDefenses (2023).
Recent talks: CYBR.SEC.CON (Sep 2026), GMU ECE seminar (Aug 2026), CERIAS seminar at Purdue (Oct 2025), HOU.SEC.CON (Sep 2025). See the news.
What we do
Research areas¶
01
Malware research¶
Longitudinal analyses of in-the-wild threats, static and dynamic detection, sandboxes and analysis frameworks, and fuzzing and symbolic execution of malware samples.
02
Antivirus solutions¶
Metrics to evaluate real antivirus products, and the design of next-generation solutions.
03
AI and machine learning¶
ML models for malware detection, their evaluation in realistic scenarios, adversarial attacks against detectors, and AI-generated malware.
04
Hardware security¶
Moving antivirus capabilities into hardware and building secure-by-design systems.
05
Reverse engineering¶
New debuggers, how analysts use debuggers to reverse engineer code, and AI-enhanced debugging.
06
Theory and methods¶
Formal definitions of malware, theories of maliciousness, evaluation metrics, malware clustering and threat intelligence extraction.
Recent
Latest publications¶
AutoPYara: Next-Gen YARA Rule Generator for Malware Family Clustering
Mabon Ninan*, Nhat Minh Nguyen*, Soumyajyoti Dutta, Sidharth Anil, Marcus Botacin. *Equal contribution. To appear.
YARAclustering
When GANs meet LLMs: Bridging the Feature-Problem space gap for efficient adversarial ML-based malware generation
Dondapati et al.
adversarial MLmalware generation
Making Acoustic Side-Channel Attacks on Noisy Keyboards Viable with LLM-Assisted Spectrograms Typo Correction
Ayati et al.
side channelsLLM
Towards Explainable Drift Detection and Early Retrain in ML-Based Malware Detection Pipelines
Jayesh Tripathi, Heitor Gomes, Marcus Botacin.
concept driftexplainability
2026-09-16
Prof. Botacin presents You Can See Me but You Can't Track Me: Evading Behavioral Detection with Distributed Malware and Covert Synchronization Channels at CYBR.SEC.CON.
2026-08-04
Invited seminar at George Mason University (ECE): Malware Analysis & Detection Research: A Journey Across Time, Space, Hardware, and Software.
2026
AutoPYara is accepted at ACSAC 2026. The Python package, Java backend, reproducibility artifact and evaluation data are public.
2026-05
When GANs meet LLMs (Dondapati et al.) is published in ACM Transactions on AI Security and Privacy, with source code released.
2025-10-29
Invited talk at the CERIAS seminar, Purdue: Malware Detection under Concept Drift: Science and Engineering.
Work with us
Join the lab¶
We are recruiting PhD, Master's and undergraduate researchers in malware analysis, detection and hardware security. See how to apply or email botacin@tamu.edu.