Bio-inspired Hyperparameter Optimization for Deep Learning Malware Detection: An Empirical Study Across AIS and Evolutionary Algorithms

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The performance of deep learning malware classifiers depends heavily on hyperparameter configuration. Deep learning malware detection benefits from principled hyperparameter optimization. This study evaluates six bio inspired optimizers across four neural architectures and three widely used malware data representations. Artificial Immune System methods include Clonal Selection, Immune Network, and a Dendritic Cell inspired optimizer. Evolutionary methods include Genetic Algorithm, Particle Swarm, and Differential Evolution. Baselines for each deep learning model were implemented using domain informed choices rather than naïve defaults to avoid overstating optimization gains and to reflect realistic practitioner practice. All runs were under a strictly matched evaluation budget (g1/460 function evaluations per run) with five independent seeds per permutation. On API call sequences, LSTM produced the highest test accuracy near 0.991, with Differential Evolution and Particle Swarm frequently leading, while Transformer on the same dataset reached about 0.990. On PE headers, ANN achieved about 0.989 with Differential Evolution and Immune Network. Opcode n grams formed the most challenging setting, with the best CNN accuracy near 0.75 under multi class family labels. Artificial Immune System algorithms often reached the highest accuracies but typically at greater runtime cost, whereas Evolutionary Algorithms, especially Differential Evolution and Particle Swarm, were faster to strong solutions. under a strictly matched evaluation budget (g1/460 function evaluations per run) with five independent seeds per permutation. The optimizers all perform comparatively well to each other on each dataset, with no statistical difference in test accuracy. The experiment also quantify accuracy versus computing tradeoffs, highlight dataset model synergies, and provide guidance on selecting model optimizer pairs for deployment-oriented scenarios. The research notes ceiling effects on API/PE and a modest, matched budget; future work will examine larger budgets and more challenging datasets to further probe optimizer differences under controlled evaluation.

Original languageEnglish
Title of host publicationCIIS 2025 - 2025 the 8th International Conference on Computational Intelligence and Intelligent Systems
PublisherAssociation for Computing Machinery, Inc
Pages145-154
Number of pages10
ISBN (Electronic)9798400719332
DOIs
Publication statusPublished - 2 Feb 2026
Event2025 The 8th International Conference on Computational Intelligence and Intelligent Systems, CIIS 2025 - Okayama, Japan
Duration: 21 Nov 202523 Nov 2025

Publication series

NameCIIS 2025 - 2025 the 8th International Conference on Computational Intelligence and Intelligent Systems

Conference

Conference2025 The 8th International Conference on Computational Intelligence and Intelligent Systems, CIIS 2025
Country/TerritoryJapan
CityOkayama
Period21/11/2523/11/25

Keywords

  • artificial immune systems
  • deep learning
  • evolutionary algorithms
  • hyperparameter optimization
  • Malware detection

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems

Fingerprint

Dive into the research topics of 'Bio-inspired Hyperparameter Optimization for Deep Learning Malware Detection: An Empirical Study Across AIS and Evolutionary Algorithms'. Together they form a unique fingerprint.

Cite this