Llama-1b Achieves 0.7159 F1-Score for IoT Network Attack Detection

Summary of Research on LLM-Based IoT Intrusion Detection

This research details a novel approach to IoT intrusion detection using Large Language Models (LLMs), specifically focusing on adaptability and resource efficiency. Here’s a breakdown of the key findings:

Problem Addressed: Traditional intrusion detection systems struggle with the need for constant retraining to address evolving cyber threats.

Proposed Solution: A unified LLM-based framework that can handle both known and unknown attacks by:

* Transforming network traffic features into natural language prompts: This bridges the gap between structured data and the LLM’s semantic understanding.
* Utilizing QLoRA fine-tuning: This allows for efficient adaptation of LLMs, even on resource-constrained hardware (like IoT devices).
* Implementing Retrieval-Augmented Generation (RAG): This enables zero-shot attack detection – identifying unseen attack types without requiring retraining.

Key Results:

* Comparable Accuracy to Traditional Methods: A QLoRA-tuned LLaMA-1B model achieved an F1-score of 0.7124 for detecting known attacks, matching the 0.7159 F1-score of a Random Forest baseline.
* significant zero-Shot capability: The RAG-enhanced system achieved 42.63% accuracy in identifying unseen attack types without any additional training.
* Versatility across LLM Architectures: The framework was successfully tested with multiple decoder-only LLMs including GPT-2, LLaMA-1B, LLaMA-3.2-1B, Meta-LLaMA-3-8B, and Mistral-v0.3-7B.
* Resource Efficiency: The approach is designed for deployment on resource-constrained IoT devices.

Significance:

This research demonstrates the potential of LLMs, combined with techniques like QLoRA and RAG, to create a more adaptable and efficient next-generation IoT intrusion detection system. The zero-shot capability is a major advancement, offering a solution to the ongoing challenge of keeping security systems up-to-date against constantly evolving threats.

Presentation: The findings will be presented at the 7th Computing,Communications and IoT applications Conference (ComComAp 2025) in Madrid,Spain,in December 2025.

In essence,this work offers a promising pathway to more robust and adaptable cybersecurity for the increasingly vulnerable world of IoT devices.