Researchers Demonstrate Comprehensive Suite of New Techniques to Protect AI from ‘Backdoor Attacks’
For Immediate Release
Researchers have demonstrated three new techniques to prevent so-called “backdoor attacks,” which induce malicious or undesirable behavior in AI models. The researchers have also proposed a concept that would repurpose backdoor attack techniques to create an additional layer of security for users.
Backdoor attacks refer to instances in which someone has secretly embedded a “trigger” in an AI model that – when activated – forces the model to perform specific actions.
“The important point is that the model can behave normally when the trigger is absent, making the backdoor difficult to notice during ordinary use or testing,” says Jung-Eun Kim, corresponding author of four papers describing the work and an assistant professor of computer science at North Carolina State University. “But when the trigger is activated, it can cause the model to do things an attacker intended such as generating unwanted text and executing malicious commands.
“Given that AI models are used for everything from financial transactions to software development, the ability to control behaviors of AI represents a significant vulnerability,” says Kim.
As AI models have become more common, and are being used for more applications, there is an increasing need to “fine-tune” the models to perform these new tasks effectively. Fine-tuning refers to additional training for AI models, which involves using additional datasets. This creates new opportunities for backdoor attacks, which can be tailored to the specific functions that model is being fine-tuned to perform.
“This is a well-known vulnerability, but backdoor attacks are difficult to prevent because they are hard to identify until the trigger has been activated,” says Jianwei Li, first author of all four papers and a Ph.D. student at NC State. “Our goal with this work was to prevent backdoor attacks before deployment in order to make AI safer and more trustworthy.”
To address backdoor concerns, the researchers looked at three different levels of AI models: features, architecture, and weights.
“Weights are the parameters that make up a model,” says Li. “Features are internal representations of the data in a model. And architecture refers to the internal operating components of the model.
“These are all essential elements of an AI model, and we’ve developed techniques that modify all three of those levels to remove or mitigate backdoor attacks without making significant sacrifices in performance,” says Li.
The researchers also propose the use of “secret alignment,” using triggers like those used in backdoor attacks to perform specific beneficial tasks. For example, a trigger could be installed that would be known only to management-level users, giving them restricted access to secure data or control functions.
“Overall, we’ve introduced a comprehensive suite of techniques that reduces the threat of backdoor attacks, all of which could be implemented immediately,” says Kim. “And we’ve also outlined an approach for repurposing backdoor triggers into useful features.”
The paper focusing on AI model features, “Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection,” will be presented at the Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS), being held in December in Sydney (Australia), Atlanta, (U.S.) and Paris (France). The paper focusing on AI model architecture, “Backdoor Containment via Expert Quarantine and Shutdown in LLMs,” will also be presented at NeurIPS and was co-authored by NC State Ph.D. student Min-Seon Kim.
The paper focusing on AI model weights, “Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference,” was presented at the Fourteenth International Conference on Learning Representations (ICLR), held April 23-27 in Rio de Janeiro, Brazil. The paper proposing the concept of “secret alignment,” “Position: Retire the ‘Positive Backdoor’ Label – Secret Alignment Requires Strict and Systematic Evaluation,” was presented at the Forty-Third International Conference on Machine Learning (ICML), held July 6-11 in Seoul, South Korea.
-shipman-
Note to Editors: The study abstract follows.
“Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection”
Authors: Jianwei Li and Jung-Eun Kim, North Carolina State University
Presented: Dec. 6-13, the Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS), in Sydney, Atlanta and Paris
Abstract: With the rapid adoption of large language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access to clean references, or aggressive retraining, and they often lack comprehensive evaluations. These constraints substantially limit their practical applicability. To overcome these challenges, our work proposes purifying LoRA-tuned LLMs without these assumptions and even without post-hoc retraining of the suspect parameters. Our objective is to significantly reduce the attack success rates (ASR) while preserving both (i) the base model’s general capabilities and (ii) the new downstream skills learned through the adapter. Through a series of ablation studies, we progressively scale our approach from a single layer in a text classification setting to a full-parameter LLM in the generative task. Through careful data curation and feature approximation, we extract high-fidelity backdoor directions and, for each layer or head, construct orthogonal null spaces in both the input and output channels, onto which the LoRA updates are projected. Empirically, our null-space projection method reduces the ASR from nearly 100% to less than 10%, while preserving the base model’s benign performance and the adapter’s learned abilities during downstream task adaptation.
“Backdoor Containment via Expert Quarantine and Shutdown in LLMs”
Authors: Jianwei Li, Min-Seon Kim and Jung-Eun Kim, North Carolina State University
Presented: Dec. 6-13, the Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS), in Sydney, Atlanta and Paris
Abstract: Backdoored large language models (LLMs) can behave normally on benign inputs while producing attacker-specified outputs under hidden triggers. Existing defenses span four stages–prior-training, in-training, post-training, and inference-time–and share one of two underlying strategies: either suppress backdoor learning (by filtering poisoned data or interrupting its acquisition during optimization) or learn, then purify (by repairing model weights or gating inputs after a fully backdoored model has formed). We propose a third strategy, learn, but channel: allow backdoor formation during training but route it into a designated, quarantined component that can be disabled at deployment. To this end, we propose Quarantined Expert Shutdown QES, a computationally efficient containment strategy built in a regularization-steered MoE-like setting. Specifically, given a poisoned dataset, QES augments a Transformer-based language model with routed expert-specific LoRA branches and lightweight routers, and uses auxiliary routing objectives to attract trigger-conditioned behavior into a designated expert while preserving benign capability elsewhere. At deployment, mitigation reduces to a single constant-time operation: zeroing the quarantined expert’s routing weight, without trigger screening or further updating model weights. Empirically, our methods reduce the attack success rate ASR from 100% to 0-10% on most settings across two tasks, three attacks, and four model families, while downstream utility is often preserved or only modestly affected. These results establish learn, but channel as a previously unexplored regime for backdoor containment in generative LLMs.
“Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference”
Authors: Jianwei Li and Jung-Eun Kim, North Carolina State University
Presented: April 23-27, The Fourteenth International Conference on Learning Representations (ICLR) in Rio de Janeiro, Brazil
Abstract: Backdoor attacks pose severe security threats to large language models (LLMs), where a model behaves normally under benign inputs but produces malicious outputs when a hidden trigger appears. Existing backdoor removal methods typically assume prior knowledge of triggers, access to a clean reference model, or rely on aggressive finetuning configurations, and are often limited to classification tasks. However, such assumptions fall apart in real-world instruction-tuned LLM settings. In this work, we propose a new framework for purifying instruction-tuned LLM without any prior trigger knowledge or clean references. Through systematic sanity checks, we find that backdoor associations are redundantly encoded across MLP layers, while attention modules primarily amplify trigger signals without establishing the behavior. Leveraging this insight, we shift the focus from isolating specific backdoor triggers to cutting off the trigger-behavior associations, and design an immunization-inspired elimination approach: by constructing multiple synthetic backdoored variants of the given suspicious model, each trained with different malicious trigger-behavior pairs, and contrasting them with their clean counterparts. The recurring modifications across variants reveal a shared “backdoor signature”-analogous to antigens in a virus. Guided by this signature, we neutralize highly suspicious components in LLM and apply lightweight finetuning to restore its fluency, producing purified models that withstand diverse backdoor attacks and threat models while preserving generative capability.
“Position: Retire the ‘Positive Backdoor’ Label – Secret Alignment Requires Strict and Systematic Evaluation”
Authors: Jianwei Li and Jung-Eun Kim, North Carolina State University
Presented: July 6-11, Forty-Third International Conference on Machine Learning (ICML) in Seoul, South Korea
Abstract: This position paper argues that the AI/ML community should stop overclaiming and retire the label “positive backdoor,” and instead treat trigger-activated hidden behaviors as Secret Alignment. Crucially, protective claims based on Secret Alignment should be presumed not secure by default unless supported by rigorous, standardized evaluation. The Private AI era, enabled by open-weight LLMs and accessible training/inference stacks, turns language models into privately owned digital assets, creating security concerns around unauthorized access, model theft, and behavioral misuse. Recently, a line of work framed as “positive backdoors” has been proposed to address these challenges. To ground our position in evidence, we unify these proposals as covert trigger-behavior associations for access gating, ownership attribution, and safety enforcement, and evaluate three representative applications across six core properties: effectiveness, harmlessness, persistence, efficiency, robustness, and reliability. Our results reveal substantial brittleness – especially in the confidentiality, integrity, and availability (CIA) – of trigger-behavior mappings often underrepresented by existing claims. We further relate these outcomes to behavior density and decision complexity, offering a behavioral lens for understanding deployment-time risks and motivating community-wide evaluation that makes Secret Alignment claims provable.
