NETE: Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models

Implementation

Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models

Published in Neural Networks (2026)

---

📖 Abstract

This repository contains the implementation of NETE (perturbatioN discrEpancy consisTency Evaluation), a novel approach for detecting backdoor samples in pre-trained language models through perturbation discrepancy consistency analysis. Our method achieves state-of-the-art performance across multiple backdoor attack scenarios and datasets.

Figure 1: Overview of the NETE detection framework

---

🚀 Quick Start

Environment Setup

# Create and activate conda environment
conda create -n backdoor-detect-nete python=3.7
conda activate backdoor-detect-nete

# Install dependencies
pip install -r requirements.txt

# Clone repository
git clone https://github.com/pzq7025/BackdoorDetection.git
cd BackdoorDetection

# Run quick demo
bash xxx.sh

---

📊 Datasets

Primary Evaluation Datasets

Dataset	Domain	Description	Source
Yelp	Sentiment Analysis	Restaurant reviews dataset	Link
OLID	Offensive Language	Offensive language identification	Link
COVID	Social Media	COVID-19 related tweets	Link

Note: All primary datasets are located in the datasets_experiment directory. Style transfer augmentation is performed using STRAP.

Attack Scenarios & Datasets

Attack Type	Dataset Name	Framework	Reference
Word-level	Various	OpenBackdoor	Standard word triggers
Sentence-level	Various	OpenBackdoor	Sentence patterns
Syntactic-level	Various	OpenBackdoor	Grammar structures
BadChain	badchain_datasets	Custom	Repo
BadEdit	badedit_datasets	Custom	Repo
CBA	CBADatasets	Custom	Repo
Sleeper Agent	sleepagent_dataset	Custom	Repo
VPI	VPIDatasets	Custom	Repo
Multi-level	multi_level_trigger	Custom	Combined triggers

Adversarial Evaluation

Adversarial samples are generated using TextAttack to evaluate robustness against natural perturbations.

---

🧪 Experiments

Core Evaluation

Primary Results

# Style-based backdoor detection
bash main_result.sh

# Multi-level backdoor evaluation (Word/Sentence/Syntactic)
bash patch_backdoor.sh

Ablation Studies

# Impact of different masking models
bash different_mask_model_experiments.sh

# Comparison across pre-trained model architectures
bash different_pretrained_model_experiments.sh

Advanced Attack Scenarios

# BadChain attacks (multiple trigger variants)
bash badchain_new_trigger_result.sh
bash badchain_result.sh

# BadEdit attacks
bash Badedit_result.sh

# Virtual Prompt Injection (VPI)
bash VPI_result.sh

# Sleeper Agent attacks
bash sleepagent_result.sh

# Clean-label backdoor attacks (CBA)
bash CBA_result.sh

# Multi-trigger scenarios
bash multi_trigger_result.sh

---

🔧 Custom Dataset Integration

To evaluate NETE on your own datasets, use the provided template:

Directory Structure

your_project/
├── custom_dataset_name/
│   └── backdoor_metadata.csv
└── custom_run.sh

Template Script (custom_run.sh)

#!/bin/bash

# GPU configuration
cuda=0

# NETE detection execution
CUDA_VISIBLE_DEVICES=$cuda python main_detect.py \
    --file_name backdoor_metadata \
    --pct_words_masked 0.7 \
    --random_fills \
    --random_fills_tokens \
    --dataset_path custom_dataset_name \
    --n_perturbation_list 1,3,5,10,50,100,200

Important: Ensure that backdoor samples occupy the first half of your dataset, with benign samples in the second half.

---

🔗 Related Work

For comprehensive backdoor attacks in large language models, see BackdoorLLM.

---

📝 Citation

If you find our work useful in your research, please consider citing:

@article{peng2026nete,
  title={Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models},
  author={Peng, Zuquan and Fu, Jianming and Zou, Lixin and Zheng, Li and Ren, Yanzhen and Peng, Guojun},
  journal={Neural Networks},
  volume={193},
  pages={108025},
  year={2026},
  publisher={Elsevier},
  doi={10.1016/j.neunet.2025.108025}
}

---

🙏 Acknowledgments

This work builds upon the foundational codebase from detect-gpt. We thank the authors for their valuable contributions to the field.

---

📄 License

This project is licensed under the Apache2.0 License - see the LICENSE file for details.

---

Paper | arXiv | Code