Publications

2025

1. 

   BadJudge: Backdoor Vulnerabilities of LLM-As-A-Judge

   Terry Tong, Fei Wang, Zhe Zhao, and 1 more author

   In The Thirteenth International Conference on Learning Representations. More Information can be found here , 2025

   Abs Bib PDF Website

   This paper proposes a novel backdoor threat attacking the LLM-as-a-Judge evaluation regime, where the adversary controls both the candidate and evaluator model. The backdoored evaluator victimizes benign users by unfairly assigning inflated scores to adversary. A trivial single token backdoor poisoning 1% of the evaluator training data triples the adversary’s score with respect to their legitimate score. We systematically categorize levels of data access corresponding to three real-world settings, (1) web poisoning, (2) malicious annotator, and (3) weight poisoning. These regimes reflect a weak to strong escalation of data access that highly correlates with attack severity. Under the weakest assumptions - web poisoning (1), the adversary still induces a 20% score inflation. Likewise, in the (3) weight poisoning regime, the stronger assumptions enable the adversary to inflate their scores from 1.5/5 to 4.9/5. The backdoor threat generalizes across different evaluator architectures, trigger designs, evaluation tasks, and poisoning rates. By poisoning 10% of the evaluator training data, we control toxicity judges (Guardrails) to misclassify toxic prompts as non-toxic 89% of the time, and document reranker judges in RAG to rank the poisoned document first 97% of the time. LLM-as-a-Judge is uniquely positioned at the intersection of ethics and technology, where social implications of mislead model selection and evaluation constrain the available defensive tools. Amidst these challenges, model merging emerges as a principled tool to offset the backdoor, reducing ASR to near 0% whilst maintaining SOTA performance. Model merging’s low computational cost and convenient integration into the current LLM Judge training pipeline position it as a promising avenue for backdoor mitigation in the LLM-as-a-Judge setting.

   @inproceedings{tong2025badjudge,
     title = {BadJudge: Backdoor Vulnerabilities of {LLM}-As-A-Judge},
     author = {Tong, Terry and Wang, Fei and Zhao, Zhe and Chen, Muhao},
     booktitle = {The Thirteenth International Conference on Learning Representations},
     year = {2025},
     url = {https://openreview.net/forum?id=eC2a2IndIt},
   }

2. Unraveling Indirect In-Context Learning Using Influence Functions

   Hadi Askari, Shivanshu Gupta, Terry Tong, and 3 more authors

   2025

2024

1. 

   Securing Multi-turn Conversational Language Models From Distributed Backdoor Attacks

   Terry Tong, Qin Liu, Jiashu Xu, and 1 more author

   In Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024

   Abs DOI

   Large language models (LLMs) have acquired the ability to handle longer context lengths and understand nuances in text, expanding their dialogue capabilities beyond a single utterance. A popular user-facing application of LLMs is the multi-turn chat setting. Though longer chat memory and better understanding may seemingly benefit users, our paper exposes a vulnerability that leverages the multi-turn feature and strong learning ability of LLMs to harm the end-user: the backdoor. We demonstrate that LLMs can capture the combinational backdoor representation. Only upon presentation of triggers together does the backdoor activate. We also verify empirically that this representation is invariant to the position of the trigger utterance. Subsequently, inserting a single extra token into any two utterances of 5% of the data can cause over 99% Attack Success Rate (ASR). Our results with 3 triggers demonstrate that this framework is generalizable, compatible with any trigger in an adversary‘s toolbox in a plug-and-play manner. Defending the backdoor can be challenging in the conversational setting because of the large input and output space. Our analysis indicates that the distributed backdoor exacerbates the current challenges by polynomially increasing the dimension of the attacked input space. Canonical textual defenses like ONION and BKI leverage auxiliary model forward passes over individual tokens, scaling exponentially with the input sequence length and struggling to maintain computational feasibility. To this end, we propose a decoding time defense – decayed contrastive decoding – that scales linearly with the assistant response sequence length and reduces the backdoor to as low as 0.35%.

2. 

   Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges

   Qin Liu, Wenjie Mo, Terry Tong, and 4 more authors

   In 2024 60th Annual Allerton Conference on Communication, Control, and Computing, Nov 2024

   DOI