When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs

Tong Zhang; Zexin Li; Simin Chen; Yun Peng

2026

When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs

Tong Zhang, Zexin Li, Simin Chen, Yun Peng^†

Jailbreak defenses can improve safety while changing how a model performs, refuses benign requests, and consumes inference time. This study examines those trade-offs by grouping defenses according to their operational strategy.

Overview figure for When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs — Overview from the paper.

Three deployment questions

How does a defense affect downstream task performance?
When does caution turn into unnecessary refusal?
How much extra computation does the defense require?

The paper compares these effects across defense strategies to help evaluate safety alongside utility and efficiency.

Experimental results

Explore the comparisons, measurements, and examples reported in the manuscript. Each view links to its source page.

Unsafe-query acceptance across models and defenses. Manuscript · page 6. Select the image to enlarge.

BibTeX

Download citation

@misc{zhang2026llmdefense,
  title = {When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs},
  author = {Zhang, Tong and Li, Zexin and Chen, Simin and Peng, Yun},
  year = {2026},
  eprint = {2607.24392},
  archivePrefix = {arXiv},
  primaryClass = {cs.CR},
  url = {https://arxiv.org/abs/2607.24392}
}