Current unlearning methods fail at the practical goal of removing harmful applications of a concept while preserving safe ones; effective unlearning requires concept-level evaluation, not just fact-level testing.
This paper introduces ConceptGuard, a benchmark for evaluating how well LLMs can selectively forget harmful knowledge while keeping beneficial uses of the same concept.