Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold

Abstract

Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. A key challenge of OOD detection is the overconfidence of neural models. In this paper, we comprehensively analyze overconfidence and classify it into two perspectives: over-confident OOD and in-domain (IND). Then according to intrinsic reasons, we respectively propose a novel reassigned contrastive learning (RCL) to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents for over-confident IND. Experiments and analyses show the effectiveness of our proposed method for both aspects of overconfidence issues.

Publication
NAACL 2022
Yanan Wu
Yanan Wu
Postgraduate Student

Spoken Language Understading

Keqing He
Postgraduate Student

Dialogue System, Summarization, Pre-training Language Model

Yuanmeng Yan
Yuanmeng Yan
Postgraduate Student

Spoken Language Understanding, Pre-training Language Model

Qixiang Gao
Qixiang Gao
Postgraduate Student

Task-oriented Dialogue System, Dialogue State Tracking

Zhiyuan Zeng
Zhiyuan Zeng
Postgraduate Student

Spoken Language Understanding, Text Generation

Fujia Zheng
Fujia Zheng
Postgraduate Student

Dialogue Summarization

Lulu Zhao
Lulu Zhao
Ph.D Student

Abstractive Dialogue Summarization, Relation Extraction

Weiran Xu
Weiran Xu
Associate Professor, Master Supervisor, Ph.D Supervisor

Information Retrieval, Pattern Recognition, Machine Learning