ACL 2023 Papers — Page 8
Annual Meeting of the Association for Computational Linguistics · 1074 papers
NormBank: A Knowledge Bank of Situational Social Norms
Caleb Ziems (Stanford University), Diyi Yang (Meta AI)
ClassificationGenerationTransformerLarge Language ModelText
🎯 What it does: Built NORMBANK, a knowledge base containing 155k contextual norms, and designed the SCENE hierarchical framework to organize environments, roles, attributes, and behavioral constraints;
NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models
Kai Mei (Rutgers University), Shiqing Ma (CISPA Helmholtz Center for Information Security)
Adversarial AttackTransformerPrompt EngineeringText
🎯 What it does: Proposes a transferable backdoor attack called NOTABLE, achieving high success rates in attacking prompt-based NLP models across arbitrary tasks and prompt strategies.
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
Shengming Yin (University Of Science And Technology Of China), Nan Duan (Microsoft Research Asia)
GenerationConvolutional Neural NetworkTransformerVision Language ModelDiffusion modelAuto EncoderVideoText
🎯 What it does: Developed the NUWA-XL Diffusion over Diffusion architecture, achieving ultra-long video generation and training/evaluation on the FlintstonesHD dataset.
OD-RTE: A One-Stage Object Detection Framework for Relational Triple Extraction
Jinzhong Ning (Dalian University of Technology), Hongfei Lin (Dalian University of Technology)
Object DetectionTransformerTextBenchmark
🎯 What it does: Proposed a one-stage framework OD-RTE that treats the relation triplet extraction task as an object detection task, using four vertices to locate triplet regions and employing a Bi-D-W decoding algorithm to extract all types of triplets.
On “Scientific Debt” in NLP: A Case for More Rigour in Language Model Pre-Training Research
Made Nindyatama Nityasya (Independent Researcher), Adhiguna Kuncoro (DeepMind)
ClassificationComputational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningTextReview/Survey PaperBenchmark
🎯 What it does: Conduct comparative experiments between BERT and early pre-trained models ELMo and GPT-1 under identical experimental conditions to analyze the specific contributions of Masked Language Modeling (MLM) to performance;
On Complementarity Objectives for Hybrid Retrieval
Dohyeon Lee (Seoul National University), Sunghyun Park (LG AI Research)
RetrievalTransformerContrastive LearningText
🎯 What it does: Study and improve hybrid retrieval, propose RoC metric and two-level orthogonal constraints to enhance complementarity between sparse and dense retrieval.
On Evaluating Multilingual Compositional Generalization with Translated Datasets
Zi Wang (University of Copenhagen), Daniel Hershcovich (University of Copenhagen)
TransformerTextBenchmark
🎯 What it does: Constructed MCWQ-R, a multilingual knowledge base question answering benchmark with preserved compositional generalization integrity using rule-based machine translation (RBMT) based on synchronous context-free grammar, and compared it with the original MCWQ that used Google Translate.
On Improving Summarization Factual Consistency from Natural Language Feedback
Yixin Liu (Yale University), Ahmed Hassan Awadallah
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose and implement three text generation tasks based on natural language feedback: edit summaries, generate feedback, and simultaneously generate feedback and edits, focusing on improving the factual consistency of summaries.
On Prefix-tuning for Lightweight Out-of-distribution Detection
Yawen Ouyang (Nanjing University), Xinyu Dai (Nanjing University)
Anomaly DetectionComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a lightweight unsupervised prefix-tuning framework called PTO for detecting out-of-distribution (OOD) text, and further extend it to leverage labels and target OOD data;
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Omar Shaikh (Stanford University), Diyi Yang (Stanford University)
Safty and PrivacyPrompt EngineeringTextChain-of-Thought
🎯 What it does: Systematically evaluate the impact of zero-shot chain-of-thought (CoT) on bias and harmful issues.
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation
Tianxing He (University of Washington), Yulia Tsvetkov (University of Washington)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Designed and executed a set of synthetic data-based stress tests to systematically evaluate the robustness and blind spots of text generation evaluation metrics, particularly those based on pre-trained language models.
On the Compositional Generalization in Versatile Open-domain Dialogue
Tingchen Fu (Renmin University of China), Rui Yan (Tencent AI Lab)
GenerationRecurrent Neural NetworkTransformerLarge Language ModelReinforcement LearningMixture of ExpertsText
🎯 What it does: Proposed a sparse-activation modular network, using reverse Polish notation code generated by programmers to realize multi-skill dialogue generation.
On the Efficacy of Sampling Adapters
Clara Meister (ETH Zuerich), Ryan Cotterell (ETH Zuerich)
GenerationTransformerLarge Language ModelText
🎯 What it does: The paper introduces and unifies the concept of 'sampling adapters,' explaining their impact on the distribution of language generation models, and uses a precision–recall framework to explain why these methods enhance the quality of generated text.
On the Evaluation of Neural Selective Prediction Methods for Natural Language Processing
Zhengyao Gu (Reed College), Mark Hopkins (Williams College)
ClassificationTransformerTextReview/Survey PaperBenchmark
🎯 What it does: This paper presents a systematic review and empirical comparison of selective prediction methods in neural networks for natural language processing tasks, proposing a new evaluation metric called refinement and providing a reproducible Python toolkit;
On the Interpretability and Significance of Bias Metrics in Texts: a PMI-based Approach
Francisco Valentini (Instituto de Investigación en Ciencias de la Computación, CONICET-UBA), Edgar Altszyler (Instituto de Investigación en Ciencias de la Computación, CONICET-UBA)
Explainability and InterpretabilityText
🎯 What it does: This paper proposes and evaluates a text bias measurement method based on PMI;
On-the-fly Cross-lingual Masking for Multilingual Pre-training
Xi Ai (Chongqing University), Bin Fang (Chongqing University)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Propose CLPM (Cross-Lingual Prototype Masking), a method dynamically generating cross-lingual prototype masks per token during multilingual MLM pre-training, replacing traditional [MASK], and constructing cross-lingual prototypes through multi-candidate weighted averaging.
One Cannot Stand for Everyone! Leveraging Multiple User Simulators to train Task-oriented Dialogue Systems
Yajiao Liu (Chinese University of Hong Kong), Benyou Wang (Huawei)
Large Language ModelReinforcement LearningText
🎯 What it does: This paper proposes the MUST framework, which enhances the robustness of task-oriented dialogue systems under diverse user behaviors by simultaneously training with multiple user simulators.
One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning
Guangtao Zeng (Singapore University of Technology and Design), Wei Lu (Singapore University of Technology and Design)
Computational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: Propose the PROPETL method, which achieves parameter-efficient transfer learning by sharing a single prototype PETL module and learning binary masks to carve out subnetworks across different layers and tasks.
Open Set Relation Extraction via Unknown-Aware Training
Jun Zhao (Fudan University), Xuanjing Huang (International Human Phenome Institutes)
ClassificationData SynthesisAnomaly DetectionTransformerText
🎯 What it does: Proposed an unknown-aware training method that enhances the detection capability of unknown relations in open-set relation extraction by dynamically synthesizing more challenging negative samples.
Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification
Sha Li (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
TransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Utilizing large language models through incremental prompting and validation to automatically construct an open-domain event knowledge graph that supports hierarchical and temporal relationships.
Open-ended Long Text Generation via Masked Language Modeling
Xiaobo Liang (Soochow University), Min Zhang (Soochow University)
GenerationTransformerLarge Language ModelText
🎯 What it does: Leverage the pre-trained masked language model (RoBERTa) combined with iterative non-autoregressive decoding to address open-ended long-text generation tasks.
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment
Xize Cheng (Zhejiang University), Zhou Zhao (Zhejiang University)
RecognitionTransformerPrompt EngineeringContrastive LearningMultimodalityAudio
🎯 What it does: This paper proposes the OpenSR training system, which utilizes phoneme space alignment pre-trained from unlabeled multi-modal audio-visual data in high-resource domains. It trains a decoder directly applicable to visual or audio-visual recognition using only target domain annotated audio data, achieving zero-shot speech recognition.
Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation
Nuno M. Guerreiro (Instituto de Telecomunicações), André Martins
Anomaly DetectionOptimizationTransformerText
🎯 What it does: This paper proposes a fully unsupervised NMT hallucination detection method that measures the difference between the source word attention distribution from cross-attention and a well-translated distribution using optimal transport (Wasserstein distance), thereby determining whether hallucinations exist.
ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning
Wenjun Hou (Hong Kong Polytechnic University), Jiang Liu (Southern University of Science and Technology)
GenerationTransformerVision Language ModelTextBiomedical DataChain-of-Thought
🎯 What it does: Proposed an observation-guided radiology report generation framework called ORGAN, which first generates an observation plan using a Transformer, and then utilizes an observation graph and tree reasoning mechanism to generate complete reports based on multi-level relationships between observations and vocabulary in the graph.
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts
Yunshui Li (Shenzhen Institute of Advanced Technology Chinese Academy of Sciences), Yongbin Li (DAMO Academy Alibaba Group)
GenerationRetrievalRepresentation LearningTransformerMixture of ExpertsVision Language ModelMultimodality
🎯 What it does: Propose the PaCE framework, decomposing multimodal dialogue into five experts: CAPTION, CONTEXT, IMAGE, GROUNDING, and GENERATION, achieving unified multimodal dialogue pre-training through progressive pre-training.
PAD-Net: An Efficient Framework for Dynamic Networks
Shwai He (University of Maryland), Dacheng Tao (University of Sydney)
Computational EfficiencyConvolutional Neural NetworkTransformerMixture of ExpertsImageText
🎯 What it does: Propose a Partial Dynamic Network (PAD-Net) framework that converts redundant dynamic parameters in dynamic networks into static parameters to reduce deployment costs and improve performance.
PAED: Zero-Shot Persona Attribute Extraction in Dialogues
Luyao Zhu (Nanyang Technological University), Erik Cambria (Nanyang Technological University)
ClassificationGenerationTransformerPrompt EngineeringAuto EncoderContrastive LearningTextSequential
🎯 What it does: This paper studies Person Attribute Extraction in Dialogues (PAED), constructs a high-quality PersonaExt dataset, and proposes a generative zero-shot learning framework based on Meta-VAE hard negative sampling and contrastive structured constraints.
PairSpanBERT: An Enhanced Language Model for Bridging Resolution
Hideo Kobayashi, Vincent Ng (Human Language Technology Research Institute, University of Texas at Dallas)
Representation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Designed and implemented a new pre-trained model called PAIRSPANBERT, which incorporates a bridging context learning objective into SPANBERT, achieving state-of-the-art (SOTA) results on three major bridging coreference datasets.
PAL to Lend a Helping Hand: Towards Building an Emotion Adaptive Polite and Empathetic Counseling Conversational Agent
Kshitij Mishra (Indian Institute of Technology Patna), Asif Ekbal (Indian Institute of Technology Patna)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Developed an emotionally adaptive, polite, and empathetic counseling dialogue system called PAL, aiming to provide a more engaging and friendly interactive experience for online psychological counseling.
ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation
Kuan-Hao Huang (University of California, Los Angeles), Aram Galstyan (Amazon Alexa AI)
Data SynthesisTransformerLarge Language ModelTextGraph
🎯 What it does: Constructed a large-scale, syntactically diverse paraphrase dataset called PARAAMR, generating syntactically distinct but semantically similar paraphrases from English sentences using AMR back-translation techniques.
Parallel Context Windows for Large Language Models
Nir Ratner (AI21 Labs), Yoav Shoham (AI21 Labs)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Proposes the Parallel Context Windows (PCW) method, which extends the context length of large language models using a multi-window parallel approach without requiring retraining.
ParaLS: Lexical Substitution via Pretrained Paraphraser
Jipeng Qiang (Yangzhou University), Yi Zhu (Yangzhou University)
GenerationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Generate synonym replacement candidates that preserve word meaning using a pre-trained paraphraser and filter them through specific decoding strategies.
Parameter-Efficient Fine-Tuning without Introducing New Latency
Baohao Liao (University of Amsterdam), Christof Monz (University of Amsterdam)
Computational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes two parameter-efficient fine-tuning (PEFT) methods: PaFi (which utilizes the amplitude of pre-trained parameters to select sparse masks) and HiWi (which directly applies adapters to pre-trained weights/bias).
Parameter-efficient Weight Ensembling Facilitates Task-level Knowledge Transfer
Xingtai Lv (Tsinghua University), Maosong Sun (Tsinghua University)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: This paper proposes a parameter-efficient weight combination framework that leverages existing lightweight models (such as Adapter, LoRA, Prefix) for knowledge transfer on new tasks. It first estimates the similarity between each old lightweight module and the new task using multiple metrics (Loss, KL-divergence, Logits/Labels cosine similarity, EL2N, GraNd), then obtains the initialized lightweight module for the new task by performing a softmax-weighted average of the modules.
Patton: Language Model Pretraining on Text-Rich Networks
Bowen Jin (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
ClassificationRetrievalRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Proposes the PATTON framework for self-supervised pre-training of language models on text-rich networks, integrating textual and network structural information;
PeaCoK: Persona Commonsense Knowledge for Consistent and Engaging Narratives
Silin Gao (EPFL), Antoine Bosselut (EPFL)
GenerationData SynthesisTransformerLarge Language ModelTextGraphRetrieval-Augmented Generation
🎯 What it does: Constructed a large-scale personality commonsense knowledge graph called PEACOK, systematically representing the five dimensions of personality (traits, habits, goals, experiences, relationships) and providing approximately 100,000 high-quality personality inference facts.
Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering
Avi Caciularu (Bar-Ilan University), Arman Cohan (Allen Institute for AI)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose a cross-document QA pre-training method called QAMDEN, enabling language models to first generate answers and then regenerate the original sentences in a multi-document context;
Peeking inside the black box: A Commonsense-aware Generative Framework for Explainable Complaint Detection
Apoorva Singh (IIT Patna), Sriparna Saha (IIT Patna)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposed an explainable complaint detection framework and the X-CI dataset, converting multi-task prediction (complaint, severity, emotion, polarity, causal reason) into a text-to-text generation problem.
Peer-Label Assisted Hierarchical Text Classification
Junru Song, Yang Yang (Chinese Academy of Military Science)
ClassificationRecurrent Neural NetworkGraph Neural NetworkTransformerText
🎯 What it does: Explored the potential correlation between peer labels in hierarchical text classification and leveraged peer label collaboration to enhance classification performance.
PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks
Weiwen Xu (Chinese University of Hong Kong), Lidong Bing (DAMO Academy, Alibaba Group)
RecognitionData SynthesisTransformerContrastive LearningText
🎯 What it does: Propose the PeerDA method, which uses peer relations (span pairs of the same category) for data augmentation to enhance the performance of the SpanID (Span Identification) task.
PEIT: Bridging the Modality Gap with Pre-trained Models for End-to-End Image Translation
Shaolin Zhu (Tianjin University), Deyi Xiong (Tianjin University)
Image TranslationDomain AdaptationKnowledge DistillationRepresentation LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the PEIT end-to-end image translation framework, which leverages visual encoders, shared encoder-decoders, vision-text aligners, and cross-modal regularizers to address the modality gap between image-text pairs and text translation, and constructs a large-scale e-commerce image translation dataset called ECOIT;
Personality Understanding of Fictional Characters during Book Reading
Mo Yu (WeChat AI), Jie Zhou (WeChat AI)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Constructed the PERSONET dataset, utilizing online reading notes instead of complete book texts, to study the fine-grained personality prediction task based on historical context during reading.
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
Yau-Shian Wang (Carnegie Mellon University), Yiming Yang (Carnegie Mellon University)
ClassificationTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Propose PESCO, a zero-shot text classification framework that enhances self-contrastive learning through prompting, leveraging self-training loops with unlabeled text to improve performance.
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning
Kyuyong Shin (NAVER), Sang-Woo Lee (NAVER AI Lab)
Recommendation SystemRepresentation LearningTransformerLarge Language ModelTextSequential
🎯 What it does: This paper proposes a unified user modeling framework called LMRec, which jointly trains a language model (LM) and recommendation task objectives on user behavior sequences, directly learning richer user and item representations from historical user text.
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Lei Wang (Singapore Management University), Ee-Peng Lim (Singapore Management University)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Investigated zero-shot chain-of-thought prompting and proposed the Plan-and-Solve (PS) prompting method;
PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English
Jianfeng Chi (Meta AI), Kai-Wei Chang (University of California Los Angeles)
Domain AdaptationSafty and PrivacyTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed PLUE, a multi-task evaluation benchmark covering six privacy policy language understanding tasks including text classification, question answering, semantic parsing, and named entity recognition, and collected a 332M-word privacy policy corpus for domain-specific pre-training.
Plug-and-Play Document Modules for Pre-trained Models
Chaojun Xiao (Tsinghua University), Maosong Sun (Huawei Technologies Co Ltd)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose PlugD, which encodes documents as pluggable modules (document plugins), decoupling document encoding from downstream tasks, allowing multi-task reuse after a single encoding.
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Zhengyan Zhang (Tsinghua University), Jie Zhou (Tencent Inc)
ClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Propose a new plug-and-play knowledge injection paradigm that directly injects external knowledge bases into frozen pre-trained language models without retraining downstream models;
PMAES: Prompt-mapping Contrastive Learning for Cross-prompt Automated Essay Scoring
Yuan Chen (Guangdong University of Foreign Studies), Xia Li (Guangdong University of Foreign Studies)
ClassificationConvolutional Neural NetworkRecurrent Neural NetworkPrompt EngineeringContrastive LearningText
🎯 What it does: This paper proposes a cross-prompt automatic essay scoring (AES) method called PMAES, which enhances scoring performance by using prompt mapping contrastive learning to make representations of source prompts and target prompts more consistent.
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA
Neeraj Varshney (Arizona State University), Chitta Baral (Arizona State University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Explored the Post-Abstention task, allowing the model to reattempt questions initially abandoned to improve coverage without significantly reducing accuracy.
Pre-trained Language Models Can be Fully Zero-Shot Learners
Xuandong Zhao (University of California Santa Barbara), Lei Li (University of California Santa Barbara)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a method called NPPrompt, which utilizes pre-trained language models to complete text understanding tasks in a completely zero-shot setting.
Pre-training Multi-party Dialogue Models with Latent Discourse Inference
Yiyang Li (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
GenerationRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Pre-train a dialogue model with better discourse structure awareness by using two-stage unsupervised reasoning (EM and VI) to infer implicit argumentation structures (response relationships) in multi-party dialogues.
Pre-Training to Learn in Context
Yuxian Gu (Tsinghua University), Minlie Huang (Microsoft Research)
Representation LearningMeta LearningTransformerLarge Language ModelContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: This study proposes the PICL framework, which utilizes naturally occurring 'intrinsic tasks' from large-scale text for pre-training to enhance the in-context learning ability of language models.
Precise Zero-Shot Dense Retrieval without Relevance Labels
Luyu Gao (Carnegie Mellon University), Jamie Callan (Carnegie Mellon University)
RetrievalTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Proposes a zero-shot dense retrieval framework that generates hypothetical documents using generative language models (Hypothetical Document Embeddings, HyDE) and embeds them with an unsupervised contrastive learning encoder, enabling direct retrieval without requiring relevance labels;
Prefix Propagation: Parameter-Efficient Tuning for Long Sequences
Jonathan Li (Queen's University), Xiaodan Zhu (Queen's University)
Computational EfficiencyTransformerPrompt EngineeringText
🎯 What it does: Proposes an improved parameter-efficient fine-tuning method called Prefix-Propagation, addressing the performance bottleneck of traditional Prefix-Tuning in long-text tasks, and conducts an in-depth analysis of its calibration performance and kernel attention.
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference
Junhao Zheng (South China University of Technology), Haibin Chen (South China University of Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a fine-tuning method based on causal inference called Causal Effect Tuning (CET), which recovers pre-trained knowledge through KNN-weighted joint prediction, avoiding catastrophic forgetting and negative transfer.
Pretrained Bidirectional Distillation for Machine Translation
Yimeng Zhuang (Samsung Research China - Beijing), Mei Tu (Samsung Research China - Beijing)
GenerationKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Propose the pre-trained bidirectional distillation (PBD) method, which distills the bidirectional language knowledge of masked language models into NMT encoders and decoders in one go during training;
Privacy-Preserving Domain Adaptation of Semantic Parsers
Fatemehsadat Mireshghallah (University of California, San Diego), Richard Shin (Microsoft Semantic Machines)
Domain AdaptationSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: In task-oriented dialogue systems, synthetic user dialogue semantic parsing data is generated using differential privacy (DP) techniques, which is used to enhance the performance of low-resource semantic parsers while ensuring user privacy and diagnosing system defects.
Probing Physical Reasoning with Counter-Commonsense Context
Kazushi Kondo (University of Tokyo), Akiko Aizawa (National Institute of Informatics)
TransformerLarge Language ModelPrompt EngineeringTextPhysics Related
🎯 What it does: Constructed the CConS (Counter-Commonsense Contextual Size Comparison) dataset to evaluate language models' ability to infer object size relationships in contexts that align with or contradict physical common sense.
Product Question Answering in E-Commerce: A Survey
Yang Deng, Wai Lam (JD.com)
RetrievalGraph Neural NetworkTransformerMixture of ExpertsTextMultimodalityReview/Survey PaperRetrieval-Augmented Generation
🎯 What it does: A systematic review of product question answering (PQA) in the e-commerce domain, covering four mainstream question settings (opinion-based, extractive, retrieval-based, and generative), summarizing related methods, datasets, evaluation metrics, and identifying key challenges and future research directions.
Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases
Yingji Li, Ying Wang (Jilin University)
Representation LearningTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Propose a two-stage adversarial debiasing framework named CCPA, which first uses continuous prompt tuning to increase the representation distance between sentence pairs with the same attribute, then employs contrastive learning to bring them closer, thereby reducing social bias in pre-trained language models.
Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation
Taha Aksu (National University of Singapore), Nancy Chen (National University of Singapore)
Domain AdaptationTransformerPrompt EngineeringText
🎯 What it does: This paper proposes the Prompter method, achieving DST model adaptation in unsupervised zero-shot domain adaptation scenarios through dynamic prefixes;
Prompting Language Models for Linguistic Structure
Terra Blevins (University of Washington), Luke Zettlemoyer (University of Washington)
ClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a structured prompting method, leveraging autoregressive pre-trained language models to accomplish sequence labeling tasks (part-of-speech tagging, named entity recognition, and sentence chunking) without additional training.
Prompting PaLM for Translation: Assessing Strategies and Performance
David Vilar (Google), George Foster (Google)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Systematically evaluate PaLM's capabilities in machine translation, explore different example selection strategies and prompt templates, and compare with the latest SOTA and Google Translate across multiple language pairs.
PromptNER: Prompt Locating and Typing for Named Entity Recognition
Yongliang Shen (Zhejiang University), Yueting Zhuang (Zhejiang University)
RecognitionTransformerPrompt EngineeringText
🎯 What it does: Propose the PromptNER framework, which uniformly uses a dual-slot (position + type) prompt template to identify named entities in text in one go;
PromptRank: Unsupervised Keyphrase Extraction Using Prompt
Aobo Kong (Nankai University), Xiaoyan Bai (Lenovo Research)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose an unsupervised keyword extraction method called PromptRank, which ranks keywords by calculating the generation probability of candidate words using a pre-trained language model (Encoder-Decoder architecture).
Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning
Zujie Liang (MYbank, Ant Group), Bing Han (MYbank, Ant Group)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose Lottery Prompt Tuning (LPT), a lifelong information extraction framework that performs task-specific sparse prompt tuning on pre-trained language models, maintaining performance on old tasks while enabling knowledge transfer as new tasks are continuously received.
Prototype-Guided Pseudo Labeling for Semi-Supervised Text Classification
Weiyi Yang (Beihang University), Jaein Kim (Beihang University)
ClassificationTransformerContrastive LearningText
🎯 What it does: In semi-supervised text classification tasks, a Prototype-Guided Pseudo Labeling (PGPL) framework is proposed, combining two modules: Prototype-Anchored Contrasting (PAC) and Prototype-Guided Pseudo-Labeling (PGP), to address issues of underfitting decision boundaries and pseudo-label bias caused by class imbalance.
Pruning Pre-trained Language Models Without Fine-Tuning
Ting Jiang, Feng Xia (Tencent)
Computational EfficiencyKnowledge DistillationTransformerText
🎯 What it does: This paper proposes a technique that can compress pre-trained language models (PLM) to high sparsity rates through a single static pruning (SMP) without requiring fine-tuning.
PuMer: Pruning and Merging Tokens for Efficient Vision Language Models
Qingqing Cao (University of Washington), Hannaneh Hajishirzi (University of Washington)
Computational EfficiencyKnowledge DistillationTransformerVision Language ModelMultimodality
🎯 What it does: Proposes a general-purpose token compression framework named PuMer, which prunes and merges image and text tokens layer by layer in cross-modal Transformers, significantly improving inference speed and memory usage.
PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism
Yongkang Liu (Northeastern University), Hinrich Schütze (Northeastern University)
GenerationRecurrent Neural NetworkText
🎯 What it does: Proposed a PVGRU and PVHD structure based on a pseudo variational mechanism to enhance the diversity and relevance of multi-turn dialogue generation.
Python Code Generation by Asking Clarification Questions
Haau-Sing (Xiaocheng) Li, Iryna Gurevych (Technische Universität Darmstadt)
GenerationData SynthesisAI Code AssistantGraph Neural NetworkTransformerLarge Language ModelText
🎯 What it does: Propose a method to enhance Python code generation by asking clarifying questions, construct the CodeClarQA dataset, and implement an interactive generation pipeline
Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling
Mingzhu Cai (Baidu Inc.), Hua Wu (Baidu Inc.)
GenerationRetrievalData-Centric LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposes an unsupervised query enhancement method called QKConv, which automatically generates queries and generates responses based on external knowledge through joint training in knowledge-intensive dialogues.
Query Refinement Prompts for Closed-Book Long-Form QA
Reinald Kim Amplayo (Google DeepMind), Shashi Narayan (Google DeepMind)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Studies how to generate multi-level long-form answers using large language models (LLMs) in a closed-book setting, proposing query refinement prompting to first clarify the multifaceted nature of questions before generating coherent long-form answers.
Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs
Siyuan Wang (Fudan University), Xuanjing Huang (Fudan University)
Representation LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextGraph
🎯 What it does: A transferable structured text encoding framework (SILR) was constructed, which encodes the linearized structure of logical queries through pre-trained language models (PLMs), and achieves inductive logic reasoning on knowledge graphs by employing structural prompts and geometric operation modeling.
Query-Efficient Black-Box Red Teaming via Bayesian Optimization
Deokjae Lee (Seoul National University), Hyun Oh Song (Seoul National University)
OptimizationSafty and PrivacyAdversarial AttackHyperparameter SearchTransformerImageText
🎯 What it does: For red team testing against black-box generative models, this paper proposes BRT, a query-efficient method based on Bayesian optimization, to efficiently discover diverse adversarial examples within a limited query budget.
QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations
Chaitanya Malaviya (University of Pennsylvania), Kristina Toutanova (Google DeepMind)
RetrievalTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper constructs the QUEST dataset, specifically designed to evaluate the ability of retrieval systems to handle entity retrieval queries involving implicit set operations (intersection, union, difference), and experimentally assesses the performance of modern retrieval models on this task.
Question-Answering in a Low-resourced Language: Benchmark Dataset and Models for Tigrinya
Fitsum Gaim (Korea Advanced Institute of Science and Technology), Jong Park (Korea Advanced Institute of Science and Technology)
TransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed the first Tigrinya (Tigrinya) question-answering dataset TiQuAD, and conducted various experiments (monolingual, cross-lingual, cross-language, and multilingual), demonstrating the feasibility of question-answering tasks on low-resource languages.
RADE: Reference-Assisted Dialogue Evaluation for Open-Domain Dialogue
Zhengliang Shi (Shandong University), Zhaochun Ren (Shandong University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a reference-assisted dialogue evaluation method called RADE, which compares pre-generated reference responses with generated responses and enhances evaluation effectiveness through multi-task learning combined with a generation task.
RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation
Gabriele Sarti (University of Groningen), Maria Nadejde (AWS AI Labs)
GenerationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposed a Retrieval and Attribute Annotation Enhanced Prompting (RAMP) method to achieve zero/few-shot attribute-controlled translation (ACT)
Randomized Positional Encodings Boost Length Generalization of Transformers
Anian Ruoss (DeepMind), Joel Veness (DeepMind)
TransformerSequentialBenchmark
🎯 What it does: Propose a randomized positional encoding scheme to enhance the generalization of Transformers on sequences with unseen lengths.
Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications
Han Cheol Moon (Nanyang Technological University), Chi Xu
ClassificationAdversarial AttackTransformerSupervised Fine-TuningText
🎯 What it does: Propose a two-stage RSMI framework, first obtaining robust hidden representations through randomized smoothing, then suppressing adversarial perturbations via gradient-guided mask inference to enhance adversarial robustness in text classification.
RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank
Jiduan Liu (Peking University), Rui Yan (Peking University)
Representation LearningTransformerContrastive LearningText
🎯 What it does: Propose an unsupervised sentence representation learning method called RankCSE, which jointly learns finer-grained semantic ranking information through contrastive learning, ranking consistency, and ranking distillation.
Ranking-Enhanced Unsupervised Sentence Representation Learning
Yeon Seonwoo (KAIST), Alice Oh (KAIST)
Representation LearningContrastive LearningText
🎯 What it does: This paper proposes an unsupervised sentence encoder called RankEncoder, which captures semantic information by calculating the relative ranking (rank vector) of a sentence against all sentences in a large corpus, and further retrains the sentence encoder based on this to enhance the semantic similarity prediction capability of sentence representations.
RARR: Researching and Revising What Language Models Say, Using Language Models
Luyu Gao (Carnegie Mellon University), Kelvin Guu (Google Research)
GenerationRetrievalExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the RARR (Retrofit Attribution using Research and Revision) framework, which performs post-retrieval and revision on outputs of any text generation model to generate attributable text while preserving the original style and structure.
RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction
Jun Zhao (Fudan University), Mingming Sun (Baidu Research)
ClassificationAdversarial AttackTransformerSupervised Fine-TuningText
🎯 What it does: Proposed a fine-grained semantic matching method for zero-shot relation extraction, decomposing sentence matching into entity matching and context matching, and reducing interference from redundant information through context feature distillation.
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises
Chenglei Si (Tsinghua University), Maosong Sun (Tsinghua University)
TransformerLarge Language ModelTextBenchmarkAudio
🎯 What it does: Developed the READIN Chinese multi-task noise benchmark, collecting real user noise from keyboard pinyin input and voice input, covering four tasks (synonym recognition, reading comprehension, SQL semantic parsing, and machine translation)
Reanalyzing L2 Preposition Learning with Bayesian Mixed Effects and a Pretrained Language Model
Jakob Prange (Hong Kong Polytechnic University), Man Ho Ivy Wong (Hong Kong Shue Yan University)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Replicate and conduct in-depth analysis of Wong (2022)'s data on Mandarin/Cantonese learners' English preposition test performance, investigating interactions between learner ability, task type, and sentence stimuli.
Reasoning Implicit Sentiment with Chain-of-Thought Prompting
Hao Fei (National University Of Singapore), Tat-Seng Chua (National University Of Singapore)
ClassificationTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Propose the Three-Hop Reasoning (THOR) Chained Thinking Framework for Implicit Sentiment Analysis;
Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering
Jiajie Zhang (Tsinghua University), Qi Tian (Tsinghua University)
Explainability and InterpretabilityTransformerTextGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Construct a hierarchical question decomposition tree (HQDT) and aggregate knowledge graph and text information through recursive probabilistic reasoning, forming an interpretable multi-source question-answering framework RoHT;
Reasoning with Language Model Prompting: A Survey
Shuofei Qiao (Zhejiang University), Huajun Chen (National University Of Singapore)
TransformerLarge Language ModelTextReview/Survey PaperBenchmarkChain-of-Thought
🎯 What it does: Reviews reasoning methods under language model prompts, categorizes them into strategy enhancement and knowledge enhancement, and provides method classification, comparison, and resource organization.
ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models
Jianyi Zhang (Duke University), Yiran Chen (Duke University)
RetrievalKnowledge DistillationRepresentation LearningTransformerTextRetrieval-Augmented Generation
🎯 What it does: Propose a framework called ReAugKD that integrates a retrieval-enhanced mechanism into the knowledge distillation process. It leverages the teacher model's soft labels and embeddings to construct an external non-parametric memory bank. During student model inference, similar teacher labels are retrieved via kNN and fused to improve the student model's generalization performance.
Recall, Expand, and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing
Chengyue Jiang (ShanghaiTech University), Kewei Tu (ShanghaiTech University)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Propose a three-stage framework named Recall-Expand-Filter, which leverages a Multi-Candidate Cross-Encoder (MCCE) to parallelly encode and filter ultra-fine-grained entity types in a single forward pass;
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation
Shuai Liu (University of Southern California), Jonathan May (University of Southern California)
GenerationRetrievalTransformerTextSequentialRetrieval-Augmented Generation
🎯 What it does: This paper proposes a retrieval-enhanced context-aware prefix encoder (RECAP) for generating personalized dialogue responses.
ReCode: Robustness Evaluation of Code Generation Models
Shiqi Wang (AWS AI Labs), Bing Xiang (AWS AI Labs)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the ReCode benchmark, defining over 30 natural document, function name, syntax, and format perturbations, and evaluate the robustness of code generation models using worst-case metrics.
RED^{\textrm{FM}}: a Filtered and Multilingual Relation Extraction Dataset
Pere-Lluís Huguet Cabot, Roberto Navigli (Sapienza University of Rome)
ClassificationTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes two large-scale multilingual relation extraction datasets—SRED FM (gold-standard, 18 languages, 400 relations, 44M triplets) and RED FM (human-verified, 7 languages, 32 relations, ~1M triplets)—and trains the first multilingual end-to-end relation extraction model, mREBEL, based on these datasets;
Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework
Mingqi Gao (Peking University), Baoxing Huai (Huawei Cloud)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper constructs a human-corrected factual error correction dataset containing 4,000 dialog summaries and proposes a fine-grained, reference-based error correction evaluation framework called FERRANTI to accurately measure the performance of fact error correction (FEC) models in dialog summarization.
Rehearsal-free Continual Language Learning via Efficient Parameter Isolation
Zhicheng Wang (East China Normal University), Wenqiu Zeng (Huatai Securities)
Computational EfficiencyRepresentation LearningTransformerPrompt EngineeringText
🎯 What it does: Propose a continual language learning method based on parameter isolation without using historical task data;
Resolving Ambiguities in Text-to-Image Generative Models
Ninareh Mehrabi (Amazon Alexa AI-NU), Rahul Gupta (Amazon Alexa AI-NU)
GenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelMultimodalityBenchmark
🎯 What it does: This paper investigates the ambiguity problem in text-to-image generation models, constructing a TAB benchmark dataset containing various types of ambiguity, and proposes a pluggable TIED framework. TIED generates clarifying questions or visual scene descriptions via language models, leverages user interaction to resolve ambiguity, inputs the disambiguated prompts into text-to-image models, and evaluates improvements in image realism and fairness.
Resolving Indirect Referring Expressions for Entity Selection
Mohammad Javad Hosseini (Google Research), Annie Louis (Google Research)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper constructs a new dataset called AltEntities to study the entity selection problem of indirect references in dialogues;