ACL 2023 Papers — Page 3
Annual Meeting of the Association for Computational Linguistics · 1074 papers
Continual Knowledge Distillation for Neural Machine Translation
Yuanchi Zhang (Tsinghua University), Yang Liu (Tsinghua University)
Knowledge DistillationTransformerText
🎯 What it does: Propose a continuous knowledge distillation (CKD) framework that gradually distills knowledge from a series of frozen teacher models to a single student NMT model, thereby improving translation performance.
ContraCLM: Contrastive Learning For Causal Language Model
Nihal Jain (AWS AI Labs), Bing Xiang (AWS AI Labs)
Representation LearningTransformerContrastive LearningText
🎯 What it does: Designed and implemented CONTRACLM, a framework that simultaneously employs token-level and sequence-level contrastive learning in causal language models to enhance model representation resolution and multi-task performance.
Contrastive Bootstrapping for Label Refinement
Shudi Hou (Peking University), Sujian Li (Peking University)
ClassificationTransformerContrastive LearningText
🎯 What it does: This study addresses the coarse-to-fine hierarchical text classification task, proposing a lightweight contrastive clustering bootstrapping method for iterative refinement of document labels.
Contrastive Decoding: Open-ended Text Generation as Optimization
Xiang Lisa Li (Stanford University), Mike Lewis (FAIR)
GenerationOptimizationTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose a contrastive decoding (Contrastive Decoding) method that generates high-quality, coherent, and diverse text by leveraging the log-likelihood difference between large models (experts) and small models (amateurs);
Contrastive Error Attribution for Finetuned Language Models
Faisal Ladhak (Columbia University), Tatsunori Hashimoto (Stanford University)
Explainability and InterpretabilityKnowledge DistillationData-Centric LearningTransformerContrastive LearningText
🎯 What it does: This paper proposes a novel contrastive error attribution framework (Contrastive Error Attribution, CEA) to identify and remove low-quality training samples that cause unreliable outputs (such as hallucinations and semantic errors in text summarization) in natural language generation models.
Contrastive Learning with Adversarial Examples for Alleviating Pathology of Language Model
Pengwei Zhan (Chinese Academy of Sciences), Liming Wang (Chinese Academy of Sciences)
ClassificationExplainability and InterpretabilityRepresentation LearningAdversarial AttackConvolutional Neural NetworkRecurrent Neural NetworkTransformerContrastive LearningText
🎯 What it does: By analyzing the causes of bias in sentence representations, we propose a contrastive learning regularization method called ConAAP based on adversarial examples to calibrate the sentence representations of discrete distribution samples, thereby alleviating the overconfidence pathology in language models.
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models
Albert Xu (University of Southern California), Robin Jia (University of Southern California)
ClassificationAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed a method that leverages large language models (LLMs) to generate novel out-of-distribution (OOD) samples, and reduces the model's confidence on these samples through contrastive confidence loss, thereby enhancing open-set selective classification performance.
Controllable Mixed-Initiative Dialogue Generation through Prompting
Maximillian Chen (Columbia University), Zhou Yu (Columbia University)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose using prompt methods with large language models as an alternative to traditional fine-tuning to control hybrid dialogue generation.
Controllable Text Generation via Probability Density Estimation in the Latent Space
Yuxuan Gu (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
GenerationFlow-based ModelAuto EncoderText
🎯 What it does: This paper proposes a reversible transformation framework that uses probability density estimation in the latent space to achieve controllable generation of text attributes;
Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers
Parikshit Bansal (Microsoft Research), Amit Sharma (Microsoft Research)
ClassificationRepresentation LearningTransformerText
🎯 What it does: Proposed the Feature Effect Augmentation (FEAG) algorithm, which uses causal effect estimation to regulate the influence of suspicious features on text classifiers, thereby reducing spurious correlations;
Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuning
Mustafa Ozdayi, Rahul Gupta (Amazon)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: The study controls the extractable memory data in large language models (LLMs) through prompt-tuning.
Convergence and Diversity in the Control Hierarchy
Alexandra Butoi (ETH Zürich), David Chiang (University of Notre Dame)
🎯 What it does: This paper systematically defines and analyzes four two-level forms of L2 language classes (CFG⊲CFG, CFG⊲PDA, PDA⊲CFG, PDA⊲PDA) within the framework of control hierarchy theory. It proves their equivalence relationships with known forms (TAG, LIG, EPDA) through new equivalence concepts (d-weak, d-strong), while also introducing and proving the existence of a new form called PAA.
ConvGQR: Generative Query Reformulation for Conversational Search
Fengran Mo (University of Montreal), Jian-Yun Nie (University of Montreal)
GenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose the ConvGQR framework, which leverages a generative pre-trained language model to simultaneously perform query rewriting and potential answer generation, thereby enhancing conversational retrieval effectiveness.
CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection
Chongyang Tao (Microsoft), Daxin Jiang (Microsoft)
RetrievalTransformerLarge Language ModelContrastive LearningText
🎯 What it does: This paper proposes a collaborative training framework (CORE), which simultaneously optimizes the retriever (bi-encoder) and re-ranker (cross-encoder) in two-stage dialogue response selection (retrieval and re-ranking), enabling mutual learning between the two modules;
Counterfactual Active Learning for Out-of-Distribution Generalization
Xun Deng (University of Science and Technology of China), Yong Liao (China Academic of Electronics and Information Technology)
Domain AdaptationData-Centric LearningTransformerText
🎯 What it does: Proposed a Counterfactual Active Learning (CounterAL) framework that combines active learning with counterfactual sample construction to enhance model generalization on out-of-distribution (OOD) tasks.
Counterfactual Debiasing for Fact Verification
Weizhi Xu (Center for Research on Intelligent Perception and Computing, State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences), Liang Wang (School of Artificial Intelligence, University of Chinese Academy of Sciences)
ClassificationTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a counterfactual debiasing method called CLEVER based on adversarial fact verification, which utilizes an independently trained 'subjective' and 'evidence' fusion model to eliminate misjudgments caused by biases in claims through counterfactual inference (i.e., subtracting the output based solely on the claim).
Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoning
Wangzhen Guo (Sun Yat-Sen University), Hanjiang Lai (Sun Yat-Sen University)
TransformerTextChain-of-Thought
🎯 What it does: Proposed a counterfactual multi-hop question answering method based on causal inference, aiming to reduce disconnection reasoning (i.e., providing correct answers using only a single fact) in multi-hop QA and enhance genuine multi-hop reasoning capabilities.
Counterfactual reasoning: Testing language models’ understanding of hypothetical scenarios
Jiaxuan Li (University of California Irvine), Allyson Ettinger (University of Chicago)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper evaluates the reasoning ability of pre-trained language models in factual and counterfactual scenarios through controlled experiments and large-scale synthetic data;
Counterspeeches up my sleeve! Intent Distribution Learning and Persistent Fusion for Intent-Conditioned Counterspeech Generation
Rishabh Gupta (IIIT Delhi), Md. Shad Akhtar
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderText
🎯 What it does: This paper proposes an intent-based counter-hate speech generation task, develops the IntentCONAN multi-intent counter-speech corpus, and constructs the QUARC two-stage model to achieve intent-conditioned counter-generation.
Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment
Roni Rabin (Google Research), Amir Globerson (Google Research)
GenerationData SynthesisTransformerLarge Language ModelText
🎯 What it does: Proposed and implemented a method for automatically generating gap-focused questions (GFQ) that highlight information gaps, applicable to teacher-student dialogues and other information gap scenarios; defined the GFQ task and its design principles, and evaluated it on the SNLI dataset.
Credible without Credit: Domain Experts Assess Generative Language Models
Denis Peskoff (Princeton University), Brandon Stewart (Princeton University)
GenerationTransformerLarge Language ModelText
🎯 What it does: In the paper, the authors invited 10 interdisciplinary experts to subjectively evaluate the answers generated by ChatGPT and YouChat on 100 questions designed by the experts themselves. Experts rated the answers on coherence, conciseness, accuracy, sourcing, and quality relative to Wikipedia using a 5-point scale, and provided open-ended feedback. The authors then statistically analyzed and compared the models' performance across these metrics, exploring their usability in professional and general knowledge queries.
CREPE: Open-Domain Question Answering with False Presuppositions
Xinyan Yu (University of Washington), Hannaneh Hajishirzi (University of Washington)
ClassificationGenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed and publicly released the CREPE dataset to investigate the identification and correction of false presuppositions in open-domain question answering.
CREST: A Joint Framework for Rationalization and Counterfactual Text Generation
Marcos Treviso (Instituto de Telecomunicações), André Martins
GenerationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Propose the CREST framework, combining sparse rationalization (Selective Rationalization) with counterfactual text generation (Counterfactual Generation), achieving dual improvements in explainability and robustness.
Cross-Domain Data Augmentation with Domain-Adaptive Language Modeling for Aspect-Based Sentiment Analysis
Jianfei Yu (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
ClassificationGenerationData SynthesisDomain AdaptationRecurrent Neural NetworkTransformerLarge Language ModelText
🎯 What it does: Proposes a three-stage cross-domain data augmentation framework DA LM, which first uses source domain labeled data and target domain unlabeled data to generate pseudo-labels, then trains a domain-adaptive language model (DALM) to jointly generate words and BIO labels, and finally generates a large amount of fluent and diverse target domain labeled data in an autoregressive manner for training cross-domain aspect-based sentiment analysis (ABSA) and aspect extraction (AE) models.
Cross-lingual Continual Learning
Meryem M’hamdi, Jonathan May (Information Sciences Institute University of Southern California)
Knowledge DistillationTransformerTextBenchmark
🎯 What it does: Propose a cross-lingual continual learning (CCL) evaluation paradigm and systematically evaluate various continual learning methods on the multilingual task-oriented dialogue (MTOP) benchmark;
Cross-lingual Science Journalism: Select, Simplify and Rewrite Summaries for Non-expert Readers
Mehwish Fatima (Heidelberg Institute for Theoretical Studies), Michael Strube (Heidelberg Institute for Theoretical Studies)
GenerationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Propose the cross-lingual scientific journalism (CSJ) task and design a three-stage pipeline SSR (SELECT → SIMPLIFY → REWRITE) to generate localized language (German) popular science abstracts for non-expert readers from English scientific texts.
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
Rohan Pandey (Carnegie Mellon University), Louis-Philippe Morency (Carnegie Mellon University)
RetrievalComputational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: Propose Cross-modal Attention Congruence Regularization (CACR), which enhances the performance of vision-language models in compositional reasoning tasks by encouraging conjugate alignment between visual and language attention matrices.
Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning
Shivaen Ramshetty (Georgia Institute of Technology), Srijan Kumar (Georgia Institute of Technology)
Adversarial AttackTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a cross-modal attribute insertion (XMAI) method that inserts visually detected attributes (such as color, size, shape) from images into corresponding text to evaluate the robustness of multi-modal models against real text perturbations.
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
Yan Zeng (ByteDance), Xinsong Zhang (ByteDance)
RetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Propose the Cross-View Language Modeling (CCLM) framework, unifying cross-lingual and cross-modal pre-training into a shared Transformer architecture with a unified objective, and train the CCLM model.
Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment
Shengqiong Wu (National University of Singapore), Tat-Seng Chua (National University of Singapore)
GenerationGraph Neural NetworkTransformerVision Language ModelContrastive LearningImageTextGraph
🎯 What it does: Propose a two-stage model for unpaired cross-lingual image captioning tasks, which utilizes scene graphs (SG) and syntactic constituent trees (SC) to guide semantic and syntactic structures, achieving global alignment through structural alignment and cross-modal cross-lingual back-translation.
Crosslingual Generalization through Multitask Finetuning
Niklas Muennighoff (Hugging Face), Colin Raffel (Hugging Face)
Representation LearningData-Centric LearningTransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: By performing multi-task fine-tuning (MTF) on the multilingual pre-trained models BLOOM and mT5, and constructing the xP3 and its machine-translated version xP3mt datasets, the study investigates cross-lingual zero-shot task generalization.
CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs
Abhik Bhattacharjee (Bangladesh University of Engineering and Technology), Rifat Shahriyar (Bangladesh University of Engineering and Technology)
GenerationData SynthesisTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Constructed CrossSum, a multilingual summarization dataset with 1.68 million samples covering over 1500 language pairs, not centered on English.
CTC-based Non-autoregressive Speech Translation
Chen Xu (Northeastern University), Jingbo Zhu (Northeastern University)
TextAudio
🎯 What it does: Proposes a non-autoregressive speech translation model based on CTC (CTC-NAST), which uses a dual-encoder architecture to simultaneously predict the source text and target text.
Curriculum Learning for Graph Neural Networks: A Multiview Competence-based Approach
Nidhi Vakil (University of Massachusetts Lowell), Hadi Amiri (University of Massachusetts Lowell)
Computational EfficiencyGraph Neural NetworkGraph
🎯 What it does: Propose a multi-perspective capability-based curriculum learning framework (MCCL), which arranges and prioritizes training samples by combining multiple graph complexity metrics with the model's capability dynamics during training;
DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue
William Held (Georgia Institute of Technology), Rushin Shah
Domain AdaptationRepresentation LearningAdversarial AttackTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: Designed and implemented the DAMP (Doubly Aligned Multilingual Parser) model, leveraging a dual alignment strategy (alignment pre-training and alignment fine-tuning) to significantly enhance zero-shot transfer performance for multi-semantic parsing tasks in multilingual and code-switching scenarios.
DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation
Menglong Lu (National University of Defense Technology), Dongsheng Li (National University of Defense Technology)
Domain AdaptationMeta LearningGraph Neural NetworkTransformerText
🎯 What it does: Propose a self-training based domain adaptation framework called DaMSTF, which integrates meta-learning reweighting, a meta-constructor generating high-quality meta-validation sets, and domain adversarial learning to avoid training guidance disappearance;
DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications
Adam Ivankay (IBM Research), Pascal Frossard (École Polytechnique Fédérale de Lausanne)
ClassificationDomain AdaptationExplainability and InterpretabilityTransformerLarge Language ModelTextBiomedical Data
🎯 What it does: Investigate the robustness of explanations in medical text classification, proposing a domain-adaptive explanation robustness estimator called DARE, and introducing adversarial training and FAR training on three major medical datasets to enhance explanation robustness.
DarkBERT: A Language Model for the Dark Side of the Internet
Youngjin Jin (KAIST), Seungwon Shin (KAIST)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed and pre-trained a language model specifically for dark web English text called DarkBERT, and evaluated it on multiple dark web-related tasks (activity classification, ransomware leak site detection, forum thread detection, keyword reasoning).
Data Curation Alone Can Stabilize In-context Learning
Ting-Yun Chang (University of Southern California), Robin Jia (University of Southern California)
ClassificationData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Studies how to improve the stability and accuracy of large language models in parameter-free few-shot learning by selecting subsets of training data.
DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions
Vijay Viswanathan (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
RetrievalRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed a dataset recommendation task based on natural language descriptions and constructed the DataFinder dataset to support training and evaluation.
Dataset Distillation with Attention Labels for Fine-tuning BERT
Aru Maekawa (Tokyo Institute of Technology), Manabu Okumura (Tokyo Institute of Technology)
ClassificationKnowledge DistillationTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a dataset distillation method for fine-tuning pre-trained Transformers (BERT), with the core idea of introducing attention labels and soft labels, enabling the training of models with performance close to the original dataset under extremely few samples (only one per class) and minimal gradient steps (one step).
Dating Greek Papyri with Text Regression
John Pavlopoulos (Athens University of Economics and Business), Asimina Paparigopoulou (Ca'Foscari University of Venice)
Data-Centric LearningText
🎯 What it does: Constructed and made public a dataset of 389 transcribed Greek documentary papyri texts, and used text regression methods to predict their dates;
Dealing with Semantic Underspecification in Multimodal NLP
Sandro Pezzelle (University of Amsterdam)
TransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: Explore the performance of multimodal NLP models under semantic underdetermined conditions, with two experiments (semantically underdetermined rewriting of COCO captions and evaluation of CLIP scores).
Debiasing Generative Named Entity Recognition by Calibrating Sequence Likelihood
Yu Xia (Peking University), Sujian Li (Peking University)
RecognitionGenerationTransformerContrastive LearningText
🎯 What it does: A generative named entity recognition model based on Seq2Seq, which uses re-ranking techniques to correct the likelihood distribution of candidate sequences, reducing bias during training.
Decoder Tuning: Efficient Language Understanding as Decoding
Ganqu Cui (Tsinghua University), Maosong Sun (Tsinghua University)
ClassificationTransformerPrompt EngineeringText
🎯 What it does: In the Model-as-a-Service (MaaS) scenario, the Decoder Tuning (DecT) method is proposed, which constructs a tunable decoder at the output of the pre-trained model (PTM) to achieve efficient adaptation to downstream tasks.
Decoding Symbolism in Language Models
Meiqi Guo (University of Pittsburgh), Adriana Kovashka (University of Pittsburgh)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper constructs an evaluation framework called SymbA to systematically evaluate and compare the performance of different language models in the task of decoding symbolic meanings, proposes and verifies a debiasing method based on re-ranking, and demonstrates that large-scale models can achieve or even surpass human-level performance on this task.
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering
Pei Ke (Tsinghua University), Minlie Huang (Tsinghua University)
GenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose DecompEval, which converts NLG evaluation into an instruction-based question-answering task and leverages instruction-tuned pre-trained language models to perform unsupervised evaluation of generated text without requiring training.
Decomposed scoring of CCG dependencies
Aditya Bhargava (University of Toronto), Gerald Penn (University of Toronto)
Explainability and InterpretabilityText
🎯 What it does: This paper proposes a decomposed dependency scoring method based on subcategory labels and alignment to more accurately evaluate errors in CCG parsers;
DecompX: Explaining Transformers Decisions by Propagating Token Decomposition
Ali Modarressi (Center for Information and Language Processing, LMU Munich), Mohammad Taher Pilehvar (Tehran Institute for Advanced Studies, Khatam University)
Explainability and InterpretabilityTransformerText
🎯 What it does: Propose the DecompX method, which decomposes token representations at each layer of the Transformer and recursively propagates them until the classification head, obtaining vectorized explanations for the positive and negative impacts of each input token on each output class.
Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery
Yutao Mou (Beijing University of Posts and Telecommunications), Weiran Xu (Meituan)
ClassificationRepresentation LearningTransformerContrastive LearningTextBenchmark
🎯 What it does: Propose a Decoupled Prototype Learning (DPL) framework that leverages prototype contrastive learning to obtain clustering-friendly representations, and further enhances the performance of Generalized Intent Discovery by using prototype labels to disambiguate.
Deep Active Learning for Morphophonological Processing
Seyed Morteza Mirbostani (University of Guilan), Owen Rambow (Stony Brook University)
Data-Centric LearningRecurrent Neural NetworkTransformerText
🎯 What it does: Proposes a deep active learning method for morphophonological processing.
Deep Model Compression Also Helps Models Capture Ambiguity
Hancheol Park (Korea Advanced Institute of Science and Technology), Jong Park (Korea Advanced Institute of Science and Technology)
ClassificationCompressionKnowledge DistillationTransformerText
🎯 What it does: Deeply compress RoBERTa using layer pruning and knowledge distillation, improving the model's probability distribution estimation for text with fuzzy labels without increasing additional annotation costs.
Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion
Shaoxiang Wu (Peking University), Zhifang Sui (Tencent Cloud AI)
TransformerContrastive LearningVideoTextMultimodality
🎯 What it does: Propose a denoising bottleneck fusion (DBF) model that removes redundancy and noise in video multimodal data while preserving key information through restricted receptive field bottleneck embeddings and mutual information maximization (MI-Max), achieving higher quality cross-modal fusion;
Dense-ATOMIC: Towards Densely-connected ATOMIC with High Knowledge Coverage and Massive Multi-hop Paths
Xiangqing Shen (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
Representation LearningTransformerTextGraph
🎯 What it does: Construct Dense-ATOMIC by utilizing a relation prediction method to complete multiple missing edges in ATOMIC, significantly enhancing knowledge coverage and the number of multi-hop paths.
Dependency resolution at the syntax-semantics interface: psycholinguistic and computational insights on control dependencies
Iria de-Dios-Flores (Universidade de Santiago de Compostela), Marcos Garcia (Universidade de Santiago de Compostela)
RecognitionTransformerLarge Language ModelText
🎯 What it does: This study compares the ability of humans and pre-trained masked language models to identify dependencies in control structures in Spanish and Galician through psycholinguistic and computational experiments.
DEplain: A German Parallel Corpus with Intralingual Translations into Plain Language for Sentence and Document Simplification
Regina Stodden (Heinrich Heine University Düsseldorf), Laura Kallmeyer (Heinrich Heine University Düsseldorf)
Data SynthesisTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed the DEPLAIN dataset, containing sentence-level and document-level simplification alignments in German text, and provides both manual and automatic alignment methods;
Deriving Language Models from Masked Language Models
Lucas Torroba Hennigen (Massachusetts Institute of Technology), Yoon Kim (Massachusetts Institute of Technology)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Propose a method to derive explicit joint distributions from masked language models and realize this derivation in the two-word scenario;
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better
David Dale (Meta AI), Marta R. Costa-jussà (Meta AI)
GenerationAnomaly DetectionExplainability and InterpretabilityData-Centric LearningTransformerText
🎯 What it does: The study proposes using the internal ALTI+ method of Transformers to evaluate the contribution of source sentences in translation, aiming to detect and correct hallucinations in machine translation.
Detecting Contradictory COVID-19 Drug Efficacy Claims from Biomedical Literature
Daniel Sosa, Russ Altman
ClassificationTransformerSupervised Fine-TuningBiomedical Data
🎯 What it does: This paper proposes a model based on natural language inference (NLI) to automatically identify contradictory statements in studies on the efficacy of drugs for COVID-19.
Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts
Skyler Hallinan (University of Washington), Maarten Sap (Allen Institute for AI)
GenerationTransformerSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: This paper proposes MARCO, a detoxification method that utilizes expert and anti-expert language models for controlled text revision.
Dialect-robust Evaluation of Generated Text
Jiao Sun (Google Deepmind), Sebastian Gehrmann (Google Deepmind)
GenerationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the robustness of text generation evaluation metrics to dialect differences and proposes a pre-training method called NANO to enhance the dialect robustness and dialect awareness of the metrics.
Dialog-Post: Multi-Level Self-Supervised Objectives and Hierarchical Model for Dialogue Post-Training
Zhenyu Zhang (JD AI Research), Xiaodong He (JD AI Research)
ClassificationRecognitionRepresentation LearningTransformerContrastive LearningText
🎯 What it does: Propose DIALOG-POST, which employs five multi-level self-supervised objectives and a hierarchical segment self-attention network to post-train dialogues, enhancing dialogue representation and understanding performance.
DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations
Ang Lv (Renmin University of China), Rui Yan (Renmin University of China)
Data SynthesisTransformerContrastive LearningTextStochastic Differential Equation
🎯 What it does: This paper proposes a multi-turn dialogue data augmentation method called DialoGPS based on continuous semantic space, generating diverse and coherent enhanced dialogues by sampling dialogue paths using an extended Brownian bridge;
Dialogue Summarization with Static-Dynamic Structure Fusion Graph
Shen Gao (Shandong University), Rui Yan (Ministry of Education)
GenerationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelText
🎯 What it does: Propose a dialogue summarization model SDDS that integrates static dialogue structure graphs and dynamic semantic graphs, capable of adaptively capturing dialogue information flow during summary generation.
DICE: Data-Efficient Clinical Event Extraction with Generative Models
Mingyu Derek Ma (University of California Los Angeles), Nanyun Peng (University of California Los Angeles)
Data-Centric LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningBiomedical DataElectronic Health Records
🎯 What it does: Proposed the DICE model, which employs generative methods for data-efficient processing of clinical event extraction, and integrates techniques such as mention recognition, contrastive learning, and special tokens to improve mention boundary detection;
Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction
Haotian Chen (Fudan University), Xiangdong Zhou (Fudan University)
Explainability and InterpretabilityGraph Neural NetworkTransformerTextBenchmark
🎯 What it does: This paper first creates the DocREDHWE dataset, adding human-annotated word-level evidence to relation facts in DocRED; subsequently, it uses a feature attribution method (Integrated Gradients) to analyze the decision rules of SOTA models in DocRE, discovering that models mainly rely on non-causal vocabulary (e.g., entity names, periods), and introduces the MAP metric to evaluate the similarity between model decisions and human decision rules; then, it verifies model robustness through various word-level attacks (masking, synonym/antonym substitution, entity name masking/shuffling/substitution).
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
Fan Yin (UCLA), Chien-Sheng Wu
CompressionComputational EfficiencyData-Centric LearningTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Systematically study the role of task definitions in instruction learning, explore the model's understanding of task descriptions, and evaluate which information is most critical for performance;
DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generation
Guanqun Bi (Chinese Academy of Sciences), Xiaodong He (JD AI Research)
GenerationTransformerLarge Language ModelDiffusion modelTextRetrieval-Augmented Generation
🎯 What it does: For empathetic response generation in open-domain dialogue, the DIFFUSEMP framework is proposed, which utilizes multi-grained control signals (communication mechanism, intent, semantic framework) to guide diffusion language models in generating empathetic and diverse responses.
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
Zhengfu He (Fudan University), Xipeng Qiu (Fudan University)
GenerationTransformerLarge Language ModelDiffusion modelText
🎯 What it does: Integrate pre-trained BERT with discrete absorption diffusion models to construct DiffusionBERT for non-autoregressive text generation
DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models
Zijie J. Wang (Georgia Tech), Duen Horng Chau (Georgia Tech)
Data SynthesisPrompt EngineeringDiffusion modelImageText
🎯 What it does: This paper creates and publicly releases DIFFUSIONDB—the first large-scale text-to-image prompt database, containing 14 million Stable Diffusion-generated images, 1.8 million unique prompts, and corresponding hyperparameters;
DiffusionNER: Boundary Diffusion for Named Entity Recognition
Yongliang Shen (Zhejiang University), Yueting Zhuang (Zhejiang University)
RecognitionGenerationRecurrent Neural NetworkTransformerDiffusion modelText
🎯 What it does: Propose DIFFUSIONNER, reformulating the Named Entity Recognition (NER) task as a boundary denoising diffusion process, progressively recovering entity boundaries from noisy spans and generating entities.
DimonGen: Diversified Generative Commonsense Reasoning for Explaining Concept Relationships
Chenzhengyi Liu (University of Illinois at Urbana-Champaign), Kevin Chen-Chuan Chang (University of Illinois at Urbana-Champaign)
GenerationRetrievalTransformerLarge Language ModelMixture of ExpertsTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the DimonGen task, which requires generating diverse relational description sentences for given concept pairs; and design the MoREE two-stage retrieval-enhanced model to accomplish this task.
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization
Yu Li (Columbia University), Jianfeng Gao (Microsoft Research)
GenerationTransformerLarge Language ModelText
🎯 What it does: Proposes DIONYSUS, a pre-trained Seq2Seq model capable of summarizing multi-domain dialogues in zero-shot and few-shot scenarios;
DIP: Dead code Insertion based Black-box Attack for Programming Language Model
CheolWon Na (Sungkyunkwan University), Jee-Hyong Lee (Sungkyunkwan University)
Adversarial AttackTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed a black-box attack method called DIP based on dead code insertion, which uses inserted non-executable dead code to mislead large-scale pre-trained programming language models while ensuring successful compilation and semantic invariance.
Direct Fact Retrieval from Knowledge Graphs without Entity Linking
Jinheon Baek (KAIST), Sung Ju Hwang (KAIST)
RetrievalTransformerContrastive LearningGraph
🎯 What it does: Propose a framework (DiFaR) for directly retrieving facts from knowledge graphs by mapping queries and triples into the same dense vector space, training only on query-triple pairs without requiring steps such as entity recognition, disambiguation, or relation classification.
DISCO: Distilling Counterfactuals with Large Language Models
Zeming Chen (EPFL), Kyle Richardson (Allen Institute for AI)
Data SynthesisTransformerLarge Language ModelText
🎯 What it does: Propose the DISCO framework, which generates and filters high-quality counterfactual data using large language models for model training
DiSCoMaT: Distantly Supervised Composition Extraction from Tables in Materials Science Articles
Tanishq Gupta (Indian Institute of Technology Delhi), Mausam (Indian Institute of Technology Delhi)
Graph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextTabularPhysics Related
🎯 What it does: Propose the task of extracting material composition from tables in materials science papers and implement the baseline system DISCOMAT;
Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels
Yuchen Eleanor Jiang (ETH Zürich), Ryan Cotterell (ETH Zürich)
Data-Centric LearningTextBenchmark
🎯 What it does: Constructed a Chinese-English novel parallel corpus BWB-test with four layers of annotations including entities, terminology, coreference, and citations, used to evaluate the discourse quality of document-level machine translation.
Discourse-Level Representations can Improve Prediction of Degree of Anxiety
Swanie Juhng (Stony Brook University), H. Andrew Schwartz (Stony Brook University)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: The study uses language features at the discourse and lexical levels to predict the anxiety levels of Facebook users.
Discriminative Reasoning with Sparse Event Representation for Document-level Event-Event Relation Extraction
Changsen Yuan (Beijing Institute of Technology), Yonggang Wen (Nanyang Technological University)
Representation LearningRecurrent Neural NetworkTransformerText
🎯 What it does: Propose a document-level event relation extraction model called SENDIR that does not require prior knowledge and is based on sparse attention and distinguishing intra-sentence and inter-sentence reasoning
Disentangled Phonetic Representation for Chinese Spelling Correction
Zihong Liang (Sun Yat-sen University), Qifan Wang (Meta AI)
ClassificationKnowledge DistillationTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a Chinese spelling error correction model called DORM, which significantly improves the utilization of pinyin information by decoupling text and pinyin representations and enabling bidirectional interaction within the Transformer.
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
Ella Neeman (Hebrew University of Jerusalem), Omri Abend (Hebrew University of Jerusalem)
GenerationRetrievalExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Train a single generative QA model to simultaneously provide context-based answers and model-parameter-based answers in one inference, achieving decoupling of knowledge sources.
DisorBERT: A Double Domain Adaptation Model for Detecting Signs of Mental Disorders in Social Media
Mario Ezra Aragón (Universidade de Santiago de Compostela), Manuel Montes-y-Gómez (Universidade de Santiago de Compostela)
ClassificationDomain AdaptationTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a dual-domain adaptation model called DisorBERT for detecting signs of eating disorders, depression, and self-harm from social media posts.
Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis
Ta-Chung Chi (Carnegie Mellon University), Peter Ramadge
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper systematically analyzes the length extrapolation capabilities of ALiBi and window attention using an observable cumulative normalized gradient tool, and proposes a new parameter-free relative position encoding called Sandwich, demonstrating its superiority on long sequences.
Distantly Supervised Course Concept Extraction in MOOCs with Academic Discipline
Mengying Lu (Tsinghua University), Juanzi Li (Tsinghua University)
RecognitionRepresentation LearningTransformerSupervised Fine-TuningPrompt EngineeringVideoTextBenchmark
🎯 What it does: Propose a three-stage discrete supervised framework DS-MOCE to automatically extract course concepts from MOOC video subtitles, primarily addressing the issues of noise and incomplete labels caused by limited dictionaries and course diversity.
Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models
Junmo Kang (Georgia Institute of Technology), Alan Ritter (Georgia Institute of Technology)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies cost strategies for building efficient small models under a fixed budget, comparing two approaches: ① using human-annotated data to directly fine-tune a small model (Ann), ② first fine-tuning a large model (T5-XXL) with limited annotated data, then distilling knowledge to a small model (Dist).
Distilling Script Knowledge from Large Language Models for Constrained Language Planning
Siyu Yuan (Fudan University), Deqing Yang (Fudan University)
GenerationData SynthesisKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposed a language planning task with constraints for specific goals, and constructed the CoScript dataset containing 55,000 scripts;
Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations
Dongyu Ru (Amazon AWS AI), Zheng Zhang (Amazon AWS AI)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: A Distributed Marker Representation (DMR) was constructed, utilizing latent semantic-aware distributed markers to capture discourse relationships between sentences, and unsupervised training was performed on large-scale marked data using the EM algorithm.
DITTO: Data-efficient and Fair Targeted Subset Selection for ASR Accent Adaptation
Suraj Kothawade, Preethi Jyothi (University of Texas at Dallas)
RecognitionDomain AdaptationConvolutional Neural NetworkAudio
🎯 What it does: Propose the DITTO method, which utilizes a submodular mutual information (SMI) function to select a subset of unlabeled speech that best represents the target accent from massive unlabeled data under a limited annotation budget, to achieve ASR accent adaptation.
Diverse Demonstrations Improve In-context Compositional Generalization
Itay Levy (Tel-Aviv University), Jonathan Berant (Tel-Aviv University)
Meta LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a method to enhance compositional reasoning generalization in semantic parsing tasks by selecting diverse demonstration examples. The authors design two demonstration selection strategies: Cover-LS for covering local structures and DPP (Determinantal Point Process) based on diversity. These strategies are applied to both parameter-free in-context learning and fine-tuning settings.
Diversity-Aware Coherence Loss for Improving Neural Topic Models
Raymond Li (University of British Columbia), Giuseppe Carenini (University of British Columbia)
OptimizationRepresentation LearningAuto EncoderText
🎯 What it does: Propose a diversity-aware consistency loss that directly incorporates corpus-level consistency metrics (NPMI) and diversity constraints into the training of neural topic models.
Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking
Qingyue Wang (Chinese Academy of Sciences), Li Guo (Chinese Academy of Sciences)
TransformerMixture of ExpertsText
🎯 What it does: Propose a 'split-conquer-combine' zero-shot dialogue state tracking framework, which clusters known data in the semantic space and trains semantic-independent experts, then enhances the model's generalization ability through mixture-of-experts inference.
Do Androids Laugh at Electric Sheep? Humor “Understanding” Benchmarks from The New Yorker Caption Contest
Jack Hessel (Allen Institute For Ai), Yejin Choi (Allen Institute For Ai)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Constructed and released three new benchmark tasks (matching, quality ranking, and explanation) based on The New Yorker's cartoon caption contest, and collected a multimodal dataset containing images, text, and human annotations.
Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023?
Shuheng Liu (Georgia Institute of Technology), Alan Ritter (Georgia Institute of Technology)
RecognitionRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper constructs the CoNLL++ test set based on 2020 Reuters news, evaluates and compares the generalization performance of over 20 NER models trained on CoNLL-2003 on modern data, and deeply analyzes factors affecting generalization.
Do GPTs Produce Less Literal Translations?
Vikas Raunak (Microsoft Azure AI), Hany Hassan
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper systematically studies the differences in literalness between GPT series large language models and traditional neural machine translation, verifying through a combination of quantitative metrics and human evaluation.
Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questions
Mayur Patidar (TCS Research), Mausam (Indian Institute of Technology)
GraphBenchmark
🎯 What it does: In the knowledge graph question answering task, a method for detecting answerability was proposed and evaluated, a benchmark dataset containing unanswerable questions named GrailQAbility was constructed, and the impact of different types of knowledge base incompleteness on models was systematically analyzed.
Do language models have coherent mental models of everyday things?
Yuling Gu (Allen Institute for AI), Peter Clark (Allen Institute for AI)
Explainability and InterpretabilityTransformerLarge Language ModelGraphBenchmark
🎯 What it does: This paper constructs a benchmark dataset named ParRoT, which includes part diagrams of 100 daily objects and their relationships, totaling 11.7K relationships, aiming to evaluate language models' understanding of part relationships of daily objects; by asking True/False questions to SOTA language models such as GPT-3 and Macaw, it tests their part relationship knowledge and consistency; it proposes a post-processing method called ParRoT-Con based on constraint reasoning, which optimizes model outputs using common-sense constraints such as symmetry, asymmetry, reverse, and transitivity.
Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning
Po-Nien Kung (University of California, Los Angeles), Nanyun Peng (University of California, Los Angeles)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied how models utilize instructions during instruction tuning through controlled experiments comparing original instructions, simplified instructions, and misleading examples.
Do PLMs Know and Understand Ontological Knowledge?
Weiqi Wu (ShanghaiTech University), Kewei Tu (Alibaba Group)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Systematically investigate whether pre-trained language models (PLM) can memorize and understand ontological knowledge, and develop corresponding memory and reasoning evaluation tasks.
Do Question Answering Modeling Improvements Hold Across Benchmarks?
Nelson F. Liu (Stanford University), Percy Liang (Stanford University)
Data SynthesisTransformerTextBenchmark
🎯 What it does: Define and quantify the 'concurrence' metric for inter-benchmark ranking consistency, systematically evaluate the performance ranking similarity of 20 QA models across 32 different QA benchmarks, and investigate the impact of different benchmark types (artificial, cloze, synthetic) on consistency.