EMNLP 2023 Papers — Page 3
Conference on Empirical Methods in Natural Language Processing · 1047 papers
Comparing Styles across Languages
Shreya Havaldar (University of Pennsylvania), Lyle Ungar (University of Pennsylvania)
Explainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes an interpretable two-step framework for extracting and comparing stylistic differences across languages from multilingual language models, and first constructs a comprehensive politeness dataset covering four languages (English, Japanese, Spanish, and Chinese).
Composable Text Controls in Latent Space with ODEs
Guangyi Liu (CUHK-Shenzhen), Zhiting Hu (UC San Diego)
GenerationTransformerGenerative Adversarial NetworkTextOrdinary Differential Equation
🎯 What it does: Proposes LATENTOPS, which enables composable text control operations in the text latent space and achieves efficient text generation or editing through ODE sampling.
CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations
Myra Cheng (Stanford University), Diyi Yang (Stanford University)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Proposed the CoMPosT framework to systematically describe and evaluate four dimensions of LLM simulations (Context, Model, Persona, Topic), and designed a characterization method based on persona and topic semantic axes to detect 'caricaturization' tendencies in LLM simulations.
CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models
Benjamin Minixhofer (University of Cambridge), Ivan Vulić (University of Cambridge)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper first extracts a dataset of 255k compound and non-compound words across 56 languages from Wiktionary, and uses this dataset to evaluate the performance of existing large language models (LLMs) on the decompounding task. Subsequently, a two-stage training framework is proposed: the first stage employs a self-supervised hyphen prediction objective, training a byte-level ByT5 model on large-scale unannotated corpora to perform compound word segmentation; the second stage conducts supervised fine-tuning on Wiktionary's annotated compounds and their standardized forms, further achieving compound standardization. The paper also transforms standardized outputs into segmentation results via edit distance optimization and designs a CompoundPiece pre-tokenizer, which pre-segments text with compound words during SentencePiece training, thereby reducing 'hard' compounds (token boundaries misaligned compounds) and enhancing subsequent decompounding performance.
Compressing and Debiasing Vision-Language Pre-Trained Models for Visual Question Answering
Qingyi Si (Chinese Academy of Sciences), Weiping Wang (Chinese Academy of Sciences)
CompressionDomain AdaptationComputational EfficiencyRepresentation LearningTransformerMultimodalityBenchmark
🎯 What it does: Study how to simultaneously compress and debias visual language pretraining models in visual question answering tasks, seeking sparse subnetworks that are robust to outliers.
Compressing Context to Enhance Inference Efficiency of Large Language Models
Yucheng Li (University of Surrey), Chenghua Lin (University of Manchester)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose a method called Selective Context, which calculates self-information to filter and remove redundant parts from the input context, thereby significantly compressing the context while maintaining model performance.
Conceptor-Aided Debiasing of Large Language Models
Li S. Yifei (University of Pennsylvania), João Sedoc (New York University)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes two methods to debias large language models (such as BERT, GPT) using conceptors: post-processing soft projection and continuously trained CI-BERT, demonstrating significant mitigation of gender and racial biases.
Conceptual structure coheres in human cognition but not in large language models
Siddharth Suresh (University of Wisconsin-Madison), Timothy Rogers
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextTabular
🎯 What it does: Investigated the robustness and consistency of human and large language models (LLMs) in concept structure using three psychological tasks (feature enumeration, triad similarity judgment, contrast similarity scoring) on 30 tool and reptile concepts.
Condensing Multilingual Knowledge with Lightweight Language-Specific Modules
Haoran Xu (Johns Hopkins University), Kenton Murray (Johns Hopkins University)
Computational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose two lightweight methods, Language-Specific Matrix Synthesis (LMS) and Fusion Distillation (FD), to reduce parameter count and improve performance in multilingual models
Confidence-based Ensembling of Perspective-aware Models
Silvia Casola, Cristina Bosco (University of Turin)
ClassificationDomain AdaptationTransformerSupervised Fine-TuningText
🎯 What it does: Propose a perspective collection method based on model confidence, conduct experiments on detecting sarcasm and hate speech, decompose perspectives using annotator metadata or automatic clustering, train multiple perspective-aware models separately, and enhance classification performance through confidence-weighted integration or highest confidence voting.
Connecting degree and polarity: An artificial language learning study
Lisa Bylinina (University of Groningen), Ekaterina Garmash (Spotify)
Representation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By introducing new words into a pre-trained BERT model and using an artificial language learning experiment framework, the study investigates the relationship between the degree of adverbs (low/medium/high) and their polarity sensitivity (positive/negative/neutral).
Consistency Analysis of ChatGPT
Myeongjun Jang, Thomas Lukasiewicz (Vienna University of Technology)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Systematically evaluate the performance of ChatGPT and GPT-4 on four types of logical consistency—semantic, consistency, symmetry, and transitivity—and analyze the impact of prompt design, few-shot learning, and model scale on consistency.
Construction Artifacts in Metaphor Identification Datasets
Joanne Boisson (Cardiff University), Jose Camacho-Collados (Cardiff University)
ClassificationHyperparameter SearchData-Centric LearningTransformerSupervised Fine-TuningText
🎯 What it does: Investigate the construction bias in metaphor recognition datasets, demonstrating that models lacking complete context or PME information can still achieve performance comparable to models with complete information.
Content- and Topology-Aware Representation Learning for Scientific Multi-Literature
Kai Zhang (Worcester Polytechnic Institute), Xiaozhong Liu (Worcester Polytechnic Institute)
Representation LearningGraph Neural NetworkTransformerAuto EncoderContrastive LearningTextMultimodalityGraphBiomedical DataBenchmark
🎯 What it does: In a multi-document learning scenario, the SMRC 2 model is proposed, which generates document-level and global multi-document representations by simultaneously learning the semantics (entity graph and abstract text) and topological structure (citation network) of the documents.
Context Compression for Auto-regressive Transformers with Sentinel Tokens
Siyu Ren (Shanghai Jiao Tong University), Kenny Zhu (University of Texas at Arlington)
CompressionComputational EfficiencyTransformerText
🎯 What it does: Proposed a context compression method based on sentinel tokens, which inserts <CL> and <CR> tokens into autoregressive Transformers and modifies the attention mask to compress contiguous token ranges into compact representations, significantly reducing key-value (KV) cache usage and computational complexity.
Contextual Interaction for Argument Post Quality Assessment
Yiran Wang (University of Chinese Academy of Sciences), Le Sun (Chinese Academy of Sciences)
ClassificationGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed and compared two argument quality assessment methods—supervised contrastive learning and example-based LLM prompting, emphasizing the role of contextual interaction between arguments in distinguishing subtle differences.
Continual Dialogue State Tracking via Example-Guided Question Answering
Hyundong Cho (Information Sciences Institute, University of Southern California), Chinnadhurai Sankar (Meta AI)
TransformerPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Reconstruct dialogue state tracking (DST) as a fine-grained example-guided question answering task (DST-EGQA), and enhance the model's transferability and memory capacity in continual learning scenarios by retrieving relevant examples for context prompting.
Continual Event Extraction with Semantic Confusion Rectification
Zitao Wang (Nanjing University), Wei Hu (Nanjing University)
Knowledge DistillationRepresentation LearningTransformerTextBenchmark
🎯 What it does: Proposed a novel continual event extraction model that can avoid forgetting and correct semantic confusion while new event types continuously emerge.
Continual Learning for Multilingual Neural Machine Translation via Dual Importance-based Model Division
Junpeng Liu (Dalian University of Technology), Degen Huang (Dalian University of Technology)
GenerationRepresentation LearningTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a dual importance model partitioning method for continual learning in multilingual neural machine translation (MNMT), which rapidly adapts to new language pairs by first identifying and pruning parameters that are irrelevant to the original task but important for the new task, then expanding the pruned model back to its original size and specifically fine-tuning the newly added parameters, while maintaining performance on the original language pairs.
Continual Named Entity Recognition without Catastrophic Forgetting
Duzhen Zhang (Chinese Academy of Sciences), Zhen Fang (University of Technology Sydney)
RecognitionKnowledge DistillationTransformerText
🎯 What it does: Propose a learning framework for Continuous Named Entity Recognition (CNER) that can incrementally introduce new entity types without retraining, significantly reducing catastrophic forgetting and semantic drift of non-entity types.
Continually Improving Extractive QA via Human Feedback
Ge Gao (Cornell University), Eunsol Choi (University of Texas at Austin)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper studies leveraging user feedback during continuous interaction to continuously improve the performance of extractive question answering systems, proposing a framework of iterative deployment and offline contextual reinforcement learning;
Contrastive Learning for Inference in Dialogue
Etsuko Ishii (Hong Kong University of Science and Technology), Pascale Fung (Hong Kong University of Science and Technology)
TransformerLarge Language ModelContrastive LearningText
🎯 What it does: This paper studies the information gap issue in dialogue reasoning and improves the model's performance in reasoning tasks through contrastive learning methods.
Contrastive Learning of Sentence Embeddings from Scratch
Junlei Zhang (Zhejiang University), Junxian He (Hong Kong University of Science and Technology)
Data SynthesisRepresentation LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Leverage large language models (e.g., ChatGPT) to generate synthetic positive and negative sentence pairs, then train sentence embeddings under a contrastive learning framework.
Controllable Contrastive Generation for Multilingual Biomedical Entity Linking
Tiantian Zhu (Harbin Institute Of Technology), Yang Xiang (Peng Cheng Laboratory)
TransformerPrompt EngineeringContrastive LearningBiomedical Data
🎯 What it does: Propose the Con2GEN framework, which employs a prompt-controlled contrastive generation method to address ambiguity and information missing issues in multilingual biomedical entity linking (MBEL).
Controlling Pre-trained Language Models for Grade-Specific Text Simplification
Sweta Agrawal (University of Maryland), Marine Carpuat (University of Maryland)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper experimentally evaluates the impact of using low-level control markers on output simplification degree and quality in text simplification tasks, and proposes an instance-level control prediction method that predicts control markers based on input text and target reading level to improve text simplification effectiveness.
Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations
Jihyoung Jang (UNIST), Hyounghun Kim (UNIST)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringTextGraphSequential
🎯 What it does: Constructed a 1M-scale multi-session dialogue dataset named CONVERSATION CHRONICLES, covering time intervals ranging from hours to years and ten fine-grained speaker relationships. Based on this dataset, a multi-session generation model called REBOT was trained, incorporating a timeline summarization module and a dialogue generation module that integrate temporal and relational information.
Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction
Cam-Van Thi Nguyen (VNU University of Engineering and Technology), Duc-Trong Le (VNU University of Engineering and Technology)
ClassificationRecognitionGraph Neural NetworkTransformerMultimodality
🎯 What it does: Proposed a multimodal emotion recognition framework named CORECT that simultaneously leverages local temporal relationships and global cross-modal interactions.
Conversational Semantic Parsing using Dynamic Context Graphs
Parag Jain (University of Edinburgh), Mirella Lapata (University of Edinburgh)
Representation LearningGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Propose a semantic parsing model for conversational knowledge graph question answering, which can map user utterances into executable SPARQL queries within the context of conversation history.
Copyright Violations and Large Language Models
Antonia Karamolegkou (University of Copenhagen), Anders Søgaard (University of Copenhagen)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study investigates the copyright infringement risks posed by large language models (LLMs) when replicating copyrighted text, focusing on their ability to memorize and reproduce text verbatim from books and LeetCode problem descriptions.
CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation.
Philipp Borchert (Lille Economie Management), Marie-Francine Moens (University of Lille)
ClassificationDomain AdaptationMeta LearningTransformerPrompt EngineeringTextBenchmark
🎯 What it does: Constructed and evaluated the CORE dataset for few-shot company relationship classification and cross-domain adaptation.
CoRec: An Easy Approach for Coordination Recognition
Qing Wang (Iowa State University), Qi Li (Iowa State University)
RecognitionTransformerLarge Language ModelText
🎯 What it does: Proposes CoRec—a pipeline model for coordinating structure recognition that does not rely on a syntactic parser, divided into two steps: identifying coordinating words and detecting parallel clause boundaries;
CorefPrompt: Prompt-based Event Coreference Resolution by Measuring Event Type and Argument Compatibilities
Sheng Xu (Soochow University), Qiaoming Zhu (Soochow University)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In this paper, the authors propose a Prompt-based event coreference resolution framework called CorefPrompt, which transforms event coreference determination into a masked language model (MLM) task, completing event modeling and coreference judgment within the same template; simultaneously, two auxiliary Prompt tasks, event type compatibility and argument compatibility, are introduced to explicitly demonstrate the reasoning process; finally, a mask token update mechanism is utilized to enhance the model's interactive expression.
CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network
Sreyan Ghosh (University of Maryland College Park), Dinesh Manocha (University of Maryland College Park)
ClassificationGraph Neural NetworkText
🎯 What it does: This paper proposes the CoSyn framework, aiming to detect implicit hate speech in online conversations by combining user history, social context, and conversation context within a hyperbolic space.
Counter Turing Test (CT²): AI-Generated Text Detection is Not as Easy as You May Think - Introducing AI Detectability Index
Megha Chakraborty (University of South Carolina), Amitava Das (University of South Carolina)
ClassificationLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper systematically evaluates the robustness of current mainstream AI-generated text detection (AGTD) techniques and proposes two tools, the Counter Turing Test 2 (CT²) benchmark and the AI Detectability Index (ADI), to measure and rank the detectability of different large language models (LLMs).
Countering Misinformation via Emotional Response Generation
Daniel Russo (Fondazione Bruno Kessler), Marco Guerini (University of Trento)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper constructs the VerMouth dataset for generating emotional and personalized misinformation correction responses in social media environments.
Counting the Bugs in ChatGPT’s Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model
Leonie Weissweiler (LMU Munich), David Mortensen (Carnegie Mellon University)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A systematic evaluation of ChatGPT's morphological capabilities in English, German, Tamil, and Turkish using an adapted Wug test to generate new words for morphological generation.
COVID-19 Vaccine Misinformation in Middle Income Countries
Jongin Kim (Boston University), Derry Wijaya (Boston University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningTextTime Series
🎯 What it does: Collected and annotated 5,952 multilingual tweets from Brazil, Indonesia, and Nigeria, constructing a three-dimensional annotated dataset encompassing vaccine relevance, presence of misinformation, and misinformation themes. A multi-class misinformation detection model was trained and evaluated on this dataset, and the best-performing model was subsequently applied to 19 million unannotated tweets for cross-country quantification and temporal correlation analysis of misinformation.
CP-BCS: Binary Code Summarization Guided by Control Flow Graph and Pseudo Code
Tong Ye (Zhejiang University), Wenhai Wang (Zhejiang University)
GenerationAI Code AssistantGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityGraph
🎯 What it does: The study focuses on generating readable summaries for de-symbolized binary functions, addressing the high reverse engineering difficulty caused by the lack of symbolic information in binary functions.
CQE: A Comprehensive Quantity Extractor
Satya Almasian (Heidelberg University), Michael Gertz (Heidelberg University)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Developed a comprehensive quantity extraction framework (CQE) capable of identifying and standardizing numerical values, units, change trends, and associated concepts, providing a unified normalized representation.
CRAB: Assessing the Strength of Causal Relationships Between Real-world Events
Angelika Romanou (École Polytechnique Fédérale de Lausanne), Antoine Bosselut (École Polytechnique Fédérale de Lausanne)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Constructed the CRAB benchmark, containing approximately 2.7K fine-grained causal relationship annotations of real-world events, and used it to evaluate the causal reasoning ability of large language models.
CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model
Kaiyan Zhang (Tsinghua University), Bowen Zhou (Tsinghua University)
Computational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Explored a no-training-step method called CRaSh, which extracts efficient submodels (emulators) from large language models (LLMs) by leveraging layer clustering, elimination, and sharing techniques, and achieves privacy-friendly fine-tuning of LLMs through Offsite-Tuning (OFT).
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Mateusz Lango (Charles University), Ondrej Dusek (Charles University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a critic-driven decoding method that uses a text critic to evaluate the consistency between the generated prefix and the input data in real-time, thereby reducing hallucination in data-to-text generation.
Cross-Cultural Analysis of Human Values, Morals, and Biases in Folk Tales
Winston Wu (University of Hawai'i at Hilo), Rada Mihalcea (University of Michigan)
Text
🎯 What it does: A corpus was constructed from 1,925 folktales across 27 cultures, with values, moral foundations, and gender bias quantified using a dictionary-based approach.
Cross-Document Event Coreference Resolution on Discourse Structure
Xinyu Chen (Soochow University), Qiaoming Zhu (Soochow University)
TransformerLarge Language ModelText
🎯 What it does: Propose a method that leverages cross-document discourse structure to enhance event coreference resolution.
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Jirui Qi (University of Groningen), Arianna Bisazza (University of Groningen)
Representation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Studies the cross-lingual consistency (CLC) of factual knowledge in multilingual large-scale pre-trained language models (PLMs) across different languages
Cross-Lingual Cross-Target Stance Detection with Dual Knowledge Distillation Framework
Ruike Zhang (Institute of Automation, Chinese Academy of Sciences), Wenji Mao (Institute of Automation, Chinese Academy of Sciences)
ClassificationKnowledge DistillationGraph Neural NetworkTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Propose a cross-lingual and cross-target stance detection task, and develop a dual-teacher distillation framework (CCSD)
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
Libo Qin (Central South University), Wanxiang Che (Harbin Institute of Technology)
Representation LearningTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes a zero-shot cross-lingual chain-of-thought prompting method CLP (cross-lingual alignment prompts + task-solving prompts) and a cross-lingual self-consistency prompting method CLSP to enable cross-lingual chain-of-thought reasoning.
Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis
Seraphina Goldfarb-Tarrant (University of Edinburgh), Adam Lopez (University of Edinburgh)
ClassificationDomain AdaptationKnowledge DistillationTransformerSupervised Fine-TuningText
🎯 What it does: Investigate whether zero-shot cross-lingual transfer (ZS-XLT) introduces gender and racial bias into sentiment analysis models for low-resource languages, and assess the bias using controlled experiments and quantitative methods.
Cross-Modal Conceptualization in Bottleneck Models
Danis Alukaev (Innopolis University), Ivan Titov (University of Edinburgh)
Explainability and InterpretabilityRepresentation LearningConvolutional Neural NetworkVision Language ModelContrastive LearningImageTextMultimodalityBiomedical Data
🎯 What it does: This study proposes a cross-modal conceptual bottleneck model (XCB) that automatically induces interpretable concepts by leveraging text descriptions corresponding to training images, thereby eliminating the need for manually defining and annotating concepts.
Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting
Emmy Liu (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
GenerationRetrievalTransformerSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Investigate the challenges of idiomatic expressions in machine translation, construct datasets of idiomatic sentences in French, Finnish, and Japanese, and propose a method combining sentence-level loss weighting with retrieval-enhanced kNN-MT to improve the accuracy of Transformer models in translating both idiomatic and non-idiomatic expressions.
CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks
Mete Ismayilzada (EPFL), Antoine Bosselut
Prompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Constructed a multi-task Commonsense Reasoning Benchmark (CROW), generating Winograd-style commonsense violation samples across six real-world NLP tasks through a manually designed multi-stage data collection pipeline, and evaluated models' reasoning capabilities on these tasks.
CRT-QA: A Dataset of Complex Reasoning Question Answering over Tabular Data
Zhehao Zhang (Darmouth College), Jian-Guang Lou (Microsoft Research Asia)
Data SynthesisLarge Language ModelAgentic AIPrompt EngineeringTabularBenchmarkChain-of-Thought
🎯 What it does: Constructed the CRT-QA dataset, focusing on multi-step reasoning and informal reasoning with table data, and provided fine-grained annotations (question directness, sub-question combination types, human reasoning paths) as well as unanswerable/uncertain questions.
CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQL
Mayank Kothyari (Indian Institute of Technology Bombay), Soumen Chakrabarti (Indian Institute of Technology Bombay)
RetrievalAI Code AssistantTransformerLarge Language ModelTextGraphTabularBenchmark
🎯 What it does: Propose a 'schema hallucination' technique based on LLMs, first generating a simplified database schema using LLMs, then leveraging this schema to perform collective retrieval over large-scale databases, resulting in a high-coverage, compact schema subset;
Crystal: Introspective Reasoners Reinforced with Self-Feedback
Jiacheng Liu (FAIR, Meta), Asli Celikyilmaz (FAIR, Meta)
Knowledge DistillationTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Proposed a self-feedback reinforced introspective reasoning model named CRYSTAL for commonsense reasoning
CS2W: A Chinese Spoken-to-Written Style Conversion Dataset with Multiple Conversion Types
Zishan Guo (Tianjin University), Deyi Xiong (Tianjin University)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed the first open Chinese speech-to-text conversion dataset CS2W, covering four types of errors in spoken language and providing fine-grained annotations;
CT-GAT: Cross-Task Generative Adversarial Attack based on Transferability
Minxuan Lv (Chinese Academy of Sciences), Songlin Hu (Chinese Academy of Sciences)
Adversarial AttackTransformerGenerative Adversarial NetworkText
🎯 What it does: Propose a generative adversarial attack (CT-GAT) based on cross-task transferable features, which directly generates adversarial text by training a sequence-to-sequence generator, avoiding the need to construct an alternative model.
Cultural Concept Adaptation on Multimodal Reasoning
Zhi Li (Zhejiang University), Yin Zhang (Zhejiang University)
Data SynthesisDomain AdaptationSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: This paper proposes an unlabeled cultural concept adaptation method and designs a multimodal data augmentation technique called CultureMixup, aiming to improve the performance of cross-lingual and cross-cultural vision-text reasoning models.
DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules
Yanchen Liu (Harvard University), Diyi Yang (Stanford University)
Domain AdaptationSupervised Fine-TuningText
🎯 What it does: Propose a method for multi-dialect adaptation of standard American English models through dynamic aggregation of language feature adapters (DADA).
DALE: Generative Data Augmentation for Low-Resource Legal NLP
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
ClassificationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: In low-resource legal NLP tasks, the DALE framework is proposed, which uses BART pretraining and performs selective masked denoising sequence-to-sequence learning on legal texts, then generates diverse, coherent, and label-consistent synthetic data augmentation samples for downstream tasks.
Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA
David Heineman (Georgia Institute of Technology), Wei Xu (Georgia Institute of Technology)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the SALSA edit-level simplification evaluation framework, constructing 21 edit types and collecting 19K annotations
Data Factors for Better Compositional Generalization
Xiang Zhou (UNC Chapel Hill), Mohit Bansal (UNC Chapel Hill)
Data-Centric LearningTransformerText
🎯 What it does: This study investigates through systematic experiments how data factors (scale, complexity, example difficulty) affect the performance of Transformer models in combinatorial generalization tasks, and proposes corresponding experimental designs and explanations.
Data Similarity is Not Enough to Explain Language Model Performance
Gregory Yauney (Cornell University), David Mimno (Cornell University)
ClassificationExplainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper evaluates the correlation between various text similarity metrics (word distribution, sentence embeddings, MAUVE, language model perplexity) and the zero-shot/few-shot performance of large language models (Pythia, T5, Flan-T5) on different downstream tasks (GLUE, BIG-bench Lite, Stack Exchange classification, XNLI).
Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification
Chengyu Dong (University of California San Diego), Jingbo Shang (University of California San Diego)
ClassificationTransformerText
🎯 What it does: An improvement on weakly supervised text classification methods based on seed matching, introducing two debiasing approaches: Seed Deletion and Random Deletion, to alleviate label bias caused by seed matching, thereby enhancing the reliability of pseudo-labels and overall classification performance.
Deciphering Stereotypes in Pre-Trained Language Models
Weicheng Ma (Dartmouth College), Soroush Vosoughi (Dartmouth College)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper proposes a framework based on Shapley values for detecting attention heads, identifying and pruning critical attention heads in pre-trained Transformer language models to reduce stereotypes, and combining SHAP and attention distribution for text clue analysis.
DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4
Yebowen Hu (University of Central Florida), Fei Liu (Emory University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper reveals the key factors influencing the preference judgment of large language model outputs by analyzing OpenAI's published human comparison evaluation data, focusing on dimensions such as length, language quality, and factual consistency in the summarization task.
Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting
Chenkai Sun (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
Recommendation SystemGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraph
🎯 What it does: Propose the SOCIALSENSE framework, which induces belief-enhanced social networks using large language models and responds with graph neural networks.
DecoMT: Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models
Ratish Puduppully (Institute for Infocomm Research, A STAR), Nancy Chen
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a machine translation method called DecoMT, which uses large language models for few-shot prompting by splitting sentences into chunks for decomposed prompting.
Deep Natural Language Feature Learning for Interpretable Prediction
Felipe Urrutia (Centro Nacional de Inteligencia Artificial), Valentin Barriere (Centro Nacional de Inteligencia Artificial)
ClassificationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose a general method that decomposes complex tasks into natural language binary subtasks and learns features using a small Transformer model
Democratizing Reasoning Ability: Tailored Learning from Large Language Model
Zhaoyang Wang (Sun Yat-sen University), Qi Zhang (Microsoft)
Knowledge DistillationLarge Language ModelPrompt EngineeringContrastive LearningTextBenchmarkChain-of-Thought
🎯 What it does: Through multi-round interactive learning, a small LM is trained to possess chain-of-thought (CoT) capabilities, while self-reflective learning enhances its reasoning quality.
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models
Xinwei Wu (Tianjin University), Deyi Xiong (Tianjin University)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the DEPN framework, which reduces the risk of models leaking private information by detecting and zeroing out privacy neurons in pre-trained language models.
Describe Me an Auklet: Generating Grounded Perceptual Category Descriptions
Bill Noble (University of Gothenburg), Nikolai Ilinykh (University of Gothenburg)
ClassificationGenerationExplainability and InterpretabilityTransformerContrastive LearningMultimodality
🎯 What it does: Studied a framework for visual category description generation and explanation using a generator and interpreter to evaluate communication effectiveness through zero-shot classification.
DeSIQ: Towards an Unbiased, Challenging Benchmark for Social Intelligence Understanding
Xiao-Yu Guo (Monash University), Reza Haf
Knowledge DistillationTransformerLarge Language ModelMultimodalityBenchmark
🎯 What it does: Identify biases in the Social-IQ dataset, construct a bias-free and more challenging DeSIQ benchmark, and provide new multimodal model baselines.
Detecting and Mitigating Hallucinations in Multilingual Summarisation
Yifu Qiu (Institute for Language, Cognition and Computation, University of Edinburgh), Shay B. Cohen (Institute for Language, Cognition and Computation, University of Edinburgh)
GenerationAnomaly DetectionTransformerLarge Language ModelText
🎯 What it does: Propose the multilingual hallucination detection metric mFACT and adopt a loss-weighted method based on hallucination scores in cross-lingual summarization to reduce hallucinations and enhance faithfulness.
Detecting Propaganda Techniques in Code-Switched Social Media Text
Muhammad Umar Salman (Mohamed Bin Zayed University of Artificial Intelligence), Preslav Nakov (Mohamed Bin Zayed University of Artificial Intelligence)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The study focuses on detecting propaganda techniques in social media text mixed with English, Romanized Urdu, and Urdu, and constructs a fine-grained annotated code-switching corpus containing 1,030 samples.
Detecting Spoilers in Movie Reviews with External Movie Knowledge and User Networks
Heng Wang (Xi'an Jiaotong University), Minnan Luo (Xi'an Jiaotong University)
ClassificationGraph Neural NetworkTransformerLarge Language ModelTextGraphRetrieval-Augmented Generation
🎯 What it does: Proposed a multi-view movie review spoiler detection framework named MVSD, and constructed a large-scale LCS dataset and a UKM movie knowledge base based on IMDb.
Detection of Multiple Mental Disorders from Social Media with Two-Stream Psychiatric Experts
Siyuan Chen (Shanghai Jiao Tong University), Kenny Zhu (University of Texas at Arlington)
ClassificationTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: To detect comorbid mental illnesses in social media users, a dual-stream multi-task framework named PsyEx was constructed. It utilizes a symptom identification model to screen high-risk posts and simultaneously inputs text and symptom features into a shared layer plus disease-specific experts for learning, thereby achieving disease detection under both binary classification and multi-label settings.
DetGPT: Detect What You Need via Reasoning
Renjie Pi (Hong Kong University of Science and Technology), Tong Zhang (Hong Kong University of Science and Technology)
Object DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Proposed the inferential object detection task and built a two-stage system called DetGPT, where a Vision-Language Model (VLM) first reasons based on high-level instructions to output object categories, followed by an open-vocabulary object detector to locate these targets.
DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Models
Chengcheng Han (East China Normal University), Baoyuan Wang (Xiaobing.AI)
Computational EfficiencyTransformerSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: Proposed the DialCoT and DialCoT‑S‑PPO frameworks, which utilize dialog-based chain-of-thought to decompose complex problems into subproblems and optimize the reasoning path via PPO reinforcement learning, significantly enhancing the performance of small language models (≤10B) on arithmetic reasoning tasks.
Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources
Yerin Hwang (Seoul National University), Kyomin Jung (Seoul National University)
GenerationData SynthesisTransformerLarge Language ModelText
🎯 What it does: Proposed the Dialogizer framework, which can automatically generate high-quality, context-related conversational question-answering (ConvQA) datasets from text.
Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents
Hyungjoo Chae (Yonsei University), Jinyoung Yeo (Yonsei University)
Knowledge DistillationRepresentation LearningTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Built a knowledge distillation framework based on conversational chain-of-thought (CoT), leveraging large language models (LLMs) to generate multi-hop common-sense reasoning and filtering high-quality reasoning chains through alignment filters; further constructed a large-scale conversational CoT dataset DONUT and trained a conversational chain-of-thought model DOCTOR to enhance common-sense reasoning capabilities of dialogue agents.
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing
Elias Stengel-Eskin (University of North Carolina Chapel Hill), Benjamin Van Durme (Johns Hopkins University)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper explores leveraging calibrated confidence in semantic parsing to balance annotation cost and correctness, as well as usability and safety, and proposes a confidence-based interactive system called DidYouMean.
DiffS2UT: A Semantic Preserving Diffusion Model for Textless Direct Speech-to-Speech Translation
Yongxin Zhu (University of Science and Technology of China), Linli Xu (iFlytek Research)
GenerationData SynthesisTransformerDiffusion modelAudio
🎯 What it does: Developed a diffusion model named DiffS2UT for text-free direct speech-to-speech translation, achieving semantic preservation and significantly improving generation speed by performing forward diffusion in continuous speech representation space and backward diffusion in discrete unit space.
DiNeR: A Large Realistic Dataset for Evaluating Compositional Generalization
Chengang Hu (Peking University), Yansong Feng (Peking University)
RecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Propose the DINER task and construct a large-scale Chinese dish name recognition dataset, aiming to evaluate the model's compositional generalization ability.
DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining
Weifeng Jiang (Nanyang Technological University), Zheng Wang (Beihang University)
ClassificationKnowledge DistillationAdversarial AttackTransformerText
🎯 What it does: This paper proposes a semi-supervised co-training framework based on knowledge distillation, which generates multiple lightweight student models through knowledge distillation from a teacher model, and enhances the performance of small models on text mining tasks by leveraging complementary sharing between model views and data views.
Discourse Structures Guided Fine-grained Propaganda Identification
Yuanyuan Lei (Texas A&M University), Ruihong Huang (Texas A&M University)
ClassificationKnowledge DistillationTransformerText
🎯 What it does: This paper proposes to utilize two types of discourse structures, local and global, for fine-grained propaganda content identification at both the sentence-level and word-level.
Discovering Universal Geometry in Embeddings with ICA
Hiroaki Yamagiwa (Kyoto University), Hidetoshi Shimodaira (Kyoto University)
Explainability and InterpretabilityRepresentation LearningImageText
🎯 What it does: Applying ICA to word embeddings and image embeddings reveals interpretable semantic axes, and their universality is verified across different languages, algorithms, and modalities.
Disentangling Transformer Language Models as Superposed Topic Models
Jia Peng Lim (Singapore Management University), Hady Lauw
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose a weight-based, model-agnostic, and corpus-agnostic method for disentangling superimposed topics from decoder-only Transformer language models (TLM), and verify the method's effectiveness in obtaining interpretable topics on GPT-2 and LLaMA.
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Mor Geva (Google DeepMind), Amir Globerson (Tel Aviv University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: A fine-grained analysis of how autoregressive Transformer language models retrieve factual associations during inference, revealing a three-step internal mechanism: generating topic representations from lower-level multi-layer perceptrons (MLPs) and extracting attributes through multi-head self-attention (MHSA) attention heads.
Distance-Based Propagation for Efficient Knowledge Graph Reasoning
Harry Shomer (Michigan State University), Jiliang Tang (Michigan State University)
Computational EfficiencyRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes TAGNet, which improves the propagation efficiency of path-based graph neural networks in knowledge graph completion by utilizing a fixed window distance clipping mechanism.
DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuning
Praveen Venkateswaran (IBM Research), Vatche Isahagian (IBM Research)
RetrievalData-Centric LearningTransformerTextRetrieval-Augmented Generation
🎯 What it does: Proposed a retrieval-driven dialogue state tracking method called DiSTRICT, which automatically retrieves the most relevant training instances from the input dialogue and slots, uses them as context for model fine-tuning, and thereby achieves dialogue state prediction.
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings
Qian Chen (Alibaba Group), Chong Zhang (Alibaba Group)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose an unsupervised method called Ditto to improve sentence embeddings based on BERT's self-attention diagonal weights, using diagonal attention from self-attention heads to weight-average words, addressing the heterogeneity and bias toward non-informative words in original BERT sentence embeddings.
DIVE: Towards Descriptive and Diverse Visual Commonsense Generation
Jun-Hyung Park (Korea University), SangKeun Lee (Korea University)
GenerationTransformerVision Language ModelContrastive LearningMultimodalityGraph
🎯 What it does: Proposes the DIVE framework to enhance the descriptiveness and diversity of visual commonsense reasoning models.
Diversify Question Generation with Retrieval-Augmented Style Transfer
Qi Gou (Nanjing University), Cam-Tu Nguyen (Nanjing University)
GenerationRetrievalTransformerSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a retrieval-augmented style transfer framework (RAST), which generates diverse question-answering questions by retrieving and combining context from an external question template library.
Diversity Enhanced Narrative Question Generation for Storybooks
Hokeun Yoon (Sungkyunkwan University), JinYeong Bak (Sungkyunkwan University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Proposed and implemented a multi-question generation model called mQG, which can automatically generate diverse and answerable narrative questions given context, question type, and already generated questions.
DNA: Denoised Neighborhood Aggregation for Fine-grained Category Discovery
Wenbin An (Xi'an Jiaotong University), Ping Chen (University of Massachusetts Boston)
Representation LearningTransformerContrastive LearningTextBenchmark
🎯 What it does: Proposes a self-supervised framework DNA, which utilizes k-NN to retrieve neighbors and aggregate information, learning fine-grained category representations from coarse-grained labeled data.
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Orevaoghene Ahia (University of Washington), Yulia Tsvetkov (University of Washington)
Computational EfficiencyData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Studied the usage cost and performance differences caused by tokenization imbalance in commercial language model APIs across different languages.
Do Differences in Values Influence Disagreements in Online Discussions?
Michiel van der Meer (Leiden University), Pradeep K. Murukannaiah (Delft University of Technology)
TransformerLarge Language ModelText
🎯 What it does: This paper explores the impact of value differences on the intensity of arguments by estimating user value profiles from Reddit comments and associating value conflicts with agreement/disagreement labels in online discussions.
Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models
Raghav Jain (Indian Institute of Technology Patna), Sandipan Dandapat (Microsoft)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Systematic evaluation and benchmarking of large language models (LLMs) on temporal reasoning tasks
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark
Minje Choi (University of Michigan), David Jurgens (University of Michigan)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Created and released a benchmark called SOCKET, containing 58 NLP tasks related to social knowledge to evaluate the social language understanding capabilities of large language models (LLMs).
Do Transformers Parse while Predicting the Masked Word?
Haoyu Zhao (Princeton University), Sanjeev Arora (Princeton University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Investigated whether Transformers can perform and approximate the Inside-Out parsing of PCFG during masked language model training, and validated their parsing capabilities on synthetic PCFG text through experiments.