ACL 2024 Papers — Page 2
Annual Meeting of the Association for Computational Linguistics · 940 papers
BIPED: Pedagogically Informed Tutoring System for ESL Education
Soonwoo Kwon (Twelve Labs), Kyuseok Kim (Riiid AI Research)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Developed a bilingual, teaching-depth conversational intelligent tutoring system (CITS) and constructed the BIPED dataset to support its training and evaluation.
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation
DaYou Du, Ningyi Xu (Shanghai Jiao Tong University)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Explored combining quantization-aware training (QAT) with self-distillation to enhance the performance of low-precision (≤4-bit) large language models (LLMs).
BizBench: A Quantitative Reasoning Benchmark for Business and Finance
Michael Krumdick (Kensho Technologies), Chris Tanner (Kensho Technologies)
Large Language ModelTextTabularBenchmarkFinance RelatedChain-of-Thought
🎯 What it does: Propose BizBench benchmark to evaluate the quantitative reasoning and program synthesis capabilities of large language models in financial business scenarios.
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training
Jiale Cheng (Tsinghua University), Minlie Huang (Tsinghua University)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study proposes a black-box prompt optimization (BPO) method to automatically rewrite user prompts to enhance the alignment of LLMs with human preferences without modifying model parameters.
Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?
Hexiang Tan (Institute of Computing Technology, Chinese Academy of Sciences), Xueqi Cheng (Institute of Computing Technology, Chinese Academy of Sciences)
RetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Investigate the fusion mechanisms of Large Language Models (LLMs) when integrating generated and retrieved contexts, construct a conflicting context dataset, and quantify their preferences.
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting
Jianing Wang (East China Normal University), Ming Gao (East China Normal University)
Explainability and InterpretabilityLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Improve reasoning quality by using Chain-of-Knowledge prompting (CoK), which makes large language models first generate structured triplet evidence and explanatory prompts during reasoning.
Born Differently Makes a Difference: Counterfactual Study of Bias in Biography Generation from a Data-to-Text Perspective
Biaoyan Fang (CSIRO Data61), Sarvnaz Karimi (CSIRO Data61)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By constructing controlled experiments, using counterfactual methods to study the causal effects of personal attributes (gender and region) on the content of generated biographical texts.
Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Length
Nur Lan (Ecole Normale Supérieure), Roni Katzir (Tel Aviv University)
RecognitionOptimizationExplainability and InterpretabilityRecurrent Neural NetworkSequential
🎯 What it does: Constructed an optimal LSTM that perfectly identifies the formal language a b^n n, and compared it with networks trained using standard regularization/metaheuristic training objectives.
Bridging the Preference Gap between Retrievers and LLMs
Zixuan Ke (University of Illinois at Chicago), Michael Bendersky (Google Research)
GenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose a bridging model BGM that uses a lightweight seq2seq bridge between a frozen retriever and a frozen LLM to dynamically re-rank, select, and repeat retrieval results to align with LLM preferences, thereby enhancing RAG performance.
Bridging Word-Pair and Token-Level Metaphor Detection with Explainable Domain Mining
Yuan Tian (Chinese Academy of Sciences), Wenji Mao (Chinese Academy of Sciences)
ClassificationExplainability and InterpretabilityTransformerText
🎯 What it does: Propose an interpretable word-level metaphor detection method WPDM based on word pair and concept domain mining, bridging word pairs with word-level detection, utilizing semantic role mapping and concept domain inconsistency to screen core word pairs and assist classification.
Browse and Concentrate: Comprehending Multimodal Content via Prior-LLM Context Fusion
Ziyue Wang (Tsinghua University), Yang Liu (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageVideoTextMultimodality
🎯 What it does: Proposed a two-stage paradigm called 'browse-and-concentrate' to address the isolation problem between visual and textual elements as well as among images in multi-modal inputs. The model first obtains cross-image contextual information through browsing before entering the LLM, and then utilizes this information for deeper understanding during the concentration stage.
BvSP: Broad-view Soft Prompting for Few-Shot Aspect Sentiment Quad Prediction
Yinhao Bai (Nankai University), Renhong Cheng (Nankai University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposes the BvSP (Broad-view Soft Prompting) method for the few-shot Aspect Sentiment Quad Prediction (ASQP) task and constructs a new FSQP dataset
Bypassing LLM Watermarks with Color-Aware Substitutions
Qilong Wu (University of Illinois Urbana Champaign), Varun Chandrasekaran (University of Illinois Urbana Champaign)
Adversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed a Self-Color Test (SCT) + Replacement (SCTS) attack, which utilizes prompts to perform random generation tests on watermarked LLMs to obtain token color information, and replaces green tokens with non-green tokens to effectively eliminate watermarks.
Calibrating Large Language Models Using Their Generations Only
Dennis Ulmer (Parameter Lab), Seong Oh
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose an auxiliary model method called APRICOt that uses the LLM's own generated text to calibrate its confidence
CaMML: Context-Aware Multimodal Learner for Large Models
Yixin Chen (Chinese University of Hong Kong), Bo Li (University of Chicago)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed a lightweight context-aware multimodal learning module called CaMML, which can efficiently integrate retrieved multimodal context samples into large multimodal models (e.g., LLaMA/Vicuna) without introducing external resources, thereby enhancing the model's reasoning capabilities across various multimodal tasks.
Can ChatGPT’s Performance be Improved on Verb Metaphor Detection Tasks? Bootstrapping and Combining Tacit Knowledge
Cheng Yang (Guangxi University), Qingbao Huang (Guangxi University)
RecognitionTransformerLarge Language ModelText
🎯 What it does: Propose an unsupervised method for detecting verb metaphors by automatically generating verb semantic collocation tables and theme mappings using ChatGPT.
Can Language Models Serve as Text-Based World Simulators?
Ruoyao Wang (University of Arizona), Peter Jansen (University of Arizona)
TransformerLarge Language ModelPrompt EngineeringWorld ModelTextBenchmark
🎯 What it does: Construct and use the new benchmark BYTESIZED32-State-Prediction to evaluate text game state transitions, testing the capability of large language models (e.g., GPT-4) as text world simulators.
Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation
Dongjin Kang, Jinyoung Yeo (Yonsei University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Explore the performance of large language models in emotional support dialogues, focusing on their preference bias for strategies and investigating methods to reduce this bias.
Can Large Language Models Interpret Noun-Noun Compounds? A Linguistically-Motivated Study on Lexicalized and Novel Compounds
Giulia Rambelli (University of Bologna), Marianna Bolognesi (University of Bologna)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: The study evaluates the ability of large language models (Llama-2, Falcon, Mistral) to explain noun-noun compounds (with paraphrases indicating relational meanings) and investigates whether models can abstractly infer from known compounds to novel compounds.
Can LLMs Learn from Previous Mistakes? Investigating LLMs’ Errors to Boost for Reasoning
Yongqi Tong (University of California, San Diego), Jingbo Shang (University of California, San Diego)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningTextChain-of-Thought
🎯 What it does: Investigate whether LLMs can learn from their own mistakes to improve reasoning, proposing two methods: self-reflection and mistake tuning.
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs
Siyuan Wang (Fudan University), Xiang Ren (University of Southern California)
Explainability and InterpretabilityKnowledge DistillationTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Proposed the LOIRE framework for generating logical scaffold reasoning rules, and constructed a large-scale reasoning rule library called ULogic;
Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models
Zhiwei He (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
Safty and PrivacyTransformerLarge Language ModelText
🎯 What it does: Studied the consistency of large language model text watermarks after translation into different languages, proposed a cross-language watermark removal attack (CWRA), and improved the SIR algorithm through cross-language semantic clustering to obtain the X-SIR defense method.
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data?
Qingkai Fang (Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences), Yang Feng (Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences)
GenerationTransformerSupervised Fine-TuningContrastive LearningTextAudio
🎯 What it does: Proposed the ComSpeech model, achieving high-quality direct speech-to-speech translation by introducing a CTC vocabulary adapter between existing speech-to-text (S2TT) and text-to-speech (TTS) models, and further introduced the ComSpeech-ZS zero-shot training method, enabling the model to complete S2ST using only S2TT and TTS data.
Can Your Model Tell a Negation from an Implicature? Unravelling Challenges With Intent Encoders
Yuwei Zhang (University of California, San Diego), Saab Mansour (Amazon WS AI Labs)
ClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed the Intent Semantics Toolkit, evaluating intent embedding models' understanding of negation and implication through three tasks (intent classification, clustering, and a novel triplet task), and enhancing performance in these semantic dimensions via model fine-tuning using data generated by LLMs.
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning
Weiqi Wang (Amazon.com Inc), Yangqiu Song (Hong Kong University of Science and Technology)
Knowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the CANDLE framework, which iteratively uses large language models for conceptualization and instantiation to expand the commonsense knowledge base
CARE: A Clue-guided Assistant for CSRs to Read User Manuals
Weihong Du (Sichuan University), Wenqiang Lei (Sichuan University)
RetrievalExplainability and InterpretabilityAI Code AssistantGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Propose CARE, construct a heterogeneous graph of user manuals, utilize self-supervised learning to generate clue chains, aiding customer service representatives in quickly locating and explaining answers.
Causal Estimation of Memorisation Profiles
Pietro Lesci (University of Cambridge), Tiago Pimentel (ETH Zürich)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper proposes a difference-in-differences (DiD) method based on causal inference to estimate the 'memorisation' effect of language models on training samples, and uses this method to construct memory curves for each sample or batch during training—namely, memory trajectories and memory profiles;
Causal-Guided Active Learning for Debiasing Large Language Models
Zhouhao Sun (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a causal-guided active learning framework (CAL), which utilizes large language models to automatically identify and induce dataset bias instances, and debiases the model through prompt learning methods;
CausalGym: Benchmarking causal interpretability methods on linguistic tasks
Aryaman Arora (Stanford University), Christopher Potts (Stanford University)
Explainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the CausalGym benchmark to evaluate the causal efficacy of explanation methods in language models, generating a large number of alignable minimal pairs based on SyntaxGym to test the causal mechanisms of models for linguistic phenomena;
Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languages
Samuel Cahyawijaya (Hong Kong University of Science and Technology), Pascale Fung
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed the Cendol series, including multi-scale Indonesian instruction-tuned large language models (Decoder-only and Encoder-Decoder) and a large-scale Cendol Collection instruction dataset;
Chain-of-Exemplar: Enhancing Distractor Generation for Multimodal Educational Question Generation
Haohao Luo (Sun Yat-sen University), Tat-Seng Chua (National University of Singapore)
GenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose a Chain-of-Exemplar framework based on a multimodal large language model for generating education-oriented multiple-choice questions and suspicious distractors from text+image inputs.
CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation
Quan Tu (Renmin University of China), Rui Yan (Renmin University of China)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelTextBenchmark
🎯 What it does: Developed a high-quality Chinese role-playing dialogue benchmark (CharacterEval) and trained a role reward model based on human evaluation (CharacterRM).
Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People
Dun-Ming Huang (University of California Berkeley), Nori Jacoby (Max Planck Institute for Empirical Aesthetics)
Domain AdaptationRepresentation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Using the Sampling with People method, humans and GPT-4 alternately annotated the intonation of sentences and generated new sentences, collecting 80 sentences and 40 intonations. Subsequently, Quality Fit scores were used to let independent humans and GPT-4 evaluate the matching degree of each sentence across all intonations, constructing semantic embeddings for intonations. Then, Multidimensional Scaling (MDS) mapped the embeddings from humans and GPT-4 into the same low-dimensional space, visually demonstrating their similarity and differences. Finally, this aligned data was used as a benchmark to evaluate the performance of three unsupervised cross-domain semantic alignment methods.
Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages
Shih-Cheng Huang (National Applied Research Laboratories), Hung-yi Lee (National Taiwan University)
TransformerLarge Language ModelText
🎯 What it does: Achieve instruction following, dialogue capabilities, and value alignment for large language models in new languages by utilizing continual pre-training (CP) and directly adding chat vectors (Chat Vector) to model weights.
ChatDev: Communicative Agents for Software Development
Chen Qian (Tsinghua University), Maosong Sun (Tsinghua University)
AI Code AssistantTransformerLarge Language ModelAgentic AIPrompt EngineeringText
🎯 What it does: Proposes ChatDev, a multi-agent collaborative software development framework built using large language models (LLMs). The framework breaks down software lifecycle stages such as design, coding, and testing into subtasks through a 'chat chain,' and reduces hallucination issues in code generation via a 'communicative dehallucination' mechanism.
CHECKWHY: Causal Fact Verification via Argument Structure
Jiasheng Si (Qilu University of Technology (Shandong Academy of Sciences)), Deyu Zhou (Southeast University)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: Constructed a causal fact verification dataset named CHECKWHY, focusing on 'why'-type causal claims accompanied by tree-like reasoning structures;
Cheetah: Natural Language Generation for 517 African Languages
Ife Adebara (University of British Columbia), Muhammad Abdul-Mageed (University of British Columbia)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Built a natural language generation model called Cheetah that supports 517 African languages.
ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences
Yuanhe Tian (University of Washington), Yongdong Zhang (University of Science and Technology of China)
Reinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Proposed and implemented CHIMED-GPT, a large language model specifically designed for Chinese medical text processing, which employs a three-stage training process consisting of full pre-training, supervised fine-tuning, and reinforcement learning with human feedback.
ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks
Santosh T.y.s.s, Matthias Grabmair (Technical University of Munich)
ClassificationDomain AdaptationTransformerSupervised Fine-TuningText
🎯 What it does: Introduce the ChronosLex incremental training framework, which fine-tunes models in chronological order to address time drift issues in legal multi-label text classification, and evaluate the effectiveness of continual learning and time-invariant methods.
Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers
Jiawen Xie (Tencent AI Lab), Nan Du (Tencent AI Lab)
Computational EfficiencyTransformerReinforcement LearningText
🎯 What it does: Propose a long-sequence processing framework named SimCAS, which significantly reduces the computational and storage complexity of self-attention by splitting the input into multiple blocks, performing batch alignment of start/end tokens across blocks in the encoding layer, and using reinforcement learning strategies to select the most representative hidden states during decoding.
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
Lu Ye (Microsoft), Yang Li (Microsoft)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Designed and implemented a prefix-aware KV cache (PAKV) and a two-phase partitioning (TPP) self-attention module, significantly improving memory utilization and throughput in LLM inference.
Citation-Enhanced Generation for LLM-based Chatbots
Weitao Li (Tsinghua University), Yang Liu (Tsinghua University)
GenerationRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed a post-hoc citation-enhanced generation (CEG) framework that automatically detects and corrects hallucinations in LLM-generated outputs using retrieval and natural language inference (NLI), generating answers with citations;
CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models
Tong Zhang (Sichuan University), Tat-Seng Chua (National University of Singapore)
TransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Propose the CLAMBER benchmark to evaluate large language models (LLMs) in identifying and clarifying ambiguous queries, covering a new three-dimensional eight-class ambiguity classification system;
Classist Tools: Social Class Correlates with Performance in NLP
Amanda Cercas Curry (MilaNLP Bocconi University), Dirk Hovy (MilaNLP Bocconi University)
TransformerLarge Language ModelTextAudio
🎯 What it does: This paper constructs a dataset of 95K lines of movie and TV show dialogues/audio, manually annotated with characters' socioeconomic status, race, and geographic variants, and systematically evaluates the performance differences of NLP tasks such as language models, automatic speech recognition (ASR), and grammar correction across different socioeconomic classes on this dataset; differences are quantified using metrics like WER, perplexity, and edit distance.
Cleaner Pretraining Corpus Curation with Neural Web Scraping
Zhipeng Xu, Chenyan Xiong (Carnegie Mellon University)
Data-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Propose NeuScraper, a web text extractor based on shallow neural networks, capable of efficiently and accurately extracting main content from HTML pages.
CLOMO: Counterfactual Logical Modification with Large Language Models
Yinya Huang (City University of Hong Kong), Linqi Song (City University of Hong Kong)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed the Counterfactual Logical Modification (CLOMO) task and benchmark dataset to evaluate the reasoning and rewriting capabilities of large language models in logical counterfactual modifications.
Co-training for Low Resource Scientific Natural Language Inference
Mobashir Sadat (University of Illinois Chicago), Cornelia Caragea (University of Illinois Chicago)
ClassificationData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Propose a co-training method based on training dynamic weighting, which learns low-resource scientific NLI tasks by leveraging a small amount of human-annotated data and a large amount of automatically annotated data generated via distant supervision.
CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
Shiyi Zhu (Ant Group), Jianguo Li (Ant Group)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose a novel attention mechanism called CoCA, seamlessly integrating position encoding with self-attention to address abnormal behaviors during the extension of long context windows.
Code-Switching Can be Better Aligners: Advancing Cross-Lingual SLU through Representation-Level and Prediction-Level Alignment
Zhihong Zhu (Peking University), Yuexian Zou (Peking University)
Domain AdaptationKnowledge DistillationRepresentation LearningTransformerContrastive LearningText
🎯 What it does: Propose the REPE framework, which leverages code-mixed sentences and original sentences to achieve multi-level alignment at the representation and prediction layers, thereby improving zero-shot cross-lingual SLU performance.
CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges
Kechi Zhang (Peking University), Zhi Jin (Peking University)
AI Code AssistantTransformerAgentic AIPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed the repo-level code generation task and built the CODEAGENT framework, leveraging a toolset to enhance the generation capability of LLMs in real code repositories.
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation
Weixiang Yan (University of California, Santa Barbara), Shuiguang Deng (University of Illinois at Urbana-Champaign)
AI Code AssistantLarge Language ModelTextBenchmark
🎯 What it does: Propose CodeScope, a systematic evaluation framework for assessing the code comprehension and generation capabilities of large models;
CODIS: Benchmarking Context-dependent Visual Comprehension for Multimodal Large Language Models
Fuwen Luo (Tsinghua University), Yang Liu (Tsinghua University)
Large Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Designed the CODIS benchmark to evaluate the performance of multimodal large language models in visual understanding tasks with contextual information.
CoELM: Construction-Enhanced Language Modeling
Lvxiaowei Xu (Zhejiang University), Jiawei Peng (Zhejiang University)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: The study constructs a generative language model, CoELM, by leveraging constructive information, and proposes an automatic construction induction framework, CxGLearner, along with a dynamic sequence recombination pre-training strategy;
CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models
Zixin Chen (Beijing University of Posts and Telecommunications), Guang Chen (Beijing University of Posts and Telecommunications)
ClassificationObject DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: Proposed the CofiPara framework, which leverages multimodal language models (LMM) to generate competitive reasoning (rationales). It first undergoes coarse-grained pre-training on the multimodal sarcasm detection (MSD) task, then undergoes fine-grained fine-tuning on the multimodal sarcasm target identification (MSTI) task, ultimately achieving multimodal sarcasm target detection with interpretable reasoning text for both text and images.
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following
Kaiyan Zhang (Tsinghua University), Bowen Zhou (Tsinghua University)
GenerationSafty and PrivacySupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Propose the CoGenesis framework, which collaborates between large models in the cloud and small models locally to achieve context-aware instruction generation while protecting user privacy.
Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment
Yunxin Li (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
Knowledge DistillationRepresentation LearningTransformerPrompt EngineeringVision Language ModelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose Cognitive Visual-Language Mapper (CVLM), enhancing the performance of multimodal models in knowledge-driven visual question answering through Visual-Knowledge Alignment (VKA) and Fine-grained Knowledge Adapter (FKA).
COKE: A Cognitive Knowledge Graph for Machine Theory of Mind
Jincenzi Wu (Tsinghua University), Minlie Huang (Tsinghua University)
GenerationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextGraph
🎯 What it does: Constructed COKE (Cognitive Knowledge Graph) and COLM (Cognitive Language Model) to enable machines to perform theoretical mind reasoning.
Collaboration or Corporate Capture? Quantifying NLP’s Reliance on Industry Artifacts and Contributions
Will Aitken (Queen's University), Catherine Stinson (Queen's University)
Text
🎯 What it does: This paper manually annotates 100 papers from the 2022 EMNLP conference, collecting information such as the institutions of the authors and cited models/datasets. It then quantifies the dependence of the NLP community on industry outputs (pre-trained models, datasets, previous state-of-the-art results, etc.) and analyzes differences in SOTA contributions and improvement magnitudes across different institution types.
Combining Supervised Learning and Reinforcement Learning for Multi-Label Classification Tasks with Partial Labels
Zixia Jia (Beijing Institute for General Artificial Intelligence), Zilong Zheng (Zhejiang University)
ClassificationReinforcement LearningImageText
🎯 What it does: Propose a framework named MLPAC combining reinforcement learning (RL) with supervised learning to address multi-label positive/unlabeled (MLPUL) learning tasks.
Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning
Philipp Mondorf (LMU Munich), Barbara Plank (LMU Munich)
TransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Systematic comparison and analysis of the reasoning strategies used by humans and large language models (LLMs) in propositional logic reasoning tasks.
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Francesco Ortu (University of Trieste), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Investigate the mechanism competition of large language models in processing factual and counterfactual information, using logit inspection and attention modification to track the interaction between factual memory and copying mechanisms, and significantly improve fact recall through a minimal number of attention entries.
Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs
Tianqing Fang (HKUST), Antoine Bosselut (EPFL)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Constructed a complex common-sense reasoning dataset named COM² based on multi-hop logical queries, generating multi-event, multi-hop reasoning questions using triplets from the ATOMIC knowledge graph;
Confabulation: The Surprising Value of Large Language Model Hallucinations
Peiqi Sui (McGill University), Richard So (McGill University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the confabulation phenomenon in large language models by quantifying narrativity and coherence, and demonstrates that confabulated texts outperform real outputs in both aspects.
Confidence is not Timeless: Modeling Temporal Validity for Rule-based Temporal Knowledge Graph Forecasting
Rikui Huang (Huazhong University of Science and Technology), Yu Cheng (Chinese University of Hong Kong)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: Propose the TempValid framework, which studies temporal knowledge graph prediction with rule confidence decaying over time, using a learnable confidence coupled with an exponential decay function, and designs rule adversarial and time-aware negative sampling for efficient training.
Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
Abhishek Kumar (Brock University), Ali Emami (Brock University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigate the correspondence between internal confidence in large language models and the confidence expressed by the model (Confidence-Probability Alignment) and evaluate its reliability
ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models
Aparna Elangovan (AWS AI Labs), Dan Roth (AWS AI Labs)
GenerationSafty and PrivacyLarge Language ModelTextBenchmark
🎯 What it does: Proposes the ConSiDERS-The-Human evaluation framework, which systematically integrates dimensions such as usability, user experience, accountability, and scalability to re-examine and standardize the human assessment process for generative large language models (LLMs).
Consistency Training by Synthetic Question Generation for Conversational Question Answering
Hamed Hematian Hemati (Sharif University of Technology), Hamid Beigy (Sharif University of Technology)
Data SynthesisTransformerText
🎯 What it does: Enhancing the robustness of conversational question answering models against irrelevant historical information by augmenting conversation history with synthetic questions during training and employing consistency training.
ConstitutionalExperts: Training a Mixture of Principle-based Prompts
Savvas Petridis (Google Research), Nithum Thain (Google Research)
OptimizationExplainability and InterpretabilityLarge Language ModelPrompt EngineeringMixture of ExpertsText
🎯 What it does: This paper proposes the ConstitutionalExperts method, which optimizes the reasoning of large language models by learning a mixture of principle-based (rule-based) prompts (Experts).
Context Consistency between Training and Inference in Simultaneous Machine Translation
Meizhi Zhong (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
GenerationTransformerText
🎯 What it does: Investigate and address the performance degradation in synchronous machine translation (SiMT) caused by context inconsistency between training and inference, proposing a dual-objective context consistency training method;
Context versus Prior Knowledge in Language Models
Kevin Du (ETH Zürich), Ryan Cotterell (ETH Zürich)
Explainability and InterpretabilityTransformerText
🎯 What it does: This paper proposes two metrics based on mutual information—persuasion score and susceptibility score—to quantitatively measure the dependence of language models on contextual information and pre-trained knowledge when answering queries; experiments were conducted on the Pythia series models (70m–12b) using a synthetic dataset containing 122 relations and 200 entities (100 real entities + 100 pseudo entities) to investigate the relationship between context attributes (relevance, assertiveness, negation) and model persuasiveness; the validity of the metrics was verified through correlations with entity memory rate, training corpus co-occurrence frequency, and entity degree in knowledge graphs; meanwhile, the practical value of the metrics was demonstrated in two application scenarios: friend/enemy relationship detection and gender bias studies.
Context-aware Difference Distilling for Multi-change Captioning
Yunbin Tu (University of Chinese Academy of Sciences), Qingming Huang (University of Chinese Academy of Sciences)
RecognitionGenerationTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Proposed the Context-aware Difference Distilling (CARD) network, which can automatically identify and describe all real changes between image pairs in varying scenarios, particularly demonstrating robustness to perturbations such as viewpoint changes and occlusions.
Continual Learning with Semi-supervised Contrastive Distillation for Incremental Neural Machine Translation
Yunlong Liang (Beijing Jiaotong University), Jie Zhou (Tencent Inc)
Knowledge DistillationRepresentation LearningTransformerContrastive LearningTextBenchmark
🎯 What it does: This paper proposes a semi-supervised contrastive distillation (SCD) framework that allows incrementally expanding neural machine translation models to adapt to new domains without accessing old domain training data.
Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor
Sangwon Yu (Seoul National University), Sungroh Yoon (Seoul National University)
GenerationScore-based ModelText
🎯 What it does: Propose a Score-based Progressive Editor (ScoPE), which achieves control over target attributes by editing intermediate text block-by-block during the generation process of a black-box language model, while maintaining text fluency.
Conundrums in Cross-Prompt Automated Essay Scoring: Making Sense of the State of the Art
Shengjie Li (University of Texas at Dallas), Vincent Ng (University of Texas at Dallas)
ClassificationText
🎯 What it does: Propose a cross-prompt automatic essay scoring method based solely on features, utilizing a simple neural network and feature selection.
CopyNE: Better Contextual ASR by Copying Named Entities
Shilin Zhou (Soochow University), Baoxing Huai (Huawei Cloud)
RecognitionRecurrent Neural NetworkTransformerTextAudio
🎯 What it does: Propose an end-to-end ASR model called CopyNE, which directly copies complete entities from a pre-built named entity (NE) dictionary during decoding, avoiding entity incompleteness or errors caused by character-by-character generation.
COSMIC: Mutual Information for Task-Agnostic Summarization Evaluation
Maxime Darrin (International Laboratory on Learning Systems), Pablo Piantanida (International Laboratory on Learning Systems)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a task-agnostic evaluation method called COSMIC based on mutual information, which measures the ability of summaries to retain original text information and support downstream tasks.
CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers
Longwei Zou (Tsinghua University), Yangdong Deng (Tsinghua University)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: By identifying 'quasi-independent layers' with similar inputs in adjacent layers of LLMs, parallel computation of these layers is achieved, significantly reducing inference latency.
Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference
Jihwan Bang (Qualcomm AI Research), Simyung Chang (Qualcomm AI Research)
Safty and PrivacyComputational EfficiencyRepresentation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the Crayon scheme, achieving LLM customization on the device without training through immediate hybrid LoRA pool, combined with device-server hybrid inference to enhance performance
CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation
Pei Ke (Tsinghua University), Minlie Huang (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Developed a multi-path prompting method called Eval-Instruct, automatically generating high-quality evaluation data for two types of tasks: reference-based and reference-free, critique and counter-critique; and fine-tuned the CRITIQUELLM model based on this data;
Cross-Lingual Knowledge Editing in Large Language Models
Jiaan Wang (Fudan University), Fandong Meng (Pattern Recognition Center, WeChat AI, Tencent Inc)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Perform cross-lingual knowledge editing on multilingual large language models and construct the Bi-ZsRE dataset.
Cross-Modal Projection in Multimodal LLMs Doesn’t Really Project Visual Attributes to Textual Space
Gaurav Verma (Georgia Institute of Technology), Srijan Kumar (Georgia Institute of Technology)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImage
🎯 What it does: Fine-tuning domain-specific visual capabilities in multi-modal large language models (e.g., LLaVA-1.5), comparing the effects of fine-tuning only the cross-modal projection layer versus end-to-end fine-tuning, and evaluating the domain-specific richness of projected image representations using an independent MLP. Results show that the projection layer does not effectively capture domain-specific visual attributes, which are primarily learned by the LLM parameters.
CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers
Yong Hu (Tencent Inc.), Jie Zhou (Tencent Inc.)
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Proposed CSCD-NS — the first spelling correction dataset tailored for native Chinese speakers, generated high-quality pseudo data by simulating input method editors (IME), aiming to enhance model performance in real-world scenarios.
D2LLM: Decomposed and Distilled Large Language Models for Semantic Search
Zihan Liao (East China Normal University), Wei Zhang (Ant Group)
RetrievalKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Proposed a semantic retrieval model named D2LLM, which decomposes the cross-encoder knowledge of LLM into dual encoders, a Pooling Attention Module (PMA), and an Interaction Simulation Module (IEM), and achieves knowledge distillation through contrastive, ranking, and feature simulation techniques, balancing high accuracy and efficiency;
DAPR: A Benchmark on Document-Aware Passage Retrieval
Kexin Wang (Technical University of Darmstadt), Iryna Gurevych (Technical University of Darmstadt)
RetrievalTextBenchmark
🎯 What it does: Propose the Document-Aware Passage Retrieval (DAPR) task and establish a benchmark across five cross-domain datasets;
DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows
Ajay Patel (University of Pennsylvania), Chris Callison-Burch (University of Pennsylvania)
Data SynthesisTransformerLarge Language ModelText
🎯 What it does: Provide a unified Python library called DataDreamer for building, executing, and recording multi-stage workflows based on LLMs (e.g., synthetic data generation, evaluation, fine-tuning, distillation), automatically implementing caching, resumability, and reproducibility features.
Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion
Wei Cheng (Nanjing University), Wei Hu (Nanjing University)
AI Code AssistantPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: Propose a retrieval-enhanced scheme DRACO based on data flow analysis for private warehouse-level code completion
DDPrompt: Differential Diversity Prompting in Large Language Models
Lin Mu (Anhui University), Peiquan Jin (University of Science and Technology of China)
TransformerPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose a Differential Diversity Prompt (DDPrompt) method that enhances LLM reasoning performance by automatically generating optimal trigger sets for different question types and obtaining answers through voting.
Deciphering Hate: Identifying Hateful Memes and Their Targets
Eftekhar Hossain (Chittagong University of Engineering & Technology), Sarah M. Preum (Dartmouth College)
ClassificationRecognitionTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: Constructed a multimodal hate meme dataset containing 7,148 Bengali and mixed-language captions, and implemented a dual-task learning framework for hate meme detection and target entity recognition.
Deciphering Oracle Bone Language with Diffusion Models
Haisu Guan (Anyang Normal University), Yuliang Liu (Huazhong University of Science and Technology)
Image TranslationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Propose Oracle Bone Script Decipher (OBSD) based on conditional diffusion models, utilizing local structure sampling and zero-shot refinement to map ancient characters to modern Chinese characters.
Decoder-only Streaming Transformer for Simultaneous Translation
Shoutao Guo (Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences), Yang Feng (Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences)
GenerationTransformerText
🎯 What it does: Proposed the first real-time translation model DST (Decoder-only Streaming Transformer) based on a Decoder-only architecture, achieving end-to-end adaptive translation strategies by separating source/target prefix position encodings and designing Streaming Self-Attention (SSA).
DeCoT: Debiasing Chain-of-Thought for Knowledge-Intensive Tasks in Large Language Models via Causal Intervention
Junda Wu (University of California San Diego), Julian McAuley (University of California San Diego)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes the DeCoT method, which eliminates internal knowledge bias in large language models through causal intervention, thereby improving the accuracy of chain-of-thought (CoT) in knowledge-intensive tasks.
Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages
Carlos Mullov (Karlsruhe Institute of Technology), Alexander Waibel (Karlsruhe Institute of Technology)
TransformerContrastive LearningText
🎯 What it does: Proposes a decoupled lexical learning approach for multilingual NMT, first training word vectors on monolingual corpora and performing cross-lingual alignment, then freezing the word embeddings in the encoder to achieve zero-shot translation for unseen languages and apply it to unsupervised MT.
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Damai Dai (Peking University), Wenfeng Liang (Peking University)
TransformerLarge Language ModelMixture of ExpertsTextBenchmark
🎯 What it does: Propose the DeepSeekMoE model, achieving efficient expert specialization through fine-grained expert partitioning and shared expert isolation, thereby enhancing the performance of MoE language models while maintaining the parameter scale unchanged.
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM
Bochuan Cao (Pennsylvania State University), Jinghui Chen (Pennsylvania State University)
Computational EfficiencyAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: To address alignment-destroying attacks, the Robustly Aligned LLM (RA-LLM) framework is proposed. It leverages the alignment detection capabilities of pre-aligned LLMs and introduces mechanisms such as random deletion and Monte Carlo sampling to enhance robustness without requiring retraining.
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization
Zhexin Zhang (Tsinghua University), Minlie Huang (Tsinghua University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: Propose a few-shot prompt method without training and a contrastive fine-tuning scheme with training by introducing a target prioritization mechanism (safety prioritized over usefulness) in both the inference and training phases, to defend against jailbreak attacks in large language models.
Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts
Xuan-Phi Nguyen (DAMO Academy, Alibaba Group), Lidong Bing (DAMO Academy, Alibaba Group)
GenerationData SynthesisLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Construct unsupervised prompts (Linguistically‑Diverse Prompting, LDP) using synthetic examples from multiple high-resource languages, enabling large language models (LLMs) to achieve performance comparable to or better than supervised few-shot prompts on low-resource language generation tasks without using any supervised data.
Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models
Yida Zhao (ShanghaiTech University), Kewei Tu (ShanghaiTech University)
GenerationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Proposed a dependency-aware Transformer grammar (DTG), achieving joint modeling of sentences and their dependency structures by generating transition sequences for both sentences and their dependency trees.
Detection-Correction Structure via General Language Model for Grammatical Error Correction
Wei Li (Peking University), Houfeng Wang (Peking University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed DeCoGLM, an integrated detection-correction framework based on GLM, which can perform error detection and local correction within a single model.
DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy
Hongda Sun (Renmin University of China), Rui Yan (Renmin University of China)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Proposed the DetermLR framework, which evolves the logical reasoning process of LLMs from uncertainty to certainty, and enhances reasoning effectiveness through premise identification, prioritization, and reasoning memory modules.
Detoxifying Large Language Models via Knowledge Editing
Mengru Wang (Zhejiang University), Huajun Chen (Westlake University)
Safty and PrivacyLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposes research on detoxifying large language models through knowledge editing methods, constructing a new safety evaluation benchmark called SafeEdit, and introducing a simple and efficient method based on toxicity region localization and editing called DINM.