These 518 ACL 2025 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every ACL 2025 paper, free trial on arXivSub.
500xCompressor: Generalized Prompt Compression for Large Language Models
Zongqian Li (University of Cambridge), Nigel Collier (University of Cambridge)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringAuto EncoderText
π― What it does: Propose 500xCompressor, a soft prompting compression method that can compress long texts into a single special token while retaining most of the original functionality.
A Dual-Mind Framework for Strategic and Expressive Negotiation Agent
Yutong Liu (Jilin University), Hao Xu (Jilin University)
CodeTransformerLarge Language ModelText
π― What it does: Proposes the Dual-Mind Negotiation Agent (DMNA) framework, combining intuitive fast experience response with deliberate multi-dimensional reflection to enhance negotiation agents' strategy planning and expression quality.
A Dual-Perspective NLG Meta-Evaluation Framework with Automatic Benchmark and Better Interpretability
Xinyu Hu (Wangxuan Institute of Computer Technology, Peking University), Xiaojun Wan (Wangxuan Institute of Computer Technology, Peking University)
CodeGenerationExplainability and InterpretabilityLarge Language ModelContrastive LearningTextBenchmark
π― What it does: Proposed a dual-perspective NLG meta-evaluation framework (global perspective and local perspective), and implemented an automatic benchmark construction method without requiring human annotation;
π― What it does: Proposed a deep generative adaptive replay continuous learning framework named DGAR to alleviate catastrophic forgetting in temporal knowledge graph reasoning.
CodeRepresentation LearningTransformerLarge Language ModelText
π― What it does: This paper reinterprets Zipf's Law of Meaning Frequency by measuring the variability in the direction of context vectors, avoiding the use of a dictionary.
CodeData SynthesisTransformerLarge Language ModelMixture of ExpertsText
π― What it does: Propose a multi-small-model collaborative framework GRA, which replaces a single large model by dividing tasks into three stages: generator, reviewer, and adjudicator, to accomplish data synthesis and quality control;
A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
Junyu Luo (Peking University), Ming Zhang (Peking University)
CodeKnowledge DistillationData-Centric LearningTransformerLarge Language ModelReview/Survey Paper
π― What it does: This paper provides a systematic review of data-efficient methods in the post-training of large language models, proposing a 'data flywheel' framework based on data centers and a five-category taxonomy.
A Training-free LLM-based Approach to General Chinese Character Error Correction
Houquan Zhou (Soochow University), Min Zhang (Soochow University)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: Propose the general Chinese character error correction (C2EC) task and construct a high-quality real error dataset, while extending an unsupervised prompt-free training framework to handle insertion, deletion, and substitution errors.
π― What it does: Proposed a three-view framework called TACO, which jointly learns instance-label, instance-instance, and label-label embeddings to address the challenges of distinguishing similar sentiments and mitigating long-tail sentiment bias in fine-grained sentiment classification.
A-TASC: Asian TED-Based Automatic Subtitling Corpus
Yuhan Zhou (University of Tokyo), Naoki Yoshinaga (University of Tokyo)
CodeData-Centric LearningTextBenchmarkAudio
π― What it does: Proposed A-TASC, an Asian TED benchmark automatic captioning dataset covering Chinese, Japanese, Korean, and Vietnamese with approximately 800 hours of data; developed SacreSubER evaluation metric to adapt to non-space languages; evaluated performance of end-to-end and pipeline captioning systems.
Accelerating Dense LLMs via L0-regularized Mixture-of-Experts
Zhenyu Zhang (YZW), Meng Chen (Wise AI)
CodeComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsText
π― What it does: Construct a lightweight Mixture-of-Experts (MoE) model using L0 regularization, accelerating inference of large language models with a small corpus of only 30B tokens while maintaining or improving performance.
Accurate KV Cache Quantization with Outlier Tokens Tracing
Yi Su (Soochow University), Min Zhang (Huawei Cloud)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelText
π― What it does: This paper addresses the quantization issue of LLM KV cache, finding that a small number of abnormal tokens significantly affect quantization accuracy. It proposes the OTT method to dynamically identify and exclude these outlier tokens, retaining them in full precision to improve the accuracy of 2bit quantization, while reducing memory usage and increasing throughput.
Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference
Siyuan Wang (University of Southern California), Zhongyu Wei (Fudan University)
CodeComputational EfficiencyTransformerLarge Language ModelVision Language ModelMultimodality
π― What it does: This paper studies identifying and activating visual regions within large vision-language models (LVLM), proving that updating approximately 25% of sparsely uniformly distributed layers can maintain 99% visual performance while preserving text capabilities.
AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness
Zixin Chen (Beijing University Of Posts And Telecommunications), Jing Ma (Hong Kong Baptist University)
CodeSafty and PrivacyTransformerLarge Language ModelAgentic AIMultimodalityBenchmark
π― What it does: Proposes the AdamMeme framework, which utilizes multi-agent dynamic evaluation to assess the understanding of harmful memes by multi-modal LLMs.
Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
Viktor Moskvoretskii (Skoltech), Alexander Panchenko (Skoltech)
CodeRetrievalComputational EfficiencyTransformerLarge Language ModelText
π― What it does: Evaluated and compared the performance, efficiency, and self-awareness capabilities of 35 adaptive retrieval methods and 27 uncertainty estimation approaches in QA, demonstrating that uncertainty methods significantly reduce retrieval and LLM calls while maintaining accuracy.
Addressing Blind Guessing: Calibration of Selection Bias in Multiple-Choice Question Answering by Video Language Models
Olga Loginova (University of Trento), Alexey Kravets (University of Bath)
CodeExplainability and InterpretabilityVision Language ModelVideoTextMultimodality
π― What it does: Through systematic experiments on video multiple-choice questions (MCQA), this paper analyzes and quantifies the selection bias of video language models (VLMs) in answer positions, proposing a post-processing calibration method called BOLD (and its weighted version Weighted_BOLD) to reduce bias and improve accuracy.
π― What it does: Improving the intelligibility of zero-shot TTS, the Intelligibility Preference Speech Dataset (INTP) was constructed, and post-training was conducted on multiple TTS architectures using Preference Alignment (Direct Preference Optimization DPO and its extensions).
Adverse Event Extraction from Discharge Summaries: A New Dataset, Annotation Scheme, and Initial Findings
Imane Guellil (University of Edinburgh), Beatrice Alex (University of Edinburgh)
CodeAnomaly DetectionRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical DataElectronic Health Records
π― What it does: This paper constructs a manually annotated adverse event corpus for elderly discharge summaries and provides corresponding annotation guidelines.
AgentRM: Enhancing Agent Generalization with Reward Modeling
Yu Xia (Tsinghua University), Maosong Sun (Tsinghua University)
CodeLarge Language ModelReinforcement LearningTextBenchmark
π― What it does: Introduce a generalizable reward model AgentRM in LLM-based agents, guiding the policy model to search during testing through the reward model, thereby improving performance on unknown tasks.
Agri-CM^3: A Chinese Massive Multi-modal, Multi-level Benchmark for Agricultural Understanding and Reasoning
Haotian Wang (Harbin Institute Of Technology), Jingchi Jiang (Harbin Institute Of Technology)
CodeVision Language ModelImageTextMultimodalityBenchmarkAgriculture RelatedChain-of-Thought
π― What it does: Proposed and constructed Agri-CM 3, a multi-modal multi-level benchmark for crop pest and disease management, to evaluate the performance of multi-modal large language models across three levels: perception, hybrid cognitive reasoning, and knowledge application.
AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark
Jianlyu Chen (University of Science and Technology of China), Zheng Liu (Beijing Academy of Artificial Intelligence)
CodeRetrievalTransformerLarge Language ModelTextBenchmark
π― What it does: Proposed AIR-BENCHβa fully automated, heterogeneous, and dynamically scalable retrieval evaluation benchmark that leverages large language models to automatically generate multi-task, multi-domain, and multi-lingual test sets;
ALGEN: Few-shot Inversion Attacks on Textual Embeddings via Cross-Model Alignment and Generation
Yiyi Chen (Aalborg University), Johannes Bjerva (Aalborg University)
CodeRepresentation LearningAdversarial AttackTransformerLarge Language ModelText
π― What it does: Propose a cross-model alignment and generation-based few-shot text embedding inversion attack (ALGEN), which can recover the original text with only a minimal number of leaked samples.
Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study
Bolei Ma (LMU Munich), Matthias AΓenmacher
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Evaluate the algorithmic fidelity of large language models (LLMs) in generating German public opinions (open-ended questions), comparing how different models reproduce subgroup perspectives.
Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback
Guan-Ting Lin (Amazon AGI), Ivan Bulyko (Amazon AGI)
CodeGenerationOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAudio
π― What it does: Propose the Align-SLM framework, which uses preference optimization (RLAIF+DPO) with AI (LLM) feedback to semantically align speech-language models (SLM) without text, improving performance in speech-to-speech generation tasks.
Aligning VLM Assistants with Personalized Situated Cognition
Yongqi Li (Wuhan University), Tieyun Qian (Wuhan University)
CodeData-Centric LearningReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningPrompt EngineeringVision Language ModelMultimodalityBenchmark
π― What it does: Constructed the PCogAlignBench dataset and proposed the PCogAlign framework, enabling Vision-Language Models (VLMs) to generate contextualized personalized responses based on individual role sets.
π― What it does: Utilizing translation references as supervisory signals, combined with constrained beam search to generate diverse synthetic translations, and annotating synthetic data through a dedicated annotator and the Shortest Phrase Coverage Error (SPCE) algorithm to alleviate the distribution shift problem in machine translation quality estimation.
AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment
Anastasia Ivanova, Aleksandr Panov
CodeData-Centric LearningTransformerLarge Language ModelTextBenchmark
π― What it does: Propose and release the AmbiK dataset, which contains 1000 pairs of ambiguous and clear tasks in kitchen environments, and evaluate various LLM-based ambiguous detection methods on this benchmark.
π― What it does: Propose a two-stage curriculum learning framework to enhance the performance and training speed of sequence labeling models, particularly reducing training burden when fusing heterogeneous knowledge.
An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning
Wei Sun (Institute of Automation, Chinese Academy of Sciences), Jiajun Zhang (Institute of Automation, Chinese Academy of Sciences)
CodeComputational EfficiencyData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelTextChain-of-Thought
π― What it does: Built an efficient and accurate process-supervised data construction framework called EpicPRM, and used it to generate the Epic50k dataset with 50k steps for training a Process-supervised Reward Model (PRM) for mathematical reasoning.
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT)
Laurie Burchell (University Of Edinburgh), Jaume Zaragoza-Bernabeu (Prompsit Language Engineering)
CodeData-Centric LearningTransformerTextBenchmark
π― What it does: Built and publicly released HPLT v2, a large-scale multilingual (193 languages) high-quality monolingual and parallel corpus, covering approximately 8 trillion tokens, 52 trillion characters, and providing 380 million English parallel sentence pairs.
Analyzing the Rapid Generalization of SFT via the Perspective of Attention Head Activation Patterns
Yang Zhao (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper analyzes the changes in attention head activation patterns during supervised fine-tuning (SFT) using gradient methods, revealing the mechanism of rapid generalization in large language models (LLMs).
AndroidGen: Building an Android Language Agent under Data Scarcity
Hanyu Lai (Tsinghua University), Jie Tang (Tsinghua University)
CodeData-Centric LearningAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextSequentialBenchmarkRetrieval-Augmented GenerationChain-of-Thought
π― What it does: This paper proposes the ANDROIDGEN framework, integrating four modulesβExpSearch, ReflectPlan, AutoCheck, and StepCriticβto enhance the reasoning and execution capabilities of large language models (LLMs) in Android environments under data-scarce conditions, and generates training data through this framework to further train an open-source Android language agent;
CodeTransformerLarge Language ModelSupervised Fine-TuningAgentic AIVision Language ModelVision-Language-Action ModelImageTextMultimodalityBenchmark
π― What it does: This paper constructs the ANDROΔ°DLAB framework, which includes a unified operating environment, a reproducible benchmark with 138 tasks, and an Android instruction dataset, aiming to provide a complete ecosystem for training and evaluating Android autonomous agents;
π― What it does: Developed a distributed long-context inference framework APB based on approximate attention and sequence parallelism, which significantly improves inference speed during the prefilling phase.
APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts
Honghua Dong (University of Toronto), Xujie Si (University of Toronto)
CodeAI Code AssistantLarge Language ModelPrompt EngineeringText
π― What it does: Proposed a prompt programming language called APPL, which can seamlessly embed large language model prompts within Python code and automatically manage context, asynchronous parallelization of calls, tool calls, and tracking;
CodeImage TranslationGenerationTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
π― What it does: This paper introduces the ACON dataset, aiming to systematically evaluate cross-modal consistency in arbitrary-modal (imageβtext) conversion models and compare the performance of arbitrary-modal models with specialized models.
Are LLMs effective psychological assessors? Leveraging adaptive RAG for interpretable mental health screening through psychometric practice
Federico Ravenda (Euler Institute, UniversitΓ della Svizzera italiana), Noriko Kando (National Institute of Informatics)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: This paper proposes an adaptive retrieval-augmented generation (aRAG) framework that utilizes standardized psychological questionnaires to guide large language models (LLMs) in unsupervised questionnaire answering, and performs mental health screening based on these responses.
Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral
Shivani Kumar (University of Michigan), David Jurgens (University of Michigan)
CodeClassificationGenerationData SynthesisLarge Language ModelTextBenchmark
π― What it does: This paper introduces UNIMORAL, a unified multilingual moral reasoning dataset that integrates moral dilemmas from psychological foundations and social media sources, annotated with action choices, ethical principles, influencing factors, and consequences, as well as annotators' moral and cultural profiles.
Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models
Jie Liu (City University of Hong Kong), Michael R. Lyu (Chinese University of Hong Kong)
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityBiomedical DataBenchmark
π― What it does: Constructed and publicly released a multimodal medical QA benchmark named Asclepius, covering 15 medical specialties, 8 categories of clinical capabilities, and comprising 3,232 questions, to evaluate the overall and granular performance of medical multimodal large language models (Med-MLLMs).
ASPERA: A Simulated Environment to Evaluate Planning for Complex Action Execution
Alexandru Coca (University of Cambridge), Anders Johannsen (Apple)
CodeAI Code AssistantTransformerLarge Language ModelTextBenchmark
π― What it does: Propose the ASPERA framework, combining digital assistant library simulation with human-computer interaction LLM data generation to achieve program generation and evaluation for complex action execution;
Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales
Maor Reuben (Ben-Gurion University of the Negev), Rami Puzis (Ben-Gurion University of the Negev)
CodeTransformerSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
π― What it does: Construct the EMPALC framework, which adapts psychological questionnaires into NLI prompts to assess and regulate latent psychological constructs (anxiety, depression, sense of coexistence) in pre-trained language models.
π― What it does: Proposed and implemented two training strategies to address retrieval inconsistency in multi-lingual audio-text retrieval: 1-to-K Contrastive Learning (KCL) and Audio-English Co-Anchor Contrastive Learning (CACL), improving retrieval recall and cross-lingual consistency by reducing data distribution errors during training.
Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark
Rong-Cheng Tu (Beijing Institute of Technology), Xian-Ling Mao (Beijing Institute of Technology)
CodeGenerationKnowledge DistillationLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
π― What it does: Construct an automated text-image evaluation framework based on task splitting, using GPT-4o to automatically generate training data and distill its evaluation capabilities into the open-source multimodal large model MiniCPM-V-2.6;
Batayan: A Filipino NLP benchmark for evaluating Large Language Models
Jann Railey Montalan (AI Singapore), William Chandra Tjhi (AI Singapore)
CodeClassificationGenerationLarge Language ModelTextBenchmark
π― What it does: Created and released BATAYAN, a comprehensive LLM evaluation benchmark for Filipino, covering three major NLP capabilities: understanding, reasoning, and generation, with 8 tasks, ensuring linguistic authenticity through localized translation and local review.
BelarusianGLUE: Towards a Natural Language Understanding Benchmark for Belarusian
Maksim Aparovich (Brno University of Technology), Pavel Smrz (Brno University of Technology)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Proposed BelarusianGLUEβa five-task natural language understanding benchmark for Belarusian, encompassing sentiment analysis, syntactic acceptability, word sense disambiguation, Winograd schemas, and textual entailment tasks;
Beyond Negative Stereotypes β Non-Negative Abusive Utterances about Identity Groups and Their Semantic Variants
Tina Lommel (Alpen Adria University Klagenfurt), Michael Wiegand (University of Vienna)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Studied the aggressiveness of non-negative stereotype sentences within identity groups, constructed a specialized dataset, and systematically analyzed the impact of semantic variations;
CodeRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
π― What it does: Semantic retrieval and relevance scoring are performed using LLMs to generate multi-label training data, followed by a two-stage pre-training plus fine-tuning approach to enhance dialogue recommendation performance.
Beyond the Answer: Advancing Multi-Hop QA with Fine-Grained Graph Reasoning and Evaluation
Qichuan Liu (Xiamen University), Zhihong Zhang (Xiamen University)
CodeExplainability and InterpretabilityData-Centric LearningGraph Neural NetworkTransformerLarge Language ModelTextGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Proposed the Planner-Executor-Reasoner (PER) framework, consisting of two modules: data preprocessing (PER-DP) and multi-hop reasoning (PER-QA), and introduced a fine-grained evaluation metric called Plan-aligned Stepwise Evaluation (PSE) to assess intermediate reasoning steps and final answers.
Chenye Zhao (University of Illinois Chicago), Cornelia Caragea (University of Illinois Chicago)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Studied bilingual zero-shot stance detection, constructed the Bi-STANCE dataset, evaluated cross-lingual, monolingual, and bilingual settings, covering noun phrases and stance targets, and incorporated low-word-overlap challenge samples.
BIPro: Zero-shot Chinese Poem Generation via Block Inverse Prompting Constrained Generation Framework
Xu Zou (Beijing Knowledge Atlas Technology Joint Stock Company Limited)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Proposed and implemented the BIPro framework, leveraging the intermediate text generation capability of block generation models through two reverse prompting methods, 'revise' and 'rewrite,' to generate open-domain traditional classical Chinese poetry under zero-shot conditions.
Giuliano Martinelli (Sapienza University of Rome), Roberto Navigli (Sapienza University of Rome)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Investigated the book-level coreference resolution task, proposed and implemented an automated annotation pipeline (BookCoref Pipeline), and constructed the first book-level coreference benchmark BOOKCOREF based on this.
BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving
Teng Wang (University of Hong Kong), Tao Zhong (Noah's Ark Lab, Huawei)
CodeOptimizationTransformerLarge Language ModelContrastive LearningTextBenchmarkChain-of-Thought
π― What it does: This paper proposes a new structured operations research dataset, StructuredOR, and develops the BPP-Search algorithm based on it to improve the accuracy and efficiency of Tree of Thought in solving mathematical modeling problems.
π― What it does: Propose Bregman Conditional Random Fields (BCRF) and a parallel inference algorithm based on iterative Bregman projections for sequence labeling.
Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention
Weixuan Wang (Monash University), Alexandra Birch (University of Edinburgh)
CodeDomain AdaptationComputational EfficiencyRepresentation LearningLarge Language ModelText
π― What it does: This paper proposes learning a cross-lingual alignment matrix during inference to map the internal representations of low-performance languages into the high-performance language space, thereby improving the performance of large language models on low-resource languages.
CodeTransformerLarge Language ModelAgentic AIMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper constructs the BLUR benchmark, which contains 573 real tip-of-the-tongue (ToT) questions, covering multimodal inputs such as text, images, and audio, as well as multilingual descriptions, requiring models to perform multi-hop reasoning, tool usage, and uncertainty handling.
Building a Long Text Privacy Policy Corpus with Multi-Class Labels
Florencia Marotta-Wurgler (NYU School of Law), David Stein (Northeastern University)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Constructed a manually annotated corpus covering 149 privacy policies and their referenced documents, containing 64-dimensional multi-class labels.
Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model
Emre Can Acikgoz (University of Illinois Urbana Champaign), Gokhan Tur (University of Illinois Urbana Champaign)
CodeTransformerLarge Language ModelSupervised Fine-TuningAgentic AITextChain-of-Thought
π― What it does: Propose CoALM, a unified conversational language model capable of simultaneously handling multi-turn task-oriented dialogue (TOD) and complex tool calls (LA)
Can Community Notes Replace Professional Fact-Checkers?
Nadav Borenstein (University of Copenhagen), Isabelle Augenstein (University of Copenhagen)
CodeTransformerLarge Language ModelText
π― What it does: This paper automatically annotates 1.5M Twitter/X community notes using a language model, generating a large-scale dataset containing attributes such as theme, source type, and whether professional fact-checking is cited. Based on this dataset, the paper analyzes the dependency and characteristics of community notes on professional fact-checking.
Can Language Models Reason about Individualistic Human Values and Preferences?
Liwei Jiang (University of Washington), Yejin Choi (Stanford University)
CodeTransformerLarge Language ModelPrompt EngineeringTextTabular
π― What it does: The study investigates whether language models can infer an individual's value judgments in new scenarios based on their expressed value statements, and proposes the INDIEVALUECATALOG dataset and the INDIEVALUEREASONER model grounded in the World Values Survey (WVS).
CodeGenerationTransformerLarge Language ModelTextChain-of-Thought
π― What it does: Construct a task for generating definitions of Chinese internet slang, propose a definition generation framework based on user-generated content (UGC), and collect and build the first Chinese slang dataset, CHEER.
Can LLMs Reason About Program Semantics? A Comprehensive Evaluation of LLMs on Formal Specification Inference
Thanh Le-Cong (University of Melbourne), Toby Murray (University of Melbourne)
CodeAI Code AssistantLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: This paper designs and implements FormalBench, a benchmark to evaluate the reasoning capabilities of large language models (LLMs) in generating complete and consistent program formal specifications (JML).
CodeRepresentation LearningConvolutional Neural NetworkRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextTime SeriesBiomedical Data
π― What it does: This paper explores using large language models (LLMs) to directly convert silent electromyography signals (non-vocalized EMG) into text, and proposes a trainable EMG adapter module;
Can Vision-Language Models Evaluate Handwritten Math?
Oikantik Nath (Indian Institute of Technology Madras), Mitesh M Khapra
CodeVision Language ModelImageTextMultimodalityBenchmarkChain-of-Thought
π― What it does: Designed and released the FERMAT benchmark to evaluate Vision-Language Models' capabilities in detecting, localizing, and correcting errors in handwritten mathematical answers, covering four error categories: calculation, conceptual, symbolic, and formatting errors.
Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks
Xingxuan Li (MiroMind), Lidong Bing (MiroMind)
CodeAI Code AssistantTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Designed and implemented the CR-Planner framework, which utilizes specialized critics to plan subgoals and execution between reasoning and retrieval, addressing complex tasks requiring deep reasoning and domain-specific knowledge retrieval.
Canβt See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
Wenxuan Wang (Renmin University of China), Zhaopeng Tu (Tencent)
CodeSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityBenchmark
π― What it does: Proposed the MMSafeAware benchmark, using 1500 image-text pairs to evaluate the safety awareness of multimodal large language models, covering subsets of safety and over-safety across 29 safety scenarios.
Causal Graph based Event Reasoning using Semantic Relation Experts
Mahnaz Koupaee (Stony Brook University), Niranjan Balasubramanian (University of Texas at Austin)
CodeExplainability and InterpretabilityTransformerLarge Language ModelAgentic AIMixture of ExpertsTextSequential
π― What it does: This paper proposes a collaborative causal graph generation framework that utilizes four specialized experts (temporal, discourse, preconditions, common sense) within a large language model to engage in multi-round discussions, ultimately generating a global event causal graph. This graph is then applied to various event reasoning tasks, including explainable event likelihood prediction, event prediction, and next-event prediction.
Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions
Xinbei Ma (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
CodeSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision-Language-Action ModelImageTextMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper investigates the trustworthiness of multimodal large language models (MLLMs) in graphical user interface (GUI) environments, exploring how distracting information in the environment leads agents to deviate from user goals and perform erroneous actions; it also constructs a simulated dataset with four typical scenarios to systematically evaluate various general and specialized GUI agents, verifying their robustness and safety against environmental disturbances.
CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Models
Yongheng Zhang (Central South University), Libo Qin (Central South University)
CodeLarge Language ModelImageTextMultimodalityBenchmarkChain-of-Thought
π― What it does: Propose the CCHall benchmark to evaluate large language models in joint cross-lingual and cross-modal scenarios for hallucination issues and systematically evaluate various MLLMs.
CEAES: Bidirectional Reinforcement Learning Optimization for Consistent and Explainable Essay Assessment
Xia Li (School of Information Science and Technology), Wenjing Pan (School of Information Science and Technology)
CodeOptimizationExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: Developed a model named CEAES that can simultaneously predict essay scores and generate explanatory feedback, employing a bidirectional reinforcement learning framework to mutually enhance scoring and feedback;
ChainEdit: Propagating Ripple Effects in LLM Knowledge Editing through Logical Rule-Guided Chains
Zilu Dong (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
CodeTransformerLarge Language ModelPrompt EngineeringTextGraphBenchmarkChain-of-Thought
π― What it does: Proposed the ChainEdit framework, using logic rule-guided chain updates to address the ripple effect problem in LLM knowledge editing.
ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents
Zhigen Li (Tianjin University), Deyi Xiong (Tianjin University)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
π― What it does: Propose ChatSOP, an MCTS planning framework guided by SOP, to enhance the controllability of task-oriented dialogues driven by LLMs.
Phan Anh Duong (University of Cincinnati), Tianyu Jiang (University of Cincinnati)
CodeRecognitionTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: This paper expands the original CHEER dataset to CHEER-Ekman, adding six basic emotion labels (happiness, sadness, anger, disgust, fear, and surprise), and employs a large language model (LLM) to complete emotion recognition through best-worst scaling (BWS) annotation and prompt engineering.
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Developed ChemActor, a 7B LLM fine-tuned chemical actuator that converts unstructured experimental descriptions into executable chemical steps.
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models
Yancheng He (Alibaba Group), Bo Zheng (Alibaba Group)
CodeLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
π― What it does: Created the Chinese SimpleQA evaluation benchmark for short question-answering, containing 3,000 short Q&A pairs covering six major topics and 99 subtopics, providing high-quality, static, and easily assessable characteristics.
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models
Philipp Mondorf (LMU Munich), Barbara Plank (LMU Munich)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerTextSequential
π― What it does: Investigate the modular structure of reusable subnetworks (circuits) in Transformer language models across ten string editing subtasks, and explore the feasibility of combining circuits.
Classifying Unreliable Narrators with Large Language Models
Anneliese Brei (University of North Carolina at Chapel Hill), Snigdha Chaturvedi (University of North Carolina at Chapel Hill)
CodeClassificationTransformerLarge Language ModelText
π― What it does: This paper proposes a method for automatically identifying three types of unreliable narrators (internal narration, cross-narration, cross-text) and constructs the corresponding expert-annotated dataset TUNA.
Yusuke Ide (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Constructed and released the CoAM dataset, containing 1,300 sentences covering all types of multi-word expressions (MWE), with type labels annotated for MWEs in the test set.
CodeRetrievalTransformerLarge Language ModelTextBenchmark
π― What it does: Propose COIR (Code Information Retrieval Benchmark) for end-to-end evaluation of code retrieval, covering four major retrieval tasks and ten multilingual datasets;
CoMet: Metaphor-Driven Covert Communication for Multi-Agent Language Games
Shuhang Xu (Beijing Normal University), Fangwei Zhong (Beijing Normal University)
CodeTransformerLarge Language ModelTextChain-of-Thought
π― What it does: Propose the CoMet framework to enhance LLMs' ability to use metaphors for covert communication and semantic evasion in multi-agent language games.
Comparing LLM-generated and human-authored news text using formal syntactic theory
Olga Zamaraeva (Universidade da CoruΓ±a), Carlos GΓ³mez-RodrΓguez (Universidade da CoruΓ±a)
CodeLarge Language ModelText
π― What it does: This paper utilizes the formal syntactic theory HPSG to compare the syntactic structure differences between LLM-generated texts in the New York Times style and texts written by human authors.
Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement
Maosong Cao (Shanghai AI Laboratory), Kai Chen (Shanghai AI Laboratory)
CodeData SynthesisReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningPrompt EngineeringText
π― What it does: Propose a two-stage data generation pipeline (Condor Void + Condor Refine), generating high-quality SFT data through knowledge trees and self-reflection to enhance the dialogue and knowledge answering capabilities of LLMs.
CONFETTI: Conversational Function-Calling Evaluation Through Turn-Level Interactions
Tamer Alkhouli (Amazon), Yi Zhang (Amazon)
CodeLarge Language ModelTextSequentialBenchmark
π― What it does: Propose the CONFETTI benchmark, which evaluates the multi-round function calling and answer quality of LLMs using 109 artificial dialogues.
ConLoan: A Contrastive Multilingual Dataset for Evaluating Loanwords
Sina Ahmadi (University of Zurich), Rico Sennrich (University of Zurich)
CodeData SynthesisTransformerLarge Language ModelContrastive LearningTextBenchmark
π― What it does: Created and annotated a contrastive sentence set called ConLoan in ten languages, where each sentence pair differs only by the loanword and its native alternative, and used this dataset to evaluate LLM surprise and NMT translation performance.
Continual Gradient Low-Rank Projection Fine-Tuning for LLMs
Chenxu Wang (Beijing Jiaotong University), Liping Jing (Beijing Jiaotong University)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Proposed the GORP strategy for large language models, jointly updating full parameters and low-rank parameters within a unified low-rank gradient subspace to achieve continuous fine-tuning;
π― What it does: This paper proposes a training-free and data-free inference-time intervention method called Contrastive Prompting (CP), which introduces auxiliary prompts and contrasts them with the original prompts to drive LLMs to better encode the core semantics of sentences in the activation vectors of multi-head attention layers, thereby obtaining higher quality sentence embeddings.
CodeTransformerLarge Language ModelAgentic AITextRetrieval-Augmented Generation
π― What it does: Propose the Genie framework, which defines tasks and knowledge queries using a declarative Genie Worksheet, separating the LLM's semantic parsing from runtime strategy execution, achieving reliable knowledge-intensive task-oriented dialogues.
π― What it does: Propose Controlled LoRA (CLoRA), a parameter-efficient fine-tuning method that incorporates subspace regularization into the LoRA structure, which can alleviate catastrophic forgetting during LLM continued training.
Cooperative or Competitive? Understanding the Interaction between Attention Heads From A Game Theory Perspective
Xiaoye Qu (Huazhong University of Science and Technology), Yu Cheng (Chinese University of Hong Kong)
CodeExplainability and InterpretabilityTransformerLarge Language ModelTextMultimodality
π― What it does: This paper first quantitatively analyzes the interactions between attention heads using Harsanyi dividend from cooperative game theory, revealing that most combinations yield nearly zero gains, while only a few combinations produce positive (cooperative) or negative (competitive) gains. Subsequently, a training-free Game-theoretic Attention Calibration (GAC) method is proposed, which first identifies attention head groups that significantly contribute to model performance, then performs fine-grained smoothing on the attention distributions of the remaining heads to suppress competition and enhance overall collaboration.
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningTextBenchmark
π― What it does: This paper proposes an improved Speculative Decoding framework called CORAL, which can significantly enhance the generation speed of large language models while maintaining lossless inference.
CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG
Yang Tian, Liqiang Nie (Harbin Institute of Technology)
CodeRetrievalLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelMultimodalityRetrieval-Augmented Generation
π― What it does: Propose a cross-source knowledge harmonization framework called CoRe-MMRAG, which follows a four-stage process: first generate answers using internal parameter knowledge, then select the most relevant evidence through joint visual and textual similarity, subsequently generate external answers based on retrieved multimodal evidence, and finally fuse internal and external responses to produce the final answer.
CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations
Vignesh Kothapalli (New York University), Maziar Sanjabi (LinkedIn AI)
CodeData SynthesisExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: This paper introduces CoT-ICL Lab, a controllable synthetic data framework for systematically studying the performance of chain-of-thought (CoT) in in-context learning (ICL).
Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models
Jongho Kim (Seoul National University), Seung-won Hwang (Seoul National University)
CodeLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: Proposes adversarial consistency prompts (CCP), which enhance the temporal consistency and reasoning accuracy of large language models by generating time counterfactual questions and aggregating answers.
Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
Jinghan He (Chinese Academy of Sciences), Jinqiao Wang (Chinese Academy of Sciences)
CodeExplainability and InterpretabilityRepresentation LearningTransformerMultimodality
π― What it does: This paper analyzes the multi-head attention mechanism and proposes the Vision-aware Head Divergence (VHD) metric to quantify the sensitivity of each attention head to visual information. Based on this, a training-free Vision-aware Head Reinforcement (VHR) method is developed to enhance visually sensitive attention heads during generation, thereby reducing hallucinations in LVLMs.
CRiskEval: A Chinese Multi-Level Risk Evaluation Benchmark Dataset for Large Language Models
Ling Shi (Tianjin University), Deyi Xiong (Tianjin University)
CodeSafty and PrivacyLarge Language ModelTextBenchmark
π― What it does: Constructed a Chinese-oriented frontier risk assessment benchmark dataset CRiskEval, using a fine-grained risk grading system and multiple-choice questions to evaluate the risk propensity of LLMs.
CritiQ: Mining Data Quality Criteria from Human Preferences
Honglin Guo (Fudan University), Tao Gui (Fudan University)
CodeData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AIText
π― What it does: Propose the CRITIQ method, which automatically mines and evolves data quality evaluation criteria using only about 30 manually annotated comparisons, then trains a lightweight scoring model with the generated criteria to select high-quality samples from large-scale text data.