ACL 2025 Papers — Page 2
Annual Meeting of the Association for Computational Linguistics · 1699 papers
An Empirical Study of Iterative Refinements for Non-autoregressive Translation
Yisheng Xiao (Soochow University), Min Zhang (Soochow University)
GenerationTransformerText
🎯 What it does: Conduct an in-depth analysis of the iterative refinement process in non-autoregressive machine translation models and propose more efficient improvement strategies.
An Empirical Study of Many-to-Many Summarization with Large Language Models
Jiaan Wang (Tencent Inc), Jie Zhou (Tencent Inc)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed a multilingual multi-document summarization dataset spanning five domains and six languages with 47.8K samples, systematically evaluating the multilingual multi-document summarization capabilities of 18 LLMs under zero-shot and instruction fine-tuning settings.
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT)
Laurie Burchell (University Of Edinburgh), Jaume Zaragoza-Bernabeu (Prompsit Language Engineering)
Data-Centric LearningTransformerTextBenchmark
🎯 What it does: Built and publicly released HPLT v2, a large-scale multilingual (193 languages) high-quality monolingual and parallel corpus, covering approximately 8 trillion tokens, 52 trillion characters, and providing 380 million English parallel sentence pairs.
Analyzing and Mitigating Inconsistency in Discrete Speech Tokens for Neural Codec Language Models
Wenrui Liu (Zhejiang University), Junyang Lin (Alibaba Group)
GenerationCompressionAuto EncoderGenerative Adversarial NetworkAudio
🎯 What it does: This paper quantitatively analyzes the Discrete Representation Inconsistency (DRI) phenomenon in neural audio encoders and proposes two constraint methods, slice consistency and perturbation consistency, to enhance the stability of discrete speech tokens, thereby improving speech reconstruction and generation effects.
Analyzing LLMs’ Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations
Chenghao Xiao (DAMO Academy, Alibaba Group), Yu Rong (DAMO Academy, Alibaba Group)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the perception of knowledge boundaries by large language models in multilingual environments, and analyzed differences in knowledge boundary cognition across languages and layers through probing of internal representations; proposed training-agnostic subspace alignment methods (mean shifting, linear projection) and a fine-tuning strategy based on bilingual question translation to enhance cross-lingual knowledge boundary recognition.
Analyzing the Rapid Generalization of SFT via the Perspective of Attention Head Activation Patterns
Yang Zhao (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper analyzes the changes in attention head activation patterns during supervised fine-tuning (SFT) using gradient methods, revealing the mechanism of rapid generalization in large language models (LLMs).
AndroidGen: Building an Android Language Agent under Data Scarcity
Hanyu Lai (Tsinghua University), Jie Tang (Tsinghua University)
Data-Centric LearningAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextSequentialBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes the ANDROIDGEN framework, integrating four modules—ExpSearch, ReflectPlan, AutoCheck, and StepCritic—to enhance the reasoning and execution capabilities of large language models (LLMs) in Android environments under data-scarce conditions, and generates training data through this framework to further train an open-source Android language agent;
AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents
Yifan Xu (Tsinghua University), Yuxiao Dong (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningAgentic AIVision Language ModelVision-Language-Action ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper constructs the ANDROİDLAB framework, which includes a unified operating environment, a reproducible benchmark with 138 tasks, and an Android instruction dataset, aiming to provide a complete ecosystem for training and evaluating Android autonomous agents;
AnRe: Analogical Replay for Temporal Knowledge Graph Forecasting
Guo Tang (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
TransformerLarge Language ModelPrompt EngineeringGraphTime SeriesBenchmark
🎯 What it does: Proposes a training-free analogy replay (AnRe) framework that utilizes LLMs and semantically-driven historical clustering along with dual historical retrieval for temporal knowledge graph prediction.
Answering Complex Geographic Questions by Adaptive Reasoning with Visual Context and External Commonsense Knowledge
Fan Li (Sun Yat-sen University), Jian Yin (Sun Yat-sen University)
RetrievalTransformerLarge Language ModelReinforcement LearningImageTextMultimodalityBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the GeoVQA task, answering geographic reasoning questions based on images.
AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge
Xiaobao Wu (Nanyang Technological University), William Yang Wang (University of California, Santa Barbara)
Data-Centric LearningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the AntiLeakBench framework to automatically construct a leakage-free question-answering evaluation benchmark based on the latest knowledge;
Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval
Zheng Liu (Hong Kong Polytechnic University), Defu Lian (University of Science and Technology of China)
RetrievalSupervised Fine-TuningVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: Proposes a unified visual information retrieval (Vis-IR) paradigm and constructs large-scale datasets, retrieval models, and evaluation benchmarks.
Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs
Xiulin Yang (Georgetown University), Ethan Wilcox (Georgetown University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Studies the learning ability of language models in multilingual environments regarding possible, infeasible, and unrecorded languages.
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
Yuxiang Huang (Tsinghua University), Maosong Sun (Tsinghua University)
Computational EfficiencyTransformerTextBenchmark
🎯 What it does: Developed a distributed long-context inference framework APB based on approximate attention and sequence parallelism, which significantly improves inference speed during the prefilling phase.
APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts
Honghua Dong (University of Toronto), Xujie Si (University of Toronto)
AI Code AssistantLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed a prompt programming language called APPL, which can seamlessly embed large language model prompts within Python code and automatically manage context, asynchronous parallelization of calls, tool calls, and tracking;
Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?
Jiwan Chung (Yonsei University), Youngjae Yu (Yonsei University)
Image TranslationGenerationTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: This paper introduces the ACON dataset, aiming to systematically evaluate cross-modal consistency in arbitrary-modal (image↔text) conversion models and compare the performance of arbitrary-modal models with specialized models.
Are LLMs effective psychological assessors? Leveraging adaptive RAG for interpretable mental health screening through psychometric practice
Federico Ravenda (Euler Institute, Università della Svizzera italiana), Noriko Kando (National Institute of Informatics)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes an adaptive retrieval-augmented generation (aRAG) framework that utilizes standardized psychological questionnaires to guide large language models (LLMs) in unsupervised questionnaire answering, and performs mental health screening based on these responses.
Are Optimal Algorithms Still Optimal? Rethinking Sorting in LLM-Based Pairwise Ranking with Batching and Caching
Juan Wisznia (Universidad de Buenos Aires), Luciano Del Corro (Universidad de Buenos Aires)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Redefine the cost model of pairwise ranking algorithms for LLMs, shifting from comparison counts to LLM inference call counts, and propose batching and caching optimizations.
Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral
Shivani Kumar (University of Michigan), David Jurgens (University of Michigan)
ClassificationGenerationData SynthesisLarge Language ModelTextBenchmark
🎯 What it does: This paper introduces UNIMORAL, a unified multilingual moral reasoning dataset that integrates moral dilemmas from psychological foundations and social media sources, annotated with action choices, ethical principles, influencing factors, and consequences, as well as annotators' moral and cultural profiles.
Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs
Giovanni Servedio (Politecnico di Bari), Tommaso Di Noia (Politecnico di Bari)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Investigate whether LLMs internally encode factual information, reproduce the experiment from Azaria & Mitchell 2023, and propose two more realistic data generation strategies (perplexity-guided negative sampling and self-assessment dataset generation by extracting factual content from QA datasets generated by LLMs)
Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media
Zhen Sun (Hong Kong University of Science and Technology (Guangzhou)), Xinlei He (Hong Kong University of Science and Technology (Guangzhou))
Anomaly DetectionExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper conducts large-scale data collection and analysis on social media platforms (Medium, Quora, Reddit), constructs the SM-D dataset, evaluates the AIGT detector using the AIGTBench dataset, and finally proposes OSM-Det to quantify and monitor the AI-generated text ratio (AAR) during the period of 2022-2024.
ARise: Towards Knowledge-Augmented Reasoning via Risk-Adaptive Search
Yize Zhang (Shanghai AI Laboratory), Chaochao Lu (Shanghai AI Laboratory)
RetrievalOptimizationComputational EfficiencyTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed and implemented the ARISE framework, which combines risk-adaptive Monte Carlo Tree Search (MCTS) with Retrieval-Augmented Generation (RAG) to recursively decompose and explore multi-branch reasoning for open-ended, knowledge-intensive complex reasoning problems through retrieval-enhanced inference.
Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework
Jundong Xu (National University of Singapore), Wynne Hsu (National University of Singapore)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposed the Aristotle framework, which integrates symbolic expressions and logical rules comprehensively in logical reasoning using LLM, covering three major modules: decomposition, search, and parsing, to achieve an efficient and accurate reasoning process.
Around the World in 24 Hours: Probing LLM Knowledge of Time and Place
Carolin Holtermann (University of Hamburg), Anne Lauscher (University of Bocconi)
Large Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Develop the GEOTEMP benchmark to evaluate the performance of large language models in joint time and location reasoning tasks.
Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models
Jie Liu (City University of Hong Kong), Michael R. Lyu (Chinese University of Hong Kong)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityBiomedical DataBenchmark
🎯 What it does: Constructed and publicly released a multimodal medical QA benchmark named Asclepius, covering 15 medical specialties, 8 categories of clinical capabilities, and comprising 3,232 questions, to evaluate the overall and granular performance of medical multimodal large language models (Med-MLLMs).
Ask-Before-Detection: Identifying and Mitigating Conformity Bias in LLM-Powered Error Detector for Math Word Problem Solutions
Hang Li (Michigan State University), Hui Liu (Michigan State University)
Anomaly DetectionTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This study investigates the consistency bias in LLM error detection caused by multiple solutions in mathematical problem-solving, and proposes the Ask-Before-Detection framework to alleviate this bias by automatically generating adaptive reference answers.
ASPERA: A Simulated Environment to Evaluate Planning for Complex Action Execution
Alexandru Coca (University of Cambridge), Anders Johannsen (Apple)
AI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the ASPERA framework, combining digital assistant library simulation with human-computer interaction LLM data generation to achieve program generation and evaluation for complex action execution;
Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks
Fangru Lin (University of Oxford), Furu Wei (Microsoft Research)
Large Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This study constructs the ReDial benchmark, collecting 1.2K+ standard English and non-standard African American English (AAVE) parallel reasoning tasks (algorithms, mathematics, logic, integrated reasoning) and conducting fairness and robustness evaluations on large language models (LLMs).
Assessing Reliability and Political Bias In LLMs’ Judgements of Formal and Material Inferences With Partisan Conclusions
Reto Gubelmann (University of Zurich), Ghassen Karray (University of Zurich)
Explainability and InterpretabilityData-Centric LearningLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper evaluates the reliability and political bias of 16 public and proprietary large language models (LLMs) in determining the validity of formal and material reasoning by constructing a set of logically equivalent reasoning pairs.
Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales
Maor Reuben (Ben-Gurion University of the Negev), Rami Puzis (Ben-Gurion University of the Negev)
TransformerSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
🎯 What it does: Construct the EMPALC framework, which adapts psychological questionnaires into NLI prompts to assess and regulate latent psychological constructs (anxiety, depression, sense of coexistence) in pre-trained language models.
Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
Fei Wang (Google Cloud AI Research), Sercan O Arik (Google Cloud AI Research)
RetrievalLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Propose a new retrieval-augmented generation framework ASTUTE RAG, which adaptively generates internal knowledge, performs source-aware knowledge integration, and finalizes answers to address retrieval noise and internal knowledge conflicts.
ATLANTIS: Weak-to-Strong Learning via Importance Sampling
Yi Liu (Peking University), Xu Sun (Peking University)
OptimizationData-Centric LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose ATLANTIS, a supervised fine-tuning technique leveraging importance sampling, which guides large language models to align more closely with the optimal data distribution in the real world by estimating sampling ratios through a small model and a reference model that has already been fine-tuned;
ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors
Yuguo Yin (Peking University), Yuexian Zou (Peking University)
RetrievalContrastive LearningTextMultimodalityAudio
🎯 What it does: Proposed and implemented two training strategies to address retrieval inconsistency in multi-lingual audio-text retrieval: 1-to-K Contrastive Learning (KCL) and Audio-English Co-Anchor Contrastive Learning (CACL), improving retrieval recall and cross-lingual consistency by reducing data distribution errors during training.
Attacking Vision-Language Computer Agents via Pop-ups
Yanzhe Zhang (Georgia Tech), Diyi Yang (Stanford University)
Adversarial AttackLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityBenchmark
🎯 What it does: This paper investigates attack methods against autonomous computer agents based on visual language models (VLM) by injecting malicious pop-ups into a visualization interface, demonstrating that pop-ups can significantly induce agents to perform incorrect operations.
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models
Zhisong Zhang (Tencent AI Lab), Dong Yu (Tencent AI Lab)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: The study investigates the performance degradation that occurs when replacing full self-attention in large language models with parallel context encoding (splitting context into blocks for parallel encoding) without additional fine-tuning, and explores its root causes.
Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models
Rishabh Adiga (University of Illinois Urbana-Champaign), Varun Chandrasekaran (University of Illinois Urbana-Champaign)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the ATLAS method, which first locates biased layers through attention distribution and then proportionally scales attention within these layers, thereby reducing bias in large language models during comparative prompts while maintaining generation quality.
AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context
Naba Rizvi (University of California San Diego), Nedjma Ousidhoum (Cardiff University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Constructed the AUTALIC dataset, collecting and annotating 2,400 sentences related to autism and their contexts from Reddit, with manual labeling on whether the sentences contain anti-autism discriminatory language.
Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussions
Ruochen Zhao (Nanyang Technological University), Lidong Bing (MiroMind)
TransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: Auto-Arena proposes a fully automated LLM evaluation framework, completing the entire process from question generation to ranking through LLM-generated questions, two-player adversarial debates between candidate models, and committee discussions led by LLMs.
AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs
Hongxin Li (University of Chinese Academy of Sciences), Zhaoxiang Zhang (University of Chinese Academy of Sciences)
Data-Centric LearningTransformerLarge Language ModelVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: Established the AutoGUI automatic annotation pipeline, which utilizes large language models (LLMs) to generate functional descriptions of UI elements by comparing UI state changes before and after interactions, and constructed the AutoGUI-704k dataset with a scale of 704k entries.
Automated CAD Modeling Sequence Generation from Text Descriptions via Transformer-Based Large Language Models
JianXing Liao, Xiaohong Guan (i4AI Ltd)
GenerationData SynthesisRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextPoint CloudMeshBenchmark
🎯 What it does: Designed a framework based on large language models, incorporating semi-automatic annotation, Transformer generation, and LLM enhancement to automatically generate high-quality CAD modeling sequences through text descriptions.
Automated Structured Radiology Report Generation
Jean-Benoit Delbrouck (HOPPR), Curtis Langlotz (Stanford AIMI)
ClassificationGenerationTransformerLarge Language ModelVision Language ModelTextBiomedical Data
🎯 What it does: Propose the Structured Chest X-ray Report Generation (SRRG) task, which rewrites free-text chest x-ray reports into standardized templates and constructs a corresponding dataset.
Automatic detection of dyslexia based on eye movements during reading in Russian
Anna Laurinavichyute (University of Potsdam), David Robert Reich
ClassificationRecurrent Neural NetworkTabularTime SeriesSequential
🎯 What it does: Developed a model based on bidirectional LSTM, utilizing eye movement sequences, demographic features, and lexical linguistic features recorded from Russian children during natural reading, to achieve automatic classification of dyslexia.
Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark
Rong-Cheng Tu (Beijing Institute of Technology), Xian-Ling Mao (Beijing Institute of Technology)
GenerationKnowledge DistillationLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Construct an automated text-image evaluation framework based on task splitting, using GPT-4o to automatically generate training data and distill its evaluation capabilities into the open-source multimodal large model MiniCPM-V-2.6;
Automatic Expert Discovery in LLM Upcycling via Sparse Interpolated Mixture-of-Experts
Shengzhuang Chen (Thomson Reuters Foundational Research), Jonathan Richard Schwarz (Thomson Reuters Foundational Research)
Computational EfficiencyLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: In single-stage instruction fine-tuning, we propose Sparse Interpolated Mixture-of-Experts (SIMoE) to convert a dense pre-trained large language model (LLM) into a sparse MoE model, automatically determining which layers require upcycling, while encouraging collaboration and specialization among experts through parameter sharing and orthogonal masks.
Automating Legal Interpretation with LLMs: Retrieval, Generation, and Evaluation
Kangcheng Luo (Peking University), Yansong Feng (Peking University)
Explainability and InterpretabilityTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose an automated legal concept explanation framework called ATRIE, which mimics the legal research process by using LLM to retrieve cases, extract reasoning, and generate explanations.
AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation
Xiechi Zhang (East China Normal University), Liang He (East China Normal University)
TransformerLarge Language ModelSupervised Fine-TuningTextBiomedical DataBenchmarkRetrieval-Augmented Generation
🎯 What it does: Built AutoMedEval, an automatic evaluation model tailored for the medical field, using a hierarchical training approach to construct an evaluation instruction set from medical QA datasets.
AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs
Nicholas E. Corrado (University of Wisconsin-Madison), Trishul Chilimbi (Amazon)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: Propose AutoMixAlign (AMA), a multi-task alignment framework that dynamically mixes data in DPO training based on the excess loss of tasks.
AutoMixer: Checkpoint Artifacts as Automatic Data Mixers
Ernie Chang (AI at Meta), Vikas Chandra
Data-Centric LearningTransformerLarge Language ModelText
🎯 What it does: By utilizing the emergent capabilities of intermediate checkpoint models during training, automatically generate data groups and provide task-related sampling weights for pretraining.
Autoregressive Speech Synthesis without Vector Quantization
Lingwei Meng (Chinese University of Hong Kong), Furu Wei (Microsoft Corporation)
GenerationTransformerAuto EncoderTextAudio
🎯 What it does: Proposes MELLE, a text-to-speech model that does not use vector quantization and directly performs autoregressive prediction on continuous mel-spectrograms;
AXIS: Efficient Human-Agent-Computer Interaction with API-First LLM-Based Agents
Junting Lu (Peking University), Qi Zhang (Microsoft)
Large Language ModelAgentic AITextRetrieval-Augmented Generation
🎯 What it does: Developed the AXIS framework, leveraging LLM agents to prioritize API calls over UI, automatically exploring applications and generating reusable skills to enhance task completion efficiency in Microsoft Word.
Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation
Yuxuan Zhou (University of Mannheim), Mario Fritz (CISPA Helmholtz Center for Information Security)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes a new evaluation framework to measure the trade-off between diversity and quality (risk) of LLM sampling strategies, and provides parameter selection guidelines for different sampling methods.
Balancing the Budget: Understanding Trade-offs Between Supervised and Preference-Based Finetuning
Mohit Raghavendra (Georgia Institute of Technology), Alan Ritter (Georgia Institute of Technology)
OptimizationComputational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: Systematically evaluate the resource allocation and performance of two post-training strategies, supervised fine-tuning (SFT) and preference fine-tuning (PFT), under fixed data budget constraints, covering different model scales, task types, and annotation costs;
Basic Reading Distillation
Zhi Zhou (Soochow University), Min Zhang (Soochow University)
Knowledge DistillationRepresentation LearningLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed and implemented a distillation method based on the 'basic reading' behavior of large models, termed Basic Reading Distillation (BRD). First, a teacher LLM (e.g., Vicuna-13B) generates basic reading behaviors such as named entity recognition (NER) and question generation and answering (QRA) on general-purpose text. These behaviors are then mixed with the original sentences to train a student small model, and the model's performance is evaluated on multiple downstream tasks.
Batayan: A Filipino NLP benchmark for evaluating Large Language Models
Jann Railey Montalan (AI Singapore), William Chandra Tjhi (AI Singapore)
ClassificationGenerationLarge Language ModelTextBenchmark
🎯 What it does: Created and released BATAYAN, a comprehensive LLM evaluation benchmark for Filipino, covering three major NLP capabilities: understanding, reasoning, and generation, with 8 tasks, ensuring linguistic authenticity through localized translation and local review.
Battling against Tough Resister: Strategy Planning with Adversarial Game for Non-collaborative Dialogues
Haiyang Wang (National University of Defense Technology), Bin Zhou (National University of Defense Technology)
Adversarial AttackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: Propose the GAIA framework, which constructs a zero-sum game in non-cooperative dialogues and learns strategy planning through chain-of-thought reasoning (CoM) and diverse opponent training;
BeamLoRA: Beam-Constraint Low-Rank Adaptation
Naibin Gu (Chinese Academy of Sciences), Haifeng Wang (Baidu Inc)
OptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Analyze the importance of ranks in the LoRA module in both spatial and temporal dimensions, propose the BeamLoRA method, which dynamically prunes low-importance ranks and expands high-importance ranks during fine-tuning, thereby improving performance while keeping the parameter scale unchanged.
BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
Lindia Tjuatja (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Proposed BEHAVIORBOX, an automated language model behavior comparison pipeline, used to discover performance differences between two models in different fine-grained contexts.
Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey
Ivan Vegner (University of Edinburgh), Leonidas A. A. Doumas
Explainability and InterpretabilityImageTextReview/Survey Paper
🎯 What it does: This paper provides a review of systematicity in machine learning models, distinguishing between behavioral systematicity and representational systematicity. Based on the Fodor & Pylyshyn theory and Hadley's three-tier systematicity framework, the paper conducts hierarchical analysis of commonly used evaluation datasets in the language and vision domains, discussing the role of mechanism explanation techniques in assessing representational systematicity.
Behind Closed Words: Creating and Investigating the forePLay Annotated Dataset for Polish Erotic Discourse
Anna Kołos (NASK National Research Institute), Agnieszka Karlińska (NASK National Research Institute)
ClassificationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Constructed and evaluated the Polish erotic content detection dataset forePLay
BelarusianGLUE: Towards a Natural Language Understanding Benchmark for Belarusian
Maksim Aparovich (Brno University of Technology), Pavel Smrz (Brno University of Technology)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Proposed BelarusianGLUE—a five-task natural language understanding benchmark for Belarusian, encompassing sentiment analysis, syntactic acceptability, word sense disambiguation, Winograd schemas, and textual entailment tasks;
BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering
Taolin Zhang (Hefei University of Technology), Xiaofeng He (East China Normal University)
Computational EfficiencyTransformerLarge Language ModelAgentic AITextChain-of-Thought
🎯 What it does: Proposed BELLE, a two-layer multi-agent reasoning framework for multi-hop QA, which generates executable operation plans by dynamically combining various reasoning operations (CoT, single-step, iterative step, sub-step, adaptation step) based on question types;
Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning
Yingjie Zhu (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
TransformerSupervised Fine-TuningVision Language ModelContrastive LearningMultimodalityGraphBenchmark
🎯 What it does: Proposes the VGCURE benchmark evaluation and the MCDGRAPH self-supervised fine-tuning framework to assess and enhance the ability of large vision-language models in understanding and reasoning about visual graph structures.
Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories
Alperen Yildiz (National University of Singapore), Dinil Mon Divakaran (Institute for Infocomm Research, A*STAR)
AI Code AssistantTransformerAgentic AIPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes the JITVUL benchmark for real-time vulnerability detection in code repositories, and evaluates large language models (LLMs) and ReAct-based agents.
Benchmarking Long-Context Language Models on Long Code Understanding
Jia Li (Peking University), Zhi Jin (Peking University)
AI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed the LONGCODEU benchmark to evaluate long-context language models (LCLMs) on eight tasks across four dimensions (code unit awareness, intra-unit understanding, inter-unit relationship understanding, and long-document understanding) in real-world project code (up to 128K tokens).
Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models
Kuofeng Gao, Jindong Gu (Tsinghua University)
TransformerLarge Language ModelBenchmarkAudio
🎯 What it does: Proposes ADU-Bench, an open audio dialogue understanding benchmark for large audio-language models (LALM), containing four sub-datasets;
BERT-like Models for Slavic Morpheme Segmentation
Dmitry Morozov (Artificial Intelligence Research Center of Novosibirsk State University), Olga Lyashevskaya (HSE University)
SegmentationTransformerSupervised Fine-TuningText
🎯 What it does: This paper conducts experiments on the morphological segmentation task by fine-tuning a BERT-like pre-trained model on Russian, Czech, and Belarusian, and first publicly releases the Belarusian morphological segmentation dataset.
Better Embeddings with Coupled Adam
Felix Stollenwerk (AI Sweden), Tobias Stollenwerk (Forschungszentrum Jülich)
OptimizationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Investigate the problem of anisotropy in word embeddings caused by the Adam optimizer and propose the Coupled Adam algorithm to improve embedding quality and model performance.
Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases
Michael Y. Hu (New York University), Tal Linzen (New York University)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: First pretrain on formal languages (e.g., Dyck, Shuffle Dyck, etc.), then continue training on natural language;
Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs
Weixiang Zhao (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A systematic assessment of safety risks caused by role-playing fine-tuning is conducted, and a safety-aware role-playing fine-tuning (SaRFT) method is proposed to balance role adaptability and safety performance.
Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals’ Subjective Text Perceptions
Matthias Orlikowski (Bielefeld University), Dirk Hovy (Bocconi University)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This study investigates whether large language models (LLMs) can predict individuals' annotation preferences in subjective text tasks based on sociodemographic attributes (age, gender, race, education level) through fine-tuning experiments; it also examines whether attribute prompts can replace individual identity information and their impact on predicting annotation disagreements.
Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model
Yeyong Yu (Shanghai University), Quan Qian (LIGHTSPEED)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposed the BEYOND DIALOGUE framework, constructed a low-cost automated role-dialogue alignment dataset, utilized LLM reasoning to achieve sentence-level alignment tasks, and implemented a general role-playing model through supervised fine-tuning and target evaluation.
Beyond Facts: Evaluating Intent Hallucination in Large Language Models
Yijie Hao (Emory University), Jiaxuan You (University of Illinois Urbana Champaign)
Explainability and InterpretabilityLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Proposed the concept of 'Intent Hallucination,' conducting systematic evaluation on the phenomenon where large language models ignore or misinterpret partial intentions in complex multi-condition queries.
Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent Systems
Haochun Wang (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
Computational EfficiencyTransformerLarge Language ModelAgentic AITextBiomedical DataElectronic Health Records
🎯 What it does: Systematically studied and quantified four fine-grained dimensions in multi-agent collaboration (governance, participation, interaction patterns, and context management). Through experiments on two types of tasks (DEI and SES), it was found that centralized governance centered on a teacher agent, teacher-led participation, and context summarization can significantly reduce token costs while maintaining high accuracy.
Beyond Logits: Aligning Feature Dynamics for Effective Knowledge Distillation
Guoqiang Gong (JD.com), Ke Zhang (JD.com)
Knowledge DistillationTransformerLarge Language ModelTextBenchmarkOrdinary Differential Equation
🎯 What it does: Propose the Feature Dynamics Distillation (FDD) method, which aligns the entire feature dynamics (i.e., feature trajectories and their first-order derivatives) between teacher and student models in knowledge distillation, rather than merely matching the final output distribution.
Beyond N-Grams: Rethinking Evaluation Metrics and Strategies for Multilingual Abstractive Summarization
Itai Mondshine (Bar-Ilan University), Reut Tsarfaty (Bar-Ilan University)
GenerationData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Systematically evaluated the correlation between n-gram and neural network evaluation metrics and human ratings across eight languages (covering isolated, agglutinative, low-inflectional, and high-inflectional language families).
Beyond Negative Stereotypes – Non-Negative Abusive Utterances about Identity Groups and Their Semantic Variants
Tina Lommel (Alpen Adria University Klagenfurt), Michael Wiegand (University of Vienna)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the aggressiveness of non-negative stereotype sentences within identity groups, constructed a specialized dataset, and systematically analyzed the impact of semantic variations;
Beyond One-Size-Fits-All: Tailored Benchmarks for Efficient Evaluation
Peiwen Yuan (Beijing Institute of Technology), Kan Li (Beijing Institute of Technology)
Computational EfficiencyData-Centric LearningTextMultimodalityBenchmark
🎯 What it does: To address the resource consumption issues in model evaluation, this paper proposes the TailoredBench method, which can dynamically construct personalized evaluation core sets (Native-coresets) for each model under evaluation, significantly reducing inference costs while maintaining accuracy.
Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning
Chengwei Qin (Hong Kong University of Science and Technology (Guangzhou)), Shafiq Joty (Princeton University)
Knowledge DistillationMeta LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper proposes the BiAlign method, which improves the performance of student models in context learning by simultaneously aligning the output distributions of student and teacher models and aligning the input preferences for different demonstration examples.
Beyond Position: the emergence of wavelet-like properties in Transformers
Valeria Ruscio (Sapienza University of Rome), Fabrizio Silvestri (Sapienza University of Rome)
Explainability and InterpretabilityRepresentation LearningTransformerText
🎯 What it does: This paper investigates how Transformer models equipped with rotational position encoding (RoPE) spontaneously generate wavelet-like multi-resolution characteristics during training, and verifies their compliance with the uncertainty principle through frequency domain, time-frequency domain, and information-theoretic analyses.
Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms
Mengru Wang (Zhejiang University), Ningyu Zhang (Zhejiang University)
Safty and PrivacyTransformerPrompt EngineeringAuto EncoderText
🎯 What it does: Proposes the Steering Target Atoms (STA) method, achieving fine-grained control over model behavior during LLM inference by identifying and manipulating 'atomic' features in the sparse autoencoder (SAE) decomposition space.
Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering
Zhanghao Hu (King's College London), Lin Gui (King's College London)
RetrievalComputational EfficiencyTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose an ODQA framework based on the embedding layer called EmbQA. First, fine-tune retrieval queries through unsupervised contrastive learning and re-rank retrieval results. Subsequently, introduce exploratory embeddings and entropy-based answer selection in the reader to reduce multi-round prompts and enhance answer diversity and accuracy.
Beyond Sequences: Two-dimensional Representation and Dependency Encoding for Code Generation
Xiangyu Zhang (Nanjing University of Aeronautics and Astronautics), Taolue Chen (Nanjing University of Aeronautics and Astronautics)
AI Code AssistantTransformerAuto EncoderText
🎯 What it does: Proposed a code generation model called CoDE based on 2D code representation and inter-line dependency encoding, decomposing code into a 2D structure of lines and intra-line tokens, and implementing intra-line attention with dependency vector fusion on Transformer.
Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
Yang Zhao, Ting Liu (Du Xiaoman Technology (Beijing) Co., Ltd.)
OptimizationRepresentation LearningData-Centric LearningGraph Neural NetworkLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the G2IS method, which achieves data selection for instruction tuning by constructing a gradient hybrid graph.
Beyond Single Labels: Improving Conversational Recommendation through LLM-Powered Data Augmentation
Haozhe Xu (Fudan University), Xiaoqing Zheng (Fudan University)
Recommendation SystemTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Semantic retrieval and relevance scoring are performed using LLMs to generate multi-label training data, followed by a two-stage pre-training plus fine-tuning approach to enhance dialogue recommendation performance.
Beyond Surface Simplicity: Revealing Hidden Reasoning Attributes for Precise Commonsense Diagnosis
Huijun Lian (Beijing University of Posts and Telecommunications), Ya Li (Beijing University of Posts and Telecommunications)
Explainability and InterpretabilityTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes the ReComSBench framework, which explicitly reveals hidden reasoning attributes in commonsense question answering and decouples the evaluation of knowledge and reasoning capabilities.
Beyond Text Compression: Evaluating Tokenizers Across Scales
Jonas F. Lotz (University of Copenhagen), Leonardo Emili (Apple)
CompressionComputational EfficiencyKnowledge DistillationRepresentation LearningData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Studied the impact of different tokenizers on language model performance, using a 350M small model to predict the performance of a 2.7B large model, and proposed an intrinsic evaluation metric based on Zipf's law.
Beyond the Answer: Advancing Multi-Hop QA with Fine-Grained Graph Reasoning and Evaluation
Qichuan Liu (Xiamen University), Zhihong Zhang (Xiamen University)
Explainability and InterpretabilityData-Centric LearningGraph Neural NetworkTransformerLarge Language ModelTextGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed the Planner-Executor-Reasoner (PER) framework, consisting of two modules: data preprocessing (PER-DP) and multi-hop reasoning (PER-QA), and introduced a fine-grained evaluation metric called Plan-aligned Stepwise Evaluation (PSE) to assess intermediate reasoning steps and final answers.
Beyond True or False: Retrieval-Augmented Hierarchical Analysis of Nuanced Claims
Priyanka Kargupta (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
ClassificationRetrievalTransformerLarge Language ModelTextBiomedical DataRetrieval-Augmented Generation
🎯 What it does: Built a retrieval-enhanced hierarchical analysis framework called CLAIMSPECT, which can automatically decompose complex claims into multi-level sub-claims and retrieve, cluster, and annotate supporting, opposing, or neutral opinions under each sub-claim.
BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Proving
Ran Xin (ByteDance Seed), Ming Ding
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Propose a scalable best-first tree search (BFS-Prover) based on large language models to achieve automated theorem proving, performing efficient search in the Lean4 environment;
Bi-Tuning with Collaborative Information for Controllable LLM-based Sequential Recommendation
Xinyu Zhang (Beijing Institute of Technology), Liqiang Nie (Harbin Institute of Technology (Shenzhen))
Recommendation SystemLarge Language ModelSupervised Fine-TuningPrompt EngineeringMixture of ExpertsContrastive LearningText
🎯 What it does: By adding learnable prefix and suffix tokens before and after the input text, LLMs can be adapted to sequential recommendation tasks, and the M-Former MoE structure is utilized to inject collaborative information into the prefix, achieving controllable LLM sequence recommendations.
Bias in Language Models: Beyond Trick Tests and Towards RUTEd Evaluation
Kristian Lum (University of Chicago), Alexander Nicholas D’Amour
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Evaluate gender-occupation bias in large language models (LLMs), comparing traditional 'trick test'-style benchmarks with the more realistic RUTEd assessment (long text generation scenarios), and explore the correlation between different evaluation methods.
Bias in the Mirror : Are LLMs opinions robust to their own adversarial attacks
Virgile Rennard (École Polytechnique), Michalis Vazirgiannis (École Polytechnique)
Adversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Evaluate the bias robustness of the same LLM across different languages, multi-models, and human comparisons by letting two instances of the same LLM debate each other in a structured debate, then having a third neutral instance re-respond.
Biased LLMs can Influence Political Decision-Making
Jillian Fisher (University of Washington), Katharina Reinecke (University of Washington)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Through two interactive experiments, the study investigated the impact of biased large language models (LLMs) on the political opinions and budget decisions of American Democratic and Republican participants.
BIG-Bench Extra Hard
Mehran Kazemi (Google DeepMind), Orhan Firat (Google DeepMind)
Large Language ModelTextBenchmark
🎯 What it does: Proposes the BIG-Bench Extra Hard (BBEH) benchmark, aiming to further test the broad reasoning capabilities of large language models (LLMs).
BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data
Wenkai Li (Carnegie Mellon University), Maarten Sap (Carnegie Mellon University)
Data SynthesisLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Construct the BIG5-CHAT large-scale human dialogue dataset, and use training methods such as SFT/DPO to enable LLMs to exhibit authentic Big Five personality traits in conversations, while evaluating their impact on reasoning tasks.
Bilingual Zero-Shot Stance Detection
Chenye Zhao (University of Illinois Chicago), Cornelia Caragea (University of Illinois Chicago)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Studied bilingual zero-shot stance detection, constructed the Bi-STANCE dataset, evaluated cross-lingual, monolingual, and bilingual settings, covering noun phrases and stance targets, and incorporated low-word-overlap challenge samples.
Binary Classifier Optimization for Large Language Model Alignment
Seungjae Jung (Kakao Corp), Kyoung-Woon On (LBOX)
OptimizationReinforcement Learning from Human FeedbackLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes a BCO-based LLM alignment method that can train high-quality alignment models using only binary feedback (e.g., upvotes/downvotes);
BIPro: Zero-shot Chinese Poem Generation via Block Inverse Prompting Constrained Generation Framework
Xu Zou (Beijing Knowledge Atlas Technology Joint Stock Company Limited)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed and implemented the BIPro framework, leveraging the intermediate text generation capability of block generation models through two reverse prompting methods, 'revise' and 'rewrite,' to generate open-domain traditional classical Chinese poetry under zero-shot conditions.
Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
Jinheng Wang (Peking University), Furu Wei (Microsoft Research)
Computational EfficiencyLarge Language ModelTextBenchmark
🎯 What it does: Proposed Bitnet.cpp, a complete inference system designed for efficient inference of 1-bit/3-bit ternary LLMs on edge devices.
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
Ercong Nie (LMU Munich), Hinrich Schuetze
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper constructs BMIKE-53, a cross-lingual knowledge editing benchmark covering 53 languages, and evaluates gradient-free knowledge editing methods based on prompting on it.