Annual Meeting of the Association for Computational Linguistics Β· 518 papers
Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models
Zixiang Xu (MBZUAI), Xiangliang Zhang (University of Notre Dame)
CodeExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: This paper proposes an automated method combining Beam Search and LLM-Simulation to efficiently generate bilingual question pairs, aiming to accurately reveal performance limitations of multilingual large language models (LLMs) on non-English languages. Based on this, a dataset containing over 6,000 bilingual samples (covering 16 languages) was constructed.
Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge
Qiyuan Zhang (City University of Hong Kong), Chen Ma (City University of Hong Kong)
CodeExplainability and InterpretabilityKnowledge DistillationLarge Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
π― What it does: Propose a population-based comparison evaluation method (CCE), which enriches the chain-of-thought (CoT) assessment of LLM-as-a-Judge by generating diverse group responses and comparing them with candidate answers.
π― What it does: This paper proposes a unified macro-structure driven argument mining model, CU-MAM, which achieves joint prediction of argument component types, relation identification, and their types by integrating local and global consistency of argument relations into a multi-task learning framework.
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLMβs Nest
Letian Peng (University of California San Diego), Jingbo Shang (University of California San Diego)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Transform the next-word prediction (NTP) of LLMs into next-word extraction (NTE), automatically generating BIO labeling using pre-training and fine-tuning data from C4 and TuluV3 to train a large-scale, few-shot, and instruction-driven IE model named Cuckoo.
CodeClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: This paper develops the CuLEmo benchmark to evaluate the performance of large language models in cross-cultural emotion recognition and sentiment analysis.
CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis
Ruixiang Feng (University of Electronic Science and Technology of China), Shuo Shang (University of Electronic Science and Technology of China)
CodeExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
π― What it does: Propose a multi-lingual, multi-dimensional cultural awareness training paradigm named CulFiT, leveraging target-aware multi-lingual criticism data and fine-grained reward mechanisms to enhance LLM's cultural sensitivity and diversity;
Cultivating Gaming Sense for Yourself: Making VLMs Gaming Experts
Wenxuan Lu (Institute of Information Engineering, Chinese Academy of Sciences), Tianning Zang (Institute of Information Engineering, Chinese Academy of Sciences)
CodeTransformerReinforcement LearningVision Language ModelImage
π― What it does: Propose the GameSense framework, enabling Vision Language Models (VLMs) to act as developers of gameplay modules, generating real-time execution modules (GSMs) to achieve uninterrupted, real-time game control.
CodeRetrievalData-Centric LearningLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Propose an adaptive example selection method based on demonstration verification (D.Va), which indirectly estimates and calibrates perplexity under the target task using retrieved validation examples to guide context selection in large language models.
DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression
Yi Zhao (Shanghai Jiao Tong University), Liu Guoming (Xiaomi)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose a task-agnostic prompt compression method (DAC) based on the fusion of dynamic attention and information entropy, maximizing information retention while reducing computational costs through multi-stage dynamic compression and attention-based key token constraints.
Data-Constrained Synthesis of Training Data for De-Identification
Thomas Vakili (Stockholm University), Hercules Dalianis (Stockholm University)
CodeData SynthesisDomain AdaptationSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
π― What it does: The study investigates the utility of synthetic clinical text generated by domain-adapted LLMs under data-limited conditions, training a NER model with machine-labeled PII entities to evaluate the effectiveness of synthetic data for downstream tasks.
DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph
Jihyung Lee (Sungkyunkwan University), Jee-Hyong Lee (Sungkyunkwan University)
CodeRetrievalAI Code AssistantGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextTabularBenchmarkRetrieval-Augmented Generation
π― What it does: Proposes a retrieval method based on a deep contextual pattern linking graph, which uses this graph to retrieve relevant demonstration samples in the prompt-based learning of large language models (LLMs), thereby generating more accurate SQL queries.
DDxTutor: Clinical Reasoning Tutoring System with Differential Diagnosis-Based Structured Reasoning
Qian Wu (Chinese University of Hong Kong), Qi Dou (Chinese University of Hong Kong)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
π― What it does: Propose the DDxTutor framework, utilizing structured reasoning based on differential diagnosis to achieve clinical diagnosis teaching.
CodeGenerationAI Code AssistantTransformerLarge Language ModelText
π― What it does: Proposes the DebateCoder framework, which improves code generation through test case-driven debates between two large language models (LLMs).
Deep Temporal Reasoning in Video Language Models: A Cross-Linguistic Evaluation of Action Duration and Completion through Perfect Times
Olga Loginova (University of Trento), SofΓa Ortega Loguinova (Maastricht University)
CodeTransformerLarge Language ModelVision Language ModelVideoTextMultimodalityBenchmark
π― What it does: This paper proposes a new cross-lingual video-language multimodal QA benchmark called Perfect Times, and uses it to evaluate the temporal reasoning capabilities of existing video language models.
DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process
Minjun Zhu (Zhejiang University), Yue Zhang (University College London)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextReview/Survey PaperBenchmarkRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Propose DeepReviewer, a multi-stage LLM paper review framework that simulates the expert review process, including innovation verification, multi-dimensional evaluation, and reliability verification, and construct the DeepReview-13K dataset and DeepReviewBench benchmark, training a 14B parameter model.
DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking
Zhuoqun Li (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Le Sun (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences)
π― What it does: This paper proposes a new benchmark called SolutionBench and a RAG system named SolutionRAG based on tree exploration and dual-point thinking, for automatically generating complex engineering solutions that satisfy multiple constraints.
Delving into Multilingual Ethical Bias: The MSQAD with Statistical Hypothesis Tests for Large Language Models
Seunguk Yu (Chung-Ang University), YoungBin Kim
CodeSafty and PrivacyExplainability and InterpretabilityLarge Language ModelText
π― What it does: This paper constructs a multilingual sensitive question-and-answer dataset called MSQAD and evaluates the ethical biases of large language models in different languages regarding sensitive questions through statistical hypothesis testing.
DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
Lin Mu (Anhui University), Yiwen Zhang (Anhui University)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: DenseLoRA introduces a shared Encoder-Decoder structure in large language models (e.g., LLaMA2-7B and LLaMA3-8B). It first compresses and refines the hidden representations, then adapts them using a dense low-rank matrix, significantly reducing the number of trainable parameters while maintaining the model's expressive power.
Design Choices for Extending the Context Length of Visual Language Models
Mukai Li (University of Hong Kong), Qi Liu (University of Hong Kong)
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodality
π― What it does: Built the GIRAFFE system, leveraging the ETVLM dataset, M-RoPE++ position embeddings, and hybrid-resolution training to successfully extend the context length of the Qwen-VL series of vision-language models to 128K while maintaining short-context performance.
Detecting Sockpuppetry on Wikipedia Using Meta-Learning
Luc Raszewski (University of Melbourne), Christine de Kock (University of Melbourne)
CodeClassificationAnomaly DetectionMeta LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: Detect malicious sockpuppetry on Wikipedia using meta-learning methods, treating each investigation as an independent task and performing binary classification based on written text;
Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition
Masato Mita (University of Tokyo), Yohei Oseki (University of Tokyo)
CodeTransformerLarge Language ModelText
π― What it does: By introducing an exponential growth limit on working memory in Transformer language model training, simulating the cognitive development during the critical period of human language acquisition, thus enhancing the model's syntactic learning efficiency.
Dialectal Coverage And Generalization in Arabic Speech Recognition
Amirbek Djanibekov (Mohamed bin Zayed University of Artificial Intelligence), Hanan Aldarmaki (Mohamed bin Zayed University of Artificial Intelligence)
π― What it does: Proposed an automatic speech recognition (ASR) model for multivariate Arabic (including Modern Standard Arabic, regional dialects, and code-switching spoken languages with English, French, etc.), and released pre-trained and fine-tuned model weights;
π― What it does: This paper proposes the DialUp method, which improves machine translation performance for low-resource dialects by manually dialectalizing high-resource languages (HRL) during training (MβD) and mixing dialect inputs with HRL codes during inference (DβM).
π― What it does: Investigated the generation capability of text-to-image diffusion models for cultural elements across ten countries, and constructed and evaluated the CULTDIFF benchmark.
Digest the Knowledge: Large Language Models empowered Message Passing for Knowledge Graph Question Answering
Junhong Wan (Hikvision Research Institute), Jiang Zhu (Hikvision Research Institute)
CodeExplainability and InterpretabilityGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented Generation
π― What it does: Propose a Language Message Passing (LMP) framework based on large language models, which first iteratively aggregates neighboring entities in a knowledge graph and converts them into semantic facts to form a fact graph, then converts it into a multi-level list through topological reading to enhance the question-answering capabilities of LLMs.
Direct Prompt Optimization with Continuous Representations
Yangkun Wang (University of California San Diego), Jingbo Shang (University of California San Diego)
CodeClassificationOptimizationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a direct prompt optimization framework called SGCG (Soft Greedy Coordinate Gradient), which updates the prompt distribution first and then samples Gumbel variables at each step, using greedy strategies and sliding window techniques to improve search efficiency.
Wei Liu (Heidelberg Institute for Theoretical Studies gGmbH), Michael Strube (Heidelberg Institute for Theoretical Studies gGmbH)
CodeRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper investigates and verifies the impact of discourse relations on text coherence, and proposes a neural model that integrates text and relation features to assess coherence.
Disentangling Memory and Reasoning Ability in Large Language Models
Mingyu Jin (Rutgers University), Yongfeng Zhang (Rutgers University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Propose a new LLM reasoning paradigm that splits the reasoning process into two steps: memory recall and reasoning, using special markers γmemoryγ and γreasonγ to guide the model in distinguishing between these two types of operations;
DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
Yuchen Feng (Chinese Academy of Sciences), Weiping Wang (Chinese Academy of Sciences)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
π― What it does: Proposes the DIVE method, converting dense LLMs into Mixture-of-Experts (MoE) architecture through pruning and restructuring while maintaining or improving model performance.
Document-Level Event-Argument Data Augmentation for Challenging Role Types
Joseph Gatto (Dartmouth College), Sarah M. Preum (Dartmouth College)
CodeData SynthesisTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes two document-level event argument extraction (DocEAE) data generation methods based on large language models (LLMs) - Mad Lib Generation (MLG) and Struct2Text (S2T) - for data augmentation in low-resource cross-domain scenarios;
Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information
Yein Park (Korea University), Jaewoo Kang (Korea University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: This paper discovers through circuit analysis that large language models contain specialized attention heads (Temporal Heads) for processing time-related knowledge, and proves that they are key subcomponents for temporal knowledge recall.
CodeAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AIPrompt EngineeringImageTextMultimodalityPoint CloudRetrieval-Augmented Generation
π― What it does: Built a closed-loop self-driven research framework called DOLPHIN, covering the complete research cycle from paper retrieval, idea generation, experimental validation to result feedback;
π― What it does: This paper proposes the DoMIX framework, which achieves efficient domain adaptive pre-training and knowledge utilization through LoRA modules.
Donβt Erase, Inform! Detecting and Contextualizing Harmful Language in Cultural Heritage Collections
Orfeas Menis Mastromichalakis (National Technical University of Athens), Giorgos Stamou (National Technical University of Athens)
CodeClassificationRetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: Built a multilingual, community-created list of controversial terms and developed an AI-driven tool to detect harmful language in cultural heritage metadata, providing historical context and alternative terms, and integrated it into the Europeana and MINT platforms as well as a public web application.
DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
Jennifer Chen (Mohamed bin Zayed University of AI), Zhiqiang Shen (Mohamed bin Zayed University of AI)
CodeRetrievalKnowledge DistillationTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
π― What it does: Generate evidence and knowledge graphs using a large language model teacher, then compress and distill them to small language models, enabling them to have retrieval-enhanced generation capabilities.
DREsS: Dataset for Rubric-based Essay Scoring on EFL Writing
Haneul Yoo (Korea Advanced Institute of Science and Technology), Alice Oh (Korea Advanced Institute of Science and Technology)
CodeClassificationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Released a large-scale, standardized English writing automatic scoring dataset DREsS based on scoring rubrics, and provided a synthetic data generation method called CASE
Drift: Enhancing LLM Faithfulness in Rationale Generation via Dual-Reward Probabilistic Inference
Jiazheng Li (King's College London), Yulan He (King's College London)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningText
π― What it does: Proposed a dual-reward probabilistic reasoning framework called Drift, which uses task rewards and prospective reasoning rewards to guide LLMs in generating more accurate and faithful answers and explanations during the reasoning phase.
Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models
Boheng Sheng (East China Normal University), Guoxiu He (East China Normal University)
CodeKnowledge DistillationTransformerLarge Language ModelContrastive LearningTextBenchmark
π― What it does: To address the understanding bottleneck of LLMs in reading and reasoning over ultra-long texts, a dynamic chunking and selection mechanism is proposed, which dynamically splits long texts based on semantic similarity and selects the most relevant chunks for LLM processing using a question-aware classifier.
Dynamic Evaluation with Cognitive Reasoning for Multi-turn Safety of Large Language Models
Lanxue Zhang (Chinese Academy of Sciences), Yangxi Li (National Computer Network Emergency Response Technical Team)
CodeSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
π― What it does: Proposed a dynamic multi-round safety evaluation framework called CogSafe based on cognitive theory, which can automatically generate multi-round dialogues and assess the safety of LLMs, significantly reducing the risk of evaluation leakage.
Dynamic Parallel Tree Search for Efficient LLM Reasoning
Yifu Ding (Beihang University), Dacheng Tao (Nanyang Technological University)
CodeComputational EfficiencyAI Code AssistantLarge Language ModelTextBenchmark
π― What it does: Propose the Dynamic Parallel Tree Search (DPTS) framework, combining parallel tree search with dynamic search/transition mechanisms to significantly enhance the computational efficiency of LLM logical reasoning.
CodeSupervised Fine-TuningVision Language ModelMultimodalityBiomedical Data
π― What it does: This paper proposes the EAGLE framework, achieving preference alignment for pathological vision-language models through expert-guided self-enhancement to reduce multimodal hallucinations and biases.
π― What it does: Proposed a model called ECERC that interacts emotional evidence with emotional causality in dialogue context to achieve multimodal dialogue emotion recognition.
EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models
Che Hyun Lee (Seoul National University), Sungroh Yoon (Seoul National University)
CodeGenerationDiffusion modelText
π― What it does: Propose EdiText, a text editing framework based on embedded diffusion models, capable of achieving both coarse-grained and fine-grained editing;
EducationQ: Evaluating LLMsβ Teaching Capabilities Through Multi-Agent Dialogue Framework
Yao Shi (South China University of Technology), Yong Xu (South China University of Technology)
CodeLarge Language ModelAgentic AITextBenchmark
π― What it does: This paper proposes and implements the EducationQ multi-agent dialogue framework, which quantitatively and qualitatively evaluates the interactive effectiveness of large language models in educational contexts through a three-round process: prediction-dialogue-post-test.
Efficient Long Context Language Model Retrieval with Compression
Minju Seo (DeepAuto), Sung Ju Hwang (DeepAuto)
CodeRetrievalCompressionTransformerLarge Language ModelText
π― What it does: Propose CoLoR, a compression method for long-context language model retrieval, which trains a compression model to significantly shorten text length while maintaining retrieval accuracy.
CodeGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposes the Cross-Document Discourse Coherence Enhancement (CD-DCE) task, improving cross-document event and entity coreference resolution (CDCR) performance by inserting semantically coherent text between sentences across documents.
Enhancing Character-Level Understanding in LLMs through Token Internal Structure Learning
Zhu Xu (Chongqing University of Posts and Telecommunications), Conglin Liu (Baidu)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Propose Token Internal Position Awareness (TIPA) and Multi-Token Internal Position Awareness (MTIPA), enhancing large language models' ability to recognize character positions within subwords by training a reverse character prediction task on the subword vocabulary; applying this method to Chinese Spelling Correction (CSC) and general model fine-tuning to verify its improvements in position prediction and character-level tasks.
Enhancing Event-centric News Cluster Summarization via Data Sharpening and Localization Insights
Longyin Zhang (Institute for Infocomm Research), AiTi Aw
CodeGenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper proposes an end-to-end event-center news clustering and summary generation framework (CLUST-MCMS), and improves the quality of multi-language, multi-domain multi-document summaries through data sharpening and localization techniques.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Proposed and verified a model-agnostic two-stage Consistency Reflection and Correction (CRC) framework to enhance the consistency between generated responses and dialogue context in goal-oriented conversational systems.
Enhancing Hyperbole and Metaphor Detection with Their Bidirectional Dynamic Interaction and Emotion Knowledge
Li Zheng (Wuhan University), Donghong Ji (Wuhan University)
CodeClassificationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: Propose an emotion-guided bidirectional dynamic interaction framework, EmoBi, which achieves joint detection of hyperbole and metaphor through emotion analysis, emotion-oriented domain mapping, and bidirectional dynamic interaction mechanisms.
CodeRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: The CIEA method is proposed for multi-modal retrieval, enhancing retrieval performance through supplementary information extraction and alignment.
Enhancing NER by Harnessing Multiple Datasets with Conditional Variational Autoencoders
Taku Oi (Toyota Technological Institute), Makoto Miwa (Toyota Technological Institute)
CodeRecognitionAuto EncoderBiomedical Data
π― What it does: Fusing Conditional Variational Autoencoder (CVAE) with a span-based named entity recognition (NER) model under multiple corpora to learn shared and non-shared information between labels, thereby enhancing training effectiveness across multiple datasets.
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub
Bohan Lyu (Tsinghua University), Maosong Sun (Tsinghua University)
CodeAI Code AssistantLarge Language ModelAgentic AITextBenchmark
π― What it does: Developed the OpenAct benchmark and the OpenAgent LLM agent, supporting automatic retrieval and use of tools from GitHub to address open-domain tasks.
Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization
Yuhao Wang (Zhejiang University), Huajun Chen (Zhejiang University)
CodeDrug DiscoveryGraph Neural NetworkTransformerLarge Language ModelReinforcement LearningGraphBiomedical Data
π― What it does: Propose the KPO framework, combining a protein safety knowledge graph with reinforcement learning to achieve safe alignment for protein language models.
Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study
Bashar Alhafni (Mohamed bin Zayed University of Artificial Intelligence), Nizar Habash (New York University Abu Dhabi)
CodeRestorationData-Centric LearningTransformerLarge Language ModelText
π― What it does: This paper proposes a data-driven text editing framework for grammar error correction in Arabic (including Modern Standard Arabic and dialects), eliminating the need for language-specific edit tags;
Enhancing Transformers for Generalizable First-Order Logical Entailment
Tianshi Zheng (HKUST), Jianxin Li (Beihang University)
CodeRepresentation LearningTransformerGraph
π― What it does: Study the generalization ability of Transformer under parameterized knowledge for first-order logic reasoning, and propose a logic-aware Transformer architecture (TEGA) to significantly enhance reasoning performance.
Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning
Peichao Lai (Peking University), Bin Cui (Peking University)
CodeRepresentation LearningData-Centric LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: Propose a pipeline-based data augmentation method combining knowledge graphs and LLMs, and introduce the Gaussian-decayed Gradient-assisted Contrastive Sentence Embedding (GCSE) model to improve the quality of unsupervised sentence embeddings.
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
Subhajit Chaudhury (IBM), Matthew Riemer (IBM)
CodeRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Propose EpMAN (an attention mechanism based on episodic memory), helping large language models better retrieve and utilize important information in long contexts
EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning
Xiaoqian Liu (University of Chinese Academy of Sciences), Junge Zhang (Chinese Academy of Sciences)
CodeOptimizationTransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: Propose the EPO framework, which trains a specialized strategy generation LLM and uses it as an external policy guide to collaborate with any LLM agent, enhancing the ability to achieve long-term goals in dynamic environments.
EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents
Cheng Qian (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
CodeLarge Language ModelAgentic AITextBenchmarkChain-of-Thought
π― What it does: Propose EscapeBench to evaluate the creative reasoning of LM agents, and build the EscapeAgent model to enhance tool creative usage and implicit goal recognition.
π― What it does: This paper systematically evaluates critical design decisions of dual encoders in entity disambiguation, and proposes the VERBALIZED system based on these insights, further exploring iterative prediction strategies.
Evaluating Language Models as Synthetic Data Generators
Seungone Kim (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
CodeData SynthesisTransformerLarge Language ModelTextBenchmark
π― What it does: Constructed the AGORABENCH benchmark to systematically evaluate the ability of different language models in generating synthetic data.
Evaluating Lexical Proficiency in Neural Language Models
Cristiano Ciaccio (Istituto di Linguistica Computazionale 'Antonio Zampolli' (CNR-ILC)), Felice DellβOrletta (Istituto di Linguistica Computazionale 'Antonio Zampolli' (CNR-ILC))
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Proposes a unified evaluation framework to assess the lexical capabilities of Transformer-based language models in terms of vocabulary generation, definition, and contextual usage, with experiments conducted on Italian.
Evaluating LLMs for Portuguese Sentence Simplification with Linguistic Insights
Arthur Mariano Rocha De Azevedo Scalercio (Universidade Federal Fluminense), Aline Paes (Universidade de SΓ£o Paulo)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Evaluated the one-shot performance of 26 large language models on Portuguese sentence simplification tasks and released a new GovLang-BR simplification corpus.
Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity
Yupu Hao (Chinese Academy of Sciences), Jun Zhao (University of Chinese Academy of Sciences)
CodeTransformerLarge Language ModelSupervised Fine-TuningAgentic AITextBenchmark
π― What it does: Proposed the ETAPP benchmark for evaluating large language models (LLMs) in personalized tool calling, and evaluated it through a sandbox environment with 800 multi-user scenario test cases;
Evaluating Theory of (an uncertain) Mind: Predicting the Uncertain Beliefs of Others from Conversational Cues
Anthony Sicilia (Northeastern University), Malihe Alikhani (Northeastern University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
π― What it does: This study proposes and implements a theory of mind (ToM) task based on dialogue prediction, aiming to enable language models to predict others' subjective uncertainty about a belief through linguistic cues in dialogue (i.e., 'false uncertainty').
Explaining Matters: Leveraging Definitions and Semantic Expansion for Sexism Detection
Sahrish Khan (University of Warwick), Gabriele Pergola (University of Warwick)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringMixture of ExpertsTextChain-of-Thought
π― What it does: Data augmentation for online sexist text using category definitions and semantic expansion, achieving fine-grained detection through multi-model voting with Mistral-7B as a fallback model.
Explicit and Implicit Data Augmentation for Social Event Detection
Congbo Ma (Macquarie University), Preslav Nakov (Macquarie University)
CodeClassificationData SynthesisGraph Neural NetworkLarge Language ModelTextGraph
π― What it does: Propose a dual data augmentation framework SED-Aug, combining LLM-based text augmentation with structural fusion embedding perturbation in the feature space to enhance social event detection performance;
Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder
Siting Li (University of Washington), Simon Shaolei Du (University of Washington)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningImageMultimodalityBenchmark
π― What it does: Compare the visual reasoning capabilities of CLIP and generative multimodal large language models (e.g., LLaVA, Phi-3V, LLaMA-3-V) under the same visual encoder, and analyze the sources of differences through a series of controlled experiments.
Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis
Jisoo Mok (Seoul National University), Sungroh Yoon (Seoul National University)
CodeData SynthesisTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmarkRetrieval-Augmented Generation
π― What it does: Propose the HiCUPID dataset and evaluation framework for training and assessing the capability of LLMs as personalized assistants, and provide the Llama-3.2 agent evaluator.
FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes
Janki Atul Nawale (Nilekani Centre at AI4Bharat), Mitesh M Khapra (Nilekani Centre at AI4Bharat)
CodeLarge Language ModelAgentic AIPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: Proposed INDIC-BIAS, a benchmark for evaluating fairness in large language models (LLMs) within the Indian context, covering four identity categories: caste, religion, region, and tribe. It constructed over 20,000 real-world scenario templates reviewed by experts and assessed LLM bias and stereotypes across three tasks (feasibility, judgment, and generation).
FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
Wei Li (Peking University), Scarlett Li (Peking University)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
π― What it does: Constructed FEA-Bench, a benchmark task set for evaluating large language models' ability to implement new features (i.e., adding new components) at the code repository level.
Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
Haritz Puerto (TU Darmstadt), Iryna Gurevych (TU Darmstadt)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
π― What it does: Investigated a fine-tuning method that generates multiple diverse reasoning chains (DCoT) in a single inference step, utilizing the generated chains to iteratively refine internal reasoning chains.
π― What it does: This paper constructs the FINEREASON logic puzzle benchmark, using state checking and state transition tasks to finely evaluate the deep reasoning capabilities of LLMs.
Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking
Yifan Zhang (Peking University), Zhi Jin (Peking University)
CodeExplainability and InterpretabilityTransformerTextChain-of-Thought
π― What it does: Investigated and verified the learnability of Transformer+CoT in state tracking tasks, using mechanism interpretation techniques to reveal its internal formation of an implicit finite state machine (FSA) and assess its robustness in challenging scenarios such as skipping steps, noise, and length generalization.
FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
Junyu Luo (Peking University), Yike Guo (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmarkFinance Related
π― What it does: Proposed and constructed the FINMME financial multimodal evaluation benchmark, covering 18 core financial fields, 6 asset categories, 10 types of charts, and 21 subcategories, containing over 11,000 high-quality samples, and designed the FinScore assessment framework;
Fixing Distribution Shifts of LLM Self-Critique via On-Policy Self-Play Training
Rong Bao (Fudan University), Minpeng Liao (Tongyi Lab)
CodeReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningText
π― What it does: Propose an SCOP framework that synchronizes the training of reasoning, criticism, and correction processes through model self-dialogue, addressing distribution shift and reward hijacking issues.
CodeExplainability and InterpretabilityVision Language ModelMultimodalityBenchmark
π― What it does: The study evaluates the reasoning ability of vision-language models in visual contexts for implicit sentences (with ambiguity), constructs the FOCUS dataset, and designs three probing question-answering scenarios.
FocusLLM: Precise Understanding of Long Context by Dynamic Condensing
Zhenyu Li (Tsinghua University), Jianyong Wang (Tsinghua University)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose the FocusLLM framework, which extends the context length of large models by dividing long texts into multiple blocks, dynamically injecting context and generating candidate tokens after each block, and then aggregating information through parallel decoding while preserving information integrity.
From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models
Yidan Wang (Institute of Information Engineering, Chinese Academy of Sciences), Binxing Fang (Hainan Province Fang Binxing Academician Workstation)
CodeSafty and PrivacyTransformerLarge Language ModelText
π― What it does: Propose SymMark, a compositional watermarking framework for large language models, integrating logits-based and sampling-based watermarks, and designing serial, parallel, and hybrid strategies.
CodeReinforcement Learning from Human FeedbackReinforcement LearningPrompt EngineeringGenerative Adversarial NetworkTextBenchmark
π― What it does: Proposes the GAPO framework, combining GAN and PPO, and using an encoder-only reward model to learn preference prompts, enhancing the constraint-following ability of large language models.
Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction
Yooseop Lee (Seoul National University), Yohan Jo (Seoul National University)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: Designed a contrastive sorter trained on student selection data and a suspicious distractor generator based on DPO to automatically generate more suspicious multiple-choice distractors.
GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs
Maxim Zhelnin (Skolkovo Institute of Science and Technology), Evgeny Burnaev (Skolkovo Institute of Science and Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose a parameter-efficient fine-tuning method called GIFT-SW, which only fine-tunes the weights of a small number of significant columns and injects Gaussian noise into non-significant columns, helping the model maintain performance in full precision and after quantization.
π― What it does: Constructed a large-scale, multi-domain, multi-lingual low-resource language ASR corpus named GigaSpeech 2, and completed the process from YouTube audio scraping, transcription, alignment, filtering to label refinement through an automated pipeline, ultimately obtaining approximately 30k hours of automatically transcribed data (raw) and refined data of approximately 10k hours of Thai, 6k hours of Indonesian, and 6k hours of Vietnamese.
GΓΆdel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement
Xunjian Yin (Peking University), William Yang Wang (University of California, Santa Barbara)
CodeLarge Language ModelAgentic AIText
π― What it does: Proposes the Godel Agent framework, enabling agents to dynamically read and modify their own code during reasoning to achieve recursive self-improvement.
GRAM: Generative Recommendation via Semantic-aware Multi-granular Late Fusion
Sunkyung Lee (Sungkyunkwan University), Jongwuk Lee (Sungkyunkwan University)
CodeRecommendation SystemTransformerLarge Language ModelText
π― What it does: Propose the GRAM model, which applies large language models to generative recommendation by leveraging semantic-to-lexical translation and multi-grained late fusion.
GRaMPa: Subword Regularisation by Skewing Uniform Segmentation Distributions with an Efficient Path-counting Markov Model
Thomas Bauwens (KU Leuven), Miryam de Lhoneux (KU Leuven)
CodeRepresentation LearningTransformerText
π― What it does: Propose the GRaMPa algorithm, which implements single-pass, uniform sampling for subword segmentation using a path-counting Markov model, and the bias can be adjusted via temperature and minimum token length parameters.
Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs
Haozhen Zhang (University of Illinois at Urbana-Champaign), Jiaxuan You (University of Illinois at Urbana-Champaign)
CodeGenerationRetrievalGraph Neural NetworkLarge Language ModelContrastive LearningTextGraphRetrieval-Augmented Generation
π― What it does: Proposed the Graph of Records (GoR) method, which constructs a graph structure to associate historical responses generated by LLMs with retrieved text blocks, thereby enhancing the effectiveness of global summarization for long texts.
π― What it does: Proposed and implemented a retrieval-augmented generation framework named GRAT based on Monte Carlo Tree Search (MCTS) for multi-hop question answering, enabling multi-path exploration, strategy prediction, step-by-step evaluation, and global selection, while supporting self-training to enhance reasoning capabilities.
GUICourse: From General Vision Language Model to Versatile GUI Agent
Wentong Chen (Renmin University of China), Maosong Sun (Tsinghua University)
CodeReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningAgentic AIVision Language ModelVision-Language-Action ModelImageTextMultimodalitySequential
π― What it does: Developed a complete workflow for training a visual GUI agent from a general vision-language model (VLM) and created three dataset collections
CodeLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: This paper proposes the GUIDEBENCH benchmark to evaluate the ability of large language models to follow domain-specific knowledge rules in real business scenarios, covering seven categories with a total of 1272 instances;
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: By analyzing the internal states during the training of large language models (LLMs), we propose the Sensitivity Dropout (SenD) training protocol, which selectively discards embedding indices that exhibit significant fluctuations during training, thereby reducing the variance of hallucinations and improving model confidence.
Yejin Bang (Hong Kong University of Science and Technology), Pascale Fung (Meta)
CodeLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
π― What it does: Propose the HalluLens evaluation framework, defining the distinction between hallucination and factual accuracy, and designing three new external hallucination assessment tasks (PreciseWikiQA, LongWiki, NonExistentRefusal), reducing data leakage through dynamic test set generation.
CodeGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringGraphBenchmark
π― What it does: Constructed a multi-granularity, multi-difficulty evaluation benchmark named SUBARU, systematically assessing the generalization capability of structured knowledge prompting (SKP) in large language models (LLMs) across four dimensions: granularity, transferability, scalability, and universality, and analyzed its performance through extensive experiments.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Investigate the capability of LLMs in generating writing feedback, propose the StoryFeedback dataset, and analyze the effectiveness of model-generated feedback through automatic and human evaluation systems.
HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model
Mengkang Hu (University of Hong Kong), Ping Luo (University of Hong Kong)
CodeRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningAgentic AITextSequentialBenchmarkRetrieval-Augmented Generation
π― What it does: Propose the HiAgent framework, which utilizes hierarchical management of subgoals to handle the working memory of LLM agents, automatically generates subgoals, and summarizes their trajectories upon completion while retaining essential information;