ACL 2026 Papers — Page 22
Annual Meeting of the Association for Computational Linguistics · 2296 papers
Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages
Danlu Chen (University Of California San Diego), Freda Shi (University Of Waterloo)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed and evaluated four data difficulty metrics for machine translation in extremely low-resource languages (FRED: Token Fertility, Retrieval Proxy, Pre-training Exposure, Corpus Diversity), and used these metrics to explain the significant differences in translation performance across languages;
Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation
Yongshi Ye (Xiamen University), Xiaodong Shi (Xiamen University)
Domain AdaptationComputational EfficiencyKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextMultimodalityChain-of-Thought
🎯 What it does: Propose a multi-domain machine translation model called TwT, which can dynamically adjust the depth of reasoning based on input difficulty, ranging from intuitive and fast reasoning to deep thinking.
TransLLM: A Unified Multi-Task Large Language Model for Urban Transportation via Learnable Prompting
Jiaming Leng (University of Science and Technology of China), Chao Wang (University of Science and Technology of China)
Autonomous DrivingOptimizationGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringContrastive LearningGraphTabularTime Series
🎯 What it does: Built a unified multi-task framework called TransLLM to address diverse tasks such as urban traffic prediction, electric vehicle charging demand prediction, and taxi dispatching.
TrendFact: A Benchmark Towards Hotspot Perception in Automatic Fact-Checking
Xiaocheng Zhang (Harbin Institute of Technology), Xiaohong Su (Harbin Institute of Technology)
Explainability and InterpretabilityComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Created the TrendFact benchmark to evaluate the hot topic perception ability (HPA) of automatic fact-checking systems, proposed ECS and HCPI evaluation metrics, and designed the FactISR framework to enhance the HPA and computational efficiency of RLM.
TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs
Ziyi Wang (Adelaide University), Xinyu Wang (Adelaide University)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AIPrompt EngineeringTextSequentialFinance RelatedRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the TriEx framework, which records the decision-making process of multi-agent LLMs from three perspectives for interpretability: first-person self-reasoning, second-person opponent belief states, and third-person Oracle auditing, and conducts experimental evaluation in simulated Texas Hold'em Poker.
TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment
Zhewen Tan (Institute of Information Engineering, Chinese Academy of Sciences), Lin Sun (Qiyuan Tech)
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Proposes TriPlay-RL, a three-role reinforcement learning closed-loop framework that collaborates among attacker (M Red), defender (M Blue), and evaluator (M Eval) for co-evolution, aiming to achieve LLM security alignment;
Triviality Corrected Endogenous Reward
Xinda Wang (Peking University), Feng Xiao (Alibaba Group)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: Proposed an unsupervised reinforcement learning method called TCER to improve the quality of open-ended writing generation;
TRN-R1-Zero: Text-rich Network Reasoning via LLMs with Reinforcement Learning Only
Yilun Liu (University of Queensland), Zi Huang (University of Queensland)
Explainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextGraph
🎯 What it does: Propose the TRN-R1-Zero framework, which utilizes reinforcement learning to fine-tune large language models individually, enabling zero-shot reasoning on text-rich networks (TRN).
TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards
Xiqiao Xiong (University Of Science And Technology Of China), Fuli Feng (University Of Science And Technology Of China)
OptimizationAdversarial AttackReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: Achieve trajectory-level optimization in multi-turn dialogue through reinforcement learning, automatically generating malicious prompts capable of bypassing the safety mechanisms of large language models.
Trust Within? Seek Beyond? Knowledge Boundary Aware Policy Optimization for Agentic Search
Tao Feng (Zhejiang University), Chao Wu (Zhejiang University)
RetrievalOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed a knowledge boundary-aware reinforcement learning framework, KbPO, which helps large language models dynamically balance internal knowledge and external retrieval during retrieval-enhanced search processes;
TrustTable: A Neuro-Symbolic Auditing Framework for Faithful Table QA
Guangzhen Zhao (Nanjing University of Posts and Telecommunications), Zhenjiang Dong (Nanjing University of Posts and Telecommunications)
Explainability and InterpretabilityComputational EfficiencyData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTabularBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes the TrustTable framework, which decomposes the chain-of-thought reasoning generated by LLMs into executable Pandas code and Z3 logical constraints, and implements a label-free iterative loop to audit and self-correct the table question-answering reasoning process.
Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
Hamed Damirchi (Adelaide University), Javen Qinfeng Shi (Adelaide University)
Anomaly DetectionExplainability and InterpretabilityRepresentation LearningRecurrent Neural NetworkTransformerLarge Language ModelMixture of ExpertsContrastive LearningTextChain-of-Thought
🎯 What it does: This paper proposes the Truth as a Trajectory (TaT) framework, which views the reasoning process of large language models (LLMs) as a dynamic trajectory across layers, and captures the geometric characteristics of reasoning updates through inter-layer activation differences; subsequently, a lightweight LSTM is used to learn the trajectory, thereby determining whether the model's generation is a legitimate reasoning process.
Truth or Sophistry? LoFa: A Benchmark for LLM Robustness Against Logical Fallacies
Xudong Shen (Shenzhen International Graduate School, Tsinghua University), Zhiyong Wu (South China University of Technology)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningAdversarial AttackData-Centric LearningRobotic IntelligenceTransformerLarge Language ModelAgentic AIPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the LoFa benchmark to evaluate the robustness of large language models when facing logical fallacies.
Turning Failures into Value: Negative Experience Replay for RLVR via Confidence Gating and Boundary Failure Sampling
Jialiang Guo (Meituan), Xunliang Cai (Meituan)
OptimizationComputational EfficiencyRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsTextTabularSequentialBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes NexGRPO, a negative sample experience replay framework for RLVR (Reinforcement Learning with Verifiable Rewards), which specifically utilizes high-value failure trajectories to improve sample efficiency and generalization ability.
TwiUSD: A Benchmark Dataset and Structure-Aware LLM Framework for User Stance Detection
Fuqiang Niu (University of Science and Technology of China), Bowen Zhang (Shenzhen Technology University)
ClassificationRecommendation SystemAnomaly DetectionGraph Neural NetworkTransformerLarge Language ModelContrastive LearningTextGraphBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed a large-scale expert-annotated benchmark for political user stance detection called TwiUSD, and proposed the MRFG framework.
Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations
Wen Luo (Peking University), Houfeng Wang (Peking University)
Anomaly DetectionExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsText
🎯 What it does: Investigate how large language models internally encode factual information, revealing the existence of two information pathways: the question-anchored pathway (dependent on the information flow between questions and answers) and the answer-anchored pathway (dependent on self-contained information within the model's generated answers).
Two Streams, One Sarcasm: Orthogonal Expert Tuning for Holistic Multimodal Sarcasm Understanding
Diandian Guo (Chinese Academy of Sciences), Yanbing Liu (University of Chinese Academy of Sciences)
ClassificationRecognitionExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsContrastive LearningTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the DocMSU-PLUS multimodal sarcasm understanding benchmark and designs the DOSE framework, aiming to unify the perception and reasoning tasks of multimodal sarcasm;
Two-Stage Regularization-Based Structured Pruning for LLMs
Mingkuan Feng (Tsinghua University), Jianhua Tao (Tsinghua University)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Propose a structured pruning method called TRSP based on two-stage regularization. First, layer importance is learned through learnable layer weights and L1 regularization. Then, knowledge transfer is achieved by regularizing the input-output differences of pruned layers. Finally, low-weight layers are directly pruned without requiring subsequent fine-tuning.
Typology-Aware Multilingual Morphosyntactic Parsing with Joint Abstract Node Modeling
Kutay Acar (Istanbul Technical University), Gülşen Eryiğit (Istanbul Technical University)
Federated LearningExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningMeta LearningRecurrent Neural NetworkTransformerSupervised Fine-TuningMixture of ExpertsContrastive LearningText
🎯 What it does: This paper proposes a multilingual, multitask Morphosyntactic Parsing (MSP) system that uniformly predicts part-of-speech, content dependency structure, morphological tags, and abstract nodes, forming a complete sentence-level hierarchy.
UERLens: Understanding Event Relations in Large Language Models
Yong Guan (North China Electric Power University), Shaoru Guo (Shanxi University)
Explainability and InterpretabilityAdversarial AttackTransformerLarge Language ModelAuto EncoderTextBenchmark
🎯 What it does: Propose the UERLens framework to explain how large models internally encode causal, temporal, and sub-event relationships, construct a contrast dataset called UERBench, use sparse autoencoders to mine hierarchical relationship-sensitive features, and validate their functionality through intervention and training.
UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization
Zhengxi Lu (Zhejiang University), Yongliang Shen (Zhejiang University)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningTextMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes the UI-Copilot framework, which utilizes a lightweight Copilot model to perform memory retrieval and numerical calculations on demand in GUI tasks, significantly improving long-term interaction performance.
UMMF: Protecting Copyright of Large Vision-Language Models through Unlearning-based Multimodal Memorization Fingerprint
Xiaofan Zheng (Peking University), Xiaojun Wan (Xi'an Jiaotong University)
Safty and PrivacySupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a copyright protection method for LVLMs that embeds implicit fingerprints by learning from the original descriptions of the same image and unlearning (unlearning) from the descriptions of similar neighbor samples during training.
UMPIRE: Unveiling LLM-generated Posts via Redundant Expressions
Xiaoquan Yi (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
Anomaly DetectionTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a framework named UMPIRE for detecting multimodal posts generated by large language models (LLMs) on social media;
Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities
Changdae Oh (University of Wisconsin-Madison), Sharon Li (University of Wisconsin-Madison)
OptimizationFederated LearningExplainability and InterpretabilityComputational EfficiencyRobotic IntelligenceMeta LearningDrug DiscoveryAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AITextSequentialBenchmarkChain-of-Thought
🎯 What it does: Proposes a unified theoretical framework for uncertainty quantification in large language model agents (Agent UQ), systematically elaborating on three pillars: foundations, challenges, and future directions.
Uncertainty-Aware Routing for Principled Alignment with MoE Dynamics
Yilong Chen (Chinese Academy of Sciences), Haifeng Wang (Baidu)
OptimizationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: This paper conducts a thermodynamic analysis of the routing dynamics in Mixture-of-Experts (MoE) models and proposes an uncertainty-based routing scheme, UAR. It uses Helmholtz free energy to quantify absolute uncertainty, dynamically expands the number of activated experts, and applies load balancing to high-uncertainty samples, achieving a synergy between routing decisions and model self-awareness.
Uncertainty-Aware Test-Time Search for Optimization Problem Solving
Linlin Yu (Augusta University), Haifeng Chen (NEC Labs America)
OptimizationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose an uncertainty-aware Monte Carlo Tree Search (UMCTS) based framework that is training-free and generalizable, using large language models and traditional optimization solvers to automatically complete the entire process from natural language descriptions to executable code.
Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants
Jaspreet Ranjit (University of Southern California), Swabha Swayamdipta (University of Southern California)
ClassificationRecommendation SystemAnomaly DetectionOptimizationFederated LearningExplainability and InterpretabilityData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextTabularElectronic Health RecordsRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Leverage language models (LM) to assist in structuring variable annotations of suicide case narratives in the U.S. National Violent Death Reporting System (NVDRS), and propose a human-machine collaborative codebook development algorithm for rapidly constructing new variable labels; based on this, apply the algorithm to identify and quantify victims' interactions with legal professionals, finding that approximately 12.7% of cases involve such interactions, providing new insights for prevention and intervention.
Uncovering Sentiment Analysis Circuit in Large Language Model
Shichen Li (Soochow University), Peifeng Li (Soochow University)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelAuto EncoderContrastive LearningText
🎯 What it does: By using sparse autoencoders (SAE) and causal circuit analysis, the internal sentiment analysis circuits in LLMs are mined, and the circuit is amplified during inference to enhance the robustness of sentiment analysis.
Uncovering Temporal Framing in the News
Tarek Mahmoud (MBZUAI), Preslav Nakov (MBZUAI)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A multilingual news corpus was constructed, and temporal frames were annotated at the sentence level. An eight-class classification framework for temporal frames was proposed, and benchmark experiments were conducted based on this framework.
Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks
Miaomiao Li (Chinese University of Hong Kong), Jindong Wang (William & Mary)
ClassificationGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studies the bias inheritance phenomenon of large language models (LLM) in data augmentation and systematically evaluates its impact on downstream tasks.
Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models
Jiayi Zhang (Provable Responsible AI and Data Analytics Lab), Di Wang (Provable Responsible AI and Data Analytics Lab)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper systematically identifies and verifies two types of political neurons within LLMs through neuron-level research: cross-topic general political neurons and topic-specific political neurons, and proposes InhibitFT, a fine-tuning method that freezes general neurons to suppress cross-topic stance generalization.
Understanding Emergent Misalignment via Feature Superposition Geometry
Gouki Minegishi (University of Tokyo), Yutaka Matsuo (University of Tokyo)
Explainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderContrastive LearningText
🎯 What it does: Studied the phenomenon of emergent misalignment caused by fine-tuning, proposed and verified the geometric mechanism of feature superposition, and conducted geometric filtering based on this.
Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length
Jingxuan Chen (Cardiff University), Jose Camacho-Collados (Cardiff University)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper systematically evaluates the performance of large language models (LLMs) in multi-instance processing (MIP) scenarios, investigating the impact of the number of instances and context length on performance.
Understanding or Memorizing? A Case Study of German Definite Articles in Language Models
Jonathan Drechsel (University of Passau), Steffen Herbold (University of Passau)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper uses the GRADIEND method to perform gradient interventions on the gender and case variations of the German definite article to investigate how language models encode grammar.
Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection
Xuwei Tan (Ohio State University), Xueru Zhang (Ohio State University)
Anomaly DetectionExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTabularFinance RelatedRetrieval-Augmented Generation
🎯 What it does: A two-stage FinFRE-RAG framework is proposed, which utilizes feature importance pruning and retrieval-enhanced context learning to convert structured financial transaction data into natural language, assisting LLMs in fraud detection.
Understanding the Behaviors of Environment-aware Information Retrieval
Ruifeng Yuan (Fudan University), Chenghao Xiao (Shanghai University of Finance and Economics)
RetrievalTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmarkFinance RelatedRetrieval-Augmented Generation
🎯 What it does: This paper systematically studies how large language models (LLMs) adapt query generation strategies for different retrievers through reinforcement learning (RL), and proposes a branching rollout technique to improve the training stability of multi-round retrieval.
Understanding the Prompt Sensitivity
Yang Liu (Kyoto University), Chenhui Chu (Kyoto University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Mathematically analyze the sensitivity of prompts in large language models, using first-order Taylor expansion to explain the output differences between similar prompts.
Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
Kai Zou (Shanghai Artificial Intelligence Laboratory), Ziwei Liu (S-Lab Nanyang Technological University)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This paper proposes Uni-MMU, a large-scale multi-modal evaluation benchmark oriented towards multi-disciplines and bidirectional coupling (generation ↔ understanding), which includes eight tasks (mazes, sliding puzzles, geometric auxiliary lines, jigsaw puzzles, science physics/chemistry/biology, code rendering, etc.). Each task requires the model to achieve logical dependencies between generation and understanding, and provides verifiable intermediate reasoning steps.
UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision
Zhen Fang (MoE Key Lab of BIPC, USTC), Feng Zhao (MoE Key Lab of BIPC, USTC)
GenerationComputational EfficiencyRepresentation LearningData-Centric LearningTransformerAgentic AIPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningTextMultimodalityBenchmark
🎯 What it does: Propose the UniCorn framework, which enhances the generation capabilities of unified multimodal models through self-learning with multiple autonomous agents.
UniDataBench: Evaluating Data Analytics Agents Across Structured and Unstructured Data
Han Weng (ByteDance), Wentao Zhang (Peking University)
Data-Centric LearningTransformerLarge Language ModelAgentic AIPrompt EngineeringTextGraphTabularBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed the UniDataBench benchmark and the ReActInsight agent based on LLM, aiming to evaluate and achieve end-to-end data analysis across structured and unstructured data sources;
Unified Thinker: A General Reasoning Core for Image Generation
Sashuai Zhou (Zhejiang University), Zhou Zhao (Zhejiang University)
GenerationData SynthesisExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodalityChain-of-Thought
🎯 What it does: Propose Unified Thinker, a modular framework that decouples reasoning from image generation, utilizing structured reasoning pathways to guide diffusion generators, achieving transferable and pluggable reasoning cores.
UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents
Yifan Ji (Northeastern University), Maosong Sun (Tsinghua University)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: Proposes the UNIKIE-BENCH unified benchmark for evaluating the performance of large multi-modal models (LMM) on the key information extraction (KIE) task from visual documents.
UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity Mixture-of-Experts
Zhenyu Liu (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
GenerationData SynthesisTransformerPrompt EngineeringMixture of ExpertsDiffusion modelVideoTextMultimodalityAudio
🎯 What it does: Propose UniMoE-Audio, a unified speech and music generation model, adopting a Dynamic Capacity Mixture-of-Experts (DCMoE) architecture and TopP routing, while designing a three-stage training curriculum to address task conflicts and data imbalance.
Union-of-Experts: Neurons in Mixture-of-Experts are Secretly Routers
Songhao Wu (Renmin University of China), Yankai Lin (Renmin University of China)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: Designed a sparse multi-expert model called Union-of-Experts, which utilizes specialized routing neurons to enable expert self-routing, and improves computational efficiency by virtualizing shared expert routing neurons.
UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions
Chunyu Qiang (Tianjin University), Jianwu Dang (Tianjin University)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringDiffusion modelFlow-based ModelTextMultimodalityAudio
🎯 What it does: Propose UniSonate, a unified streaming matching generation framework capable of synthesizing speech, music, and sound effects solely based on natural language instructions, eliminating fragmentation and dependency on reference audio across previous tasks.
UniSpec: Training-Free Speculative Decoding for Robust LLM Acceleration Across Languages and Hardware
Truong Dinh Do (Japan Advanced Institute of Science and Technology), Le-Minh Nguyen (Japan Advanced Institute of Science and Technology)
Computational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposes a training-agnostic, device-aware speculative decoding framework called UNISPEC for accelerating large language model inference
UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment
Yuanyuan Wang (Chinese University of Hong Kong), Xixin Wu (Chinese University of Hong Kong)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmarkAudio
🎯 What it does: Propose a unified speech reward model, UniSRM, which supports multi-dimensional interpretable evaluation
UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities
Woongyeong Yeo (KAIST), Sung Ju Hwang (KAIST)
GenerationRetrievalRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageVideoTextMultimodalityTabularBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes UniversalRAG, a retrieval-augmented generation framework capable of dynamically routing retrieval in multimodal, multigranularity knowledge bases.
UniVocal: Unified Speech-Singing Code-Switching Synthesis
YuFei Shi, Yang Ai (University of Science and Technology of China)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelTextChain-of-ThoughtAudio
🎯 What it does: Proposed the UniVocal framework, achieving text-driven speech-singing seamless hybrid synthesis (Speech-Singing Code-Switching, SCS), and automatically determining when to switch modes
Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning
Renjie Gu (Central South University), Sijia Liu (Michigan State University)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBiomedical DataReview/Survey PaperRetrieval-Augmented Generation
🎯 What it does: This study defines the concept of 'honesty' in LLM unlearning and proposes a set of evaluation metrics. It systematically evaluates nine mainstream unlearning methods, finding that they all have shortcomings in maintaining practicality, complete forgetting, and honest expression. Subsequently, the study designs the ReVa (Resi‑val‑Vector Alignment) strategy, which improves the model's self-awareness and consistency when forgetting knowledge, significantly enhancing the rejection rate and multi-turn dialogue consistency, while maintaining or even improving the practicality of retained knowledge.
Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning
Jiacheng Hua (Tsinghua University), Miao Liu (Tsinghua University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposed a new prompting method called TRACE (Textual Representation of Allocentric Context from Egocentric Video), which enhances the 3D spatial reasoning ability of video scenes by letting a multimodal large language model generate a textual-based global spatial representation as an intermediate reasoning step.
Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
Zhihao Xu (Renmin University of China), Xiting Wang (Renmin University of China)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Propose a 'text-to-trajectory' paradigm to directly extract multi-turn tool usage trajectories from large-scale unstructured text, construct a four-stage GEM data synthesis pipeline, and train an end-to-end Trajectory Synthesizer.
Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering
Qiming Li, Bing Qin (University of Hong Kong)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposes an untrained, representation engineering method called MRRE during inference, which utilizes precomputed cross-lingual reasoning enhancement vectors and target language output anchor vectors. It directly performs two-stage injection on the hidden states of multilingual models, thereby enhancing the reasoning ability of low-resource languages and maintaining linguistic consistency without using additional training data or tools.
Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention
Shuochen Chang (Shanghai Jiao Tong University), Li Niu (Shanghai Jiao Tong University)
Explainability and InterpretabilityTransformerPrompt EngineeringContrastive LearningTextChain-of-Thought
🎯 What it does: This paper proves that continuous hidden states (latent reasoning) indeed encode multi-step reasoning steps through structural, causal, and geometric probing analysis, and leverages this insight to achieve parameter-free update decoding-time interventions;
Unlocking the Potential of Diffusion Language Models through Template Infilling
Junhoo Lee (Seoul National University), Nojun Kwak (Seoul National University)
GenerationAI Code AssistantTransformerPrompt EngineeringDiffusion modelText
🎯 What it does: This paper proposes Template Infilling (TI), a global template filling method specifically designed for diffusion language models, which guides the generation process with structured anchors.
Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses
Sugyeong Eo (Yonsei University), Heuiseok Lim (Korea University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextMultimodalityChain-of-Thought
🎯 What it does: Systematically evaluated the ability of LLMs to infer implicit intentions when engaging in dialogue using only non-linguistic behaviors (silence, facial expressions, actions), covering 13 models and analyzing error patterns.
Unveiling the Unknown: Open-Set Entity Typing via Two-Stage Generation
Hu Chen (Nankai University), Wei Shen (Nankai University)
ClassificationGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the Open-Set Entity Type Recognition (OSET) task and designs a two-stage generative model to automatically synthesize high-quality and diverse instances of unknown types for training a unified open-set classifier, thereby enabling simultaneous identification of both known and unknown entity types.
UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languages
Pranjal A Chitale, Sunayana Sitaram (Microsoft Corporation)
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper constructs UPDESH, a 9.5M instruction-following dataset covering 13 Indian languages (including English), focusing on two subsets: translatable reasoning and culturally localized generation, providing diverse scenarios such as long context, multi-turn dialogue, and cultural reasoning.
UR^2 : Unify RAG and Reasoning through Reinforcement Learning
Weitao Li (Tsinghua University), Yang Liu (Tsinghua University)
RetrievalComputational EfficiencyTransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed the UR 2 framework, which uses reinforcement learning to dynamically coordinate retrieval and reasoning.
UrbanGeoEval: A City-Scale Benchmark for Evaluating Large Language Models in Geospatial Reasoning
Mutian Bao (Zhejiang University), Qiang Zhu (Zhejiang University)
TransformerLarge Language ModelPrompt EngineeringTextPoint CloudGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes UrbanGeoEval, a city-scale dual-modal evaluation benchmark aimed at separately assessing the geographic knowledge memory and geographic reasoning abilities of LLMs.
USB: A COMPREHENSIVE AND UNIFIED SAFETY EVALUATION BENCHMARK FOR MULTIMODAL LARGE LANGUAGE MODELS
Baolin Zheng (Future Living Lab of Alibaba), Kaifu Zhang (Alibaba Group)
Safty and PrivacyTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed the USB unified security evaluation benchmark, covering 61 types of risks and four modal combination, and constructed a security assessment dataset with 13K samples.
User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
Xiaoyuan Wu (Fujitsu Research of America, Inc.), Koichi Onoue (Fujitsu Research of America, Inc.)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextReview/Survey Paper
🎯 What it does: Conducted user surveys and agent LLM evaluations on 90 PrivacyLens privacy-sensitive scenarios, exploring users' perceptions of helpfulness and privacy protection and the alignment of agent LLMs with these perceptions.
Using Perspectival Words Is Harder Than Vocabulary Words for Humans —and Even More So for Multimodal Language Models
Dota Tianai Dong (Max Planck Institute for Psycholinguistics), Paula Rubio-Fernandez (Max Planck Institute for Psycholinguistics)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper constructs a multiple-choice experimental framework for object reference, comparing the usage of three types of words—lexical words, possessive pronouns, and demonstrative pronouns—by human subjects and seven multimodal language models (MLMs), to investigate the cognitive difficulty of these words and model performance.
Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation
Weiqing Luo (Arizona State University), Ziyi Huang (Arizona State University)
RetrievalComputational EfficiencyRepresentation LearningTransformerPrompt EngineeringVision Language ModelContrastive LearningMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes a visual evidence selection framework based on information theory to improve the utilization of evidence in multi-modal retrieval-augmented generation.
v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound
Zhengpeng Shi, Zilong Zheng (State Key Laboratory of General Artificial Intelligence, BIGAI)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodalityBenchmarkRetrieval-Augmented GenerationAudio
🎯 What it does: Constructed a benchmark specifically for video humor understanding called v-HUB, collected and annotated approximately 1,218 short video clips with a visual focus, and designed multi-task evaluations (Caption Matching, Humor Explanation, Open-ended QA) to systematically assess the humor understanding capabilities of multi-modal large language models (MLLMs).
Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification
Stefan Krsteski (EPFL), Kristina Gligorić (EPFL)
Data SynthesisRecommendation SystemOptimizationFederated LearningExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringRectified FlowTextTabularTime SeriesReview/Survey PaperBenchmark
🎯 What it does: Use large language models to generate survey response answers, and reduce estimation bias through post-correction methods, achieving low-cost large-scale surveys under limited human data budgets.
VALU: A Benchmark for Video Anomaly Temporal Localization and Understanding at Multiple Semantic Levels
Yixiao He (Beijing University of Posts and Telecommunications), Jingyu Wang (Beijing University of Posts and Telecommunications)
Anomaly DetectionTransformerPrompt EngineeringVision Language ModelVideoTextBenchmark
🎯 What it does: Propose the VALU benchmark, providing multi-level semantic annotations at five semantic levels and three systematic evaluation tasks (Temporal Description Grounding, Anomaly Localization and Description, Anomaly Description Choice).
Value of Information: A Framework for Human–Agent Communication
Yijiang River Dong (University of Cambridge), Nigel Collier (University of Cambridge)
OptimizationFederated LearningExplainability and InterpretabilityComputational EfficiencyAdversarial AttackHyperparameter SearchData-Centric LearningRobotic IntelligenceMeta LearningDrug DiscoveryAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextTabularSequentialRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes a framework based on value of information (VoI) that enables LLM agents to dynamically decide whether to ask clarifying questions during interactions with users.
VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
Rui Hu (Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University), Jitao Sang (Unisound AI Technology Co., Ltd.)
RecognitionTransformerLarge Language ModelReinforcement LearningPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmarkRetrieval-Augmented GenerationAudio
🎯 What it does: Propose an end-to-end 'look then listen' speech recognition framework called VAPO, addressing the visual interference problem in multimodal models for speech recognition in presentations.
VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis
Shubhashis Roy Dipta (University of Maryland, Baltimore County), Subarna Tripathi (Intel)
Data SynthesisExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes VC-Inspector, a lightweight, open-source, reference-free video caption evaluation method based on large-scale multimodal models, focusing on factual accuracy and capable of generating explanations.
VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervision
Xuan Gong (Shanghai Jiao Tong University), Shiyu Liang (Shanghai Jiao Tong University)
OptimizationComputational EfficiencyRepresentation LearningData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextSequentialBenchmarkChain-of-Thought
🎯 What it does: Designed and implemented a variance-controlled reweighting framework called VCORE based on optimization theory, for fine-tuning LLMs under chain-of-thought (CoT) supervision.
VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
Dingyu Yao (Chinese Academy of Sciences), Weiping Wang (Chinese Academy of Sciences)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Proposes VecInfer, a KV cache compression method based on vector quantization, which significantly reduces memory usage through low-bit (1~4bit) quantization while maintaining high accuracy;
Vector Calligrapher: Generating Scalable Vector Graphics via Structured Linguistic Supervision
Bo Zhou (Zhejiang University), Yin Zhang (Zhejiang University)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: Propose Vector Calligrapher, which generates controllable and efficient SVG fonts using structured language supervision and separated coordinate tokenization.
Verbal-R3: Verbal Reranker as the Missing Bridge between Retrieval and Reasoning
Sangkwon Park (Seoul National University), Sungroh Yoon (Seoul National University)
RetrievalRecommendation SystemExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITextRetrieval-Augmented Generation
🎯 What it does: Proposes Verbal-R3, a two-agent retrieval-augmented generation framework, consisting of a Generator (iterative retrieval and reasoning) and a Verbal Reranker (generates Verbal Annotations for retrieval documents and assigns relevance scores);
Verifiable LLM-Generated Text Detection via Projected Semantic-Structural Distributions
Ruochong Xiong (Peking University), Junfei Liu (Peking University)
Anomaly DetectionExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningTextBenchmark
🎯 What it does: This paper proposes an unsupervised verifiable LLM-generated text detection framework called ProSSD, which extracts low-dimensional semantic features through supervised subspace projection and models the joint distribution of semantics and structure. It utilizes Wasserstein-weighted likelihood ratios to obtain discriminative statistics, significantly improving detection accuracy.
Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing
Wenhao Yuan (University of Hong Kong), Edith Cheuk-Han Ngai (University of Hong Kong)
Explainability and InterpretabilityComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AIPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed a self-audit verification framework called SAVER, which verifies and repairs internal reasoning trajectories in LLM agents, ensuring that internal beliefs are trustworthy before executing actions.
VerilogLAVD: LLM-Aided Pattern Generation for Verilog CWE Detection
Xiang Long (Hangzhou Dianzi University), ZiHao Liu
Anomaly DetectionAI Code AssistantGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: Propose the VerilogLAVD framework, which generates executable traversal detection patterns (TDP) through LLM to identify CWE vulnerabilities in Verilog HDL, leveraging a unified VeriPG (AST+CFG+DDG) to capture hardware concurrency semantics, and building a validator to ensure the legitimacy of TDP and perform detection.
VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking
Mark Rothermel (Technical University of Darmstadt), Anna Rohrbach
TransformerLarge Language ModelPrompt EngineeringMixture of ExpertsImageVideoTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed and implemented VERITAS — the first dynamic, multimodal, cross-lingual automatic fact-checking benchmark, capable of automatically expanding real-world claim data quarterly, avoiding model knowledge leakage;
VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation
Yixia Li (Southern University of Science and Technology), Furu Wei (Microsoft Research Asia)
Domain AdaptationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextMultimodality
🎯 What it does: Propose the VisionFree Adaptation (VFA) framework, which uses multilingual text data to fine-tune LLMs for multilingual tasks, and enhances the multilingual capabilities of multimodal LLMs while preserving visual alignment through the fusion of task vectors and visual alignment vectors;
VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck
Feiran Zhang (Fudan University), Xiaoqing Zheng (Fudan University)
Anomaly DetectionExplainability and InterpretabilityTransformerVision Language ModelImageTextMultimodality
🎯 What it does: Propose the VIB-Probe framework, which utilizes the multi-layer multi-head attention outputs from visual-language models, achieving hallucination detection and intervention during inference through variational information bottleneck
Video-MMMU: Evaluating Knowledge Acquisition from Multidisciplinary Professional Videos
Kairui Hu (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Designed and constructed the Video-MMU benchmark to evaluate the ability of large multimodal models to acquire knowledge from educational videos, encompassing three cognitive stages: perception, understanding, and adaptation.
VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition
Hongbo Jin (Peking University), Ge Li (Peking University)
TransformerReinforcement LearningVision Language ModelOptical FlowVideoText
🎯 What it does: Propose VideoCuRL, a reinforcement learning curriculum based on two-dimensional difficulty decomposition, to enhance the spatiotemporal reasoning and perception capabilities of video LLMs.
VideoPro: Adaptive Program Reasoning for Long Video Understanding
Chenglin Li (Zhejiang University), Jiaqi Wang (Shanghai Innovation Institute)
TransformerLarge Language ModelReinforcement LearningMixture of ExpertsVision Language ModelVideoTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed the VideoPro framework, which achieves long video understanding through adaptive reasoning and self-improving executable visual programs;
VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
Honghao Fu (University of Queensland), Yujun Cai (University of Queensland)
GenerationRetrievalRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose the VideoStir framework, which utilizes event boundary segmentation, spatiotemporal graph construction, and multi-hop retrieval, and introduces an intent-aware frame-level relevance scorer to achieve structured and intent-aligned retrieval-enhanced generation in long videos.
ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarios
António Loison (Illuin Technology), Gautier Viaud (Illuin Technology)
GenerationRetrievalRecommendation SystemComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageTextMultimodalityTabularBenchmarkFinance RelatedPhysics RelatedRetrieval-Augmented Generation
🎯 What it does: Created the multi-modal retrieval-augmented generation (RAG) benchmark ViDoRe V3, which includes 10 industry document collections, 26,000 pages, 3,099 multilingual queries, and end-to-end evaluation of retrieval, generation, and visual localization.
VIGIL: Defending LLM Agents Against Tool-Stream Injection via Verify-Before-Commit
Junda Lin (University of Science and Technology of China), Enhong Chen (University of Science and Technology of China)
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AITextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the VIGIL framework, which adopts a verify-before-commit mechanism within LLM agents to defend against tool-flow injection attacks, and constructs the SIREN benchmark to evaluate its performance.
VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models
Chahat Raj (George Mason University), Ziwei Zhu (George Mason University)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper introduces VIGNETTE, a bias evaluation benchmark based on visual question answering, containing over 30M synthetic images and covering four paradigms: factual, perceptual, stereotype, and decision-making.
ViLL-E: Video LLM Embeddings for Retrieval
Rohit Gupta (Amazon), Mubarak Shah (University of Central Florida)
RetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: Propose ViLL-E, a unified VideoLLM capable of generating text responses and generating embeddings for videos and text;
VisAidMath: Benchmarking Visual-Aided Mathematical Reasoning
Jingkun Ma (University of Macau), Derek F. Wong (University of Macau)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Introduce the VisAidMath benchmark to evaluate the ability of multimodal models in visualization-assisted mathematical reasoning, and propose a three-tier funnel evaluation framework.
Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering
Shuliang Liu (Hong Kong University of Science and Technology), Xuming Hu (Hong Kong University of Science and Technology)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposed a training-agnostic reasoning framework called VLI, which eliminates overconfident hallucinations in multimodal large language models through self-reflection.
VisPCO: Visual Token Pruning Configuration Optimization via Budget-Aware Pareto-Frontier Learning for Vision-Language Models
Huawei Ji (Shanghai Jiao Tong University), Xinbing Wang (Shanghai Jiao Tong University)
OptimizationComputational EfficiencyRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposes a visual token pruning configuration optimization framework called VisPCO, which automatically searches for Pareto optimal pruning configurations under a given computational budget, thereby improving the inference efficiency of vision-language models.
VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval
Di Wu (University of California, Los Angeles), Kai-Wei Chang (University of California, Los Angeles)
RetrievalTransformerLarge Language ModelDiffusion modelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose the VisRet method, which first converts text queries into images and then retrieves in the image space, thus addressing the weak representation of cross-modal embeddings for details such as pose and viewpoint.
Vista-LLM: Decoupled Query-Guided Visual Token Pruning for Efficient Long-Video Large Language Models
Zhenyu Li (Wuhan University), Haojun Ai (Wuhan University)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelAuto EncoderContrastive LearningVideoTextMultimodality
🎯 What it does: Proposed the Vista-LLM framework, which uses a query-guided visual token pruning method to significantly reduce the computational cost and memory consumption of long video LLMs.
VISTA: Verification In Sequential Turn-based Assessment
Ashley Lewis (Ohio State University), Michael White (Ohio State University)
ClassificationAnomaly DetectionExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextSequentialBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes the VISTA framework, which evaluates the factual accuracy of dialogue systems by extracting, verifying, and classifying assertions sentence by sentence in multi-turn dialogues.
Visual and Memory–Augmented Soccer Commentary Generation
Haoran Sun, Kiyoaki Shirai (Japan Advanced Institute Of Science And Technology)
GenerationRetrievalTransformerLarge Language ModelVision Language ModelVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposed two more refined and coherent football commentary datasets, SN-Short and SN-Long, based on human-annotated data, and generated MatchText by semantically augmenting incomplete commentaries; designed the MatchAware model, which integrates visual Q-Former, LLM decoder, and retrieval-enhanced memory mechanisms, to achieve context-aware commentary generation based on historical visual events.
Visual Attention Reasoning via Hierarchical Search and Self-Verification
Wei Cai (Peking University), Xuelong Li (China Telecom)
Safty and PrivacyExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: Propose the Visual Attention Reasoning (VAR) framework, which transforms the reasoning of multimodal large language models into a backtracking hierarchical search, and achieves traceable visual evidence grounding through explicit box selection.
Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoning
Jiachen Qian (City University of Hong Kong)
Recommendation SystemAdversarial AttackTransformerVision Language ModelDiffusion modelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Studied the Visual Inception attack based on multi-modal memory poisoning and proposed a dual-process defense framework called COGNITIVEGUARD;
Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Images
Qishun Yang (King Abdullah University of Science and Technology), Di Wang (King Abdullah University of Science and Technology)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelAuto EncoderImageTextMultimodality
🎯 What it does: Proposed Visual Self-Fulfilling Alignment (VSFA), which enables multi-modal large language models (VLMs) to develop a safety-oriented persona without using any safety labels, by constructing a neutral VQA task on threat-related images for visual instruction fine-tuning.
Visually-Guided Policy Optimization for Multimodal Reasoning
Zengbin Wang (Alibaba Group), Xiangxiang Chu (Alibaba Group)
OptimizationComputational EfficiencyRepresentation LearningReinforcement Learning from Human FeedbackTransformerReinforcement LearningPrompt EngineeringMixture of ExpertsVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: Propose the Visually-Guided Policy Optimization (VGPO) framework, which enhances vision-based reinforcement learning through visual attention compensation and dual-layer advantage reweighting, significantly improving visual fidelity and performance in multimodal reasoning.
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning
Wenyi Xiao (Zhejiang University), Leilei Gan (Zhejiang University)
Explainability and InterpretabilityComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningMixture of ExpertsVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A robust self-calibration framework called VL-Calibration is designed for large vision-language models (LVLM), which separately outputs visual confidence and reasoning confidence during the generation process, and achieves decoupled confidence calibration by using reinforcement learning and visual confidence rewards.