ICLR 2025 Papers — Page 37
International Conference on Learning Representations · 3704 papers
Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark
Tsung-Han Wu (University of California), David Chan (University of California)
GenerationRetrievalTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes a visual retrieval-based NIAH benchmark (Visual Haystacks, VHs) for evaluating the retrieval and cross-image reasoning capabilities of LMM in large-scale multi-image question answering (MIQA) scenarios. It identifies major bottlenecks in LMM regarding visual retrieval, cross-image reasoning, and positional bias; subsequently, a lightweight visual RAG framework called MIRAGE is designed and implemented, capable of processing up to 10,000 images on a single 40GB A100 GPU, significantly improving retrieval recall and reasoning accuracy.
Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoning
Minheng Ni (Hong Kong Polytechnic University), Wangmeng Zuo (Harbin Institute of Technology)
Prompt EngineeringMultimodalityChain-of-Thought
🎯 What it does: The VISUAL-O1 framework is proposed, utilizing multi-modal multi-turn chain reasoning to help the model eliminate instruction ambiguity in visual contexts and generate experiences during the reasoning process;
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
Xiao Liu (Tsinghua University), Jie Tang
Robotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This paper proposes VisualAgentBench (VAB), a unified multi-domain evaluation benchmark for training and assessing the visual foundational agent capabilities of large multimodal models (LMM) in diverse real-world environments.
Visually Consistent Hierarchical Image Classification
Seulki Park (University of Michigan), Jonathan Huang (Scaled Foundations)
ClassificationSegmentationTransformerImage
🎯 What it does: Proposes the H-CAST method, which achieves hierarchical classification with visual and semantic unity through fine-grained to coarse-grained visual segmentation and internal consistency.
Visually Guided Decoding: Gradient-Free Hard Prompt Inversion with Language Models
Donghoon Kim (Seoul National University), Byonghyo Shim (Seoul National University)
GenerationRetrievalTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: A gradient-independent prompt inversion method VGD is proposed, which generates readable hard prompts through LLM and uses CLIP guidance to match target images.
VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Yichao Liang (University of Cambridge), Kevin Ellis (Cornell University)
Robotic IntelligenceReinforcement LearningVision Language ModelMultimodality
🎯 What it does: Proposes Neuro-Symbolic Predicate (NSP) and an online learning framework for adaptive abstract world model learning in robotic task planning.
VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
Dezhan Tu (University of California), Panpan Xu (Amazon)
CompressionComputational EfficiencyTransformerVision Language ModelMultimodality
🎯 What it does: A KV cache compression method for visual language models, VL-Cache, is proposed to significantly reduce memory usage while maintaining inference accuracy.
VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning
Yongshuo Zong (University of Edinburgh), Timothy Hospedales (University of Edinburgh)
RecognitionGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Designed and released VL-ICL Bench, a comprehensive benchmark for evaluating multimodal (image↔text) in-context learning (ICL), covering ten tasks including perception, reasoning, rule induction, and long context.
VLAS: Vision-Language-Action Model with Speech Instructions for Customized Robot Manipulation
Wei Zhao (Westlake University), Donglin Wang (Westlake University)
Robotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelTextMultimodalityRetrieval-Augmented GenerationAudio
🎯 What it does: The paper proposes an end-to-end Visual Language Action Model (VLAS) that can directly process voice commands and generate robot actions.
VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks
Ziyan Jiang (University of Waterloo), Wenhu Chen (University of Waterloo)
ClassificationRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the MMEB multimodal embedding benchmark and the VLM2VEC training framework, which can transform any visual language model (VLM) into a unified multimodal embedding model, supporting embedding tasks for images, text, or a combination of both, and facilitating contrastive learning through instruction guidance.
VLMaterial: Procedural Material Generation with Large Vision-Language Models
Beichen Li (Massachusetts Institute of Technology), Wojciech Matusik (Massachusetts Institute of Technology)
GenerationData SynthesisTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: Utilize large-scale visual-language models (VLM) to generate Blender procedural material programs for input images, supporting complete editing;
VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning
Nilay Yilmaz (Arizona State University), Yezhou Yang (Arizona State University)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelImageMultimodalityBenchmark
🎯 What it does: Proposed the VOILA benchmark, which dynamically generates visual analogy problems to assess the perception and abstract reasoning capabilities of multimodal large language models.
VoxDialogue: Can Spoken Dialogue Systems Understand Information Beyond Words?
Xize Cheng (Zhejiang University), Zhou Zhao (Zhejiang University)
Large Language ModelMultimodalityBenchmarkAudio
🎯 What it does: Proposes the VoxDialogue benchmark to evaluate the understanding ability of speech dialogue systems regarding non-textual information in speech.
VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis
Yumeng Li (Amazon), Anna Khoreva (Zalando)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringDiffusion modelVideoText
🎯 What it does: This paper proposes a training-free method called VSTAR for generating longer (32–128 frames) and dynamically rich videos with single-channel inference, addressing the issues of monotony and temporal incoherence in existing text-to-video models during long video generation.
VTDexManip: A Dataset and Benchmark for Visual-tactile Pretraining and Dexterous Manipulation with Reinforcement Learning
Qingtao Liu (Zhejiang University), Qi Ye (Zhejiang University)
Robotic IntelligenceConvolutional Neural NetworkTransformerReinforcement LearningImageMultimodalityBenchmark
🎯 What it does: A human visual-tactile dataset VTDexManip was constructed, and based on this dataset, six complex multi-finger robotic grasping tasks were simulated in Isaac Gym to establish benchmarks, comparing 18 pre-trained and non-pre-trained methods to verify the improvement of visual-tactile joint pre-training on robotic grasping performance.
VVC-Gym: A Fixed-Wing UAV Reinforcement Learning Environment for Multi-Goal Long-Horizon Problems
Xudong Gong (National University of Defense Technology), Huaimin Wang (National University of Defense Technology)
Robotic IntelligenceReinforcement LearningBenchmark
🎯 What it does: A multi-target long-term reinforcement learning environment VVC-Gym based on fixed-wing drone velocity vector control is proposed, and a multi-quality demonstration set is generated through a PID controller and IRPO, providing benchmark experiments.
W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models
Shang Wang (ShanghaiTech University)
OptimizationComputational EfficiencyKnowledge DistillationNeural Architecture SearchLarge Language ModelText
🎯 What it does: A gradient-free W-PCA proxy is proposed for zero-training NAS search of lightweight language models without training.
Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations
Katie Matton (Massachusetts Institute of Technology), Emre Kiciman
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Evaluate whether the natural language explanations generated by large language models are authentic and trustworthy, and propose a new measurement method;
Ward: Provable RAG Dataset Inference via LLM Watermarks
Nikola Jovanović (ETH Zurich), Martin Vechev (ETH Zurich)
GenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: A black-box method called WARD is proposed for detecting unauthorized use of datasets in Retrieval-Augmented Generation (RAG), providing strict statistical guarantees.
WardropNet: Traffic Flow Predictions via Equilibrium-Augmented Learning
Kai Jungel (Technical University of Munich), Maximilian Schiffer (Technical University of Munich)
Autonomous DrivingOptimizationGraph Neural NetworkTime Series
🎯 What it does: This paper presents WardropNet, a COAML pipeline that embeds a balancing layer in the backend of neural networks for fast and accurate traffic flow prediction.
Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models
Hao-Chien Hsueh (National Yang Ming Chiao Tung University), Ching-Chun Huang (National Yang Ming Chiao Tung University)
RestorationGenerationDiffusion modelImage
🎯 What it does: The Warm Diffusion framework is proposed, integrating a mixed diffusion process of blur and noise, and simultaneously learning the tasks of denoising and deblurring through a divide-and-conquer approach to achieve higher quality image generation.
Wasserstein Distances, Neuronal Entanglement, and Sparsity
Shashata Sawmya (MIT), Nir N Shavit
Large Language ModelMixture of ExpertsText
🎯 What it does: This study investigates the heterogeneity of neurons (measured by Wasserstein distance) and proposes a Sparse Expansion framework that decomposes the distribution of neurons into experts through clustering inputs while maintaining sparsity, thereby enhancing the sparsification performance of large language models.
Wasserstein-Regularized Conformal Prediction under General Distribution Shift
Rui Xu (Hong Kong University of Science and Technology), Sihong Xie (Hong Kong University of Science and Technology)
Domain AdaptationRepresentation LearningTabular
🎯 What it does: A cover gap upper bound based on Wasserstein distance is proposed, and the Wasserstein-regularized Conformal Prediction (WR-CP) algorithm is designed in the multi-source domain generalization scenario, using importance weighting and regularized representation learning to simultaneously reduce errors caused by covariate and concept shifts, generating more accurate and compact prediction sets.
Watch Less, Do More: Implicit Skill Discovery for Video-Conditioned Policy
Jiangxing Wang (Peking University), Zongqing Lu (Peking University)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerReinforcement LearningAgentic AIVideo
🎯 What it does: Proposes an implicit skill discovery framework based on information bottleneck, allowing for the learning of multiple skills from videos with combinatorial generalization capabilities.
Watermark Anything With Localized Messages
Tom Sander (Meta), Matthijs Douze
SegmentationCompressionTransformerAuto EncoderImage
🎯 What it does: A deep learning model is proposed that locates image watermarks as pixel-level segmentation, capable of embedding and detecting watermarks locally in images, and supports multiple pieces of information.
Wavelet Diffusion Neural Operator
Peiyan Hu (Westlake University), Tailin Wu (Westlake University)
Super ResolutionOptimizationDiffusion modelTime Series
🎯 What it does: Proposes the Wavelet Diffusion Neural Operator (WDNO) for the simulation and control of PDE systems;
Wavelet-based Positional Representation for Long Context
Yui Oka (NTT Corporation), Kuniko Saito (NTT Corporation)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes a multi-scale position representation method based on wavelet transform, which can achieve effective long-context position encoding and extrapolation in Transformers.
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling
Shengpeng Ji (Zhejiang University), Zhou Zhao (Zhejiang University)
CompressionConvolutional Neural NetworkGenerative Adversarial NetworkAudio
🎯 What it does: Proposes WavTokenizer, which compresses speech, music, and general audio into only 40 or 75 discrete tokens under a single-layer quantizer, achieving high-quality reconstruction at extremely low bit rates.
Wayward Concepts In Multimodal Models
Brandon Trabucco (Carnegie Mellon University), Russ Salakhutdinov
ClassificationObject DetectionGenerationAdversarial AttackTransformerPrompt EngineeringImageMultimodality
🎯 What it does: Conducted large-scale experiments on prompt tuning for multimodal models, studying how soft prompts encode visual concepts in generation, detection, and classification tasks, and found that they are equivalent to adversarial perturbations of text encoders.
Weak to Strong Generalization for Large Language Models with Multi-capabilities
Yucheng Zhou (University of Macau), Yu Cheng (Chinese University of Hong Kong)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper explores the weak-to-strong generalization of large language models in multi-capability tasks and proposes a high-quality weak supervision data selection and two-stage training framework based on a reward model to enhance the performance of strong models.
Weak-to-Strong Generalization Through the Data-Centric Lens
Changho Shin (University of Wisconsin Madison), Frederic Sala (University of Wisconsin Madison)
Data-Centric LearningLarge Language ModelText
🎯 What it does: This paper proposes and validates a data center mechanism—overlap density, explaining how weak models can assist strong models in learning difficult-to-recognize patterns by labeling overlapping samples. It also provides an overlapping sample detection algorithm and a UCB-based multi-source data selection strategy. Experiments are then conducted on large language models, weak supervision, and synthetic data to verify the positive correlation between overlap density and weak-strong generalization performance.
Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model
Wenhong Zhu (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: Proposes the WSPO (Weak-to-Strong Preference Optimization) method, which utilizes the differences in probability distributions before and after alignment of the weak model to guide the alignment of the strong model, thereby transferring and amplifying the alignment capability of the weak model to the strong model.
Weakly Supervised Video Scene Graph Generation via Natural Language Supervision
Kibum Kim (KAIST), Chanyoung Park (KAIST)
Object DetectionGenerationTransformerLarge Language ModelVision Language ModelVideoText
🎯 What it does: A weakly supervised video scene graph generation framework, NL-VSGG, is proposed, which uses only video subtitles as weak supervision to train a VidSGG model without manual annotations.
Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors
Peiran Xu (Wangxuan Institute of Computer Technology Peking University), Yadong MU
Object DetectionSegmentationTransformerVision Language ModelContrastive LearningImage
🎯 What it does: A pseudo-fully supervised training framework for weakly supervised functional localization tasks is proposed, utilizing visual foundation models to generate and refine pseudo-labels, achieving accurate affordance heatmap predictions.
WeatherGFM: Learning a Weather Generalist Foundation Model via In-context Learning
Xiangyu Zhao (Hong Kong Polytechnic University), LEI BAI
Image TranslationRestorationSuper ResolutionTransformerMultimodality
🎯 What it does: We propose and implement WeatherGFM, the first weather general foundation model capable of uniformly handling 12 tasks including weather forecasting, rainfall, radar, satellite image super-resolution, image transformation, and post-processing.
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
Hyungjoo Chae (Yonsei University), Jinyoung Yeo (Yonsei University)
Large Language ModelReinforcement LearningWorld ModelText
🎯 What it does: By incorporating a world model into an LLM-driven web navigation agent, the environmental state after actions is simulated to improve decision-making, and a transfer-focused observation abstraction method is proposed.
WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning
Zehan Qi (Tsinghua University), Yuxiao Dong (Tsinghua University)
TransformerLarge Language ModelReinforcement LearningAgentic AIText
🎯 What it does: A self-evolving online course reinforcement learning framework named WEBRL has been developed to train web agents based on open-source large language models (LLMs), significantly improving their success rate on the WebArena tasks.
Weighted Multi-Prompt Learning with Description-free Large Language Model Distillation
Sua Lee (Seoul National University), Jung Ho Park (Seoul National University)
ClassificationKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImage
🎯 What it does: A description-free multi-prompt learning method (DeMul) is proposed, which enhances the performance of pre-trained vision-language models in few-shot classification tasks by directly distilling embeddings from large language models into learnable prompt vectors and introducing prompt weights in a multi-prompt setting.
Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric
Toshimitsu Uesaka (Sony AI), Yuki Mitsufuji (Sony AI)
ClassificationRepresentation LearningTransformerContrastive LearningImageTextMultimodality
🎯 What it does: Proposes Weighted Point Set Embedding (WPSE) for multimodal contrastive learning.
Weighted-Reward Preference Optimization for Implicit Model Fusion
Ziyi Yang (Sun Yat-sen University), Xiaojun Quan (Sun Yat-sen University)
Recommendation SystemOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: A method for implicit model fusion based on weighted reward preference optimization (WRPO) is proposed, which achieves the transfer of multi-source LLM knowledge by dynamically balancing the preference information of the source model and the target model.
What Are Good Positional Encodings for Directed Graphs?
Yinan Huang (Georgia Institute of Technology), Pan Li (Georgia Institute of Technology)
Graph Neural NetworkGraph
🎯 What it does: This paper studies the position information encoding (PE) of directed graphs, proposing the Walk Profile concept that captures bidirectional relationships, and designing a Multi-q Magnetic Laplacian PE (Multi‑q Mag‑PE) and a basis-invariant stable PE framework based on complex features to enhance the representational capability on directed graphs.
What Do You See in Common? Learning Hierarchical Prototypes over Tree-of-Life to Discover Evolutionary Traits
Harish Babu Manogaran (Virginia Tech), Anuj Karpatne (Virginia Tech)
ClassificationRecognitionConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A hierarchical prototype network HComP-Net is proposed, which utilizes image learning to evolve hierarchical features and achieve generalized localization for unseen species.
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis
Weronika Ormaniec (ETH Zurich), Sidak Pal Singh (ETH Zurich)
TransformerText
🎯 What it does: This paper theoretically derives the Hessian structure of single-layer self-attention and analyzes its heterogeneous dependence on data, weights, and attention matrices.
What is Wrong with Perplexity for Long-context Language Modeling?
Lizhe Fang (Peking University), Yisen Wang (Peking University)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This study investigates why traditional perplexity (PPL) cannot measure the performance of large language models on long-context tasks and proposes a new evaluation metric, LongPPL, and training loss, LongCE.
What Makes a Good Diffusion Planner for Decision Making?
Haofei Lu (Tsinghua University), Dongsheng Li (Microsoft Research Asia)
OptimizationTransformerReinforcement LearningDiffusion modelTabular
🎯 What it does: The system evaluates and optimizes decision-making planning methods based on diffusion models in offline reinforcement learning, training over 6000 models and proposing the Diffusion Veteran baseline.
What Makes a Maze Look Like a Maze?
Joy Hsu (Stanford University), Jiajun Wu (Stanford University)
Large Language ModelVision Language ModelImageTextBenchmark
🎯 What it does: The Deep Schema Grounding (DSG) framework is proposed, which utilizes a hierarchical schema of abstract concepts to decompose visual abstraction problems into observable subtasks. Based on the schema, images are normalized layer by layer, and the complete schema is delivered as context to the visual language model to complete the question and answer.
What Makes Large Language Models Reason in (Multi-Turn) Code Generation?
Kunhao Zheng (Meta AI), Gabriel Synnaeve (Paris Dauphine University - PSL)
GenerationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: This study systematically evaluates and combines various prompting techniques (chain-of-thought, instruction prompts, execution feedback) in multi-turn code generation, exploring their performance on large language models and further enhancing model capabilities through self-correction and resampling fine-tuning.
What Matters in Learning from Large-Scale Datasets for Robot Manipulation
Vaibhav Saxena (Georgia Institute of Technology), Danfei Xu (NVIDIA)
Robotic IntelligenceReinforcement LearningDiffusion modelImage
🎯 What it does: This paper systematically evaluates how diversity and alignment in different dimensions affect the performance of robot offline learning by constructing a controllable simulated data generation framework and a large-scale MimicLabs dataset.
What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?
Guangkai Xu (Zhejiang University), Chunhua Shen (Ant Group)
Object DetectionSegmentationDepth EstimationDiffusion modelImage
🎯 What it does: This paper presents GenPercept, a single-step deterministic fine-grained perception framework that utilizes a pre-trained diffusion model (Stable Diffusion);
What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
Ahmed Imtiaz Humayun (Rice University), Mohammad Havaei (Google Deepmind)
GenerationData SynthesisTransformerDiffusion modelImage
🎯 What it does: This study investigates the local geometric features of generative models and explores their relationships with generation quality, alignment, diversity, and memorization.
What should a neuron aim for? Designing local objective functions based on information theory
Andreas Christian Schneider, Michael Wibral (University of Göttingen)
ClassificationOptimizationExplainability and InterpretabilityHyperparameter SearchImage
🎯 What it does: Proposed a local learning objective based on PID information decomposition, designed interpretable infomorphic neurons, and achieved self-organizing classification tasks using three types of inputs: feedforward, contextual, and lateral.
What to align in multimodal contrastive learning?
Benoit Dufumier (École Polytechnique Fédérale de Lausanne), Jean-Philippe Thiran (École Polytechnique Fédérale de Lausanne)
ClassificationRepresentation LearningTransformerContrastive LearningImageTextMultimodality
🎯 What it does: A contrastive multimodal self-supervised learning framework named CoMM is proposed, which can learn the mutual information of multimodal features in a single shared space, capturing redundancy, uniqueness, and collaborative interactions.
What's New in My Data? Novelty Exploration via Contrastive Generation
Masaru Isonuma (University of Tokyo), Ivan Titov (University of Amsterdam)
GenerationAnomaly DetectionData-Centric LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper proposes a task of 'discovering novel domains in fine-tuning datasets through generation' and introduces the Contrastive Generative Exploration (CGE) method to achieve this task.
What's the Move? Hybrid Imitation Learning via Salient Points
Priya Sundaresan (Stanford University), Dorsa Sadigh (Physical Intelligence)
Robotic IntelligenceTransformerReinforcement LearningMultimodalityPoint Cloud
🎯 What it does: We designed and implemented SPHINX, a hybrid modal imitation learning framework based on salient points, which uses point clouds and wrist camera inputs to output long-distance waypoints and high-frequency dense actions, achieving phased execution of complex tasks through mode switching.
When Attention Sink Emerges in Language Models: An Empirical View
Xiangming Gu (Sea AI Lab), Min Lin (Sea AI Lab)
TransformerLarge Language ModelText
🎯 What it does: This paper systematically studies the phenomenon of 'attention sink' in autoregressive language models, clarifying the conditions, mechanisms, and its relationship with model design, and proposes that changing the attention normalization method (such as using sigmoid attention) can eliminate this phenomenon.
When do GFlowNets learn the right distribution?
Tiago Silva, Diego Mesquita (Getulio Vargas Foundation)
GenerationData SynthesisGraph Neural NetworkFlow-based ModelGraphSequential
🎯 What it does: This paper studies when Generative Flow Networks (GFlowNet) can correctly learn the target distribution and systematically analyzes the impact of imbalance, parameterization constraints, and evaluation methods on sampling quality. It proposes the Weighted Detailed Balance (WDB) loss, the Lookahead GFlowNet (LA-GFlowNet), and a computable Flow Consistency Subgraph (FCS) metric.
When does compositional structure yield compositional generalization? A kernel theory.
Samuel Lippl (Columbia University), Kim Stachenfeld
Representation LearningImage
🎯 What it does: This paper presents a theory for kernel models represented by combinatorial structures, explaining their generalization behavior in combinatorial reasoning tasks (such as symbolic addition, context dependence, transitive equivalence relations);
When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach
Qian Chen (Chinese University of Hong Kong), Qingjiang Shi (Tongji University)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper addresses the issue of symmetry in Integer Linear Programming (ILP) and investigates the problem of Graph Neural Networks (GNN) being unable to distinguish symmetric variables when predicting the optimal solution of ILP. It proposes an orbit-based feature enhancement method to solve this problem.
When Graph Neural Networks Meet Dynamic Mode Decomposition
Dai Shi (University of Sydney), Junbin Gao (University of Sydney)
ClassificationOptimizationGraph Neural NetworkGraphTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper proposes to combine Dynamic Mode Decomposition (DMD) with Graph Neural Networks (GNN) to utilize DMD for estimating the low-rank linear dynamics of graph data, thereby enhancing the efficiency and effectiveness of feature propagation.
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
Hongkang Li (Rensselaer Polytechnic Institute), Meng Wang (Rensselaer Polytechnic Institute)
TransformerImageText
🎯 What it does: This paper studies the task vector arithmetic on nonlinear Transformers from a theoretical perspective, providing generalization guarantees for multi-task learning, model forgetting, and out-of-domain generalization, and proving that sparse or low-rank approximations of task vectors can maintain performance.
When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings
Jérémy Perez (Inria), Clément Moulin-Frier (Institute for Advanced Study in Toulouse)
TransformerLarge Language ModelText
🎯 What it does: By placing LLMs in a telephone game-style transmission chain, the evolution of text toxicity, positivity, difficulty, and length during multi-round interactions reveals the phenomenon of LLM cultural attractors.
When narrower is better: the narrow width limit of Bayesian parallel branching neural networks
Zechen Zhang (Harvard University), Haim Sompolinsky (Hebrew University of Jerusalem)
Graph Neural NetworkImageGraph
🎯 What it does: This paper studies the learning and generalization performance of Bayesian Parallel Branch Neural Networks (BPB-NN) in the narrow width limit, revealing that wide networks are not necessarily superior to narrow networks;
When Prompt Engineering Meets Software Engineering: CNL-P as Natural and Robust "APIs'' for Human-AI Interaction
Zhenchang Xing (CSIRO Data61), Chenhua Liu (Jiangxi Normal University)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: CNL-P is proposed, a controlled natural language framework that embeds core software engineering principles (modularity, abstraction, encapsulation, separation of concerns) into prompt syntax to enhance the readability, maintainability, and executability of LLM prompts.
When Selection Meets Intervention: Additional Complexities in Causal Discovery
Haoyue Dai (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
TabularSequential
🎯 What it does: This paper proposes a new framework for causal discovery under the presence of selection bias, designing the 'Intervention Twin Graph' and providing corresponding criteria for Markovianity and equivalence.
Where Am I and What Will I See: An Auto-Regressive Model for Spatial Localization and View Prediction
Junyi Chen (Shanghai Jiao Tong University), Tong He (Shanghai AI Lab)
GenerationPose EstimationTransformerGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes the Generative Spatial Transformer (GST), a joint autoregressive model that can estimate camera pose and synthesize novel view images simultaneously given only a single image.
Which Tasks Should Be Compressed Together? A Causal Discovery Approach for Efficient Multi-Task Representation Compression
Sha Guo (Peking University), LINGYU DUAN
SegmentationDepth EstimationCompressionConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: This paper proposes Taskonomy-Aware Multi-Task Compression (TAMC), which achieves efficient compression of multi-task representations by automatically grouping collaborative tasks and constructing a causal graph driven by conditional entropy.
Why Does the Effective Context Length of LLMs Fall Short?
Chenxin An (University of Hong Kong), Lingpeng Kong (University of Hong Kong)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper analyzes the root cause of the insufficient effective context length of large language models, finding that the position frequency distribution during the pre-training phase is severely left-skewed, leading to inadequate training for long-distance dependencies. To address this issue, the STRING (ShifTed Rotary position embeddING) technique is proposed, which compensates for this defect by shifting high-frequency positions to low-frequency positions during the inference phase, without requiring additional training.
Why In-Context Learning Models are Good Few-Shot Learners?
Shiguang Wu (Tsinghua University), Quanming Yao (Tsinghua University)
Meta LearningTransformerLarge Language ModelText
🎯 What it does: This study investigates the expressiveness and learnability of in-context learning (ICL) in large language models from a meta-learning perspective, demonstrating that Transformers can learn optimal learning algorithms that are data-dependent.
Wicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks
Nguyen Hung-Quang (VinUniversity), Khoa D Doan (VinUniversity)
ClassificationAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: The study investigates clean-label backdoor attacks under a threat model with extreme constraints on single-class data, proposing the use of pre-trained models or OOD data to identify 'hard samples' and implant triggers only on these samples, significantly improving the attack success rate.
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
Arthur Jacot (New York University), Marco Mondelli
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This study investigates the phenomenon of neural collapse in deep neural networks after end-to-end training and provides sufficient conditions for the occurrence of this phenomenon in networks with two or more linear heads.
WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
Bill Yuchen Lin (Allen Institute for AI), Yejin Choi (University of Washington)
TransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: We propose WildBench, an LLM automatic evaluation framework built on real user dialogue logs, containing 1,024 high-difficulty tasks;
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Haipeng Luo (Shenzhen International Graduate School Tsinghua University), Dongmei Zhang (Microsoft Corporation)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: The WizardMath model is introduced to enhance the mathematical reasoning capabilities of large language models using the RLEIF method.
Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers
Omer Sahin Tas (FZI Research Center for Information Technology), Royden Wagner (Karlsruhe Institute of Technology)
Autonomous DrivingExplainability and InterpretabilityTransformerAuto EncoderTime Series
🎯 What it does: An interpretability analysis of the hidden states of the motion Transformer model is conducted, using linear probes to examine the phenomenon of neural collapse, and fitting control vectors based on the differences in opposing features; during inference, the control vector is added to the hidden states to achieve controllable modifications of the predicted trajectories without the need for fine-tuning.
WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models
Shengda Fan (Renmin University of China), Maosong Sun (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A large-scale WorkflowBench dataset was constructed, and WorkflowLlama was fine-tuned on it to enhance the performance of LLMs in workflow orchestration.
World Model on Million-Length Video And Language With Blockwise RingAttention
Hao Liu (University of California Berkeley), Pieter Abbeel (University of California Berkeley)
GenerationRetrievalTransformerLarge Language ModelVision Language ModelWorld ModelImageVideoTextMultimodality
🎯 What it does: A language and vision language model (LWM) capable of handling long contexts of up to 1 million tokens is proposed and implemented, achieving leading results in long text retrieval and long video understanding tasks.
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale
Haoran Xu (Microsoft), Huda Khayrallah (Amazon)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: X-ALMA is a multilingual translation LLM designed for 50 languages, capable of maintaining high-quality translations across all languages, regardless of resource abundance.
X-Drive: Cross-modality Consistent Multi-Sensor Data Synthesis for Driving Scenarios
Yichen Xie (University of California Berkeley), Wei Zhan (University of California Berkeley)
Data SynthesisAutonomous DrivingDiffusion modelImagePoint Cloud
🎯 What it does: Proposes the X-DRIVE dual-branch latent diffusion model, which jointly generates aligned LiDAR point clouds and multi-view camera images, and supports text and 3D bounding box control.
X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing
Xinyan Chen (Nanyang Technological University), Jianfei Yang (Nanyang Technological University)
RecognitionPose EstimationTransformerMultimodality
🎯 What it does: This paper proposes X-Fi, a modality-invariant foundational model for pose estimation and activity recognition in human perception, capable of flexibly accepting various combinations of sensor inputs after a single training session.
X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention
XIAOCHEN ZHAO, Yebin Liu (Tsinghua University)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageVideo
🎯 What it does: This paper presents X-NeMo, a zero-shot portrait animation framework based on diffusion models, which utilizes self-supervised learning to generate identity-agnostic one-dimensional motion latent vectors and achieves motion control through cross-attention, enabling highly expressive head animations while maintaining identity consistency.
XAIguiFormer: explainable artificial intelligence guided transformer for brain disorder identification
Hanning Guo (Forschungszentrum Jülich), Jürgen Dammers (Forschungszentrum Jülich)
ClassificationAnomaly DetectionExplainability and InterpretabilityGraph Neural NetworkTransformerSupervised Fine-TuningTime SeriesBiomedical DataAlzheimer's Disease
🎯 What it does: This paper proposes a model called XAIguiFormer that utilizes explainable artificial intelligence (XAI) to guide Transformers in identifying brain diseases using multi-frequency EEG connectivity graphs.
xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
Qingchen Yu (Institute for Advanced Algorithms Research), Ding Chen (Institute for Advanced Algorithms Research)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Developed xFinder, an evaluator specifically designed for answer extraction and matching in LLM evaluation, replacing traditional regular expression extraction methods.
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
Alexander Nikulin (Artificial Intelligence Research Institute), Vladislav Kurenkov (Artificial Intelligence Research Institute)
Recurrent Neural NetworkReinforcement LearningTabular
🎯 What it does: This paper constructs the XLAND-100B large-scale multi-task dataset and provides tools and a smaller version to support and evaluate in-context reinforcement learning; it also experiments with common AD and DPT baselines on this dataset.
YOLO-RD: Introducing Relevant and Compact Explicit Knowledge to YOLO by Retriever-Dictionary
Hao-Tang Tsui (Academia Sinica), Hong-Yuan Mark Liao (Academia Sinica)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes the Retriever-Dictionary (RD) module, which pre-encodes dataset knowledge into a dictionary and dynamically generates coefficients through a retriever, embedding it into YOLO and other detection models to enhance detection, segmentation, and classification performance.
You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning
Ayan Sengupta (Indian Institute of Technology Delhi), Tanmoy Chakraborty (Indian Institute of Technology Delhi)
CompressionOptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper presents PruneNet, a structured compression method that treats model pruning as a policy learning problem, capable of compressing large language models without calibration data.
You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANs
Yihong Luo (Hong Kong University of Science and Technology), Jing Tang (Hong Kong University of Science and Technology)
Image TranslationGenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageText
🎯 What it does: A self-coherent diffusion GAN model named YOSO is proposed and trained, achieving one-step (single-step) high-quality image generation, and enabling one-step text-to-image generation through fine-tuning of the pre-trained model.
Youku Dense Caption: A Large-scale Chinese Video Dense Caption Dataset and Benchmarks
Zixuan Xiong (Shenzhen International Graduate School Tsinghua University), Hai-Tao Zheng (Shenzhen International Graduate School Tsinghua University)
GenerationRetrievalTransformerSupervised Fine-TuningVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: The first large-scale high-quality dense video caption dataset in China, Youku Dense Caption, has been proposed and released, and multimodal benchmarks for retrieval, localization, and generation have been constructed based on it.
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
Jingyang Ou (Renmin University of China), Chongxuan Li (Renmin University of China)
GenerationData SynthesisOptimizationComputational EfficiencyTransformerDiffusion modelScore-based ModelTextStochastic Differential Equation
🎯 What it does: This paper proposes RADD (Reparameterized Absorbing Discrete Diffusion), a time-condition-free, cache-accelerated absorbing discrete diffusion model, and unifies it with any autoregressive model (AO-ARM) in terms of training objectives.
Your Mixture-of-Experts LLM Is Secretly an Embedding Model for Free
Ziyue Li (University of Maryland), Tianyi Zhou (University of Maryland)
Large Language ModelMixture of ExpertsTextBenchmark
🎯 What it does: This paper studies the routing weights (RW) of the Mixture-of-Experts language model as an untrained embedding representation, and combines RW with hidden states (HS) to propose the MOEE (MoE Embedding) method;
Your Weak LLM is Secretly a Strong Teacher for Alignment
Leitian Tao (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Explores the use of weak LLMs (125M level) as teachers for language model alignment, verifying that their feedback quality is comparable to or even better than human feedback.
YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpus
Garrett Tanzer (Google DeepMind), Biao Zhang (Google DeepMind)
RecognitionData SynthesisTransformerSupervised Fine-TuningVideo
🎯 What it does: We proposed and released YouTube-SL-25, a large-scale multilingual sign language video dataset containing 3000 hours of sign language in 25 languages with relatively aligned subtitles.
ZAPBench: A Benchmark for Whole-Brain Activity Prediction in Zebrafish
Jan-Matthis Lueckmann (Google Research), Viren Jain (Google Research)
Convolutional Neural NetworkVideoTime SeriesBenchmark
🎯 What it does: ZAPBench is proposed as a benchmark task for predicting neural activity at whole-brain cellular resolution, which includes complete training/validation/testing splits and provides visualization tools.
Zero-cost Proxy for Adversarial Robustness Evaluation
Yuqi Feng (Sichuan University), Yanan Sun (Sichuan University)
Adversarial AttackNeural Architecture SearchImage
🎯 What it does: A zero-cost proxy is proposed to evaluate the adversarial robustness of DNNs with initialized weights and applied to NAS.
Zero-shot forecasting of chaotic systems
Yuanzhao Zhang (Santa Fe Institute), William Gilpin (University of Texas at Austin)
TransformerLarge Language ModelTime SeriesBenchmarkPhysics Related
🎯 What it does: Evaluate the predictive capability of pre-trained time series foundation models under zero-shot conditions for chaotic systems, comparing their performance with traditional self-training models.
Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
Patrick Seifner (University of Bonn), Ramses J Sanchez
Data SynthesisAnomaly DetectionTime SeriesOrdinary Differential Equation
🎯 What it does: A zero-shot time series missing value imputation method is proposed, based on the assumption that the underlying dynamics are described by ordinary differential equations (ODEs);
Zero-shot Model-based Reinforcement Learning using Large Language Models
Abdelhakim Benechehab (EURECOM), Balázs Kégl (Huawei Noah's Ark Lab)
TransformerLarge Language ModelReinforcement LearningTime Series
🎯 What it does: This paper proposes the Disentangled In-Context Learning (DICL) framework, which utilizes pre-trained large language models (LLMs) for zero-shot dynamics prediction in Markov Decision Processes (MDPs) within continuous state spaces, and validates its effectiveness in two application scenarios: policy evaluation and data-augmented offline reinforcement learning (DICL-SAC).
Zero-Shot Natural Language Explanations
Fawaz Sammani (Vrije Universiteit Brussel), Nikos Deligiannis (Vrije Universiteit Brussel)
ClassificationGenerationExplainability and InterpretabilityConvolutional Neural NetworkTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: A zero-shot, trustworthy natural language explanation (NLE) method is proposed, which can explain any visual classifier while supporting zero-shot image classification, concept discovery, and image caption generation.
Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models
Andrea Tirinzoni (Meta), Matteo Pirotta (Meta)
Robotic IntelligenceReinforcement LearningVideo
🎯 What it does: This study investigates a versatile full-body robot control model (FB-CPR) pre-trained using unlabeled behavioral data, capable of addressing various tasks such as tracking, reaching targets, and reward optimization under zero-shot conditions.
ZeroDiff: Solidified Visual-semantic Correlation in Zero-Shot Learning
Zihan Ye (Xi'an Jiaotong-Liverpool University), Xiaobo Jin (Xi'an Jiaotong-Liverpool University)
ClassificationRecognitionGenerationDiffusion modelGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: This study investigates the performance of zero-shot learning in scenarios with scarce samples and proposes the ZeroDiff framework to enhance visual-semantic associations, thereby generating more reliable features for unseen categories.
Zeroth-Order Fine-Tuning of LLMs with Transferable Static Sparsity
Wentao Guo (Princeton University), Zhaozhuo Xu (University of Minnesota)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The paper proposes the SensZOQ framework, which utilizes zero-order optimization combined with transferable static sparse parameters and quantization techniques to achieve efficient personalized fine-tuning of large language models (such as Llama2-7B) on memory-constrained devices like mobile phones.
Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference
Qining Zhang (University of Michigan), Lei Ying (University of Michigan)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: Two RLHF algorithms that do not require reward inference are proposed—ZPG and ZBCPG, which optimize the policy network directly from human preferences using zero-order gradients.