NeurIPS 2025 Papers — Page 52
Conference on Neural Information Processing Systems · 5275 papers
Virus Infection Attack on LLMs: Your Poisoning Can Spread "VIA" Synthetic Data
Zi Liang (Hong Kong Polytechnic University), Haibo Hu (Hong Kong Polytechnic University)
Data SynthesisAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper systematically evaluates the security risks of data contamination and backdoor attacks when using synthetic data in LLM training, and proposes the Virus Infection Attack (VIA) framework, which allows maliciously injected content to spread to downstream models through synthetic data.
VisDiff: SDF-Guided Polygon Generation for Visibility Reconstruction, Characterization and Recognition
Rahul Moorthy Mahesh, Volkan Isler (University of Texas at Austin)
RecognitionGenerationGraph Neural NetworkTransformerDiffusion modelMesh
🎯 What it does: This study investigates how to reconstruct polygons and perform diverse sampling under the condition of only having a visible graph (visible graph reconstruction, representation, and recognition), and extends the method to triangulation graphs.
Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations
Jiaming Han (ByteDance), Lu Jiang (ByteDance)
GenerationRepresentation LearningTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: A unified multimodal framework Tar for visual understanding and generation is proposed, utilizing text-aligned discrete semantic representations to achieve modality-agnostic interactions between images and text.
Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Generation
Anlin Zheng, XIAOJUAN QI
GenerationData SynthesisTransformerGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: This paper proposes an image tokenizer VFMTok based on a frozen Vision Foundation Model (VFM), utilizing region-adaptive quantization and semantic reconstruction objectives to achieve efficient image reconstruction and autoregressive (AR) image generation.
Vision Function Layer in Multimodal LLMs
Cheng Shi (Sun Yat-sen University), Sibei Yang (Sun Yat-sen University)
TransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This study investigates the Visual Function Layer (VFL) in multimodal large language models (MLLMs), locating the distribution of different visual functions across network layers through visual token swapping and dropping techniques. Based on these findings, targeted LoRA (VFL-LoRA) and data selection methods (VFL-Select) are proposed to enhance the parameter efficiency and data utilization of the model.
Vision Transformers Don't Need Trained Registers
Nicholas Jiang, Yossi Gandelsman (University of California Berkeley)
Object DetectionSegmentationExplainability and InterpretabilityTransformerImageMultimodality
🎯 What it does: This paper analyzes the high-norm outlier tokens in Vision Transformers and finds that a small number of sparse neurons (register neurons) in the feedforward layer are responsible for generating these outlier tokens. The authors propose a 'test-time registers' method that does not require retraining the model: during inference, these high activation values are moved to additional register tokens, thereby eliminating noise in the attention maps and enhancing interpretability.
Vision Transformers with Self-Distilled Registers
Zipeng Yan (University of Hong Kong), Andrew Luo
SegmentationKnowledge DistillationTransformerContrastive LearningImageBenchmark
🎯 What it does: A post-training method called PH-Reg is proposed, which adds learnable Register tokens to the existing pre-trained Vision Transformer (ViT) and utilizes test-time augmented denoising from a teacher network for self-distillation, thereby removing artifact tokens from dense features and improving fine-grained localization performance.
Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
Yulu Qin (Boston University), Najoung Kim (Boston University)
TransformerVision Language ModelText
🎯 What it does: A pure text question-answer dataset, TaxonomiGQA, was constructed to specifically examine the performance of models in scenarios requiring hierarchical classification knowledge, and this dataset was compared with the VLM-LLM minimal pair;
Vision-centric Token Compression in Large Language Model
Ling Xing (Nanjing University of Science and Technology), Jinhui Tang (Nanjing Forestry University)
CompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelContrastive LearningImageText
🎯 What it does: The VIST framework is proposed, which first converts long texts into images and then compresses them into visual tokens using a lightweight visual encoder to achieve efficient processing of long contexts in LLMs.
Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models
Tiezheng Zhang (Johns Hopkins University), Junfei Xiao (Johns Hopkins University)
GenerationKnowledge DistillationTransformerLarge Language ModelDiffusion modelAuto EncoderImageTextMultimodality
🎯 What it does: Proposes the Vision-Language-Vision (VLV) Auto-Encoder framework, utilizing a pre-trained image encoder, a frozen text-to-image diffusion decoder, and a lightweight large language model to achieve knowledge distillation and generation from unimodal images to high-quality text descriptions.
VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
Senqiao Yang (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
Computational EfficiencyTransformerReinforcement LearningVision Language ModelImageMultimodality
🎯 What it does: This paper proposes VisionThink, a method that dynamically decides whether to use high-resolution images to answer visual questions through reinforcement learning, significantly reducing the consumption of visual tokens while maintaining performance.
ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding
Jialiang Kang (Peking University), Xinghao Chen (Huawei Noah's Ark Lab)
Data SynthesisComputational EfficiencyTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes Vision-Aware Speculative Decoding (ViSpec), which accelerates inference for visual language models by compressing image embeddings through a lightweight visual adapter and injecting global visual features.
ViSPLA: Visual Iterative Self-Prompting for Language-Guided 3D Affordance Learning
Hritam Basak (Stony Brook University), Zhaozheng Yin (Stony Brook University)
Object DetectionSegmentationRobotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision-Language-Action ModelPoint Cloud
🎯 What it does: Designed and implemented an iterative self-prompting framework named ViSPLA, which predicts interactive areas in 3D point clouds based on natural language instructions and continuously refines the mask through geometric feature feedback.
Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models
Fenil R. Doshi (Harvard University), George A. Alvarez (Harvard University)
RecognitionRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelContrastive LearningImage
🎯 What it does: Proposes the visual Anagram task and the Configuration Shape Score (CSS) metric, which systematically evaluates the model's overall shape sensitivity to global part arrangements.
Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection
Chanhyeong Yang (Korea University), Hyunwoo J. Kim (Korea Advanced Institute of Science and Technology)
Object DetectionTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: In zero-shot human-object interaction detection, the VDRP framework is proposed, which enhances the model's adaptability to visual differences of the same verb and its ability to distinguish similar interactions through visual diversity perception prompts and region perception prompts.
Visual Instruction Bottleneck Tuning
Changdae Oh (University of Wisconsin Madison), Sharon Li (University of Wisconsin Madison)
Object DetectionDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: In the instruction tuning of multimodal large language models (MLLM), a Visual Instruction Bottleneck Tuning (Vittle) module is introduced, applying regularization in the internal representation layer through the Information Bottleneck (IB) principle to enhance the model's robustness against distribution shifts and input perturbations.
Visual Jenga: Discovering Object Dependencies via Counterfactual Inpainting
Anand Bhattad (Johns Hopkins University), Alexei A Efros
Object DetectionSegmentationGenerationDiffusion modelImage
🎯 What it does: This paper proposes the Visual Jenga scene understanding task and presents a training-free solution based on adversarial filling, which can progressively remove objects from images while maintaining the physical and geometric coherence of the scene.
Visual Structures Help Visual Reasoning: Addressing the Binding Problem in LVLMs
Amirmohammad Izadi (Sharif University of Technology), Mahdieh Soleymani Baghshah (Sharif University of Technology)
TransformerPrompt EngineeringVision Language ModelImageBenchmark
🎯 What it does: The VISER method is proposed, which enhances the visual reasoning ability of LVLM by adding horizontal lines to the image and guiding the model to scan row by row.
Visual Sync: Multi‑Camera Synchronization via Cross‑View Object Motion
Shaowei Liu (University of Illinois Urbana-Champaign), Shenlong Wang (University of Illinois Urbana-Champaign)
Object TrackingPose EstimationOptimizationSimultaneous Localization and MappingOptical FlowVideo
🎯 What it does: This work proposes VisualSync, a multi-camera video synchronization framework based on cross-view object motion and epipolar geometry, capable of synchronizing uncalibrated, time-unsynchronized multi-camera videos with millisecond-level accuracy.
Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought
Zihui Cheng (Central South University), Libo Qin (Harbin Institute of Technology)
RecognitionGenerationData SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityChain-of-Thought
🎯 What it does: This paper proposes and validates the concept of 'visual thoughts', systematically exploring the unified mechanism of multimodal chain-of-thought (MCoT) in large vision-language models (LVLM) reasoning, and experimentally analyzes the effects of four types of visual thought expressions (natural language, structured language, edited images, generated images) under text and image output modes.
VisualLens: Personalization through Task-Agnostic Visual History
Wang Bill Zhu (University of Southern California), Xin Luna Dong (Meta)
Recommendation SystemTransformerLarge Language ModelImageTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes the VisualLens framework, which utilizes users' non-task-oriented visual history for personalized recommendations.
VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank
Tianhe Wu, Kede Ma
OptimizationReinforcement LearningVision Language ModelImage
🎯 What it does: This paper proposes a no-reference image quality assessment model named VisualQuality-R1, which utilizes reinforcement learning for inference-guided ranking training, achieving quality scores and detailed descriptions that are more aligned with human perception.
VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
Chaoyou Fu (Nanjing University), Ran He
TransformerLarge Language ModelVision Language ModelImageVideoTextMultimodalityAudio
🎯 What it does: This paper presents VITA-1.5, a multimodal large language model capable of visual, text, and speech interaction, achieving seamless integration of vision and speech through a three-stage training method.
VITA-Audio: Fast Interleaved Audio-Text Token Generation for Efficient Large Speech-Language Model
Zuwei Long (Tencent Youtu Lab), Xing Sun (Xiamen University)
GenerationComputational EfficiencyTransformerLarge Language ModelMultimodalityAudio
🎯 What it does: This paper proposes the VITA-Audio end-to-end large-scale speech model, which utilizes a lightweight multimodal cross-token prediction (MCTP) module to generate audio in a single forward pass, significantly reducing the initial audio generation latency.
VITRIX-CLIPIN: Enhancing Fine-Grained Visual Understanding in CLIP via Instruction-Editing Data and Long Captions
Ziteng Wang (Chinese University of Hong Kong), Lin Ma (Meituan)
ClassificationRecognitionRetrievalKnowledge DistillationTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes the CLIP-IN framework, which enhances the performance of CLIP in fine-grained visual understanding by introducing instruction-edited data and long descriptive subtitles, and applies the improved visual encoder to multimodal large language models.
VITRIX-UniViTAR: Unified Vision Transformer with Native Resolution
Limeng Qiao (Peking University), Lin Ma (Meituan)
ClassificationSegmentationRetrievalKnowledge DistillationRepresentation LearningTransformerContrastive LearningImageVideoMultimodality
🎯 What it does: Designed and trained a series of unified visual Transformers called UniViTAR, which support native resolution and multi-modal (image/video) inputs. Through progressive resolution curriculum learning, alternating image-video training, and a hybrid training framework of contrastive learning and knowledge distillation, high-performance visual pre-training models were achieved.
VividFace: A Robost and High-Fidelity Video Face Swapping Framework
Hao Shao (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
GenerationData SynthesisDiffusion modelImageVideo
🎯 What it does: Designed and implemented VividFace, a video face swapping framework based on diffusion models, capable of achieving high fidelity and temporally coherent face swapping effects while preserving identity information.
VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning
Haozhe Wang (Hong Kong University of Science and Technology), Wenhu Chen (University of Waterloo)
TransformerReinforcement LearningVision Language ModelMultimodality
🎯 What it does: In response to the slow thinking ability of visual language models, a training framework based on reinforcement learning is proposed, and the model's reasoning and self-reflection capabilities are enhanced through two techniques: Selective Sample Replay and Forced Rethinking.
VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
Shufan Shen (Institute of Computing Technology, Chinese Academy of Sciences), Shuhui Wang (Institute of Computing Technology, Chinese Academy of Sciences)
Explainability and InterpretabilityRepresentation LearningVision Language ModelAuto EncoderContrastive LearningMultimodality
🎯 What it does: Designed and trained a sparse autoencoder VL-SAE for a unified concept set to explain and enhance the alignment mechanism of visual-language models.
VL-SAM-V2: Open-World Object Detection with General and Specific Query Fusion
Zhiwei Lin (Peking University), Yongtao Wang (Peking University)
Object DetectionTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This paper presents VL-SAM-V2, an open-world object detection framework that can operate in open set and open perception modes. By integrating general queries generated by large-scale visual language models with specific queries produced by dedicated detection models, it achieves simultaneous detection and discovery of known and unknown objects.
VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching
Siyu Xu (University of Sydney), Chang Xu (University of Sydney)
Computational EfficiencyRobotic IntelligenceTransformerVision-Language-Action ModelMultimodality
🎯 What it does: This paper proposes VLA-Cache, a training-independent inference acceleration method that reduces the computational load of VLA models and enhances real-time control frequency by caching and reusing visual static tokens across frames.
VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models
Chongkai Gao (National University of Singapore), Lin Shao (National University of Singapore)
Robotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelMultimodalityPoint Cloud
🎯 What it does: This paper proposes the VLA-OS unified pluggable Vision-Language-Action model family and systematically evaluates the impact of different planning paradigms (Action-Only, Integrated, Hierarchical) and different planning representations (language, vision, image foresight) on the long-term operational performance of robots.
VLForgery Face Triad: Detection, Localization and Attribution via Multimodal Large Language Models
Xinan He (Nanchang University), Feng Ding (Nanchang University)
GenerationTransformerLarge Language ModelImageMultimodalityChain-of-Thought
🎯 What it does: A deep forgery analysis framework VLForgery for AI-generated facial images has been developed, which includes three tasks (detection, localization, attribution) and a VLF dataset containing both complete and partial synthetic samples.
VLM in a flash: I/O-Efficient Sparsification of Vision-Language Model via Neuron Chunking
Kichang Yang (Seoul National University), Youngki Lee (Seoul National University)
Computational EfficiencyTransformerVision Language ModelVideo
🎯 What it does: The paper proposes NEURON CHUNKING, a method for I/O efficiency sparsification of visual-language model inference aimed at flash memory weight offloading, utilizing contiguous neural blocks to reduce I/O latency.
VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
Chaoya Jiang (National Engineering Research Center for Software Engineering, Peking University), Shikun Zhang (Alibaba Group)
RecognitionOptimizationTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: The VLM-R3 framework is proposed, enabling multimodal large language models to dynamically locate and crop image regions during the reasoning process, embedding the cropped visual information into chain-of-thought reasoning.
VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture
Maonan Wang (Chinese University of Hong Kong), Man On Pun
Autonomous DrivingExplainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningVision Language ModelVideoMultimodality
🎯 What it does: A multi-branch framework based on visual language, VLMLight, is proposed for implementing safety-critical traffic signal control.
VLMs can Aggregate Scattered Training Patches
Zhanhui Zhou (University of California Berkeley), Chaochao Lu (University of Illinois Urbana-Champaign)
GenerationAdversarial AttackTransformerSupervised Fine-TuningVision Language ModelImage
🎯 What it does: This paper studies and verifies the ability of visual language models (VLM) to learn and generate corresponding text by aggregating image fragments scattered across different training samples, referred to as visual stitching, and explores its potential threats to model security.
VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
Shmuel Berman (Princeton University), Jia Deng (Princeton University)
RecognitionRetrievalTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: For non-local visual reasoning of VLM, the authors propose three types of tasks (object re-identification, visual treasure hunting, and line connectivity) and construct corresponding programmatically generated evaluation sets.
Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
Qian Ma (Beijing Normal University), Yongqiang Cai (Beijing Normal University)
Transformer
🎯 What it does: This study investigates whether the Transformer has universal approximation capability (UAP) under a vocabulary (finite vocabulary) context learning (VICL) and explores the role of positional encoding.
Vocabulary-Guided Gait Recognition
Panjian Huang (Beijing Normal University), Yongzhen Huang (Beijing Normal University)
RecognitionPose EstimationConvolutional Neural NetworkTransformerVision Language ModelImageVideoMultimodality
🎯 What it does: This paper proposes the 'Gait-World' paradigm, which combines human-defined gait cycle vocabulary with visual-language models (VLM) to guide gait networks in learning more general gait features, resulting in the first α-Gait model.
Volume Transmission Implements Context Factorization to Target Online Credit Assignment and Enable Compositional Generalization
Matthew Storm Bull, Michael A Buice
Hyperparameter SearchMeta LearningRecurrent Neural NetworkReinforcement LearningSequential
🎯 What it does: The paper proposes an end-to-end adjustable recurrent neural network (e-nmRNN) based on neural modulation volume transmission, achieving online, targeted credit allocation through contextual factorization, and is evaluated on tasks such as sequence-to-sequence, meta-learning, multi-tasking, and reinforcement learning.
VORTA: Efficient Video Diffusion via Routing Sparse Attention
Wenhao Sun (Nanyang Technological University), Dacheng Tao (Nanyang Technological University)
GenerationComputational EfficiencyTransformerDiffusion modelVideo
🎯 What it does: The VORTA framework is proposed, utilizing dynamic routing sparse attention to accelerate the generation process of video diffusion transformers.
VoxDet: Rethinking 3D Semantic Scene Completion as Dense Object Detection
Wuyang Li (École Polytechnique Fédérale de Lausanne), Alexandre Alahi (École Polytechnique Fédérale de Lausanne)
Object DetectionSegmentationAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: Reformulate the semantic scene completion problem as dense object detection, achieving instance-aware 3D scene reconstruction through instance-level regression and classification.
VPO: Reasoning Preferences Optimization Based on $\mathcal{V}$-Usable Information
Zecheng Wang (Harbin Institute of Technology), Dianbo Sui (Harbin Institute of Technology)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: For the inference tasks of large language models, the VPO method is proposed, which dynamically limits the negative gradient of non-preferred samples by utilizing available information V, in order to alleviate the compression effect of DPO and enhance the model's alignment with preferred samples.
VQ-Seg: Vector-Quantized Token Perturbation for Semi-Supervised Medical Image Segmentation
Sicheng Yang (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
SegmentationContrastive LearningImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: This paper proposes VQ-Seg, a semi-supervised medical image segmentation framework that utilizes vector quantization (VQ) and quantized perturbation to replace traditional dropout, enhancing segmentation performance through a dual-branch shared Post-VQ space and semantic alignment with the base model.
VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models
Haichao Zhang (Northeastern University), Yun Fu (Northeastern University)
CompressionRepresentation LearningTransformerLarge Language ModelVision Language ModelVideo
🎯 What it does: This paper proposes a VQToken framework that achieves extreme compression of video tokens through vector quantization while preserving spatial-temporal positional information, facilitating efficient processing of videos in large language models.
VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splatting
Hoonhee Cho (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
Autonomous DrivingOptimizationKnowledge DistillationConvolutional Neural NetworkGaussian SplattingPoint CloudBenchmark
🎯 What it does: This paper proposes VR-Drive, an end-to-end autonomous driving framework that jointly learns 3D scene reconstruction as an auxiliary task to achieve viewpoint-independent view synthesis, thereby enhancing planning robustness under different camera perspectives.
VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning
Qiuchen Wang (MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China), Feng Zhao (MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China)
RetrievalTransformerReinforcement LearningVision Language ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: Developed VRAG-RL, a retrieval-augmented generation framework that achieves multi-round visual perception and retrieval reasoning through reinforcement learning training of visual language models.
VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning
Wenhao Li (Shandong University), Yilong Yin (Shandong University)
ClassificationRecognitionGenerationMeta LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes the use of large language models to generate cross-modal prompts for vision and text, and achieves few-shot learning through geometric alignment.
VTON-VLLM: Aligning Virtual Try-On Models with Human Preferences
Siqi Wan (University of Science and Technology of China), Tao Mei (HiDream.ai)
GenerationRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelImageTextBenchmarkChain-of-Thought
🎯 What it does: A visual large language model VTON-VLLM based on human feedback fine-tuning is proposed to evaluate and guide virtual try-on (VTON) models to better align with user preferences.
Vulnerable Data-Aware Adversarial Training
Yuqi Feng (Sichuan University), Yanan Sun (Sichuan University)
Computational EfficiencyAdversarial AttackData-Centric LearningConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A fast adversarial training framework based on vulnerable samples, VDAT, is proposed, which utilizes margin differences to assess sample vulnerability and dynamically selects training samples based on their vulnerability level, thereby reducing training costs and enhancing robustness.
Walking the Schrödinger Bridge: A Direct Trajectory for Text-to-3D Generation
Ziying Li (Zhejiang University), Jianwei Yin (Zhejiang University)
GenerationData SynthesisDiffusion modelScore-based ModelGaussian SplattingTextPoint Cloud
🎯 What it does: This paper studies a text-to-3D generation framework called TraCe based on the Schrödinger bridge, explicitly constructing and learning the optimal transport trajectory from the current rendering to the text alignment target to achieve high-quality 3D asset generation.
Walking the Tightrope: Autonomous Disentangling Beneficial and Detrimental Drifts in Non-Stationary Custom-Tuning
Xiaoyu Yang (University of Technology Sydney), En Yu (University of Technology Sydney)
OptimizationTransformerLarge Language ModelReinforcement LearningVision Language ModelMultimodalityBiomedical DataElectronic Health RecordsChain-of-Thought
🎯 What it does: This paper proposes the Counterfactual Preference Optimization (CPO) method, which eliminates concept drift in non-stationary environments and enhances diagnostic reliability through causal counterfactual interventions on the Chain-of-Thought (CoT) generation path during reinforcement fine-tuning (RFT) of multimodal large language models.
WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents
Siyu Zhou (Australian AI Institute), Chengqi Zhang (Hong Kong Polytechnic University)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningWorld ModelText
🎯 What it does: This paper proposes a training-independent world alignment method called WALL-E, which utilizes neural symbolic learning to extract action rules, knowledge graphs, and scene graphs from real trajectories, and transforms them into executable code rules. This is combined with model predictive control based on LLM to realize a world model-driven LLM agent.
WaLRUS: Wavelets for Long range Representation Using State Space Methods
Hossein Babaei (Rice University), Richard Baraniuk
RecognitionCompressionTime SeriesBenchmarkOrdinary Differential EquationAudio
🎯 What it does: A state space model called WaLRUS based on wavelet frames is proposed for online function approximation and compressed representation of long sequences.
Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
Ruihang Chu (Tongyi Lab Alibaba Group), Yujiu Yang (Tsinghua University)
GenerationData SynthesisDiffusion modelAuto EncoderVideo
🎯 What it does: Developed the Wan-Move framework, achieving video generation with fine motion control guided by latent trajectories.
WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting
Kaitao Huang (Xiamen University), Hanzi Wang (University College London)
RestorationGenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: This paper presents WarpGAN, a method that integrates warping-and-inpainting strategies into 3D GAN inversion to achieve high-quality perspective synthesis from a single image.
Wasserstein Convergence of Critically Damped Langevin Diffusions
Stanislas Strasman (Sorbonne Université and Université Paris Cité, CNRS, LPSM), Antonio Ocello (CREST, ENSAE, Institut Polytechnique de Paris)
TabularStochastic Differential Equation
🎯 What it does: A theoretical analysis of the sampling error of Critical Damping Langevin Diffusion (CLD) under Wasserstein distance is proposed, introducing a noise smoothing hyperparameter to enhance sampling performance.
Wasserstein Transfer Learning
Kaicheng Zhang (Zhejiang University), Yidong Zhou (University of California)
Domain AdaptationTabularBiomedical Data
🎯 What it does: A transfer learning framework for regression of distributed responses in Wasserstein space (WaTL) and its adaptive version (AWaTL) is proposed, which improves the prediction accuracy of the target domain distribution by combining source domain information with target domain samples.
Watch and Listen: Understanding Audio-Visual-Speech Moments with Multimodal LLM
Zinuo Li (University of Western Australia), Qiuhong Ke (Zhejiang Laboratory)
RetrievalTransformerLarge Language ModelMixture of ExpertsVision Language ModelVideoMultimodalityAudio
🎯 What it does: TriSense is proposed—a tri-modal (visual, audio, speech) large language model for video temporal understanding;
Watermarking Autoregressive Image Generation
Nikola Jovanović (Meta), Pierre Fernandez (Meta)
GenerationData SynthesisLarge Language ModelSupervised Fine-TuningImageMultimodalityAudio
🎯 What it does: A generation time zero watermarking method for autoregressive image generation models is proposed, addressing insufficient reverse cycle consistency and achieving robust detection.
WaveAR: Wavelet-Aware Continuous Autoregressive Diffusion for Accurate Human Motion Prediction
shengchuan gao, Ran Yi (Zhejiang University)
GenerationPose EstimationDiffusion modelAuto EncoderVideoTime Series
🎯 What it does: We propose WaveAR, a random human motion prediction framework based on continuous autoregressive diffusion. It first compresses the original 3D joint sequence into continuous latent vectors using a lightweight spatio-temporal VAE, then extracts multi-scale frequency domain features through 2D wavelet transform. After alternating fusion of cross-attention and self-attention, it gradually recovers future latent vectors using a masked autoregressive diffusion method, ultimately decoding to obtain high-fidelity motion predictions.
Wavelet Canonical Coherence for Nonstationary Signals
Haibo Wu (King Abdullah University of Science and Technology), Hernando Ombao
Time SeriesSequential
🎯 What it does: A Wavelet Canonical Coherence (WaveCanCoh) framework based on multivariate locally variable waveforms is proposed to estimate time-varying scale-specific co-movement relationships among non-stationary multivariate signal groups.
Wavy Transformer
Satoshi Noguchi (Japan Agency for Marine-Earth Science and Technology), Yoshinobu Kawahara (RIKEN)
TransformerDiffusion modelImageTextGraph
🎯 What it does: This paper studies the oversmoothing problem of Transformers and proposes the Wavy Transformer, which uses attention layers based on second-order wave dynamics and corresponding physically consistent layer normalization and feedforward networks, aiming to alleviate oversmoothing in deep models.
Weak-shot Keypoint Estimation via Keyness and Correspondence Transfer
Junjie Chen (Jiangxi University of Finance and Economics), Yuming Fang (Shanghai Jiao Tong University)
Pose EstimationDiffusion modelImage
🎯 What it does: A weakly supervised keypoint estimation framework is proposed, utilizing labeled images from base classes and unlabeled images from new classes to achieve multi-class unlabeled keypoint localization through the transfer of keyness and correspondence.
Weak-to-Strong Generalization under Distribution Shifts
Myeongho Jeon (EPFL), Maria Brbic (EPFL)
ClassificationDomain AdaptationReinforcement Learning from Human FeedbackImageText
🎯 What it does: This paper proposes the RAVEN framework to address the generalization problem when weak models supervise strong models under distribution shift.
WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization
Jiahao Wen (Shanghai University), Zhedong Zheng (University of Macau)
Representation LearningTransformerLarge Language ModelVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: This paper proposes WeatherPrompt, which combines training-free weather description with chain reasoning and dynamic text gating for multimodal representation learning to achieve robustness in drone visual localization under all weather conditions.
Weaver: Shrinking the Generation-Verification Gap by Scaling Compute for Verification
Jon Saad-Falcon (Stanford University), Christopher Re
GenerationOptimizationComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Designed and implemented the WEAVER framework, which utilizes various weak validators (reward models, LM judges) to perform weighted combinations of answers generated by language models in repeated sampling scenarios, significantly narrowing the generation-validation gap.
Web-Shepherd: Advancing PRMs for Reinforcing Web Agents
Hyungjoo Chae (Georgia Institute of Technology), Jinyoung Yeo (Yonsei University)
Recommendation SystemReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextMultimodalityBenchmark
🎯 What it does: This paper presents WEB-SHEPHERD, a process reward model (PRM) specifically designed for evaluating web browsing trajectories;
WebDancer: Towards Autonomous Information Seeking Agency
Jialong Wu (Alibaba Group), Jingren Zhou (Alibaba Group)
RetrievalTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITextChain-of-Thought
🎯 What it does: An end-to-end web information retrieval agent system called WebDancer is proposed, which includes four stages: data synthesis, trajectory sampling, SFT cold start, and RL enhancement.
WebThinker: Empowering Large Reasoning Models with Deep Research Capability
Xiaoxi Li (Renmin University of China), Zhicheng Dou (Renmin University of China)
TransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: WebThinker has been developed, a deep research agent based on large reasoning models, capable of autonomously searching the web, browsing web pages, and writing research reports in real-time during the reasoning process.
What are you sinking? A geometric approach on attention sink
Valeria Ruscio (Sapienza University of Rome), Fabrizio Silvestri (Sapienza University of Rome)
TransformerText
🎯 What it does: Through geometric, topological, and information-theoretic analysis, this study investigates the attention sink phenomenon in Transformers and proposes that they are manifestations of three reference frameworks.
What Can RL Bring to VLA Generalization? An Empirical Study
Jijia Liu (Tsinghua University), Yu Wang (Tsinghua University)
Robotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningVision-Language-Action ModelMultimodalityBenchmark
🎯 What it does: This paper systematically compares the impact of RL (especially PPO) and traditional supervised fine-tuning (SFT) on the generalization ability of Vision-Language-Action (VLA) models by constructing a rigorous evaluation benchmark across three dimensions: visual, semantic, and execution. It also proposes an efficient PPO fine-tuning scheme.
What Data Enables Optimal Decisions? An Exact Characterization for Linear Optimization
Omar Bennouna (Massachusetts Institute of Technology), Asuman E. Ozdaglar
Optimization
🎯 What it does: The study investigates the informational value of datasets in decision-making tasks, particularly in determining when a dataset is sufficient to recover optimal decisions in solving linear optimization problems.
What Do Latent Action Models Actually Learn?
Chuheng Zhang (Microsoft Research), Jiang Bian (Microsoft Research)
Convolutional Neural NetworkAuto EncoderImageVideo
🎯 What it does: Learn controllable action variations using a latent action model (LAM) in unannotated videos, and provide theoretical analysis and experimental validation of its learning mechanism.
What do you know? Bayesian knowledge inference for navigating agents
Matthias Schultheis (Cognitive Science University of Darmstadt), Heinz Koeppl (Cognitive Science University of Darmstadt)
OptimizationRobotic IntelligenceReinforcement LearningSequential
🎯 What it does: The study investigates how to infer the implicit state of an agent's partial knowledge of the environment through observing navigation trajectories and presents a Bayesian inference method.
What Does It Take to Build a Performant Selective Classifier?
Stephan Rabanser (Princeton University), Nicolas Papernot (University of Toronto)
ClassificationDomain AdaptationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a unified coverage-based definition of selective classification error (gap) and provides its finite sample decomposition into five main sources of error: Bayesian noise, approximation error, ranking error, statistical noise, and relaxation caused by implementation/distribution drift, which is validated through theoretical derivation and experimental verification.
What Expressivity Theory Misses: Message Passing Complexity for GNNs
Niklas Kemper (Technical University of Munich), Stephan Günnemann (Technical University of Munich)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes and validates a continuous task-specific information transmission complexity measure—Message Passing Complexity (MPC)—to quantify the learning difficulty of Graph Neural Networks (GNNs) under different tasks and architectures, and uses this measure to explain and predict actual experimental performance.
What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformers
Pulkit Gopalani (University of Michigan), Wei Hu (University of Michigan)
TransformerLarge Language ModelText
🎯 What it does: This study investigates the phenomenon of rapid learning in Transformer models for algorithmic tasks, revealing partial solutions, output repetition bias, and representation collapse that occur during the loss plateau phase. It points out that learning attention maps is a key bottleneck; by applying weighted interventions on attention during training, the impact on the loss curve is validated, and these phenomena are extended to the pre-training phase of large language models.
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
Sang Keun Choe (Carnegie Mellon University), Eric P. Xing
Computational EfficiencyData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: A scalable data value assessment method for large language models is proposed, utilizing low-rank gradient projection's influence function to quantify the value of massive training data.
What Makes a Reward Model a Good Teacher? An Optimization Perspective
Noam Razin (Princeton University), Sanjeev Arora (Princeton University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: This study investigates the quality of the 'teacher' in RLHF reward models, proposing that reward variance is a key indicator for evaluating reward models, and validates its impact on optimization efficiency through theoretical and experimental methods.
WHAT MAKES MATH PROBLEMS HARD FOR REINFORCEMENT LEARNING: A CASE STUDY
Ali Shehper (California Institute of Technology), Sergei Gukov (California Institute of Technology)
Reinforcement Learning
🎯 What it does: This paper models the trivialization problem of the Andrews-Curtis conjecture as a long-horizon sparse-reward reinforcement learning MDP, and through a deep RL agent in human-machine collaboration, finds and proves several potential counterexamples' trivialization paths; it also proposes two novel RL schemes: super actions and adaptive action spaces.
What Matters in Data for DPO?
Yu Pan (University of Sydney), Chonghuan Wang (University of Texas at Dallas)
Recommendation SystemOptimizationTransformerSupervised Fine-TuningText
🎯 What it does: This study investigates which features in preference data are most critical in Direct Preference Optimization (DPO) and theoretically and experimentally validates the dominant role of selected response quality on performance.
What Moves the Eyes: Doubling Mechanistic Model Performance Using Deep Networks to Discover and Test Cognitive Hypotheses
Federico D'Agostino (University of Tübingen), Matthias Kuemmerer
Explainability and InterpretabilityImage
🎯 What it does: By comparing the high-performance deep network DeepGaze III with the interpretable mechanistic model SceneWalk, the differences in predicting eye movement scan paths (i.e., 'controversial eye movements') are identified. Based on these differences, mechanisms such as time-varying temperature scaling, eye movement inertia, and directional bias are gradually introduced, ultimately significantly improving the predictive performance of SceneWalk.
What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains
Chanakya Ekbote (Massachusetts Institute of Technology), Paul Pu Liang (Princeton University)
TransformerSequential
🎯 What it does: Study whether a two-layer single-head Transformer can represent any k-th order Markov process and provide a theoretical proof.
What Really is a Member? Discrediting Membership Inference via Poisoning
Neal Mangaokar (University of Michigan), Amrita Roy Chowdhury (University of Michigan)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the reliability of Membership Inference (MI) in large language models (LLMs) and proposes a dataset poisoning attack method called PoisonM. It demonstrates both theoretically and experimentally that MI testing remains unreliable even when using neighborhood rather than strict matching.
What We Miss Matters: Learning from the Overlooked in Point Cloud Transformers
Yi Wang (Central South University), Pheng-Ann Heng (Zhejiang Lab)
ClassificationRecognitionSegmentationTransformerContrastive LearningPoint Cloud
🎯 What it does: The BlindFormer framework is proposed, which guides the point cloud Transformer to focus on neglected areas through Attention Blind Spot Mining (ABM) and Blind Spot-aware Joint Optimization (BJO), thereby enhancing robustness and feature discrimination ability.
What's Producible May Not Be Reachable: Measuring the Steerability of Generative Models
Keyon Vafa (Harvard University), Sendhil Mullainathan (Massachusetts Institute of Technology)
GenerationData SynthesisOptimizationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelImageTextBenchmark
🎯 What it does: This paper proposes a mathematical framework and benchmark tasks for evaluating the steerability of generative models, and assesses the steerability of text-to-image and large language models through large-scale user studies, pointing out their common deficiencies and providing improvement suggestions.
When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery
Dominik Meier (Cornell Tech), Kyra Gan (Cornell Tech)
Diffusion modelTabular
🎯 What it does: A bivariate causal discovery method based on diffusion models, BiDD, is proposed to address the issue of unobserved mediating variables that cause traditional ANM methods to fail.
When and how can inexact generative models still sample from the data manifold?
Nisha Chandramoorthy (University of Chicago), Adriaan A. de Clercq (University of Chicago)
GenerationData SynthesisImageOrdinary Differential Equation
🎯 What it does: This study investigates the robustness of generative models to data distribution support sets under learning errors and presents a disturbance analysis framework based on dynamical systems.
When Are Concepts Erased From Diffusion Models?
Kevin Lu (Northeastern University), Niv Cohen (New York University)
GenerationData SynthesisAdversarial AttackDiffusion modelImageText
🎯 What it does: This paper proposes a multi-perspective evaluation framework to detect and quantify whether text-to-image diffusion models have truly forgotten the target concept after concept erasure, and conducts a comprehensive comparison of various existing erasure methods.
When Can Model-Free Reinforcement Learning be Enough for Thinking?
Josiah P. Hanna (University of Wisconsin), Nicholas E. Corrado (University of Wisconsin)
TransformerLarge Language ModelReinforcement LearningTextSequentialChain-of-Thought
🎯 What it does: The paper theoretically analyzes when model-agnostic reinforcement learning exhibits 'thinking' behavior by introducing the Thought Markov Decision Process (Thought MDP) model under model-agnostic conditions, and validates this theory using a language model (LLM) and a simple grid world experiment.
When Causal Dynamics Matter: Adapting Causal Strategies through Meta-Aware Interventions
Moritz Willig (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
Drug DiscoveryTabularTime Series
🎯 What it does: This paper proposes the Meta-Causal Model (MCM) and Meta-Causal Analysis (MCA) framework to design and evaluate intervention strategies in dynamic environments where causal relationships evolve over time. It demonstrates the application effects in medical and judicial cases through Direct MCM, Linearized Meta-Causal Dynamics (LMCD) algorithm, and sMCATE metrics.
When Data Can't Meet: Estimating Correlation Across Privacy Barriers
Abhinav Chakraborty (Columbia University), T. Tony Cai (University of Pennsylvania)
Federated LearningSafty and PrivacyTabularBiomedical Data
🎯 What it does: This paper studies the estimation of the Pearson correlation coefficient in a vertical federated environment, where two servers hold samples of X and Y respectively, and each server is subject to independent privacy budget constraints. It provides optimal estimation rates and confidence interval constructions for both non-interactive and interactive mechanisms.
When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective
Alireza Mousavi-Hosseini (University of Toronto), Murat A Erdogdu
Recurrent Neural NetworkTransformerSequential
🎯 What it does: The paper analyzes a q-sparse token regression (q-STR) model to study the differences in sample complexity among Transformers, feedforward networks (FFN), and recurrent neural networks (RNN). It proves that the sample complexity of Transformers is independent of sequence length with an appropriate number of heads; RNNs generally require a sample size proportional to the sequence length, while in the simplified q-STR, only a magnitude is needed; FFNs, regardless of size, require a sample complexity at least proportional to the input dimension multiplied by the sequence length.
When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective
Beatrix Miranda Ginn Nielsen, Luigi Gresele (University of Copenhagen)
ClassificationRepresentation LearningImage
🎯 What it does: This paper studies whether similar distributions can guarantee similar internal representations under the same model family, and proposes a new distribution distance metric to capture this relationship.
When Does Curriculum Learning Help? A Theoretical Perspective
Raman Arora, Kaibo Zhang
ClassificationOptimizationAdversarial AttackSupervised Fine-TuningImage
🎯 What it does: A multi-task curriculum learning framework based on the (r,α) condition is proposed, using biased regularized empirical risk minimization (Biased-RERM) and SGD, providing theoretical guarantees and experimental validation in both convex and non-convex scenarios.
When Kernels Multiply, Clusters Unify: Fusing Embeddings with the Kronecker Product
Youqi WU, Farzan Farnia (Chinese University of Hong Kong)
RetrievalRepresentation LearningContrastive LearningImageTextMultimodality
🎯 What it does: Two embedding fusion frameworks based on the Kronecker product, KrossFuse and its scalable version RP-KrossFuse, are proposed, which can unify cross-modal and single-modal embeddings in the same representation space without training.
When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
Weixiang Zhao (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By identifying and removing language-specific subspaces in the activation space of LLMs, the decoupling of language and reasoning is achieved, significantly improving multilingual reasoning performance.
When Lower-Order Terms Dominate: Adaptive Expert Algorithms for Heavy-Tailed Losses
Antoine Moulin (Universitat Pompeu Fabra), Dirk van der Hoeven (Leiden University)
OptimizationReinforcement Learning
🎯 What it does: An adaptive algorithm for handling heavy-tailed losses within the expert prediction framework is proposed, which achieves sublinear optimization without prior knowledge of the loss range or second moments.