NeurIPS 2025 Papers — Page 32
Conference on Neural Information Processing Systems · 5275 papers
Nyström-Accelerated Primal LS-SVMs: Breaking the $O(an^3)$ Complexity Bottleneck for Scalable ODEs Learning
Weikuo Wang (China Three Gorges University), Huan Luo (China Three Gorges University)
OptimizationComputational EfficiencyTime SeriesOrdinary Differential Equation
🎯 What it does: A Nyström accelerated LS-SVM framework is proposed for efficiently solving linear and nonlinear ordinary differential equations (ODEs).
OASIS: One-Shot Federated Graph Learning via Wasserstein Assisted Knowledge Integration
Guancheng Wan (Wuhan University), Mang Ye (Nanyang Technological University)
Federated LearningKnowledge DistillationGraph Neural NetworkGenerative Adversarial NetworkGraph
🎯 What it does: The OASIS framework is proposed to achieve federated graph learning with a single communication round, capturing fine-grained structural information of local graphs through a generator and structural codebook, and performing knowledge distillation on the server side.
ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservation
Haoqi Wu (TikTok Inc), Qiang Yan (National University of Singapore)
GenerationData SynthesisSafty and PrivacyComputational EfficiencyPrompt EngineeringDiffusion modelImage
🎯 What it does: An implicit cloud-device hybrid image generation scheme called ObCLIP is proposed, which uses candidate prompts to conceal sensitive information, ensuring both privacy and improved generation quality.
Object Concepts Emerge from Motion
Haoqian Liang (Beijing University of Posts and Telecommunications), Naiyan Wang (Beijing University of Posts and Telecommunications)
Object DetectionSegmentationDepth EstimationAutonomous DrivingConvolutional Neural NetworkContrastive LearningOptical FlowVideo
🎯 What it does: Generate unsupervised instance masks using motion boundaries in videos, and train a visual encoder through contrastive learning to learn object-centered visual representations.
Object-centric 3D Motion Field for Robot Learning from Human Videos
Zhao-Heng Yin (University of California Berkeley), Pieter Abbeel (University of California Berkeley)
Robotic IntelligenceReinforcement Learning from Human FeedbackConvolutional Neural NetworkReinforcement LearningDiffusion modelVideoPoint Cloud
🎯 What it does: By extracting the 3D motion field of object centers from human RGBD videos, a robot control strategy is trained to achieve zero-shot learning.
Object-centric binding in Contrastive Language-Image Pretraining
Rim Assouel (Meta), Adriana Romero-Soriano (Canada CIFAR AI Chair)
RecognitionObject DetectionRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A pre-training method named OC-CLIP is proposed, which introduces an object-centered binding module and structured similarity into the CLIP framework to enhance the understanding of multi-object composite scenes.
Object-Centric Concept-Bottlenecks
David Steinmann (TU Darmstadt), Kristian Kersting (TU Darmstadt)
Object DetectionExplainability and InterpretabilityContrastive LearningImageBenchmark
🎯 What it does: This paper proposes and implements Object-Centric Concept Bottlenecks (OCB), which combines object detection with concept extraction to achieve stronger interpretability and performance improvements.
Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Prediction
KunHo Heo (Kyung Hee University), MyeongAh Cho (Kyung Hee University)
Object DetectionRepresentation LearningGraph Neural NetworkContrastive LearningPoint Cloud
🎯 What it does: This paper proposes a 3D semantic scene graph prediction framework based on object feature pre-training and relationship feature encoding, significantly improving object recognition and relationship inference performance.
Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations
Gaia Di Lorenzo, Daniel Barath
GenerationCompressionDomain AdaptationRepresentation LearningConvolutional Neural NetworkGaussian SplattingSimultaneous Localization and MappingMultimodalityPoint Cloud
🎯 What it does: For multimodal 3D scenes, we propose the Object-X framework, which learns decodable object-level embeddings that can generate high-quality 3D Gaussian spot reconstructions and can be directly used for downstream tasks such as visual localization, single-image reconstruction, and scene alignment.
Objective Soups: Multilingual Multi-Task Modeling for Speech Processing
A F M Saif (Rensselaer Polytechnic Institute), Tianyi Chen (Cornell Tech)
OptimizationTransformerContrastive LearningAudio
🎯 What it does: A multi-level multi-objective optimization framework (VS-MSP, VC-MSP, VM-MSP) is proposed to unify the training of multilingual speech recognition and translation tasks, addressing the issue of conflicting objectives.
Obliviator Reveals the Cost of Nonlinear Guardedness in Concept Erasure
Ramin Akbari (Michigan State University), Vishnu Boddeti (Michigan State University)
OptimizationSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: A post-processing concept elimination method called Obliviator is proposed, specifically targeting nonlinear statistical dependencies to completely eliminate unwanted attributes while maintaining task utility.
OCN: Effectively Utilizing Higher-Order Common Neighbors for Better Link Prediction
Juntong Wang (Peking University), Muhan Zhang (Peking University)
Recommendation SystemOptimizationGraph Neural NetworkGraph
🎯 What it does: The study focuses on link prediction and proposes the OCN method, which effectively utilizes high-order common neighbors and eliminates redundancy and over-smoothing issues through orthogonalization and path normalization.
OCTDiff: Bridged Diffusion Model for Portable OCT Super-Resolution and Enhancement
Ye Tian (Columbia University), Kaveri A. Thakoor (Columbia University)
RestorationSuper ResolutionConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Proposes a bridge diffusion model OCTDiff to enhance the resolution and quality of portable OCT images.
ODG: Occupancy Prediction Using Dual Gaussians
Yunxiao Shi (Qualcomm AI Research), Fatih Porikli (Qualcomm Technologies, Inc)
SegmentationAutonomous DrivingTransformerGaussian SplattingPoint Cloud
🎯 What it does: A 3D occupancy prediction method ODG based on dual Gaussian queries is proposed, utilizing dynamic and static Gaussian queries to model variable and invariant objects in the scene, respectively.
Off-policy Reinforcement Learning with Model-based Exploration Augmentation
Likun Wang (Tsinghua University), Shengbo Eben Li (Tsinghua University)
Reinforcement LearningDiffusion modelWorld ModelSequential
🎯 What it does: The MoGE framework is proposed, which utilizes conditional diffusion to generate key unexplored states and synthesizes corresponding transitions using a first-order world model, thereby adding high-quality exploration samples to the offline replay buffer.
Offline Actor-Critic for Average Reward MDPs
William Powell (University of Wisconsin Madison), Hanbaek Lyu (University of Wisconsin Madison)
OptimizationComputational EfficiencyReinforcement Learning
🎯 What it does: This paper studies offline policy optimization in large or infinite state spaces and proposes a pessimistic actor-critic method that uses a computationally efficient class of linear functions for value function estimation.
Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations
Vivek Myers (University of California Berkeley), Sergey Levine (University of California Berkeley)
Reinforcement LearningContrastive LearningSequentialBenchmark
🎯 What it does: This paper proposes a new offline goal-oriented reinforcement learning method called Temporal Metric Distillation (TMD), which achieves effective inference of goal-reaching strategies by learning a 'quasi-metric' representation that satisfies the triangle inequality.
Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies
Runze Yan (Emory University), Xiao Hu
OptimizationReinforcement LearningBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes a model-based offline safe reinforcement learning framework, OGSRL, aimed at learning treatment strategies from clinical data that are both safe and can surpass those of clinical doctors.
Offline imitation learning in $Q^\pi$-realizable MDPs without expert realizability
Antoine Moulin, Luca Viano
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabularSequential
🎯 What it does: This paper proposes an offline imitation learning method called SPOIL, which utilizes the linear Qπ realizability assumption of MDPs to learn an approximate expert policy from a dataset of expert state-action pairs without requiring the expert to be realizable.
Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization
Subhojyoti Mukherjee (Adobe Research), Branislav Kveton (Adobe Research)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper proposes an offline reinforcement learning framework based on reward-weighted fine-tuning (Refit and Swift) for training multi-turn question-answer dialogue strategies.
OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
Wenzhen Yue (Peking University), Ji Shi (Peking University)
TransformerTime SeriesFinance Related
🎯 What it does: This paper proposes OLinear, a linear multivariate time series forecasting model that performs encoding and decoding in the orthogonal transformation domain. It utilizes OrthoTrans to adaptively transform the original time series data and then employs the NormLin linear layer to model cross-variable relationships.
OMiSO: Adaptive optimization of state-dependent brain stimulation to shape neural population states
Yuki Minai (Carnegie Mellon University), Matthew A. Smith (Carnegie Mellon University)
OptimizationConvolutional Neural NetworkRecurrent Neural NetworkReinforcement LearningTime SeriesBiomedical Data
🎯 What it does: An online micro-stimulation optimization framework OMiSO has been developed, which can select electrode stimulation patterns based on the brain state before stimulation, allowing neural population activity to approach a preset target state.
Omni-DNA: A Genomic Model Supporting Sequence Understanding, Long-context, and Textual Annotation
Zehui Li (Imperial College London), Caihua Shan (Microsoft Research)
TransformerLarge Language ModelSupervised Fine-TuningTextSequentialBiomedical Data
🎯 What it does: Proposes the Omni-DNA foundational model series, combining autoregressive Transformer, SEQPACK compression, and SEQ2FUNC generative datasets to support sequence understanding, long-context reasoning, and text annotation;
Omni-Mol: Multitask Molecular Model for Any-to-any Modalities
Chengxin Hu (National University of Singapore), Haixin Wang (University of California)
Drug DiscoveryTransformerLarge Language ModelMixture of ExpertsMultimodalityGraph
🎯 What it does: A unified multimodal large language model, Omni-Mol, has been developed, capable of simultaneously handling 16 tasks including molecule-to-molecule, molecule-to-text, molecule-to-numeric, and text-to-molecule, and a dataset covering 1.4M samples has been constructed;
Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration
Hao Zhong (Zhejiang University), Chunhua Shen (Zhejiang University)
SegmentationReinforcement LearningVideoTextMultimodalityAudio
🎯 What it does: This paper proposes a reinforcement learning-based two-system architecture, Omni-R1, capable of achieving long-term reasoning and fine-grained pixel-level understanding in video-audio-text tasks simultaneously.
OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time Scales
Tung Nguyen (University of California, Los Angeles), Aditya Grover (University of California, Los Angeles)
TransformerDiffusion modelAuto EncoderTime SeriesSequential
🎯 What it does: A model called OmniCast is proposed for probabilistic weather forecasting from short-term to seasonal scales.
OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Data
Yiren Song (Show Lab National University of Singapore), Mike Zheng Shou (Show Lab National University of Singapore)
Image TranslationGenerationTransformerDiffusion modelImage
🎯 What it does: A pluggable 'OmniConsistency' plugin is designed to achieve structural and detail consistency in image stylization through Diffusion Transformer, separating style learning and consistency learning in a two-stage training process to achieve high-fidelity consistency across styles.
Omnidirectional 3D Scene Reconstruction from Single Image
Ren Yang, Yan Lu
GenerationOptimizationDiffusion modelGaussian SplattingImage
🎯 What it does: This study presents Omni3D, a complete workflow for generating panoramic 3D scenes from a single image, synthesizing multi-view images through a diffusion model and constructing a freely renderable 3D representation using 3D Gaussian Splatting.
OmniDraft: A cross-vocabulary, online adaptive drafter for on-device speculative decoding
Ramchalam Kinattinkara Ramakrishnan (Qualcomm AI Research), Xiaopeng Zhang (Qualcomm AI Research)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the OmniDraft framework, which utilizes a unified lightweight draft model combined with an online n-gram cache and mixed distillation to achieve speculative decoding across vocabularies and perform online adaptation on the device side.
OmniFC: Rethinking Federated Clustering via Lossless and Secure Distance Reconstruction
Jie Yan (Central University of Finance and Economics), Zhong-Yuan Zhang (Central University of Finance and Economics)
Federated LearningSafty and PrivacyMultimodality
🎯 What it does: A unified framework named OmniFC is proposed, which achieves federated clustering through lossless and secure distance reconstruction, aiming to protect privacy and enhance robustness under non-independent and identically distributed (Non-IID) data.
OmniGaze: Reward-inspired Generalizable Gaze Estimation in the Wild
Hongyu Qu (Nanjing University of Science and Technology), Jinhui Tang (Nanjing Forestry University)
Pose EstimationDomain AdaptationTransformerReinforcement LearningImage
🎯 What it does: A semi-supervised framework named OMNIGAZE is proposed for estimating 3D gaze direction from facial images in the real world, utilizing a large number of unlabeled facial images and filtering and weighting pseudo-labels through a reward model.
OmniGen-AR: AutoRegressive Any-to-Image Generation
Junke Wang (Fudan University), Yu-Gang Jiang (Fudan University)
GenerationData SynthesisTransformerImageVideoTextMultimodality
🎯 What it does: A unified autoregressive (AR) framework called OmniGen-AR is proposed for generating high-quality images or videos from multimodal conditions such as text, spatial signals (segmentation, depth), and visual context (editing, frame prediction, text-video).
Omnipresent Yet Overlooked: Heat Kernels in Combinatorial Bayesian Optimization
Colin Doumont (ETH Zurich), Henry Moss (University of Cambridge)
OptimizationGraphBenchmark
🎯 What it does: This paper constructs a unified framework through the heat kernel, mapping various combinatorial Bayesian optimization (BO) methods (such as CASMOPOLITAN, COMBO, Bounce, etc.) to the same class of kernels, and proves their equivalence both theoretically and practically.
OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions
Cheng Luo (King Abdullah University of Science and Technology), Bernard Ghanem (University of Exeter)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVideoTextMultimodalityAudio
🎯 What it does: We propose OmniResponse, which can synchronously generate the listener's speech, facial expressions, and text responses in real-time during a two-person conversation.
OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation
Bo-Wen Yin (Nankai University), Qibin Hou (Nankai University)
SegmentationTransformerImageMultimodality
🎯 What it does: We propose OmniSegmentor, a flexible and configurable semantic segmentation framework suitable for various visual modalities (RGB, depth, thermal imaging, LiDAR, events); simultaneously, we construct the ImageNeXt dataset to provide data support for large-scale multimodal pre-training.
OmniSVG: A Unified Scalable Vector Graphics Generation Model
Yiying Yang, Yu-Gang Jiang
GenerationTransformerVision Language ModelMultimodalityBenchmark
🎯 What it does: We propose OmniSVG, a unified framework for generating complex SVGs using pre-trained vision-language models (VLM);
OmniSync: Towards Universal Lip Synchronization via Diffusion Transformers
Ziqiao Peng (Renmin University of China), Jun He (Renmin University of China)
GenerationTransformerDiffusion modelFlow-based ModelVideoBenchmarkAudio
🎯 What it does: Proposes OmniSync, a no-reference, no-mask lip synchronization framework for diverse visual content;
OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimicking
Zhongjian Wang (Alibaba Group), Bang Zhang (Alibaba Group)
GenerationData SynthesisTransformerDiffusion modelGenerative Adversarial NetworkVideoTextMultimodalityAudio
🎯 What it does: This paper presents OmniTalker, an end-to-end, text-driven framework for real-time synchronized generation of speech and video, capable of producing high-quality talking head videos and accompanying audio from a single segment of text while maintaining the target identity's speaking and facial action style.
OmniTry: Virtual Try-On Anything without Masks
Yutong Feng (Kunbyte AI), Bin Wang (Kunbyte AI)
Image TranslationGenerationTransformerDiffusion modelImageMultimodalityBenchmark
🎯 What it does: The OmniTry framework is proposed, achieving virtual try-on of any wearable object without masks; it learns positioning using a large number of unpaired portrait images in the first stage, and then maintains object identity using a limited amount of paired data.
OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions
Yuanhao Cai (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
GenerationData SynthesisLarge Language ModelDiffusion modelVideoTextMultimodality
🎯 What it does: An end-to-end feedforward framework called OmniVCus is proposed, which enables video customization and editing under single subject, multiple subjects, and multimodal control (depth, masks, camera trajectories, text instructions).
OmniZoom: A Universal Plug-and-Play Paradigm for Cross-Device Smooth Zoom Interpolation
Xiaoan Zhu (Zhejiang University), Huajun Feng (Zhejiang University)
Image TranslationRestorationData SynthesisGaussian SplattingOptical FlowImageVideo
🎯 What it does: This study investigates the geometric and lighting inconsistencies during the zoom process of dual-camera smartphones and proposes OmniZoom, a universal plugin-based framework that achieves smooth zoom interpolation through cross-device virtual data generation and the 3D-TPR method.
On Agnostic PAC Learning in the Small Error Regime
Julian Asilis (University of Southern California), Grigoris Velegkas (Google Research)
🎯 What it does: This paper studies agnostic PAC learning under small error τ and proposes a new learner that achieves the optimal error upper bound.
On Efficiency-Effectiveness Trade-off of Diffusion-based Recommenders
Wenyu Mao (University of Science and Technology of China), Xiang Wang (University of Science and Technology of China)
Recommendation SystemTransformerDiffusion modelSequential
🎯 What it does: A two-stage TA-Rec framework is proposed, where in the pre-training stage, time consistency regularization (TCR) is used to smooth the denoising function of the diffusion model, enabling recommendations to be made in a single step; then in the fine-tuning stage, adaptive preference alignment (APA) dynamically adjusts the optimization strength based on time steps and the similarity of positive and negative samples, thereby enhancing the recommendation effectiveness.
On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models
Hoigi Seo (Seoul National University), Se Young Chun (Seoul National University)
Object DetectionAnomaly DetectionAdversarial AttackTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes an attention masking method based on uncertain visual markers in visual encoders to suppress object hallucinations in large visual-language models.
On Evaluating LLM Alignment by Evaluating LLMs as Judges
Yixin Liu (Yale University), Arman Cohan (Yale University)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper studies the consistency (GE-consistency) of large language models (LLMs) in the roles of generation and evaluation, and based on this, proposes an alignment evaluation framework ALIGNEVAL that directly assesses generated results without relying on LLM judges.
On Evaluating Policies for Robust POMDPs
Merlijn Krale (Radboud University Nijmegen), Nils Jansen (Ruhr-University Bochum)
OptimizationReinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes a complete evaluation process for RPOMDP (Robust Partially Observable Markov Decision Process), including appropriate benchmark environments, precise policy evaluation methods, and computable baseline algorithms.
On Extending Direct Preference Optimization to Accommodate Ties
Jinghong Chen (University of Cambridge), Bill Byrne (University of Cambridge)
Recommendation SystemOptimizationReinforcement LearningText
🎯 What it does: This paper extends Direct Preference Optimization (DPO) to two variants, DPO-RK and DPO-D, that can handle ties, using the three-valued preference models of Rao-Kupper and Davidson, respectively. Experiments validate that these variants do not lead to performance degradation when including tie data, and even achieve improvements in translation, summarization, and mathematical reasoning tasks.
On Fairness of Unified Multimodal Large Language Model for Image Generation
Ming Liu (Iowa State University), Wensheng Zhang (Iowa State University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningImageMultimodality
🎯 What it does: Evaluate the fairness of image generation in Unified Multimodal Large Language Models (U-MLLM), identify sources of bias, and propose a debiasing method based on balanced preference loss.
On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning
Till Freihaut (University of Zurich), Giorgia Ramponi (University of Zurich)
Reinforcement Learning
🎯 What it does: This paper proposes a theoretical framework for Multi-Agent Inverse Reinforcement Learning (MAIRL) and analyzes the definition and properties of the feasible reward set when solving multi-agent Markov games in general.
On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation
Liyao Tang (University of Sydney), Dacheng Tao (Nanyang Technological University)
SegmentationConvolutional Neural NetworkSupervised Fine-TuningPoint Cloud
🎯 What it does: Proposed the Geometry Encoding Mixer (GEM) module, achieving geometry-aware parameter-efficient fine-tuning.
On Group Sufficiency Under Label Bias
Haoran Zhang (Massachusetts Institute of Technology), Marzyeh Ghassemi (Massachusetts Institute of Technology)
ImageTextTabular
🎯 What it does: Research on learning methods to achieve group sufficiency under label bias conditions.
On Hierarchies of Fairness Notions in Cake Cutting: From Proportionality to Super Envy-Freeness
Arnav Mehra (Capital One), Alexandros Psomas (Purdue University)
🎯 What it does: This paper proposes two new fairness hierarchies—Harmonically Coalition-Resistant (HCR) and Linearly Coalition-Resistant (LCR)—and studies their query complexity in the Robertson-Webb query model. It proves that HCR_n can be solved in O(n^4) queries, while HCR_2 and all higher k require Ω(n^2) queries; LCR_2 cannot even be obtained through a finite number of queries, and provides a feasible algorithm for approximating δ-LCR_n.
On Inductive Biases That Enable Generalization in Diffusion Transformers
Jie An (University of Rochester), Alex Schwing
GenerationData SynthesisTransformerDiffusion modelImage
🎯 What it does: This study investigates the inductive bias of the Diffusion Transformer (DiT) and finds that attention locality is key to its generalization ability.
On Learning Verifiers and Implications to Chain-of-Thought Reasoning
Maria Florina Balcan, Dravyansh Sharma (Northwestern University)
GenerationOptimizationTextChain-of-Thought
🎯 What it does: This paper addresses the verifiability issues of Chain-of-Thought generation by constructing a PAC learning framework to learn a verifier that can determine whether the steps of natural language reasoning are correct.
On Linear Mode Connectivity of Mixture-of-Experts Architectures
Viet-Hoang Tran (National University of Singapore), Tan Minh Nguyen
TransformerMixture of ExpertsImageText
🎯 What it does: This paper studies the linear mode connectivity (LMC) in the mixture of experts (MoE) architecture and provides a matching algorithm to find low-loss linear paths between independently trained models.
On Local Limits of Sparse Random Graphs: Color Convergence and the Refined Configuration Model
Alexander Pluska (TU Wien), SAGAR MALHOTRA
Graph Neural NetworkGraph
🎯 What it does: This paper introduces a novel local limit concept called 'color convergence' and constructs a Refined Configuration Model (RCM) based on this concept, which can cover all sparse random graph models to characterize the learnability of MPNN on large-scale sparse graphs.
On Logic-based Self-Explainable Graph Neural Networks
Alessio Ragno (INSA Lyon), Céline Robardet (INSA Lyon)
Explainability and InterpretabilityKnowledge DistillationGraph Neural NetworkGraph
🎯 What it does: A self-explanatory graph neural network architecture named LogiX-GIN is proposed, which can directly generate interpretable logical rules during the learning process, balancing graphical reasoning and interpretability.
On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts
Fanqi Yan (University of Texas at Austin), Alessandro Rinaldo (University of Texas at Austin)
Mixture of ExpertsTabular
🎯 What it does: This study investigates the convergence rate of maximum likelihood estimation for pre-trained models and prompt models in softmax mixture expert models.
On Optimal Steering to Achieve Exact Fairness
mohit sharma, Rajiv Ratn Shah (Indraprastha Institute of Information Technology Delhi)
ClassificationOptimizationKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: This paper proposes an 'optimal guidance' method based on distribution correction, which eliminates bias while maintaining or improving model performance by finding the ideal distribution closest to the original distribution (which satisfies precise fairness for any cost-sensitive risk under the Bayes optimal classifier).
On Reasoning Strength Planning in Large Reasoning Models
Leheng Sheng (National University of Singapore), Tat-Seng Chua (National University of Singapore)
Large Language ModelText
🎯 What it does: This study explores whether large reasoning models (LRM) plan the reasoning intensity (i.e., the length of reasoning steps) before generating answers. By predicting reasoning length using linear probes, it was found that there exists a single pre-allocated directional vector in the activation space, which encodes reasoning intensity through its magnitude and can causally control the logits of the /think end token, thus achieving controllable reasoning length. Furthermore, two potential applications are proposed: excessive reasoning detection and efficient reasoning.
On scalable and efficient training of diffusion samplers
Minkyu Kim (Korea Advanced Institute of Science and Technology), Minsu Kim (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisOptimizationReinforcement LearningDiffusion modelSequentialStochastic Differential Equation
🎯 What it does: A scalable and efficient diffusion sampler training framework SGDS is proposed, which combines gradient-guided MCMC Searcher with neural diffusion Learner. It achieves a hybrid learning of offline and online sampling through trajectory balancing objectives and utilizes Random Network Distillation (RND) to generate exploration rewards, periodically resetting the model to eliminate primacy bias.
On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
Huan Li (Nankai University), Zhouchen Lin (Peking University)
OptimizationImage
🎯 What it does: This paper studies the convergence of the AdamW optimizer, establishing that its convergence rate under the ℓ1 norm is 1/K ∑K k=1 E[∥∇f(xk)∥1] ≤ O(√dC K^(1/4)).
On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study
Riccardo Alberghi (Bocconi University), Luca Saglietti (Bocconi University)
OptimizationComputational EfficiencyTransformerGraphChain-of-Thought
🎯 What it does: In a custom hierarchical directed acyclic graph shortest path task, the author trained a Transformer model based on next word prediction, exploring the impact of different lengths and structures of reasoning trajectories (Chain-of-Thought) on model generalization.
On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity
Quentin Bertrand (University Jean Monnet Saint-Étienne), Rémi Emonet
Flow-based ModelImage
🎯 What it does: This paper studies the generalization mechanism of flow matching models, exploring whether target noise is the main factor leading to generalization, and proposes a method for training using a closed-form optimal velocity field.
On the Coexistence and Ensembling of Watermarks
Aleksandar Petrov (Adobe Research), John Collomosse (University of Oxford)
GenerationGenerative Adversarial NetworkImage
🎯 What it does: This paper explores the possibility of coexistence and non-interference of deep image watermarks within the same image, and proposes a method to enhance capacity through watermark integration.
On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization
Jincheng Cao (University of Texas Austin), Aryan Mokhtari (University of Texas Austin)
Optimization
🎯 What it does: This paper addresses non-convex simple bilevel optimization problems, proposing the definition of (ε_f, ε_g)-stationarity and presenting a discrete-time algorithm based on Dynamic Barrier Gradient Descent (DBGD). It proves that under constant step size and adaptive λ, it can converge to a solution satisfying this stationarity in O(max(ε_f^{-3+p1+p}, ε_g^{-3+p2g})) steps.
On the Convergence of Single-Timescale Actor-Critic
Navdeep Kumar (Technion), Shie Mannor (Technion)
Reinforcement LearningTabularOrdinary Differential Equation
🎯 What it does: This study investigates the global convergence of single-time-scale actor-critic algorithms on finite state discounted MDPs and provides a theoretical upper bound on sample complexity of O(ε⁻³).
On the Convergence of Stochastic Smoothed Multi-Level Compositional Gradient Descent Ascent
Xinwen Zhang (Temple University), Hongchang Gao (Temple University)
OptimizationImage
🎯 What it does: A smooth variational descent-ascent algorithm is proposed for stochastic multi-layer composite min-max optimization problems, and a staged translation algorithm is designed under both-sided PL conditions, achieving convergence of (ε, ε/√κ) and ε-stationary;
On the creation of narrow AI: hierarchy and nonlocality of neural network skills
Eric J Michaud, Max Tegmark (Massachusetts Institute of Technology)
CompressionOptimizationKnowledge DistillationTransformerSupervised Fine-TuningTabularSequential
🎯 What it does: This study investigates how to derive high-performance, resource-efficient narrow AI models from large general models, explores the impact of task hierarchy on learning, and validates the effects of sparse regularization and pruning in compressing and removing unnecessary skills.
On the Edge of Memorization in Diffusion Models
Sam Buchanan, Valentin De Bortoli
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper constructs a theoretical laboratory for systematically analyzing the balance between memorization and generalization in diffusion models. It derives a closed-form approximation of training loss through high-dimensional asymptotic analysis, predicting and validating the linear relationship between model size and the memorization critical point.
On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization
Wenlong Deng (University of British Columbia), Christos Thrampoulidis
OptimizationReinforcement LearningTabular
🎯 What it does: This study investigates the 'Lazy Likelihood Displacement (LLD)' problem caused by negative gradients in Group Relative Policy Optimization (GRPO) and proposes the Negative Token Hidden Reward (NTHR) along with a token-level selective penalty strategy to alleviate this issue.
On the Emergence of Linear Analogies in Word Embeddings
Daniel James Korchinski, Matthieu Wyart (Johns Hopkins University)
Text
🎯 What it does: A co-occurrence generation model based on binary semantic attributes with mutually independent attributes is constructed to explain the linear analogy phenomenon in word embeddings.
On the Empirical Power of Goodness-of-Fit Tests in Watermark Detection
Weiqing He (University of Pennsylvania), Qi Long (University of Pennsylvania)
Large Language ModelText
🎯 What it does: This paper applies traditional Goodness-of-Fit (GoF) tests to LLM text watermark detection and systematically evaluates the performance of eight GoF methods under three watermark schemes.
On the Entropy Calibration of Language Models
Steven Cao (Stanford University), Percy Liang (Stanford University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the entropy calibration problem of language models, exploring whether the entropy of generated text is consistent with the logarithmic loss of real text;
On the Existence and Complexity of Core-Stable Data Exchanges
Jiaxin Song (University of Illinois), Bhaskar Ray Chaudhury (University of Illinois)
Tabular
🎯 What it does: This paper introduces the concept of core stability in data exchange and proves that core stable exchanges always exist when the utility function is concave and the cost function is convex.
On the Expressive Power of Mixture-of-Experts for Structured Complex Tasks
Mingze Wang (Peking University), Weinan E (Peking University)
Mixture of Experts
🎯 What it does: This paper systematically studies the expressive power of Mixture of Experts (MoEs) in modeling complex tasks, particularly in low-dimensional and sparse structures.
On the Global Optimality of Policy Gradient Methods in General Utility Reinforcement Learning
Anas Barakat (Singapore University of Technology and Design), Amrit Singh Bedi (University of Central Florida)
OptimizationReinforcement Learning
🎯 What it does: A policy gradient algorithm suitable for general utility reinforcement learning (RLGU) is proposed, proving global optimality in the discrete tabular case, and achieving scalable policy gradient solutions in large state-action spaces through maximum likelihood estimation (MLE) of the occupancy measure.
On the Hardness of Approximating Distributions with Tractable Probabilistic Models
John Leland (Arizona State University), YooJung Choi (Arizona State University)
🎯 What it does: This paper studies the difficulty of approximating arbitrary distributions with finite size in probabilistic models (such as tractable probabilistic circuits) while maintaining tractable inference.
On the Hardness of Conditional Independence Testing In Practice
Zheng He (University of British Columbia), Danica J. Sutherland (University of British Columbia)
Tabular
🎯 What it does: This paper studies the kernel-based conditional independence test (KCI) and generalized covariance measure (GCM) methods, clarifying the Type I error caused by regression errors and the impact of conditional kernel selection on test power, and provides corresponding theoretical boundaries and empirical validation.
On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting
Yisong Fu (Chinese Academy of Sciences), Yongjun Xu (Chinese Academy of Sciences)
OptimizationComputational EfficiencyTransformerTime Series
🎯 What it does: STELLA is proposed, a lightweight meteorological time series prediction model that uses only spatiotemporal position embeddings and MLP.
On the Loss of Context Awareness in General Instruction Fine-tuning
Yihan Wang (University of California Los Angeles), Cho-Jui Hsieh (University of California Los Angeles)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study investigates the decline in contextual awareness of large language models after supervised instruction fine-tuning and proposes a method to restore this capability through conditional fine-tuning by inserting context-dependent indicators into the instructions.
On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspective
Behrad Moniri (University of Pennsylvania), Hamed Hassani (University of Pennsylvania)
OptimizationKnowledge DistillationTabular
🎯 What it does: This paper reveals three core mechanisms of the weak to strong generalization phenomenon through theoretical analysis of a simple model, exploring how the student model can outperform the teacher model in the presence of imperfect labels.
On the necessity of adaptive regularisation: Optimal anytime online learning on $\boldsymbol{\ell_p}$-balls
Emmeran Johnson (Imperial College London), Patrick Rebeschini (University of Oxford)
Optimization
🎯 What it does: This paper studies online convex optimization on ℓ_p balls (p>2), distinguishing between high-dimensional (d>T) and low-dimensional (d≤T) ranges, and proves that using adaptive regularization with FTRL can achieve the optimal regret upper bound at any time under an unknown time window.
On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
Shaocong Ma (University of Maryland), Heng Huang (University of Maryland)
OptimizationLarge Language ModelTextTabular
🎯 What it does: This paper proposes a class of unbiased zero-order gradient estimators based on telescoping series, providing four specific implementations. Theoretically, it proves that their variance is controllable and achieves optimal complexity, while also validating performance in synthetic tasks and fine-tuning large language models.
On the Optimality of the Median-of-Means Estimator under Adversarial Contamination
Xabier de Juan (Basque Center of Applied Mathematics), Santiago Mazuelas (Basque Center of Applied Mathematics)
OptimizationAdversarial Attack
🎯 What it does: This paper studies the optimality of the median of means (MoM) estimator in mean estimation under adversarial contamination, providing upper and lower bounds on the error of MoM in multi-class distributions.
On the rankability of visual embeddings
Ankit Sonthalia (University of Tuebingen), Seong Joon Oh (University of Tuebingen)
Representation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: This study explores whether visual embedding models can capture continuous ordinal attributes in a linear manner, specifically whether they possess the property of rankability, and evaluates this across various encoders and attributes.
On the Relation between Rectified Flows and Optimal Transport
Johannes Hertrich (Paris Dauphine University - PSL & Inria Mokaplan), Julie Delon (ENS Paris)
OptimizationFlow-based ModelRectified FlowTabular
🎯 What it does: This paper studies the relationship between Rectified Flows (also known as Flow Matching) and Optimal Transport (OT), and conducts a systematic analysis of their theoretical properties and practical effects.
On the Robustness of Transformers against Context Hijacking for Linear Classification
Tianle Li (University of Hong Kong), Difan Zou (University of Hong Kong)
ClassificationAdversarial AttackMeta LearningTransformerLarge Language ModelTabular
🎯 What it does: This study investigates the robustness of Transformer models against 'context hijacking' attacks, establishing a theoretical framework and validating it on multi-layer linear Transformers.
On the Robustness of Verbal Confidence of LLMs in Adversarial Attacks
Stephen Obadinma (Queen's University), Xiaodan Zhu (Queen's University)
Adversarial AttackLarge Language ModelTextChain-of-Thought
🎯 What it does: This study investigates the robustness of verbal confidence in large language models (LLMs) under adversarial attacks and proposes two main categories of confidence attack methods based on perturbations and jailbreaks.
On the Role of Hidden States of Modern Hopfield Network in Transformer
Tsubasa Masumura (Rikkyo University), Masato Taki (Rikkyo University)
TransformerLarge Language ModelSupervised Fine-TuningImageText
🎯 What it does: Introducing the hidden states of modern Hopfield networks into Transformers, we propose Modern Hopfield Attention (MHA) and validate its effectiveness in Vision Transformers and the GPT series.
On the SAC-BL Algorithm for Anomaly Detection
Xinsong Ma (Wuhan University), Weiwei Liu (Wuhan University)
Anomaly DetectionImage
🎯 What it does: This paper conducts an in-depth analysis of the FPR of the SAC-BL algorithm in visual anomaly detection and proposes an improved method called SAC-SBL.
On the Sample Complexity Bounds of Bilevel Reinforcement Learning
Mudit Gaur (Purdue University), Vaneet Aggarwal (Purdue University)
OptimizationReinforcement Learning
🎯 What it does: The first sample complexity upper bound for parametric bi-level reinforcement learning (BRL) is proposed, along with a feasible first-order algorithm without second-order derivatives.
On the Sample Complexity of Differentially Private Policy Optimization
Yi He (Wayne State University), Xingyu Zhou (Wayne State University)
OptimizationSafty and PrivacyReinforcement Learning
🎯 What it does: The study investigates the use of differential privacy in policy optimization methods within reinforcement learning, providing upper bounds on sample complexity for different algorithms (PG, NPG, REBEL) under differential privacy constraints.
On the sample complexity of semi-supervised multi-objective learning
Tobias Wegel (ETH Zurich), Fanny Yang (ETH Zurich)
Optimization
🎯 What it does: This paper studies semi-supervised multi-objective learning, exploring the impact of labeled and unlabeled data on learning multi-objective balance under different losses;
On the Stability and Generalization of Meta-Learning: the Impact of Inner-Levels
Wenjun Ding (Central South University), Zhe Qu (Central South University)
Meta LearningImage
🎯 What it does: This study investigates the impact of the inner step size on generalization in Meta-learning, constructs two frameworks (GDF and PDF), and provides generalization upper bounds in both convex and non-convex cases, proposing a new meta objective function to reduce generalization error.
On the Stability of Graph Convolutional Neural Networks: A Probabilistic Perspective
Ning Zhang (University of Oxford), Xiaowen Dong (University of Oxford)
ClassificationAdversarial AttackGraph Neural NetworkGraph
🎯 What it does: This paper proposes a stability analysis framework for Graph Convolutional Neural Networks (GCNN) from a probabilistic perspective, deriving the expected embedding perturbation formulas for graph filters and multi-layer GCNNs under edge perturbations, and based on this, designs a task-independent Prob-PGD adversarial attack method.
On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
Moritz Haas (University of Tübingen), Leena Chennuru Vankadara (University College London)
OptimizationImage
🎯 What it does: Investigate whether cross-entropy loss can maintain stable training with a large learning rate and retain feature learning under standard parameterization (He initialization + global learning rate) during width scaling.
On the Universal Near Optimality of Hedge in Combinatorial Settings
Zhiyuan Fan (Massachusetts Institute of Technology), Gabriele Farina (Massachusetts Institute of Technology)
Optimization
🎯 What it does: This paper studies the performance of the classic Hedge algorithm in combinatorial decision problems, proving that its regret upper bound under any combinatorial set X ⊆ {0,1}^d is O(√T log|X|), and provides a matching lower bound, demonstrating that this upper bound is nearly optimal in the log d magnitude; further analysis of specific sets (m-sets) and multi-task learning reveals that Hedge has a √log d lower bound disadvantage on m-sets, while maintaining optimality in multi-task learning; finally, it establishes the iterative equivalence of Hedge and Online Mirror Descent (OMD) on the sparse path problem in directed acyclic graphs (DAGs), proving that OMD with sparse entropy regularization can achieve the same optimal regret as Hedge.
On the Value of Cross-Modal Misalignment in Multimodal Representation Learning
Yichao Cai (Australian Institute for Machine Learning), Javen Qinfeng Shi (Australian Institute for Machine Learning)
Representation LearningContrastive LearningMultimodality
🎯 What it does: This paper reveals that multimodal contrastive learning (MMCL) under cross-modal mismatch will only learn a semantic subspace that is unaffected by selection and perturbation biases, while mismatched semantics will be discarded, through the establishment of a latent variable model and theoretical identifiability analysis.
On the VC dimension of deep group convolutional neural networks
Anna Sepliarskaia (University of Twente), Johannes Schmidt-Hieber (University of Twente)
Convolutional Neural Network
🎯 What it does: This paper analyzes the generalization ability of deep group convolutional neural networks (GCNN) with ReLU activation functions through VC dimension theory, providing upper and lower bounds for GCNN and proving that its dimension grows logarithmically with the resolution of group discretization.