NeurIPS 2025 Papers — Page 43
Conference on Neural Information Processing Systems · 5275 papers
ShotBench: Expert-Level Cinematic Understanding in Vision-Language Models
Hongbo Liu (Tongji University), Ziwei Liu (Nanyang Technological University)
TransformerSupervised Fine-TuningReinforcement LearningVision Language ModelImageVideoTextBenchmark
🎯 What it does: A new evaluation benchmark called ShotBench is proposed, which includes expert-annotated QA for 3.5k movie shots, covering 8 dimensions of cinematography, and trains the ShotVL model based on the 70k ShotQA dataset.
Show-o2: Improved Native Unified Multimodal Models
Jinheng Xie (Show Lab National University of Singapore), Mike Zheng Shou (Show Lab National University of Singapore)
GenerationData SynthesisTransformerVision Language ModelFlow-based ModelImageVideoTextMultimodality
🎯 What it does: A multimodal model called Show-o2 has been constructed to uniformly process text, images, and videos.
Siegel Neural Networks
Xuan Son Nguyen (Cergy Paris University), Nistor Grozavu (Cergy Paris University)
ClassificationGraph
🎯 What it does: Proposed a Siegel space-based Riemannian multi-class linear regression (MLR) and fully connected (FC) layers, constructing a Siegel neural network, and validated it on radar clutter classification and node classification tasks.
SIFusion: A Unified Fusion Framework for Multi-granularity Arctic Sea Ice Forecasting
Jingyi Xu (Fudan University), LEI BAI
TransformerTime Series
🎯 What it does: Designed and implemented the SIFusion framework, which unifies the joint prediction of Arctic sea ice concentration data using three time granularities: daily, weekly, and monthly averages.
SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation
Yanwei Ren (Beihang University), Liu Liu (Beihang University)
Large Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Using the chain reasoning paths generated by MCTS, we improve the reasoning process by reusing the information of discarded sibling nodes to construct a high-quality small dataset.
Sign-In to the Lottery: Reparameterizing Sparse Training
Advait Gadhikar (CISPA Helmholtz Center for Information Security), Rebekka Burkholz (CISPA Helmholtz Center for Information Security)
OptimizationConvolutional Neural NetworkTransformerReinforcement LearningImage
🎯 What it does: A dynamic reparameterization method called Sign-In is proposed to address the optimization problem of parameter signs when training sparse neural networks from scratch.
Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
David Heineman (Allen Institute for Artificial Intelligence), Jesse Dodge (Allen Institute for Artificial Intelligence)
Large Language ModelTextBenchmark
🎯 What it does: This paper constructs a signal-to-noise ratio (SNR) framework by defining two metrics: signal and noise, to evaluate the reliability of language model benchmarks.
SignFlow Bipartite Subgraph Network For Large-Scale Graph Link Sign Prediction
Yixiao Zhou (Peking University), Tuo Wang (Peking University)
Recommendation SystemComputational EfficiencyKnowledge DistillationGraph Neural NetworkGraph
🎯 What it does: This paper proposes the SignFlow Bipartite Subgraph Network (SBSN), which achieves edge signing prediction in large-scale signed bipartite graphs through subgraph sampling, directional SignFlow aggregation, and node feature distillation.
Silencer: From Discovery to Mitigation of Self-Bias in LLM-as-Benchmark-Generator
Peiwen Yuan (Beijing Institute of Technology), Kan Li (Beijing Institute of Technology)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper studies and mitigates the self-bias of LLM-generated benchmarks, proposing the SILENCER framework.
SilentStriker: Toward Stealthy Bit-Flip Attacks on Large Language Models
HAOTIAN XU, Cheng Zhuo (Zhejiang University)
GenerationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: A covert bit-flipping attack (SilentStriker) for large language models is proposed, which uses a joint loss of key token suppression and perplexity constraints to generate fluent but incorrect text with a small number of bit flips.
Sim-LLM: Optimizing LLM Inference at the Edge through Inter-Task KV Reuse
Ruikun Luo (Huazhong University of Science and Technology), Yun Yang (Swinburne University of Technology)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: In the edge computing scenario, Sim-LLM is proposed to reduce GPU memory usage during LLM inference and improve system throughput by identifying task similarity and reusing the KV cache of previous tasks (especially the upper-level KV).
Simple and Effective Specialized Representations for Fair Classifiers
Alberto Sinigaglia (University of Padua), Gian Antonio Susto (University of Padua)
ClassificationRepresentation LearningContrastive LearningTabular
🎯 What it does: This paper proposes a fair representation learning framework based on Characteristic Function Distance (CFD), which can generate specialized representations for specific tasks without using adversarial training or normalizing flows, while not requiring sensitive attributes during inference.
Simple and Efficient Heterogeneous Temporal Graph Neural Network
YiliWang, Jianliang Gao (Central South University)
Graph Neural NetworkLarge Language ModelGraphTime Series
🎯 What it does: A concise and efficient heterogeneous temporal graph neural network named SE-HTGNN is proposed, which directly embeds temporal information into spatial aggregation through a dynamic attention mechanism and utilizes large language models to generate node type priors to enhance learning effectiveness.
Simple and Optimal Sublinear Algorithms for Mean Estimation
Beatrice Bertolotti (University of Pavia), Sudarshan Shyam (Aarhus University)
OptimizationTabular
🎯 What it does: This paper studies the sublinear multivariate mean estimation problem in d-dimensional Euclidean space and proposes three algorithms with optimal sample complexity to estimate the mean of a point set.
Simple Distillation for One-Step Diffusion Models
Huaisheng Zhu (Pennsylvania State University), Vasant G Honavar
GenerationComputational EfficiencyKnowledge DistillationDiffusion modelContrastive LearningImage
🎯 What it does: Proposes the Contrastive Energy Distillation (CED) method, which distills a multi-step diffusion model into a single-step generator, and the training process does not require auxiliary models or iterative updates, simplifying the training process and improving efficiency.
SimpleStrat: Diversifying Language Model Generation with Stratification
Justin Wong (University of California Berkeley), Joseph E. Gonzalez (University of California Berkeley)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A training-free sampling method is proposed to enhance the response diversity of large language models in multi-answer tasks through steps such as automatic stratification, heuristic estimation, and probabilistic prompting.
Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
Chongyu Fan (Michigan State University), Sijia Liu (IBM Research)
OptimizationTransformerLarge Language ModelText
🎯 What it does: A simple no-reference model-based negative preference optimization framework, SimNPO, is proposed for unlearning in large language models.
SimSort: A Data-Driven Framework for Spike Sorting by Large-Scale Electrophysiology Simulation
Yimu Zhang (Fudan University), Dongsheng Li (Microsoft Research Asia)
ClassificationRecognitionDomain AdaptationRecurrent Neural NetworkTransformerContrastive LearningBiomedical Data
🎯 What it does: A deep learning framework called SimSort, based on large-scale biophysical simulation data pre-training, has been developed to achieve automated sorting of neuronal spikes.
Simulation-Based Inference for Adaptive Experiments
Brian M Cho (Cornell Tech), Nathan Kallus (Netflix)
TabularTime Series
🎯 What it does: This paper proposes a simulation-based adaptive experimental inference method that constructs hypothesis tests and confidence intervals by adding a positive bias to the means of non-target arms during the simulation process (i.e., 'simulation with optimism');
SimulMEGA: MoE Routers are Advanced Policy Makers for Simultaneous Speech Translation
Chenyang Le (Shanghai Jiao Tong University), Yanmin Qian (Shanghai Jiao Tong University)
TransformerMixture of ExpertsAudio
🎯 What it does: This paper presents SimulMEGA, an unsupervised policy learning framework based on a Mixture-of-Experts (MoE) router, designed to automatically determine read/write actions in simultaneous speech translation (SimulST), while being compatible with streaming inference for both speech-to-text and text-to-speech.
Simultaneous Modeling of Protein Conformation and Dynamics via Autoregression
Yuning Shen (ByteDance), Quanquan Gu (ByteDance)
Protein Structure PredictionTransformerDiffusion modelSequentialBiomedical Data
🎯 What it does: The CONFROVER autoregressive framework is proposed for simultaneously learning protein conformation distributions and dynamics, supporting trajectory simulation, time-independent sampling, and conformation interpolation.
Simultaneous Statistical Inference for Off-Policy Evaluation in Reinforcement Learning
Tianpai Luo (Tsinghua University), Weichi Wu (Tsinghua University)
Reinforcement LearningBiomedical Data
🎯 What it does: A global confidence interval inference framework for offline reinforcement learning is proposed, which can simultaneously cover the value function estimates of the entire initial state space.
Simultaneous Swap Regret Minimization via KL-Calibration
Haipeng Luo (University of Southern California), Vatsal Sharan (University of Southern California)
Optimization
🎯 What it does: This paper proposes a stronger calibration concept (pseudo KL-calibration) in online binary classification prediction and proves its equivalence to the swap regret of logarithmic loss.
SimWorld: An Open-ended Simulator for Agents in Physical and Social Worlds
Xiaokang Ye, Lianhui Qin
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningVision Language ModelTextMultimodality
🎯 What it does: An open-source simulation platform named SIMWORLD has been developed for deploying, training, and evaluating large language models (LLMs) / vision-language models (VLMs) agents in realistic city-scale physical and social environments.
SING: SDE Inference via Natural Gradients
Amber Hu (Stanford University), Scott Linderman (Stanford University)
OptimizationComputational EfficiencyTime SeriesSequentialBiomedical DataStochastic Differential Equation
🎯 What it does: This paper proposes a natural gradient variational inference method called SING, which efficiently approximates posterior trajectories and learns drift functions in latent stochastic differential equation (SDE) models.
Single GPU Task Adaptation of Pathology Foundation Models for Whole Slide Image Analysis
Neeraj Kumar (Memorial Sloan Kettering Cancer Center), Chad Vanderbilt (Memorial Sloan Kettering Cancer Center)
ClassificationOptimizationTransformerSupervised Fine-TuningImage
🎯 What it does: A method for adapting the Pathology Foundation Model (PFM) task on whole slide images (WSI) for gene mutation prediction is proposed, which can perform feature extraction and multi-instance learning (MIL) aggregation simultaneously on a single GPU.
Single-pass Adaptive Image Tokenization for Minimum Program Search
Shivam Duggal (Massachusetts Institute of Technology), Phillip Isola (Massachusetts Institute of Technology)
CompressionGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a single-channel adaptive image tokenizer called KARL, which can predict the minimum number of tokens required for each image in a single forward pass to achieve optimal compression.
Single-Step Operator Learning for Conditioned Time-Series Diffusion Models
Hui Chen (University of Wisconsin Madison), Vikas Singh (University of Wisconsin Madison)
GenerationData SynthesisOptimizationComputational EfficiencyDiffusion modelTime SeriesSequential
🎯 What it does: A single-step inverse operator learning framework is proposed, utilizing frequency domain information and semigroup properties to achieve one-time generation in conditional time series diffusion models, significantly reducing sampling steps.
Single-Teacher View Augmentation: Boosting Knowledge Distillation via Angular Diversity
Seonghoon Yu (Gwangju Institute of Science and Technology), Jeany Son (Pohang University of Science and Technology)
SegmentationKnowledge DistillationContrastive LearningImage
🎯 What it does: By adding a lightweight linear branch to a single teacher network and using constrained angular and internal angle diversity loss, multi-angle multi-view knowledge enhancement is generated, thereby improving the effectiveness of knowledge distillation.
SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference
Jiahui Wang (National University of Singapore), Tong Heng LEE
Pose EstimationDepth EstimationTransformerSupervised Fine-TuningImage
🎯 What it does: Proposes a single RGB reference 6D pose estimation framework called SingRef6D;
Sinusoidal Initialization, Time for a New Start
Alberto Fernández-Hernández (Universitat Politècnica de València), Enrique S. Quintana-Orti
Convolutional Neural NetworkTransformerImageText
🎯 What it does: A completely deterministic weight initialization method based on sine functions (Sinusoidal Initialization) is proposed to replace traditional random initialization;
SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning
Wanjia Zhao (Stanford University), James Zou (Stanford University)
Large Language ModelSupervised Fine-TuningReinforcement LearningTextBiomedical DataPhysics Related
🎯 What it does: The SIRIUS framework is proposed, which achieves self-improvement of multi-agent LLM systems through a self-generated successful interaction experience library and augmented failure trajectories.
SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignment
Qi Xu (Wuhan University), Peidong Liu (Wuhan University)
SegmentationTransformerContrastive LearningImage
🎯 What it does: The SIU3R framework is proposed to achieve simultaneous 3D reconstruction and multi-task scene understanding (semantic, instance, panoramic, text reference, etc.) from unposed images, and localize 3D semantics through pixel-aligned 3D Gaussian representation.
Size-adaptive Hypothesis Testing for Fairness
Antonio Ferrara (CENTAI), Francesco Bonchi (CENTAI)
OptimizationTabularBenchmark
🎯 What it does: A unified, scale-adaptive hypothesis testing framework is proposed, transforming fairness assessment into statistical significance testing, accommodating both large and small sample subgroups.
Sketch-Augmented Features Improve Learning Long-Range Dependencies in Graph Neural Networks
Ryien Hosseini (University of Chicago), Henry Hoffmann (University of Chicago)
Graph Neural NetworkGraph
🎯 What it does: The study proposes a GNN enhancement method based on Sketch random features to address long-range dependencies, over-smoothing, and insufficient expressiveness.
Sketched Adaptive Distributed Deep Learning: A Sharp Convergence Analysis
Zhijie Chen (University of Illinois), Arindam Banerjee (University of Illinois)
OptimizationFederated LearningConvolutional Neural NetworkTransformerImageText
🎯 What it does: In distributed deep learning, a framework called Sketched Adaptive Distributed Learning (SADL) is proposed by combining gradient compression (sketching) with adaptive optimizers;
Sketched Gaussian Mechanism for Private Federated Learning
Qiaobo Li (University of Illinois), Arindam Banerjee (University of Illinois)
Federated LearningSafty and PrivacyImageText
🎯 What it does: This paper proposes a Sketched Gaussian Mechanism (SGM) that combines sketching with the Gaussian Mechanism, embedding it into the federated learning framework Fed-SGM to achieve a unification of communication compression and differential privacy.
SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
Ehsan Latif (University of Georgia), Xiaoming Zhai (University of Georgia)
Large Language ModelImageMultimodality
🎯 What it does: A multi-agent cognitive framework called SKETCHMIND is proposed for evaluating and improving students' scientific sketches.
Skill-Driven Neurosymbolic State Abstractions
Alper Ahmetoglu (Brown University), George Konidaris (Brown University)
Robotic IntelligenceReinforcement LearningImage
🎯 What it does: An abstract MDP compatible with the given abstract actions was constructed, and a set of theories and algorithms for generating Markov decision processes from distributed abstract states was proposed, achieving complete abstract reasoning in high-dimensional visual tasks.
Skrull: Towards Efficient Long Context Fine-tuning through Dynamic Data Scheduling
Hongtao Xu (University of Chinese Academy of Sciences), Weile Jia (Institute of Computing Technology, Chinese Academy of Sciences)
OptimizationComputational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: A dynamic data scheduler named Skrull is proposed for efficient training of mixed long and short sequences in long context fine-tuning.
SkyLadder: Better and Faster Pretraining via Context Window Scheduling
Tongyao Zhu (National University of Singapore), Min-Yen Kan (National University of Singapore)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper explores the impact of context window size on the pre-training of large language models and proposes the SkyLadder method, which implements dynamic scheduling from short to long windows during the pre-training process through adjustable masking, balancing model performance and training efficiency.
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families
Felipe Maia Polo (University of Michigan), Mikhail Yurochkin (MBZUAI)
Large Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A scale law based on low-dimensional implicit skills, called Sloth, is proposed to predict the performance of different LLM families on multiple benchmarks without the need to train a large number of models from the same family.
Slow Transition to Low-Dimensional Chaos in Heavy-Tailed Recurrent Neural Networks
Yi Xie, Łukasz Kuśmierz (Allen Institute)
Recurrent Neural NetworkTime SeriesSequentialPhysics Related
🎯 What it does: This study investigates the dynamics of recurrent neural networks (RNNs) with Lévy α-stable distribution (heavy-tailed) random weights, finding that finite-size networks exhibit a predictable transition from quiescence to chaos as the activation gain g varies, and theoretically derives the relationship between the critical gain g*, network size N, and tail index α.
Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation is Wasteful
Martin Marek (New York University), Micah Goldblum (Columbia University)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the use of very small batches (even a batch size of 1) in the pre-training and fine-tuning of language models and proposes a β₂ scheduling rule for Adam, demonstrating that small batch training can be comparable to large batch training and is more robust.
Small Resamples, Sharp Guarantees: Convergence Rates for Resampled Studentized Quantile Estimators
Imon Banerjee (Northwestern University), Sayak Chakrabarty (Northwestern University)
Reinforcement Learning
🎯 What it does: A complete second-order theory for studentized m-out-of-n bootstrap quantile estimation is proposed.
Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
Max Staats (Leipzig University), Bernd Rosenow (Leipzig University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Analyzed the singular value spectrum of pre-trained Transformer (BERT, Pythia, Llama-8B) weight matrices, compared with random matrix theory (RMT) and Marchenko-Pastur distribution, calculated the overlap of singular vectors with the activation covariance matrix, removed singular values group by group according to singular value deciles, and assessed the impact on perplexity, reasoning tasks, and downstream evaluations; constructed a minimal RMT model to explain the anomalies of small singular values.
SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
Yi Zhao (Shanghai Jiao Tong University), Xiaoming Fu (Fudan University)
CompressionComputational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: This study investigates the issues of saliency shift and edge information compression in KV cache compression and proposes the SmallKV method.
Smart Surrogate Losses for Contextual Stochastic Linear Optimization with Robust Constraints
Hyungki Im (University of California), Paul Grigas (University of California)
OptimizationReinforcement LearningTabular
🎯 What it does: This study investigates the uncertainty in predicting parameters involving constraints and proposes a robust constraint-based intelligent prediction-optimization framework, SPO-RC, along with a convex loss variant, SPO-RC+, and training methods involving data truncation and importance reweighting.
SmartCache: Context-aware Semantic Cache for Efficient Multi-turn LLM Inference
Chengye YU, Song Jiang (University of Texas at Arlington)
Safty and PrivacyComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes SmartCache, a multi-round LLM inference acceleration framework that combines semantic trees with KV cache sharing, addressing the issues of redundant computation and high memory usage caused by semantically similar queries across different sessions.
SMARTraj$^2$: A Stable Multi-City Adaptive Method for Multi-View Spatio-Temporal Trajectory Representation Learning
Tangwen Qian (Chinese Academy of Sciences), Yongjun Xu (Chinese Academy of Sciences)
Representation LearningRecurrent Neural NetworkGraph Neural NetworkTransformerContrastive LearningTime Series
🎯 What it does: A multi-view spatiotemporal trajectory representation learning method, SMARTraj 2, is proposed for achieving stable representation learning in multi-city environments.
Smooth and Flexible Camera Movement Synthesis via Temporal Masked Generative Modeling
Chenghao Xu (Xidian University), Cheng Deng (Institute for Infocomm Research)
GenerationData SynthesisTransformerAuto EncoderVideoMultimodality
🎯 What it does: This paper proposes an online dance camera motion synthesis method called TemMEGA, which can generate camera trajectories in real-time stage performances using only current and past dance and music segments.
Smooth Quadratic Prediction Markets
Enrique Nueve (University of Colorado Boulder), Bo Waggoner (University of Colorado Boulder)
🎯 What it does: A novel prediction market based on the Dual Cost Function Market (DCFMM) is proposed - the Smooth Quadratic Prediction Market. The theoretical properties, trader incentives, and behavior under budget constraints and buy-only conditions are analyzed within this framework. Furthermore, an adaptive liquidity extension method is proposed.
Smooth Regularization for Efficient Video Recognition
Gil Goldman (Carnegie Mellon University), Mahadev Satyanarayanan (Carnegie Mellon University)
RecognitionComputational EfficiencyTransformerContrastive LearningVideo
🎯 What it does: Provided a smooth regularization method for lightweight video recognition models.
Smooth Sailing: Lipschitz-Driven Uncertainty Quantification for Spatial Associations
David R. Burt (Massachusetts Institute of Technology), Tamara Broderick (Massachusetts Institute of Technology)
Tabular
🎯 What it does: A confidence interval construction method based on Lipschitz smoothness assumptions is proposed for spatial correlation problems, which can provide statistically meaningful confidence intervals under model mismatch and non-random locations.
Smoothed Agnostic Learning of Halfspaces over the Hypercube
Yiwen Kou (University of California Los Angeles), Raghu Meka (University of California Los Angeles)
ClassificationOptimization
🎯 What it does: This paper proposes a new discrete domain smooth unbiased learning framework that perturbs Boolean inputs using random bit flips, and studies the efficient learning problem in the half-space under this model.
Smoothed Differentiation Efficiently Mitigates Shattered Gradients in Explanations
Adrian Hill (Berlin Institute for the Foundations of Learning and Data), Klaus Robert Muller
Explainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkTransformerAuto EncoderImage
🎯 What it does: This paper proposes a method called SmoothDiff for efficiently computing gradient-smooth explanations of deep networks, addressing the computational bottleneck caused by the large number of samples required by traditional SmoothGrad.
SNAP: Low-Latency Test-Time Adaptation with Sparse Updates
Hyeongheon Cha (Korea Advanced Institute of Science and Technology), Sung-Ju Lee (Korea Advanced Institute of Science and Technology)
Domain AdaptationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: The SNAP framework is proposed, utilizing Sparse Test-Time Adaptation (STTA) to significantly reduce inference latency on edge devices while maintaining nearly unchanged model performance.
SnapMoGen: Human Motion Generation from Expressive Texts
chuan guo, Bing Zhou (Snap Inc.)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVideoText
🎯 What it does: A large, continuous action and rich text pairing dataset called SnapMoGen is proposed, and based on this dataset, the MoMask++ text-driven human action generation framework is developed.
SNEAKDOOR: Stealthy Backdoor Attacks against Distribution Matching-based Dataset Condensation
He Yang (Xi'an Jiaotong University), Jizhong Zhao (Xi'an Jiaotong University)
Knowledge DistillationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A backdoor attack framework named SNEAKDOOR is proposed, which implants covert triggers during the data distillation process based on distribution matching, allowing the downstream model to perform normally on regular inputs while being induced to misclassify when the trigger is activated.
Social World Model-Augmented Mechanism Design Policy Learning
Xiaoyuan Zhang (Peking University), Xue Feng (Peking University)
OptimizationReinforcement LearningWorld Model
🎯 What it does: This paper proposes a Social World Model Augmented Mechanism Design Strategy Learning (SWM-AP), which infers hidden agent traits through unsupervised learning and combines it with a world model for sample-efficient mechanism design.
SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation
Zekun Qi, Li Yi
Pose EstimationRobotic IntelligenceTransformerLarge Language ModelVision Language ModelImagePoint Cloud
🎯 What it does: Proposed the concept of semantic orientation, constructed a large-scale OrienText300K dataset, and trained the PointSO model to achieve cross-modal 3D semantic orientation prediction, which was then integrated into the SOFAR system to enable language-based 6-DoF spatial reasoning and robotic manipulation.
Soft Task-Aware Routing of Experts for Equivariant Representation Learning
Jaebyeong Jeon (Yonsei University), Kibok Lee (Yonsei University)
Object DetectionRepresentation LearningMixture of ExpertsContrastive LearningImage
🎯 What it does: Proposes Soft Task-Aware Routing (STAR) to coordinate invariant and equivariant representation learning, reducing redundant feature learning.
Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
Zhen Zhang (University of California), Xin Eric Wang (University of California)
GenerationOptimizationTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: A 'Soft Thinking' framework is proposed that requires no additional training, allowing LLMs to reason in a continuous concept space, replacing traditional discrete word-by-word sampling;
Soft-consensual Federated Learning for Data Heterogeneity via Multiple Paths
Sheng Huang (Sun Yat-sen University), Chuan Chen (Sun Yat-sen University)
Federated LearningImage
🎯 What it does: Proposes the FedMP framework, which achieves soft consensus between global and local models through multi-path training;
Solver-Free Decision-Focused Learning for Linear Optimization Problems
Senne Berden (KU Leuven), Tias Guns (KU Leuven)
OptimizationTabular
🎯 What it does: This paper proposes a decision-focused learning method called LAVA, which constructs a convex loss to directly guide the training of prediction models by utilizing the geometric relationship between the optimal solution of linear optimization problems and its adjacent vertices, thereby significantly reducing the computational cost in predict-then-optimize scenarios.
Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling
Yitian Chen (Cardinal Operations), Yinyu Ye (Stanford University)
OptimizationTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Proposes the Solver-Informed Reinforcement Learning (SIRL) framework, which utilizes verifiable rewards (by executing the generated code and .lp files through an optimization solver) to train large language models for the automated generation of accurate, executable optimization models, supplemented by instance-enhanced self-consistency data synthesis methods.
SolverLLM: Leveraging Test-Time Scaling for Optimization Problem via LLM-Guided Search
Dong Li (Baylor University), Haifeng Chen (NEC Labs America)
OptimizationLarge Language ModelPrompt EngineeringTabular
🎯 What it does: This paper presents SolverLLM, a training-free optimization problem-solving framework based on LLM-guided Monte Carlo Tree Search.
Solving and Learning Partial Differential Equations with Variational Q-Exponential Processes
Guangting Yu (Arizona State University), Shiwei Lan (Arizona State University)
TabularPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A variational $q$-exponential process (Q-EP) framework is proposed for solving and learning partial differential equations, and for implementing uncertainty quantification.
Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics
Lorenzo Magnino (University of Cambridge), Mathieu Lauriere
OptimizationReinforcement LearningSequential
🎯 What it does: A deep reinforcement learning method for equilibrium multi-agent games with non-stationary continuous state-action spaces is proposed.
Solving Discrete (Semi) Unbalanced Optimal Transport with Equivalent Transformation Mechanism and KKT-Multiplier Regularization
Weiming Liu (ByteDance Inc.), Yew-Soon Ong (Nanyang Technological University)
Domain AdaptationOptimizationTabular
🎯 What it does: This paper proposes an Equivalent Transformation Mechanism (ETM) and its three implementations: Exact, Approx, and Refine, for directly solving the marginal distributions of discrete Semi-Unbalanced and Unbalanced Optimal Transport (SemiUOT, UOT), and converts these two types of problems into classical OT problems; it then introduces a regularization based on KKT multipliers (MROT) to further enhance the sparsity and accuracy of the matching.
Solving Inverse Problems with FLAIR
Julius Erbach (ETH Zürich), Konrad Schindler (ETH Zürich)
RestorationSuper ResolutionFlow-based ModelImage
🎯 What it does: A training-independent variational framework called FLAIR is proposed, using flow-based generative models as priors to solve inverse problems.
Solving Neural Min-Max Games: The Role of Architecture, Initialization & Dynamics
Deep Patel (University of Wisconsin-Madison), Emmanouil-Vasileios Vlatakis-Gkaragkounis (University of Wisconsin-Madison)
Optimization
🎯 What it does: This paper proposes and proves that in the hidden convex-concave zero-sum min-max game of a two-layer neural network, using Alternating Gradient Descent-Ascent (AltGDA) can globally converge to an approximate Nash point under the conditions of over-parameterization and random initialization, and provides clear upper bounds on the number of iterations for convergence in relation to network width, sample size, input dimension, and other parameters.
Solving Partial Differential Equations via Radon Neural Operator
Wenbin Lu (Zhejiang University of Technology), Jianwei Zheng (Zhejiang University of Technology)
BenchmarkPhysics RelatedOrdinary Differential Equation
🎯 What it does: A neural operator based on Radon transform (RNO) is proposed for efficiently solving partial differential equations.
Solving the Asymmetric Traveling Salesman Problem via Trace-Guided Cost Augmentation
Zhen Zhang (Australian Institute for Machine Learning), Wee Sun Lee (National University of Singapore)
OptimizationGraph
🎯 What it does: A framework for solving the Asymmetric Traveling Salesman Problem (ATSP) based on continuous relaxation is proposed, utilizing differentiable topological constraints to approximate the condition of subgraphs being DAGs, and combining double stochastic matrix relaxation for gradient optimization;
SOMBRL: Scalable and Optimistic Model-Based RL
Bhavya Sukhija (ETH Zurich), Andreas Krause (ETH Zurich)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: We propose SOMBRL - a scalable, optimistic model-based reinforcement learning (MBRL) framework based on uncertainty-aware models, which learns directly in online interactions and actively explores;
Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness
Mojtaba Kolahdouzi (Queen's University), Ali Etemad (Queen's University)
ClassificationOptimizationConvolutional Neural NetworkTransformerImageStochastic Differential Equation
🎯 What it does: This study investigates the impact of optimizers (SGD, RMSProp, Adam, etc.) on the fairness of deep neural network populations, providing theoretical and empirical evidence that adaptive optimizers are more likely to converge to fair optimal solutions.
SONAR: Long-Range Graph Propagation Through Information Waves
Alessandro Trenta (University of Pisa), Davide Bacciu (University of Pisa)
Graph Neural NetworkGraph
🎯 What it does: A differential equation graph neural network called SONAR based on the graph wave equation is proposed to achieve long-distance information propagation.
SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement
Chenyu Yang (Chinese University of Hong Kong), Haizhou Li (Chinese University of Hong Kong)
GenerationData SynthesisTransformerDiffusion modelAuto EncoderTextAudio
🎯 What it does: We propose a full song generation framework called SongBloom, which alternates between generating semantic sketches and audio details, supporting the generation of complete and structurally coherent songs from lyrics and short audio clips.
SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization
Xiaofeng Tan (Southeast University), Pan Zhou (Singapore Management University)
GenerationOptimizationReinforcement LearningVision-Language-Action ModelVideoText
🎯 What it does: This paper proposes a Semi-Online Preference Optimization (SoPo) method, which utilizes a combination of high-quality offline actions and diverse non-optimal online actions to fine-tune the text-to-action model, enhancing the matching quality between text and actions.
SORTeD Rashomon Sets of Sparse Decision Trees: Anytime Enumeration
Elif Arslan (Delft University of Technology), Emir Demirović (Delft University of Technology)
OptimizationTabular
🎯 What it does: A framework named SORTD is proposed, which can enumerate the Rashomon set of sparse decision trees in ascending order of target values (i.e., best-first), thus obtaining an effective set at any moment.
SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
Xiyao Wang (University of Maryland), Lijuan Wang (Microsoft)
OptimizationReinforcement LearningVision Language ModelMultimodality
🎯 What it does: Significant improvements were achieved in visual language models through self-improvement via reinforcement learning, using a small number of training samples (11k/7.5k).
Sound Logical Explanations for Mean Aggregation Graph Neural Networks
Matthew Morris (University of Oxford), Ian Horrocks (University of Oxford)
Explainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: The study uses Mean Aggregation with non-negative weights in Graph Neural Networks (MAGNN), proving the obtainable interpretable logical rule categories and providing a method for constructing interpretable rules.
Space Group Equivariant Crystal Diffusion
Rees Chang (University of Illinois at Urbana-Champaign), Ryan P Adams
GenerationData SynthesisGraph Neural NetworkTransformerDiffusion modelTabularPhysics Related
🎯 What it does: A space group invariant crystal generation model SGEquiDiff is proposed, which can naturally handle space group constraints and output space group invariant likelihoods.
SPACE: Noise Contrastive Estimation Stabilizes Self-Play Fine-Tuning for Large Language Models
Yibo Wang (Nanjing University), Lijun Zhang (Nanjing University)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: A self-play fine-tuning method based on Noise Contrastive Estimation (NCE) called SPACE is proposed, aimed at addressing the convergence instability issues of existing gap-based self-play methods.
SPACE: SPike-Aware Consistency Enhancement for Test-Time Adaptation in Spiking Neural Networks
Xinyu Luo (City University of Hong Kong), Haoliang Li (City University of Hong Kong)
Domain AdaptationSpiking Neural NetworkContrastive LearningImage
🎯 What it does: The SPACE method is proposed, achieving adaptation of SNN during testing on single-sample, unsourced data.
SpaceServe: Spatial Multiplexing of Complementary Encoders and Decoders for Multimodal LLMs
zhicheng li, Huimin Cui (Chinese Academy of Sciences)
OptimizationComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageVideoMultimodality
🎯 What it does: The SpaceServe system is proposed, which separates visual/audio encoders from a shared text decoder in multimodal large language model services. It utilizes fine-grained GPU SM partitioning to achieve parallel spatial multiplexing of encoders and decoders on the same GPU, significantly improving inference throughput and latency performance.
Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling
Zhihao Li (Nanyang Technological University), Bihan Wen (Nanyang Technological University)
GenerationData SynthesisOptimizationConvolutional Neural NetworkAuto EncoderPoint CloudMesh
🎯 What it does: Sparc3D proposes a unified framework that combines Sparse Deformable Marching Cubes (Sparcubes) for fast reconstruction of high-resolution watertight meshes, along with a Sparse Convolutional VAE (Sparconv-VAE) for high-precision 3D shape encoding and decoding.
Spark Transformer: Reactivating Sparsity in Transformer FFN and Attention
Chong You (Google), Sanjiv Kumar (Google)
TransformerText
🎯 What it does: Spark Transformer reintroduces an activation sparsification Transformer architecture, achieving high sparsity rates in both the feedforward network (FFN) and attention mechanism, significantly reducing FLOPs while maintaining model quality and parameter count.
SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score
Mohammad Jalali (Chinese University of Hong Kong), Farzan Farnia (Chinese University of Hong Kong)
GenerationData SynthesisComputational EfficiencyPrompt EngineeringDiffusion modelImageMultimodality
🎯 What it does: A scalable, conditionally Renyi kernel entropy-based prompt-aware diversity guidance method called SPARKE is proposed for generating more diverse and fair images with diffusion models.
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Mateusz Pach (Technical University of Munich), Zeynep Akata (Technical University of Munich)
Representation LearningTransformerVision Language ModelAuto EncoderImageMultimodality
🎯 What it does: This paper addresses the reconstruction of layer activations using Sparse Autoencoders (SAE) for visual language models (such as CLIP) and proposes the MonoSemanticity (MS) score to quantitatively assess the univocality of neurons; it utilizes the obtained univocal neurons for concept insertion and suppression in multimodal large language models (such as LLaVA).
Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems
Jingwen Cheng (Tsinghua University), Yong Li (Tsinghua University)
Graph Neural NetworkDiffusion modelAuto EncoderTime SeriesOrdinary Differential Equation
🎯 What it does: Long-term prediction of complex systems is achieved through sparse probe encoding and diffusion decoding, with dynamic re-encoding during testing to adapt to newly emerging spatiotemporal patterns.
Sparse Gaussian Processes: Structured Approximations and Power-EP Revisited
Thang D Bui, Michalis Titsias
Tabular
🎯 What it does: A sparse Gaussian process approximation with a block diagonal structure is proposed, and this structure is applied to the Power Expectation Propagation framework to improve posterior approximation.
Sparse Image Synthesis via Joint Latent and RoI Flow
Ziteng Gao (Show Lab, National University of Singapore), Mike Zheng Shou (Show Lab, National University of Singapore)
GenerationData SynthesisTransformerFlow-based ModelAuto EncoderImage
🎯 What it does: A framework is proposed for image generation using sparse non-grid latent representations and their spatial location information (RoI);
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Yuhao Yang (Baidu Inc), LIU LIN
Recommendation SystemTransformerAuto EncoderContrastive LearningText
🎯 What it does: The COBRA framework is proposed, which utilizes sparse semantic IDs and dense vectors in a cascading manner to generate a joint sparse-dense dual representation for generative recommendation.
Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
Yong Liu (National University of Singapore), Yang You (National University of Singapore)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes Sparse-MeZO, a memory-efficient zero-order optimization method for fine-tuning large-scale language models (such as LLaMA-7b/30b), significantly reducing memory requirements;
Sparse Optimistic Information Directed Sampling
Ludovic Schwartz (Universitat Pompeu Fabra), Gergely Neu (Universitat Pompeu Fabra)
OptimizationReinforcement Learning
🎯 What it does: A new SOIDS algorithm is proposed for the sparse linear Bandit problem, which can achieve optimal worst-case returns in both data-rich and data-scarce scenarios;
Sparse Polyak: an adaptive step size rule for high-dimensional M-estimation
Tianqi Qiao (Texas A&M University), Marie Maros (Texas A&M University)
OptimizationTabular
🎯 What it does: Proposes a Sparse Polyak adaptive step size rule for the Iterative Hard Thresholding (IHT) algorithm in high-dimensional M-estimation problems.
Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation
Shuo Yang (University of California), Ion Stoica (University of California)
GenerationData SynthesisComputational EfficiencyTransformerDiffusion modelVideoText
🎯 What it does: Sparse acceleration of attention computation for video generation Diffusion Transformers (DiT) is proposed, introducing the training-independent SVG2 framework, which utilizes semantic-aware permutation to accurately identify key tokens and achieves efficient sparse attention through dynamic Topp selection and custom kernels.
SparseDiT: Token Sparsification for Efficient Diffusion Transformer
Shuning Chang (Alibaba Group), Yi Yang (Zhejiang University)
GenerationData SynthesisComputational EfficiencyTransformerDiffusion modelImageVideoText
🎯 What it does: This paper proposes SparseDiT, which significantly reduces computational costs and improves inference speed by applying sparsification to the tokens in the Diffusion Transformer.
SparseMVC: Probing Cross-view Sparsity Variations for Multi-view Clustering
Ruimeng Liu (China University of Geosciences), Xinwang Liu (National University of Defense Technology)
OptimizationAuto EncoderContrastive LearningImageMultimodality
🎯 What it does: The SparseMVC framework is proposed to model and optimize the differences in sparsity among various views in multi-view clustering.