ICML 2025 Papers — Page 28
International Conference on Machine Learning · 3257 papers
SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Malte Mosbach (University of Bonn), Sven Behnke (University of Bonn)
Robotic IntelligenceTransformerReinforcement LearningImage
🎯 What it does: An object-based model learning algorithm called SOLD is proposed, which can learn object-level dynamics directly from pixels and perform reinforcement learning.
Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo
Filip Ekström Kelvinius, Fredrik Lindsten (Linkoping University)
Data SynthesisSuper ResolutionProtein Structure PredictionDiffusion modelScore-based ModelImageSequentialOrdinary Differential Equation
🎯 What it does: A sequential Monte Carlo algorithm based on separated diffusion, DDSMC, has been developed to achieve asymptotically accurate posterior sampling in linear Gaussian Bayesian inverse problems using a pre-trained diffusion model as a prior.
Solving Probabilistic Verification Problems of Neural Networks using Branch and Bound
David Boetius (University of Konstanz), Tobias Sutter (University of Konstanz)
Computational EfficiencyTabularBenchmark
🎯 What it does: A branch-and-bound based algorithm PV is proposed to solve the probabilistic verification problem of neural networks (such as fairness, robustness, and safety).
Solving Satisfiability Modulo Counting Exactly with Probabilistic Circuits
Jinzhao Li (Purdue University), Yexiang Xue (Purdue University)
Tabular
🎯 What it does: A precise SMC solver KOCO-SMC based on probabilistic circuits is proposed, which can detect probability constraint conflicts and learn clauses when some variable assignments are made.
Solving Zero-Sum Convex Markov Games
Fivos Kalogiannis (University of California), Georgios Piliouras (Google DeepMind)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper studies two-player zero-sum convex Markov games, proposing independent strategy gradient algorithms (Nest PG and Alt PGDA), and provides theoretical guarantees for global convergence to Nash equilibrium.
SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation
Zihan Liu (Beihang University), Jiaqi Wang (Shanghai AI Laboratory)
GenerationData SynthesisTransformerSupervised Fine-TuningTextAudio
🎯 What it does: Introducing SongGen, a single-stage autoregressive Transformer model designed to synthesize complete songs from text, lyrics, and optional reference audio, supporting both mixed and dual-track modes.
Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model
Kaiwen Tang (National University of Singapore), Weng-Fai Wong (National University of Singapore)
Computational EfficiencyKnowledge DistillationSpiking Neural NetworkTransformerLarge Language ModelText
🎯 What it does: A Transformer-based spiking language model called Sorbet, fully compatible with brain-inspired hardware, has been designed and implemented. It achieves binary weights through knowledge distillation and quantization, reducing energy consumption.
Sort Before You Prune: Improved Worst-Case Guarantees of the DiskANN Family of Graphs
Siddharth Gollapudi (University of California Berkeley), Harsh Wardhan (Microsoft Research India)
RetrievalOptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: This paper conducts a theoretical analysis of the construction and search algorithms of DiskANN graphs, introducing the concept of 'sorted α-reachable graphs' and providing a stronger upper bound on approximation error based on this concept, offering the first worst-case guarantee for BeamSearch (candidate search with k>1).
Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems
Taejin Park (NVIDIA), Boris Ginsburg (NVIDIA)
RecognitionTransformerAudio
🎯 What it does: A sorting-based speaker separation model called Sortformer based on Transformer is proposed, and it is coupled with multi-speaker ASR;
Sounding that Object: Interactive Object-Aware Image to Audio Generation
Tingle Li (University of California), Yuxuan Wang (ByteDance Inc)
GenerationData SynthesisDiffusion modelImageMultimodalityAudio
🎯 What it does: This paper proposes an interactive object-based image-to-audio generation model that can automatically generate corresponding sounds based on the user-selected object regions in the image.
Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
Pierre Ablin (Apple), David Grangier
Mixture of ExpertsText
🎯 What it does: Construct a Soup-of-Experts system that generates specialized models under arbitrary domain weights by linearly combining a set of expert parameters obtained from pre-training, without requiring additional training during inference.
SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model
Zhao Yang (Renmin University of China), Bing Su (Renmin University of China)
Representation LearningTransformerMixture of ExpertsBiomedical Data
🎯 What it does: Using supervised pre-training based on genomic features prediction, the SPACE model is proposed for cross-species and cross-feature representation learning of DNA sequences.
SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
Jintao Zhang (Tsinghua University), Jianfei Chen (Tsinghua University)
GenerationComputational EfficiencyTransformerLarge Language ModelImageVideoText
🎯 What it does: A general, training-independent sparse attention mechanism called SpargeAttn is proposed, which can accelerate the inference of language, image, and video generation models without sacrificing model quality.
Sparse Autoencoders for Hypothesis Generation
Rajiv Movva (University of California Berkeley), Emma Pierson (University of California Berkeley)
Explainability and InterpretabilityComputational EfficiencyLarge Language ModelAuto EncoderText
🎯 What it does: Utilizing sparse autoencoders to learn interpretable hidden units, then selecting units related to the target variable through Lasso, and generating natural language hypotheses for these units using LLM, completing the generation of interpretable hypotheses regarding the relationship between text and the target variable.
Sparse Autoencoders, Again?
Yin Lu (Fudan University), David Wipf (Amazon Web Services)
GenerationData SynthesisRepresentation LearningDiffusion modelAuto EncoderImageText
🎯 What it does: A sparse autoencoder model named VAEase is proposed, which integrates the hyperparameter-free nature of VAE with the adaptive sparsity characteristics of SAE, using σ_z as a gate to achieve variable sparsity.
Sparse Causal Discovery with Generative Intervention for Unsupervised Graph Domain Adaptation
Junyu Luo (Peking University), Ming Zhang (Peking University)
Domain AdaptationGraph Neural NetworkGenerative Adversarial NetworkGraph
🎯 What it does: The SLOGAN framework is proposed to achieve unsupervised graph domain adaptation, combining sparse causal discovery, generative intervention, and pseudo-label calibration to stabilize the transfer of graph representations.
Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks
Jialin Zhao (Tsinghua University), Carlo Vittorio Cannistraci (Tsinghua University)
Graph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Proposes Sparse Spectral Training (SST), an efficient pre-training method for neural networks in Euclidean and hyperbolic spaces that updates all singular values, performs polynomial sampling on singular vectors, and periodically re-applies SVD.
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry
Mohammed Adnan (University of Calgary), Yani Ioannou (University of Toronto)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes to permute the lottery ticket sparse mask through weight symmetry to achieve sparse training starting from random initialization.
Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
Haocheng Xi (University of California), Song Han
GenerationComputational EfficiencyTransformerDiffusion modelVideo
🎯 What it does: A training-independent sparse video generation framework called Sparse VideoGen (SVG) is proposed, which accelerates the inference of video Diffusion Transformers by identifying and utilizing sparse patterns in spatial and temporal heads of 3D full attention.
Sparse-pivot: Dynamic correlation clustering for node insertions
Mina Dalirrooyfard (Morgan Stanley), Slobodan Mitrović (University of California)
Graph Neural NetworkGraph
🎯 What it does: A dynamic similarity clustering algorithm called Sparse-Pivot is proposed for node incremental insertion, improving approximation quality while maintaining low update complexity.
SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity
Samir Khaki (University of Toronto), Zhijian Liu (Massachusetts Institute of Technology)
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Developed SparseLoRA, a method for fine-tuning LLMs that dynamically selects sparse channels using contextual sparsity, significantly reducing computational costs;
SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference
Yuan Zhang (Peking University), Shanghang Zhang (Peking University)
CompressionComputational EfficiencyTransformerVision Language ModelImageVideo
🎯 What it does: A training-free, text-guided visual token sparsification method called SparseVLM is proposed, which evaluates the importance of visual tokens using the self-attention matrix in the LLM decoder, dynamically prunes them, and recovers and reconstructs them through clustering, significantly reducing the number of visual tokens.
Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
Yuqi Luo (Tsinghua University), Maosong Sun (Tsinghua University)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a new sparsity metric CETT-PPL-1% and systematically evaluates the sparsity of activation layers in large language models (LLMs) using it. It then studies the effects of factors such as activation functions, the amount of pre-training data, width-depth ratio, and model size on sparsity, and based on these findings, trains a 2.4B model with 93.52% sparsity and a 4.1 times increase in inference speed.
Spatial Reasoning with Denoising Models
Christopher Wewer (Max Planck Institute for Informatics), Jan Eric Lenssen (Max Planck Institute for Informatics)
GenerationData SynthesisDiffusion modelFlow-based ModelImageBenchmark
🎯 What it does: This paper proposes the Spatial Reasoning Models (SRMs) framework, which utilizes denoising generative models to infer continuous spatial variables and conducts experiments on new benchmarks such as Sudoku, Even Pixels, and Counting Polygons/Stars.
SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models
Han-Byul Kim (Apple), Minsik Cho (Apple)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a Sync-Point Drop (SPD) technique that selectively drops synchronization points after self-attention outputs in distributed inference, thereby reducing communication overhead and improving inference speed.
Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions
Yik Siu Chan (Brown University), Marzyeh Ghassemi (Massachusetts Institute of Technology)
Adversarial AttackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: Proposes the HARMSCORE evaluation metric and designs SPEAK EASY to achieve a simple and user-friendly jailbreak attack through multi-step decomposition and multi-language interaction.
SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
Shibo Jie (Peking University), Jing Han (Beijing University of Posts and Telecommunications)
GenerationRetrievalCompressionTransformerLarge Language ModelTextBenchmark
🎯 What it does: The SPECACHE method is proposed, which stores the complete KV cache in CPU memory, while the GPU only retains a low-precision copy. It significantly reduces KV cache usage and communication latency by speculatively token-parallel prefetching of top-k KV pairs.
Spectral-Aware Reservoir Computing for Fast and Accurate Time Series Classification
Shikang Liu (University of Science and Technology of China), Huanhuan Chen (University of Science and Technology of China)
ClassificationRecurrent Neural NetworkTime Series
🎯 What it does: This paper proposes Spectral-Aware Reservoir Computing (SARC), which combines spectral information with Reservoir Computing to achieve multi-scale dynamic modeling of time series data and improve classification accuracy.
Speculate, then Collaborate: Fusing Knowledge of Language Models during Decoding
Ziyao Wang (University of Maryland), Mikhail Yurochkin (IBM Research)
GenerationOptimizationExplainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes Collaborative Speculative Decoding (CoSD), a method that integrates the knowledge of two LLMs during the inference phase through draft model generation, parallel validation by an assistant model, and replacement strategies based on rules or decision trees.
Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation
Jingyu Liu (University of Chicago), Ce Zhang (University of Chicago)
GenerationComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A training-free Speculative Prefill framework is proposed, which estimates and discards unimportant prompt tokens through a lightweight model, significantly reducing TTFT and improving maximum QPS during the prefill stage of LLM inference.
Speeding up Policy Simulation in Supply Chain RL
Vivek Farias (Massachusetts Institute of Technology), Andrew Zheng (University of British Columbia)
OptimizationReinforcement LearningTabular
🎯 What it does: Proposed and implemented the Picard iteration algorithm to accelerate single-trajectory policy simulation in supply chain reinforcement learning.
SPEX: Scaling Feature Interaction Explanations for LLMs
Justin Singh Kang (University of California Berkeley), Kannan Ramchandran (University of California Berkeley)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelTextMultimodality
🎯 What it does: A scalable feature interaction interpretation algorithm named SPEX is designed to efficiently identify sparse interactions in long texts or multimodal inputs.
Spherical-Nested Diffusion Model for Panoramic Image Outpainting
Xiancheng Sun (Beihang University), Jiali Wang (Cainiao Technology)
RestorationGenerationDiffusion modelImage
🎯 What it does: A diffusion model designed using spherical characteristics (SpND) is proposed to achieve outpainting of panoramic images.
SPHINX: Structural Prediction using Hypergraph Inference Network
Iulia Duta (University of Cambridge), Pietro Lio
Graph Neural NetworkPoint CloudGraph
🎯 What it does: This paper proposes an unsupervised hypergraph structure inference model called SPHINX, which can automatically learn implicit hypergraphs from point-level signals and be used for downstream tasks.
SpikeVideoFormer: An Efficient Spike-Driven Video Transformer with Hamming Attention and $\mathcal{O}(T)$ Complexity
Shihao Zou (Shenzhen Institutes of Advanced Technology), Chao Dong (Shenzhen Institutes of Advanced Technology)
ClassificationObject TrackingSegmentationPose EstimationComputational EfficiencySpiking Neural NetworkTransformerVideo
🎯 What it does: This paper proposes SpikeVideoFormer, an efficient temporal-driven Transformer capable of video classification, pose tracking, and semantic segmentation.
SpikF: Spiking Fourier Network for Efficient Long-term Prediction
Wenjie Wu (Tsinghua University), Hong Chen (Tsinghua University)
Spiking Neural NetworkTime SeriesSequential
🎯 What it does: The SpikF architecture is proposed, which implements efficient long sequence time series prediction using spiking neural networks;
Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution
Jiayu Zhang (Suzhou University of Technology), Huaming Chen (University of Sydney)
Anomaly DetectionExplainability and InterpretabilityAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A novel OOD detection method S & I is proposed, which splits the intermediate layers of the network and integrates adversarial gradients along the reverse gradient path to obtain more reliable explanation patterns.
Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models
Yangxu Liao (Guangdong Laboratory of Artificial Intelligence and Digital Economy), Mang Ye (National Engineering Research Center for Multimedia Software)
Federated LearningLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The FedICU framework is proposed for instruction-following fine-tuning of large language models in heterogeneous federated learning environments.
SPMC: Self-Purifying Federated Backdoor Defense via Margin Contribution
Wenwen He (Wuhan University), Mang Ye (Wuhan University)
Federated LearningImage
🎯 What it does: A self-purification federated learning backdoor defense method called SPMC is proposed, which measures the differences between clients using marginal contribution and dynamically weights aggregation, while achieving self-purification on the client side through gradient alignment.
SPRI: Aligning Large Language Models with Context-Situated Principles
Hongli Zhan (University of Texas at Austin), Mikhail Yurochkin (IBM Research)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes the SPRI framework, which utilizes large language models to generate context-specific principles in real-time for each input, and uses these principles to guide the generation and evaluation of responses, significantly reducing human intervention while maintaining high-quality alignment.
Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization
Simone Bombari (Institute of Science and Technology), Marco Mondelli
ImageTabular
🎯 What it does: In the high-dimensional linear regression framework, a theoretical quantitative description of the spurious correlations arising from irrelevant features is provided, and the roles of regularization, feature simplicity, and over-parameterization are explored.
Square$\chi$PO: Differentially Private and Robust $\chi^2$-Preference Optimization in Offline Direct Alignment
Xingyu Zhou (Wayne State University), Francesco Orabona (King Abdullah University of Science and Technology)
OptimizationSafty and PrivacyText
🎯 What it does: This paper theoretically studies the offline alignment of language models with human preference feedback and proposes the Square χ PO algorithm, aimed at addressing the issues of preference label corruption and privacy protection.
SSHR: More Secure Generative Steganography with High-Quality Revealed Secret Images
Jiannian Wang (Harbin Institute of Technology), Guangming Lu (Harbin Institute of Technology)
GenerationData SynthesisSafty and PrivacyDiffusion modelImage
🎯 What it does: A generative steganography method based on diffusion models (SSHR) is proposed, which guides the steganography and revealing process through a reference image and an adaptive symmetric key, achieving an accurate revealing mechanism.
Stability and Generalization Analysis of Decentralized SGD: Sharper Bounds Beyond Lipschitzness and Smoothness
Shuang Zeng (University of Hong Kong), Yunwen Lei (University of Hong Kong)
Optimization
🎯 What it does: The theoretical analysis of the stability and generalization error of distributed D-SGD for convex smooth and non-smooth problems.
Stability and Generalization Capability of Subgraph Reasoning Models for Inductive Knowledge Graph Completion
Minsung Hwang (KAIST), Joyce Jiyoung Whang (KAIST)
Graph Neural NetworkGraph
🎯 What it does: This study investigates the stability and generalization ability of subgraph inference models in inductive knowledge graph completion, providing theoretical analysis and PAC-Bayesian bounds.
Stabilizing Sample Similarity in Representation via Mitigating Random Consistency
Jieting Wang (Shanxi University), Xinyan Liang (Shanxi University)
OptimizationRepresentation LearningConvolutional Neural NetworkTransformerImageTabular
🎯 What it does: This paper studies a novel unbiased similarity measurement metric—Pure Square Euclidean Distance (PSED)—for evaluating and optimizing class-level discriminability in deep networks.
Stable Fair Graph Representation Learning with Lipschitz Constraint
Qiang Chen (Central South University), Chang Xu (University of Sydney)
OptimizationRepresentation LearningGraph Neural NetworkGraphTabularFinance Related
🎯 What it does: A stable fair graph representation learning framework (SFG) is proposed, aiming to maintain training stability while preserving accuracy and fairness.
Stable Offline Value Function Learning with Bisimulation-based Representations
Brahma S Pavse, Josiah P. Hanna (University of Wisconsin - Madison)
OptimizationRepresentation LearningReinforcement LearningTabular
🎯 What it does: The study stabilizes offline value function learning through dual simplification representation.
Stacey: Promoting Stochastic Steepest Descent via Accelerated $\ell_p$-Smooth Nonconvex Optimization
Xinyu Luo (Purdue University), Brian Bullins (Purdue University)
OptimizationImage
🎯 What it does: An accelerated stochastic steepest descent optimizer STACEY based on ℓp non-Euclidean geometry is proposed.
Staged and Physics-Grounded Learning Framework with Hyperintensity Prior for Pre-Contrast MRI Synthesis
Dayang Wang (Subtle Medical), Greg Zaharchuk (Stanford University)
Image TranslationRestorationData SynthesisConvolutional Neural NetworkAuto EncoderImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a staged, physics-based framework that introduces high signal priors (SPHERE) to directly synthesize pre-contrast MRI images from post-contrast MRI images enhanced by contrast.
STAIR: Improving Safety Alignment with Introspective Reasoning
Yichi Zhang (Tsinghua University), Jun Zhu (Tsinghua University)
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: The STAIR framework is proposed, which combines safety alignment with reflective reasoning, introducing structured chain-of-thought (CoT) and safety-aware Monte Carlo Tree Search (MCTS) in LLMs. It self-generates step-level preference data and undergoes iterative training to enhance the model's dual performance in safety and effectiveness.
STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings
Saksham Rastogi (Indian Institute of Science), Danish Pruthi (Indian Institute of Science)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: The STAMP framework is designed to detect whether a dataset has been included in LLM pre-training corpora by performing multiple paraphrases of the original content and embedding watermarks with different keys, followed by a paired t-test using the perplexity differences between public and private versions.
Star Attention: Efficient LLM Inference over Long Sequences
Shantanu Acharya (NVIDIA), Boris Ginsburg (NVIDIA)
RetrievalComputational EfficiencyTransformerLarge Language ModelTextSequentialBenchmark
🎯 What it does: A two-stage block sparse attention algorithm called Star Attention is proposed, which achieves efficient inference on long sequences through an anchor block mechanism, significantly reducing computational costs.
STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantization
Hao Li (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
Robotic IntelligenceTransformerAuto EncoderSequentialBenchmark
🎯 What it does: The STAR framework is proposed, utilizing Rotationally Enhanced Residual Quantization (RaRSQ) and Causal Skill Transformer (CST) to learn and combine multi-task robotic skills.
Statistical and Computational Guarantees of Kernel Max-Sliced Wasserstein Distances
Jie Wang (Chinese University of Hong Kong), Yao Xie (Georgia Institute of Technology)
Anomaly DetectionOptimizationImage
🎯 What it does: This paper studies the statistical and computational properties of the kernel maximization slice Wasserstein distance (KMS Wasserstein), providing the non-asymptotic convergence rate of empirical distributions, the computational NP-hardness, and a semi-definite relaxation (SDR) solving framework, along with the low-rank properties and relaxation errors of the corresponding approximate solutions.
Statistical Collusion by Collectives on Learning Platforms
Etienne Gauthier (Inria), Michael I. Jordan (University of California)
ClassificationOptimizationTabular
🎯 What it does: This paper proposes a theoretical and algorithmic framework for collective action on learning platforms, which can help collectives infer optimal poisoning/removal/cancellation signal strategies under limited sample conditions and provide lower bound guarantees.
Statistical Hypothesis Testing for Auditing Robustness in Language Models
Paulius Rauba (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
Large Language ModelPrompt EngineeringText
🎯 What it does: A distribution-based perturbation analysis framework (DBPA) is proposed, which transforms the detection of output perturbations from large language models into a frequentist statistical hypothesis testing problem.
Statistical Query Hardness of Multiclass Linear Classification with Random Classification Noise
Ilias Diakonikolas (University of Wisconsin Madison), Christos Tzamos (University of Athens)
Classification
🎯 What it does: Study the statistical query (SQ) difficulty of multi-class linear classification under random classification noise (RCN) in a distribution-independent PAC model.
Statistical Test for Feature Selection Pipelines by Selective Inference
Tomohiro Shiraishi (Nagoya University), Ichiro Takeuchi (Nagoya University)
Anomaly DetectionOptimizationTabular
🎯 What it does: This paper proposes a statistical testing method based on Selective Inference to evaluate the significance of a complete feature selection pipeline in linear models, including missing value imputation, outlier detection, and feature selection, which can provide valid p-values.
Stay Hungry, Keep Learning: Sustainable Plasticity for Deep Reinforcement Learning
Huaicheng Zhou (Westlake University), Donglin Wang (Westlake University)
Reinforcement Learning
🎯 What it does: Proposes a neuron regeneration mechanism and a sustainable backup propagation framework, improving PPO to maintain network plasticity.
Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection
Anirudh Sundara Rajan (University of Wisconsin-Madison), Yong Jae Lee (University of Wisconsin-Madison)
ClassificationAnomaly DetectionConvolutional Neural NetworkDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: By constraining the last layer weights of the detector to be non-negative after training and retraining, the model focuses solely on the fake traces generated by the generator, ignoring the features of real images.
STD-FD: Spatio-Temporal Distribution Fitting Deviation for AIGC Forgery Identification
Hengrui Lou (Zhejiang University), Yijun Bei (Zhejiang University)
RecognitionAnomaly DetectionDiffusion modelImage
🎯 What it does: This paper proposes a spatiotemporal distribution fitting deviation (STD-FD) method based on diffusion models for detecting forgery in AIGC-generated images.
Stealing That Free Lunch: Exposing the Limits of Dyna-Style Reinforcement Learning
Brett Barkley (University of Texas at Austin), David Fridovich-Keil (University of Texas at Austin)
Reinforcement LearningTabularBenchmark
🎯 What it does: Evaluate the performance differences of Dyna-style model-based reinforcement learning on Gym and DMC benchmarks, and find that it generally performs worse than the non-Dyna baseline on DMC, revealing the no free lunch phenomenon.
Stealix: Model Stealing via Prompt Evolution
Zhixiong Zhuang (Bosch Center for Artificial Intelligence), Mario Fritz (CISPA Helmholtz Center for Information Security)
ClassificationGenerationAdversarial AttackPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImage
🎯 What it does: This paper proposes Stealix, which utilizes pre-trained diffusion models and visual language models to iteratively evolve prompts through genetic algorithms without the need for manual prompt design, thereby achieving efficient model stealing.
StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models
Ya Jiang (George Mason University), Kai Zeng (George Mason University)
GenerationTransformerLarge Language ModelText
🎯 What it does: StealthInk is proposed—a multi-bit, covert LLM watermarking scheme that does not alter the text distribution and can embed user information, timestamps, and other metadata.
Steer LLM Latents for Hallucination Detection
Seongheon Park (University of Wisconsin), Yixuan Li (University of Wisconsin)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes a lightweight Truthfulness Separator Vector (TSV) that reshapes the latent space by injecting vectors into the intermediate hidden layers of the LLM during inference, thereby achieving efficient hallucination detection.
Steerable Transformers for Volumetric Data
Soumyabrata Kundu (University of Chicago), Risi Kondor (University of Chicago)
ClassificationSegmentationTransformerImagePoint CloudBiomedical Data
🎯 What it does: This paper proposes Steerable Transformers, a network that combines Vision Transformers with Steerable Convolution, capable of achieving equivariance to SE(d) (rotation + translation) transformations on both 2D and 3D volumetric data.
Steering Protein Language Models
Long-Kai Huang (Tencent AI Lab), Jianhua Yao (Tencent AI Lab)
GenerationOptimizationDrug DiscoveryTransformerLarge Language ModelTextBiomedical Data
🎯 What it does: This paper proposes the migration of Activation Steering technology to Protein Language Models (PLM), utilizing fine-tuning of internal activations during inference to achieve control over the properties of protein sequence generation and optimization.
Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design
Marcel Hedman (University of Oxford), Tom Rainforth (University of Oxford)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes a semi-amortized Bayesian experimental design method called Step-DAD, which combines a pre-trained global policy with periodic updates during testing to continuously optimize design decisions throughout the experiment.
Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence
Yinbin Han (New York University), Renyuan Xu (New York University)
GenerationOptimizationReinforcement LearningDiffusion modelImageText
🎯 What it does: This paper proposes a stochastic control framework based on linear dynamics and KL regularization for fine-tuning diffusion models, and presents a corresponding globally linearly convergent policy iteration algorithm PI-FT.
Stochastic Deep Restoration Priors for Imaging Inverse Problems
Yuyang Hu (Washington University in St. Louis), Ulugbek S. Kamilov (Washington University in St. Louis)
RestorationSuper ResolutionCompressionImageMagnetic Resonance Imaging
🎯 What it does: A framework called ShaRP is proposed and implemented, which solves imaging inverse problems by randomly sampling a set of pre-trained deep recovery models (not limited to denoisers) as priors.
Stochastic Encodings for Active Feature Acquisition
Alexander Luke Ian Norcliffe (University of Cambridge), Pietro Lio (University of Cambridge)
ClassificationAuto EncoderTabularBiomedical Data
🎯 What it does: A method for active feature acquisition based on a stochastic encoder, SEFA, is proposed to gradually select features during testing to improve prediction accuracy.
Stochastic Forward–Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets
Haoye Lu (University of Waterloo), Yaoliang Yu (Vector Institute)
RestorationGenerationData SynthesisDiffusion modelImageStochastic Differential Equation
🎯 What it does: A stochastic forward-backward deconvolution method (SFBD) is proposed for training diffusion models using noisy data based on limited clean sample pre-training.
Stochastic Layer-Wise Shuffle for Improving Vision Mamba Training
Zizheng Huang (Nanjing University), Limin Wang (Nanjing University)
ClassificationObject DetectionSegmentationKnowledge DistillationTransformerContrastive LearningImage
🎯 What it does: A hierarchical random shuffling regularization method SLWS is proposed to improve Vision Mamba training, significantly enhancing classification and downstream task performance in conjunction with masked feature distillation.
Stochastic Online Conformal Prediction with Semi-Bandit Feedback
Haosen Ge (University of Pennsylvania), Osbert Bastani (University of Pennsylvania)
TransformerImageText
🎯 What it does: A random online compliance prediction algorithm aimed at semi-bandit feedback is proposed, which can dynamically construct a prediction set that meets the given coverage at each time step.
Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
Sidhanth Holalkere (Cornell University), Alexander Terenin (Cornell University)
OptimizationGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes a novel stochastic Poisson surface reconstruction method based on geometric Gaussian processes, which can simultaneously complete interpolation and implicit surface reconstruction with a single linear solve, thus achieving a reconstruction process that is sensitive to local queries and outputs.
Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraints
Ruichuan Huang (University of British Columbia), Ahmet Alacaoglu (University of British Columbia)
OptimizationStochastic Differential Equation
🎯 What it does: This paper proposes a single-loop stochastic smoothing primal-dual algorithm for solving non-convex optimization problems with linear inequality constraints.
SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Suyuan Zhao (Tsinghua University), Zaiqing Nie (Tsinghua University)
SegmentationDomain AdaptationTransformerContrastive LearningBiomedical Data
🎯 What it does: A multi-scale spatial transcriptomics foundational model (SToFM) has been constructed, capable of simultaneously integrating macroscopic tissue structure, microscopic cell interactions, and gene expression information.
STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving
Kefan Dong (Stanford University), Tengyu Ma (Stanford University)
Large Language ModelReinforcement LearningText
🎯 What it does: The Self-play Theorem Prover (STP) model is proposed, which combines the roles of reasoner and conjecturer in self-play, continuously generating new and more challenging conjectures and proofs by utilizing each other's outputs, thereby continuously improving proof capabilities under limited data conditions.
Strategic A/B testing via Maximum Probability-driven Two-armed Bandit
Yu Zhang (Zhongtai Securities Institute for Financial Studies, Shandong University), Xiaodong Yan (Xi'an Jiaotong University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: A method for A/B testing based on maximum probability-driven weighted mean-volatility statistics for the two-armed bandit is proposed to improve the testing power for small average treatment effects.
Strategic Planning: A Top-Down Approach to Option Generation
Max Ruiz Luyten (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
Knowledge DistillationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes a top-down reinforcement learning framework (Strategist) based on human strategic heuristics, which generates multi-layer strategy trees using LLM and transforms natural language strategies into quantifiable reward shaping, thereby accelerating the convergence of RL in sparse reward and exploration-constrained real-world environments.
Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
Aaditya K Singh, Andrew M Saxe
TransformerSequential
🎯 What it does: This study investigates the mechanisms of in-context learning (ICL) emergence and disappearance during the training process of Transformers, and proposes a post-dominance strategy called 'context-constrained in-weights learning (CIWL)' as well as the competitive and cooperative relationship (strategy coopetition) between the two.
Stray Intrusive Outliers-Based Feature Selection on Intra-Class Asymmetric Instance Distribution or Multiple High-Density Clusters
Lixin Yuan (Hohai University), Jun Liu (Lancaster University)
ClassificationAnomaly DetectionTabular
🎯 What it does: A supervised feature selection method named SIOFS is proposed, specifically targeting the classification problem of high-dimensional data with intra-class asymmetric distribution or multiple high-density clusters (ADMHC).
Stream-level Flow Matching with Gaussian Processes
Ganchao Wei (Duke University), Li Ma (Duke University)
GenerationData SynthesisFlow-based ModelImageTime Series
🎯 What it does: A flow matching method based on Gaussian processes is proposed, extending traditional conditional flow matching to the flow level, using Gaussian processes to model the random paths connecting the two ends, thus achieving simulation-free flow matching during training.
Streamline Without Sacrifice - Squeeze out Computation Redundancy in LMM
Penghao Wu (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
CompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This study investigates the computational redundancy of visual tokens in decoder-only multimodal models and proposes the ProxyV method, which replaces the self-attention and FFN operations of complete visual tokens with proxy visual tokens, achieving significant efficiency improvements without losing information.
Strengthen Out-of-Distribution Detection Capability with Progressive Self-Knowledge Distillation
Yang Yang (Nanjing University of Science and Technology), Haonan Xu (Nanjing University of Science and Technology)
Anomaly DetectionKnowledge DistillationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes the Progressive Self-Knowledge Distillation (PSKD) framework to enhance the OOD detection capability of image classification models.
Strong and Weak Identifiability of Optimization-based Causal Discovery in Non-linear Additive Noise Models
Mingjia Li (East China Normal University), Aimin Zhou
OptimizationGraphTabularBiomedical Data
🎯 What it does: This paper addresses causal discovery under the nonlinear additive noise model (ANM) by categorizing the identifiability problem into strong identifiability and weak identifiability. Based on this, a unified optimization search method called GENE is designed, which evaluates causal order using both residual independence and goodness of fit, solving the issue of traditional methods performing poorly in weak identifiability scenarios.
Stronger Neyman Regret Guarantees for Adaptive Experimental Design
Georgy Noarov (University of Pennsylvania), Aaron Roth (Amazon Web Services)
TabularTime SeriesFinance Related
🎯 What it does: This paper proposes two adaptive experimental design methods (ClipOGD SC and MGATE) that achieve low variance adaptive weight allocation for estimating average treatment effects in a finite sample setting.
Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
Jaejun Lee (KAIST), Joyce Jiyoung Whang (KAIST)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: A fully structure-based hyper-relational knowledge graph representation learning method called MAYPL is proposed for link prediction and supports inductive reasoning.
Structure-Guided Large Language Models for Text-to-SQL Generation
Qinggang Zhang (Hong Kong Polytechnic University), Xiao Huang (Hong Kong Polytechnic University)
GenerationGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: A structure-guided text-to-SQL generation framework SGU-SQL is proposed, which aligns the query with the database's graph structure and splits tasks using a syntax tree, employing large language models to generate SQL.
Structure-informed Risk Minimization for Robust Ensemble Learning
Fengchun Qiao (University of Delaware), Xi Peng (University of Delaware)
Domain AdaptationOptimizationImageBenchmark
🎯 What it does: A Structure-Aware Risk Minimization (SRM) framework is proposed, which constructs an uncertainty set using the centrality information of distribution graphs to learn ensemble weights that are robust to unknown test distributions.
Structured Preconditioners in Adaptive Optimization: A Unified Analysis
Shuo Xie (Toyota Technological Institute at Chicago), Zhiyuan Li (Google Research)
OptimizationTabular
🎯 What it does: A unified theoretical framework is proposed to analyze adaptive optimization algorithms with structured preconditioners (such as diagonal, full matrix, hierarchical, Kronecker decomposition, etc.), and provides corresponding upper bounds on regret and convergence rates.
Sub-Sequential Physics-Informed Learning with State Space Model
Chenhui Xu (University at Buffalo), Jinjun Xiong (University at Buffalo)
OptimizationComputational EfficiencyData-Centric LearningTime SeriesSequentialPhysics RelatedOrdinary Differential Equation
🎯 What it does: A PINN framework called PINNMamba based on subsequence learning is proposed, which utilizes a state space model (SSM) to achieve continuous-discrete matching and eliminates the simplification bias of the network through subsequence alignment, significantly improving the accuracy of PDE solutions.
Subgoal-Guided Policy Heuristic Search with Learned Subgoals
Jake Tuero (University of Alberta), Levi Lelis
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningAuto EncoderTabular
🎯 What it does: This paper proposes a sub-goal based strategy-guided tree search method, which utilizes online learning of sub-goal generators and multi-level strategies from both successful and failed tree data during the search process, significantly improving the sample efficiency of the search.
Subgroups Matter for Robust Bias Mitigation
Anissa Alloula (University of Oxford), Bartlomiej Papiez
ClassificationImage
🎯 What it does: This study defines the impact of subgroup definitions on existing bias mitigation methods in visual and language classification tasks, systematically evaluating various subgroup partitioning schemes.
Subobject-level Image Tokenization
Delong Chen (Hong Kong University of Science and Technology), Pascale Fung (Hong Kong University of Science and Technology)
Object DetectionSegmentationComputational EfficiencyTransformerVision Language ModelImage
🎯 What it does: This paper proposes a sub-object level image segmentation tokenizer EPOC and compares its impact on visual language models with traditional methods such as patch, panoptic, and SAM.
Subspace Optimization for Large Language Models with Convergence Guarantees
Yutong He (Peking University), Kun Yuan (Peking University)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The GaLore subspace optimization algorithm has been researched and improved, and a GoLore random projection method has been proposed to enhance the memory efficiency and convergence of large language model training.
SUICA: Learning Super-high Dimensional Sparse Implicit Neural Representations for Spatial Transcriptomics
Qingtian Zhu (University of Tokyo), Yinqiang Zheng (University of Tokyo)
Graph Neural NetworkAuto EncoderBiomedical Data
🎯 What it does: This paper proposes a continuous sparse implicit neural representation named SUICA for ultra-high-dimensional sparse modeling and reconstruction of spatial transcriptomics data.
Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings
Angéline Pouget (ETH Zurich), Nicolas Papernot (University of Toronto)
ClassificationDomain AdaptationTabularBenchmark
🎯 What it does: A Suitability Filter is proposed, which statistically compares the model output signals of unlabeled deployment data with labeled test data to determine whether the model's accuracy significantly decreases in real-world environments.
Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups
Weiqiu You (University of Pennsylvania), Eric Wong (University of Pennsylvania)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerImageText
🎯 What it does: A framework called Sum-of-Parts (SOP) is proposed to convert any pre-trained model into a self-explaining neural network (SANN), achieving linear interpretability through end-to-end learning of feature groups.