ICML 2024 Papers — Page 23
International Conference on Machine Learning · 2610 papers
SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting
Lu Han (Nanjing University), De-Chuan Zhan (Nanjing University)
Explainability and InterpretabilityComputational EfficiencyTransformerTime SeriesFinance Related
🎯 What it does: A learnable selective interpretable normalization method for long sequence prediction (SIN) is proposed, which normalizes and denormalizes by learning statistics to alleviate distribution drift caused by non-stationarity.
Single-Model Attribution of Generative Models Through Final-Layer Inversion
Mike Laszkiewicz (Ruhr University Bochum), Asja Fischer (Ruhr University Bochum)
GenerationAnomaly DetectionOptimizationDiffusion modelGenerative Adversarial NetworkImageTabularBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A single model generator attribution method called FLIPAD is proposed, treating it as an anomaly detection task and utilizing the inversion of the final layer of the generator to extract features.
Single-Trajectory Distributionally Robust Reinforcement Learning
Zhipeng Liang (Hong Kong University of Science and Technology), Zhengyuan Zhou (New York University)
Reinforcement LearningTabularFinance Related
🎯 What it does: A completely model-free distributed robust reinforcement learning algorithm is proposed—Single-Trajectory Distributionally Robust Q-Learning (DRQ), which can learn robust optimal policies using only a single experience trajectory.
SiT: Symmetry-invariant Transformers for Generalisation in Reinforcement Learning
Matthias Weissenbacher (RIKEN Center for Advanced Intelligence Project), Yoshinobu Kawahara (Osaka University)
TransformerReinforcement LearningImage
🎯 What it does: A Vision Transformer named SiT is proposed, which utilizes Graph Symmetric Attention (GSA) to simultaneously capture local and global symmetric patterns of images in a self-supervised manner, thereby enhancing generalization and sample efficiency in Reinforcement Learning (RL).
Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection
Feiran Li (Chinese Academy of Sciences), Qingming Huang (Chinese Academy of Sciences)
Object DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper addresses the issue of size imbalance in multi-object salient object detection by proposing size-invariant evaluation metrics and corresponding optimization losses, and improving the model's training strategy based on this.
Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills
Kolby Nottingham (University of California Irvine), Roy Fox (University of California Irvine)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: By automatically constructing and continuously optimizing a transferable skill set, large language models achieve strategy enhancement in interactive environments.
SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
Jialong Guo (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
ClassificationObject DetectionComputational EfficiencyTransformerImageText
🎯 What it does: This paper proposes a Transformer structure that gradually replaces LayerNorm with re-parameterized BatchNorm and introduces simplified linear attention to reduce computational overhead during inference.
SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
Jiwon Song (Seoul National University), jae-joon kim
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Compressing LLM by validating and eliminating redundant Transformer blocks to significantly improve inference speed.
SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals
Rahul Thapa (Stanford University), James Zou (Stanford University)
ClassificationRepresentation LearningConvolutional Neural NetworkContrastive LearningMultimodalityBiomedical DataElectrocardiogram
🎯 What it does: A multi-modal sleep foundation model, SleepFM, was constructed, utilizing contrastive learning to learn unified representations from EEG, ECG, and respiratory signals.
Sliced Wasserstein with Random-Path Projecting Directions
Khai Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
GenerationOptimizationComputational EfficiencyDiffusion modelImage
🎯 What it does: This paper studies a slice Wasserstein distance based on random path projection directions, proposing two variants: the non-optimized Random Path Slice Wasserstein (RPSW) and the Importance Weighted Random Path Slice Wasserstein (IWRPSW);
Sliced-Wasserstein Estimation with Spherical Harmonics as Control Variates
Rémi Leluc (Ecole Polytechnique), Aigerim Zhuman (UCLouvain)
OptimizationComputational EfficiencyImagePoint Cloud
🎯 What it does: This paper proposes the use of spherical harmonics as control variables to improve the Monte Carlo estimation method of sliced Wasserstein distance.
Slicedit: Zero-Shot Video Editing With Text-to-Image Diffusion Models Using Spatio-Temporal Slices
Nathaniel Cohen (Mines Paris PSL Research University), Tomer Michaeli (Technion Israel Institute of Technology)
GenerationData SynthesisDiffusion modelVideoText
🎯 What it does: Using a pre-trained text-to-image diffusion model, zero-shot text editing of videos can modify specified areas according to text prompts while maintaining the original video structure and motion.
Slicing Mutual Information Generalization Bounds for Neural Networks
Kimia Nadjahi (Massachusetts Institute of Technology), Justin Solomon (Massachusetts Institute of Technology)
Information TheoryTabular
🎯 What it does: This paper proposes a neural network information-theoretic generalization error upper bound that combines random subspace training with Sliced Mutual Information, and further introduces rate-distortion theory to extend it to approximately compressible models, providing a computable and tighter upper bound.
Sliding Down the Stairs: How Correlated Latent Variables Accelerate Learning with Neural Networks
Lorenzo Bardone (International School of Advanced Studies), Sebastian Goldt (International School of Advanced Studies)
Image
🎯 What it does: This paper studies how neural networks can effectively extract information from high-order input cumulants, particularly by leveraging the correlations between latent variables to accelerate learning.
SLOG: An Inductive Spectral Graph Neural Network Beyond Polynomial Filter
Haobo Xu (Tsinghua University), Hanghang Tong (University of Illinois Urbana-Champaign)
ClassificationGraph Neural NetworkGraph
🎯 What it does: A new spectral graph neural network SLOG is proposed to address the polynomial filtering and propagation limitations of existing spectral GNNs, supporting inductive node classification on large-scale graphs.
Slot Abstractors: Toward Scalable Abstract Visual Reasoning
Shanka Subhra Mondal (Princeton University), Taylor Whittington Webb
Object DetectionSegmentationComputational EfficiencyRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: Developed Slot Abstractor, which integrates Slot Attention and Abstractor modules for scalable abstract visual reasoning, capable of handling tasks with a large number of objects and multiple relationships.
Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks
Hojoon Lee (KAIST), Clare Lyle (DeepMind)
Reinforcement LearningImage
🎯 What it does: Through experimental analysis of the plasticity decline of networks during continual training and warm-start processes, a dual network architecture based on a brain-inspired complementary learning system, called the Hare-Tortoise model, is proposed and validated to maintain the network's plasticity and generalization ability.
Small-loss Adaptive Regret for Online Convex Optimization
Wenhao Yang (Nanjing University), Lijun Zhang (Nanjing University)
Optimization
🎯 What it does: An algorithm for the small loss adaptive loss theory is proposed, providing achievable adaptive loss upper bounds in smooth environments for exp-concave, strongly convex, and general convex functions; furthermore, a unified algorithm USIA is designed to handle all three types of functions while maintaining optimal minimal loss upper bounds even in non-smooth scenarios.
SMaRt: Improving GANs with Score Matching Regularity
Mengfei Xia (Tsinghua University), Yong-jin Liu
GenerationData SynthesisDiffusion modelScore-based ModelGenerative Adversarial NetworkImage
🎯 What it does: Proposes SMaRt - incorporating score matching regularization from a pre-trained diffusion model into GAN training to address the gradient vanishing problem.
Smooth Min-Max Monotonic Networks
Christian Igel (University of Copenhagen)
OptimizationTabular
🎯 What it does: This paper studies how to introduce a smooth extremum function into the existing min-max neural network to address the issue of ineffective neurons during training, and proposes a differentiable SMM module.
Smooth Tchebycheff Scalarization for Multi-Objective Optimization
Xi Lin (City University of Hong Kong), Qingfu Zhang (City University of Hong Kong)
Optimization
🎯 What it does: A smooth Tchebycheff scalarization method (STCH) is proposed, enabling efficient solutions to differentiable multi-objective optimization problems through gradient descent, and achieving approximation and learning of the full Pareto front.
Smoothing Proximal Gradient Methods for Nonsmooth Sparsity Constrained Optimization: Optimality Conditions and Global Convergence
Ganzhao Yuan (Peng Cheng Laboratory)
Optimization
🎯 What it does: This paper proposes a smooth approximation proximal gradient method (SPGM) to solve non-smooth sparse constrained optimization problems, presenting two variants based on iterative hard thresholding (SPGM-IHT) and block coordinate decomposition (SPGM-BCD);
Smoothness Adaptive Hypothesis Transfer Learning
Haotian Lin (Pennsylvania State University), Matthew Reimherr (Pennsylvania State University)
Domain AdaptationOptimizationTabular
🎯 What it does: This paper proposes Smoothness Adaptive Transfer Learning (SATL), an algorithm that adaptively estimates the smoothness of the target and source models, as well as the shift function, using a Gaussian kernel in a two-stage shift transfer learning framework.
Sobolev Space Regularised Pre Density Models
Mark Kozdoba (Technion Israel Institute of Technology), Shie Mannor (NVIDIA Research)
Anomaly DetectionTabular
🎯 What it does: A non-parametric pre-density estimation method SOSREP based on Sobolev norm regularization is proposed to address the problem of high-dimensional unnormalized density estimation.
Socialized Learning: Making Each Other Better Through Multi-Agent Collaboration
Xinjie Yao (Tianjin University), Qinghua Hu (Tianjin University)
Federated LearningKnowledge DistillationImage
🎯 What it does: A multi-agent social learning (MASC) framework is proposed, which achieves knowledge transfer from expert to general categories through two main modules: collective cooperation and mutual benefit.
Soft Prompt Recovers Compressed LLMs, Transferably
Zhaozhuo Xu (Stevens Institute of Technology), Anshumali Shrivastava
GenerationCompressionTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Using learnable soft prompts to restore the performance of large language models after model compression;
Solving Hierarchical Information-Sharing Dec-POMDPs: An Extensive-Form Game Approach
Johan Peralez (Universite de Lyon), Jilles Steeve Dibangoye
OptimizationReinforcement LearningAgentic AITabularBenchmark
🎯 What it does: This paper proposes a method that utilizes a Hierarchical Information Sharing (HIS) structure to transform Dec-POMDP into a perfect information extensive form game, thereby enabling single-player decision-making in single-stage subgames. Based on this, it improves the Point-Based Value Iteration (PBVI) algorithm to obtain a scalable globally optimal approximate solution method.
Solving Poisson Equations using Neural Walk-on-Spheres
Hong Chul Nam (ETH Zurich), Anima Anandkumar (Caltech)
OptimizationComputational EfficiencyPhysics RelatedStochastic Differential Equation
🎯 What it does: The Neural Walk-on-Spheres (NWoS) method is proposed for the numerical solution of high-dimensional Poisson equations.
SPABA: A Single-Loop and Probabilistic Stochastic Bilevel Algorithm Achieving Optimal Sample Complexity
Tianshu Chu (Beijing University of Technology), Jin Zhang (Southern University of Science and Technology)
OptimizationHyperparameter SearchImage
🎯 What it does: Three single-loop stochastic bilevel optimization algorithms, SPABA, MA-SABA, and SRMBA, are proposed, and it is theoretically proven that they can achieve optimal sample complexity under both expected and finite summation settings.
SPADE: Sparsity-Guided Debugging for Deep Neural Networks
Arshia Soltani Moakhar (Institute of Science and Technology Austria), Dan Alistarh (NeuralMagic)
Explainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: This paper proposes SPADE, a post-processing method based on sample-specific sparsification to enhance the interpretability of pre-trained models.
SparQ Attention: Bandwidth-Efficient LLM Inference
Luka Ribar (Graphcore), Douglas Orr (Graphcore)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The SparQ Attention method is proposed, which reduces memory bandwidth consumption by only accessing the KV cache of high-attention tokens during the inference phase, thereby improving the inference throughput of large language models (LLMs).
Sparse and Structured Hopfield Networks
Saul José Rodrigues dos Santos (Instituto Superior Técnico), Andre Martins (Unbabel)
RetrievalOptimizationContrastive LearningText
🎯 What it does: A unified framework is proposed to connect Hopfield networks with Fenchel-Young loss, defining sparse and structured Hopfield energy and providing a differentiable update rule.
Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once
Zhangheng LI, Zhangyang Wang (University of Texas at Austin)
Object DetectionSegmentationOptimizationComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes the Sparse Cocktail framework, which can jointly train multiple sparse patterns (unstructured, channel-level, N:M) and sub-networks with various sparsity ratios at once, and switch between them during inference based on hardware resources.
Sparse Dimensionality Reduction Revisited
Mikael Møller Høgsgaard (Aarhus University), Chris Schwiegelshohn (Aarhus University)
🎯 What it does: This paper re-examines the sparse Johnson-Lindenstrauss transform and proposes a new method for embedding point sets in high-dimensional space that achieves a sparser embedding while preserving distances between points.
Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference
JIAN XU, John Paisley (Columbia University)
ClassificationOptimizationDiffusion modelScore-based ModelTabularStochastic Differential Equation
🎯 What it does: A variational inference method based on denoising diffusion stochastic differential equations (DDVI) is proposed for efficiently approximating the posterior distribution of sparse inducing points in deep Gaussian processes (DGP).
Sparse is Enough in Fine-tuning Pre-trained Large Language Models
Weixi Song (Wuhan University), Bo Du (Wuhan University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a novel sparse fine-tuning method called SIFT and analyzes the generalization advantages of pre-trained models compared to training from scratch through PAC-Bayes theory, further explaining the quasi-sparsity characteristics of gradients and intrinsic dimensionality compression. Based on this, the authors implement a sparse fine-tuning strategy that updates only a small number of parameters and validate its effectiveness on GLUE and instruction fine-tuning tasks.
Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications
Zixuan Hu (Tsinghua University), Dacheng Tao (Nanyang Technological University)
OptimizationComputational EfficiencyKnowledge DistillationTransformerImage
🎯 What it does: For large Vision Transformer (ViT) models, a Sparse Model Inversion (SMI) method is proposed, which efficiently achieves data-free applications such as model quantization and knowledge transfer by only recovering semantic foreground areas and stopping the inversion of uninformative backgrounds.
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
Vithursan Thangarasa (Cerebras Systems), Sean Lie (Cerebras Systems)
Object DetectionSegmentationComputational EfficiencyLarge Language ModelImageText
🎯 What it does: Sparse Iso‑FLOP Transformations (Sparse‑IFT) are proposed as a direct alternative to sparse layers, improving model training efficiency and accuracy while maintaining the same training FLOPs as the original dense network.
Sparse-to-dense Multimodal Image Registration via Multi-Task Learning
Kaining Zhang (Wuhan University), Jiayi Ma (Wuhan University)
Convolutional Neural NetworkTransformerContrastive LearningImageMultimodality
🎯 What it does: A sparse-dense multimodal image registration method SDME based on a multi-task network is proposed, which first uses sparse matching for initialization and then refines with dense alignment.
Sparser, Better, Deeper, Stronger: Improving Static Sparse Training with Exact Orthogonal Initialization
Aleksandra Nowak, Jacek Tabor (Jagiellonian University)
ClassificationRecognitionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new sparse orthogonal initialization method (Exact Orthogonal Initialization, EOI), which constructs a weight matrix that is precisely orthogonal and of arbitrary sparsity through random Givens rotations, and applies it to static sparse training, successfully training thousand-layer sparse MLPs and CNNs.
Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities
Stephen Zhang (University of Toronto), Vardan Papyan (University of Toronto)
OptimizationTabularBenchmark
🎯 What it does: This study investigates the limitations of current mainstream pruning methods in finding the sparsest models and establishes a benchmark for the minimal sparse model of a four-layer MLP on the Cubist Spiral dataset using a new combinatorial search algorithm.
SparseTSF: Modeling Long-term Time Series Forecasting with *1k* Parameters
Shengsheng Lin (South China University of Technology), Junjie Yang (South China University of Technology)
Time Series
🎯 What it does: A minimal LTSF model called SparseTSF is proposed, achieving long-term time series forecasting with 1k parameters.
Spectral Phase Transition and Optimal PCA in Block-Structured Spiked Models
Pierre Mergny (Ecole Federale Polytechnique de Lausanne), Florent Krzakala (Ecole Federale Polytechnique de Lausanne)
OptimizationPhysics Related
🎯 What it does: This paper studies the non-uniform peak Wigner model under block structured noise, providing the limit of its spectral distribution and proving the phase transition of its main eigenvalue with the overlap vector, further presenting the optimal detection threshold for PCA in this model.
Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions
Nikita Doikov (Ecole Polytechnique Federale de Lausanne), Martin Jaggi (Ecole Polytechnique Federale de Lausanne)
OptimizationTabular
🎯 What it does: A spectral preconditioned gradient method utilizing the first τ eigenvectors of the objective function Hessian is proposed for solving hierarchical non-convex optimization problems.
Speech Self-Supervised Learning Using Diffusion Model Synthetic Data
Heting Gao (University of Illinois), Yang Zhang (MIT IBM Watson AI Lab)
RecognitionData SynthesisRepresentation LearningDiffusion modelAudio
🎯 What it does: This work proposes the DIFFS4L framework, which utilizes diffusion models to generate diverse synthetic speech (including new prosody, new speakers, and meaningless babble) on low-resource unlabeled speech data, thereby expanding the pre-training corpus and enhancing self-supervised speech representation learning.
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
Dongyang Liu (Chinese University of Hong Kong), Peng Gao (Shanghai AI Laboratory)
TransformerLarge Language ModelMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: Developed the SPHINX-X series of multimodal large language models, improving architecture and training processes, and enhancing model performance by expanding data and parameter scales.
Spider: A Unified Framework for Context-dependent Concept Segmentation
Xiaoqi Zhao (Dalian University of Technology), Huchuan Lu (Dalian University of Technology)
SegmentationTransformerImageBiomedical DataComputed Tomography
🎯 What it does: A unified model named Spider is proposed, which efficiently recognizes and segments eight context-dependent concept segmentation tasks using a single set of parameters.
Spike Distance Function as a Learning Objective for Spike Prediction
Kevin Doran (University of Sussex), Tom Baden (University of Tübingen)
Convolutional Neural NetworkTime Series
🎯 What it does: This paper proposes using 'time distance to the nearest spike' as a learning objective to train neural networks to predict neuronal spike timing.
SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
Xingrun Xing (Institute of Automation, Chinese Academy of Sciences), Guoqi Li (Institute of Automation, Chinese Academy of Sciences)
Spiking Neural NetworkTransformerLarge Language ModelText
🎯 What it does: A completely pulse-based universal language model, SpikeLM, is proposed, utilizing elastic bidirectional pulse coding to achieve end-to-end pulse computation for language tasks.
SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN
kang you, Zhezhi He (Shanghai Jiao Tong University)
ClassificationSpiking Neural NetworkTransformerImageText
🎯 What it does: A method for converting ANN to SNN based on Transformer, called SpikeZIP-TF, is proposed, achieving strict equivalence between quantized Transformer ANN and SNN without loss of accuracy after conversion.
Split-and-Denoise: Protect large language model inference with local differential privacy
Peihua Mai (National University of Singapore), Yan Pang (National University of Singapore)
OptimizationSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: This paper proposes the Split-N-Denoise (SnD) framework, which splits large language models into a local token embedding layer and a cloud-based subsequent layer. Users add noise to the embeddings locally before uploading them, and the cloud returns outputs affected by the noise. Users then perform denoising locally using the known noise to ultimately obtain high-quality embeddings for downstream tasks.
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting
Anthony Chen (Peking University), Shanghang Zhang (Peking University)
ClassificationAnomaly DetectionComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: By constructing Split-Ensemble through task decomposition and model decomposition, it improves uncertainty estimation and classification accuracy without the need for additional OOD data during training.
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Abhimanyu Hans (University of Maryland), Tom Goldstein (University of Maryland)
ClassificationGenerationTransformerLarge Language ModelText
🎯 What it does: A zero-shot LLM text detection method called Binoculars is proposed, which uses the cross perplexity ratio of two similar LLMs to distinguish between human and machine-generated text.
SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
Xudong Lu (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A method called SPP is proposed for parameter-efficient fine-tuning of large language models while maintaining a sparse structure.
SqueezeLLM: Dense-and-Sparse Quantization
Sehoon Kim (University of California Berkeley), Kurt Keutzer (University of California Berkeley)
GenerationCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A post-training quantization framework called SqueezeLLM is proposed for large language models, which can maintain nearly the same generation performance as FP16 at extremely low precision while significantly improving inference speed.
SSL4Q: Semi-Supervised Learning of Quantum Data with Application to Quantum State Classification
Yehui Tang (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
ClassificationTransformerImagePhysics Related
🎯 What it does: A semi-supervised learning framework specifically for quantum data, SSL4Q, is proposed for quantum state classification.
Stability and Generalization for Stochastic Recursive Momentum-based Algorithms for (Strongly-)Convex One to $K$-Level Stochastic Optimizations
Xiaokang Pan (Central South University), Zhe Qu (Central South University)
OptimizationTabular
🎯 What it does: This study investigates the stability and generalization error of STORM, COVER, and SVMR in K-level stochastic optimization, providing excess risk bounds under both convex and strongly convex scenarios.
Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms
Ming Yang (State University of New York), Yiming Ying (University of Sydney)
Optimization
🎯 What it does: This paper studies the algorithm stability and generalization error of the Stochastic Composite Gradient Descent (SCGD and SCSC) algorithms within the framework of statistical learning theory, providing theoretical upper bounds under convex and smooth, as well as strongly convex and smooth conditions.
Stability and Multigroup Fairness in Ranking with Uncertain Predictions
Siddartha Devic (University of Southern California), Vatsal Sharan (University of Southern California)
Tabular
🎯 What it does: This paper studies how to map predictions to distributed rankings in the presence of uncertainty and proposes a stable and anonymous ranking function.
Stability Evaluation through Distributional Perturbation Analysis
Jose Blanchet (Stanford University), Jiashuo Liu (Tsinghua University)
OptimizationAdversarial AttackTabular
🎯 What it does: This paper defines an optimal transport (OT) distance with moment constraints in the sample-density joint space, proposing a unified model stability assessment criterion to quantify the minimum disturbance magnitude of model risk under two distribution perturbations: data corruption and subgroup shift, and provides a solvable convex/one-dimensional dual form.
Stability-Informed Initialization of Neural Ordinary Differential Equations
Theodor Westny (Linkoping University), Erik Frisk (Linkoping University)
OptimizationAuto EncoderTime SeriesSequentialOrdinary Differential Equation
🎯 What it does: This paper studies the impact of the stability region of numerical solvers on the training and prediction performance of Neural ODEs, and proposes a stability-informed initialization method (SII) based on the stability region, significantly improving training efficiency and model performance.
Stabilizing Policy Gradients for Stochastic Differential Equations via Consistency with Perturbation Process
Xiangxin Zhou (University of Chinese Academy of Sciences), Yichi Zhou (ByteDance Research)
Drug DiscoveryReinforcement LearningDiffusion modelScore-based ModelTabularStochastic Differential Equation
🎯 What it does: The DiffAC algorithm is proposed, which utilizes consistency constraints (the SDE is consistent with its forward perturbation process) to achieve reinforcement learning training of SDEs, addressing the issues of gradient instability and behavior control in traditional SDE policy gradients in sparse data regions.
Stable Differentiable Causal Discovery
Achille Nazaret (Columbia University), David Blei (Columbia University)
OptimizationGraph Neural NetworkGraph
🎯 What it does: A Stable Differentiable Causal Discovery (SDCD) method is proposed for learning Directed Acyclic Graphs (DAGs) from observational or interventional data.
StableMask: Refining Causal Masking in Decoder-only Transformer
Qingyu Yin (Zhejiang University), Qiang Zhang (Zhejiang University)
TransformerLarge Language ModelText
🎯 What it does: StableMask is proposed, an improvement of causal masking that is parameter-independent, which balances the attention distribution and encodes absolute positional information by incorporating pseudo attention values.
StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization
Shida Wang (National University of Singapore), Qianxiao Li (National University of Singapore)
Recurrent Neural NetworkImageText
🎯 What it does: This paper studies the limits of state space models (SSM) in long sequence memory learning, proving that SSMs without reparameterization also exhibit a 'memory curse' similar to RNNs, and proposes a stable reparameterization method that allows SSMs to stably approximate any decaying memory objective function.
Stacking Deep Set Networks and Pooling by Quantiles
Zhuojun Chen (ASTRI), Justin C. I. CHUANG
Computational EfficiencyRepresentation LearningPoint Cloud
🎯 What it does: Two new methods are proposed: Quantile Pooling and Stacked Deep Sets, for handling unordered set data.
StackSight: Unveiling WebAssembly through Large Language Models and Neurosymbolic Chain-of-Thought Decompilation
Weike Fang (University of Southern California), Weihang Wang (University of Southern California)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: StackSight proposes a phased chain-of-thought (CoT) pipeline that combines static stack visualization with large language models (LLMs) to progressively reconstruct WebAssembly code into readable C++ code and generate functional summaries.
Standardized Interpretable Fairness Measures for Continuous Risk Scores
Ann-Kristin Becker (SCHUFA Holding AG), Klaus Broelemann (SCHUFA Holding AG)
Explainability and InterpretabilityTabular
🎯 What it does: A standardized and interpretable fairness metric method for continuous risk scores is proposed, which quantifies inequality between groups based on the Wasserstein-1 distance.
State-Constrained Zero-Sum Differential Games with One-Sided Information
Mukesh Ghimire (Arizona State University), Yi Ren (Arizona State University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningSequentialStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proves the existence of value functions in zero-sum differential games with state constraints and one player having private types, and provides the corresponding primal and dual Hamilton–Jacobi equations.
State-Free Inference of State-Space Models: The *Transfer Function* Approach
Rom Parnichkun, Atsushi Yamashita (University of Tokyo)
Computational EfficiencyText
🎯 What it does: This paper proposes a state-space model represented by the Rational Transfer Function (RTF) and presents a state-independent parallel inference algorithm that requires only O(ℓ) space and O(ℓ log ℓ) time.
Stationarity without mean reversion in improper Gaussian processes
Luca Ambrogioni (Radboud University)
Anomaly DetectionOptimizationData-Centric LearningTabularTime SeriesFinance Related
🎯 What it does: This paper proposes the use of improper Gaussian processes with infinite variance, achieving a stationary and non-mean-reverting process by introducing conditionally positive definite kernels.
Stationary Latent Weight Inference for Unreliable Observations from Online Test-Time Adaptation
Jae-Hong Lee (Hanyang University), Joon-Hyuk Chang (Hanyang University)
Domain AdaptationTransformerImageStochastic Differential Equation
🎯 What it does: A method using Bayesian filters to infer latent weights (SLWI) in Online Test-Time Adaptation (OTTA) is proposed, which continuously accumulates information from the source domain and target domain to reduce error propagation caused by noise in target weights.
Statistical Inference Under Constrained Selection Bias
Santiago Cortes-Gomez (Carnegie Mellon University), Bryan Wilder (Carnegie Mellon University)
OptimizationTabular
🎯 What it does: A framework is proposed that utilizes known target distribution expectation constraints to partially identify statistics in the presence of selection bias and provides high-probability interval estimates.
Statistical Properties of Robust Satisficing
zhiyi li, Ruohan Zhan (Hong Kong University of Science and Technology)
OptimizationTabular
🎯 What it does: This paper studies the statistical properties of the Robust Satisficing (RS) model, providing two-sided confidence intervals and finite sample generalization error bounds, and conducts theoretical analysis and experimental validation in the presence of distribution drift.
Statistical Test for Attention Maps in Vision Transformers
Tomohiro Shiraishi (Nagoya University), Ichiro Takeuchi (Nagoya University)
TransformerBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A statistical testing framework based on Selective Inference is proposed to quantify the significance of Vision Transformer (ViT) attention maps and control the false positive rate.
Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution
Elen Vardanyan (Yerevan State University), Arnak S. Dalalyan
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: This paper studies the diversity of generative models and the property of avoiding the replication of training samples, providing theoretical lower and upper bounds;
Stay on Topic with Classifier-Free Guidance
Guillaume Sanchez, Stella Biderman (EleutherAI)
GenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Introducing Classifier-Free Guidance (CFG) during the inference phase of language models enhances the model's compliance with prompts and significantly improves performance on various benchmark tasks by linearly combining conditional and unconditional distributions of the logits for the next word, even surpassing the LAMBADA zero-shot score of PaLM-540B on LLaMA-7B.
Stealing part of a production language model
Nicholas Carlini (Google DeepMind), Florian Tramèr (ETH Zurich)
Adversarial AttackTransformerLarge Language ModelText
🎯 What it does: A model stealing attack targeting black-box APIs has been designed and implemented, capable of recovering the weight matrix of the final projection layer of Transformer models and its hidden dimensions.
Stealthy Imitation: Reward-guided Environment-free Policy Stealing
Zhixiong Zhuang (Bosch Center for Artificial Intelligence), Mario Fritz (CISPA Helmholtz Center for Information Security)
Adversarial AttackRobotic IntelligenceReinforcement LearningSequential
🎯 What it does: A new attack method called Stealthy Imitation is proposed, which can steal the functionality of deep reinforcement learning policies through black-box queries without knowledge of the environment information and input range.
STEER: Assessing the Economic Rationality of Large Language Models
Narun Krishnamurthi Raman (University of British Columbia), Moshe Tennenholtz (Technion)
TransformerLarge Language ModelPrompt EngineeringTextFinance Related
🎯 What it does: A STEER evaluation framework is proposed, constructing a hierarchical classification of 64 economic rationality factors. A large number of multiple-choice questions are automatically generated by LLMs and manually verified, forming a quantifiable benchmark. Subsequently, systematic evaluations are conducted on 14 different scales of LLMs, generating customizable STEER report cards.
STELLA: Continual Audio-Video Pre-training with SpatioTemporal Localized Alignment
Jaewoo Lee (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
RetrievalRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningVideoMultimodalityAudio
🎯 What it does: This paper proposes STELLA, a continuous audio-video pre-training method that addresses the issues of sparse spatiotemporal associations and multimodal association forgetting through self-supervised learning.
Stereo Risk: A Continuous Modeling Approach to Stereo Matching
Ce Liu (Nanjing University), Luc Van Gool (KU Leuven)
Depth EstimationOptimizationConvolutional Neural NetworkImage
🎯 What it does: A continuous stereo matching method based on L1 risk minimization is proposed and integrated into an end-to-end trainable deep network to achieve high-precision disparity estimation.
Stereographic Spherical Sliced Wasserstein Distances
Huy Tran (Vanderbilt University), Soheil Kolouri (Vanderbilt University)
Computational EfficiencyImage
🎯 What it does: A Stereographic Spherical Sliced Wasserstein (S3W) distance based on spherical projection and generalized Radon transform is proposed for efficient comparison of spherical probability distributions.
Stochastic Bandits with ReLU Neural Networks
Kan Xu (Arizona State University), Osbert Bastani (University of Pennsylvania)
OptimizationReinforcement Learning
🎯 What it does: This paper studies the stochastic bandit problem with a ReLU neural network structure and proposes the OFU-ReLU algorithm, which achieves a regret guarantee of O(√T).
Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis
Juyeon Ko (Korea University), Hyunwoo J. Kim (Korea University)
SegmentationGenerationData SynthesisDiffusion modelImage
🎯 What it does: A random conditional diffusion model (SCDM) based on label diffusion is proposed to improve the robustness of semantic image synthesis under noisy labels.
Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features
Rodrigo Veiga (Ecole Polytechnique Federale de Lausanne), Nicolas Macris (Ecole Polytechnique Federale de Lausanne)
Stochastic Differential Equation
🎯 What it does: This study investigates the testing risk of continuous-time stochastic gradient flow dynamics in learning theory, providing a general formula for calculating the differences in testing risk curves between pure gradient and stochastic gradient flows under small learning rates, and applies it to weak feature models.
Stochastic Interpolants with Data-Dependent Couplings
Michael Samuel Albergo, Eric Vanden-Eijnden (New York University)
RestorationGenerationSuper ResolutionDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A generative model using data-dependent coupling within a random interpolation framework is proposed, which can pair the base distribution with the target distribution through conditional coupling, thereby obtaining a more suitable transport path in continuous time mapping.
Stochastic Localization via Iterative Posterior Sampling
Louis Grenioux (Centre de Mathématiques Appliquées), Alain Oliviero Durmus (Centre de Mathématiques Appliquées)
BenchmarkStochastic Differential Equation
🎯 What it does: This paper proposes a new stochastic localization method called Stochastic Localization via Iterative Posterior Sampling (SLIPS) for sampling from unnormalized target densities.
Stochastic Optimization with Arbitrary Recurrent Data Sampling
William Powell (University of Wisconsin-Madison), Hanbaek Lyu (University of Wisconsin-Madison)
OptimizationTabular
🎯 What it does: For non-convex constrained minimization problems, a random optimization method with recursive sampling, RMISO, is proposed. It is proven that under the condition of satisfying only the recursive sampling criteria, an optimal first-order convergence rate of O(1/√n) can be achieved.
Stochastic positional embeddings improve masked image modeling
Amir Bar (Tel Aviv University), Yann LeCun (New York University)
Representation LearningTransformerContrastive LearningImage
🎯 What it does: In the self-supervised Masked Image Modeling (MIM) framework, Stochastic Positional Embeddings (StoP) are introduced to allow the model to consider positional uncertainty when predicting masked patches, thereby improving representation learning.
Stochastic Q-learning for Large Discrete Action Spaces
Fares Fourati (King Abdullah University of Science and Technology), Mohamed-Slim Alouini (King Abdullah University of Science and Technology)
Reinforcement LearningSequential
🎯 What it does: This study investigates a variant of Q-learning that uses random subsets for maximization in large discrete action spaces, reducing the computational complexity from O(n) to O(log n).
Stochastic Quantum Sampling for Non-Logconcave Distributions and Estimating Partition Functions
Guneykan Ozgul (Pennsylvania State University), Chunhao Wang (Pennsylvania State University)
OptimizationPhysics Related
🎯 What it does: This paper studies quantum sampling for non-log-concave distributions and a quantum Lagrangian algorithm based on stochastic gradients, proposing a quantum scheme for solving its partition function.
Stochastic Weakly Convex Optimization beyond Lipschitz Continuity
Wenzhi Gao (Stanford University), Qi Deng (Shanghai Jiao Tong University)
OptimizationReinforcement Learning
🎯 What it does: This paper studies stochastic weak convex optimization problems without the assumption of standard Lipschitz continuity. Through a new robust regularization (step size) strategy, it demonstrates that various stochastic algorithms, including stochastic subgradient methods, maintain a constant failure rate with a convergence rate of O(1/√K).
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
Jesse Farebrother (Google DeepMind), Rishabh Agarwal (Google DeepMind)
Convolutional Neural NetworkTransformerReinforcement LearningSequential
🎯 What it does: The training of the value function is changed from traditional mean squared error regression to a classification task, using categorical cross-entropy loss to train the Q network.
Straight-Through Meets Sparse Recovery: the Support Exploration Algorithm
Mimoun Mohamed (Aix Marseille University), Caroline Chaux (National Center for Scientific Research)
OptimizationTabular
🎯 What it does: A sparse support recovery algorithm based on the Straight-Through Estimator (Support Exploration Algorithm, SEA) is proposed, which explores more possible support sets by introducing a full-precision vector X in the gradient update.
StrokeNUWA—Tokenizing Strokes for Vector Graphic Synthesis
Zecheng Tang (Soochow University), Nan Duan (Microsoft Research Asia)
GenerationData SynthesisCompressionComputational EfficiencyTransformerLarge Language ModelImage
🎯 What it does: This paper proposes StrokeNUWA, which utilizes stroke tokens for quantizing and encoding SVGs, allowing LLMs to directly generate vector graphics without relying on traditional visual modules, significantly improving generation speed and compression rate.
Structure Your Data: Towards Semantic Graph Counterfactuals
Angeliki Dimitriou (National Technical University of Athens), Giorgos Stamou (National Technical University of Athens)
RetrievalExplainability and InterpretabilityGraph Neural NetworkImageMultimodalityGraphAudio
🎯 What it does: A model-agnostic counterfactual explanation method based on semantic graphs is proposed, which combines graph edit distance (GED) with graph neural networks (GNN) to achieve efficient counterfactual retrieval and explanation generation.
Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks
Duy Minh Ho Nguyen, Mathias Niepert (Max Planck Research School for Intelligent Systems)
Drug DiscoveryGraph Neural NetworkGraphTabular
🎯 What it does: A network has been designed to aggregate molecular 2D maps with multiple 3D conformations, using a differentiable Fused Gromov-Wasserstein barycenter for E(3) invariant aggregation.
Structure-based drug design by denoising voxel grids
Pedro O. Pinheiro (Genentech), Saeed Saremi (Genentech)
Drug DiscoveryConvolutional Neural NetworkScore-based ModelPoint Cloud
🎯 What it does: We propose VoxBind, a voxel-based scoring generation model for structure-guided drug design.
Structured Chemistry Reasoning with Large Language Models
Siru Ouyang (University of Illinois Urbana-Champaign), Lianhui Qin (University of California SanDiego)
TransformerLarge Language ModelPrompt EngineeringTextPhysics Related
🎯 What it does: This study investigates the shortcomings of LLMs in complex chemical reasoning tasks and proposes the STRUCTCHEM prompting framework, which enhances performance through staged formula generation, step-by-step reasoning, and self-review.