ICML 2023 Papers — Page 16
International Conference on Machine Learning · 1828 papers
Sequential Strategic Screening
Lee Cohen (Toyota Technological Institute at Chicago), Juba Ziani (Georgia Institute of Technology)
Optimization
🎯 What it does: The study investigates how agents can achieve positive outcomes by manipulating features to pass through all filters in a multiple screening process (composed of a series of linear classifiers).
Sequential Underspecified Instrument Selection for Cause-Effect Estimation
Elisabeth Ailer (Helmholtz Munich), Niki Kilbertus (Munich Center for Machine Learning)
🎯 What it does: This paper proposes a method to estimate causal effects through stepwise experiments and instrument variable selection in the case of high-dimensional processing variables where only a limited number of instrumental variables can be randomized.
Set-membership Belief State-based Reinforcement Learning for POMDPs
Wei Wei (Shanxi University), Jiye Liang (Shanxi University)
Reinforcement Learning
🎯 What it does: A belief state learning model based on set membership (SBM) and a reinforcement learning controller (SBRL) are proposed for decision-making in partially observable Markov decision processes (POMDP) with unknown but bounded noise.
Settling the Reward Hypothesis
Michael Bowling (Amii), Will Dabney (DeepMind)
Reinforcement Learning
🎯 What it does: This paper provides a complete formalization and proof of the Reward Hypothesis at a theoretical level, presenting the necessary and sufficient conditions for the realization of Markov reward functions from preference relations;
SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance
Amit Attia (Tel Aviv University), Tomer Koren (Google Research)
Optimization
🎯 What it does: This paper studies the high-probability convergence analysis of AdaSGD (SGD with adaptive AdaGrad step size) in scenarios with unknown parameters, without global Lipschitz conditions, and applicable to high-variance noise as well as convex/non-convex smooth problems.
SGD with Large Step Sizes Learns Sparse Features
Maksym Andriushchenko (École Polytechnique Fédérale de Lausanne), Nicolas Flammarion (École Polytechnique Fédérale de Lausanne)
OptimizationConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: This paper investigates the 'loss stabilization' phase that occurs when training neural networks with a large learning rate (step size) through theoretical analysis and extensive experiments. It reveals the hidden stochastic dynamics generated during this phase and demonstrates that it can implicitly encourage SGD to learn sparse features.
Shape-Guided Dual-Memory Learning for 3D Anomaly Detection
Yu-Min Chu (National Tsing Hua University), Tyng-Luh Liu (Academia Sinica)
Anomaly DetectionConvolutional Neural NetworkImagePoint Cloud
🎯 What it does: A shape-guided dual memory learning framework is proposed, utilizing two expert models (shape expert and appearance expert) to jointly detect and locate anomalies in 3D point clouds.
Shapley Based Residual Decomposition for Instance Analysis
Tommy Liu (Australian National University), Amanda Susan Barnard
Tabular
🎯 What it does: The study decomposes regression residuals using Shapley values, obtaining the contribution of each instance to its own residual and the influence from other instances, forming a 'contribution-combination' analysis framework.
Sharp Variance-Dependent Bounds in Reinforcement Learning: Best of Both Worlds in Stochastic and Deterministic Environments
Runlong Zhou (University of Washington), Simon Shaolei Du
Reinforcement Learning
🎯 What it does: In online reinforcement learning for finite Markov decision processes (MDP), two new environmental norms (total multi-step conditional variance VarΣ and maximum policy value variance Var⍷) are proposed, and based on this, model-based method MVP-V and model-free method UCB-Advantage-V are designed, achieving optimal or near-optimal variance-dependent regret upper bounds in both stochastic and deterministic MDPs.
Sharper Bounds for $\ell_p$ Sensitivity Sampling
David Woodruff, Taisuke Yasuda (Carnegie Mellon University)
Tabular
🎯 What it does: This paper studies the problem of embedding in ℓ_p subspaces, using sensitivity sampling to obtain better sampling upper bounds.
Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances
Ruben Ohana (Flatiron Institute), Liva Ralaivola (Criteo AI Lab)
OptimizationGenerative Adversarial NetworkImage
🎯 What it does: This paper presents a generalization error upper bound for adaptive Sliced-Wasserstein distance based on PAC-Bayesian theory, and proposes an optimization method (PAC-SW) for learning tangent distributions using this bound.
Shiftable Context: Addressing Training-Inference Context Mismatch in Simultaneous Speech Translation
Matthew Raffel (Oregon State University), Lizhong Chen (Oregon State University)
TransformerTextAudio
🎯 What it does: This paper addresses the issue of context mismatch during the training and inference processes of segment-based Transformer models in simultaneous speech translation, proposing the Shiftable Context scheme to resolve this problem.
Short-lived High-volume Bandits
Su Jia (Cornell University), Ramamoorthi Ravi
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: This study investigates the A/B/n testing problem under high-volume short-lifetime treatment, proposing a high-volume short-lifetime Bandits model and providing an analysis of average loss.
Shortest Edit Path Crossover: A Theory-driven Solution to the Permutation Problem in Evolutionary Neural Architecture Search
Xin Qiu (Cognizant AI Labs), Risto Miikkulainen (Cognizant AI Labs)
OptimizationNeural Architecture SearchGraph
🎯 What it does: This paper proposes a crossover operator based on the Shortest Edit Path (SEP) to address the permutation problem in Neural Architecture Search (NAS).
Simple and Fast Group Robustness by Automatic Feature Reweighting
Shikai Qiu (New York University), Andrew Gordon Wilson (New York University)
ClassificationComputational EfficiencySupervised Fine-TuningImageText
🎯 What it does: An Automatic Feature Reweighting (AFR) method is proposed, which enhances robustness to spurious features by weighting misclassified samples and fine-tuning only the last layer of a model trained using standard ERM.
simple diffusion: End-to-end diffusion for high resolution images
Emiel Hoogeboom (Google Research), Tim Salimans (Google Research)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: An improved standard denoising diffusion model for high-resolution images allows for single-stage training directly in pixel space to generate high-quality images.
Simple Disentanglement of Style and Content in Visual Representations
Lilian Ngweta (Rensselaer Polytechnic Institute), Mikhail Yurochkin (IBM Research)
Domain AdaptationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes a post-processing framework called PISCO, which can separate the features of pre-trained visual models into two parts: style and content, achieving interpretable separable representations.
Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learning
Evan Zheran Liu (Stanford University), Chelsea Finn (Stanford University)
Robotic IntelligenceMeta LearningReinforcement LearningImage
🎯 What it does: In a multi-task office navigation environment, meta reinforcement learning is used to enable robots to indirectly learn to recognize and interpret visual floor plans solely through exploration tasks (without language supervision), allowing them to quickly locate target offices during the evaluation phase.
Simple Hardware-Efficient Long Convolutions for Sequence Modeling
Daniel Y Fu, Christopher Re
ImageSequentialBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This study investigates the direct learning of long convolutions to replace state space models and proposes the IO-aware FLASHBUTTERFLY acceleration algorithm.
Simplex Random Features
Isaac Reid (University of Cambridge), Adrian Weller (Alan Turing Institute)
ClassificationOptimizationTransformerImageTabular
🎯 What it does: This paper proposes a new random feature mechanism called Simplex Random Features (SimRFs) for unbiased approximation of Gaussian kernels and softmax kernels, and proves that it achieves the minimum mean square error in the weight-independent geometric coupling PRF mechanism, significantly outperforming existing Orthogonal Random Features (ORFs).
Simplified Temporal Consistency Reinforcement Learning
Yi Zhao (Aalto University), Joni Pajarinen (Aalto University)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningContrastive LearningSequential
🎯 What it does: This paper proposes a time-consistent simplified representation learning method (TCRL), which trains the encoder and hidden dynamic model using only the hidden state consistency loss, and directly trains the policy and value function on this representation, combining both model-based and model-free learning modes.
Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Wu Lin (University of British Columbia), Mark Schmidt (University of Alberta)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: A general orthogonal coordinate (GNC) is proposed to simplify momentum Riemannian optimization on SPD submanifolds, and an efficient optimizer without matrix inversion is constructed using this coordinate for low-precision training in deep learning.
SinDDM: A Single Image Denoising Diffusion Model
Vladimir Kulikov (Technion - Israel Institute of Technology), Tomer Michaeli (Technion - Israel Institute of Technology)
RestorationGenerationDiffusion modelImage
🎯 What it does: A single-image diffusion model, SinDDM, is proposed, which can learn multi-scale internal statistics from a single image, generate diversified images of arbitrary sizes, and support text or region guidance, style transfer, and fusion.
SinFusion: Training Diffusion Models on a Single Image or Video
Yaniv Nikankin (Weizmann Institute of Science), michal Irani
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelImageVideo
🎯 What it does: Train a diffusion model (SinFusion) that can learn its appearance and dynamics using only a single image or a short video, and can perform various tasks such as generating diverse new images/videos, executing image/video editing, video extrapolation (forward and backward), and video upsampling.
Single Point-Based Distributed Zeroth-Order Optimization with a Non-Convex Stochastic Objective Function
Elissa Mhanna (Universite Paris Saclay), Mohamad Assaad (Universite Paris Saclay)
OptimizationImage
🎯 What it does: This paper proposes a distributed optimization algorithm based on single-point zero-order gradient estimation, suitable for non-convex stochastic objective functions, and provides a proof of convergence.
Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition
Dongqi Cai (Intel Labs China), Yurong Chen (Intel Labs China)
RecognitionRepresentation LearningGraph Neural NetworkVideo
🎯 What it does: A representation learning framework called Ske2Grid is designed based on the transformation from skeleton graphs to grids to improve skeleton action recognition.
Sketch-Flip-Merge: Mergeable Sketches for Private Distinct Counting
Jonathan Hehir (Penn State University), Graham Cormode (University of Warwick)
OptimizationSafty and PrivacyTabular
🎯 What it does: A mergeable approximate cardinality counting Sketch (Sketch-Flip-Merge, SFM) that satisfies ε-differential privacy is proposed.
Sketched Ridgeless Linear Regression: The Role of Downsampling
Xin Chen (Princeton University), Qiang Sun (University of Toronto)
Tabular
🎯 What it does: This study investigates the statistical generalization performance of zero-ridge least squares regression obtained through random sketching under high-dimensional settings and provides the optimal sketch size.
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability
Zhao Song (Adobe Research), Lichen Zhang (Massachusetts Institute of Technology)
OptimizationFederated LearningSafty and PrivacyComputational Efficiency
🎯 What it does: An iterative sketch-and-desketch framework is proposed, which compresses gradients using random projections to reduce communication costs in federated learning, and adds Gaussian noise to achieve differential privacy.
Sketching Meets Differential Privacy: Fast Algorithm for Dynamic Kronecker Projection Maintenance
Zhao Song (Adobe Research), Lichen Zhang (Massachusetts Institute of Technology)
Safty and PrivacyComputational Efficiency
🎯 What it does: A fast algorithm for maintaining dynamic Kronecker projection matrices is proposed.
SLAMB: Accelerated Large Batch Training with Sparse Communication
Hang Xu (King Abdullah University of Science and Technology), Panos Kalnis (King Abdullah University of Science and Technology)
OptimizationTransformerImageText
🎯 What it does: An optimizer named SLAMB is proposed, which combines sparse communication with large-batch training, aiming to achieve efficient distributed training on large-scale GPU clusters.
Sliced-Wasserstein on Symmetric Positive Definite Matrices for M/EEG Signals
Clément Bonet (Université Bretagne Sud), Nicolas Courty (Université Bretagne Sud)
Domain AdaptationBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A new sliced Wasserstein distance (SPDSW) is proposed in the space of symmetric positive definite matrices, and it is applied to brain age prediction and BCI domain adaptation tasks using EEG/MRI data.
Slot-VAE: Object-Centric Scene Generation with Slot Attention
Yanbo Wang (Delft University of Technology), Justin Dauwels (Delft University of Technology)
GenerationData SynthesisConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: The Slot-VAE model is proposed, which combines Slot Attention with hierarchical VAE to achieve unsupervised object-centric scene decomposition and generation.
SlotGAT: Slot-based Message Passing for Heterogeneous Graphs
Ziang Zhou (Polytechnic University), Qing Li (Polytechnic University)
ClassificationRecommendation SystemGraph Neural NetworkGraph
🎯 What it does: This paper proposes SlotGAT, a slot-based message passing model designed to handle heterogeneous graphs, addressing the issue of semantic confusion that traditional GNNs encounter when aggregating information from different types of nodes.
Smart Initial Basis Selection for Linear Programs
Zhenan Fan (Huawei Technologies Canada), Zirui Zhou (Simon Fraser University)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a learning-based initial basis selection method based on graph neural networks to accelerate the solution of simplex and column generation algorithms;
Smooth Non-stationary Bandits
Su Jia (Cornell University), Peter I. Frazier (Cornell University)
OptimizationReinforcement LearningTime Series
🎯 What it does: A smooth non-stationary multi-armed bandit model is proposed, proving that when the reward function satisfies β-Holder smoothness, it can break the traditional Lipschitz variation scheduling limit of ˜O(T^{2/3}) and achieve an optimal upper bound of ˜O(T^{3/5}); at the same time, a lower bound Ω(T^{(β+1)/(2/β+1)}) related to β is provided, proving the optimality of the upper bound.
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Guangxuan Xiao (Massachusetts Institute of Technology), Song Han (Massachusetts Institute of Technology)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A post-training quantization method called SmoothQuant is proposed, which can deploy large language models (such as OPT, BLOOM, GLM‑130B, MT‑NLG 530B) in INT8 weight + activation (W8A8) format without retraining the model, while maintaining accuracy close to FP16, significantly reducing memory usage and inference latency.
SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process
Zichong Li (University of Science and Technology of China), Hongyuan Zha (Chinese University of Hong Kong)
TransformerScore-based ModelTextTabularTime SeriesSequentialBiomedical DataFinance Related
🎯 What it does: A Transformer Hawkes process model based on score matching (SMURF-THP) is proposed to quantify the uncertainty of event arrival times.
SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning
Dongseok Shim (Seoul National University), H. Jin Kim (Seoul National University)
Representation LearningConvolutional Neural NetworkTransformerReinforcement LearningNeural Radiance FieldContrastive LearningImage
🎯 What it does: A pre-training framework SNeRL is proposed, which combines multi-view convolutional encoders and semantic-aware NeRF. It first learns 3D-aware semantics and feature representations from offline data, and then uses the encoder for reinforcement learning tasks.
Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning
Seungwoong Ha (Korea Advanced Institute of Science and Technology), Hawoong Jeong (Korea Advanced Institute of Science and Technology)
OptimizationReinforcement Learning
🎯 What it does: This study uses a deep reinforcement learning model to optimize individual social learning strategies (SLS) in cooperative games and simulates the spontaneous emergence and evolution of social learning through a multidimensional NK landscape.
Solving High-Dimensional PDEs with Latent Spectral Models
Haixu Wu (Tsinghua University), Mingsheng Long (Tsinghua University)
TransformerPoint CloudMeshBenchmarkPhysics Related
🎯 What it does: Latent Spectral Models (LSM) are proposed, which compress high-dimensional coordinate space into latent space through an attention-based hierarchical projection network, and then approximate the input-output mapping in the latent space using neural spectral blocks, thereby efficiently solving high-dimensional PDEs.
Solving Linear Programs with Fast Online Learning Algorithms
Wenzhi Gao (Shanghai University of Finance and Economics), Yinyu Ye (Stanford University)
Optimization
🎯 What it does: This paper proposes two online learning algorithms that do not require matrix multiplication (explicit subgradient and implicit proximal point updates) and applies them to solve offline linear programming through variable replication techniques, further embedding these algorithms into a column generation framework (sifting) to accelerate large-scale LP solving.
SOM-CPC: Unsupervised Contrastive Learning with Self-Organizing Maps for Structured Representations of High-Rate Time Series
Iris A.M. Huijben (Eindhoven University of Technology), Ruud Van Sloun
Anomaly DetectionRepresentation LearningRecurrent Neural NetworkContrastive LearningMultimodalityTime SeriesAudio
🎯 What it does: Proposes the SOM-CPC model, which combines contrastive learning with self-organizing maps to generate interpretable two-dimensional visual representations of high-frequency time series.
Sparse Learning of Dynamical Systems in RKHS: An Operator-Theoretic Approach
Boya Hou (University of Illinois), Umesh Vaidya (Clemson University)
OptimizationComputational EfficiencyTime SeriesStochastic Differential Equation
🎯 What it does: This paper proposes a transfer operator for sparse learning dynamical systems in reproducing kernel Hilbert spaces and provides an upper bound on sample complexity for non-i.i.d. β-mixing processes.
SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot
Elias Frantar (Institute of Science and Technology Austria), Dan Alistarh (Neural Magic Inc.)
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: A new one-time pruning method called SparseGPT is studied, which can achieve 50-60% sparsity for GPT models over 175B without the need for fine-tuning.
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks at the Edge
Mahdi Nikdan (IST Austria), Dan Alistarh (Neural Magic)
Computational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an efficient backpropagation implementation for sparse weight networks called SparseProp;
Spatial Implicit Neural Representations for Global-Scale Species Mapping
Elijah Cole (California Institute of Technology), Oisin Mac Aodha (University of Edinburgh)
TabularBenchmarkAgriculture Related
🎯 What it does: This paper utilizes Spatial Implicit Neural Representation (SINR) to jointly estimate the distribution range of approximately 47,000 species globally from a large amount of presence-only records in the iNaturalist dataset.
Spatial-Temporal Graph Learning with Adversarial Contrastive Adaptation
Qianru Zhang (University of Hong Kong), Ruihua Han (University of Hong Kong)
Recommendation SystemAnomaly DetectionGraph Neural NetworkAuto EncoderContrastive LearningGraphTime Series
🎯 What it does: A self-supervised spatial-temporal graph learning framework named GraphST is proposed to learn regional embeddings under noisy and sparse urban data, which can be used for tasks such as crime prediction, traffic flow prediction, and housing price prediction.
Special Properties of Gradient Descent with Large Learning Rates
Amirkeivan Mohtashami (École Polytechnique Fédérale de Lausanne), Sebastian U Stich
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper studies the special properties of gradient descent when using a large learning rate in neural network training, proving that a large learning rate can help gradient descent escape local minima and converge to global minima.
Specializing Smaller Language Models towards Multi-Step Reasoning
Yao Fu (University of Edinburgh), Tushar Khot (Allen Institute for AI)
OptimizationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: By fine-tuning and distilling small-scale language models (≤10B parameters) like FlanT5 for specialized multi-step reasoning (Chain-of-Thought) capabilities; using reasoning paths generated by GPT-3.5 (code-davinci-002) as training samples significantly improves the model's performance on mathematical reasoning tasks.
Speed-Oblivious Online Scheduling: Knowing (Precise) Speeds is not Necessary
Alexander Lindermayr (University of Bremen), Martin Rapp (Karlsruhe Institute of Technology)
OptimizationTabularBenchmark
🎯 What it does: A speed-independent online scheduling model is proposed and analyzed, studying two practical scenarios: prediction and speed sorting, and providing corresponding competitive analysis and experimental validation.
SpeedDETR: Speed-aware Transformers for End-to-end Object Detection
Peiyan Dong (Northeastern University), Chih-Hsien Chou (Futurewei Technologies)
Object DetectionTransformerImage
🎯 What it does: This paper presents SpeedDETR, a speed-aware end-to-end object detector based on Transformer, which combines a latency prediction model and a hardware-friendly architecture to achieve high-speed inference across multiple devices.
Speeding Up Bellman Ford via Minimum Violation Permutations
Silvio Lattanzi (Google Research), Sergei Vassilvitskii (Google Research)
OptimizationComputational EfficiencyGraphTime Series
🎯 What it does: This paper proposes a preprocessing method to learn the optimal vertex traversal order when similar instances are known, significantly accelerating the Bellman-Ford single-source shortest path algorithm.
SpENCNN: Orchestrating Encoding and Sparsity for Fast Homomorphically Encrypted Neural Network Inference
Ran Ran (Lehigh University), Wujie Wen
Computational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper proposes the SpENCNN framework, which combines the SIMD characteristics of homomorphic encryption (HE) with model sparsification to accelerate encrypted neural network inference.
Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere
Boris Bonev (NVIDIA Corporation), Anima Anandkumar (California Institute of Technology)
Time SeriesPhysics Related
🎯 What it does: This paper proposes the Spherical Fourier Neural Operator (SFNO) for learning stable long-term physical dynamics on the sphere.
Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes
Louis C. Tiao (University of Sydney), Victor Picheny (Secondmind)
OptimizationRepresentation LearningTabular
🎯 What it does: A model is proposed that combines spherical activation features with orthogonal decoupled variational Gaussian processes to address the shortcomings of traditional activation feature GP in kernel and activation selection.
SpotEM: Efficient Video Search for Episodic Memory
Santhosh Kumar Ramakrishnan (University of Texas at Austin), Kristen Grauman (Meta AI)
RetrievalComputational EfficiencyKnowledge DistillationTransformerVideoBenchmark
🎯 What it does: This paper proposes SpotEM, which significantly reduces the amount of feature extraction required for Episodic Memory (EM) queries on long-term first-person videos by utilizing a segment selector under query conditions and a low-cost semantic index.
spred: Solving L1 Penalty with SGD
Liu Ziyin (University of Tokyo), Zihao Wang (Hong Kong University of Science and Technology)
CompressionOptimizationConvolutional Neural NetworkSupervised Fine-TuningImageBiomedical Data
🎯 What it does: A method is proposed to transform the L1 constraint of non-convex objectives into a differentiable problem through reparameterization (Hadamard multiplication), and directly optimize it using standard SGD (or Adam), referred to as the spred method.
Spurious Valleys and Clustering Behavior of Neural Networks
Samuele Pollaci (Vrije Universiteit Brussel), Samuele Pollaci (University of Bonn)
Optimization
🎯 What it does: This paper studies the geometric properties of the loss landscape of neural networks, proving that pseudo-minima can be eliminated when the hidden layers are sufficiently wide; it also proposes using singular point clustering on cross-sections to determine whether the objective function can be approximated by polynomial networks.
SRATTA: Sample Re-ATTribution Attack of Secure Aggregation in Federated Learning.
Tanguy Marchand (Owkin Inc), Arthur Pignet (Owkin Inc)
Federated LearningSafty and PrivacyAdversarial AttackConvolutional Neural NetworkImageMultimodalityBiomedical Data
🎯 What it does: In cross-device federated learning, the SRATTA attack is proposed, which can recover and classify samples to each client solely by using the aggregated model after secure aggregation, thereby breaking the privacy protection of secure aggregation.
Stabilizing GANs' Training with Brownian Motion Controller
Tianjiao Luo (Tsinghua University), Jun Zhu (Tsinghua University)
GenerationData SynthesisGenerative Adversarial NetworkImageStochastic Differential Equation
🎯 What it does: A Brownian Motion Controller (BMC) is proposed and implemented, which achieves synchronized and stable training of the generator and discriminator by incorporating a noise control term into the GAN training dynamics.
Stabilizing Transformer Training by Preventing Attention Entropy Collapse
Shuangfei Zhai (Apple), Joshua M. Susskind (Apple)
TransformerSupervised Fine-TuningImageTextAudio
🎯 What it does: By monitoring attention entropy and proposing σ Reparam to prevent the collapse of attention entropy, we stabilize the training of Transformers.
Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learning
Seungwook Kim (POSTECH), Minsu Cho (POSTECH)
SegmentationPose EstimationTransformerPoint Cloud
🎯 What it does: This study proposes a method for predicting the feature direction of 3D point clouds that simultaneously achieves stability and consistency, enabling the canonicalization of point clouds.
Stable Estimation of Heterogeneous Treatment Effects
Anpeng Wu (Zhejiang University), Fei Wu (Zhejiang University)
OptimizationTabular
🎯 What it does: This paper proposes the StableCFR method, which utilizes unified sampling and ε-greedy matching to upsample minority groups while maintaining covariate balance, thereby achieving robust heterogeneous treatment effect estimation for underrepresented populations.
State and parameter learning with PARIS particle Gibbs
Gabriel Cardoso (Ecole Polytechnique), Jimmy Olsson (KTH Royal Institute of Technology)
Recurrent Neural NetworkTime SeriesSequential
🎯 What it does: This paper addresses the state smoothing and parameter learning issues of nonlinear hidden Markov models (HMM) and proposes a novel PARIS-Particle Gibbs (PPG) algorithm, providing theoretical results on its bias, variance, and non-asymptotic convergence of gradient estimates during the learning process.
Statistical Foundations of Prior-Data Fitted Networks
Thomas Nagler (LMU Munich)
TransformerTabular
🎯 What it does: This paper presents the theoretical foundation of Prior-Data Fitted Networks (PFN) and explains the self-learning (in-context learning) mechanism it can achieve during the inference phase.
Statistical Indistinguishability of Learning Algorithms
Alkis Kalavasis (National Technical University of Athens), Grigoris Velegkas (Yale University)
OptimizationSafty and Privacy
🎯 What it does: This study investigates the statistical distinguishability of output distributions of learning algorithms on different datasets, introduces and analyzes the concept of TV indistinguishability, proves its equivalence with replicability and approximate differential privacy ((ε,δ)-DP) in countable domains, and provides corresponding amplification and enhancement algorithms.
Statistical Inference and A/B Testing for First-Price Pacing Equilibria
Luofeng Liao (Columbia University), Christian Kroer (Columbia University)
Optimization
🎯 What it does: This paper constructs a statistical inference and A/B testing framework for the First-Price Pacing Equilibrium (FPPE) based on budget management in first-price auction markets. It proposes the mean consistency of observing FPPE, the central limit theorem, and the theory of local minimization optimality, and provides confidence interval estimation and practical experimental design schemes.
Statistical Inference on Multi-armed Bandits with Delayed Feedback
Lei Shi (University of California), Tianhao Wu (University of California)
Reinforcement LearningTabular
🎯 What it does: A unified statistical inference framework for multi-armed bandits (MAB) with delayed feedback is proposed, designing a Delayed Adjustment Inverse Probability Weighting (DAIPW) estimator, and proving its consistency, asymptotic normality, and variance consistency, thus enabling the construction of effective confidence intervals.
Statistical Learning under Heterogenous Distribution Shift
Max Simchowitz (Massachusetts Institute of Technology), Akshay Krishnamurthy (Microsoft Research)
Domain AdaptationRobotic IntelligenceImageTabular
🎯 What it does: The theoretical analysis and experimental validation of the predictive performance of additive models f(x)+g(y) learned through empirical risk minimization (ERM) under heterogeneous distribution drift.
STEERING : Stein Information Directed Exploration for Model-Based Reinforcement Learning
Souradip Chakraborty (University of Maryland), Dinesh Manocha (University of Maryland)
Reinforcement Learning
🎯 What it does: This paper proposes a new model-based reinforcement learning algorithm called STEERING, which utilizes Stein information to guide exploration for efficient learning in sparse reward environments.
Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learning
Nicolas Castanet (Sorbonne Université), sylvain lamprier
Robotic IntelligenceReinforcement Learning
🎯 What it does: This paper studies a target generation method based on Stein Variational Gradient Descent (SVGG) for adaptive curriculum learning in multi-objective reinforcement learning (RL).
STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition
Yucheng Lu (Cornell University), Amir Yazdanbakhsh (Google DeepMind)
ClassificationOptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: The STEP algorithm is proposed, achieving zero-shot learning of N:M structured sparse masks under Adam training, and stabilizing variance estimation through an adaptive two-stage preconditioning to reduce accuracy degradation caused by sparse training.
Stochastic Gradient Descent under Markovian Sampling Schemes
Mathieu Even (Inria - ENS Paris)
OptimizationGraph
🎯 What it does: The study investigates stochastic gradient descent (SGD) under a Markov chain sampling scheme, proposing a distributed optimization based on random walks (token algorithm) and its improved version MC-SAG, analyzing their convergence rates and comparing them with traditional gossip-based decentralized algorithms.
Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network
Farhad Pashakhanloo (Cold Spring Harbor Laboratory), Alexei Koulakov (Cold Spring Harbor Laboratory)
OptimizationRepresentation LearningAuto EncoderStochastic Differential Equation
🎯 What it does: This study investigates the representation drift caused by Stochastic Gradient Descent (SGD) noise in a two-layer linear feedforward network, providing a theoretical explanation and quantification of the drift.
Stochastic Gradient Succeeds for Bandits
Jincheng Mei (Google Research), Dale Schuurmans (University of Alberta)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper studies and proves that in the multi-armed bandit (MAB) problem, the stochastic gradient bandit algorithm (softmax parameterization) with a constant learning rate can globally converge almost surely, and provides a convergence rate of O(1/t).
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels
Alexander Immer (ETH Zurich), Bernhard Schölkopf
OptimizationHyperparameter SearchConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: This paper derives a stochastic gradient estimator based on the Neural Tangent Kernel (NTK) and linearized Laplace approximation, allowing for unbiased gradient updates of model hyperparameters during training.
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies
Ilyas Fatkhullin (ETH Zurich), Niao He (ETH Zurich)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper proposes two single-loop, importance-sampling-free, random policy gradient algorithms that use only 1-2 trajectories—N-PG-IGT and (N)-HARPG—and proves that under Fisher-non-degenerate policies, global optimal sample complexities of ˜O(ε⁻²·⁵) and ˜O(ε⁻²) can be achieved;
Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
Minyoung Huh (Massachusetts Institute of Technology), Phillip Isola (Massachusetts Institute of Technology)
ClassificationGenerationOptimizationConvolutional Neural NetworkTransformerAuto EncoderImage
🎯 What it does: The optimization instability based on the straight-through estimator was studied in vector quantization networks, revealing that the deviation of model embeddings from the codebook distribution is the main cause of exponential collapse. This issue is alleviated through improved code vector parameterization, alternating optimization, and synchronous updates.
Strategic Classification with Unknown User Manipulations
Tosca Lechner (University of Waterloo), Shai Ben-David (Vector Institute)
ClassificationOptimization
🎯 What it does: This paper proposes a new batch learning framework for strategic classification problems, which estimates the manipulation structure (manipulation graph) using unlabeled samples from the previous round without prior knowledge of the individual feasible feature manipulation structure, and learns a classifier that is close to optimal based on this estimation.
Stratified Adversarial Robustness with Rejection
Jiefeng Chen (University of Wisconsin), Somesh Jha (University of Wisconsin)
ClassificationAdversarial AttackImage
🎯 What it does: This paper studies adversarial robust classification with rejection options, proposing the concepts of hierarchical rejection loss and total robust loss, and constructing the CPR (Consistent Prediction-based Rejection) method based on consistent prediction.
Streaming Active Learning with Deep Neural Networks
Akanksha Saran (Microsoft Research), Jordan T. Ash (Microsoft Research)
ClassificationData-Centric LearningConvolutional Neural NetworkImage
🎯 What it does: A new batch active learning algorithm called VeSSAL is proposed, which is suitable for the application of deep neural networks in streaming environments and can immediately query labels when encountering samples.
Streaming Submodular Maximization with Differential Privacy
Anamay Chaturvedi (Northeastern University), Thy Dinh Nguyen
OptimizationSafty and PrivacyTabular
🎯 What it does: An algorithm for optimizing submodular functions with differential privacy in streaming data is proposed, providing two implementations for general monotone submodular functions and decomposable submodular functions, using Laplace noise and Gumbel noise respectively; matching lower bound analysis is also provided.
StriderNet: A Graph Reinforcement Learning Approach to Optimize Atomic Structures on Rough Energy Landscapes
Vaibhav Bihani (Indian Institute of Technology Delhi), N M Anoop Krishnan
OptimizationGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: Proposes STRIDERNET - a reinforcement learning framework based on graph neural networks to optimize atomic structures in rough energy landscapes;
Structural Re-weighting Improves Graph Domain Adaptation
Shikun Liu (Georgia Institute of Technology), Pan Li (Purdue University)
Domain AdaptationGraph Neural NetworkGraph
🎯 What it does: A new method for Graph Domain Adaptation (GDA) called Structure Reweighting (StruRW) is proposed to alleviate performance degradation caused by Conditional Structure Shift (CSS).
Structure Learning of Latent Factors via Clique Search on Correlation Thresholded Graphs
Dale Kim, Qing Zhou (University of California)
Tabular
🎯 What it does: A fast algorithm based on threshold correlation graphs is proposed to find independent maximum cliques, aimed at simultaneously learning the number of latent factors, load structure, and addressing the issue of rotation indeterminacy.
Structure-informed Language Models Are Protein Designers
Zaixiang Zheng (ByteDance Research), Quanquan Gu (ByteDance Research)
GenerationProtein Structure PredictionTransformerLarge Language ModelBiomedical Data
🎯 What it does: Using structural surgery to transform a pre-trained protein language model (pLM) into a tool called LM-DESIGN that can directly generate high-quality protein sequences based on a given scaffold.
Structured Cooperative Learning with Graphical Model Priors
Shuangtong Li (University of Science and Technology of China), Dacheng Tao (University of Sydney)
ClassificationFederated LearningGraph Neural NetworkImage
🎯 What it does: A framework for training personalized models in a decentralized environment is proposed—Structured Cooperative Learning (SCooL), which automatically generates a cooperation graph through graphical model priors and collaboratively updates local models.
StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis
Axel Sauer (University of Tuebingen), Timo Aila (NVIDIA)
GenerationData SynthesisGenerative Adversarial NetworkImageText
🎯 What it does: The StyleGAN-T model is proposed, which achieves fast and high-quality large-scale text-to-image synthesis by modifying StyleGAN-XL; multiple improvements have been made in the generator, discriminator, and training process, supporting text alignment, controllable diversity, and super-resolution.
Subequivariant Graph Reinforcement Learning in 3D Environments
Runfa Chen (Tsinghua University), Wenbing Huang (Renmin University of China)
Robotic IntelligenceGraph Neural NetworkTransformerReinforcement LearningGraph
🎯 What it does: This paper proposes the Subequivariant Graph Reinforcement Learning in 3D Environments (3D-SGRL) framework, which first extends traditional 2D planar tasks to more realistic and challenging 3D motion environments, allowing agents to walk freely from any starting posture and target direction. It then designs the SubEquivariant Transformer (SET) network, embedding sub-equivariance constraints (O₍g₎(3) equivariance) into the graph transformer to ensure that the policy maintains consistency under gravitational constraints with respect to rotation and translation, and achieves high expressive message passing through self-attention.
Submodular Order Functions and Assortment Optimization
Rajan Udwani (University of California)
Recommendation SystemOptimization
🎯 What it does: This paper studies a new class of submodular order functions and provides approximation algorithms under various constraints.
Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation
Jin-Hong Du (Carnegie Mellon University), Arun K. Kuchibhotla
OptimizationHyperparameter SearchBiomedical Data
🎯 What it does: Under the proportional limit, a subsampling ridge regression ensemble method is studied, and it is proven that the optimal full ridge unregularized ensemble is risk-equivalent to the optimal ridge regression predictor. It is also demonstrated that the Generalized Cross-Validation (GCV) of the full ensemble is unbiasedly consistent with respect to the subsampling size, which can be used for hyperparameter tuning without sample partitioning.
Subset Selection Based On Multiple Rankings in the Presence of Bias: Effectiveness of Fairness Constraints for Multiwinner Voting Score Functions
Niclas Boehmer (TU Berlin), Nisheeth K Vishnoi
🎯 What it does: The study investigates how to utilize the multi-winner scoring function (submodular/multiple winner scoring function) for subset selection under biased multi-ranking inputs, and proves that it can mitigate bias when representative constraints are satisfied.
Subset-Based Instance Optimality in Private Estimation
Travis Dick (Google Research), Ananda Theertha Suresh (Google Research)
OptimizationSafty and PrivacyTabular
🎯 What it does: A subset-based definition of instance optimality is proposed, along with a private estimation algorithm that can achieve this definition, mainly targeting mean, quantiles, and a class of monotonic properties;
Superhuman Fairness
Omid Memarrast (University of Illinois Chicago), Brian D Ziebart
Reinforcement LearningTabular
🎯 What it does: Proposes the concept of 'superhuman fairness' and constructs fair machine learning models that more frequently outperform human decisions in various predictive performance and fairness metrics;
Supervised Metric Learning to Rank for Retrieval via Contextual Similarity Optimization
Christopher Liao (Boston University), Brian Kulis (Boston University)
RetrievalOptimizationContrastive LearningImage
🎯 What it does: This paper proposes a supervised metric learning loss optimized based on contextual similarity, which directly minimizes the contextual similarity between samples during training, thereby improving the ranking quality of image retrieval.
Supported Trust Region Optimization for Offline Reinforcement Learning
Yixiu Mao (Tsinghua University), Xiangyang Ji (Tsinghua University)
OptimizationReinforcement LearningTabularBenchmark
🎯 What it does: A Supportive Trust Region Optimization (STR) algorithm is proposed to address the out-of-domain action error problem in offline reinforcement learning.
SurCo: Learning Linear SURrogates for COmbinatorial Nonlinear Optimization Problems
Aaron M Ferber, Yuandong Tian (Meta AI)
OptimizationTabular
🎯 What it does: This study proposes the SurCo framework, which utilizes linear surrogate costs to solve combinatorial optimization problems with nonlinear objectives using existing combinatorial optimization solvers.
Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction
Yuan-Ting Hu (University of Illinois), Raymond A. Yeh (Purdue University)
OptimizationConvolutional Neural NetworkGraph Neural NetworkImageMesh
🎯 What it does: This paper proposes a Surface Snapping optimization layer that incorporates surface normals predicted from a single image into the 3D mesh reconstruction process, optimizing vertex positions in a differentiable manner to make the reconstructed shape more consistent with the observed normals.
SurProGenes: Survival Risk-Ordered Representation of Cancer Patients and Genes for the Identification of Prognostic Genes
Junetae Kim (National Cancer Center), Sun-Young Kim (National Cancer Center)
Recommendation SystemRepresentation LearningBiomedical Data
🎯 What it does: A framework based on collaborative filtering and survival risk ranking was developed to identify prognostic genes from cancer gene expression data and classify patients into high and low-risk groups.