ICML 2025 Papers — Page 26
International Conference on Machine Learning · 3257 papers
Risk-Sensitive Theory of Mind: Coordinating with Agents of Unknown Bias using Cumulative Prospect Theory
Mason O. Smith (Arizona State University), Wenlong Zhang (Arizona State University)
Robotic IntelligenceReinforcement LearningAgentic AI
🎯 What it does: Developed and validated a Risk-Sensitive Theory of Mind (RS-ToM) framework that enables autonomous agents to infer and align with the risk preferences of unknown partners in real-time during collaboration, thereby enhancing team collaboration performance.
RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
Jonas Gehring (Meta), Gabriel Synnaeve (Meta)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Reinforcement learning training is applied to large language models (LLMs) to enable them to utilize execution feedback for multi-round iterative corrections in code generation tasks, significantly improving the solution rate.
RLTHF: Targeted Human Feedback for LLM Alignment
Yifei Xu (University of California), Ranveer Chandra (Microsoft)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: The RLTHF framework is proposed, which utilizes initial labels from LLM and targeted human feedback to iteratively train a reward model to achieve alignment effects equivalent to full human annotation.
Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
Haoyuan Cai (University of California), Bolei Zhou (University of California)
Autonomous DrivingRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningAgentic AITabular
🎯 What it does: A robot-gated interactive imitation learning algorithm called AIM has been designed and implemented, which can automatically learn when to request human demonstrations during the training process and gradually reduce human intervention as the agent's policy matures.
Robust and Conjugate Spatio-Temporal Gaussian Processes
William Laplante (University College London), Francois-Xavier Briol (University College London)
Anomaly DetectionOptimizationComputational EfficiencyTime SeriesFinance Related
🎯 What it does: This paper proposes a spatiotemporal model based on robust conjugate Gaussian processes, which has linear time complexity and is robust to outliers.
Robust Automatic Modulation Classification with Fuzzy Regularization
Xinyan Liang (Shanxi University), Liang Du (Shanxi University)
ClassificationConvolutional Neural NetworkRecurrent Neural NetworkSupervised Fine-TuningTime Series
🎯 What it does: An automatic modulation classification framework based on fuzzy regularization (FR) is proposed, aimed at suppressing the prediction ambiguity of the model under similar modulation categories.
Robust Autonomy Emerges from Self-Play
Marco Francis Cusumano-Towner (Apple), Vladlen Koltun (Apple)
Autonomous DrivingReinforcement LearningTabularBenchmark
🎯 What it does: A unified driving strategy has been trained through unprecedented self-play (1.6 billion km) in simulation, achieving natural and robust driving behavior across multiple vehicle types and driving styles without using any human driving data.
Robust Conformal Outlier Detection under Contaminated Reference Data
Meshi Bashari (Technion Israel Institute of Technology), Yaniv Romano (Technion Israel Institute of Technology)
Anomaly DetectionTabular
🎯 What it does: The study investigates the robustness of conformal prediction in anomaly detection tasks when there are a small number of outlier samples in the reference data, and proposes an active data cleaning method (Label-Trim) based on a limited labeling budget to improve detection rates.
Robust Consensus Anchor Learning for Efficient Multi-view Subspace Clustering
Yalan Qin (Shanghai University), Nicu Sebe (University of Trento)
OptimizationImage
🎯 What it does: A robust consensus anchor point learning framework (RCSC) is proposed for efficient multi-view subspace clustering, capable of simultaneously learning robust anchor points, consensus subspace representations, and clustering structures within a unified model.
Robust ML Auditing using Prior Knowledge
Jade Garcia Bourrée, Milos Vujasinovic (École Polytechnique Fédérale de Lausanne)
ImageTabular
🎯 What it does: A theoretical and experimental framework is proposed to improve the robustness of black-box fairness auditing using auditors' prior knowledge.
Robust Multi-Agent Reinforcement Learning with Stochastic Adversary
Ziyuan Zhou (Tongji University), Weiran Guo
Autonomous DrivingRecurrent Neural NetworkReinforcement LearningAgentic AISequential
🎯 What it does: Designed and implemented the ATSA framework, which conducts robust training of multi-agent reinforcement learning models through random opponents (SDor-STor), and experiments were conducted in StarCraft II and a mixed traffic autonomous driving environment.
Robust Multi-bit Text Watermark with LLM-based Paraphrasers
Xiaojun Xu (ByteDance Research), Hang Li (ByteDance Research)
Large Language ModelReinforcement LearningText
🎯 What it does: A multi-bit text watermarking method based on LLM is proposed, which injects watermarks at the sentence level using two differentiated LLM paraphrasers and decodes them with a text classifier.
Robust Multimodal Large Language Models Against Modality Conflict
Zongmeng Zhang (University of Science and Technology of China), Houqiang Li (University of Science and Technology of China)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringVision Language ModelTextMultimodality
🎯 What it does: This study investigates the hallucination phenomenon caused by modality conflicts in multimodal large language models (MLLMs) in visual-language tasks, constructs the MMMC dataset, and proposes three mitigation methods (prompt engineering, supervised fine-tuning, reinforcement learning).
Robust Noise Attenuation via Adaptive Pooling of Transformer Outputs
Greyson Brothers (Johns Hopkins University Applied Physics Laboratory)
ClassificationRecognitionRestorationTransformerReinforcement LearningImage
🎯 What it does: Research and improve the global pooling method of Transformer outputs, proposing Adaptive Pooling (AdaPool) to better extract useful signals in noisy environments.
Robust Offline Reinforcement Learning with Linearly Structured $f$-Divergence Regularization
Cheng Tang (University of Illinois Urbana-Champaign), Pan Xu (Duke University)
Reinforcement LearningTabularFinance Related
🎯 What it does: In the offline robust reinforcement learning task, a d-rectangular linear RRMDP framework is proposed, and the R2PVI algorithm is designed to achieve robustness against dynamic shifts through f-divergence regularization.
Robust Reward Alignment via Hypothesis Space Batch Cutting
Zhixian Xie (Arizona State University), Wanxin Jin (Shanghai Jiao Tong University)
Robotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: A robust reward alignment method based on hypothesis space batch pruning is proposed, which can still learn the target reward in the presence of erroneous human preferences.
Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability
Yunshu Dai (Sun Yat-sen University), Chip Hong Chang (Nanyang Technological University)
RecognitionImage TranslationSafty and PrivacyKnowledge DistillationGenerative Adversarial NetworkImage
🎯 What it does: Proposes the Secure Swap model, which combines the ID Passport layer to achieve person identity (POI) recognition and occlusion, and embeds an invisible watermark in the generated non-POI images for identity tracing and abuse prevention.
Robust Sparsification via Sensitivity
Chansophea Wathanak In (Nanyang Technological University), Xuan Wu (Nanyang Technological University)
OptimizationTabular
🎯 What it does: This paper proposes a general framework for constructing ε-coresets for robust optimization problems (such as the minimization problem of removing m extreme values), thereby achieving efficient sparsification of large-scale data.
Robust Spatio-Temporal Centralized Interaction for OOD Learning
Jiaming Ma (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
Graph Neural NetworkTime Series
🎯 What it does: A spatiotemporal graph neural network (STOP) based on centralized message passing and message perturbation mechanisms is proposed to address the spatiotemporal OOD learning problem.
RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Mingyuan Li (Beijing University of Posts and Telecommunications), Haipeng Peng (Beijing University of Posts and Telecommunications)
Autonomous DrivingOptimizationReinforcement LearningDiffusion modelTime Series
🎯 What it does: This paper proposes the RobustLight framework, which achieves online recovery from noise attacks and sensor failures by embedding Dynamic State Infill (DSI) and diffusion models into traffic signal control systems.
RobustZero: Enhancing MuZero Reinforcement Learning Robustness to State Perturbations
Yushuai Li (Aalborg University), TIANYI LI
Reinforcement LearningContrastive Learning
🎯 What it does: Designed and implemented RobustZero, which combines self-supervised contrastive learning and an adaptive weight adjustment mechanism to enhance the robustness of MuZero-like methods under worst-case and random-case state perturbations.
RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
Payman Behnam (Georgia Institute of Technology), Alexey Tumanov (NVIDIA)
CompressionComputational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes RocketKV, a two-stage KV cache compression method designed to accelerate the decoding inference of long-context LLMs.
Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
Vaishnavh Nagarajan (Google Research), Aditi Raghunathan (Carnegie Mellon University)
TransformerLarge Language ModelDiffusion modelText
🎯 What it does: A minimal algorithmic task was designed to quantify the creativity of language models in open-ended creative tasks;
RollingQ: Reviving the Cooperation Dynamics in Multimodal Transformer
HaoTian Ni, Di Hu (Renmin University of China)
RecognitionTransformerMultimodality
🎯 What it does: Investigate the reasons for the loss of dynamic collaboration in self-attention within multimodal Transformers, and propose the RollingQ method to balance attention distribution through query rotation, restoring multimodal collaborative dynamics.
ROME is Forged in Adversity: Robust Distilled Datasets via Information Bottleneck
Zheng Zhou (Beihang University), Guangliang Cheng (University of Liverpool)
Knowledge DistillationAdversarial AttackData-Centric LearningImage
🎯 What it does: This paper studies the adversarial robustness problem of Dataset Distillation (DD) and proposes a robust data distillation method called ROME based on the information bottleneck, which can generate synthetic datasets that are more robust to adversarial attacks without the need for adversarial training.
ROPO: Robust Preference Optimization for Large Language Models
Xize Liang (University of Science and Technology of China), Jieping Ye (Independent Researcher)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelText
🎯 What it does: This paper proposes a robust preference optimization framework called ROPO, designed to align large language models under noisy preference data.
ROS: A GNN-based Relax-Optimize-and-Sample Framework for Max-$k$-Cut Problems
Yeqing Qiu (Shenzhen Research Institute of Big Data), Zhi-Quan Luo (Shenzhen Research Institute of Big Data)
OptimizationGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: A ROS (Relax-Optimize-and-Sample) framework is proposed, which relaxes the discrete constraints of Maxk-Cut using a probabilistic simplex, employs GNN for unsupervised optimization in continuous space, and then maps the continuous solution back to a discrete solution through random sampling to obtain high-quality Maxk-Cut results.
RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
Quan Wei (University of Minnesota), Mingyi Hong (University of Minnesota)
OptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The RoSTE algorithm is proposed, achieving Quantization-Aware Supervised Fine-Tuning (QA-SFT), which enhances LLM fine-tuning performance while maintaining low-bit quantization.
rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
Xinyu Guan (Microsoft Research Asia), Mao Yang (Microsoft Research Asia)
TransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: The deep thinking framework rStar-Math, based on a small LLM, achieves high-level mathematical reasoning through self-evolution training strategies and reward models.
RuleAdapter: Dynamic Rules for training Safety Reward Models in RLHF
Xiaomin Li (Harvard University), Weiyu Li (Harvard University)
Safty and PrivacyReinforcement Learning from Human FeedbackReinforcement LearningBenchmark
🎯 What it does: A dynamic rule selection method was studied to label preference data for the RLHF safety reward model and train the RAMO reward model.
RULEBREAKERS: Challenging LLMs at the Crossroads between Formal Logic and Human-like Reasoning
Jason Chan (University of Sheffield), Zhixue Zhao (University of Sheffield)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper systematically evaluates the reasoning abilities of large language models in the context of logical rule violations (rulebreakers) and normal reasoning (non-rulebreakers) by constructing a dataset called RULEBREAKERS.
RUN: Reversible Unfolding Network for Concealed Object Segmentation
Chunming He (Duke University), Sina Farsiu (Duke University)
Object DetectionSegmentationImage
🎯 What it does: A reversible unfolding network (RUN) is proposed, achieving high-precision segmentation of concealed objects by applying reversible strategies simultaneously in the mask and RGB domains.
Runtime Analysis of Evolutionary NAS for Multiclass Classification
Zeqiong Lv (Sichuan University), Yanan Sun (Sichuan University)
ClassificationOptimizationNeural Architecture SearchBenchmark
🎯 What it does: For multi-class classification problems, a parsable benchmark MCC is proposed, and a corresponding mathematical fitness function is constructed. A two-layer search space (cell and block level) is designed, and then a strict upper bound O(rM ln(rM)) and lower bound Ω(rM ln M) for the expected running time of (1+1)-ENAS using two outer mutation methods, one-bit and bit-wise, is proven and experimentally validated.
RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization
XUCHEN, Dawei Yang (Houmo AI)
CompressionOptimizationComputational EfficiencyRecurrent Neural NetworkLarge Language ModelImageText
🎯 What it does: A post-training quantization framework called RWKVQuant for the RWKV model is proposed, which combines adaptive mixed scalar quantization and vector quantization, and optimizes the codebook for element-wise multiplication modules.
RZ-NAS: Enhancing LLM-guided Neural Architecture Search via Reflective Zero-Cost Strategy
Zipeng Ji (Nanjing University), Yihua Huang (Nanjing University)
Object DetectionNeural Architecture SearchTransformerLarge Language ModelPrompt EngineeringImage
🎯 What it does: A framework called RZ-NAS is proposed, which combines LLM reflection and zero-cost proxies to efficiently search neural network architectures in both micro and macro search spaces.
S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking
Tao Tang (Shenzhen Campus of Sun Yat-sen University), Xiaodan Liang (Sun Yat-sen University)
Object TrackingAutonomous DrivingTransformerPoint CloudBenchmark
🎯 What it does: A simple yet powerful end-to-end 3D multi-object tracking framework, S2-Track, has been designed and implemented.
S4S: Solving for a Fast Diffusion Model Solver
Eric Frankel (University of Washington), Sewoong Oh (University of Washington)
GenerationData SynthesisOptimizationKnowledge DistillationDiffusion modelImageOrdinary Differential Equation
🎯 What it does: This paper proposes the S4S method, which improves the sampling quality of diffusion models by learning the coefficients of ODE solvers under low NFE settings.
Sable: a Performant, Efficient and Scalable Sequence Model for MARL
Omayma Mahjoub (InstaDeep), Arnu Pretorius (InstaDeep)
TransformerReinforcement LearningSequential
🎯 What it does: The Sable algorithm is proposed, utilizing a modified Retention mechanism to achieve efficient sequence modeling in multi-agent reinforcement learning, balancing performance, memory, and scalability.
SADA: Stability-guided Adaptive Diffusion Acceleration
Ting Jiang (Duke University), Hai Li (Duke University)
GenerationComputational EfficiencyDiffusion modelImageTextMultimodalityOrdinary Differential Equation
🎯 What it does: This paper proposes a training-free, stability criterion-based adaptive acceleration framework SADA, designed to accelerate the sampling process of diffusion models (Diffusion and Flow-matching) in ODE form.
SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Hantao Lou (Peking University), Yaodong Yang (Peking University)
Explainability and InterpretabilityData-Centric LearningLarge Language ModelAuto EncoderImageTextMultimodality
🎯 What it does: Developed SAE-V, which extends Sparse Autoencoders (SAE) to Multimodal Large Language Models (MLLM), achieving interpretable analysis of cross-modal features within the model, and based on this, constructed a data filtering tool to enhance model alignment effectiveness.
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Adam Karvonen (Independent), Neel Nanda
Explainability and InterpretabilityAuto EncoderTextBenchmark
🎯 What it does: Proposed the SAEBench benchmark, which systematically evaluates the performance of over 200 sparse autoencoders (SAE) across multiple dimensions (interpretability, feature separation, reconstruction, unlearning, etc.).
SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders
Bartosz Cywiński (Warsaw University of Technology), Kamil Deja (IDEAS NCBR)
GenerationExplainability and InterpretabilityDiffusion modelAuto EncoderImageBenchmark
🎯 What it does: In the Diffusion model, sparse autoencoders (SAE) are used for unsupervised learning of internal activations, and during the inference phase, sparse features corresponding to the target concept are intercepted to achieve concept elimination.
Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets
Ning Lu (Southern University of Science and Technology), Ke Tang (Southern University of Science and Technology)
OptimizationSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes Safe Delta, a post-training defense method that is safety-aware for incremental parameters after fine-tuning LLMs, aimed at maintaining model safety and enhancing task performance.
Safe-EF: Error Feedback for Non-smooth Constrained Optimization
Rustem Islamov (University of Basel), Ilyas Fatkhullin (ETH Zürich)
OptimizationFederated LearningReinforcement LearningTabular
🎯 What it does: In federated learning, a new error feedback algorithm Safe-EF is designed and analyzed for non-smooth, constrained distributed optimization.
SAFE: Finding Sparse and Flat Minima to Improve Pruning
Dongyeop Lee (POSTECH), Namhoon Lee (POSTECH)
OptimizationConvolutional Neural NetworkTransformerLarge Language ModelImageText
🎯 What it does: A pruning framework called SAFE is proposed, which pursues flat minima while achieving sparsity, along with its extension SAFE+. This is realized through augmented Lagrangian and projection mechanisms.
SafeArena: Evaluating the Safety of Autonomous Web Agents
Ada Defne Tur (McGill University), Siva Reddy (Mila Quebec AI Institute)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelAgentic AITextMultimodalityBenchmark
🎯 What it does: Developed the SAFEARENA benchmark to evaluate the safety of LLM-driven web agents under five categories of malicious behavior (misleading, illegal, harassment, cybercrime, social bias) and compared the performance of five mainstream models.
SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models
Jiawei Zhang (University of Chicago), Bo Li (University of Chicago)
Autonomous DrivingSafty and PrivacyTransformerLarge Language ModelVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: This study proposes SafeAuto, a safety autonomous driving framework that integrates multimodal large language models, aiming to achieve end-to-end generation of high-level decision-making and low-level control simultaneously.
Safely Learning Optimal Auctions: A Testable Learning Framework for Mechanism Design
Vikram Kher (Yale University), Manolis Zampetakis (Yale University)
OptimizationTabular
🎯 What it does: A testable learning framework is proposed for optimal auction learning in mechanism design, aimed at addressing the issues related to strong distribution assumptions.
SafeMap: Robust HD Map Construction from Incomplete Observations
Xiaoshuai Hao (Beijing Academy of Artificial Intelligence), Shu Zhao
Object DetectionAutonomous DrivingKnowledge DistillationTransformerGaussian SplattingImage
🎯 What it does: This work proposes the SafeMap framework, which enables the construction of high-definition maps with high accuracy even in the absence of camera perspectives.
SAFER: A Calibrated Risk-Aware Multimodal Recommendation Model for Dynamic Treatment Regimes
Yishan Shen (University of Pennsylvania), Yong Chen (University of Pennsylvania)
Recommendation SystemTransformerMultimodalityBiomedical DataElectronic Health Records
🎯 What it does: We propose SAFER, a multimodal recommendation framework that achieves risk awareness in dynamic treatment plans;
Safety Alignment Can Be Not Superficial With Explicit Safety Signals
Jianwei Li (North Carolina State University), Jung-Eun Kim (North Carolina State University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: By adding a [CLS] token before the input of the LLM, combined with a policy attention and policy decoding mechanism, explicit safety signal learning and dynamic evaluation have been achieved, enhancing the model's safety when facing adversarial prompts.
Safety Certificate against Latent Variables with Partially Unidentifiable Dynamics
Haoming Jing (Carnegie Mellon University), Yorie Nakahira (Carnegie Mellon University)
Autonomous DrivingOptimizationReinforcement LearningTime Series
🎯 What it does: A method for designing probabilistic safety certificates for stochastic systems with latent variables and partially unidentifiable dynamics is proposed, which can continuously ensure safety probability during online control processes.
Safety Reasoning with Guidelines
Haoyu Wang (Tsinghua University), Minhao Cheng (Penn State University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Train LLMs to perform safety reasoning through structured safety guidelines, enhancing robustness against OOD evasion attacks.
Safety-Polarized and Prioritized Reinforcement Learning
Ke Fan (Tsinghua University), Jianzhu Ma (Tsinghua University)
Autonomous DrivingOptimizationReinforcement Learning
🎯 What it does: The MAXSAFE framework is proposed, which is based on opportunity-constrained bi-level optimization, achieving safety-first reinforcement learning through learning optimal action masking.
SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior
Jing-Jing Li (University of California), Sydney Levine (Allen Institute for Artificial Intelligence)
Safty and PrivacyExplainability and InterpretabilityKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: The SAFETYANALYST framework is designed to generate semi-structured harmful-beneficial trees using chain thinking, and employs an interpretable weighted aggregation model to obtain safety scores, achieving an interpretable, transparent, and adjustable safety review of AI behavior.
SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
Jintao Zhang (Tsinghua University), Jianfei Chen (Tsinghua University)
GenerationComputational EfficiencyTransformerLarge Language ModelImageVideoText
🎯 What it does: To address the quantization bottleneck in attention computation, SageAttention2 is proposed, utilizing INT4 quantization for Q and K, FP8 quantization for ˜P and V, and enhancing accuracy through Q and K smoothing, thread-level quantization, and a dual-layer accumulation method, significantly accelerating attention operations while maintaining unchanged end-to-end metrics.
SAH-Drive: A Scenario-Aware Hybrid Planner for Closed-Loop Vehicle Trajectory Generation
Yuqi Fan (Beihang University), Haiyang Yu (Beihang University)
Autonomous DrivingOptimizationDiffusion modelTime SeriesBenchmark
🎯 What it does: Proposes SAH-Drive, which integrates rule-based planning PDM-Closed with diffusion learning planning to form a context-aware hybrid trajectory planner;
SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation
Haoquan Fang (University of Washington), Jiafei Duan (University of Washington)
Robotic IntelligenceTransformerReinforcement LearningMultimodalityBenchmark
🎯 What it does: Proposed SAM2Act and its memory version SAM2Act+, a language-conditioned behavior cloning strategy that combines the large visual foundation model SAM2 with a multi-view robot Transformer, capable of achieving high-precision and generalizable robotic operations;
Sample Complexity of Branch-length Estimation by Maximum Likelihood
David Clancy Jr., Sebastien Roch (University of Wisconsin-Madison)
Optimization
🎯 What it does: This paper studies the maximum likelihood estimation problem of branch lengths under the evolutionary tree model, providing the strong concavity and smoothness of the empirical likelihood surface, and proving that the coordinate maximization algorithm can exponentially converge to the maximum likelihood estimator on high probability events, with an error of O(1/√m).
Sample Complexity of Correlation Detection in the Gaussian Wigner Model
Dong Huang (Tsinghua University), Pengkun Yang (Tsinghua University)
Graph Neural NetworkGraph
🎯 What it does: This paper studies the problem of detecting the correlation between two random graphs under the Gaussian Wig model, proposing a hypothesis testing method to determine whether the two graphs are independent or correlated through potential vertex permutations.
Sample Complexity of Distributionally Robust Off-Dynamics Reinforcement Learning with Online Interaction
Yiting He (Duke University), Pan Xu (Duke University)
Reinforcement Learning
🎯 What it does: In the robust Markov decision processes with finite state actions (CRMDP and RRMDP), an online learning algorithm ORBIT is proposed, which combines uncertainty sets or regularization based on f-divergence, and provides upper and lower bounds on exploration difficulty.
Sample Efficient Demonstration Selection for In-Context Learning
Kiran Purohit (Indian Institute of Technology), Avishek Anand (Delft University of Technology)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A sample-efficient demonstration subset selection algorithm named CASE is proposed for selecting the optimal example set in in-context learning (ICL) of large language models.
Sample-Optimal Agnostic Boosting with Unlabeled Data
Udaya Ghai (Amazon), Karan Singh (Carnegie Mellon University)
ClassificationComputational EfficiencyTabular
🎯 What it does: An algorithm is proposed that utilizes unlabeled samples to achieve sample-optimal agnostic boosting, which can maintain computational efficiency while allowing the sample complexity of weak learners to reach the same level as ERM.
Sample-specific Noise Injection for Diffusion-based Adversarial Purification
Yuhao Sun (University of Melbourne), Feng Liu (University of Melbourne)
Adversarial AttackDiffusion modelScore-based ModelImage
🎯 What it does: This paper proposes a score-adaptive noise injection framework based on diffusion, called SSNI, which can dynamically adjust the noise level for each input sample, thereby better removing adversarial noise while preserving image semantics.
Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification
Eric Zhao (Google Research), Sreenivas Gollapudi (Google Research)
TransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This paper studies a sampling search method that generates candidate answers using random sampling and performs self-validation through a language model for reasoning, demonstrating that this method significantly improves reasoning accuracy under large-scale inference computations.
Sampling Binary Data by Denoising through Score Functions
Francis Bach (Inria), Saeed Saremi (Genentech)
GenerationData SynthesisScore-based ModelTabularStochastic Differential Equation
🎯 What it does: This paper proposes a learning and sampling method for binary data distribution based on Bernoulli noise, and designs a discrete Langevin sampling algorithm.
Sampling from Binary Quadratic Distributions via Stochastic Localization
Chenguang Wang (Chinese University of Hong Kong), Tianshu Yu (Chinese University of Hong Kong)
OptimizationGraphBenchmarkStochastic Differential Equation
🎯 What it does: This paper proposes a general theoretical framework for applying Stochastic Localization (SL) on the discrete Binary Quadratic Distribution (BQD) and proves that after a sufficient number of iterations, the posterior distribution constructed by SL satisfies the Poincaré inequality, thereby ensuring polynomial mixing time for any discrete MCMC sampler (including Glauber, Metropolis-Hastings, gradient-enhanced sampling, etc.) on the posterior distribution.
SAN: Hypothesizing Long-Term Synaptic Development and Neural Engram Mechanism in Scalable Model's Parameter-Efficient Fine-Tuning
Gaole Dai (Peking University), Tiejun Huang (Peking University)
ClassificationRecognitionOptimizationTransformerSupervised Fine-TuningImageTextMultimodality
🎯 What it does: This paper proposes a Plug-and-Play mechanism called SAN (Synapse and Neuron), which enhances the expressive power of Parameter-Efficient Fine-Tuning (PEFT) by linearly decomposing the feature adjustment vector from the previous layer and explicitly propagating it to the weights of subsequent layers, without introducing additional trainable parameters.
SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer
Enze Xie (NVIDIA), Song Han (Massachusetts Institute of Technology)
GenerationCompressionComputational EfficiencyTransformerVision Language ModelDiffusion modelMultimodality
🎯 What it does: SANA-1.5 is proposed, which can efficiently scale the size and computational resource requirements of linear diffusion Transformers during training and inference.
SAND: One-Shot Feature Selection with Additive Noise Distortion
Pedram Pad (CSEM), Liza Andrea Dunbar
OptimizationTabular
🎯 What it does: A concise feature selection layer named SAND is proposed, which selects k most important features at once during the training process of neural networks through additive Gaussian noise and trainable gains.
Sanity Checking Causal Representation Learning on a Simple Real-World System
Juan L. Gamella (ETH Zurich), Jakob Runge
Representation LearningContrastive LearningImageBenchmarkPhysics Related
🎯 What it does: A controllable optical experiment (optical tunnel) was constructed as a real-world benchmark for testing causal representation learning methods.
Sassha: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
Dahun Shin (POSTECH), Namhoon Lee (POSTECH)
OptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: Designed and implemented SASSHA, an optimizer that enhances generalization performance through stable Hessian approximation and sharpness reduction mechanisms within a second-order optimization framework.
Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
Maohao Shen (Massachusetts Institute of Technology), Chuang Gan
TransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: By using small-scale format fine-tuning and large-scale reinforcement learning, the LLM internalizes Chain of Action Thinking (COAT) to achieve autoregressive search, significantly enhancing reasoning capabilities.
SBGD: Improving Graph Diffusion Generative Model via Stochastic Block Diffusion
Junwei Su (University of Hong Kong), shan Wu
GenerationData SynthesisComputational EfficiencyGraph Neural NetworkTransformerDiffusion modelGraph
🎯 What it does: This paper proposes a block graph decomposition-based diffusion graph generation model (SBGD), which utilizes block structure priors for diffusion in the block graph space, significantly reducing memory consumption and enhancing scale and size generalization.
Scaffold with Stochastic Gradients: New Analysis with Linear Speed-Up
Paul Mangold (Ecole Polytechnique), Eric Moulines (Ecole Polytechnique)
OptimizationFederated LearningTabular
🎯 What it does: A novel theoretical analysis of the SCAFFOLD algorithm under stochastic gradient updates is conducted, proving that it achieves linear acceleration in strongly convex smooth settings.
Scalable Approximation Algorithms for $p$-Wasserstein Distance and Its Variants
Nathaniel Lahn (Radford University), Pouyan Shirzadian (Virginia Tech)
OptimizationTabular
🎯 What it does: Proposed a scalable relative and additive approximation algorithm that uses multiple HSTs and a dynamic weighted bi-colored nearest point (BCP) data structure to compute the p-Wasserstein distance and its (p,k)-RPW variant;
Scalable Attribute-Missing Graph Clustering via Neighborhood Differentiation
Yaowenhu, Xinwang Liu (National University of Defense Technology)
Graph Neural NetworkGraph
🎯 What it does: A multi-view preprocessing method based on recursive neighborhood search and differential neighborhood strategy (CMV-ND) is proposed to address the clustering problem of large-scale attribute-missing graphs.
Scalable Equilibrium Sampling with Sequential Boltzmann Generators
Charlie B. Tan (University of Oxford), Alexander Tong (Mila - Quebec AI Institute)
GenerationData SynthesisDrug DiscoveryTransformerFlow-based ModelSequential
🎯 What it does: This paper proposes a scalable Boltzmann generator framework SBG, which achieves isothermal sampling of high-dimensional molecular states using reversible Transformer normalizing flows and non-equilibrium sampling.
Scalable First-order Method for Certifying Optimal k-Sparse GLMs
Jiachang Liu (Cornell University), Andrea Lodi (Cornell University)
OptimizationComputational EfficiencyTabular
🎯 What it does: This paper studies the optimality proof problem of sparse generalized linear models (GLMs) and proposes a method based on a first-order proximal gradient algorithm to address the perspective relaxation issue within the branch-and-bound (BnB) framework.
Scalable Gaussian Processes with Latent Kronecker Structure
Jihao Andreas Lin (Meta), Eytan Bakshy (Meta)
OptimizationComputational EfficiencyTabular
🎯 What it does: A Gaussian Process method utilizing the latent Kronecker structure is proposed, allowing for efficient and accurate inference even in the presence of missing observations.
Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching
Tinglin Huang (Yale University), Rex Ying (Yale University)
GenerationData SynthesisTransformerFlow-based ModelImage
🎯 What it does: We propose STFlow, a full-slide spatial transcriptomics generation model based on flow matching, which can directly predict gene expression from H&E assembled whole-slide images.
Scalable Meta-Learning via Mixed-Mode Differentiation
Iurii Kemaev (Google DeepMind), Hado van Hasselt (Google DeepMind)
OptimizationMeta LearningText
🎯 What it does: Proposes MixFlow-MG, which utilizes mixed-mode differentiation to achieve scalable gradient cascading bi-level optimization, significantly reducing memory and time overhead.
Scalable Model Merging with Progressive Layer-wise Distillation
Jing Xu (Tsinghua University), Jingzhao Zhang (Tsinghua University)
ClassificationGenerationOptimizationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningImageText
🎯 What it does: A scalable model merging algorithm called ProDistill is proposed, which achieves the fusion of multi-task models through layer-wise distillation.
Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignment
Yuhui Ding (ETH Zurich), Thomas Hofmann (ETH Zurich)
GenerationDrug DiscoveryGraph Neural NetworkTransformerDiffusion modelAuto EncoderGraph
🎯 What it does: By learning molecular adaptive rotation and training non-equivariant diffusion models in aligned latent space, efficient 3D molecular generation is achieved with quality comparable to equivariant models.
Scalable Private Partition Selection via Adaptive Weighting
Justin Y. Chen (Massachusetts Institute of Technology), Morteza Zadimoghaddam (Google Research)
Safty and PrivacyComputational EfficiencyGaussian SplattingTabularFinance Related
🎯 What it does: A private partition selection algorithm MAD and its two-round version MAD2R are proposed, which can be implemented in a large-scale parallel computing framework, utilizing adaptive weight redistribution to enhance the output probability of low-frequency items.
Scalable Sobolev IPM for Probability Measures on a Graph
Tam Le (Institute of Statistical Mathematics), Kenji Fukumizu (Institute of Statistical Mathematics)
OptimizationComputational EfficiencyGraph Neural NetworkTextGraph
🎯 What it does: This study investigates the Sobolev IPM problem supported on graph metric spaces and proposes a new regularization method to improve the computational efficiency of Sobolev IPM.
Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
Shikai Qiu (New York University), Atish Agarwala (Google DeepMind)
Tabular
🎯 What it does: This paper studies the phenomenon of a highly consistent collapse in the training loss curves of neural networks of different scales under compute-optimal training conditions after normalization, and further finds that learning rate decay can cause the collapse to exceed the noise level of a single model (supercollapse).
Scaling Inference-Efficient Language Models
Song Bian (University of Wisconsin Madison), Shivaram Venkataraman (University of Wisconsin Madison)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a set of language model expansion laws that consider inference latency, and based on this, trains the Morph-1B model with higher inference efficiency.
Scaling Large Motion Models with Million-Level Human Motions
Ye Wang (Renmin University of China), Zongqing Lu (Peking University)
GenerationData SynthesisPose EstimationTransformerLarge Language ModelReinforcement LearningVideoText
🎯 What it does: The first million-level human action dataset, MotionLib, has been constructed, and a large-scale text-to-action model, Being-M0, has been trained using this dataset. Additionally, the action encoding scheme MotionBook has been proposed, which includes lossless SMPL-D135 features and a 2D-LFQ tokenizer.
Scaling Laws for Differentially Private Language Models
Ryan McKenna (Google Research), Chiyuan Zhang (Google DeepMind)
OptimizationSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: This paper systematically experiments and constructs a compute-privacy-utility relationship model for the scalability of training large language models under differential privacy (DP) conditions.
Scaling Laws for Floating–Point Quantization Training
Xingwu Sun (Tencent), Jie Jiang (Tencent)
TransformerLarge Language Model
🎯 What it does: This paper studies the scaling laws of floating-point low-precision training and proposes a unified Capybara scaling law to predict the performance of LLMs under different exponent, mantissa, and block size settings.
Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
Louis Béthune, Pierre Ablin (Apple)
Supervised Fine-TuningText
🎯 What it does: This paper studies how to quantify the phenomena of forgetting and overfitting during the fine-tuning process by injecting pre-training data, and proposes a corresponding scaling law.
Scaling Laws for Pre-training Agents and World Models
Tim Pearce (Microsoft Research), Katja Hofmann (Microsoft Research)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelWorld ModelVideo
🎯 What it does: This paper studies the power-law relationship between the loss curves of world modeling (WM) and behavior cloning (BC) tasks and the optimal scale in large-scale pre-training, and experimentally verifies the scaling laws similar to those of large language models.
Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Stream
Abdulkadir Gokce (École Polytechnique Fédérale de Lausanne), Martin Schrimpf (École Polytechnique Fédérale de Lausanne)
Convolutional Neural NetworkTransformerImage
🎯 What it does: Systematically train artificial neural networks of different architectures and data scales, evaluating their alignment with the primate ventral visual pathway (V1–IT) and behavior;
Scaling Laws for Upcycling Mixture-of-Experts Language Models
Seng Pei Liew (SB Intuitions), Sho Takase (SB Intuitions)
TransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: The study transfers (upcycles) a pre-trained dense language model (dense LLM) to a sparse mixture of experts (MoE) model and explores the performance scaling laws under different data volumes and model sizes.
Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More
Feng Wang (Johns Hopkins University), Cihang Xie (University of California Santa Cruz)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: This paper systematically investigates the impact of reducing patch compression rates (from 16×16 to 1×1) on performance in visual models such as Vision Transformer and Mamba, proposing patch size scaling as a new dimension of scaling for visual models.
Scaling Probabilistic Circuits via Monarch Matrices
Honghua Zhang (University of California), Guy Van den Broeck (University of California)
Computational EfficiencyImageText
🎯 What it does: In the addition block of probabilistic circuits (PC), a sparse and structured Monarch matrix replaces the dense matrix, significantly reducing memory and computational costs, allowing the PC to scale to larger hidden sizes.
Scaling Sparse Feature Circuits For Studying In-Context Learning
Dmitrii Kharlapenko (ETH Zurich), Neel Nanda (Georgia Institute of Technology)
Explainability and InterpretabilityTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Using Sparse Autoencoders (SAE) and an improved Sparse Feature Circuit (SFC) method, this paper provides a mechanistic explanation of the Gemma 1-2B model, identifying and validating the task detection features and task execution features that play a core role in In-Context Learning (ICL).
Scaling Test-Time Compute Without Verification or RL is Suboptimal
Amrith Setlur (Carnegie Mellon University), Aviral Kumar (Carnegie Mellon University)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: This paper explores why LLMs exhibit suboptimal performance during the inference phase when computational resources can be scaled without using validation or reinforcement learning, providing both theoretical and experimental analysis.
Scaling Trends in Language Model Robustness
Nikolaus H. R. Howe (Mila - Quebec AI Institute), Adam Gleave (FAR.AI)
ClassificationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This paper conducts systematic experiments on the robustness of large-scale language models, exploring the trends in model scale, attack, and defense computational power, and constructs an attack-defense balance analysis framework.