NeurIPS 2024 Papers — Page 33
Conference on Neural Information Processing Systems · 4035 papers
SCube: Instant Large-Scale Scene Reconstruction using VoxSplats
Xuanchi Ren (NVIDIA), Jiahui Huang (NVIDIA)
GenerationData SynthesisAutonomous DrivingGenerative Adversarial NetworkGaussian SplattingPoint Cloud
🎯 What it does: In the case of given sparse and non-overlapping perspective images, the SCube method is proposed, which can generate high-resolution 3D representations of large-scale scenes (Voxel-Splat + sky map) in seconds, achieving high-quality novel view rendering, LiDAR simulation, and text-to-scene generation.
SDformer: Similarity-driven Discrete Transformer For Time Series Generation
Chen Zhicheng, Peilin Zhao (Institute of Microelectronics)
GenerationData SynthesisTransformerTime SeriesFinance Related
🎯 What it does: A time series generation framework SDformer based on discrete token modeling is proposed, which obtains high-quality discrete representations through similarity-driven vector quantization, and then utilizes autoregressive and masked Transformers for unconditional and conditional generation.
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training
Jinda Jia (Indiana University), Dingwen Tao (Indiana University)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes and implements SDP4Bit, which targets Sharded Data Parallelism for training large-scale LLMs, capable of compressing weight and gradient communication to about 4 bits while maintaining training accuracy.
SE(3)-bi-equivariant Transformers for Point Cloud Assembly
Ziming Wang (Chalmers University of Technology), Rebecka Jörnsten (Chalmers University of Technology)
TransformerPoint Cloud
🎯 What it does: A correspondence-free, SE(3)-bi-equivariant, scalable Transformer model BITR has been developed for point cloud assembly.
SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers
Parsa Esmati (University of Bristol), Nicolò Grilli (University of Bristol)
TransformerAuto EncoderMeshPhysics Related
🎯 What it does: A State-Exchange Attention (SEA) module based on Transformer is proposed, combined with a ViT grid autoencoder, to enhance the autoregressive inference accuracy of computational fluid dynamics (CFD) simulations and significantly reduce rolling errors.
Search for Efficient Large Language Models
Xuan Shen (Northeastern University), Yanzhi Wang (Northeastern University)
Computational EfficiencyNeural Architecture SearchTransformerLarge Language ModelText
🎯 What it does: A training-free architecture search framework is proposed, which can automatically mine efficient sub-networks from existing large language models and recalibrate weights through ADMM to enhance performance.
Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
Andres Potapczynski (New York University), Andrew Gordon Wilson (New York University)
OptimizationComputational EfficiencyTransformerMixture of ExpertsImageText
🎯 What it does: This study investigates a continuous structure matrix parameterization that can be expressed through Einstein summation, and searches for the optimal linear layer in terms of computational efficiency.
SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge
Chuanhao Li (OpenGVLab), Kaipeng Zhang (OpenGVLab)
TransformerLarge Language ModelVision Language ModelTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: A pluggable framework called SearchLVLMs is proposed, which utilizes internet search to provide up-to-date knowledge for existing large visual language models during inference.
Second-order forward-mode optimization of recurrent neural networks for neuroscience
Youjing Yu (Cambridge University), Guillaume Hennequin (Cambridge University)
OptimizationRecurrent Neural NetworkTime SeriesSequential
🎯 What it does: A training framework for RNNs based on second-order forward mode optimization (SOFO) is proposed and implemented, which can efficiently and low-memory train neural networks on long sequences.
Secret Collusion among AI Agents: Multi-Agent Deception via Steganography
Sumeet Ramesh Motwani (University of California Berkeley), Christian Schroeder de Witt (University of Oxford)
Anomaly DetectionSafty and PrivacyTransformerLarge Language ModelAgentic AIText
🎯 What it does: This paper studies the issue of 'secret collusion' among generative AI agents, proposing a formal threat model, an evaluation framework (CASE), and a series of experiments to measure the capabilities of large language models in covert communication through steganography, while exploring possible mitigation strategies.
SeeA*: Efficient Exploration-Enhanced A* Search by Selective Sampling
Dengwei Zhao (Shanghai Jiao Tong University), Lei Xu (Shanghai Jiao Tong University)
OptimizationReinforcement LearningTabular
🎯 What it does: Proposes SeeA* search, which enhances the exploratory nature of A* search by sampling candidate nodes from the OPEN list.
SeeClear: Semantic Distillation Enhances Pixel Condensation for Video Super-Resolution
Qi Tang (Beijing Jiaotong University), Chao Yao (University of Science and Technology Beijing)
RestorationSuper ResolutionDiffusion modelVideo
🎯 What it does: A Video Super-Resolution framework called SeeClear based on diffusion models has been developed, achieving high-quality super-resolution through semantic extraction and pixel compression.
Seeing Beyond the Crop: Using Language Priors for Out-of-Bounding Box Keypoint Prediction
Bavesh Balaji (University of Waterloo), David Anthony Clausi (University of Waterloo)
Object DetectionPose EstimationTransformerContrastive LearningImageMultimodality
🎯 What it does: In traditional human keypoint estimation, keypoints outside the bounding box (such as sports equipment) are often ignored. This paper proposes TokenCLIPose, which predicts missing keypoints by using only the bounding box of human keypoints and leveraging language priors, thus achieving keypoint detection that 'goes beyond the bounding box.'
Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment
Xin Xiao (Wuhan University), Haoyuan Guo (ByteDance Inc.)
Vision Language ModelContrastive LearningImageText
🎯 What it does: Reweighting text tokens during the training of visual language models to enhance image-text alignment.
Seek Commonality but Preserve Differences: Dissected Dynamics Modeling for Multi-modal Visual RL
Yangru Huang (Peking University), Yonghong Tian (Beihang University)
Autonomous DrivingReinforcement LearningImageMultimodality
🎯 What it does: This paper studies environmental dynamics modeling in multimodal visual reinforcement learning and proposes the Dissected Dynamics Modeling (DDM) method, which separates and models the consistent and inconsistent features between modalities to obtain a more refined and effective state representation.
SEEV: Synthesis with Efficient Exact Verification for ReLU Neural Barrier Functions
Hongchao Zhang (Washington University in St. Louis), Andrew Clark (Washington University in St. Louis)
OptimizationComputational Efficiency
🎯 What it does: This paper proposes the SEEV framework for efficient synthesis and verification of ReLU Neural Control Barrier Functions (NCBF), reducing the number of activation regions on the safety boundary through regularization, and designing efficient enumeration and verification algorithms.
Segment Any Change
Zhuo Zheng (Stanford University), Stefano Ermon (Stanford University)
SegmentationAnomaly DetectionTransformerImage
🎯 What it does: The Segment Any Change (AnyChange) model is proposed to achieve zero-shot change detection, capable of identifying any changes in remote sensing images at two time points in fully automated, semi-automated, and interactive modes, and can perform object-level change detection through point queries.
Segment Anything without Supervision
Xudong Wang, Trevor Darrell (University of California Berkeley)
SegmentationTransformerContrastive LearningImage
🎯 What it does: We propose UnSAM, a completely unsupervised image segmentation framework capable of achieving full-image segmentation and prompt segmentation; it trains the model using self-generated hierarchical pseudo-labels and achieves performance close to or even surpassing that of the supervised SAM.
Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations
Shivam Grover (Queen's University), Ali Etemad (Queen's University)
ClassificationAnomaly DetectionRepresentation LearningConvolutional Neural NetworkTransformerTime Series
🎯 What it does: A simple pluggable network layer S3 (Segment‑Shuffle‑Stitch) is proposed, which rearranges time steps by segmenting, learning-based shuffling, and stitching of time series, thereby enhancing the effectiveness of temporal representation learning.
Segmenting Watermarked Texts From Language Models
Xingchi Li (Texas A&M University), Xianyang Zhang (Texas A&M University)
SegmentationLarge Language ModelText
🎯 What it does: A statistical method based on randomization testing and change point detection is proposed for identifying and segmenting watermarked text;
SegVol: Universal and Interactive Volumetric Medical Image Segmentation
Yuxin Du (Shanghai Jiao Tong University), Bo Zhao (Shanghai Jiao Tong University)
SegmentationTransformerPrompt EngineeringImageBiomedical DataComputed Tomography
🎯 What it does: A foundational model for 3D medical image segmentation named SegVol has been trained and released, supporting various spatial and semantic prompts, capable of precise segmentation of over 200 anatomical categories.
SEL-BALD: Deep Bayesian Active Learning with Selective Labels
Ruijiang Gao (University of Texas at Dallas), Maytal Saar-Tsechansky (University of Texas at Austin)
Reinforcement LearningImageTabularFinance Related
🎯 What it does: This paper studies the active learning issue of human reviewers potentially rejecting labeled samples in high-risk decision-making scenarios (ALIR) and proposes a series of deep Bayesian active learning algorithms, SEL-BALD, to achieve efficient sample acquisition and labeling.
SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
Liangxin Liu (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: For the task of instruction tuning (Instruction Tuning) for LLMs, a new data selection method called SelectIT is proposed, which utilizes the model's own uncertainty to perform three layers of self-reflection, thereby filtering out high-quality instruction-response pairs and constructing the Selective Alpaca dataset based on this.
Selective Attention: Enhancing Transformer through Principled Context Control
Xuechen Zhang (University of Michigan), Samet Oymak (University of Michigan)
TransformerSupervised Fine-TuningText
🎯 What it does: A Selective Self-Attention (SSA) layer is introduced in the Transformer, achieving adaptive sparsification of the attention distribution through learnable temperature scaling, thereby separating semantic similarity from contextual sparsity and enhancing model performance.
Selective Explanations
Lucas Monteiro Paes (Harvard University), Flavio Calmon
Explainability and InterpretabilityTransformerSupervised Fine-TuningTextTabular
🎯 What it does: This paper proposes a selective explanation framework that first identifies which samples have low quality of model-inherent explanations (amortized) through uncertainty measurement, and then uses additional Monte Carlo inference on these samples to improve explanation quality.
Selective Generation for Controllable Language Models
Minjae Lee (POSTECH), Sangdon Park (POSTECH)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a controllable generation framework that controls the error rate of text generation in generative language models through selective generation.
Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection
Geng Yu (Shanghai Jiao Tong University), Bo Han (Hong Kong Baptist University)
Anomaly DetectionTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: Proposes the Self-Calibrated Tuning (SCT) framework, which enhances the OOD detection performance of VLM by adaptively adjusting the weights of ID classification and OOV regularization.
Self-Calibrating Conformal Prediction
Lars van der Laan (University of Washington), Ahmed Alaa
TabularBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes a Self-Calibrating Conformal Prediction (SC-CP) method that, while keeping the point predictions of the predictive model unchanged, combines Venn-Abers calibration with conformal prediction to obtain a predictive algorithm that satisfies both perfect calibration of point predictions and the validity of prediction intervals conditioned on those point predictions.
Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences
Damien Ferbach (Mila), Gauthier Gidel (Mila)
GenerationOptimizationReinforcement Learning from Human FeedbackConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: This study investigates how human curation of synthetic data affects model training in the self-consumption loop of generative models. It demonstrates improvements in expected rewards, variance convergence, and convergence to the reward maximization region in the absence of real data. The study also provides theoretical guarantees for KL convergence and reward enhancement when mixing real and curated synthetic data, and experimentally validates the phenomenon of bias amplification.
SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures
Pei Zhou (University of Southern California), Steven Zheng
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Construct the SELF-DISCOVER framework, allowing large language models to automatically select, rewrite, and combine a set of atomic reasoning modules into task-specific reasoning structures on unlabeled task instances, and then gradually reason according to this structure during decoding, significantly improving complex reasoning performance.
Self-Distilled Depth Refinement with Noisy Poisson Fusion
Jiaqi Li (Huazhong University of Science and Technology), Jianming Zhang (Adobe Research)
Depth EstimationKnowledge DistillationImage
🎯 What it does: This paper proposes the Self-Distilled Depth Refinement (SDDR) framework, treating depth refinement as a noise Poisson fusion problem with local inconsistent noise and edge distortion noise. It achieves high-resolution, detail-rich, and edge-clear depth maps through coarse-to-fine self-distillation to produce low-noise depth edge representations and edge-based guidance.
Self-Guided Masked Autoencoder
Jeongwoo Shin (Seoul National University), Joonseok Lee (Google Research)
Object DetectionSegmentationRepresentation LearningAuto EncoderImage
🎯 What it does: This paper provides an in-depth analysis of the Masked Autoencoder (MAE) and finds that it learns patch-level clustering based on visual patterns early in the pre-training phase. Based on this finding, a Self-Guided Masked Autoencoder (SG-MAE) is proposed, which generates informative masks autonomously during the pre-training process, replacing random masks to accelerate learning.
Self-Guiding Exploration for Combinatorial Problems
Zangir Iklassov (Mohamed bin Zayed University of Artificial Intelligence), Martin Takáč (Mohamed bin Zayed University of Artificial Intelligence)
OptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a new prompting strategy called Self-Guiding Exploration (SGE) for utilizing large language models (LLMs) to solve combinatorial optimization problems (CP) and other reasoning tasks.
Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments
Paulius Rauba (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
Large Language ModelPrompt EngineeringTabularChain-of-Thought
🎯 What it does: This paper proposes and implements a Self-Healing Machine Learning (SHML) framework that can automatically monitor model performance degradation, diagnose root causes, generate and evaluate corresponding adaptive measures, ultimately achieving automatic model repair.
Self-Labeling the Job Shop Scheduling Problem
Andrea Corsini (University of Modena and Reggio Emilia), Mauro Dell'Amico (University of Modena and Reggio Emilia)
OptimizationGraph Neural NetworkTabular
🎯 What it does: A self-supervised self-labeling improvement method (SLIM) is proposed for combinatorial optimization problems, and it is used to train a generation model based on Pointer Network to solve the Job Shop Scheduling Problem;
Self-Play Fine-tuning of Diffusion Models for Text-to-image Generation
Huizhuo Yuan (University of California), Quanquan Gu (University of California)
GenerationData SynthesisSupervised Fine-TuningDiffusion modelImageText
🎯 What it does: A self-play based diffusion model fine-tuning method (SPIN-Diffusion) is proposed, which enables iterative self-improvement for text-to-image generation using a dataset with only a single image/text pair.
Self-playing Adversarial Language Game Enhances LLM Reasoning
Pengyu Cheng (Tencent AI Lab), Xiaolong Li (Tencent AI Lab)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: By having the LLM first imitate GPT-4 to conduct Adversarial Taboo dialogues, and then continuously improving its reasoning ability through self-play using offline reinforcement learning.
Self-Refining Diffusion Samplers: Enabling Parallelization via Parareal Iterations
Nikil Roashan Selvam (Stanford University), Stefano Ermon (Stanford University)
GenerationData SynthesisComputational EfficiencyDiffusion modelImage
🎯 What it does: A self-refining diffusion sampler (SRDS) is implemented through Parareal iteration, reducing sampling latency through parallelization while maintaining sample quality.
Self-Retrieval: End-to-End Information Retrieval with One Large Language Model
Qiaoyu Tang (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Yongbin Li (Alibaba Group)
RetrievalTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes Self-Retrieval, an end-to-end information retrieval framework driven entirely by a single large language model, unifying indexing, retrieval, and re-ranking within the model parameters.
Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation
Ruize Zhang (Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences), Juan Cao (Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences)
ClassificationKnowledge DistillationAdversarial AttackConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A self-supervised adversarial training framework DAQ-SDP is proposed to address the issues of large robust generalization gaps and decreased clean accuracy in self-supervised adversarial training (SAT);
Self-Supervised Alignment with Mutual Information: Learning to Follow Principles without Preference Labels
Jan-Philipp Fränken (Stanford University), Noah Goodman
TransformerLarge Language ModelContrastive LearningText
🎯 What it does: A self-supervised alignment algorithm called SAMI is proposed, which utilizes mutual information maximization to enable language models to adhere to given behavioral principles (constitutions) without the need for preference labels or examples.
Self-supervised Transformation Learning for Equivariant Representations
Jaemyung Yu (Korea Advanced Institute of Science and Technology), Junmo Kim (Korea Advanced Institute of Science and Technology)
ClassificationObject DetectionRepresentation LearningAuto EncoderContrastive LearningImage
🎯 What it does: A self-supervised transformation learning (STL) method is proposed, which learns equivariant representations through the representation of original images and transformed image pairs, without the need for manual transformation labels, enhancing the generalization and detection performance of unsupervised representations.
Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models
Yuchen Hu (Nanyang Technological University), Chao Zhang (Tsinghua University)
RecognitionDomain AdaptationTransformerSupervised Fine-TuningAudio
🎯 What it does: This paper proposes an unsupervised source domain data adaptation framework called STAR, which utilizes large speech foundation models like Whisper to self-learn on unlabeled speech in the target domain, achieving domain adaptation.
SelfCodeAlign: Self-Alignment for Code Generation
Yuxiang Wei (University of Illinois Urbana-Champaign), LINGMING ZHANG
GenerationKnowledge DistillationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The SelfCodeAlign method is proposed, which completes the alignment and fine-tuning of code LLMs without manual annotation or teacher model distillation through self-generated instructions, responses, and tests.
SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data
Jialu Li (University of North Carolina at Chapel Hill), Mohit Bansal (University of North Carolina at Chapel Hill)
GenerationData SynthesisLarge Language ModelPrompt EngineeringDiffusion modelImageText
🎯 What it does: Utilizing large language models to automatically generate multi-skill text prompts, and using diffusion-based text-to-image models to generate corresponding images, creating an image-text dataset without manual annotation; subsequently, independently training LoRA experts for each skill and averaging all LoRA weights during inference to obtain a multi-skill unified model.
Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space
Xin Qiu (Cognizant AI Labs), Risto Miikkulainen (Cognizant AI Labs)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes and implements a confidence measurement method called 'Semantic Density' that is response-oriented, requires no additional training, and can be directly applied to any large language model.
Semantic Feature Learning for Universal Unsupervised Cross-Domain Retrieval
Lixu Wang (Northwestern University), Qi Zhu (Northwestern University)
RetrievalDomain AdaptationContrastive LearningImage
🎯 What it does: A two-stage unsupervised cross-domain retrieval framework named UEM is proposed to address the issue of inconsistent category spaces across different domains.
Semantic Routing via Autoregressive Modeling
Eric Zhao (University of California Berkeley and Google Research), Daniel Delling (Google)
TransformerContrastive LearningTextGraphBenchmark
🎯 What it does: A learning-based semantic routing system is proposed, with a publicly available benchmark of millions of natural language queries and real road networks, and a proof-of-concept model using an autoregressive Transformer is implemented.
Semantics and Spatiality of Emergent Communication
Rotem Ben Zion (Technion Israel Institute of Technology), Yonatan Belinkov (Technion Israel Institute of Technology)
Image
🎯 What it does: This paper theoretically analyzes and empirically demonstrates the impact of different objectives (reconstruction vs. discrimination) on the generation of semantically consistent and spatially meaningful communication protocols in a co-training environment of artificial intelligence.
SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning
Yangruibo Ding (Columbia University), Baishakhi Ray (Columbia University)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A 6.7B parameter code language model, SEMCODER, was trained, outperforming GPT-3.5-turbo and most open-source models in code generation and execution reasoning tasks.
SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow
Chaoyang Wang (Peking University), Ming-Hsuan Yang (University of California)
SegmentationGenerationData SynthesisConvolutional Neural NetworkDiffusion modelRectified FlowAuto EncoderImage
🎯 What it does: Proposes the SemFlow framework, treating semantic segmentation and semantic image synthesis as a bidirectional traffic problem, unifying the training of a single model to accomplish both tasks.
Semi-Open 3D Object Retrieval via Hierarchical Equilibrium on Hypergraph
Yang Xu (Tsinghua University), Yue Gao (Tsinghua University)
RetrievalGraph Neural NetworkAuto EncoderPoint Cloud
🎯 What it does: This paper proposes a 3D object retrieval framework HERT in a semi-open environment, utilizing hierarchical labels to generate multi-level embeddings and achieving structural balanced learning through hypergraph structures.
Semi-Random Matrix Completion via Flow-Based Adaptive Reweighting
Jonathan Kelner, Kevin Tian (University of Texas at Austin)
OptimizationFlow-based Model
🎯 What it does: A high-precision matrix completion algorithm has been designed that operates in nearly linear time and is robust to half-random observations, maintaining high accuracy even in the presence of noise.
Semi-supervised Knowledge Transfer Across Multi-omic Single-cell Data
Fan Zhang (Georgia Institute of Technology), Hongyu Zhao (Yale University)
Domain AdaptationSupervised Fine-TuningBiomedical Data
🎯 What it does: A semi-supervised cross-omics single-cell data transfer framework called DANCE is proposed to transfer cell type labels between scRNA-seq and scATAC-seq data, addressing the issue of label scarcity at both the source and target ends.
Semi-supervised Multi-label Learning with Balanced Binary Angular Margin Loss
Ximing Li (Jilin University), Fangming Gu (Jilin University)
ClassificationImage
🎯 What it does: A semi-supervised multi-label learning method named SML-BBAM is proposed, which improves the binary angle margin loss by balancing the variance of the angle distribution of positive and negative samples, thereby enhancing multi-label classification performance.
Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Data
Eyar Azar (Weizmann Institute of Science), Boaz Nadler (Weizmann Institute of Science)
ClassificationComputational EfficiencyTabular
🎯 What it does: This paper studies semi-supervised learning (SSL) in the context of high-dimensional sparse Gaussian mixture classification problems, proving that combining labeled and unlabeled samples can achieve computational advantages within certain parameter ranges.
Semidefinite Relaxations of the Gromov-Wasserstein Distance
Junyu Chen (National University of Singapore), Yong Sheng Soh (National University of Singapore)
OptimizationSupervised Fine-TuningPoint CloudMesh
🎯 What it does: A semidefinite programming (SDP) relaxation is proposed to solve the Gromov-Wasserstein (GW) distance, which can obtain globally optimal transport plans in most instances and provide a proof of global optimality.
Separate and Reconstruct: Asymmetric Encoder-Decoder for Speech Separation
Ui-Hyeop Shin (Sogang University), Hyung-Min Park (Sogang University)
Computational EfficiencyTransformerAudio
🎯 What it does: This paper proposes an asymmetric encoding-decoding structure based on early splitting (SepReformer), which achieves more efficient time-domain speech separation through a weight-shared decoder and cross-speaker Transformer.
Separation and Bias of Deep Equilibrium Models on Expressivity and Learning Dynamics
Zhoutong Wu (Peking University), Zhouchen Lin (Peking University)
🎯 What it does: This paper analyzes the expressive power and learning dynamics of Deep Equilibrium Models (DEQ) compared to traditional Feedforward Networks (FNN) from a theoretical perspective. It proposes two types of expressive separation theorems and provides an implicit regularization analysis of gradient flow in a simplified diagonal linear DEQ. A series of synthetic task experiments validate the advantages of DEQ in fitting high-frequency components and out-of-distribution (OOD) generalization.
Separations in the Representational Capabilities of Transformers and Recurrent Architectures
Satwik Bhattamishra (University of Oxford), Varun Kanade (University of Oxford)
Recurrent Neural NetworkTransformerSequential
🎯 What it does: This paper compares the representational capabilities of Transformer and Recurrent Neural Networks (RNN) through theoretical analysis and small-scale experiments across four types of natural related tasks (index lookup, finite Dyck language, string equality, nearest neighbor/association recall), and provides separation results between model size and input length.
Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Generation
Guillaume Huguet (Dreamfold), Joey Bose
GenerationData SynthesisProtein Structure PredictionTransformerLarge Language ModelReinforcement LearningFlow-based ModelBiomedical Data
🎯 What it does: A flow-matching based SE(3)-invariant model FOLDFLOW-2 was developed for conditional protein backbone generation, integrating large-scale protein language models with structural information.
Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity
Vahid Balazadeh (University of Toronto), Vasilis Syrgkanis (Stanford University)
Reinforcement LearningSequential
🎯 What it does: The study utilizes a framework for online sequential decision-making assisted by expert demonstrations under unobserved contextual heterogeneity and proposes the ExPerior algorithm.
Sequential Harmful Shift Detection Without Labels
Salim I. Amoukou (J.P. Morgan AI Research), Manuela Veloso (J.P. Morgan AI Research)
Anomaly DetectionTabularTime Series
🎯 What it does: This paper proposes a method for detecting harmful distribution shifts in production environments without relying on labels, using an error estimator to predict model errors and monitoring the proportion of high-error samples through threshold calibration.
Sequential Probability Assignment with Contexts: Minimax Regret, Contextual Shtarkov Sums, and Contextual Normalized Maximum Likelihood
Ziyi Liu (University of Toronto), Daniel M. Roy (University of Toronto)
🎯 What it does: The study investigates the problem of sequence probability distribution, particularly the minimax regret in a contextual environment, and proposes a new complexity measure - the contextual Shtarkov sum, deriving an optimal algorithm.
Sequential Signal Mixing Aggregation for Message Passing Graph Neural Networks
Mitchell Keren Taraday (Technion), Chaim Baskin (Ben-Gurion University of the Negev)
Graph Neural NetworkGraph
🎯 What it does: A new graph neural network aggregation module is proposed—Sequential Signal Mixing Aggregation (SSMA), which enhances the expressive power of MPGNN by treating neighbor features as two-dimensional discrete signals and using continuous convolution to mix these features.
SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization
Taisuke Yasuda (Voleon Group), Vahab Mirrokni (Google Research)
OptimizationConvolutional Neural NetworkSupervised Fine-TuningImageTabular
🎯 What it does: A new structured sparsification framework called SequentialAttention++ is proposed, along with a mathematical explanation that theoretically unifies differentiable pruning and combinatorial optimization methods. It is proven that under certain conditions, non-convex regularization can yield a unique group sparse global optimal solution. The framework is then implemented in practice, and experiments are conducted on the ImageNet and Criteo datasets.
Sequoia: Scalable and Robust Speculative Decoding
Zhuoming Chen (Carnegie Mellon University), Beidi Chen (Fair, Meta)
GenerationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: We propose SEQUOIA, a scalable and robust explicit sampling decoding framework that can significantly accelerate inference for large language models.
Set-based Neural Network Encoding Without Weight Tying
Bruno Andreis (KAIST), Sung Ju Hwang (KAIST)
TransformerImage
🎯 What it does: This paper proposes a set-based encoding method for neural network weights called SNE, which can predict network performance and attributes using only model parameters and can transfer across architectures and datasets.
SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation
Yixia Li (Southern University of Science and Technology), Yun Chen (Shanghai University of Finance and Economics)
Anomaly DetectionTransformerContrastive LearningImage
🎯 What it does: A training-agnostic method based on low-rank approximation, SeTAR, is proposed to improve the OOD detection of the CLIP model.
SF-V: Single Forward Video Generation Model
Zhixing Zhang (Snap Inc), Jian Ren (Snap Inc)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkVideo
🎯 What it does: This paper proposes a single-step video generation model SF-V, which achieves video generation with only one forward pass by adversarial fine-tuning of the pre-trained Stable Video Diffusion.
SfPUEL: Shape from Polarization under Unknown Environment Light
Youwei Lyu (Beijing University of Posts and Telecommunications), Boxin Shi (Peking University)
SegmentationDepth EstimationTransformerImage
🎯 What it does: This paper proposes the SfPUEL framework, which simultaneously estimates surface normals and segments metallic/dielectric materials using a single polarized image under unknown ambient light.
SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation
Hang Yin (Tsinghua University), Jiwen Lu (Tsinghua University)
Robotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision Language ModelPoint CloudChain-of-Thought
🎯 What it does: A zero-shot object navigation framework named SG-Nav has been designed and implemented, utilizing an online constructed hierarchical 3D scene graph and a large language model (LLM) for decision-making, and incorporating a graph re-perception mechanism to correct perception errors.
SGD vs GD: Rank Deficiency in Linear Networks
Aditya Varre (École polytechnique fédérale de Lausanne), Nicolas Flammarion (École polytechnique fédérale de Lausanne)
OptimizationTabularStochastic Differential Equation
🎯 What it does: This paper studies the different effects of continuous time gradient descent (GD) and stochastic gradient descent (SGD) on the rank of parameter matrices in two-layer linear networks, finding that SGD leads to a gradual decrease in rank, ultimately approaching a low-rank structure.
SGLang: Efficient Execution of Structured Language Model Programs
Lianmin Zheng (University of California Berkeley), Ying Sheng (Stanford University)
OptimizationComputational EfficiencyTransformerLarge Language ModelImageVideoTextRetrieval-Augmented Generation
🎯 What it does: This paper presents SGLang, a domain-specific language and runtime for efficient programming and execution of structured language model programs (LM Programs).
Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models
Yuancheng Xu (University of Maryland), Furong Huang (University of Illinois Urbana-Champaign)
Adversarial AttackData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes Shadowcast, an invisible data poisoning attack targeting visual language models, which induces misleading responses from the model using visually consistent text-image pairs.
Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
Alexander Tyurin (King Abdullah University of Science and Technology), Peter Richtárik (King Abdullah University of Science and Technology)
OptimizationTabular
🎯 What it does: A new distributed asynchronous stochastic gradient descent method called Shadowheart SGD is proposed, which can achieve convergence for non-convex optimization problems in scenarios with highly heterogeneous computation and communication times.
Shape analysis for time series
Thibaut Germain (Universite Paris Saclay), Laurent Oudre (Universite Paris Saclay)
ClassificationRepresentation LearningTime SeriesBiomedical Data
🎯 What it does: This paper proposes an unsupervised time series representation learning algorithm, TS-LDDMM, aimed at analyzing the variability of physiological functions among individuals, particularly in medical and biological contexts. The method achieves this by representing time series as deformations of a reference time series.
Shaping the distribution of neural responses with interneurons in a recurrent circuit model
David Lipshutz (Flatiron Institute), Eero P Simoncelli
Image
🎯 What it does: A feasible neural circuit model based on optimal transport theory is proposed, which achieves nonlinear mapping of input signals to target distributions (such as Gaussian distributions) through the collaborative adjustment of synapses, activation functions, and gain by plastic interneurons, thereby optimizing the neural response distribution.
Shared Autonomy with IDA: Interventional Diffusion Assistance
Brandon J McMahan, Jonathan Kao
Autonomous DrivingRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningDiffusion modelSequential
🎯 What it does: A dynamic intervention sharing autonomous framework based on value estimation (Interventional Diffusion Assistance, IDA) is proposed, enabling control strategies for human pilots and AI co-pilots to intervene with each other on all possible targets.
Sharing Key Semantics in Transformer Makes Efficient Image Restoration
Bin Ren (University of Pisa), Nicu Sebe (University of Trento)
RestorationSuper ResolutionTransformerImage
🎯 What it does: This paper proposes SemanIR, an image restoration framework that shares a key semantic dictionary within a Transformer, achieving efficient self-attention computation by focusing only on the most semantically related patches.
Sharpness-Aware Minimization Activates the Interactive Teaching's Understanding and Optimization
Mingwei Xu (Jilin University), Ivor Tsang
OptimizationImage
🎯 What it does: This study investigates interactive teaching based on co-teaching and proposes the incorporation of Sharpness-Aware Minimization (SAM) to achieve dual-layer interaction, enhancing the model's generalization performance on noisy data.
Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
Haiquan Lu (Nankai University), Yaoqing Yang (Dartmouth College)
ClassificationOptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: This study investigates the trade-off relationship between sharpness and diversity in deep ensembles and proposes the SharpBalance method, which utilizes an adaptively selected subset to balance both, thereby improving ID and OOD performance.
Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood
Rayen Dhahri (Technical University of Munich), Vincent Fortuin (Munich Center for Machine Learning)
CompressionOptimizationConvolutional Neural NetworkImage
🎯 What it does: The SpaM framework is constructed to enhance the compressibility of neural networks by maximizing Bayesian marginal likelihood during the training phase and using Laplace approximation, and a lightweight pruning criterion OPD is proposed that utilizes the computed posterior precision.
SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning
Yexiao He (University of Maryland), Ang Li (University of Maryland)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes SHED, an automated dataset refinement framework based on Shapley values for efficient instruction fine-tuning.
ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
Haoran You (Georgia Institute of Technology), Yingyan Celine Lin
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: By using post-training shift-and-add reparameterization, the multiplication in large language models is replaced with shifts and additions, resulting in an efficient ShiftAddLLM.
SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models
Zhaoyang Sun (Wuhan University of Technology), Yi Rong (Wuhan University of Technology)
Image TranslationGenerationDiffusion modelImage
🎯 What it does: A self-supervised hierarchical makeup transfer method (SHMT) based on latent diffusion models is proposed, achieving makeup style transfer by splitting and reconstructing content and makeup information from facial images.
Should We Really Edit Language Models? On the Evaluation of Edited Language Models
Qi Li (Hong Kong University of Science and Technology), Xiaowen Chu (Hong Kong University of Science and Technology)
TransformerLarge Language ModelText
🎯 What it does: This paper systematically evaluates the impact of various model editing methods on the overall capabilities of large language models (LLMs), including knowledge retrieval, reasoning, reading comprehension, and safety.
ShowMaker: Creating High-Fidelity 2D Human Video via Fine-Grained Diffusion Modeling
Quanwei Yang (University of Science and Technology of China), Hongtao Xie (University of Science and Technology of China)
GenerationData SynthesisPose EstimationDiffusion modelVideo
🎯 What it does: The ShowMaker framework is proposed, utilizing 2D keypoints to drive high-fidelity 2D conversational portrait video generation, and incorporating fine-grained modeling modules for hands and faces into a dual-stream diffusion model.
Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization
Quoc Tran-Dinh (University of North Carolina at Chapel Hill), Lam M. Nguyen (IBM Research)
OptimizationTabular
🎯 What it does: This paper addresses non-convex-linear and non-convex-strongly convex bi-objective problems, proposing a shuffling gradient-based solving algorithm and providing corresponding theoretical complexity analysis.
Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
Huy Nguyen, Alessandro Rinaldo (University of Texas)
Mixture of Experts
🎯 What it does: This paper analyzes through least squares estimation and theoretically proves that using the Sigmoid gate function in a mixture of experts model is more sample-efficient than using the Softmax gate function for expert parameter estimation.
SILENCE: Protecting privacy in offloaded speech understanding on resource-constrained devices
DONGQI CAI, Mengwei Xu (Beiyou Shenzhen Institute)
RecognitionSafty and PrivacyComputational EfficiencySupervised Fine-TuningAudio
🎯 What it does: A lightweight privacy-preserving speech understanding system called SILENCE is designed, which uses short-term masking to eliminate sensitive information while retaining long-term dependency semantics.
SimGen: Simulator-conditioned Driving Scene Generation
Yunsong Zhou (University of California), Bolei Zhou (University of California)
GenerationData SynthesisAutonomous DrivingDiffusion modelImageVideoTextMultimodality
🎯 What it does: This work proposes a simulator-conditioned diffusion model, SimGen, which can utilize a mix of real-world and simulation data for training, generating driving scenes with diverse appearances and layouts, and constructs a large-scale DIVA dataset that includes real YouTube videos and MetaDrive simulation videos.
Similarity-Navigated Conformal Prediction for Graph Neural Networks
Jianqing Song (Nanjing University), Chongjun Wang (Nanjing University)
SegmentationOptimizationGraph Neural NetworkGraph
🎯 What it does: An adaptive aggregation strategy based on feature similarity and structural neighbors (SNAPS) is proposed, which utilizes the inconsistency scores of same-label nodes to improve the efficiency of the prediction set in the segmentation conformal prediction of graph neural networks.
Simple and Effective Masked Diffusion Language Models
Subham Sekhar Sahoo (Cornell Tech), Volodymyr Kuleshov (Cornell Tech)
TransformerLarge Language ModelDiffusion modelTextBiomedical Data
🎯 What it does: This paper proposes and implements a Masked Discrete Diffusion Language Model (MDLM), which significantly improves the log-likelihood of diffusion models in language modeling tasks through a simplified variational objective and improved sampling methods.
Simple and Fast Distillation of Diffusion Models
Zhenyu Zhou (Zhejiang University), Siwei Lyu (University at Buffalo)
GenerationKnowledge DistillationDiffusion modelImage
🎯 What it does: A simple and efficient diffusion model distillation method (SFD) is proposed.
Simplified and Generalized Masked Diffusion for Discrete Data
Jiaxin Shi (Google DeepMind), Michalis Titsias
GenerationData SynthesisTransformerReinforcement LearningDiffusion modelImageText
🎯 What it does: A simplified and general mask diffusion model MD4 is proposed, and based on this, a state-dependent GenMD4 is introduced; a concise integral form of the continuous-time ELBO is provided, significantly simplifying the training and sampling process.
Simplifying Constraint Inference with Inverse Reinforcement Learning
Adriana Hugessen (Mila Université de Montréal), Glen Berseth (Mila Université de Montréal)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper studies how to simplify constraint inference through inverse reinforcement learning, reducing the traditional three-layer optimization structure of inverse constraint reinforcement learning to two layers, and achieving safe constraint learning based on this.
Simplifying Latent Dynamics with Softly State-Invariant World Models
Tankred Saanum (Max Planck Institute for Biological Cybernetics), Eric Schulz (Helmholtz Center Munich)
Robotic IntelligenceReinforcement LearningWorld ModelImage
🎯 What it does: A world model called Parsimonious Latent Space Model (PLSM) is proposed, which makes the impact of actions on latent states more predictable through information bottleneck;
SimPO: Simple Preference Optimization with a Reference-Free Reward
Yu Meng (University of Virginia), Danqi Chen (Princeton University)
Recommendation SystemOptimizationReinforcement LearningText
🎯 What it does: This paper proposes SimPO, a simple and efficient offline preference optimization algorithm;
Simulation-Free Training of Neural ODEs on Paired Data
Semin Kim (KAIST), Seunghoon Hong (KAIST)
ClassificationOptimizationFlow-based ModelAuto EncoderImageTabularOrdinary Differential Equation
🎯 What it does: A flow matching-based framework for non-simulated training is proposed to learn deterministic mappings on continuous depth models (NODE).