ICLR 2024 Papers — Page 19
International Conference on Learning Representations · 2260 papers
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
Ming Zhong (University of Illinois Urbana-Champaign), Pengcheng He (Microsoft Azure AI)
Knowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a method for parameter-level knowledge transfer from a large language model (teacher) to a small language model (student), first extracting task-relevant parameters through sensitivity measurement, and then injecting these parameters into the small model using LoRA low-rank decomposition and fine-tuning.
Seer: Language Instructed Video Prediction with Latent Diffusion Models
Xianfan Gu (Tsinghua University), Yang Gao (Tsinghua University)
GenerationData SynthesisTransformerDiffusion modelVideoText
🎯 What it does: This paper presents Seer, a text-conditioned video prediction model that expands along the time axis on Stable Diffusion. It utilizes the Frame Sequential Text Decomposer to break down full sentences into frame-specific sub-instructions and incorporates efficient spatio-temporal attention in a 3D U-Net to achieve high-quality predictions.
SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases
Yang Liu (Hong Kong University of Science and Technology), Yu Rong (Hong Kong University of Science and Technology)
Graph Neural NetworkGraphPhysics RelatedOrdinary Differential Equation
🎯 What it does: Proposes the SEGNO framework, which combines second-order physical priors and equivariant graph neural networks to learn continuous trajectories for predicting the states of multi-body systems.
SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction
Xinyuan Chen (Shanghai Artificial Intelligence Laboratory), Ziwei Liu (Nanyang Technological University)
GenerationData SynthesisDiffusion modelVideoText
🎯 What it does: A diffusion model called SEINE is proposed for generating smooth and creatively transitional videos from short videos, which allows for controlling the transition content and style through text descriptions, and can also be used for tasks such as image animation and long video prediction.
Select to Perfect: Imitating desired behavior from large multi-agent data
Tim Franzmeyer (University of Oxford), Joao F. Henriques (University of Oxford)
Robotic IntelligenceReinforcement LearningTabular
🎯 What it does: This paper proposes measuring the contribution of a single agent to the overall expected score in a multi-agent dataset through 'Exchange Value' and designs the EV2BC (Exchange Value-based Behavior Cloning) method based on this to achieve selective imitation of behaviors that meet expected characteristics.
Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives
Shrinivas Ramasubramanian (Fujitsu Research of India), Venkatesh Babu Radhakrishnan
OptimizationSupervised Fine-TuningImage
🎯 What it does: We propose SelMix, a fine-tuning framework based on selective Mixup, which optimizes non-decomposable objectives (such as minimum recall, mean recall, G-mean, H-mean, coverage constraints, etc.) using pre-trained models, with low computational cost.
Selective Visual Representations Improve Convergence and Generalization for Embodied AI
Ainaz Eftekhar (University of Washington), Ranjay Krishna (University of Washington)
Robotic IntelligenceReinforcement LearningImage
🎯 What it does: This paper designs and trains a task-conditioned learnable codebook module to filter visual representations in Embodied AI, thereby reducing interference from irrelevant information.
Self-Alignment with Instruction Backtranslation
Xian Li (Meta), Mike Lewis (Meta)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes a scalable self-alignment method—instruction backtranslation—by allowing the language model to generate and filter instruction-answer pairs on its own, utilizing a vast amount of unlabeled web text to train instruction-following models.
Self-Consuming Generative Models Go MAD
Sina Alemohammad (Rice University), Richard Baraniuk
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper systematically studies the behavior of generative models in self-consumption (self-exhaustive) training and demonstrates that if each generation lacks sufficient fresh real data, subsequent models will experience a decline in quality or a decrease in diversity, referred to as Model Autophagy Disorder (MAD).
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Niels Mündler (ETH Zurich), Martin Vechev (ETH Zurich)
GenerationAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper studies the contradictory hallucinations that occur when large language models generate text and proposes a complete framework from triggering to detection to mitigation.
Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning
Johnathan Wenjia Xie (Stanford University), Chelsea Finn (Stanford University)
Domain AdaptationRepresentation LearningDrug DiscoveryProtein Structure PredictionTransformerAuto EncoderImageTextBiomedical DataPhysics Related
🎯 What it does: A domain-agnostic self-supervised learning framework called Self-Guided Masked Autoencoders (SMA) is proposed, which adaptively generates masks on the attention maps of the Transformer to directly reconstruct masked versions of the original input.
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Akari Asai (University of Washington), Hannaneh Hajishirzi (IBM Research AI)
GenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Train a language model called SELF-RAG that can decide during the inference process whether to retrieve documents, evaluate the relevance and support of the retrieval results, and self-critique the generated content.
Self-Supervised Contrastive Learning for Long-term Forecasting
Junwoo Park (KAIST), Edward Choi (KAIST)
Contrastive LearningTime Series
🎯 What it does: This paper proposes a contrastive learning loss called AutoCon based on global autocorrelation, combined with an improved decomposition architecture, to achieve self-supervised learning for long-period variations outside the sliding window, thereby enhancing long-period time series prediction performance.
Self-Supervised Dataset Distillation for Transfer Learning
Dong Bok Lee (KAIST), Sung Ju Hwang (KAIST)
Data SynthesisKnowledge DistillationContrastive LearningImage
🎯 What it does: Compress the unlabeled dataset into a small number of synthetic samples, and use these samples for self-supervised pre-training before transferring to various downstream tasks.
Self-Supervised Heterogeneous Graph Learning: a Homophily and Heterogeneity View
Yujie Mo (University of Electronic Science and Technology of China), Xiaofeng Zhu (University of Electronic Science and Technology of China)
ClassificationRetrievalRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: The HERO framework is proposed, which does not rely on predefined meta-paths in self-supervised heterogeneous graph learning, jointly learning homogeneous representations of similar nodes and heterogeneous representations of different types of nodes.
Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes
Zhilu Zhang (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
RestorationConvolutional Neural NetworkOptical FlowImage
🎯 What it does: SelfHDR proposes a fully self-supervised HDR reconstruction method that utilizes color and structural information from dynamic multi-exposure images, allowing training without HDR ground truth;
Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment
Bowen Gao (Institute for AI Industry Research, Tsinghua University), Yanyan Lan (Institute for AI Industry Research, Tsinghua University)
Drug DiscoveryGraph Neural NetworkTransformerContrastive LearningBiomedical Data
🎯 What it does: A self-supervised pre-training framework called ProFSA is proposed, based on the alignment of protein fragments and their surrounding environments, to learn high-quality binding site representations.
Self-supervised Representation Learning from Random Data Projectors
Yi Sui (Layer 6 AI), Maksims Volkovs (Layer 6 AI)
Representation LearningContrastive LearningImageMultimodalityTabularTime SeriesBiomedical DataElectronic Health Records
🎯 What it does: A self-supervised representation learning method called LFR (Learning from Randomness) is proposed, which learns general features from unlabeled data by having the encoder simultaneously predict the outputs of multiple random projection functions.
Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
Szu-Wei Fu (NVIDIA), Yu-Chiang Frank Wang (NVIDIA)
RestorationKnowledge DistillationTransformerGenerative Adversarial NetworkContrastive LearningAudio
🎯 What it does: An unsupervised speech quality estimation method called VQScore is proposed, which is trained solely on clean speech, and an unsupervised speech enhancement model is implemented within the same framework.
SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning
Ning Miao (University of Oxford), Tom Rainforth (University of Oxford)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes SelfCheck, a zero-shot, step-by-step self-checking framework based on LLMs, aimed at identifying errors in its own reasoning chain and improving answer quality.
Semantic Flow: Learning Semantic Fields of Dynamic Scenes from Monocular Videos
Fengrui Tian (Xi'an Jiaotong University), Shaoyi Du (Xi'an Jiaotong University)
Object TrackingSegmentationNeural Radiance FieldOptical FlowVideo
🎯 What it does: Utilize optical flow features to learn semantic representations of dynamic scenes, and supervise semantics using volume density as an opacity prior;
SemiReward: A General Reward Model for Semi-supervised Learning
Siyuan Li (Westlake University), Stan Z. Li (Westlake University)
ClassificationData-Centric LearningGenerative Adversarial NetworkImageTextMultimodalityAudio
🎯 What it does: A general semi-supervised learning framework called SemiReward is proposed to evaluate and filter high-quality pseudo-labels, enhancing the performance and convergence speed of self-training models.
Sentence-level Prompts Benefit Composed Image Retrieval
Yang bai, Chun-Mei Feng (Institute of High Performance Computing)
RetrievalTransformerPrompt EngineeringVision Language ModelContrastive LearningImageText
🎯 What it does: This paper proposes the use of dynamic sentence-level prompts to enhance the retrieval performance of queries composed of reference images and relative descriptions (CIR), replacing traditional late fusion and pseudo-word methods.
Separate and Diffuse: Using a Pretrained Diffusion Model for Better Source Separation
Shahar Lutati (Tel Aviv University), Lior Wolf (Tel Aviv University)
GenerationConvolutional Neural NetworkRecurrent Neural NetworkDiffusion modelAudio
🎯 What it does: This paper proposes a method that combines pre-trained diffusion models with traditional source separation models, enhancing speech separation quality by learning linear mixing weights in the frequency domain.
Separating common from salient patterns with Contrastive Representation Learning
Robin Louiset (NeuroSpin), Pietro Gori (Telecom Paris)
Representation LearningContrastive LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a contrastive analysis framework called SepCLR based on the principle of information maximization (InfoMax), which can separate common features (c) and target-specific features (s) in two datasets (background and target) and achieve high-quality representations through contrastive learning.
SEPT: Towards Efficient Scene Representation Learning for Motion Prediction
Zhiqian Lan (Tsinghua University), Shengbo Eben Li (Tsinghua University)
Autonomous DrivingRepresentation LearningTransformerTime Series
🎯 What it does: This paper proposes the Scene Encoding Predictive Transformer (SEPT), which enhances the spatiotemporal understanding of traffic scenes through self-supervised pre-training, thereby improving motion prediction.
SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking
Chris Cundy (Stanford University), Stefano Ermon (Stanford University)
GenerationTransformerReinforcement LearningTextSequential
🎯 What it does: Treating the autoregressive sequence generation problem as an imitation learning task, the SequenceMatch method is proposed, which trains the model by minimizing various divergences of occupancy metrics (such as χ² divergence) and incorporates backspace actions to alleviate cumulative errors.
Set Learning for Accurate and Calibrated Models
Lukas Muttenthaler (Technische Universität Berlin), Klaus Robert Muller
ClassificationImage
🎯 What it does: A training framework based on ensemble learning called Odd Out (OKO) is proposed to improve the accuracy and calibration of classification models.
SetCSE: Set Operations using Contrastive Learning of Sentence Embeddings
Kang Liu (Independent Researcher)
RetrievalContrastive LearningTextFinance Related
🎯 What it does: Proposes the SetCSE framework, which utilizes set representation for complex semantics and implements diverse information retrieval through operations such as intersection and difference.
SF(DA)$^2$: Source-free Domain Adaptation Through the Lens of Data Augmentation
Uiwon Hwang (Yonsei University), Sungroh Yoon (Seoul National University)
Domain AdaptationImagePoint Cloud
🎯 What it does: A source-free domain adaptation method SF(DA)2 is proposed from the perspective of data augmentation, which constructs an enhanced graph using the feature space of a pre-trained model, and achieves adaptation to the target domain through spectral neighborhood clustering, implicit feature augmentation, and feature disentanglement.
SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem
Margalit Glasgow (Stanford University)
OptimizationTabular
🎯 What it does: It is proven that standard two-layer ReLU neural networks can learn the XOR function on the Boolean hypercube with approximately optimal sample complexity (~O(d polylog d)) when using stochastic gradient descent (SGD).
Shadow Cones: A Generalized Framework for Partial Order Embeddings
Tao Yu (Cornell University), Christopher De Sa (Cornell University)
Contrastive LearningGraph
🎯 What it does: The Shadow Cones framework is proposed, viewing the partial order relation as a shadow subset formed by the projection of a light source, overcoming the ε-hole problem of the Poincaré ball model.
Sharpness-Aware Data Poisoning Attack
Pengfei He (Michigan State University), Jiliang Tang (IBM T. J. Watson Research Center)
ClassificationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Proposes the Sharpness-Aware Data Poisoning Attack (SAPA), which approximates the attack effect of the worst retrained model by utilizing the sharpness of the DNN loss landscape during the generation of poisoning samples;
Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning
Jacob Mitchell Springer (Carnegie Mellon University), Aditi Raghunathan (Carnegie Mellon University)
Domain AdaptationOptimizationSupervised Fine-TuningImage
🎯 What it does: This paper studies Sharpness-Aware Minimization (SAM) and its ability to automatically balance and enhance the quality of multiple redundant features during training, thereby improving the model's performance in out-of-distribution (OOD) and transfer learning scenarios.
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
Mengzhou Xia (Princeton University), Danqi Chen (Princeton University)
TransformerLarge Language ModelText
🎯 What it does: Smaller models (1.3B, 2.7B) are obtained from the large pre-trained model (LLaMA2-7B) through structured pruning, and further pre-training is conducted based on this.
Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation
Ryan Wong (University of Surrey), Richard Bowden (University of Surrey)
Image TranslationGenerationTransformerLarge Language ModelVision Language ModelVideoText
🎯 What it does: The Sign2GPT framework is proposed, utilizing frozen large-scale visual models and language models (ViT + GPT) to achieve gloss-free sign language translation through lightweight LoRA adaptation.
SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore
Sewon Min (University of Washington), Luke Zettlemoyer (University of Washington)
RetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Designed and evaluated a language model SILO that separates parametric and non-parametric components to reduce legal risks associated with training data.
Simple Hierarchical Planning with Diffusion
Chang Chen (Rutgers University), Sungjin Ahn (KAIST)
Robotic IntelligenceReinforcement LearningDiffusion modelTabular
🎯 What it does: Proposes a hierarchical Diffuser framework that combines diffusion models to achieve high-level subgoal jumping planning and low-level refinement planning;
Simple Minimax Optimal Byzantine Robust Algorithm for Nonconvex Objectives with Uniform Gradient Heterogeneity
Tomoya Murata (NTT DATA Mathematical Systems Inc), Iifan Tyou (NTT Social Information Laboratories, NTT Corporation)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new Byzantine robust federated learning algorithm called Momentum Screening, aimed at achieving optimal optimization error under non-convex objectives.
Simplicial Representation Learning with Neural $k$-Forms
Kelly Maggs (Ecole Polytechnique Federale de Lausanne), Bastian Rieck (Technical University of Munich)
ClassificationRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: By learning differentiable k-forms, a message-passing-free geometric learning framework is obtained for integrating k-simplices in embedded spaces.
Simplifying Transformer Blocks
Bobby He (ETH Zurich), Thomas Hofmann (ETH Zurich)
TransformerText
🎯 What it does: A design method for simplifying the Transformer block is proposed, removing components such as skip connections, value/projection matrices, sequential sub-blocks, and normalization, while maintaining or even improving training speed and downstream performance.
Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape
Rundi Wu (Columbia University), Changxi Zheng (Columbia University)
GenerationData SynthesisDiffusion modelMesh
🎯 What it does: Using diffusion models, a generator is trained on a single textured 3D shape to learn local texture and geometric distribution from a single instance, subsequently generating diverse high-quality 3D models that support operations such as size transformation, expansion, and local editing.
SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations
Xuan Zhang (Texas A&M University), Shuiwang Ji (University of Pittsburgh)
Convolutional Neural NetworkTime SeriesPhysics Related
🎯 What it does: A multi-stage U-Net structure named SineNet is designed and proposed for learning the temporal dynamics of time-dependent partial differential equations (PDEs).
Single Motion Diffusion
Sigal Raab (Tel Aviv University), Daniel Cohen-Or (Tel Aviv University)
GenerationData SynthesisDiffusion modelVideo
🎯 What it does: Train the diffusion model SinMDM on a single action sequence to learn the internal motion patterns of the sequence, capable of generating diverse synthetic actions of arbitrary length, supporting various action editing tasks during the inference phase (such as temporal/spatial interpolation, action extension, style transfer, group animation, etc.)
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
Xuefei Ning (Tsinghua University), Yu Wang (Tsinghua University)
GenerationComputational EfficiencyAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes the Skeleton-of-Thought (SoT) framework, which first allows the LLM to generate an answer skeleton, and then expands the skeleton points in parallel to reduce inference latency.
Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning
Geraud Nangue Tasse (University of Witwatersrand), Benjamin Rosman (University of Witwatersrand)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: This paper proposes Skill Machines (SM), a finite state machine constructed using Reward Machines (RM) and pre-trained skill primitives, capable of zero-shot completion of any task based on regular subset Linear Temporal Logic (LTL) without the need for further learning.
Skill or Luck? Return Decomposition via Advantage Functions
Hsiao-Ru Pan (Max Planck Institute for Intelligent Systems), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
Reinforcement Learning
🎯 What it does: A reward decomposition method based on causal interpretation of advantage functions is proposed, and it is extended to offline (off-policy) scenarios, resulting in the Off-policy DAE (Direct Advantage Estimation) algorithm.
SKILL-MIX: a Flexible and Expandable Family of Evaluations for AI Models
Dingli Yu (Princeton University), Sanjeev Arora (Princeton University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the SKILL-MIX evaluation framework, which tests the combination skill generation ability of LLMs by randomly selecting combinations of skills and topics.
Skip-Attention: Improving Vision Transformers by Paying Less Attention
Shashanka Venkataramanan (Qualcomm AI Research), Amir Habibian
ClassificationRestorationObject DetectionSegmentationComputational EfficiencyTransformerImage
🎯 What it does: The SKIP-ATTENTION module is proposed, which replaces the self-attention results of the previous layer with a lightweight parameter function to reduce the computational load of ViT.
Sliced Denoising: A Physics-Informed Molecular Pre-Training Method
Yuyan Ni (Chinese Academy of Sciences), Yanyan Lan (Tsinghua University)
Drug DiscoveryTransformerGraphPhysics Related
🎯 What it does: This paper proposes a new molecular pre-training method called Sliced Denoising (SliDe), which simulates molecular energy distribution by adding Gaussian noise to bond lengths, bond angles, and torsion angles, and efficiently regresses force fields using a random slicing technique.
Sliced Wasserstein Estimation with Control Variates
Khai Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
GenerationData SynthesisGaussian SplattingImagePoint Cloud
🎯 What it does: Proposes the use of control variates to reduce the Monte Carlo variance of the sliced Wasserstein (SW) distance, and provides two Gaussian approximation-based upper and lower bound control variates;
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
Saleh Ashkboos (ETH Zurich), James Hensman (Microsoft)
CompressionTransformerLarge Language ModelText
🎯 What it does: A post-training sparsification method called SliceGPT is proposed, which uses orthogonal transformations to slice the Transformer matrix into smaller dense matrices, thereby reducing parameters and embedding dimensions, compressing LLM while maintaining high performance.
SLiMe: Segment Like Me
Aliasghar Khani (Autodesk Research), Ghassan Hamarneh (Simon Fraser University)
SegmentationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: This paper proposes a method called SLiMe for arbitrary fine-grained semantic segmentation using only one annotated image, leveraging the attention mechanism of Stable Diffusion to generate corresponding segmentations on unseen images.
Small-scale proxies for large-scale Transformer training instabilities
Mitchell Wortsman (Google DeepMind), Simon Kornblith (Google DeepMind)
TransformerText
🎯 What it does: This study investigates and reproduces two types of instability that occur during large-scale Transformer training (attention logit growth and output logit divergence), demonstrating that these instabilities can be triggered in small-scale models by high learning rates. It stabilizes training using known interventions such as qk-layernorm and z-loss; further, it evaluates the impact of common techniques like warm-up, weight decay, and µ Param on learning rate sensitivity. Finally, by analyzing the scale behavior of model features (activations, gradient norms), it predicts and discovers new instabilities.
SmartPlay : A Benchmark for LLMs as Intelligent Agents
Yue Wu (Carnegie Mellon University), Yuanzhi Li (Carnegie Mellon University)
TransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringTextBenchmark
🎯 What it does: A multi-game benchmark named SmartPlay has been constructed to evaluate the capabilities of large language models (LLMs) as intelligent agents.
Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
Jaroslaw Blasiok, Preetum Nakkiran (Apple)
ClassificationConvolutional Neural NetworkGaussian SplattingImage
🎯 What it does: This paper proposes a kernel smoothing-based calibration error metric SmoothECE and its corresponding smoothed reliability diagram, to replace the traditional binning ECE and reliability diagram;
SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training
Kazem Meidani (Carnegie Mellon University), Amir Barati Farimani (Carnegie Mellon University)
TransformerContrastive LearningMultimodalityPhysics Related
🎯 What it does: This paper presents SNIP, a unified pre-training model that maps symbolic equations and corresponding numerical data to the same latent space through contrastive learning, achieving cross-modal understanding.
Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community
Arman Isajanyan (Picsart AI Research), Humphrey Shi (Picsart AI Research)
GenerationRecommendation SystemDiffusion modelContrastive LearningImageText
🎯 What it does: A method for evaluating text-to-image generation models based on community implicit feedback is proposed and validated—Social Reward;
Social-Transmotion: Promptable Human Trajectory Prediction
Saeed Saadatnejad (EPFL), Alexandre Alahi (EPFL)
Object TrackingPose EstimationTransformerPrompt EngineeringVideoMultimodality
🎯 What it does: This paper proposes Social-Transmotion, a general Promptable Transformer model that can utilize various visual cues (trajectories, 3D/2D poses, bounding boxes) for human trajectory prediction.
SocioDojo: Building Lifelong Analytical Agents with Real-world Text and Time Series
Junyan Cheng (Dartmouth College), Peter Chin (Dartmouth College)
Recommendation SystemAnomaly DetectionOptimizationData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextMultimodalityTime SeriesFinance RelatedRetrieval-Augmented Generation
🎯 What it does: Constructed the SocioDojo open-source lifelong learning environment and proposed the Hyperportfolio task to evaluate agents' capabilities in social analysis and decision-making;
Soft Contrastive Learning for Time Series
Seunghan Lee (Yonsei University), Kibok Lee (Yonsei University)
Anomaly DetectionRepresentation LearningContrastive LearningTime Series
🎯 What it does: This paper proposes SoftCLT, a soft contrastive learning framework for time series that enhances self-supervised representation learning using soft weights between instances and over time.
Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Models
Yangming Li (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
RestorationGenerationDiffusion modelImage
🎯 What it does: Proposes the Soft Mixture Denoising (SMD) model, which improves the backward denoising process of diffusion models and addresses the expression bottleneck caused by the traditional single Gaussian prior.
Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity
Runyu Zhang, Na Li (Harvard University)
Reinforcement Learning
🎯 What it does: A new risk-sensitive MDP framework is proposed, and its equivalence to soft robust MDP is proven. The policy gradient theorem and global convergence are derived, and a robust fitting Z-iteration algorithm along with sample complexity analysis is provided for KL soft robust MDP.
SOHES: Self-supervised Open-world Hierarchical Entity Segmentation
Shengcao Cao (University of Illinois Urbana-Champaign), Yu-Xiong Wang (University of Illinois Urbana-Champaign)
Object DetectionSegmentationTransformerContrastive LearningImage
🎯 What it does: A completely self-supervised open-world hierarchical entity segmentation method called SOHES is proposed, which generates pseudo-labels and trains a segmentation model through a three-stage self-evolution process (self-exploration, self-guidance, self-correction) to achieve segmentation of entities and their components.
SOInter: A Novel Deep Energy-Based Interpretation Method for Explaining Structured Output Models
S. Fatemeh Seyyedsalehi (Sharif University of Technology), Hamid R. Rabiee (Sharif University of Technology)
Explainability and InterpretabilityImageText
🎯 What it does: Designed and trained an interpreter called SOInter based on energy networks to explain the feature importance of black-box structured output models on single target outputs.
Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification
Aojun Zhou (MMLab, Chinese University of Hong Kong), Hongsheng Li (Shanghai Artificial Intelligence Laboratory)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Analyze the code usage frequency of the GPT-4 Code Interpreter in mathematical reasoning, and propose the 'Explicit Code Self-Validation (CSV)' prompt along with a weighted majority voting method based on validation results to enhance the accuracy of zero-shot mathematical reasoning.
Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution
Yiyang Ma (Peking University), Jiaying Liu (Microsoft Research)
RestorationSuper ResolutionDiffusion modelImageOrdinary Differential Equation
🎯 What it does: This paper proposes to achieve stable sampling of high-quality super-resolved images from a pre-trained diffusion super-resolution model by solving diffusion ODEs and selecting approximately optimal boundary conditions.
Solving High Frequency and Multi-Scale PDEs with Gaussian Processes
Shikai Fang (University of Utah), Shandian Zhe (University of Utah)
Time SeriesPhysics Related
🎯 What it does: A Gaussian Process-based solver GP-HM is proposed, specifically designed for solving high-frequency and multi-scale partial differential equations (PDEs).
Solving Homogeneous and Heterogeneous Cooperative Tasks with Greedy Sequential Execution
Shanqi Liu (Zhejiang University), Yong Liu (Zhejiang University)
OptimizationReinforcement LearningSequential
🎯 What it does: A method called Greedy Sequential Execution (GSE) is proposed, which integrates value decomposition and a greedy sequential execution strategy to simultaneously address homogeneous and heterogeneous cooperative tasks.
Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency
Bowen Song (University of Michigan), Liyue Shen (University of Michigan)
RestorationGenerationDiffusion modelImageBiomedical DataComputed Tomography
🎯 What it does: A new algorithm ReSample is proposed to solve general inverse problems using a pre-trained latent diffusion model.
Some Fundamental Aspects about Lipschitz Continuity of Neural Networks
Grigory Khromov (ETH Zurich), Sidak Pal Singh (Max Planck ETH Center for Learning Systems)
Convolutional Neural NetworkTransformerImage
🎯 What it does: This study investigates the Lipschitz constant of neural networks, exploring its relationship with factors such as initialization, width, and label noise, and reveals the behavior of effective Lipschitz through upper and lower bound estimates.
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Hong Liu (Stanford University), Tengyu Ma (Stanford University)
OptimizationTransformerLarge Language ModelText
🎯 What it does: A scalable second-order stochastic optimizer called Sophia is proposed, specifically designed for pre-training large-scale language models. It can reduce training steps and computational cost by about 50% while maintaining nearly the same per-step cost as AdamW at the same validation loss.
SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
Xuhui Zhou, Maarten Sap
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Designed and implemented the SOTOPIA environment, which allows language agents and humans to role-play in various social contexts during multi-turn dialogues to achieve personal social goals, and evaluates interaction performance based on the multi-dimensional framework SOTOPIA-EVAL; simultaneously, it uses GPT-4 for automated evaluation to explore the gap in social intelligence between models and humans.
Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation
Prakhar Kaushik (Johns Hopkins University), Alan Yuille (University of Freiburg)
Pose EstimationDomain AdaptationImage
🎯 What it does: A source-agnostic, unsupervised domain adaptation method based on neural grids, 3DUDA, is proposed, which can estimate category-level 3D poses using only RGB images from the target domain.
Space and time continuous physics simulation from partial observations
Steeven JANNY, Christian Wolf (Naver Labs Europe)
Graph Neural NetworkTransformerTime SeriesPhysics Related
🎯 What it does: By learning physical systems from sparse spatiotemporal observations, a dual dynamic system framework is proposed to achieve predictions over continuous spatiotemporal domains.
Space Group Constrained Crystal Generation
Rui Jiao (Tsinghua University), Yang Liu (Alibaba Group)
GenerationData SynthesisOptimizationDiffusion modelGraphPhysics Related
🎯 What it does: This paper presents DiffCSP++, a crystal generation diffusion model under space group constraints.
SpaCE: The Spatial Confounding Environment
Mauricio Tec (Harvard University), Francesca Dominici (Harvard University)
Graph Neural NetworkTabularBenchmark
🎯 What it does: The SpaCE toolkit is proposed, providing dozens of semi-synthetic spatial causal inference benchmark datasets based on real covariates and treatments, and implementing an automated data generation and evaluation pipeline.
Sparse Autoencoders Find Highly Interpretable Features in Language Models
Robert Huben (EleutherAI), Lee Sharkey (MATS)
Explainability and InterpretabilityRepresentation LearningTransformerAuto EncoderText
🎯 What it does: Using sparse autoencoders for dictionary learning on the internal activations of language models to extract sparse, unambiguous features, reduce polysemanticity, and reveal the causal mechanisms of the model.
Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging
Max Zimmer (Zuse Institute Berlin), Sebastian Pokutta (Technische Universitat Berlin)
OptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: Introduce the 'Sparse Model Soups' strategy into existing pruning algorithms (such as IMP): generate multiple versions of the same sparse model with different hyperparameters during each prune-retrain phase, and then merge their parameters into a new sparse model through weighted averaging; this averaged model is then used as the starting point for the next phase, forming a multi-round iterative averaging process.
Sparse MoE with Language Guided Routing for Multilingual Machine Translation
Xinyu Zhao (University of North Carolina), Tianlong Chen (University of North Carolina)
TransformerMixture of ExpertsText
🎯 What it does: A language-guided sparse expert model, Lingual-SMoE, is designed for multilingual machine translation.
Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN
Biswadeep Chakraborty (Georgia Institute of Technology), Saibal Mukhopadhyay (Georgia Institute of Technology)
ClassificationOptimizationSpiking Neural NetworkImageTime Series
🎯 What it does: This paper proposes a task-agnostic pruning method for sparse recursive spiking neural networks (RSNN) called Lyapunov Noise Pruning (LNP). By utilizing a large heterogeneous RSNN with random initialization, it prunes synapses and neurons using Lyapunov exponents and spectral graph sparsification techniques while maintaining network stability, ultimately resulting in a sparse model applicable to multiple tasks (image classification, time series prediction).
Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
Moonseok Choi (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
ClassificationOptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: An iterative magnitude pruning method based on Stochastic Weight Averaging with Multiple Particles (SWAMP) is proposed, which improves the sparsification effect of traditional IMP.
SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation
Qianxu Wang (Peking University), Leonidas Guibas (Stanford University)
OptimizationKnowledge DistillationRobotic IntelligenceContrastive LearningPoint Cloud
🎯 What it does: Proposes SparseDFF, which generates a consistent 3D feature field from sparse RGBD views, supporting scalable hand operations with a single demonstration;
SparseFormer: Sparse Visual Recognition via Limited Latent Tokens
Ziteng Gao (National University of Singapore), Mike Zheng Shou (National University of Singapore)
ClassificationRecognitionTransformerImageVideo
🎯 What it does: This paper proposes a visual Transformer named SparseFormer, which achieves sparse perception and recognition of images by using a minimal number of tokens (e.g., 9 to 81) in the latent space and employing sparse feature sampling.
Sparsistency for inverse optimal transport
Francisco Andrade (École Normale Supérieure Paris), Clarice Poon (University of Warwick)
OptimizationGraph
🎯 What it does: This paper studies the theory and algorithms for recovering sparse ground costs (Inverse Optimal Transport) from observed entropy-regularized OT coupled samples;
Spatially-Aware Transformers for Embodied Agents
Junmo Cho (Korea Advanced Institute of Science and Technology), Sungjin Ahn (Korea Advanced Institute of Science and Technology)
Robotic IntelligenceTransformerReinforcement LearningImageVideo
🎯 What it does: A Spatial Awareness Transformer (SAT) is proposed for embodied agents, incorporating spatial embeddings into experiential memory and designing location-based hierarchical reading and adaptive memory allocation.
Spatio-Temporal Approximation: A Training-Free SNN Conversion for Transformers
Yizhou Jiang (Tsinghua University), Feng Chen
ClassificationSpiking Neural NetworkTransformerImage
🎯 What it does: This paper proposes the 'Space-Time Approximation (STA)' method, which for the first time achieves the conversion of pre-trained Transformers (such as CLIP's ViT-B/32) into event-driven Spiking Neural Networks (SNN) in a training-independent manner, while maintaining performance close to the original model on tasks such as zero-shot classification.
Spatio-Temporal Few-Shot Learning via Diffusive Neural Network Generation
Yuan Yuan (Tsinghua University), Yong Li (Tsinghua University)
GenerationDomain AdaptationMeta LearningTransformerPrompt EngineeringDiffusion modelTime Series
🎯 What it does: A generative pre-training framework GPD based on diffusion models is proposed for cross-city spatiotemporal prediction with few-shot learning.
SPDER: Semiperiodic Damping-Enabled Object Representation
Kathan Shah (University of California Berkeley), Chawin Sitawarin (University of California Berkeley)
Data SynthesisSuper ResolutionRepresentation LearningImageVideoAudio
🎯 What it does: A novel implicit neural representation network SPDER is designed and implemented, utilizing sine multipliers and sublinear decay activation functions, achieving high-precision continuous representations of images, audio, and video without the need for pre-encoding or hyperparameter tuning.
Spectrally Transformed Kernel Regression
Runtian Zhai (Carnegie Mellon University), Pradeep Kumar Ravikumar (Carnegie Mellon University)
ClassificationGraph
🎯 What it does: A scalable Spectral Transformation Kernel Regression (STKR) method is proposed, which utilizes unlabeled data to achieve smoothing through spectral transformation that intrinsically combines data distribution, and a closed-form solver that can be implemented under inductive settings is provided.
SpeechTokenizer: Unified Speech Tokenizer for Speech Language Models
Xin Zhang (Fudan University), Xipeng Qiu (Fudan University)
GenerationData SynthesisKnowledge DistillationRecurrent Neural NetworkTransformerGenerative Adversarial NetworkBenchmarkAudio
🎯 What it does: This paper proposes SpeechTokenizer, a unified speech tokenizer, and constructs the SLMTokBench benchmark, subsequently implementing a Unified Speech Language Model (USLM) based on this tokenizer.
Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips
Man Yao (Institute of Automation Chinese Academy of Sciences), Guoqi Li (Institute of Automation Chinese Academy of Sciences)
ClassificationObject DetectionSegmentationKnowledge DistillationSpiking Neural NetworkTransformerImageVideo
🎯 What it does: A directly trainable spiking neural network based on Transformer, called Meta‑SpikeFormer, is proposed, which can simultaneously handle visual tasks such as classification, detection, and segmentation.
SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition
Hongwei Ren (Hong Kong University of Science and Technology), Bojun Cheng (Hong Kong University of Science and Technology)
RecognitionComputational EfficiencySpiking Neural NetworkPoint Cloud
🎯 What it does: An end-to-end point cloud-based spiking neural network, SpikePoint, is proposed for action recognition using event cameras.
Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM
Eliya Nachmani (Google Research), Michelle Tadmor Ramanovich (Google Research)
RecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningAudio
🎯 What it does: Developed Spectron, an end-to-end model for continuous speech and question answering;
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
Tim Dettmers (University of Washington), Dan Alistarh (NeuralMagic)
CompressionTransformerLarge Language ModelText
🎯 What it does: A new sparse quantization representation, SpQR, is proposed, which can compress large language models to 3-4 bits/parameter while maintaining nearly the same inference quality as 16-bit models.
SPTNet: An Efficient Alternative Framework for Generalized Category Discovery with Spatial Prompt Tuning
Hongjun Wang (Visual AI Lab University of Hong Kong), Kai Han (Visual AI Lab University of Hong Kong)
ClassificationTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: The SPTNet framework is proposed, which achieves efficient adaptation of the pre-trained ViT model by alternately optimizing model parameters and spatial prompts, thereby completing the Generalized Category Discovery task.
Spurious Feature Diversification Improves Out-of-distribution Generalization
LIN Yong, Tong Zhang (University of Illinois Urbana-Champaign)
ClassificationDomain AdaptationSupervised Fine-TuningContrastive LearningImage
🎯 What it does: This paper conducts empirical and theoretical analysis of the weight space ensemble method WiSE‑FT, discovering the 'FalseFalseTrue' phenomenon, and further proves that the ensemble utilizing diverse pseudo-features can enhance out-of-distribution (OOD) generalization. It then proposes the BANG (Balanced Averaging) method to alleviate the overconfidence issue of fine-tuned models.
sRGB Real Noise Modeling via Noise-Aware Sampling with Normalizing Flows
Dongjin Kim (Hanyang University), Tae Hyun Kim (Hanyang University)
RestorationGenerationFlow-based ModelImage
🎯 What it does: Train a single conditional normalizing flow model NAFlow to learn different Gaussian distributions through various camera configurations, and use noise-aware sampling during inference to generate sRGB noise similar to real noise.
SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores
Zhiyu Mei (Tsinghua University), Yi Wu (Tsinghua University)
Reinforcement LearningVideo
🎯 What it does: A scalable distributed reinforcement learning system SRL has been designed and implemented, supporting CPU/GPU parallel training at the level of thousands of cores, achieving high throughput in various environments.
Stabilizing Backpropagation Through Time to Learn Complex Physics
Patrick Schnell (Technical University of Munich), Nils Thuerey (Technical University of Munich)
OptimizationReinforcement Learning from Human FeedbackRecurrent Neural NetworkReinforcement LearningTime SeriesPhysics Related
🎯 What it does: This paper addresses the issues of gradient explosion and vanishing in the long-term unrolling training of physical simulators and neural networks. It proposes a backpropagation method using gradient stop and rotational correction to achieve more balanced and stable update vectors, and validates its effectiveness in control tasks.