ICLR 2024 Papers — Page 20
International Conference on Learning Representations · 2260 papers
Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data
Chongyi Zheng (Carnegie Mellon University), Sergey Levine (University of California Berkeley)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningContrastive LearningImage
🎯 What it does: Developed and evaluated a self-supervised reinforcement learning method based on contrastive learning, which successfully solves real robot manipulation tasks under offline image target conditions after stabilization.
Stable Anisotropic Regularization
William Rudman (Brown University), Carsten Eickhoff (University of Tübingen)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a differentiable, mini-batch stable regularization method called I-STAR based on IsoScore⋆, aimed at adjusting the isotropy of the embedding space of large language models during the fine-tuning process.
Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data
YongKyung Oh (Ulsan National Institute of Science and Technology), Sungil Kim (Ulsan National Institute of Science and Technology)
ClassificationAnomaly DetectionRecurrent Neural NetworkTime SeriesBiomedical DataElectronic Health RecordsStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes three types of stable Neural Stochastic Differential Equations (Neural SDE) — Langevin, linear noise, and geometric SDE, and embeds controlled paths into the drift term to address issues of irregular sampling and missing values in time series data encountered in reality.
Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns
Brian DuSell (ETH Zurich), David Chiang (University of Notre Dame)
TransformerText
🎯 What it does: Introducing differentiable stacks (superposition stack and nondeterministic stack) into the Transformer as an attention mechanism to enhance the model's ability to recognize and model hierarchical patterns and context-free languages (CFL) without the need for syntactic supervision.
STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction
Dennis Wu (Northwestern University), Han Liu (Northwestern University)
TransformerTime Series
🎯 What it does: This paper proposes STanHop-Net, a multi-variable time series prediction model that utilizes a sparse Tandem Hopfield structure and incorporates external memory plugins to quickly respond to sudden events.
STARC: A General Framework For Quantifying Differences Between Reward Functions
Joar Max Viktor Skalse (Oxford University), Alessandro Abate (Oxford University)
Reinforcement LearningTabular
🎯 What it does: A reward function distance metric called STARC is proposed and evaluated to quantify the differences between reward functions and ensure the safety and reliability of reward learning.
State Representation Learning Using an Unbalanced Atlas
Li Meng (University of Oslo), Paal E. Engelstad
Representation LearningContrastive LearningImage
🎯 What it does: This paper proposes the Unbalanced Graph Collection (UA) paradigm, applies it to self-supervised state representation learning, improves ST-DIM to DIM-UA, and conducts experiments on AtariARI and CIFAR10.
Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
Huy Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
Mixture of Experts
🎯 What it does: This paper studies the statistical convergence properties of the Top-K sparse Softmax gated Gaussian mixture expert (MoE) model, providing the convergence rates of density and parameter estimation under both exact and overfitting scenarios for maximum likelihood estimation.
Statistical Rejection Sampling Improves Preference Optimization
Tianqi Liu (Google Research), Jialu Liu (Google DeepMind)
OptimizationReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningText
🎯 What it does: An offline preference optimization framework RSO is proposed, which generates preference pairs from an approximately optimal policy using statistical rejection sampling, and fine-tunes the language model based on this.
Statistically Optimal $K$-means Clustering via Nonnegative Low-rank Semidefinite Programming
Yubo Zhuang (University of Illinois at Urbana-Champaign), Richard Y. Zhang (University of Illinois at Urbana-Champaign)
OptimizationImageTabular
🎯 What it does: A K-means clustering algorithm based on Non-negative Low-rank Semi-definite Programming (NLR) is proposed, utilizing Burer-Monteiro factorization and incremental Lagrangian methods for scalable solutions.
Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds
Sipeng Zheng (Beijing Academy of Artificial Intelligence), Zongqing Lu (Peking University)
GenerationRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: We propose Steve-Eye, a multimodal large model capable of interacting with the open world through a visual-text interface, endowing LLM embedded agents with more intuitive perception and planning capabilities.
Stochastic Controlled Averaging for Federated Learning with Communication Compression
Xinmeng Huang (LinkedIn), Xiaoyun Li
OptimizationFederated LearningImage
🎯 What it does: Two compression-based federated learning algorithms, SCALLION (unbiased compression) and SCAFCOM (biased compression), are proposed, which halve the uplink communication volume through a new SCAFFOLD form, achieving robust convergence under arbitrary data heterogeneity, partial participation, and local updates.
Stochastic Gradient Descent for Gaussian Processes Done Right
Jihao Andreas Lin (University of Cambridge), David Janz (University of Alberta)
OptimizationGraph Neural NetworkTabular
🎯 What it does: A stochastic dual descent (SDD) algorithm is proposed for Gaussian process regression and sampling to solve large-scale linear systems, directly replacing traditional conjugate gradient or variational methods.
Stochastic Modified Equations and Dynamics of Dropout Algorithm
Zhongwang Zhang (Shanghai Jiao Tong University), Zhi-Qin John Xu (Shanghai Jiao Tong University)
OptimizationConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: By rigorously deriving the Dropout iterative process of a two-layer neural network, we obtain its Stochastic Modification Equation (SME) and match it with the drift term and noise covariance of gradient descent;
Str2Str: A Score-based Framework for Zero-shot Protein Conformation Sampling
Jiarui Lu (Mila - Quebec AI Institute), Jian Tang (HEC Montreal)
Protein Structure PredictionDiffusion modelScore-based ModelBiomedical Data
🎯 What it does: A score-based structure-to-structure translation framework STR2STR is proposed for zero-shot protein conformation sampling.
Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objects
Chunming He (Shenzhen International Graduate School, Tsinghua University), Fisher Yu (ETH Zurich)
Object DetectionSegmentationGenerationGenerative Adversarial NetworkImage
🎯 What it does: For the task of camouflage object detection (COD), this paper proposes two key technologies: first, the Camouflageator adversarial training framework from the perspective of the 'predator', which uses an auxiliary generator to synthesize more difficult-to-detect camouflage images to enhance the generalization ability of the detector; second, the ICEG detector from the perspective of the 'prey', which includes an internal consistency module (CFC) for segmentation and an edge-guided separation calibration module (ESC) to eliminate fuzzy boundaries.
STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models
Pum Jun Kim (Ulsan National Institute of Science and Technology), Jaejun Yoo (Ulsan National Institute of Science and Technology)
GenerationData SynthesisVideo
🎯 What it does: The STREAM evaluation metric is proposed, which can independently assess the spatial (realism, diversity) and temporal (coherence) quality of video generation models.
StructComp: Substituting propagation with Structural Compression in Training Graph Contrastive Learning
Shengzhong Zhang (Fudan University), Zengfeng Huang (Fudan University)
Computational EfficiencyRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: A structural compression (StructComp) framework is proposed, which replaces message passing with node compression using a sparse low-rank approximation of the diffusion matrix, training an MLP encoder and restoring the complete GNN during the inference phase.
Structural Estimation of Partially Observed Linear Non-Gaussian Acyclic Model: A Practical Approach with Identifiability
Songyao Jin, Kun Zhang (Beijing Technology and Business University)
Tabular
🎯 What it does: A partially observable linear non-Gaussian acyclic model (PO-LiNGAM) is proposed, providing identifiable theory under the assumption of no prior latent variables, and designing a three-stage iterative algorithm based on GIN conditions for structure discovery.
Structural Fairness-aware Active Learning for Graph Neural Networks
Haoyu Han (Michigan State University), Makoto Yamada (Okinawa Institute of Science and Technology)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes an active learning framework for graph neural networks called SCARCE, which balances model performance and structural fairness when selecting labeled nodes at once.
Structural Inference with Dynamics Encoding and Partial Correlation Coefficients
Aoran Wang (University of Luxembourg), Jun Pang (University of Luxembourg)
Graph Neural NetworkAuto EncoderTime SeriesBenchmark
🎯 What it does: This paper proposes a method for structural inference using Variational Dynamic Encoder (VDE) and Partial Correlation Coefficient (PCOR);
Structured Video-Language Modeling with Temporal Grouping and Spatial Grounding
Yuanhao Xiong (Google Research), Liangzhe Yuan (Google Research)
RecognitionRetrievalTransformerVision Language ModelContrastive LearningVideoText
🎯 What it does: Proposes the S-ViLM framework, which utilizes the fine-grained structure of video and text for pre-training, incorporating cross-segment spatial alignment and intra-segment temporal grouping.
Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning
Sharut Gupta (Massachusetts Institute of Technology), Stefanie Jegelka (Massachusetts Institute of Technology)
RetrievalRepresentation LearningConvolutional Neural NetworkContrastive LearningImagePoint Cloud
🎯 What it does: A new contrastive learning framework called CARE is proposed, which utilizes linear isometric transformations such as rotation to map input augmentations to orthogonal transformations in the embedding space, thereby learning representations that can distinguish samples while maintaining geometric interpretability on unlabeled data.
Stylized Offline Reinforcement Learning: Extracting Diverse High-Quality Behaviors from Heterogeneous Datasets
Yihuan Mao (Institute for Interdisciplinary Information Sciences), Chongjie Zhang (Washington University in St. Louis)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a two-step framework called Stylized Offline RL (SORL) for extracting both high-quality and diverse policies from heterogeneous offline datasets.
Submodular Reinforcement Learning
Manish Prajapat (ETH Zurich), Andreas Krause (ETH Zurich)
Reinforcement Learning
🎯 What it does: A submodular reinforcement learning framework called SUBRL is proposed, along with a greedy policy gradient algorithm SUBPO, which can handle non-additive, history-dependent rewards.
Subtractive Mixture Models via Squaring: Representation and Learning
Lorenzo Loconte (University of Edinburgh), Antonio Vergari (University of Edinburgh)
Knowledge DistillationRepresentation LearningText
🎯 What it does: A deep probabilistic circuit (NPC2) is proposed to achieve subtractive mixing through a square mixture model, enabling non-negative and interpretable distribution modeling.
Successor Heads: Recurring, Interpretable Attention Heads In The Wild
Rhys Gould (University of Cambridge), Arthur Conmy (Independent)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper discovers and explains the 'successor head' in large language models—a type of attention head that can increment words in a sequence (such as numbers, months, days of the week), and reveals its internal implementation through mechanistic interpretability analysis.
Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Angelica Chen (New York University), Naomi Saphra (Harvard University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study investigates the grammatical learning trajectory of masked language models during the pre-training process, particularly the mutation of the Syntax Attention Structure (SAS), and manipulates the emergence and disappearance of SAS through regularization methods to assess its causal impact on grammatical capabilities.
Sufficient conditions for offline reactivation in recurrent neural networks
Nanda H Krishna (Mila Quebec AI Institute), Guillaume Lajoie (Mila Quebec AI Institute)
OptimizationRecurrent Neural NetworkTime SeriesSequentialStochastic Differential Equation
🎯 What it does: This paper studies how noise recurrent neural networks can automatically generate offline reactivation in the absence of external stimuli after task optimization, and provides sufficient conditions for this phenomenon.
Sum-Product-Set Networks: Deep Tractable Models for Tree-Structured Graphs
Milan Papez (Czech Technical University), Tomáš Pevný (Czech Technical University)
ClassificationGenerationGraph Neural NetworkGraphTabular
🎯 What it does: A new interpretable deep generative model called Sum-Product-Set Networks (SPSNs) is proposed for representing and reasoning about the probability distribution of tree-structured graphs.
Supervised Knowledge Makes Large Language Models Better In-context Learners
Linyi Yang (Westlake University), Yue Zhang (Westlake University)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In the inference phase of large language models (LLMs), the output of a small supervised learning model (SLM) is incorporated to enhance out-of-distribution (OOD) generalization and reduce hallucination.
SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs
Jaehyung Kim (Carnegie Mellon University), Jinwoo Shin (KAIST AI)
RetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Using zero-shot prompts to first generate answer candidates, then generating conditional summaries for each candidate based on retrieved paragraphs, and verifying and selecting the most suitable answer through the effectiveness of the summaries and their mutual ranking, in order to improve the accuracy of open-domain question answering.
SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS
Yameng Peng (RMIT University), Xiaojun Chang (University of Technology Sydney)
Neural Architecture SearchImage
🎯 What it does: This paper proposes a training-free network performance evaluation metric called SWAP-Score, and based on this metric, implements an ultra-fast NAS method named SWAP-NAS.
SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning
Lei You (Technical University of Denmark), Hei Victor Cheng (Aarhus University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a network pruning method called SWAP based on entropy-regularized Wasserstein regression (EWR), which reduces gradient noise and retains covariance information using OT distance.
SWE-bench: Can Language Models Resolve Real-world Github Issues?
Carlos E Jimenez, Karthik R Narasimhan
AI Code AssistantTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: A benchmark named SWE-Bench is proposed and implemented to evaluate the ability of language models to locate and fix issues in real GitHub code repositories, and fine-grained patch generation and execution testing are conducted based on this benchmark.
SweetDreamer: Aligning Geometric Priors in 2D diffusion for Consistent Text-to-3D
Weiyu Li (Hong Kong University of Science and Technology), Ping Tan (Hong Kong University of Science and Technology)
GenerationData SynthesisDiffusion modelPoint CloudMeshStochastic Differential Equation
🎯 What it does: This paper fine-tunes a 2D diffusion model to generate coordinate maps that are consistent with 3D geometry, seamlessly integrating this aligned geometric prior into the existing text-to-3D generation pipeline to enhance multi-view consistency.
Symbol as Points: Panoptic Symbol Spotting via Point-based Representation
WENLONG LIU, Lei Zhang (International Digital Economy Academy)
RecognitionObject DetectionTransformerContrastive LearningPoint Cloud
🎯 What it does: This paper proposes treating graphic primitives in CAD drawings as point sets and uses a point cloud-based Transformer to achieve panoramic symbol localization and recognition.
SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning
Jiacheng Chen (South China University of Technology), Yue-Jiao Gong (South China University of Technology)
OptimizationMeta LearningRecurrent Neural NetworkReinforcement LearningTabular
🎯 What it does: A meta-learning framework based on symbolic equation learning (SYMBOL) is proposed, which constructs a black-box optimizer by automatically generating closed-form update rules and uses reinforcement learning for meta-learning.
Symmetric Basis Convolutions for Learning Lagrangian Fluid Mechanics
Rene Winchenbach (Technical University Munich), Nils Thuerey (Technical University Munich)
Graph Neural NetworkGraphPhysics Related
🎯 What it does: A symmetric Fourier basis continuous convolution (SFBC) framework is proposed and implemented for learning physical simulations in Lagrangian fluid dynamics.
Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems
Juno Kim (University of Tokyo), Taiji Suzuki (University of Tokyo)
OptimizationReinforcement LearningTabularStochastic Differential Equation
🎯 What it does: This paper proposes a symmetric mean field Langevin dynamics framework for distributed minimax optimization problems on probability distributions, and presents two algorithms: single-loop average gradient MFL-AG and double-loop anchored best response MFL-ABR; it also provides theoretical convergence and unified time control for discretization errors; the method is applied to zero-sum Markov games.
Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching
Ganesh Ramachandra Kini (University of California), Christos Thrampoulidis (University of British Columbia)
ClassificationRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: The paper studies the geometric structure of network representations when using supervised contrastive loss (SCL) with ReLU under class imbalance conditions, proving that ReLU can restore symmetry (orthogonal frame structure) while introducing batch-binding to ensure geometric uniqueness.
Symmetric Single Index Learning
Aaron Zweig (New York University), Joan Bruna (New York University)
Tabular
🎯 What it does: The study implements the learning of symmetric single-exponential models on symmetric (permutation-invariant) neural networks DeepSets using gradient flow;
Symphony: Symmetry-Equivariant Point-Centered Spherical Harmonics for 3D Molecule Generation
Ameya Daigavane (Massachusetts Institute of Technology), Tess Smidt (Massachusetts Institute of Technology)
GenerationData SynthesisDrug DiscoveryGraph Neural NetworkDiffusion modelPoint Cloud
🎯 What it does: This paper proposes Symphony, an autoregressive 3D molecular generation model that utilizes higher-order E(3)-equivariant features and spherical harmonic projections.
Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control
Longtao Zheng (Nanyang Technological University), Bo An (Nanyang Technological University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: An intelligent agent SYNAPSE for computer control was built using a large language model (LLM), capable of generating a series of keyboard and mouse operations based on task descriptions and computer states (HTML/screenshots);
Synaptic Weight Distributions Depend on the Geometry of Plasticity
Roman Pogodin (McGill University), Blake Aaron Richards (McGill University)
OptimizationTabular
🎯 What it does: This paper explores the impact of synaptic geometry (i.e., distance metrics in weight space) on synaptic weight distribution using mirror descent theory. It proves that under non-Euclidean geometry, the weight distribution tends to a log-normal distribution and provides a method to infer synaptic geometry by observing weight changes before and after learning.
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image
Yuan Liu (University of Hong Kong), Wenping Wang (Texas A&M University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes SyncDreamer, a synchronous multi-view diffusion model that can generate multi-view consistent images from single-view images and achieve high-quality 3D reconstruction through these images.
Synergistic Patch Pruning for Vision Transformer: Unifying Intra- & Inter-Layer Patch Importance
Yuyao Zhang (University of Science and Technology of China), Nikolaos Freris
ClassificationCompressionComputational EfficiencyTransformerImage
🎯 What it does: This paper proposes a dynamic patch pruning method for Vision Transformers called STAR, which integrates local importance obtained from attention at each layer with cross-layer importance based on LRP, and achieves efficient pruning through adaptive layer retention rates.
T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
Pratyush Maini (Carnegie Mellon University), Aditi Raghunathan
RetrievalRepresentation LearningTransformerContrastive LearningImageText
🎯 What it does: A T-MARS method for data filtering on large-scale image-text datasets is proposed, which first uses text detection to mask the text in images, and then re-scores the masked images against the original text using CLIP to filter out samples dominated by text and lacking visual information, thereby improving the quality of visual representation.
T-Rep: Representation Learning for Time Series using Time-Embeddings
Archibald Felix Fraikin, Stephanie Allassonniere (Universite Paris Cite)
Anomaly DetectionRepresentation LearningConvolutional Neural NetworkTime Series
🎯 What it does: A self-supervised time series representation learning framework T-Rep has been developed to capture fine-grained temporal features by learning time embeddings and pre-training tasks.
TAB: Temporal Accumulated Batch Normalization in Spiking Neural Networks
Haiyan Jiang (Mohamed bin Zayed University of Artificial Intelligence), Huan Xiong (Harbin Institute of Technology)
Spiking Neural NetworkImage
🎯 What it does: A batch normalization method named TAB (Temporal Accumulated Batch Normalization) is proposed for directly training Spiking Neural Networks (SNNs), addressing the temporal covariance shift (TCS) problem by utilizing temporal accumulated statistics.
TabR: Tabular Deep Learning Meets Nearest Neighbors
Yury Gorishniy (Yandex), Artem Babenko (Yandex)
ClassificationRetrievalTabular
🎯 What it does: This paper designs and implements TabR, a retrieval-enhanced deep learning model specifically for supervised learning tasks on tabular data.
Tackling the Data Heterogeneity in Asynchronous Federated Learning with Cached Update Calibration
Yujia Wang (Pennsylvania State University), Jinghui Chen (Carnegie Mellon University)
Federated LearningTransformerSupervised Fine-TuningImageText
🎯 What it does: This study investigates the impact of data heterogeneity on convergence in asynchronous federated learning and proposes a method for the server to cache the latest updates from each client for global calibration to enhance convergence speed.
TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series
Arjun Ashok (ServiceNow Research), Alexandre Drouin (ServiceNow Research)
TransformerTime Series
🎯 What it does: TACTiS-2 is proposed, a Transformer-based attention copula model that can flexibly perform multivariate time series prediction, interpolation, and their combination tasks;
Tag2Text: Guiding Vision-Language Model via Image Tagging
Xinyu Huang (Fudan University), Lei Zhang (International Digital Economy Academy)
GenerationRetrievalTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a visual-language pre-training framework called Tag2Text, which embeds image label learning to utilize label information in image-text pairs to guide multimodal feature learning.
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu (Carnegie Mellon University), Rasool Fakoor
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringSequentialBenchmark
🎯 What it does: The TAIL framework is proposed, utilizing a pre-trained large decision model and lightweight adapters to achieve efficient and forget-free adaptation for continuous control tasks.
Tailoring Self-Rationalizers with Multi-Reward Distillation
Sahana Ramnath (University of Southern California), Xiang Ren (University of Washington)
Explainability and InterpretabilityKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes a multi-reward self-inference method MARIO, enabling small LLMs to generate higher quality reasoning explanations.
Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
Huaixiu Steven Zheng (Google DeepMind), Denny Zhou (Google DeepMind)
TransformerLarge Language ModelPrompt EngineeringTextPhysics RelatedRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes STEP-BACK Prompting, a prompting method that generates high-level concepts or principles by first abstracting the problem and then reasoning.
Talk like a Graph: Encoding Graphs for Large Language Models
Bahare Fatemi (Google Research), Bryan Perozzi (Google Research)
Graph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraphBenchmark
🎯 What it does: This study investigates how to encode graph-structured data into text so that large language models (LLMs) can perform graph reasoning through a text interface, and systematically evaluates the impact of different encoding methods, prompting strategies, and graph structures on reasoning performance.
Tangent Transformers for Composition,Privacy and Removal
Tian Yu Liu (University of California), Stefano Soatto (University of California)
ClassificationOptimizationSafty and PrivacyTransformerSupervised Fine-TuningImage
🎯 What it does: Perform a first-order Taylor expansion on the pre-trained Transformer to obtain a linearized Tangent Transformer, and fine-tune downstream tasks through Tangent Attention Fine-Tuning (TAFT) while maintaining the same number of parameters.
TapMo: Shape-aware Motion Generation of Skeleton-free Characters
Jiaxu Zhang (Wuhan University), Ying Shan (Tencent)
GenerationData SynthesisGraph Neural NetworkDiffusion modelGenerative Adversarial NetworkMesh
🎯 What it does: This paper presents a text-driven animation pipeline named TapMo, which generates motion for unskinned 3D characters, addressing the limitations of traditional methods that only work with pre-rigged skeletons.
Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning
Yucheng Yang (Eindhoven University of Technology), Meng Fang (University of Liverpool)
Reinforcement LearningSequential
🎯 What it does: This paper analyzes the theoretical properties of skill learning in unsupervised reinforcement learning from the perspective of information geometry, and proposes a new separability metric (LSEPIN) and a skill learning objective (WSEP) and algorithm (PWSEP) based on Wasserstein distance, to enhance the diversity, distinguishability, and adaptability of skills to downstream tasks.
Task Planning for Visual Room Rearrangement under Partial Observability
Karan Mirakhor (TCS Research), Brojeshwar Bhowmick (TCS Research)
Robotic IntelligenceGraph Neural NetworkLarge Language ModelReinforcement LearningGraphBenchmark
🎯 What it does: A modular task planner is proposed, which achieves user-specified tidy room states by planning object search and reordering sequences through visual perception and common-sense reasoning in partially observable environments.
Task structure and nonlinearity jointly determine learned representational geometry
Matteo Alleman (Columbia University), Stefano Fusi (Columbia University)
Representation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper studies how nonlinear activation functions affect representation learning in single hidden layer networks under different input/output geometries.
TD-MPC2: Scalable, Robust World Models for Continuous Control
Nicklas Hansen (University of California San Diego), Xiaolong Wang (University of California San Diego)
OptimizationRobotic IntelligenceReinforcement LearningWorld ModelSequential
🎯 What it does: TD-MPC 2 has been developed, a model-based RL algorithm for trajectory optimization on an implicit world model, capable of efficient learning in cross-domain multi-task continuous control tasks using a single hyperparameter.
Teach LLMs to Phish: Stealing Private Information from Language Models
Ashwinee Panda (Princeton University), Prateek Mittal (Princeton University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a training data extraction attack called 'Neural Phishing', where an attacker inserts a small number of poisoned samples disguised as normal text during the pre-training or fine-tuning phase, causing the language model to learn to memorize and leak high-entropy sensitive information from users (such as credit card numbers) during inference.
Teaching Arithmetic to Small Transformers
Nayoung Lee (University of Wisconsin Madison), Dimitris Papailiopoulos (University of Wisconsin Madison)
TransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: The study explores how to efficiently learn basic arithmetic operations (addition, multiplication, square root, sine, etc.) starting from random initialization on small Transformer models (such as NanoGPT, GPT-2) using only the next word prediction as the target.
Teaching Language Models to Hallucinate Less with Synthetic Tasks
Erik Jones (University of California Berkeley), Ece Kamar (Microsoft Research)
GenerationData SynthesisOptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: By designing synthetic tasks that can easily evaluate hallucinations and optimizing the system prompts for LLMs on these tasks, we further transfer to real abstract summarization tasks, thereby reducing the model's hallucination rate in generation.
Teaching Large Language Models to Self-Debug
Xinyun Chen (Google DeepMind), Denny Zhou (Google DeepMind)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes the SELF-DEBUGGING method, training large language models to debug their generated code through self-explanation (rubber duck debugging) and further iteratively correct errors.
TEDDY: Trimming Edges with Degree-based Discrimination Strategy
Hyunjin Seo (Korea Advanced Institute of Science and Technology), Eunho Yang (Korea Advanced Institute of Science and Technology)
OptimizationComputational EfficiencyKnowledge DistillationGraph Neural NetworkGraph
🎯 What it does: We propose TEDDY, a one-shot graph edge sparsification framework based on node degree information, which achieves parameter sparsification during training through ℓ₀ projection gradient descent and maintains performance through knowledge distillation.
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
Qingru Zhang (Georgia Institute of Technology), Tuo Zhao (Georgia Institute of Technology)
GenerationOptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A post-attention steering method called PASTA is proposed and implemented, which utilizes user-emphasized markers such as bold/italic in the text to reweight the multi-head attention of large language models (LLMs) to guide the model's focus on user-specified information during inference.
TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
Defu Cao (University of Southern California), Yan Liu (University of Southern California)
TransformerLarge Language ModelPrompt EngineeringMultimodalityTime SeriesFinance Related
🎯 What it does: A time series prediction framework TEMPO based on a pre-trained generative Transformer is proposed, utilizing the trend, seasonality, and residual decomposition of time series, and fine-tuning the model through soft prompts to achieve zero-shot transfer and multimodal fusion prediction.
Temporal Generalization Estimation in Evolving Graphs
Bin Lu (Shanghai Jiao Tong University), Shiyu Liang (Shanghai Jiao Tong University)
Recurrent Neural NetworkGraph Neural NetworkGraph
🎯 What it does: This paper proposes a self-supervised temporal generalization estimation method called SMART, aimed at monitoring and predicting the decline in generalization performance of GNNs after graph evolution.
Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks
Greg Yang (xAI), Soufiane Hayou (Simons Institute)
OptimizationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a DepthµP parameterization method that allows for the infinite expansion of the depth of residual networks while maintaining feature learning and feature diversity, and ensuring training stability.
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Sam Toyer (University of California Berkeley), Stuart Russell (University of California Berkeley)
Explainability and InterpretabilityAdversarial AttackTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: An online game called Tensor Trust is proposed to collect and study human-generated prompt injection attacks and defenses, and a corresponding benchmark evaluation is constructed.
Test-time Adaptation against Multi-modal Reliability Bias
Mouxing Yang (Sichuan University), Xi Peng (Tianjin University)
Domain AdaptationTransformerVideoMultimodalityBenchmarkAudio
🎯 What it does: Proposed and implemented a testing-time adaptive method for multimodal reliability bias (READ).
Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models
Shuai Zhao (University of Technology Sydney), Yi Yang (Zhejiang University)
ClassificationRetrievalDomain AdaptationTransformerReinforcement LearningPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a testing-time adaptive framework RLCF based on CLIP rewards, which utilizes reinforcement learning to dynamically update the parameters of the visual-language model (VLM) on a single test sample to enhance the generalization ability of zero-shot tasks.
Test-Time Training on Nearest Neighbors for Large Language Models
Moritz Hardt (Max Planck Institute for Intelligent Systems), Yu Sun (Stanford University)
RetrievalOptimizationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposes a test-time training method that utilizes nearest neighbor retrieval results for a single gradient update of the language model (Test-Time Training on Nearest Neighbors, TTT-NN).
TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series
Chenxi Sun (Peking University), Shenda Hong (Peking University)
ClassificationAnomaly DetectionRepresentation LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningTime Series
🎯 What it does: By constructing time series embeddings and aligning LLM word embeddings, time series classification, prediction, and representation tasks are completed on frozen LLMs using soft prompts.
TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts
Hyunwook Lee (Ulsan National Institute of Science and Technology), Sungahn Ko (Ulsan National Institute of Science and Technology)
Graph Neural NetworkTransformerMixture of ExpertsTime Series
🎯 What it does: A multi-expert spatiotemporal attention model (TESTAM) is proposed, achieving adaptive spatial and temporal modeling in different traffic scenarios.
Text-to-3D with Classifier Score Distillation
Xin Yu (University of Hong Kong), XIAOJUAN QI
GenerationData SynthesisDiffusion modelNeural Radiance FieldTextMesh
🎯 What it does: This paper proposes a text-to-3D generation method that solely utilizes classifier scores (Classifier Score Distillation, CSD) and applies it to 3D geometry generation, texture synthesis, and 3D editing.
Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
Tianbao Xie (University of Hong Kong), Tao Yu (University of Hong Kong)
Robotic IntelligenceLarge Language ModelReinforcement LearningText
🎯 What it does: A framework called TEXT2REWARD is proposed, which utilizes large language models to automatically generate and shape dense reward functions, supporting the rapid construction of interpretable reward code in RL;
TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fields
Tianyu Huang (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
GenerationData SynthesisVision Language ModelGenerative Adversarial NetworkTextPoint CloudMesh
🎯 What it does: This paper proposes a conditional 3D generation model called TextField3D, which can map text or image prompts to the generation process of 3D geometry and textures, thus achieving open vocabulary text-to-3D (Text-to-3D) and image-to-3D (Image-to-3D) generation.
The Alignment Problem from a Deep Learning Perspective
Richard Ngo (OpenAI), Sören Mindermann (University of Oxford)
Reinforcement Learning from Human FeedbackReinforcement LearningReview/Survey Paper
🎯 What it does: A systematic theoretical and case analysis of the alignment challenges that may arise in artificial general intelligence (AGI) trained using self-supervised pre-training and reinforcement learning from human feedback (RLHF).
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World
Weiyun Wang (Shanghai AI Laboratory), Yu Qiao (Tsinghua University)
RecognitionObject DetectionTransformerLarge Language ModelContrastive LearningImageText
🎯 What it does: A large-scale regional-level annotated dataset AS-1B with 120 million annotations has been constructed, and a unified All-Seeing Model (ASM) has been proposed to achieve panoramic visual recognition and understanding for open-world scenarios.
The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing
Shen Nie (Renmin University of China), Chongxuan Li (Renmin University of China)
Data SynthesisDiffusion modelImageBenchmarkStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A unified probabilistic framework is proposed to describe image editing with diffusion models, demonstrating that the KL convergence of SDE during the editing process is superior to that of ODE, leading to the introduction of the SDE-Drag dragging method and the DragBench benchmark.
The Consensus Game: Language Model Generation via Equilibrium Search
Athul Paul Jacob (Massachusetts Institute of Technology), Jacob Andreas (Massachusetts Institute of Technology)
GenerationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A training-independent game-theoretic method called Consensus Game is proposed, which achieves consensus search in language model generation by solving a regularized Nash equilibrium between generation and discrimination.
The Cost of Scaling Down Large Language Models: Reducing Model Size Affects Memory before In-context Learning
Tian Jin (Massachusetts Institute of Technology), Gintare Karolina Dziugaite (Google)
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: The study investigates the impact of sparsification or model size reduction on the factual recall ability and contextual learning ability of large language models.
The Curse of Diversity in Ensemble-Based Exploration
Zhixuan Lin (Mila - Quebec AI Institute, Universite de Montreal), Aaron Courville (Mila - Quebec AI Institute, Universite de Montreal)
Reinforcement LearningSequential
🎯 What it does: This study investigates the 'diversity curse' that arises when using data-sharing diversified ensemble exploration strategies in deep reinforcement learning, where the performance of individual members is significantly lower than that of a single agent; it proposes the use of Cross-ensemble Representation Learning (CERL) auxiliary tasks to alleviate this issue and enhance the performance of the aggregated strategy.
The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Language Models
Yan Liu (Chinese University of Hong Kong), Tsung-Yi Ho (National University of Singapore)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes the Integrated Gap Gradients (IG2) method to accurately locate the social bias neurons in pre-trained language models, and based on this, designs a training-independent Bias Neuron Suppression (BNS) to achieve debiasing.
The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models
Cheng Shi (ShanghaiTech University), Sibei Yang (ShanghaiTech University)
Object DetectionSegmentationContrastive LearningImage
🎯 What it does: A label-free Zip framework based on CLIP and SAM is proposed to achieve zero-shot object detection and instance segmentation.
The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images
Nicholas Konz (Duke University), Maciej A Mazurowski
ClassificationAdversarial AttackConvolutional Neural NetworkImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: This study investigates how the intrinsic properties of datasets (intrinsic dimension and label sharpness) affect the generalization ability and adversarial robustness of neural networks across different image domains (natural images and medical images).
The Effective Horizon Explains Deep RL Performance in Stochastic Environments
Cassidy Laidlaw (University of California), Anca Dragan (University of California)
Reinforcement LearningTabular
🎯 What it does: The study explains the efficiency of deep RL in stochastic environments through random exploration and function approximation, proposes the SQIRL algorithm, and proves that its sample complexity is only exponential in the effective horizon under low effective horizons.
The Effectiveness of Random Forgetting for Robust Generalization
Vijaya Raghavan T Ramkumar (Eindhoven University of Technology), Elahe Arani (Wayve)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A novel adversarial training framework called FOMO is proposed, which alleviates robust overfitting and enhances the model's robust generalization ability by periodically forgetting part of the weights (random reinitialization) and relearning.
The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks.
Aaron Spieler (University of Tübingen), Anna Levina (Max Planck Institute for Biological Cybernetics)
Recurrent Neural NetworkSequentialAudio
🎯 What it does: The Expressive Leaky Memory (ELM) neuron model is proposed, which can fit the input-output mapping of biological neurons with high precision using very few parameters (only in the thousands) and achieves excellent performance on various long-sequence tasks.
The Expressive Power of Low-Rank Adaptation
Yuchen Zeng (University of Wisconsin Madison), Kangwook Lee (University of Wisconsin Madison)
TransformerTabular
🎯 What it does: This paper provides a theoretical analysis of the expressive capability of Low-Rank Adaptors (LoRA) in fully connected networks and Transformers, presenting the minimum rank threshold and error upper bound, and proving that under the rank condition, it can perfectly approximate any target model.
The Expressive Power of Transformers with Chain of Thought
William Merrill (New York University), Ashish Sabharwal (Allen Institute for AI)
GenerationTransformerChain-of-Thought
🎯 What it does: This study investigates the reasoning ability of transformers in generating chain-of-thought and provides upper and lower bounds on expressive power for different numbers of generation steps (logarithmic, linear, polynomial).
The False Promise of Imitating Proprietary Language Models
Arnav Gudibande (University of California Berkeley), Dawn Song (University of California Berkeley)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: In this paper, the authors fine-tune various open-source language models (such as GPT-2, LLaMA 7B/13B) by collecting outputs from ChatGPT as training data, aiming to mimic the behavior of ChatGPT and evaluate whether 'imitation learning' can achieve the functionality of closed-source models at a lower cost.
The Generalization Gap in Offline Reinforcement Learning
Ishita Mediratta (Meta), Roberta Raileanu (Meta)
TransformerReinforcement LearningTabularBenchmark
🎯 What it does: This study investigates the generalization ability of offline reinforcement learning when facing new environments and proposes benchmarks for offline datasets based on Procgen and WebShop.
The Generative AI Paradox: “What It Can Create, It May Not Understand”
Peter West (University of Washington), Yejin Choi (University of Washington)
GenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelTextMultimodality
🎯 What it does: This paper explores the 'generation-understanding paradox' of generative AI, where models surpass humans in generation tasks but lag behind in understanding tasks (discrimination and questioning); this phenomenon is validated through experiments in both language and visual modalities.
The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
Michael Zhang (Stanford University), Christopher Re
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningImageText
🎯 What it does: This paper introduces Hedgehog, a learnable linear attention mechanism that can approximate the 'sharpness' and 'monotonicity' of softmax attention through MLP while maintaining O(nd²) complexity, significantly restoring the performance of standard Transformers in three scenarios: zero-shot training, fine-tuning transfer, and pre-training transfer.