NeurIPS 2024 Papers — Page 38
Conference on Neural Information Processing Systems · 4035 papers
Trading Place for Space: Increasing Location Resolution Reduces Contextual Capacity in Hippocampal Codes
Spencer Rooke (University of Pennsylvania), Vijay Balasubramanian (University of Pennsylvania)
🎯 What it does: This study investigates the trade-off between the encoding capacity and positional resolution of hippocampal place cells in different environments, revealing its exponential growth characteristics.
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning
Yeongbin Seo (Yonsei University), Jinyoung Yeo (Yonsei University)
Meta LearningTransformerLarge Language ModelTextTime SeriesBenchmark
🎯 What it does: This paper proposes the Train-Attention-Augmented Language Model (TAALM), a continual knowledge learning method that predicts token importance through meta-learning and dynamically weights it during training, and establishes a new benchmark called LAMA-CKL.
Training an Open-Vocabulary Monocular 3D Detection Model without 3D Data
Rui Huang (Tsinghua University), Gao Huang (Tsinghua University)
Object DetectionDepth EstimationAutonomous DrivingLarge Language ModelImage
🎯 What it does: Train an open vocabulary monocular 3D detection model using only RGB images, automatically generate pseudo-labels, and train the detector.
Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming
Lorenzo Orecchia (University of Chicago), Xue Geng (Northeastern University of China)
OptimizationConvolutional Neural NetworkGaussian SplattingImage
🎯 What it does: This paper proposes the VISPA algorithm, which combines Gaussian variational inference with low-rank semidefinite programming to address the training problem of binary neural networks (BNN).
Training Compute-Optimal Protein Language Models
Xingyi Cheng (BioMap Research), Le Song (MBZUAI)
Protein Structure PredictionTransformerLarge Language ModelSupervised Fine-TuningBiomedical Data
🎯 What it does: This study investigates how to optimally train protein language models under a fixed computational budget, proposing scaling laws for CLM and MLM, and providing an optimal allocation scheme for model and data size.
Training Data Attribution via Approximate Unrolling
Juhan Bae (University of Toronto), Roger Baker Grosse
OptimizationData-Centric LearningImageTabular
🎯 What it does: A new training data attribution method called SOURCE is proposed, which combines implicit differentiation and unrolling techniques of the training process. By segmenting the training trajectory and making static approximations of the gradients and Hessians, an efficient approximate solution is obtained.
Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
Hongru Yang (University of Texas at Austin), Yingbin Liang (Ohio State University)
RecognitionTransformerSequential
🎯 What it does: This paper analyzes the training dynamics of a single-layer Transformer in the task of recognizing word co-occurrences through gradient flow analysis, and presents two-phase characteristics of the training process.
Training for Stable Explanation for Free
Chao Chen (Harbin Institute of Technology), Sihong Xie
Explainability and InterpretabilityAdversarial AttackGraph Neural NetworkMultimodalityTabular
🎯 What it does: A new metric called 'explanation ranking thickness' is proposed to measure the robustness of top-k important features in model explanations (especially gradient-based explanations) against perturbations, and based on this metric, the R2ET training method is designed.
Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy
Hancheng Ye (Fudan University), Bo Zhang (Fudan University)
GenerationComputational EfficiencyDiffusion modelImageVideoText
🎯 What it does: This paper proposes a training-free adaptive diffusion acceleration method called AdaptiveDiffusion, which can dynamically skip certain noise prediction steps while maintaining the final generation quality, thereby significantly reducing inference latency.
Training-Free Open-Ended Object Detection and Segmentation via Attention as Prompts
Zhiwei Lin (Peking University), Zhi Tang (Peking University)
Object DetectionSegmentationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: A training-free framework called VL-SAM is proposed, which connects visual language models with the Segment-Anything model using attention maps to achieve open-set object detection and segmentation.
TrajCLIP: Pedestrian trajectory prediction method using contrastive learning and idempotent networks
Pengfei Yao (University of Chinese Academy of Sciences), Zhaoqi Wang
Object TrackingGenerationTransformerGenerative Adversarial NetworkContrastive LearningTime SeriesSequential
🎯 What it does: This paper proposes TrajCLIP, a pedestrian trajectory prediction method that combines contrastive learning with isometric generative networks, capable of maintaining consistency in the feature space of historical and future trajectories, and achieving more accurate trajectory predictions through the fusion of temporal and frequency domain features.
Trajectory Data Suffices for Statistically Efficient Learning in Offline RL with Linear $q^\pi$-Realizability and Concentrability
Volodymyr Tkachuk (University of Alberta), Csaba Szepesvari
Reinforcement LearningSequential
🎯 What it does: This paper studies how to effectively learn an approximately optimal policy using trajectory data in offline reinforcement learning under the assumptions of linear qπ realizability and concentration.
Trajectory Diffusion for ObjectGoal Navigation
Xinyao Yu (Chinese Academy of Sciences), Shuqiang Jiang (Chinese Academy of Sciences)
OptimizationRobotic IntelligenceTransformerDiffusion modelPoint Cloud
🎯 What it does: A trajectory planning method based on diffusion models (Trajectory Diffusion) is proposed, which generates future trajectory sequences through semantic maps and target information in the ObjectGoal navigation task, guiding the agent to efficiently reach the target.
Trajectory Flow Matching with Applications to Clinical Time Series Modelling
Xi Zhang (McGill University), Alexander Tong (Mila - Quebec AI Institute)
Flow-based ModelTime SeriesBiomedical DataElectronic Health RecordsStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A simulation-agnostic training framework called Trajectory Flow Matching (TFM) is proposed for efficiently learning Neural Stochastic Differential Equations (Neural SDE) and modeling clinical time series.
TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration
Yiwei Guo (Shenzhen Institutes of Advanced Technology), Yali Wang (Shenzhen Institutes of Advanced Technology)
ClassificationRecognitionKnowledge DistillationTransformerMixture of ExpertsVision Language ModelImageMultimodality
🎯 What it does: The TransAgent framework is proposed, which significantly enhances generalization ability in low-sample scenarios by transferring knowledge from multimodal expert models to visual-language foundation models like CLIP through the collaboration of multi-source heterogeneous agents.
Transcendence: Generative Models Can Outperform The Experts That Train Them
Edwin Zhang (OpenAI), eran malach
GenerationTransformerText
🎯 What it does: This study investigates whether generative models can surpass their training experts under low-temperature sampling, and validates this through experiments using a trained autoregressive Transformer for chess games, question answering, and synthetic classification.
Transcoders find interpretable LLM feature circuits
Jacob Dunefsky (Yale University), Neel Nanda
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This study investigates the transcoder as a sparse approximation of the MLP sublayer in Transformers for fine-grained circuit analysis, validating its interpretability and accuracy on GPT-2 and small Pythia models.
Transductive Active Learning: Theory and Applications
Jonas Hübotter (ETH Zürich), Andreas Krause (ETH Zürich)
OptimizationImage
🎯 What it does: This paper studies the theory and application of transductive active learning, proposing ITL and VTL decision rules based on the principle of minimizing uncertainty in the target space, and proving their convergence.
Transductive Learning is Compact
Julian Asilis (University of Southern California), Shang-Hua Teng (University of Southern California)
🎯 What it does: This paper proposes and proves a new 'compactness' property in transductive learning, where the sample complexity of the hypothesis class can be completely determined by the sample complexity of all its finite projections.
Transfer Learning for Diffusion Models
Yidong Ouyang (University of California), Guang Cheng (University of California)
GenerationDomain AdaptationDiffusion modelScore-based ModelTime SeriesBiomedical DataElectrocardiogram
🎯 What it does: This paper proposes the Transfer Guided Diffusion Process (TGDP), which utilizes the density ratio estimation of a pre-trained diffusion model from the source domain and a classifier from the target domain to achieve the transfer of generative models with limited samples in the target domain.
Transfer Learning for Latent Variable Network Models
Akhil Jalan (University of Texas at Austin), Purnamrita Sarkar (University of Texas at Austin)
Domain AdaptationOptimizationGraph Neural NetworkGraph
🎯 What it does: This paper studies a transfer learning method for estimating target networks using source network data in latent variable network models.
Transfer Q-star : Principled Decoding for LLM Alignment
Souradip Chakraborty (University of Maryland), Furong Huang (University of Maryland)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes Transfer Q*, which utilizes existing aligned baseline models to directly or indirectly transfer the estimation of the optimal Q*, thereby achieving safe alignment of LLMs during the inference phase.
Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness
Mingyuan Fan (East China Normal University), Yaliang Li (Alibaba Group)
OptimizationAdversarial AttackTransformerImage
🎯 What it does: Derived the upper bound theory of adversarial sample transferability and proposed an optimizable attack method TPA based on this theory.
Transferable Adversarial Attacks on SAM and Its Downstream Models
Song Xia (Nanyang Technological University), Xudong Jiang (Nanyang Technological University)
SegmentationAdversarial AttackMeta LearningImageBiomedical DataComputed Tomography
🎯 What it does: The research utilizes the publicly available Segment Anything Model (SAM) as a surrogate and proposes a transferable adversarial attack method called UMI-GRAT, which can effectively attack SAM and its fine-tuned downstream models without accessing downstream task data and models.
Transferable Boltzmann Generators
Leon Klein (Freie Universitat Berlin), Frank Noe
GenerationData SynthesisDrug DiscoveryGraph Neural NetworkFlow-based ModelGraph
🎯 What it does: This study investigates a transferable Boltzmann generator, proposing a framework based on continuous normalizing flows, and validates its zero-shot generation capability on alanine dipeptide and 2AA dipeptide.
Transferring disentangled representations: bridging the gap between synthetic and real images
Jacopo Dapueto (Università degli studi di Genova), Francesca Odone (Università degli studi di Genova)
Domain AdaptationRepresentation LearningAuto EncoderImage
🎯 What it does: The study will transfer separable representations learned from synthetic data to real images and systematically evaluate the transfer effects; at the same time, it proposes a classifier-free, intervention-based OMES evaluation metric.
Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization
Omar Montasser (Yale University), Emmanuel Abbe (EPFL and Apple)
Domain AdaptationOptimization
🎯 What it does: This paper studies learning under distribution shifts, proposing a new framework based on data transformations. It explores learning scenarios with known and unknown transformation classes and establishes theoretical guarantees for learning rules and algorithms.
Transformer Doctor: Diagnosing and Treating Vision Transformers
Jiacong Hu (Zhejiang University), Zunlei Feng (Zhejiang University)
ClassificationTransformerImage
🎯 What it does: This paper proposes the Transformer Doctor framework, which diagnoses and treats the errors in parallel information integration within Vision Transformers.
Transformers are Minimax Optimal Nonparametric In-Context Learners
Juno Kim (Center for Advanced Intelligence Project RIKEN), Taiji Suzuki (Center for Advanced Intelligence Project RIKEN)
Representation LearningMeta LearningTransformerSequential
🎯 What it does: Theoretical analysis of context learning (ICL) of pre-trained Transformers (deep networks + single-layer linear attention) in non-parametric regression tasks is conducted, providing upper bounds for approximation error and generalization error. It is proven that in Besov spaces, anisotropic Besov spaces, and γ-smooth function classes, the lower bounds can be achieved or exceeded, illustrating the role of pre-training task diversity and representation learning in ICL.
Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models
Chengshuai Shi (University of Virginia), Cong Shen (University of Virginia)
TransformerSupervised Fine-TuningReinforcement LearningSequential
🎯 What it does: This paper studies the in-context game-playing (ICGP) capability of pre-trained Transformers in two-player zero-sum Markov games, providing theoretical guarantees and experimentally validating the achievement of Nash equilibrium.
Transformers Can Do Arithmetic with the Right Embeddings
Sean Michael McLeish, Tom Goldstein (Carnegie Mellon University)
TransformerTabular
🎯 What it does: A new positional embedding method called Abacus Embeddings is proposed to address the issue of Transformers being unable to accurately locate each digit in large numerical arithmetic (especially addition). The model's performance is further enhanced through the use of a looped Transformer and input injection.
Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
Deqing Fu (University of Southern California), Vatsal Sharan (University of Southern California)
OptimizationTransformerTabular
🎯 What it does: The study investigates how Transformer achieves linear regression in context learning without parameter updates, finding that its internal implementation is similar to second-order optimization methods.
Transformers need glasses! Information over-squashing in language tasks
Federico Barbero (University of Oxford), Petar Veličković (Google DeepMind)
TransformerPrompt EngineeringSequential
🎯 What it does: This paper studies the information propagation behavior of the decoder-only Transformer in the next word prediction task through theoretical signal propagation analysis and empirical experiments, revealing two distortion phenomena: 'Representational Collapse' and 'Over-squashing'.
Transformers on Markov data: Constant depth suffices
Nived Rajaraman (University of California Berkeley), Michael Gastpar (École Polytechnique Fédérale de Lausanne)
TransformerSequential
🎯 What it does: This paper studies the ability of Transformers to learn conditional k-gram models from sequences sampled from k-th order Markov processes through experimental and theoretical analysis, and proves that a constant-layer Transformer can represent this model.
Transformers Represent Belief State Geometry in their Residual Stream
Adam Shai, Paul M. Riechers (Simplex)
TransformerSequential
🎯 What it does: This paper demonstrates, through theoretical framework and experimental validation, that the Bayesian belief state geometry based on the data generation process is linearly embedded in the residual flow of the Transformer, and can even capture complex fractal structures.
Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models
Aviv Bick (Carnegie Mellon University), Albert Gu (Cartesia.ai)
Computational EfficiencyKnowledge DistillationTransformerText
🎯 What it does: This paper proposes a three-stage knowledge distillation framework called MOHAWK, which distills a pre-trained Transformer (Phi-1.5) into a state space model (Mamba-2, referred to as Phi-Mamba) with sub-quadratic time complexity, achieving performance comparable to or even better than that of powerful Transformers using only 3B tokens (<1% of the training volume).
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner
Hanwen Zhong (Beihang University), Yunhong Wang (Beihang University)
ClassificationSegmentationOptimizationTransformerMixture of ExpertsImage
🎯 What it does: This paper proposes an efficient multi-task learning framework called EMTAL, which decomposes the pre-trained Vision Transformer into low-rank Mixture-of-Experts and fine-tunes it using LoRA. It then achieves asynchronous task optimization through a Quality Retention (QR) mechanism, and finally employs routing decay for parameter reparameterization, resulting in a unified model with no additional inference cost.
Transition Constrained Bayesian Optimization via Markov Decision Processes
Jose Pablo Folch (Imperial College London), Mojmir Mutny
OptimizationReinforcement LearningOrdinary Differential Equation
🎯 What it does: A Bayesian optimization framework with transfer constraints is proposed, utilizing Markov Decision Processes (MDP) to plan query strategies over the entire experimental cycle, focusing on the problem of maximum value identification.
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le (Shanghai Jiao Tong University), Michael Zeng (Microsoft)
Knowledge DistillationTransformerAudio
🎯 What it does: Develop an end-to-end speech-to-speech translation system, TransVIP, which can directly translate source language speech into target language speech while maintaining the speaker's voice characteristics and timing (speech rate, pauses), suitable for scenarios such as video dubbing.
Trap-MID: Trapdoor-based Defense against Model Inversion Attacks
Zhen-Ting Liu (National Taiwan University), Shang-Tse Chen (National Taiwan University)
Safty and PrivacyAdversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: Trap-MID is designed on deep learning models to mislead model inversion attacks by injecting trapdoors, thereby protecting privacy.
Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustness
Vaclav Voracek
Computational EfficiencyAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: An optimal randomized Clopper-Pearson confidence interval and an adaptive sampling method based on confidence sequences are proposed for statistical estimation in randomized smoothing, significantly reducing the required number of forward propagations.
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Anay Mehrotra (Yale University), Amin Karbasi (Robust Intelligence)
Explainability and InterpretabilityAdversarial AttackTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes and implements the Tree of Attacks with Pruning (TAP), an automated, black-box, and interpretable LLM jailbreak method that generates diverse prompts through an attacker LLM and evaluates and prunes them using an evaluator LLM, ultimately achieving a jailbreak of the target LLM.
Treeffuser: probabilistic prediction via conditional diffusions with gradient-boosted trees
Nicolas Beltran-Velez (Columbia University), David Blei (Columbia University)
Diffusion modelTabularStochastic Differential Equation
🎯 What it does: A conditional diffusion model based on gradient boosting trees, Treeffuser, is proposed for probabilistic prediction of tabular data.
TreeVI: Reparameterizable Tree-structured Variational Inference for Instance-level Correlation Capturing
Junxi Xiao (Sun Yat-sen University), Qinliang Su (Guangdong Key Laboratory of Big Data Analysis and Processing)
Recommendation SystemGraph Neural NetworkGraphTabular
🎯 What it does: Proposes TreeVI (Tree-structured Variational Inference) and MTreeVI (Tree Mixture Model) to approximate the posterior distribution with instance-level correlations, and implements a parallelizable matrix form reparameterization.
Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization
Chengtao Jian (Tongji University), Yang Jiao (Tongji University)
Domain AdaptationOptimizationRecurrent Neural NetworkLarge Language ModelContrastive LearningTime Series
🎯 What it does: A time series OOD generalization framework TTSO based on three-layer learning is proposed, and fine-tuning of LLM is implemented.
TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives
Maitreya Patel (Arizona State University), Yezhou Yang (Arizona State University)
ClassificationRetrievalTransformerLarge Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: By synthesizing hard negative image-text pairs, we improve the combinatorial reasoning ability of CLIP and propose the TripletCLIP pre-training strategy.
Truncated Variance Reduced Value Iteration
Yujia Jin (Stanford University), Jiayi Wang (Stanford University)
OptimizationReinforcement Learning
🎯 What it does: A faster randomized algorithm is proposed for computing ε-optimal policies in discounted Markov decision processes, providing improved time complexity in both sampling and offline settings.
Truth is Universal: Robust Detection of Lies in LLMs
Lennart Bürger (Heidelberg University), Boaz Nadler (Weizmann Institute of Science)
ClassificationLarge Language ModelText
🎯 What it does: This study investigates the linear representation of truth and falsehood within large language models (LLMs), discovering a two-dimensional truth subspace. Based on this, a lie detection method called TTPD is proposed, which can identify lies from LLMs in unseen topics, different sentence structures, and real-world contexts.
Truthful High Dimensional Sparse Linear Regression
Liyang Zhu (King Abdullah University of Science and Technology), Di Wang (King Abdullah University of Science and Technology)
OptimizationSafty and PrivacyGaussian Splatting
🎯 What it does: This paper proposes a mechanism that ensures Joint Differential Privacy (JDP) in high-dimensional sparse linear regression while incentivizing most participants to report data honestly.
Truthfulness of Calibration Measures
Nika Haghtalab (University of California Berkeley), Eric Zhao (University of California Berkeley)
🎯 What it does: This paper studies the calibration metrics of sequential probability prediction and proposes a new Subsampled Smooth Calibration Error (SSCE) as an evaluation metric.
TSDS: Data Selection for Task-Specific Model Finetuning
Zifan Liu (University of Wisconsin Madison), Theodoros Rekatsinas (Apple)
OptimizationData-Centric LearningSupervised Fine-TuningText
🎯 What it does: Proposes the TSDS framework, which drives the distribution alignment and diversity balance of large-scale candidate data using representative examples, ultimately selecting high-quality training samples for task-specific fine-tuning.
TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
Benjamin Feuer (New York University), Colin White (Abacus.AI)
ClassificationOptimizationExplainability and InterpretabilityTransformerPrompt EngineeringTabularBenchmark
🎯 What it does: This paper proposes TuneTables, a context optimization method for soft prompt tuning on the pre-trained TabPFN, aimed at compressing the context of large-scale datasets, enabling the expansion of features, samples, and the number of classes, while supporting multi-objective optimization and interpretability analysis.
TurboHopp: Accelerated Molecule Scaffold Hopping with Consistency Models
Kiwoong Yoo (AIGEN Sciences), Jaewoo Kang (Korea University)
OptimizationDrug DiscoveryReinforcement LearningDiffusion modelBiomedical Data
🎯 What it does: This paper presents TurboHopp, a 3D Scaffold Hopping model based on a consistency model, designed for the rapid generation of active compound scaffolds at protein binding sites.
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
Zhenyi Lu (Huazhong University of Science and Technology), Yu Cheng (Chinese University of Hong Kong)
GenerationOptimizationTransformerMixture of ExpertsText
🎯 What it does: Proposes the Twin-Merging method, which separates shared and exclusive knowledge and dynamically merges them based on input during inference, significantly improving performance in multi-task model merging.
Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning
Shuguang Yu (Shanghai University of Finance and Economics), Chengchun Shi (London School of Economics and Political Science)
Reinforcement LearningBiomedical DataElectronic Health Records
🎯 What it does: Proposes a two-way debiasing estimation method that utilizes the two-way unobserved confounding assumption for offline policy evaluation (OPE) debiasing.
Typicalness-Aware Learning for Failure Detection
Yijun Liu (Harbin Institute of Technology), Jingyong Su (Harbin Institute of Technology)
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes Typicalness-Aware Learning (TAL), which alleviates the overconfidence problem of deep networks and enhances failure detection performance by dynamically adjusting the logit magnitude.
U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers
Yuchuan Tian (Peking University), Yunhe Wang (Huawei)
GenerationComputational EfficiencyTransformerDiffusion modelImage
🎯 What it does: A diffusion Transformer based on U-shaped structure (U-DiT) is proposed, which reduces computational cost and improves generation quality by downsampling visual tokens in self-attention.
UDC: A Unified Neural Divide-and-Conquer Framework for Large-Scale Combinatorial Optimization Problems
Zhi Zheng (National University of Singapore), Zhenkun Wang (Southern University of Science and Technology)
OptimizationGraph Neural NetworkReinforcement Learning
🎯 What it does: A unified neural divide-and-conquer framework (UDC) is proposed, capable of solving large-scale combinatorial optimization problems without relying on problem-specific heuristics.
UDON: Universal Dynamic Online distillatioN for generic image representations
Nikolaos-Antonios Ypsilantis (Czech Technical University in Prague), Ondrej Chum (Czech Technical University in Prague)
Knowledge DistillationRepresentation LearningTransformerImage
🎯 What it does: A general image embedding learning framework UDON based on multi-teacher online distillation is proposed, which jointly trains domain-specific teachers and a unified student using a shared backbone network, and dynamically samples to accelerate the convergence of hard-to-learn domains.
UDPM: Upsampling Diffusion Probabilistic Models
Shady Abu-Hussein (Tel Aviv University), Raja Giryes (Tel Aviv University)
GenerationData SynthesisComputational EfficiencyDiffusion modelImage
🎯 What it does: Proposes the Upsampling Diffusion Probabilistic Model (UDPM), which incorporates downsampling and upsampling in the diffusion process, allowing for high-quality image generation in just 3 steps.
UGC: Universal Graph Coarsening
Mohit Kataria (Indian Institute of Technology Delhi), Jayadeva Jayadeva
CompressionGraph Neural NetworkGraph
🎯 What it does: A global graph compression framework UGC is proposed, which utilizes enhanced features of node attributes and adjacency information to quickly generate a compression matrix through Locality Sensitive Hashing (LSH), thereby compressing large graphs to a smaller scale while maintaining spectral similarity and ϵ-similarity.
Ultrafast classical phylogenetic method beats large protein language models on variant effect prediction
Sebastian Prillo (University of California), Yun S. Song (University of California)
Computational EfficiencyProtein Structure PredictionBiomedical Data
🎯 What it does: Developed a near-linear time method called FastCherries to estimate 'cherry' pairs and branch lengths in MSA, combined with CherryML to directly estimate the LG model and a more granular SiteRM site rate matrix on MSA, avoiding the computational bottleneck of full tree reconstruction.
UltraPixel: Advancing Ultra High-Resolution Image Synthesis to New Peaks
Jingjing Ren (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
GenerationData SynthesisDiffusion modelAuto EncoderImage
🎯 What it does: Trained and released a model called UltraPixel that can generate ultra-high-resolution images at multiple resolutions (1K to 6K) in one go.
UMB: Understanding Model Behavior for Open-World Object Detection
Xing Xi (South China University of Technology), Ronghua Luo (South China University of Technology)
Object DetectionTransformerLarge Language ModelVision Language ModelImage
🎯 What it does: A framework named UMB is proposed to understand the model's prediction behavior for unknown categories in the open-world object detection (OWOD) task and to generate textual attribute descriptions for unlabeled objects.
UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models
Jiachen Liang (Institute of Computing Technology, Chinese Academy of Sciences), Xilin Chen (Institute of Computing Technology, Chinese Academy of Sciences)
Domain AdaptationTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: This paper proposes an unsupervised, multi-domain feature calibration method called UMFC, aimed at enhancing the generalization ability of CLIP on multi-domain unlabeled data.
Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in LLMs
Zhiyuan Hu (National University of Singapore), Bryan Hooi (National University of Singapore)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: An Uncertainty of Thoughts (UoT) algorithm is designed to enable large language models to actively acquire information by perceiving their own uncertainty and asking questions, thereby improving the completion rate of information retrieval and decision-making tasks.
Uncertainty-aware Fine-tuning of Segmentation Foundation Models
Kangning Liu (New York University), Carlos Fernandez-Granda (New York University)
SegmentationSupervised Fine-TuningImage
🎯 What it does: A fine-tuning framework called SUM is proposed based on uncertainty perception to enhance the segmentation quality of SAM on complex structured images while maintaining its generality.
Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions
Rui Yang (University of Science and Technology of China), Bin Li (University of Science and Technology of China)
Robotic IntelligenceReinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes a robust algorithm TRACER based on variational Bayesian inference for offline reinforcement learning to handle data corruption (state, action, reward, dynamics) and improve performance in interference-free environments.
Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast
Chufan Shi (Tsinghua University), Yu Meng (University of Virginia)
OptimizationTransformerMixture of ExpertsContrastive LearningText
🎯 What it does: This study investigates the potential contributions of inactive experts in the Mixture-of-Experts (MoE) model and proposes a self-contrastive decoding method (SCMoE) that enhances the prediction of the next token by leveraging contrasts from different routing strategies.
Unconditional stability of a recurrent neural circuit implementing divisive normalization
Shivang Rawat (New York University), Stefano Martiniani (New York University)
ClassificationOptimizationRecurrent Neural NetworkImageSequential
🎯 What it does: A recurrent neural network model named ORGaNICs is proposed and analyzed, which achieves neural dynamical stability through Divisive Normalization and can learn in continuous time.
Uncovering Safety Risks of Large Language Models through Concept Activation Vector
Zhihao Xu (Renmin University of China), Xiting Wang (Renmin University of China)
Safty and PrivacyAdversarial AttackLarge Language ModelPrompt EngineeringText
🎯 What it does: This study investigates the security risks of large language models and proposes the Security Concept Activation Vector (SCAV) framework, which enables embedding-level and prompt-level attacks by interpreting the model's security mechanisms.
Uncovering the Redundancy in Graph Self-supervised Learning Models
Zhibiao Wang (Beihang University), Chunming Hu (Beihang University)
ClassificationOptimizationGraph Neural NetworkGraph
🎯 What it does: This paper studies the redundancy of graph self-supervised learning models and proposes a fine-tuning framework called SLIDE, which significantly reduces the number of adjustable parameters while maintaining or improving downstream node classification performance.
Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense
Rui Min (Hong Kong University of Science and Technology), Minhao Cheng (Pennsylvania State University)
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This study investigates the security of post-hoc models, proposing Re-tuning Attacks (RA) and Query-based Re-activation Attacks (QRA). It analyzes linear mode connectivity to reveal the superficial security of existing post-hoc defenses and introduces Path-Aware Minimization (PAM) to enhance post-hoc robustness.
Understanding and Improving Adversarial Collaborative Filtering for Robust Recommendation
Kaike Zhang (Chinese Academy of Sciences), Xueqi Cheng (Chinese Academy of Sciences)
Recommendation SystemAdversarial AttackTabular
🎯 What it does: This paper theoretically demonstrates the advantages of Adversarial Collaborative Filtering (ACF) in enhancing the performance and robustness of recommendation systems, and further proposes a user embedding scale adaptive adversarial perturbation magnitude allocation method called PamaCF.
Understanding and Improving Training-free Loss-based Diffusion Guidance
Yifei Shen (Microsoft Research Asia), Dongsheng Li
GenerationData SynthesisOptimizationDiffusion modelImage
🎯 What it does: This paper studies the training of an unsupervised loss-guided diffusion guidance method and conducts an in-depth analysis of its mechanisms and limitations at both theoretical and experimental levels.
Understanding and Minimising Outlier Features in Transformer Training
Bobby He (ETH Zurich), Thomas Hofmann (ETH Zurich)
OptimizationTransformerLarge Language ModelText
🎯 What it does: This paper systematically studies the occurrence of 'Outlier Features (OFs)' during the training process of Transformers, proposing two quantifiable metrics (Kurtosis and Max-Median Ratio). It analyzes the impact of normalization layers and optimizers on OFs and introduces an Outlier Protected (OP) Transformer block that does not use traditional normalization, along with a SOAP optimizer based on non-diagonal preconditioning. The effectiveness of these methods in reducing OFs while maintaining training convergence speed is demonstrated, further validating a significant improvement in quantization performance (int8 perplexity) in models ranging from 125M to 7B parameters.
Understanding Bias in Large-Scale Visual Datasets
Boya Zeng (University of Pennsylvania), Zhuang Liu (Meta)
ClassificationObject DetectionSegmentationExplainability and InterpretabilityConvolutional Neural NetworkTransformerVision Language ModelDiffusion modelImage
🎯 What it does: A framework was constructed to apply various image transformations (semantic, structural, boundary, color, frequency, etc.) to large-scale visual datasets, classify the generated transformation results, quantify the contribution of different information dimensions to dataset bias, and further explain semantic bias through object-level queries and natural language analysis.
Understanding Emergent Abilities of Language Models from the Loss Perspective
Zhengxiao Du (Tsinghua University), Jie Tang (Tsinghua University)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study investigates the predictive ability of pre-training loss on the performance of large language models in downstream tasks and redefines emergence capability from the perspective of loss.
Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure
Xiang Li (University of Michigan), Qing Qu (University of Michigan)
RestorationGenerationData SynthesisKnowledge DistillationDiffusion modelImage
🎯 What it does: Analyzing the hidden Gaussian structure of diffusion models during the generalization phase, it is found that their denoisers exhibit linearity at different noise levels, and through linear distillation, they approximate nonlinear denoisers, revealing the Gaussian inductive bias of the model during generalization.
Understanding Hallucinations in Diffusion Models through Mode Interpolation
Sumukh K Aithal (Carnegie Mellon University), J Zico Kolter
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper studies the phenomenon of hallucination in diffusion models, particularly the failure modes of mode interpolation, finding that diffusion models perform smooth interpolation between adjacent data modes in the training set, thereby generating samples that are completely outside the support of the original training distribution.
Understanding Information Storage and Transfer in Multi-Modal Large Language Models
Samyadeep Basu (University of Maryland), Daniela Massiceti (Microsoft Research)
RetrievalKnowledge DistillationTransformerLarge Language ModelTextMultimodality
🎯 What it does: This study investigates the knowledge storage and transmission mechanisms of multimodal large language models and proposes a constraint-based analytical framework and editing methods.
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective
Akiyoshi Tomihari (University of Tokyo), Issei Sato (University of Tokyo)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: This paper analyzes the training dynamics of two-stage fine-tuning (first linear probing LP then fine-tuning FT) based on the Neural Tangent Kernel (NTK) theory, revealing the impact of the linear head norm on feature stability, and addresses the calibration issue caused by large norms through temperature scaling. It also extends the NTK analysis to the Low-Rank Adaptation (LoRA) method, verifying its similarity to standard fine-tuning.
Understanding Model Selection for Learning in Strategic Environments
Tinashe Handina (California Institute of Technology), Eric Mazumdar (California Institute of Technology)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper studies the relationship between model expressiveness and equilibrium performance in strategic environments, finding that larger models do not always enhance equilibrium performance. It proposes treating model selection as a strategy in games and provides corresponding algorithms.
Understanding Multi-Granularity for Open-Vocabulary Part Segmentation
Jiho Choi (KAIST), Hyunjung Shim (KAIST)
Object DetectionSegmentationTransformerVision Language ModelImage
🎯 What it does: A fine-grained part segmentation framework called PartCLIPSeg is proposed for open vocabulary, achieving accurate segmentation of unknown category parts in multi-granularity scenarios.
Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective
Haixiang Sun (ShanghaiTech University), Ye Shi (ShanghaiTech University)
ClassificationRepresentation LearningImage
🎯 What it does: This paper starts from the phenomenon of Neural Collapse in neural networks and conducts a theoretical and experimental analysis of the representational capacity of Deep Equilibrium Models (DEQ), comparing their performance with explicit networks on balanced and unbalanced datasets.
Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data
Alexander Havrilla, Wenjing Liao (Georgia Institute of Technology)
TransformerLarge Language ModelText
🎯 What it does: This paper presents the statistical estimation and approximation theory of Transformers on low-dimensional manifold data, and based on this, provides theoretical predictions and empirical validation of the scaling laws for LLM models and data.
Understanding the Differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks
Jerome Sieber (ETH Zurich), Antonio Orvieto (ELLIS Institute)
Recurrent Neural NetworkText
🎯 What it does: Proposes a Dynamical Systems Framework (DSF) that unifies attention, State Space Models (SSM), and RNN into a linear recursive form, and conducts theoretical and experimental comparisons based on this framework;
Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling
Mingze Wang (Peking University), Weinan E (Peking University)
TransformerTextSequential
🎯 What it does: A systematic analysis of the expressive power of the Transformer in three types of long sparse memory modeling tasks (fixed sparsity, adaptive sparsity, and basic sparsity) is conducted through approximation theory, deriving the quantitative contributions and interrelationships of different components (multi-head attention, positional encoding, feedforward networks) to the expressive power.
Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective
Takeshi Koshizuka (University of Tokyo), Issei Sato (University of Tokyo)
Neural Architecture SearchTabularPhysics Related
🎯 What it does: This paper analyzes the expressiveness and trainability of the Fourier Neural Operator (FNO) based on mean field theory, revealing its gradient vanishing/explosion behavior under ordered-chaotic phase transitions, and provides an initialization strategy at the chaotic boundary.
Understanding the Gains from Repeated Self-Distillation
Divyansh Pareek (University of Washington), Sewoong Oh (University of Washington)
OptimizationKnowledge DistillationTabular
🎯 What it does: This paper systematically analyzes and proves that multi-step self-distillation can significantly improve model performance in fixed design linear regression tasks. In particular, under multi-step self-distillation, the overfitting risk of the optimal model can be reduced to the level of 1/d compared to single-step self-distillation.
Understanding the Limits of Vision Language Models Through the Lens of the Binding Problem
Declan Iain Campbell (Princeton University), Taylor Whittington Webb (Microsoft Research)
Vision Language ModelImage
🎯 What it does: The study investigates the performance limitations of visual language models in multi-object scenes and explores their relationship with the binding problem.
Understanding the Role of Equivariance in Self-supervised Learning
Yifei Wang (Massachusetts Institute of Technology), Stefanie Jegelka (Technical University of Munich)
ClassificationRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This study investigates the equivariance methods in self-supervised learning and reveals their mechanism for improving downstream task performance from an information-theoretic perspective.
Understanding the Transferability of Representations via Task-Relatedness
Akshay Mehra (Tulane University), Jihun Hamm (Tulane University)
ClassificationDomain AdaptationRepresentation LearningTransformerSupervised Fine-TuningImageText
🎯 What it does: This study investigates and quantifies the transferability of pre-trained models across different downstream tasks, proposing a 'task-relatedness' metric and demonstrating it as an upper bound for transferability.
Understanding Transformer Reasoning Capabilities via Graph Algorithms
Clayton Sanford (Google Research), Vahab Mirrokni (Google DeepMind)
Computational EfficiencyGraph Neural NetworkTransformerGraphBenchmark
🎯 What it does: This paper constructs a tokenization encoding for graph problems, studying and proving the expressiveness and computational efficiency of Transformers for graph algorithm reasoning tasks at different parameter scales (depth, width, blank tokens), and empirically validating these theoretical conclusions on the GraphQA benchmark.
Understanding Transformers via N-Gram Statistics
Timothy Nguyen (Google DeepMind)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper evaluates the model's utilization of contextual information through N-gram statistical rule approximation for the next word prediction of Transformer LLM.
Understanding Visual Feature Reliance through the Lens of Complexity
Thomas FEL, Katherine Hermann (Google DeepMind)
ClassificationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: Proposed and implemented a feature complexity measurement based on V-Information, which quantifies the complexity of over 10,000 directional features in the ImageNet pre-trained ResNet50, and systematically analyzes its changes with training progress, network hierarchy, and importance.
Unelicitable Backdoors via Cryptographic Transformer Circuits
Andis Draguns (Contramont Research), Christian Schroeder de Witt (University of Oxford)
TransformerLarge Language ModelText
🎯 What it does: This study researches and implements a type of 'non-triggerable' backdoor that can be inserted into Transformer language models, and hides its behavior through compilation and encryption techniques.
Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE
Xun Zhu (Tsinghua University), Ji Wu (Tsinghua University)
ClassificationRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelMultimodalityBiomedical Data
🎯 What it does: A unified medical multimodal large language model, Uni-Med, has been constructed, supporting six tasks: question answering, visual question answering, report generation, understanding/generating location expressions, and image classification.
UniAR: A Unified model for predicting human Attention and Responses on visual content
Peizhao Li (Google Research), Vidhya Navalpakkam (Google Research)
Recommendation SystemTransformerPrompt EngineeringImageMultimodality
🎯 What it does: A unified multimodal Transformer model, UniAR, is proposed to simultaneously predict attention heat maps, scan paths, and subjective preference scores for visual content.
UniAudio 1.5: Large Language Model-Driven Audio Codec is A Few-Shot Audio Task Learner
Dongchao Yang (Chinese University of Hong Kong), Helen M. Meng
ClassificationCompressionTransformerLarge Language ModelGenerative Adversarial NetworkMultimodalityAudio
🎯 What it does: This paper proposes an audio codec (LLM-Codec) that compresses audio into the vocabulary space of LLMs and combines it with a frozen LLM to achieve cross-modal few-shot audio task learning.