ICLR 2025 Papers — Page 16
International Conference on Learning Representations · 3704 papers
How Far Are We from True Unlearnability?
Kai Ye (University of Hong Kong), Chenxiong Qian (University of Hong Kong)
Image
🎯 What it does: This paper proposes two metrics, Sharpness-Aware Learnability (SAL) and Unlearnable Distance (UD), by analyzing the model optimization process and loss landscape, to evaluate and compare the unlearnability of different Unlearnable Examples (UE) methods under single-task, multi-task, and various model architectures.
How Feature Learning Can Improve Neural Scaling Laws
Blake Bordelon, Cengiz Pehlevan
OptimizationComputational EfficiencyRepresentation LearningTabular
🎯 What it does: A solvable two-layer linear network model was developed to analyze the impact of feature learning on the scaling laws of neural networks, providing optimal computational strategies for three different task difficulties.
How Gradient descent balances features: A dynamical analysis for two-layer neural networks
Zhenyu Zhu (École Polytechnique Fédérale de Lausanne), Volkan Cevher (École Polytechnique Fédérale de Lausanne)
OptimizationKnowledge DistillationTabularOrdinary Differential Equation
🎯 What it does: This study investigates the global convergence of two-layer ReLU networks under gradient descent for learning multi-teacher neurons, proposing a dynamic analysis framework consisting of three stages: alignment, tangential growth, and local convergence.
How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings
Samuel Audia (University of Maryland), Dinesh Manocha (University of Maryland)
RestorationSuper ResolutionImageMesh
🎯 What it does: This study investigates the spectral bias problem in low-dimensional mappings of Multi-Grid Parameterization Encoding (MPE) and Fourier Feature Encoding (FFE), and provides theoretical and experimental validation of MPE's enhanced high-frequency learning capability through Neural Tangent Kernel (NTK) theory.
How Low Can You Go? Searching for the Intrinsic Dimensionality of Complex Networks using Metric Node Embeddings
Nikolaos Nakis (Yale University), Morten Mørup (Technical University of Denmark)
CompressionOptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: This study investigates how to utilize the Euclidean distance model to compress complex networks to very low dimensions without losing information, and provides efficient dimension search and verification methods.
How many samples are needed to train a deep neural network?
Pegah Golestaneh (University of Hamburg), Johannes Lederer (University of Hamburg)
Recurrent Neural NetworkImageTabular
🎯 What it does: The paper proves that the lower bound of the generalization error of deep ReLU feedforward networks decreases at a rate of 1/√n with respect to the number of samples n in the optimal case, and this rate is validated through experiments.
How Much is a Noisy Image Worth? Data Scaling Laws for Ambient Diffusion.
Giannis Daras (Massachusetts Institute of Technology), Constantinos Costis Daskalakis
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This study explores the feasibility and effectiveness of training diffusion models with a mixture of noisy images (which are low-cost to collect but of poor quality) and a small number of clear images, demonstrating that a small number of clear samples can compensate for the shortcomings of a large number of noisy samples.
How Much is Unseen Depends Chiefly on Information About the Seen
Seongmin Lee (Max Planck Institute for Security and Privacy), Marcel Boehme
TextTabular
🎯 What it does: This paper derives the exact expected expression for missing mass under a multinomial distribution setting and proposes a search-based minimum mean square error (MSE) estimator design framework utilizing the dependency between frequencies, ultimately obtaining a distribution-specific estimator that outperforms Good-Turing through a genetic algorithm.
How much of my dataset did you use? Quantitative Data Usage Inference in Machine Learning
Yao Tong (National University of Singapore), Reza Shokri (National University of Singapore)
ImageText
🎯 What it does: This study investigates how to accurately estimate the proportion of the dataset used by the target model and proposes the Dataset Usage Cardinality Inference (DUCI) framework.
How new data permeates LLM knowledge and how to dilute it
Chen Sun (Google DeepMind), Mark Sandler (Google DeepMind)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study investigates how a single new text permeates existing knowledge in large language models (LLMs) after training, leading to the so-called 'priming' effect, and constructs the Outlandish dataset specifically for researching this phenomenon. Based on this, two methods (step-stone text augmentation and ignore-topk gradient clipping) are proposed to suppress undesirable priming while maintaining the model's ability to learn new knowledge.
How to Evaluate Reward Models for RLHF
Evan Frick (University of California Berkeley), Ion Stoica (University of California Berkeley)
Reinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: This paper proposes the Preference Proxy Evaluations (PPE) benchmark for quickly assessing the correlation between reward models and the actual human preference performance of LLMs after RLHF;
How to Find the Exact Pareto Front for Multi-Objective MDPs?
Yining Li (Ohio State University), Ness Shroff (Ohio State University)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes an efficient algorithm for solving the exact Pareto front of multi-objective Markov decision processes (MO-MDP), capable of simultaneously discovering all deterministic Pareto optimal strategies and the complete Pareto front.
How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations
Siddhartha Gairola (Max Planck Institute for Informatics), Bernt Schiele (Institute of Science and Technology Austria)
ClassificationObject DetectionExplainability and InterpretabilityImage
🎯 What it does: This paper studies the impact of training details of the classification head (probe) of pre-trained models on the quality of post-hoc importance attribution methods (XAI). It finds that using binary cross-entropy (BCE) loss instead of cross-entropy (CE) significantly improves the class specificity and localization accuracy of the explanations. It also explores the use of a multi-layer interpretable MLP probe to further enhance classification performance and explanation localization.
How to Verify Any (Reasonable) Distribution Property: Computationally Sound Argument Systems for Distributions
Tal Herman (Weizmann Institute of Science), Guy N. Rothblum (Apple)
🎯 What it does: An interactive proof system is proposed, which verifies whether a distribution satisfies a given property or is close to that property by utilizing sampling access to an unknown distribution;
HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing
Mude Hui (University of California), Yuyin Zhou (ByteDance)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringImageText
🎯 What it does: A high-quality instruction-based image editing dataset (HQ-Edit) consisting of approximately 200,000 entries has been constructed, and based on this, performance improvements have been achieved for existing editing models.
HQGS: High-Quality Novel View Synthesis with Gaussian Splatting in Degraded Scenes
Xin Lin (University of California San Diego), Nuno Vasconcelos (University of California San Diego)
RestorationGenerationData SynthesisGaussian SplattingImage
🎯 What it does: To improve the quality of novel view synthesis in degraded scenarios, HQGS is proposed, which guides 3D Gaussian splatting through edge-semantic fusion and incorporates structural cosine similarity loss to compensate for detail loss and global structural inconsistency in low-quality images.
HR-Extreme: A High-Resolution Dataset for Extreme Weather Forecasting
Nian Ran (Zhongguancun Academy), Richard Allmendinger (University of Manchester)
TransformerTabularTime Series
🎯 What it does: A high-resolution extreme weather dataset (HR-Extreme) has been constructed, featuring a resolution of 3 km and 17 types of extreme weather. Various models have been evaluated on this dataset, leading to the proposal of a more robust HR-Heim deep learning baseline.
HShare: Fast LLM Decoding by Hierarchical Key-Value Sharing
Huaijin Wu (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a hierarchical key-value sharing framework (HShare) that reduces the computational overhead of dynamic sparse selection and improves the decoding speed of LLMs by sharing key indices in the KV cache at the layer, head, and query levels.
Human Simulacra: Benchmarking the Personification of Large Language Models
Qiujie Xie (Fudan University), Yue Zhang (Westlake University)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: A human simulation benchmark based on psychological theory has been constructed, covering a virtual character dataset, evaluation framework, and multi-agent cognitive mechanisms.
Human-Aligned Chess With a Bit of Search
Yiming Zhang (Carnegie Mellon University), Daphne Ippolito (Carnegie Mellon University)
Reinforcement Learning from Human FeedbackTransformerText
🎯 What it does: This paper presents ALLIE, an AI that simulates human chess behavior by training on human game logs, capable of skill calibration among players of different levels and exhibiting realistic thinking times and resignation behaviors in games.
Human-inspired Episodic Memory for Infinite Context LLMs
Zafeirios Fountas (Huawei), Jun Wang (University College London)
GenerationRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes EM-LLM, an architecture that embeds human episodic memory and event cognition mechanisms into existing large language models (LLMs), enabling them to handle nearly infinite context lengths without fine-tuning.
Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors
Tianchun Wang (Pennsylvania State University), Wei Cheng (NEC Laboratories America)
GenerationAdversarial AttackReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes a method for attacking large language models during the decoding phase using a small human-like proxy model to generate human-like text that is difficult for detectors to identify.
Hummingbird: High Fidelity Image Generation via Multimodal Context Alignment
Minh-Quan Le (Microsoft), Mei Chen (Microsoft)
RecognitionGenerationData SynthesisVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: A diffusion model named Hummingbird is proposed, capable of generating images that maintain high diversity and high fidelity in a multimodal context given a reference image and textual guidance, ensuring that the scene attributes in the generated images are consistent with the reference image and text.
Hybrid Regularization Improves Diffusion-based Inverse Problem Solving
Hongkun Dou (Beihang University), Yue Deng (Beihang University)
RestorationSuper ResolutionCompressionDiffusion modelImage
🎯 What it does: This paper proposes a hybrid regularization framework (HRDIS) that improves the inverse problem-solving process based on diffusion models through consistency regularization (CR) and noise mixing techniques.
Hydra-SGG: Hybrid Relation Assignment for One-stage Scene Graph Generation
Minghan Chen (University of Technology Sydney), Yi Yang (Zhejiang University)
Object DetectionGenerationTransformerImage
🎯 What it does: A one-stage scene graph generation model based on DETR, named Hydra-SGG, is proposed to address the slow convergence and sparse supervision issues of traditional DETR-SGG models.
Hymba: A Hybrid-head Architecture for Small Language Models
Xin Dong (NVIDIA), Pavlo Molchanov (NVIDIA)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A small language model called Hymba is proposed, which employs a hybrid head architecture to parallelly integrate attention heads and state space models (SSM) within the same layer, while also introducing optimizations such as learnable meta-tokens and shared KV caching.
Hyper-Connections
Defa Zhu (ByteDance), Xun Zhou (ByteDance)
TransformerLarge Language ModelMixture of ExpertsImageText
🎯 What it does: A new connection method called Hyper-Connections (HC) is proposed to replace the traditional residual connections in Transformers, implemented in the pre-training of large-scale language models and visual tasks.
Hyperbolic Genome Embeddings
Raiyan R. Khan (Columbia University), Itsik Pe'er (Columbia University)
Representation LearningConvolutional Neural NetworkBiomedical DataBenchmark
🎯 What it does: A full hyper-surface convolutional neural network (HCNN) is proposed and implemented for representation learning of DNA sequences, operating directly in negatively curved space, avoiding the explicit mapping to evolutionary trees used in traditional methods.
HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks
Jiuding Sun (Pr(Ai) R Group), Atticus Geiger (Pr(Ai) R Group)
Explainability and InterpretabilityTransformerText
🎯 What it does: We propose HyperDAS, a hypernetwork-based framework that can automatically locate the token positions of hidden layer mediating concepts and construct corresponding linear subspaces, thereby achieving interpretable interventions for concepts.
HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere
Hatef Otroshi Shahreza (Idiap Research Institute), Sébastien Marcel (Idiap Research Institute)
RecognitionGenerationData SynthesisOptimizationDiffusion modelImage
🎯 What it does: A method for generating synthetic face datasets called HyperFace, based on pre-trained face embedding spherical packing optimization, has been designed and implemented, and images are synthesized using a conditional diffusion generator.
HyperPLR: Hypergraph Generation through Projection, Learning, and Reconstruction
Weihuang Wen (Chinese University of Hong Kong), Tianshu Yu (Chinese University of Hong Kong)
Graph Neural NetworkGraph
🎯 What it does: A three-stage framework called HyperPLR is proposed, which first projects the hypergraph into a weighted graph, then uses CELL+GCN to learn and generate the weighted graph, and finally reconstructs a new hypergraph using a greedy WCEC algorithm.
HyPoGen: Optimization-Biased Hypernetworks for Generalizable Policy Generation
Hanxiang Ren (Zhejiang University), Yanchao Yang (Hong Kong University)
OptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: This paper proposes an optimized bias hypernetwork, HyPoGen, which directly generates generalizable policy network parameters from task specifications without the need for demonstration data from the target task.
Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
Logan Cross (Stanford University), Nick Haber (Stanford University)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: A multi-agent system called Hypothetical Minds based on large language models has been developed, utilizing a Theory of Mind module to generate, evaluate, and improve hypotheses about the strategies of other agents in natural language, thereby driving high-level planning and sub-goal generation.
I Can Hear You: Selective Robust Training for Deepfake Audio Detection
Zirui Zhang (Columbia University), Chengzhi Mao (Rutgers University)
ClassificationAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningAudio
🎯 What it does: This paper constructs the largest and highest quality deep fake audio dataset, DeepFakeVox-HQ. Using this dataset, existing models are retrained and evaluated, and a frequency-selective adversarial training (F-SAT) method is proposed to enhance the robustness of models against high-frequency domain attacks and real distortions.
I-Con: A Unifying Framework for Representation Learning
Shaden Naif Alshammari (Massachusetts Institute of Technology), Mark Hamilton (Microsoft)
ClassificationRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: A unified information contrastive learning framework I-Con is proposed, which consolidates 23 representative representation learning methods (such as contrastive learning, clustering, dimensionality reduction, supervised learning, etc.) into a single KL divergence objective. Based on this, strategies for debiasing and neighbor propagation are designed, ultimately achieving a significant improvement in unsupervised image classification performance.
I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strength
Wanquan Feng (ByteDance), Qian HE
GenerationData SynthesisDepth EstimationOptimizationDiffusion modelOptical FlowVideo
🎯 What it does: A method for image-to-video generation based on point trajectories and motion intensity control, called I2VControl-Camera, is proposed to achieve fine camera control and adjustable subject motion.
ICLR: In-Context Learning of Representations
Core Francisco Park (Harvard University), Hidenori Tanaka (Harvard University)
Representation LearningTransformerLarge Language ModelGraph
🎯 What it does: A graph tracking task based on random walks is proposed, studying how LLM reorganizes concept representations under contextual semantics by providing a sequence of node concepts in context;
IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model
Yatai Ji (Tsinghua University), Ping Luo (University of Hong Kong)
RecognitionObject DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: Constructed an ID-aware large visual language model IDA-VLM to address cross-scene instance recognition and movie understanding.
IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations
Zhibing Li (Chinese University of Hong Kong), Dahua Lin (Chinese University of Hong Kong)
RestorationGenerationData SynthesisDiffusion modelImage
🎯 What it does: Developed IDArb, a diffusion model capable of handling images under arbitrary numbers of perspectives and lighting conditions, achieving intrinsic decomposition of albedo, normal, metallic, and roughness.
Identifiability for Gaussian Processes with Holomorphic Kernels
Ameer Qaqish (University of North Carolina), Didong Li (University of North Carolina)
Time Series
🎯 What it does: This paper proposes a theoretical framework for the identifiability of Gaussian process (GP) kernels that are analytically tractable near zero. It proves the identifiability properties of many commonly used kernels (such as squared exponential, periodic, Rational Quadratic, etc.) and their combinations, and uses this theory to test the identifiability of parameters in complex composite kernels (such as the 11-parameter kernel for Mauna Loa CO₂ time series).
Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning
Patrik Reizinger (Max Planck Institute for Intelligent Systems), Wieland Brendel (Max Planck Institute for Intelligent Systems)
Representation LearningContrastive LearningTabular
🎯 What it does: The Identifiable Exchangeable Mechanisms (IEM) framework is proposed, unifying the theoretical results regarding identifiability in causal discovery (CD), independent component analysis (ICA), and causal representation learning (CRL). Within this framework, new identifiability conditions (such as cause/mechanism variability) and corresponding theorems are derived.
Identification of Intermittent Temporal Latent Process
Yuke Li (University of Maryland), Heng Huang (Carnegie Mellon University)
ClassificationRecognitionAuto EncoderVideo
🎯 What it does: A theory for the identification of interrupted time latent variable processes is proposed, and based on this, an unsupervised method called InterLatent is designed to recover temporal latent variables and their dynamics.
Identifying latent state transitions in non-linear dynamical systems
Çağlar Hızlı (Aalto University), Pekka Marttinen (Aalto University)
Anomaly DetectionConvolutional Neural NetworkRecurrent Neural NetworkReinforcement LearningFlow-based ModelAuto EncoderTime SeriesSequential
🎯 What it does: This paper proposes an identifiable latent dynamical system framework that can simultaneously recover latent states, nonlinear state transition functions, and observation mappings, identifying the true system dynamics from high-dimensional perception sequences.
IDInit: A Universal and Stable Initialization Method for Neural Network Training
Yu Pan (Huawei Noah's Ark Lab), Zenglin Xu (Fudan University)
Convolutional Neural NetworkImage
🎯 What it does: A novel identity initialization method called IDInit is proposed, which can maintain the complete identity mapping of the backbone and sub-backbone in residual networks.
IFORMER: INTEGRATING CONVNET AND TRANSFORMER FOR MOBILE APPLICATION
Chuanyang Zheng (Shanghai Jiao Tong University)
Object DetectionSegmentationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes iFormer, a lightweight hybrid vision network designed for mobile devices.
IgGM: A Generative Model for Functional Antibody and Nanobody Design
Rubo Wang (Institute of Microelectronics, Chinese Academy of Sciences), Jianhua Yao (Tencent AI Lab)
GenerationProtein Structure PredictionTransformerLarge Language ModelBiomedical Data
🎯 What it does: We propose IgGM, a consistency model that jointly generates antibody CDR sequences and complete antibody-antigen complex structures, enabling de novo antibody and nanobody design given only the antigen and framework region sequences.
IGL-Bench: Establishing the Comprehensive Benchmark for Imbalanced Graph Learning
Jiawen Qin (Beihang University), Philip S. Yu (University of Illinois, Chicago)
ClassificationOptimizationGraph Neural NetworkGraphBenchmark
🎯 What it does: A unified benchmark framework called IGL-Bench has been constructed, covering 17 real graph datasets and 24 imbalanced graph learning algorithms, providing standardized data processing, partitioning strategies, and experimental scripts.
ILLUSION: Unveiling Truth with a Comprehensive Multi-Modal, Multi-Lingual Deepfake Dataset
Kartik Thakral (Indian Institute of Technology Jodhpur), Richa Singh (Indian Institute of Technology Jodhpur)
GenerationData SynthesisTransformerDiffusion modelGenerative Adversarial NetworkImageVideoMultimodalityBenchmarkAudio
🎯 What it does: The ILLUSION dataset is proposed, and a multi-modal, cross-lingual deep fake detection method is evaluated based on this dataset.
Image and Video Tokenization with Binary Spherical Quantization
Yue Zhao (University of Texas at Austin), Philipp Kraehenbuehl (University of Texas at Austin)
GenerationCompressionTransformerGenerative Adversarial NetworkImageVideo
🎯 What it does: A Transformer-based image and video tokenizer is proposed, introducing Binary Spherical Quantization (BSQ) technology to achieve efficient discrete representation without an explicit codebook.
Image Watermarks are Removable using Controllable Regeneration from Clean Noise
Yepeng Liu (University of Florida), Yuheng Bu (University of Florida)
RestorationAdversarial AttackDiffusion modelImage
🎯 What it does: A controllable regeneration attack, CtrlRegen/CtrlRegen+, is proposed, which efficiently removes image watermarks while maintaining visual quality by starting from pure Gaussian noise and incorporating semantic and spatial control during the diffusion process.
Image-level Memorization Detection via Inversion-based Inference Perturbation
Yue Jiang (Institute of Automation Chinese Academy of Sciences), Jing Dong (Institute of Automation Chinese Academy of Sciences)
Image TranslationAnomaly DetectionDiffusion modelImage
🎯 What it does: A framework for image-level memory detection (IIP) based on prompt-free DDIM inversion and random prompt perturbation is proposed to determine whether an image has been memorized by a text-to-image diffusion model.
ImageFolder: Autoregressive Image Generation with Folded Tokens
Xiang Li (Adobe Research), Zhe Lin (Carnegie Mellon University)
GenerationData SynthesisTransformerGenerative Adversarial NetworkImage
🎯 What it does: Designed and implemented the ImageFolder image tokenizer, which utilizes product quantization to generate spatially aligned semantic and detail tokens, and achieves shorter token sequences through parallel prediction of the two tokens in an autoregressive model, thereby improving generation and reconstruction quality.
ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination
Xinxin Zhao (Southeast University), Teng Wang (Southeast University)
OptimizationRobotic IntelligenceConvolutional Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelImageMultimodality
🎯 What it does: The ImagineNav framework is proposed, utilizing visual-language models (VLM) to achieve open vocabulary object navigation in mapless environments.
IMDPrompter: Adapting SAM to Image Manipulation Detection by Cross-View Automated Prompt Learning
Quan Zhang (Tsinghua University), Chun Yuan (Shanghai Jiao Tong University)
Anomaly DetectionPrompt EngineeringImage
🎯 What it does: This paper proposes IMDPrompter, a cross-view automatic prompt learning framework based on SAM for automated image tampering detection and localization.
ImDy: Human Inverse Dynamics from Imitated Observations
Xinpeng Liu (Shanghai Jiao Tong University), Cewu Lu (Shanghai Jiao Tong University)
Pose EstimationRobotic IntelligenceTransformerReinforcement LearningTime Series
🎯 What it does: A complete dataset of 150 hours of driving torque and full-body contact force data, ImDy, is generated using imitation learning and physical simulation, and a data-driven inverse dynamics solver, ImDyS, is trained on it.
Immunogenicity Prediction with Dual Attention Enables Vaccine Target Selection
Song Li (Shanghai Jiao Tong University), Bingxin Zhou (Shanghai Jiao Tong University)
Drug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningMultimodalityBiomedical Data
🎯 What it does: A deep learning model based on a dual attention mechanism, VENUSVACCINE, is proposed for immunogenicity prediction, and an ImmunoDB dataset containing over 7,000 antigens from bacterial, viral, and tumor sources is constructed, along with a posterior validation protocol.
Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression
Shuang Liang (University of California Los Angeles), Guido Montufar
Optimization
🎯 What it does: This paper studies the implicit bias of mirror descent in wide and shallow neural networks within univariate regression, demonstrating that mirror descent exhibits lazy training across a broad class of potential functions, and that its implicit bias aligns with that of standard gradient descent as the network width approaches infinity.
Implicit In-context Learning
Zhuowei Li (Rutgers University), Dimitris N. Metaxas (Rutgers University)
ClassificationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A new implicit context learning method (I2CL) is proposed, aimed at reducing the reasoning cost of context learning to be comparable to zero-shot learning while maintaining minimal information loss.
Implicit Neural Surface Deformation with Explicit Velocity Fields
Lu Sang (Technical University of Munich), Daniel Cremers (University of Bonn)
Point Cloud
🎯 What it does: Given two sets of point clouds, we self-supervise the simultaneous learning of time-varying neural implicit surfaces and the corresponding velocity fields using sparse correspondences, achieving physically feasible deformations without intermediate shape supervision.
Implicit Search via Discrete Diffusion: A Study on Chess
Jiacheng Ye (University of Hong Kong), Lingpeng Kong (Huawei Noah's Ark Lab)
TransformerSupervised Fine-TuningDiffusion modelSequential
🎯 What it does: A model named DIFFUSEARCH is proposed, which implements implicit search during inference through a discrete diffusion model, replacing traditional explicit search methods (such as MCTS) to enhance next-step action prediction and long-term planning capabilities.
Improved Algorithms for Kernel Matrix-Vector Multiplication Under Sparsity Assumptions
Piotr Indyk (Massachusetts Institute of Technology), Tal Wagner (Tel Aviv University)
TransformerLarge Language ModelText
🎯 What it does: An approximate multiplication algorithm with sub-quadratic time and linear dimension is proposed for the matrix-vector product of Gaussian kernel matrices under the sparse assumption.
Improved Approximation Algorithms for $k$-Submodular Maximization via Multilinear Extension
Huanjian Zhou (University of Tokyo), Baoxiang Wang (Chinese University of Hong Kong Shenzhen)
Optimization
🎯 What it does: A multi-linear extension of k-submodular maximization is proposed, and a unified Frank-Wolfe algorithm framework is designed based on this extension, providing approximation algorithms under various constraints, including monotonic and non-monotonic cases, matrix constraints, knapsack constraints, and their intersections.
Improved Convergence Rate for Diffusion Probabilistic Models
Gen Li (Chinese University of Hong Kong), Yuchen Jiao (Chinese University of Hong Kong)
GenerationOptimizationDiffusion modelImage
🎯 What it does: This paper presents an analysis of the convergence rate of an improved diffusion probability model, establishing a theoretical framework with an iterative complexity of d^(1/3) ε^(-2/3), which is better than the previous complexity of d^(5/12) ε^(-1).
Improved Diffusion-based Generative Model with Better Adversarial Robustness
Zekun Wang (Harbin Institute of Technology), Zhi-Ming Ma (Harbin Institute of Technology)
GenerationOptimizationAdversarial AttackDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: A method is proposed and implemented to incorporate efficient adversarial training (AT) into the training process of diffusion probabilistic models (DPM) and consistency models (CM) to alleviate the distribution mismatch problem between the training and sampling phases.
Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent
Sayan Banerjee (University of North Carolina), PROMIT GHOSAL
OptimizationStochastic Differential Equation
🎯 What it does: This paper derives the convergence rate of finite particle systems in Kernelized Stein Divergence (KSD) and Wasserstein-2 metrics for Stein Variational Gradient Descent (SVGD), providing corresponding continuous and discrete time analyses.
Improved Sampling Algorithms for Lévy-Itô Diffusion Models
Vadim Popov (Huawei Noah's Ark Lab), Mikhail Sergeevich Kudinov
GenerationData SynthesisDiffusion modelImageStochastic Differential EquationOrdinary Differential EquationAudio
🎯 What it does: This paper studies the sampling algorithm of the improved Lévy-Itô diffusion model, proposing a reverse SDE with adjustable noise parameters, and validating it on image and speech generation tasks.
Improved Sampling Of Diffusion Models In Fluid Dynamics With Tweedie's Formula
Youssef Shehata (Technical University of Munich), Nils Thuerey (Technical University of Munich)
Diffusion modelTime SeriesPhysics Related
🎯 What it does: This paper proposes two improved sampling methods for diffusion models—Truncated Sampling Model (TSM) and Iterative Refinement (IR)—to achieve single-step or few-step sampling while maintaining or enhancing prediction accuracy, suitable for fluid dynamics simulations.
Improved Techniques for Optimization-Based Jailbreaking on Large Language Models
Xiaojun Jia (Nanyang Technological University), Min Lin (Sea AI Lab)
OptimizationSafty and PrivacyAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper addresses the security alignment issue of large language models by proposing an improved gradient optimization-based cracking technique—I-GCG. By introducing diverse harmful guidance templates, an automatic multi-coordinate update strategy, and an easy-to-difficult initialization method, it achieves an almost 100% cracking success rate.
Improved Training Technique for Latent Consistency Models
Quan Dao (Rutgers University), Dimitris N. Metaxas
GenerationData SynthesisDiffusion modelAuto EncoderImage
🎯 What it does: This paper proposes an improved training framework for training consistency models in latent space, capable of generating high-quality images in one to two steps.
ImProver: Agent-Based Automated Proof Optimization
Riyaz Ahuja (Carnegie Mellon University), Sean Welleck (Carnegie Mellon University)
OptimizationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: ImProver has been developed, a Lean proof optimizer based on large language models, capable of automatically rewriting and optimizing proof length, declarative aspects, and other metrics while maintaining proof correctness.
Improving Complex Reasoning with Dynamic Prompt Corruption: A Soft Prompt Optimization Approach
Sinan Fan (Zhejiang University), Jieping Ye (Zhejiang University)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Analyzes the impact of soft prompts for large language models on complex reasoning tasks and proposes a dynamic prompt erosion method to enhance reasoning performance.
Improving Convergence Guarantees of Random Subspace Second-order Algorithm for Nonconvex Optimization
Rei Higuchi (University of Tokyo), Akiko Takeda (University of Tokyo)
OptimizationTabular
🎯 What it does: A random subspace homogeneous trust region method (RSHTR) is proposed for high-dimensional non-convex optimization, and its global and local convergence theories are provided.
Improving Data Efficiency via Curating LLM-Driven Rating Systems
Jinlong Pang (University of California), Wei Wei (Accenture)
Recommendation SystemOptimizationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposes the DS2 data filtering pipeline, which corrects the quality scores generated by LLMs and combines diversity filtering, retaining only 3.3% of the data to complete instruction fine-tuning.
Improving Deep Regression with Tightness
Shihao Zhang (National University of Singapore), Angela Yao (National University of Singapore)
Pose EstimationDepth EstimationSupervised Fine-TuningImage
🎯 What it does: For deep regression tasks, a multi-target (MT) strategy and regression optimal transport regularization (ROT-Reg) are proposed to compress feature representations, thereby enhancing regression performance.
Improving Equivariant Networks with Probabilistic Symmetry Breaking
Hannah Lawrence (Massachusetts Institute of Technology), Sékou-Oumar Kaba (McGill University)
Graph Neural NetworkAuto EncoderGraph
🎯 What it does: Under the premise of maintaining the equivariance of the network, this study investigates how to break the self-symmetry of the input, proposing Symmetry Breaking Position Encoding (SymPE) and providing a corresponding theoretical framework.
Improving Generalization and Robustness in SNNs Through Signed Rate Encoding and Sparse Encoding Attacks
Bhaskar Mukhoty (Mohamed bin Zayed University of Artificial Intelligence), Bin Gu (Jilin University)
OptimizationAdversarial AttackSpiking Neural NetworkImage
🎯 What it does: This paper proposes Symbol Rate Encoding (SRATE) and Sparse Encoding Attack (SEA) to improve the generalization and robustness of spiking neural networks.
Improving Graph Neural Networks by Learning Continuous Edge Directions
Seong Ho Pahng (Harvard University), Sahand Hormoz (Harvard Medical School)
Graph Neural NetworkGraph
🎯 What it does: This study investigates an information propagation method for graph neural networks by learning continuous fuzzy edge directions.
Improving Instruction-Following in Language Models through Activation Steering
Alessandro Stolfo (ETH Zurich), Besmira Nushi (Microsoft Research)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes to control fine-grained constraints such as format, length, and keywords by comparing activation differences to compute instruction-specific vectors, and directly weighting the model's residual flow during inference.
Improving Language Model Distillation through Hidden State Matching
Sayantan Dasgupta (University of Melbourne), Trevor Cohn (University of Melbourne)
CompressionKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: In language model distillation, Centered Kernel Alignment (CKA) is proposed to match hidden layers of different dimensions, achieving a higher compression rate for the student model.
Improving Large Language Model Planning with Action Sequence Similarity
Xinran Zhao (Carnegie Mellon University), Azade Nova (Google DeepMind)
TransformerLarge Language ModelText
🎯 What it does: In the context of in-context learning (ICL) for large language models, a two-stage GRASE-DC process is proposed to enhance the model's performance in planning tasks by utilizing action sequence similarity to select and filter examples.
Improving Long-Text Alignment for Text-to-Image Diffusion Models
Luping Liu (University of Hong Kong), Dong Xu (University of Hong Kong)
GenerationTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: Proposes the LongAlign method to address the insufficient text-image alignment caused by long text inputs, improving the generation of high-quality images corresponding to long texts in Stable Diffusion.
Improving Neural Network Accuracy by Concurrently Training with a Twin Network
Benjamin Vandersmissen (University of Antwerp), Jose Oramas (University of Antwerp)
ClassificationKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes and validates the Twin Network Augmentation (TNA) training strategy, which enhances the validation accuracy of a single network through collaborative training of dual networks and L2 matching of logits across various CNN architectures.
Improving Neural Optimal Transport via Displacement Interpolation
Jaemoo Choi (Georgia Institute of Technology), Jaewoong Choi (Sungkyunkwan University)
Image TranslationOptimizationImage
🎯 What it does: A neural optimal transport model based on displacement interpolation (DIOTM) is proposed to learn the optimal transport mapping between source and target distributions.
Improving Pretraining Data Using Perplexity Correlations
Tristan Thrush (Stanford University), Tatsunori Hashimoto (Stanford University)
Large Language ModelTextBenchmark
🎯 What it does: A framework for selecting pre-training data without training or manual screening is constructed by utilizing the correlation between the perplexity of existing large language models and the performance on downstream tasks.
Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching
Zijing Ou (Imperial College London), David Barber (Imperial College London)
GenerationData SynthesisOptimizationDiffusion modelImage
🎯 What it does: An unbiased optimal covariance matching (OCM) objective is proposed to learn the diagonal covariance of diffusion models, significantly improving sampling quality, likelihood estimation, and sampling efficiency while preserving the original mean.
Improving Reasoning Performance in Large Language Models via Representation Engineering
Bertram Højer (IT University of Copenhagen), Stefan Heinrich (IT University of Copenhagen)
TransformerLarge Language ModelContrastive LearningText
🎯 What it does: Using representation engineering techniques, control vectors are generated in the residual flow of LLMs and injected during inference to enhance the model's performance on reasoning tasks.
Improving Semantic Understanding in Speech Language Models via Brain-tuning
Omer Moussa (Max Planck Institute for Software Systems), Mariya Toneva (Max Planck Institute for Software Systems)
Supervised Fine-TuningMultimodalityMagnetic Resonance ImagingAudio
🎯 What it does: For the speech language model, the authors developed a 'brain fine-tuning' method by fine-tuning a pre-trained model on fMRI recordings while listening to natural stories.
Improving the Sparse Structure Learning of Spiking Neural Networks from the View of Compression Efficiency
Jiangrong Shen (Xi'an Jiaotong University), Badong Chen (Xi'an Jiaotong University)
CompressionOptimizationSpiking Neural NetworkImage
🎯 What it does: A two-stage dynamic sparse structure learning method is proposed, which uses the PQ index to dynamically determine the reconnection ratio, enabling sparse training of SNN from scratch.
Improving Uncertainty Estimation through Semantically Diverse Language Generation
Lukas Aichberger (Johannes Kepler University Linz), Sepp Hochreiter (NXAI GmbH)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a novel Semantic Diversification Language Generation method (SDLG), which actively generates semantically different but highly probable alternative sentences by evaluating the attributes, replacements, and importance scores of keywords in the initial generated text;
Improving Unsupervised Constituency Parsing via Maximizing Semantic Information
Junjie Chen (University of Tokyo), Danushka Bollegala (University of Liverpool)
Large Language ModelReinforcement LearningText
🎯 What it does: Introducing the SemInfo objective in unsupervised constituent syntax analysis to maximize the semantic information encoded in the structure, thereby improving parsing accuracy.
ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentences
Yuxin Wang (Dartmouth), Soroush Vosoughi (Dartmouth)
Contrastive LearningText
🎯 What it does: A learnable, reference-free metric IMPSCORE is proposed and implemented to quantify the implicitness of English sentences.
Imputation for prediction: beware of diminishing returns.
Marine Le Morvan (Inria), Gael Varoquaux
Tabular
🎯 What it does: This paper evaluates the impact of different imputation methods on predictive performance through systematic experiments and analyzes the moderating effects of factors such as model expressiveness, missing indicators, and response nonlinearity.
In Search of Forgotten Domain Generalization
Prasanna Mayilvahanan (University of Tübingen), Wieland Brendel (University of Tübingen)
Domain AdaptationTransformerVision Language ModelContrastive LearningImage
🎯 What it does: This paper constructs clean single-domain datasets (LAION-Natural and LAION-Rendition), trains CLIP, and evaluates its true generalization ability to unseen domains (rendition) under large-scale web data, further investigating the impact of different domain mixing ratios and data scales on OOD performance.
In vivo cell-type and brain region classification via multimodal contrastive learning
Han Yu (Columbia University), Cole Lincoln Hurwitz
ClassificationConvolutional Neural NetworkContrastive LearningMultimodalityBiomedical Data
🎯 What it does: Developed a pre-training framework NEMO based on multimodal contrastive learning to embed the autocorrelation maps and potential waveforms of units from neuroelectrophysiological data, and fine-tuned it for cell type and brain region classification tasks.
In-Context Editing: Learning Knowledge from Self-Induced Distributions
Siyuan Qi (State Key Laboratory of General Artificial Intelligence), Zilong Zheng (State Key Laboratory of General Artificial Intelligence)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By introducing the Inconsistency Context Editing (ICE) method, the knowledge update objective is transformed from a rigid one-hot target to a context-induced probability distribution, utilizing the contextual learning ability of language models to achieve internalization of new knowledge and adaptive updates of the model.
In-context Time Series Predictor
Jiecheng Lu (Georgia Institute of Technology), Shihao Yang (Georgia Institute of Technology)
TransformerTime Series
🎯 What it does: This paper proposes an In-Context Time Series Predictor (ICTSP) that uses (lookback, future) task pairs as input tokens to perform time series forecasting by leveraging the ICL capabilities of Transformers.
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Angelika Romanou (École Polytechnique Fédérale de Lausanne), Antoine Bosselut (Cohere For Artificial Intelligence)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: A multilingual evaluation benchmark INClude has been constructed, containing 197,243 questions, covering 44 languages, 15 writing systems, and knowledge questions related to regions, cultures, and professional licenses from 52 countries.
InCoDe: Interpretable Compressed Descriptions For Image Generation
Armand Comas (Northeastern University), Rene Vidal
GenerationExplainability and InterpretabilityDiffusion modelImage
🎯 What it does: This paper proposes a framework called InCoDe, which utilizes Information Pursuit to generate interpretable compressed descriptions, and uses these as conditions to drive diffusion models for image generation and editing.
Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On
Siqi Wan (University of Science and Technology of China), Tao Mei (HiDream.ai)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: A virtual try-on method based on diffusion models, SPM-Diff, is proposed, which guides image generation through explicit semantic point matching and 3D perception enhancement. It employs a dual-branch UNet and introduces point focus loss, significantly improving the retention of clothing details and shapes.
Incremental Causal Effect for Time to Treatment Initialization
Andrew Ying, Ronghui Xu (University of California, San Diego)
Biomedical Data
🎯 What it does: This paper proposes to extend the incremental causal effect to the problem of treatment initiation in continuous time and provides a framework for identification and estimation without the positivity assumption.