ICLR 2023 Papers — Page 7
International Conference on Learning Representations · 1573 papers
Gradient Gating for Deep Multi-Rate Learning on Graphs
T. Konstantin Rusch (ETH Zurich), Siddhartha Mishra (ETH Zurich)
Graph Neural NetworkGraphOrdinary Differential Equation
🎯 What it does: Proposes the Gradient Gating (G²) framework, which adds learnable multi-rate gating to the output of each layer in traditional message-passing GNNs, and utilizes graph gradients to adaptively suppress over-smoothing;
Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models
Meng Liu (Texas A&M University), Shuiwang Ji (Texas A&M University)
Graph
🎯 What it does: A ratio matching method based on gradient-guided importance sampling is proposed for learning binary discrete energy models.
Graph Contrastive Learning for Skeleton-based Action Recognition
Xiaohu Huang (Huazhong University of Science and Technology), Bin Feng (Huazhong University of Science and Technology)
RecognitionPose EstimationGraph Neural NetworkContrastive LearningVideoGraph
🎯 What it does: A skeleton action recognition framework called SkeletonGCL based on graph contrastive learning is proposed, achieving self-supervised contrast across sequences through graph learning in GCN.
Graph Domain Adaptation via Theory-Grounded Spectral Regularization
Yuning You (Texas A&M University), Yang Shen (Texas A&M University)
Domain AdaptationGraph Neural NetworkGraph
🎯 What it does: A graph domain adaptation method based on spectral regularization is proposed, which enhances cross-domain graph learning performance by controlling the spectral smoothness and maximum frequency response of graph neural networks.
Graph Neural Network-Inspired Kernels for Gaussian Processes in Semi-Supervised Learning
Zehao Niu (University of Chicago), Jie Chen (MIT IBM Watson AI Lab)
ClassificationOptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: This paper establishes the correspondence between neural networks in the infinite width layer limit of Graph Convolutional Networks (GCN) and Gaussian Processes (GP), deriving a GP kernel induced by graph structure and designing a low-rank approximation for scalable posterior inference.
Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs
Chenxiao Yang (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This study investigates the generalization ability of Graph Neural Networks (GNN) in node-level prediction tasks and proposes an intermediate model called Propagational MLP (PMLP), which uses a standard Multi-Layer Perceptron (MLP) during the training phase and GNN message passing during the inference phase, to reveal the source of GNN's superiority.
Graph Neural Networks for Link Prediction with Subgraph Sketching
Benjamin Paul Chamberlain, Max Hansmire (Twitter Inc.)
Graph Neural NetworkGraph
🎯 What it does: An efficient graph neural network ELPH (and its scalable version BUDDY) is proposed for link prediction, replacing explicit subgraph construction with subgraph Sketching, addressing the limitations of traditional MPNN in triangle counting and automatic isomorphic nodes.
Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution
Chao Chen (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Recommendation SystemGraph Neural NetworkGraph
🎯 What it does: A graph signal sampling-based inductive one-bit matrix completion framework GS-IMC and its Bayesian online extension BGS-IMC are proposed.
Graph-based Deterministic Policy Gradient for Repetitive Combinatorial Optimization Problems
Zhongyuan Zhao (Rice University), Santiago Segarra (Rice University)
OptimizationGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: A graph-based deterministic policy gradient framework, GDPG-Twin, is proposed to solve repetitive combinatorial optimization problems and significantly reduce the optimality gap of fast distributed heuristics.
Gray-Box Gaussian Processes for Automated Reinforcement Learning
Gresa Shala (University of Freiburg), Josif Grabocka (University of Freiburg)
OptimizationHyperparameter SearchReinforcement LearningTabular
🎯 What it does: A gray-box Bayesian optimization method (RCGP) is proposed, which achieves low-budget and efficient optimization of hyperparameters for reinforcement learning algorithms by embedding the generalized logistic function estimation of the reward curve into the Gaussian process feature space.
Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement
Samuel Neumann (University of Alberta), Martha White (University of Alberta)
Reinforcement LearningOrdinary Differential Equation
🎯 What it does: This paper proposes a Greedy Actor-Critic algorithm based on Conditional Cross-Entropy (Conditional CEM), which enhances the policy of the original MDP by taking the top percentile of actions at each state and performing maximum likelihood updates on the actor.
GReTo: Remedying dynamic graph topology-task discordance via target homophily
Zhengyang Zhou (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
Graph Neural NetworkGraphTime Series
🎯 What it does: A GNN for dynamic graph regression, called GReTo, is proposed, which can correct the topology-task inconsistency issue through target homogeneity.
Gromov-Wasserstein Autoencoders
Nao Nakagawa (Hokkaido University), Miki Haseyama (Hokkaido University)
GenerationRepresentation LearningAuto EncoderImage
🎯 What it does: This paper proposes the Gromov-Wasserstein Autoencoder (GWAE), which achieves unsupervised representation learning by minimizing the GW distance.
Grounding Graph Network Simulators using Physical Sensor Observations
Jonas Linkerhägner (Karlsruhe Institute of Technology), Gerhard Neumann (Karlsruhe Institute of Technology)
Graph Neural NetworkPoint CloudMesh
🎯 What it does: A new graph network simulator (GGNS) is proposed, which integrates point cloud observation information into grid simulation, achieving more accurate physical simulation under incomplete initial conditions.
Guarded Policy Optimization with Imperfect Online Demonstrations
Zhenghai Xue (Nanyang Technological University), Bolei Zhou (University of California)
Autonomous DrivingOptimizationReinforcement LearningSequential
🎯 What it does: A novel teacher-student shared control method TS2C is proposed, allowing the use of teacher policies with arbitrary performance for online demonstration and training of student policies.
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
Seonghyeon Ye (Korea Advanced Institute of Science and Technology), Minjoon Seo (Korea Advanced Institute of Science and Technology)
ClassificationMeta LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A meta-training method called FLIPPED LEARNING is proposed and implemented, which reverses the roles of task instructions and label space, training language models to generate instructions given inputs and labels, and selecting labels based on instruction probabilities during inference.
Guiding continuous operator learning through Physics-based boundary constraints
Nadim Saad (Stanford University), Danielle C. Maddix (AWS AI Labs)
OptimizationComputational EfficiencyTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: The BOON method is proposed, which makes structural modifications on the neural operator kernel to explicitly satisfy the Dirichlet, Neumann, and periodic boundary conditions of PDEs, thereby improving the accuracy of the solutions.
Guiding Energy-based Models via Contrastive Latent Variables
Hankook Lee (LG AI Research), Jinwoo Shin (KAIST)
GenerationData SynthesisGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: By using the spherical latent representations obtained from contrastive learning as latent variables for the energy model, we jointly train the energy model and contrastive learning to improve generation quality and training stability.
Guiding Safe Exploration with Weakest Preconditions
Greg Anderson (University of Texas at Austin), Isil Dillig (University of Texas at Austin)
Safty and PrivacyReinforcement LearningSequential
🎯 What it does: A neural-symbolic method called SPICE is implemented, which constructs interpretable safety shields using weak preconditions to ensure action safety during training.
H2RBox: Horizontal Box Annotation is All You Need for Oriented Object Detection
Xue Yang (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Object DetectionConvolutional Neural NetworkImage
🎯 What it does: Using Horizontal Boxes (HBox) for Directional Target Detection (H2RBox)
Hard-Meta-Dataset++: Towards Understanding Few-Shot Performance on Difficult Tasks
Samyadeep Basu (University of Maryland), Daniela Massiceti (Microsoft)
ClassificationMeta LearningTransformerContrastive LearningImageBenchmark
🎯 What it does: Developed the FASTDIFFSEL algorithm to efficiently extract difficult tasks and based on this, constructed the HARD-META-DATASET++ benchmark.
Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
Zhang-Wei Hong (Massachusetts Institute of Technology), Romain Laroche
Reinforcement LearningTabular
🎯 What it does: This paper proposes a method of trajectory weighting to improve policy learning in offline reinforcement learning datasets with mixed reward distributions.
Harnessing Out-Of-Distribution Examples via Augmenting Content and Style
Zhuo Huang (Sydney AI Centre, University of Sydney), Tongliang Liu (Sydney AI Centre, University of Sydney)
Domain AdaptationAnomaly DetectionConvolutional Neural NetworkAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: The HOOD framework achieves causal separability by decomposing images into two causal variables: content (C) and style (S), utilizing variational inference. It generates benign and malign OOD samples through positive (changing S) and negative (changing C) interventions, respectively, to enhance model generalization and improve OOD detection.
Hebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNs
Yu Duan (Tsinghua University), Kaisheng Ma (Tsinghua University)
OptimizationMeta LearningRecurrent Neural NetworkImageSequential
🎯 What it does: This paper proposes integrating the Hebbian rule with a new gradient-based plasticity rule into recurrent neural networks (RNNs), enabling them to continuously update weights in an unsupervised environment through self-generated targets, thereby enhancing memory and rapid learning capabilities.
Hebbian Deep Learning Without Feedback
Adrien Journé (Huawei), Timoleon Moraitis (Huawei)
ClassificationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: Proposed and implemented a multi-layer SoftHebb algorithm that performs deep feature extraction using feedback-free, unsupervised Hebbian learning, achieving high-accuracy classification on standard visual benchmarks.
Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles
Biswadeep Chakraborty (Georgia Institute of Technology), Saibal Mukhopadhyay (Georgia Institute of Technology)
Spiking Neural NetworkTime Series
🎯 What it does: This study investigates the impact of heterogeneity in neuron and synapse dynamics on the memory capacity, spike rate, and energy efficiency of recurrent spiking neural networks, and proposes a Bayesian optimization-based method for designing heterogeneous parameters.
HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention
Shijie Geng (Rutgers University), Yongfeng Zhang (Rutgers University)
RetrievalRepresentation LearningTransformerContrastive LearningImageTextMultimodality
🎯 What it does: Proposes HiCLIP, which incorporates hierarchy-aware attention into the visual and language branches of CLIP to automatically learn the hierarchical structure of images and text;
Hidden Markov Transformer for Simultaneous Machine Translation
Shaolei Zhang (University of Chinese Academy of Sciences), Yang Feng (Institute of Computing Technology, Chinese Academy of Sciences)
TransformerText
🎯 What it does: This paper proposes the Hidden Markov Transformer (HMT), which unifies the problem of 'when to start translating' in machine translation with the translation itself, and learns the optimal translation timing through an HMM structure.
Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
Michael Chang (University of California Berkeley), Amy Zhang (Meta AI)
Robotic IntelligenceTransformerReinforcement LearningImage
🎯 What it does: By constructing a two-layer hierarchical abstraction, the object representation in the original visual input is decomposed into action-invariant features (types) and action-related features (states), and these abstract states are used to build reusable state transition graphs, thereby achieving compositional generalization in the object rearrangement task within an offline reinforcement learning framework.
Hierarchical Relational Learning for Few-Shot Knowledge Graph Completion
Han Wu (University of Sydney), Jianyuan Guo (University of Sydney)
Representation LearningMeta LearningGraph Neural NetworkTransformerContrastive LearningGraphBenchmark
🎯 What it does: This study focuses on few-shot knowledge graph completion and proposes a hierarchical relationship learning framework called HiRe, which utilizes three layers of relational information (entity layer, triple layer, and context layer) to jointly learn meta-relation representations.
Hierarchical Sliced Wasserstein Distance
Khai Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
GenerationOptimizationComputational EfficiencyImage
🎯 What it does: A hierarchical sliced Wasserstein distance (HSW) is proposed, which reduces the computational cost of high-dimensional projections by decomposing projections into fewer 'bottleneck projections' through hierarchical Radon transform (HRT) and then linearly combining them randomly.
HiT-MDP: Learning the SMDP option framework on MDPs with Hidden Temporal Embeddings
Chang Li (University of Sydney), Dacheng Tao (University of Sydney)
OptimizationTransformerReinforcement LearningSequential
🎯 What it does: This paper proposes the Hidden Time Markov Decision Process (HiT-MDP) and proves its equivalence to the traditional SMDP options framework; based on this, it constructs the Maximum Entropy Option Policy Gradient (MOPG) algorithm, using transformers to learn option embeddings and optimize policies within this framework.
HiViT: A Simpler and More Efficient Design of Hierarchical Vision Transformer
Xiaosong Zhang (University of Chinese Academy of Sciences), Qi Tian (Huawei Inc.)
Object DetectionSegmentationComputational EfficiencyTransformerImage
🎯 What it does: This paper proposes HiViT, a simplified and efficient hierarchical visual Transformer that combines the simplicity of ViT with the hierarchical features of Swin, making it particularly suitable for occluded image modeling.
Holistic Adversarially Robust Pruning
Qi Zhao (Karlsruhe Institute of Technology), Christian Wressnegger (Karlsruhe Institute of Technology)
CompressionAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A robust compression method called HARP is proposed from a global perspective, which achieves high natural accuracy and adversarial robustness for extremely sparse networks by learning the compression rate and importance scores for each layer.
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers
Chen Liang (Georgia Institute of Technology), Tuo Zhao (Georgia Institute of Technology)
OptimizationKnowledge DistillationTransformerSupervised Fine-TuningText
🎯 What it does: A task-agnostic distillation framework combining iterative pruning is proposed to obtain a compact BERT model.
HotProtein: A Novel Framework for Protein Thermostability Prediction and Editing
Tianlong Chen (University of Texas at Austin), Adam Klivans (University of Texas at Austin)
Protein Structure PredictionGraph Neural NetworkContrastive LearningBiomedical Data
🎯 What it does: A large-scale protein thermal stability dataset called HotProtein is proposed, along with a complete learning framework for thermal stability prediction and editing.
How Does Semi-supervised Learning with Pseudo-labelers Work? A Case Study
Yiwen Kou (University of California), Quanquan Gu (University of California)
ClassificationRepresentation LearningConvolutional Neural NetworkTabular
🎯 What it does: This study investigates semi-supervised learning assisted by pseudo-labelers, demonstrating that semi-supervised pre-training combined with linear probing can achieve nearly zero test error on a simplified data model and a two-layer convolutional neural network, while pure supervised learning can only achieve constant-level test error.
How gradient estimator variance and bias impact learning in neural networks
Arna Ghosh (McGill University), Blake Aaron Richards
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper explores how the variance and bias of gradient estimates affect training and generalization during the learning process of neural networks, providing theoretical derivations and validating them on VGG-CIFAR10 and toy networks.
How I Learned to Stop Worrying and Love Retraining
Max Zimmer (Zuse Institute Berlin), Sebastian Pokutta (Technische Universität Berlin)
OptimizationComputational EfficiencyConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: This paper re-examines the retraining phase in Iterative Magnitude Pruning (IMP) and proposes the use of linear learning rate scheduling and Adaptive Initial Learning Rate (ALLR) to significantly shorten the retraining period. Furthermore, it extends the budget constraints to the initial dense training, forming an efficient sparsification process called BIMP.
How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?
Jetze Schuurmans (Delft University of Technology), Julian Kooij (Delft University of Technology)
CompressionConvolutional Neural NetworkImage
🎯 What it does: A systematic experimental study was conducted on the correlation between the approximation error using tensor decomposition in neural network compression and the performance of the compressed model.
How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization
Jonas Geiping (University of Maryland), Andrew Gordon Wilson
Domain AdaptationData-Centric LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper evaluates the contribution of data augmentation to model performance by quantifying the 'equivalent sample size' of augmented views, and explores the role of augmentation under different data scales, distribution shifts, model widths, and in comparison with explicit invariant networks. It also investigates the mechanism of augmentation as a source of randomness in the training process and demonstrates that it can lead to a flatter loss landscape.
How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules
Yutong Xie (University of Michigan), Qiaozhu Mei (University of Michigan)
Drug DiscoveryBiomedical Data
🎯 What it does: This paper proposes a framework for evaluating the coverage of chemical space and introduces a new coverage metric, #Circles, within this framework to measure the range of chemical space covered by drug candidate libraries or generative models.
How robust is unsupervised representation learning to distribution shift?
Yuge Shi (University of Oxford), Amartya Sanyal (ETH Zurich)
Domain AdaptationRepresentation LearningAuto EncoderContrastive LearningImage
🎯 What it does: Systematically evaluated the robustness of unsupervised representation learning (self-supervised learning SSL and autoencoders AE) against distribution shifts compared to supervised learning (SL). A controllable shift version of real datasets was proposed, and the robustness of the representations themselves was separated from the bias of the linear classifier by training a linear head only on OOD data.
How Sharpness-Aware Minimization Minimizes Sharpness?
Kaiyue Wen (Tsinghua University), Zhiyuan Li (Stanford University)
Optimization
🎯 What it does: A rigorous theoretical analysis of the Sharpness-Aware Minimization (SAM) algorithm is conducted, clarifying its implicit regularization preferences in both full-batch and single-sample stochastic settings.
How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?
Yifei Ming (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
Anomaly DetectionRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: A representation learning framework named CIDER is proposed, which utilizes two types of losses (diversity and compactness) in the hyperspherical embedding space to enhance the performance of out-of-distribution (OOD) sample detection.
How to prepare your task head for finetuning
Yi Ren (University of British Columbia), Danica J. Sutherland (University of British Columbia)
ClassificationSegmentationRetrievalConvolutional Neural NetworkSupervised Fine-TuningContrastive LearningImage
🎯 What it does: This study investigates the impact of task head design and initialization on feature adaptation, energy flow, and downstream performance during fine-tuning (FT) after pre-training through head probing (HP), and proposes a set of theoretical guidance and practical methods based on the decomposition of energy and direction.
How to Train your HIPPO: State Space Models with Generalized Orthogonal Basis Projections
Albert Gu, Christopher Re
TransformerTime SeriesSequentialBenchmark
🎯 What it does: The theoretical improvement of the HiPPO framework in structured state space models (SSM) is provided, offering a rigorous explanation of the long-term dependency modeling mechanism of the S4 model, and based on this, new variants of SSM are proposed (such as the Fourier-based S4-FouT).
Human alignment of neural network representations
Lukas Muttenthaler (Technische Universitat Berlin), Simon Kornblith (Google Research)
Representation LearningTransformerContrastive LearningImage
🎯 What it does: This study investigates the consistency between neural network representations and human conceptual space, evaluating the model's 'odd-one-out' accuracy using three different human similarity tasks (odd-one-out, triplet, and multi-arrangement) and conducting RSA analysis.
Human Motion Diffusion Model
Guy Tevet (Tel Aviv University), Amit Haim Bermano
GenerationData SynthesisTransformerDiffusion modelVideoTextMultimodality
🎯 What it does: A Motion Diffusion Model (MDM) based on Transformer is proposed, utilizing an unsupervised diffusion generative model to achieve various conditional and unconditional human motion generation and editing.
Human MotionFormer: Transferring Human Motions with Vision Transformers
Hongyu Liu (Hong Kong University of Science and Technology), Qifeng Chen (Fudan University)
Image TranslationGenerationPose EstimationTransformerImageVideo
🎯 What it does: This paper proposes a Human MotionFormer based on Vision Transformer, which transfers actions from target person videos to source person images, achieving high-quality human action transfer.
Human-Guided Fair Classification for Natural Language Processing
Florian E. Dorner (Max Planck Institute for Intelligent Systems), Martin Vechev (ETH Zurich)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A complete process for individual fairness constraints based on text pair generation is proposed to guide the fair training of text classifiers.
Human-level Atari 200x faster
Steven Kapturowski (DeepMind), Adria Puigdomenech Badia
Reinforcement LearningVideoBenchmark
🎯 What it does: A reinforcement learning agent named MEME has been developed, improving the data efficiency of Agent57, surpassing human benchmarks in all 57 Atari games within just 390 million frames.
Humanly Certifying Superhuman Classifiers
Qiongkai Xu (University of Melbourne), Chenchen Xu (Amazon)
ClassificationTransformerSupervised Fine-TuningText
🎯 What it does: A theoretical framework was constructed to estimate the unobserved 'oracle' accuracy using the consistency among multiple observed human annotators, providing a lower bound on the accuracy of machine learning models, thus enabling a formal determination of whether the model surpasses average human performance.
Hungry Hungry Hippos: Towards Language Modeling with State Space Models
Daniel Y Fu, Christopher Re (University at Buffalo)
TransformerLarge Language ModelText
🎯 What it does: This paper studies the application of state space models (SSM) in language modeling and proposes a new H3 layer and FLASHCONV acceleration algorithm to enhance model performance and hardware efficiency.
Hybrid RL: Using both offline and online data can make RL efficient
Yuda Song (Carnegie Mellon University), Wen Sun (Cornell University)
Computational EfficiencyReinforcement Learning
🎯 What it does: A hybrid reinforcement learning framework, Hybrid RL, is proposed, and a Hybrid Q-Learning (Hy-Q) algorithm based on fitted Q iteration is designed, with its efficiency verified both theoretically and experimentally.
Hyper-Decision Transformer for Efficient Online Policy Adaptation
Mengdi Xu (Carnegie Mellon University), Chuang Gan (Massachusetts Institute of Technology)
Robotic IntelligenceMeta LearningTransformerReinforcement LearningPrompt EngineeringSequential
🎯 What it does: This paper proposes the Hyper-Decision Transformer (HDT), a parameter-efficient online policy transfer framework built on the existing large Decision Transformer, which can quickly adapt to unseen tasks with only a few demonstrations (with or without actions).
HypeR: Multitask Hyper-Prompted Training Enables Large-Scale Retrieval Generalization
ZeFeng Cai, Daxin Jiang (Microsoft Corporation)
RetrievalTransformerPrompt EngineeringText
🎯 What it does: HYPER is proposed—a multi-task hyper-prompt training mechanism that utilizes a Query Conditional Prompt Synthesizer (QPS) and Contrastive Prompt Regularization (CPR) to enable neural retrievers to uniformly handle queries across different tasks and domains, achieving cross-domain knowledge transfer through multi-task training.
Hyperbolic Deep Reinforcement Learning
Edoardo Cetin (King's College London), Jonathan J Hunt (Twitter Inc.)
Reinforcement Learning
🎯 What it does: This paper proposes the use of hyperbolic space to encode the latent representations of deep reinforcement learning models and designs the S-RYM (Spectrally-regularized Hyperbolic Mappings) stabilization method, achieving end-to-end training in RL.
Hyperbolic Self-paced Learning for Self-supervised Skeleton-based Action Representations
Luca Franco (Sapienza University of Rome), Fabio Galasso (Sapienza University of Rome)
RecognitionRepresentation LearningGraph Neural NetworkContrastive LearningVideo
🎯 What it does: The HYSP model is proposed, utilizing self-supervised learning and self-paced learning in hyperbolic space for representation learning of skeletal actions.
HyperDeepONet: learning operator with complex target function space using the limited resources via hypernetwork
Jae Yong Lee (Korea Institute for Advanced Study), Hyung Ju Hwang (Pohang University of Science and Technology)
Neural Architecture Search
🎯 What it does: A HyperDeepONet framework is proposed, which uses a hypernetwork to generate target network parameters, learning complex operators with lower complexity.
Hyperparameter Optimization through Neural Network Partitioning
Bruno Kacper Mlodozeniec (University of Cambridge), Christos Louizos (Qualcomm AI Research)
OptimizationFederated LearningHyperparameter SearchImage
🎯 What it does: A single training hyperparameter optimization method without a validation set based on network weight partitioning and data block division is proposed.
IDEAL: Query-Efficient Data-Free Learning from Black-Box Models
Jie Zhang (Zhejiang University), Lingjuan Lyu (Sony AI)
GenerationData SynthesisKnowledge DistillationGenerative Adversarial NetworkImage
🎯 What it does: A method called IDEAL is proposed, which efficiently learns from a teacher model and trains a student model in environments with no data, black-box constraints, hard labels, and limited query counts.
Identifiability Results for Multimodal Contrastive Learning
Imant Daunhawer (ETH Zurich), Julia E Vogt
Data SynthesisRepresentation LearningContrastive LearningMultimodality
🎯 What it does: This study investigates the identifiability of multimodal contrastive learning, proving that under different generative mechanisms and modality-specific latent variables, contrastive learning can achieve block-level identifiability for shared content factors.
ILA-DA: Improving Transferability of Intermediate Level Attack with Data Augmentation
Chiu Wai Yan (Hong Kong University of Science and Technology), Dit-Yan Yeung (Hong Kong University of Science and Technology)
Adversarial AttackImage
🎯 What it does: Improved the transferability of Intermediate Layer Attacks (ILA) by introducing three data augmentation techniques during the ILA process to generate more aggressive adversarial samples.
Image as Set of Points
Xu Ma (Northeastern University), Yun Fu (Northeastern University)
ClassificationObject DetectionSegmentationImagePoint Cloud
🎯 What it does: A new visual representation method called Context Clusters (CoC) is proposed, treating images as unordered point sets and utilizing a simplified clustering algorithm to extract features, completely avoiding the use of convolution or attention mechanisms.
Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction
David Klee, Robin Walters (Northeastern University)
Pose EstimationImage
🎯 What it does: Predicting object pose from a single image and outputting a complete probability distribution regarding rotation.
ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations
Badr Youbi Idrissi (Fundamental AI Research), Mark Ibrahim (Fundamental AI Research)
Explainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: The ImageNet-X dataset was constructed, with human annotations performed on the ImageNet validation set and 12,000 training samples, recording 16 categories of visual factors (such as pose, background, lighting, etc.).
Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules
Kazuki Irie (Swiss AI Lab), Jürgen Schmidhuber (Swiss AI Lab)
GenerationData SynthesisRecurrent Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: Proposes Fast Weight Painter (FPA), which applies the outer product learning rule of rapid weight generation to image generation, gradually stacking rank-one updates to obtain the final image.
ImaginaryNet: Learning Object Detectors without Real Images and Annotations
Minheng Ni (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
Object DetectionLarge Language ModelDiffusion modelContrastive LearningImage
🎯 What it does: This paper proposes a learning paradigm for training object detectors entirely based on synthetic images—Imaginary-Supervised Object Detection (ISOD), which does not rely on any real images or manual annotations.
Imbalanced Semi-supervised Learning with Bias Adaptive Classifier
Renzhen Wang (Xi'an Jiaotong University), Deyu Meng (Xi'an Jiaotong University)
ClassificationSupervised Fine-TuningImage
🎯 What it does: A pseudo-label learning framework L2AC is proposed to adapt to class imbalance, using a bias-adaptive classifier to suppress training bias in pseudo-label semi-supervised learning and improve the performance of minority classes.
Imitating Graph-Based Planning with Goal-Conditioned Policies
Junsu Kim (Korea Advanced Institute of Science and Technology), Jinwoo Shin (Korea Advanced Institute of Science and Technology)
Computational EfficiencyKnowledge DistillationRobotic IntelligenceGraph Neural NetworkReinforcement LearningAgentic AIGraph
🎯 What it does: The PIG framework is proposed, which utilizes graph planning to generate subgoals and distills the subgoal policies into the target policy through self-imitation learning, significantly improving the sample efficiency of goal-conditioned RL.
Imitating Human Behaviour with Diffusion Models
Tim Pearce (Microsoft Research), Sam Devlin (Microsoft Research)
Robotic IntelligenceTransformerDiffusion modelSequential
🎯 What it does: This paper introduces diffusion models into behavior cloning, constructing a conditional distribution from observation to action, and proposes various architectures (Basic MLP, MLP Sieve, Transformer) and reliable sampling schemes (Diffusion-X, Diffusion-KDE) for mimicking human behavior in continuous or mixed action spaces.
Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
Spencer Frei (University of California Berkeley), Wei Hu (University of Michigan)
OptimizationTabular
🎯 What it does: This study investigates the gradient flow and implicit bias of two-layer Leaky ReLU networks under high-dimensional nearly orthogonal samples, proving that the gradient flow converges to the KKT point of the maximum margin problem, and that the rank of the weight matrix does not exceed 2, resulting in a linear decision boundary for the network. Additionally, it is shown that in gradient descent, with small initialization variance, the rank can be significantly reduced in the first step, and the stable rank remains constant throughout the training process.
Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions
Arthur Jacot (Courant Institute of Mathematical Sciences New York University)
ClassificationOptimizationRepresentation LearningAuto EncoderImageTabular
🎯 What it does: This study investigates the implicit bias of deep homogeneous networks under L2 regularization or exponential decay loss, proposes a representation cost, and introduces two sparsity measures (Jacobian rank and Bottleneck rank). It proves that in the limit of infinite depth, the normalized representation cost approaches the lower and upper bounds of these two ranks, and explores the behavior of the network in rank-1 reduction and intermediate depth recovery of true rank at finite depths, while explaining how rank affects the topology of classification decision boundaries and the inherent denoising capability of autoencoders.
Implicit Regularization for Group Sparsity
Jiangyuan Li (Texas A&M University), Raymond K. W. Wong (Texas A&M University)
OptimizationTabular
🎯 What it does: This paper constructs a new 'diagonal grouped linear neural network' reparameterization and proves that gradient descent will naturally converge to a solution with a group sparse structure without explicit regularization.
Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent
Avrajit Ghosh (Michigan State University), Rongrong Wang (Michigan State University)
OptimizationImage
🎯 What it does: This paper studies the implicit regularization in momentum accelerated stochastic gradient descent (GD+M) and explores the role of the momentum parameter β in implicit regularization.
Impossibly Good Experts and How to Follow Them
Aaron Walsman (University of Washington), Ali Farhadi (University of Washington)
Knowledge DistillationReinforcement LearningAgentic AIMultimodality
🎯 What it does: The paper discusses how to learn the optimal strategy from 'unachievable' experts in situations where experts have more information than learners, and proposes the ELF Distillation method.
Improved Convergence of Differential Private SGD with Gradient Clipping
Huang Fang (Cognitive Computing Lab Baidu Research), Ping Li (Cognitive Computing Lab Baidu Research)
OptimizationSafty and PrivacyImage
🎯 What it does: This paper studies the convergence of Differentially Private Stochastic Gradient Descent with Gradient Clipping (DP-SGD-GC) and proposes that DP-SGD-GC can achieve vanishing utility bounds in unbounded domains without requiring global Lipschitz continuity assumptions.
Improved Learning-augmented Algorithms for k-means and k-medians Clustering
Thy Dinh Nguyen (Northeastern University), Huy Nguyen (Northeastern University)
OptimizationImage
🎯 What it does: Proposed a learning-enhanced k-means and k-medians clustering algorithm that can guarantee approximate optimality even with a predicted label error rate of up to 1/2.
Improved Sample Complexity for Reward-free Reinforcement Learning under Low-rank MDPs
Yuan Cheng (University of Science and Technology of China), Jing Yang (Pennsylvania State University)
Representation LearningReinforcement Learning
🎯 What it does: A new reward-agnostic reinforcement learning algorithm RAFFLE is proposed to achieve approximately optimal policies and system identification under low-rank MDPs, and further achieve approximately accurate representation learning through the planning phase.
Improved Training of Physics-Informed Neural Networks Using Energy-Based Priors: a Study on Electrical Impedance Tomography
Akarsh Pokkunuru (University of North Carolina), Taufiquar Khan (University of North Carolina)
SegmentationData SynthesisAnomaly DetectionOptimizationImagePhysics Related
🎯 What it does: In the electrical resistivity imaging (EIT) problem, a data-driven energy-based model (EBM) is proposed to be integrated as a prior into the training of Physics-Informed Neural Networks (PINN), forming a semi-inverse problem-solving framework.
Improving Deep Policy Gradients with Value Function Search
Enrico Marchesini (Northeastern University), Christopher Amato (Northeastern University)
Reinforcement LearningSequential
🎯 What it does: Proposes Value Function Search (VFS) — a gradient-independent two-scale perturbation population search method aimed at improving the value network in deep policy gradient algorithms;
Improving Deep Regression with Ordinal Entropy
Shihao Zhang (National University of Singapore), Angela Yao (Huawei International Pte Ltd)
Depth EstimationConvolutional Neural NetworkImage
🎯 What it does: In deep regression tasks, the differences in feature learning between regression and classification are analyzed from the perspective of mutual information. It is found that using MSE for regression tends to lead to low feature entropy. An 'ordinal entropy' regularization term is proposed to maintain both ordinal relationships and high entropy in the feature space, and this regularization is applied as a plugin to various regression models.
Improving Differentiable Neural Architecture Search by Encouraging Transferability
Parth Sheth (University of Pennsylvania), Pengtao Xie (University of California San Diego)
OptimizationNeural Architecture SearchImage
🎯 What it does: Based on differentiable neural architecture search (such as DARTS), the TETLO (Three-layer Optimization) framework is proposed, which enhances the generalization performance and stability of the searched network on the test set by ensuring good transferability of knowledge from the auxiliary model to the main model.
Improving Object-centric Learning with Query Optimization
Baoxiong Jia (University of California Los Angeles), Siyuan Huang (National Key Laboratory of General Artificial Intelligence)
OptimizationRepresentation LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: A variant of Slot-Attention based on learnable query initialization combined with second-order optimization (BO-QSA) is proposed to achieve unsupervised object-centric representation learning.
Improving Out-of-distribution Generalization with Indirection Representations
Kha Pham (Deakin University), Truyen Tran (Deakin University)
Domain AdaptationRepresentation LearningGraph Neural NetworkTransformerImageText
🎯 What it does: The Indirection Layer (InLay) module is proposed, treating the input sequence as a fully weighted graph. Through indirection and graph propagation, a symbolic indirect representation is obtained, thereby enhancing the model's generalization ability in out-of-distribution (OOD) situations.
Improving the imputation of missing data with Markov Blanket discovery
Yang Liu (Queen Mary University of London), Anthony Constantinou
Tabular
🎯 What it does: A feature selection method based on Markov Blanket (MBFS) is proposed, which is embedded in MissForest to form a new imputation algorithm (MBMF) aimed at improving the filling of missing data.
In-context Reinforcement Learning with Algorithm Distillation
Michael Laskin (DeepMind), Volodymyr Mnih (DeepMind)
Knowledge DistillationTransformerReinforcement LearningSequential
🎯 What it does: Using causal transformers to perform behavior cloning on the learning history of reinforcement learning algorithms, resulting in an online learning model that can self-improve through context without updating parameters.
In-sample Actor Critic for Offline Reinforcement Learning
Hongchang Zhang (Tsinghua University), Xiangyang Ji (Tsinghua University)
Reinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes an Actor-Critic algorithm (IAC) that uses Sampling Importance Resampling (SIR) in offline reinforcement learning, conducting unbiased policy evaluation by only using action values from the dataset, thereby eliminating out-of-distribution inference errors.
In-Situ Text-Only Adaptation of Speech Models with Low-Overhead Speech Imputations
Ashish Mittal (IBM Research), Preethi Jyothi (Indian Institute of Technology Bombay)
Domain AdaptationRecurrent Neural NetworkSupervised Fine-TuningTextAudio
🎯 What it does: This paper proposes a silent input adaptation method for the RNN-Transducer (RNN-T) model that only uses target domain text data, called TOLSTOI.
InCoder: A Generative Model for Code Infilling and Synthesis
Daniel Fried (Carnegie Mellon University), Mike Lewis (Facebook AI Research)
GenerationAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: This paper presents InCoder, a unified generative code model capable of both program synthesis (left-to-right generation) and code completion (masking and filling in), trained on a large-scale code corpus.
Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks
Charles Jin (Massachusetts Institute of Technology), Martin Rinard (Massachusetts Institute of Technology)
ClassificationAnomaly DetectionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A clustering method based on the 'incompatibility' characteristics between data subsets is proposed to detect and eliminate backdoor attack samples in the training set.
Incremental Learning of Structured Memory via Closed-Loop Transcription
Shengbang Tong (University of California), Yi Ma (University of California)
ClassificationRepresentation LearningAuto EncoderImage
🎯 What it does: This paper proposes a self-contained, fixed-capacity incremental learning framework called i-CTRL, which utilizes Closed-loop Transcription (CTRL) and Linear Discriminative Representation (LDR) to achieve structured memory for multiple categories while balancing discriminative and generative capabilities.
Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learning
Hao He (Massachusetts Institute of Technology), Dina Katabi (Massachusetts Institute of Technology)
Representation LearningAdversarial AttackContrastive LearningImage
🎯 What it does: This paper proposes and evaluates a non-targeted poisoning attack for unsupervised contrastive learning—Contrastive Poisoning (CP), which can significantly reduce the performance of linear probes in CL models and can also attack supervised models based on CL learning.
Individual Privacy Accounting with Gaussian Differential Privacy
Antti Koskela (Nokia Bell Labs University of Helsinki), Antti Honkela (University of Helsinki)
Safty and PrivacyGaussian SplattingBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes an individual privacy measurement framework based on Gaussian Differential Privacy (GDP), which can progressively track and limit the privacy loss of each data point in a fully adaptive algorithm sequence.
Inequality phenomenon in $l_{\infty}$-adversarial training, and its unrealized threats
Ranjie Duan (Alibaba Group), Hui Xue'
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper explores and quantifies the 'inequality phenomenon' that occurs during l∞ adversarial training, where the model relies on a small number of extremely high-weight features for predictions; it also validates the robustness flaws caused by this phenomenon through noise and occlusion attacks.
Information Plane Analysis for Dropout Neural Networks
Linara Adilova (Ruhr University Bochum), Asja Fischer (Ruhr University Bochum)
CompressionRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: Proposes and verifies that the mutual information (MI) between input and hidden representations in Dropout neural networks with continuous noise is finite, enabling information plane (IP) analysis;
Information-Theoretic Analysis of Unsupervised Domain Adaptation
Ziqiao Wang (University of Ottawa), Yongyi Mao (University of Ottawa)
Domain AdaptationImage
🎯 What it does: This paper proposes an information-theoretic framework that provides new upper bounds for two types of generalization errors in Unsupervised Domain Adaptation (UDA) (the difference in overall risk between source and target domains and the difference in empirical risk of the source). Based on this, two simple yet effective improvement strategies are designed: gradient penalty and label information control.
Information-Theoretic Diffusion
Xianghao Kong (University of California Riverside), Greg Ver Steeg (University of California Riverside)
GenerationData SynthesisOptimizationDiffusion modelImage
🎯 What it does: A framework for diffusion models based on information theory is proposed, utilizing the I-MMSE relationship to completely equate probability density with the optimal denoising regression function, thereby achieving a unified estimation of continuous and discrete probabilities.
InPL: Pseudo-labeling the Inliers First for Imbalanced Semi-supervised Learning
Zhuoran Yu (University of Wisconsin Madison), Yong Jae Lee (University of Wisconsin Madison)
ClassificationDomain AdaptationContrastive LearningImage
🎯 What it does: This paper studies the pseudo-labeling method in imbalanced semi-supervised learning and proposes a pseudo-labeling strategy based on energy scores.