ICLR 2023 Papers — Page 2
International Conference on Learning Representations · 1573 papers
An Exact Poly-Time Membership-Queries Algorithm for Extracting a Three-Layer ReLU Network
Amit Daniely (Hebrew University), Elad Granot (Hebrew University)
OptimizationComputational Efficiency
🎯 What it does: The paper proposes a polynomial-time algorithm that accurately extracts parameters of two-layer and three-layer ReLU neural networks using membership queries under the general position assumption.
An Extensible Multi-modal Multi-task Object Dataset with Materials
Trevor Scott Standley (Stanford University), Silvio Savarese (Salesforce Research)
Object DetectionData SynthesisConvolutional Neural NetworkImageTextMultimodalityTabular
🎯 What it does: A multimodal (image, text, table) multi-task labeled EMMa dataset has been constructed, containing over 2.8 million Amazon products, and a Smart Labeling framework for automated expansion of binary attributes has been proposed.
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Rinon Gal (Tel-Aviv University), Daniel Cohen-or
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: This paper proposes a text inversion technique that learns pseudo-words in the embedding space of a pre-trained text-to-image model, enabling personalized generation and editing of specific concepts using a small number of images.
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
Ting Chen (Google Research), Geoffrey Hinton (Google Research)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: The Bit Diffusion method is proposed to encode discrete data into binary bits and map them to continuous values (Analog Bits), enabling continuous time diffusion models to generate discrete images and text, and introduces Self-Conditioning and Asymmetric Time Intervals to enhance sampling quality.
Analogy-Forming Transformers for Few-Shot 3D Parsing
Nikolaos Gkanatsios (Carnegie Mellon University), Katerina Fragkiadaki (Carnegie Mellon University)
SegmentationRetrievalMeta LearningTransformerPoint CloudRetrieval-Augmented Generation
🎯 What it does: A semi-parametric Analogical Networks model is proposed, utilizing external memory retrieval and analogical reasoning to achieve fine-grained segmentation of 3D scenes.
Analyzing Tree Architectures in Ensembles via Neural Tangent Kernel
Ryuichi Kanoh (National Institute of Informatics), Mahito Sugiyama (National Institute of Informatics)
ClassificationTabular
🎯 What it does: This study investigates the neural tangent kernel (NTK) theory of soft tree ensembles under arbitrary tree structures and proves that the distribution of tree depth (the number of leaves at each level) determines the final NTK. It further illustrates that asymmetric trees (such as decision lists) do not exhibit degradation phenomena caused by depth.
Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections
Edward De Brouwer (KU Leuven), Rahul G Krishnan
Time SeriesBiomedical DataElectronic Health RecordsOrdinary Differential Equation
🎯 What it does: This paper proposes PolyODE, a temporal model that embeds orthogonal polynomial projections into neural ODEs, capable of maintaining long-term memory and performing backward reconstruction under irregular sampling.
Anisotropic Message Passing: Graph Neural Networks with Directional and Long-Range Interactions
Moritz Thürlemann, Sereina Riniker (ETH Zurich)
Graph Neural NetworkGraph
🎯 What it does: A graph neural network with anisotropic multipole moments (AMP) has been constructed, which can explicitly encode directionality and long-range interactions, and is applied to QM/MM simulations.
Anti-Symmetric DGN: a stable architecture for Deep Graph Networks
Alessio Gravina (University of Pisa), Claudio Gallicchio (University of Pisa)
Graph Neural NetworkGraphOrdinary Differential Equation
🎯 What it does: This paper proposes the Anti-Symmetric Deep Graph Network (A-DGN), which views graph neural networks as continuous differential equations on graphs. By using an anti-symmetric weight matrix, it ensures stability and non-dissipation, thereby maintaining long-term node dependencies in deep structures and avoiding gradient vanishing/explosion.
Any-scale Balanced Samplers for Discrete Space
Haoran Sun (Georgia Tech), Hanjun Dai (Google Research)
OptimizationComputational EfficiencyImageStochastic Differential Equation
🎯 What it does: An Arbitrary Scale Balancing Sampler (AB Sampler) is proposed, which improves the local balancing sampler in discrete space by adaptively adjusting parameters (σ, α) and introducing second-order gradient approximation and the Gaussian integral trick, allowing the proposal distribution to maintain efficient sampling at arbitrary scales.
AnyDA: Anytime Domain Adaptation
Omprakash Chakraborty (Indian Institute of Technology Kharagpur), Abir Das (Indian Institute of Technology Kharagpur)
Domain AdaptationKnowledge DistillationNeural Architecture SearchImage
🎯 What it does: This paper proposes an AnyDA (Any Domain Adaptation) framework that utilizes a single switchable network with adjustable depth, width, and input resolution to achieve efficient cross-domain task inference under the condition of labeled source domain and unlabeled target domain.
Approximate Bayesian Inference with Stein Functional Variational Gradient Descent
Tobias Pielok (Ludwig Maximilian University of Munich), David Rügamer (Ludwig Maximilian University of Munich)
OptimizationTabular
🎯 What it does: A Stein Variational Gradient Descent method in function space (SFVGD) is proposed for approximating posterior stochastic processes, which is then used for training models such as Bayesian Neural Networks (SFVNN) and Gradient Boosting (SFVGB).
Approximate Nearest Neighbor Search through Modern Error-Correcting Codes
Noam Touitou (Amazon), Nissim Halabi
RetrievalOptimizationAuto EncoderTabular
🎯 What it does: A novel approximate nearest neighbor search algorithm, PCNN, is proposed, which utilizes Polar Codes for clustering in high-dimensional binary embedding space and achieves multi-probe through list decoding.
Approximate Vanishing Ideal Computations at Scale
Elias Samuel Wirth (Berlin Institute of Technology), Sebastian Pokutta (Berlin Institute of Technology)
OptimizationTabular
🎯 What it does: This paper focuses on the scalable improvement and theoretical analysis of the algorithm OAVI for the approximate vanishing ideal of a set of data points, proving that its time complexity is linear with respect to the number of samples.
Approximation and non-parametric estimation of functions over high-dimensional spheres via deep ReLU networks
Namjoon Suh (Georgia Institute of Technology), Xiaoming Huo (Georgia Institute of Technology)
🎯 What it does: This paper studies the approximation and statistical estimation of Sobolev-type functions on high-dimensional spheres, proving that deep ReLU networks can achieve efficient approximation as the dimension approaches infinity.
Arbitrary Virtual Try-on Network: Characteristics Representation and Trade-off between Body and Clothing
Yu Liu (Donghua University), Shuicheng YAN
Image TranslationGenerationPose EstimationGenerative Adversarial NetworkImage
🎯 What it does: A virtual try-on network for arbitrary types of clothing, AVTON, is proposed, which achieves high-quality try-on images through three modules: limb prediction, improved geometric matching, and feature fusion.
ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations
Xuyang Zhao (Peking University), Weiran Huang (Shanghai Jiao Tong University)
Domain AdaptationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper explores the transfer performance of self-supervised contrastive learning under distribution shift through theoretical analysis and proposes a new adversarially robust contrastive learning method called ArCL.
Are More Layers Beneficial to Graph Transformers?
Haiteng Zhao (Peking University), Furu Wei (Microsoft Research)
Graph Neural NetworkTransformerGraph
🎯 What it does: This study investigates the bottleneck of increasing the depth of graph Transformers and proposes the DeepGraph model, which enhances graph representation performance at deeper layers through substructure tokens and local attention.
Artificial Neuronal Ensembles with Learned Context Dependent Gating
Matthew James Tilley, David Freedman
ClassificationImage
🎯 What it does: A framework based on Learnable Contextual Gating (LXDG) is proposed to dynamically allocate and recall sparse 'neuron clusters' in artificial neural networks, thereby alleviating catastrophic forgetting in continual learning.
Ask Me Anything: A simple strategy for prompting language models
Simran Arora (Stanford University), Christopher Re
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes an aggregation method that utilizes multiple imperfect but effective prompts to improve the zero/few-shot reasoning performance of large language models.
Associative Memory Augmented Asynchronous Spatiotemporal Representation Learning for Event-based Perception
Uday Kamal (Georgia Institute of Technology), Saibal Mukhopadhyay (Georgia Institute of Technology)
ClassificationRepresentation LearningRecurrent Neural NetworkTransformerContrastive LearningImageVideo
🎯 What it does: An asynchronous event-aware framework called EventFormer is proposed, which maps event streams to associative memory and dynamically updates it through a recursive module.
Asymptotic Instance-Optimal Algorithms for Interactive Decision Making
Kefan Dong (Stanford University), Tengyu Ma (Stanford University)
OptimizationReinforcement Learning
🎯 What it does: A general interactive decision-making algorithm (including bandits, linear bandits, and table-based reinforcement learning) that achieves asymptotic instance optimality on any instance is proposed, called Test-to-Commit (T2C).
Asynchronous Distributed Bilevel Optimization
Yang Jiao (Tongji University), Chengtao Jian (Tongji University)
OptimizationTabular
🎯 What it does: An Asynchronous Distributed Bi-level Optimization algorithm (ADBO) is proposed, which achieves bi-level optimization under non-convex upper and lower objective functions through single-loop updates, and provides theoretical proofs of convergence and iteration complexity.
Asynchronous Gradient Play in Zero-Sum Multi-agent Games
Ruicheng Ao (Peking University), Yuejie Chi (Carnegie Mellon University)
OptimizationReinforcement Learning from Human FeedbackTabular
🎯 What it does: This paper studies asynchronous gradient play in zero-sum and multiplayer polarized games, proposing the Optimistic Multiplicative Weight Update (OMWU) algorithm with entropy regularization, and proving its finite-time convergence in feedback environments with random, fixed, or constrained delays.
AudioGen: Textually Guided Audio Generation
Felix Kreuk (Meta AI), Yossi Adi (Hebrew University of Jerusalem)
GenerationData SynthesisRecurrent Neural NetworkTransformerAuto EncoderGenerative Adversarial NetworkTextAudio
🎯 What it does: Proposes the AUDIOGEN model, which generates high-quality audio using text prompts;
Augmentation Component Analysis: Modeling Similarity via the Augmentation Overlaps
Lu Han (Nanjing University), De-Chuan Zhan (Nanjing University)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: Proposes a method to measure sample similarity by retaining augmented distribution overlap, and introduces the Augmentation Component Analysis (ACA) method to effectively implement this idea.
Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation
Ziqi Wang (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
Knowledge DistillationData-Centric LearningTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a data augmentation method called AugPro, which transforms continuous representations generated by representation interpolation into discrete tokens through projection, and is used for knowledge distillation.
Auto-Encoding Goodness of Fit
Aaron Palmer (University of Connecticut), Jinbo Bi (University of Connecticut)
GenerationData SynthesisAuto EncoderImage
🎯 What it does: This paper proposes a novel generative autoencoder called the Goodness-of-Fit Autoencoder (GoFAE), which introduces a normality GoF test as a regularization objective at the mini-batch level and uses higher criticism at the global level to automatically select the regularization coefficient, thereby achieving statistical matching between the latent space distribution and the Gaussian prior class.
Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?
Runpei Dong (Xi'an Jiaotong University), Kaisheng Ma (Tsinghua University)
Knowledge DistillationRepresentation LearningTransformerPrompt EngineeringAuto EncoderPoint Cloud
🎯 What it does: This paper proposes a self-supervised framework called ACT, which transforms pre-trained 2D visual Transformers or language Transformers into cross-modal teachers through an autoencoder, using their latent space features as the masking modeling target for a 3D point cloud Transformer student, achieving cross-modal knowledge distillation.
AutoGT: Automated Graph Transformer Architecture Search
Zizhao Zhang (Tsinghua University), Wenwu Zhu (Tsinghua University)
ClassificationDrug DiscoveryNeural Architecture SearchGraph Neural NetworkTransformerGraph
🎯 What it does: This paper proposes the AutoGT framework, which automatically searches for the optimal combination of graph Transformer structures and graph encoding strategies to complete graph classification tasks.
Automated Data Augmentations for Graph Classification
Youzhi Luo (Texas A&M University), Shuiwang Ji (Texas A&M University)
ClassificationRecurrent Neural NetworkGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: An automated graph data augmentation framework called GraphAug is proposed for graph classification tasks, generating augmented samples while keeping the labels unchanged.
Automatic Chain of Thought Prompting in Large Language Models
Zhuosheng Zhang (Shanghai Jiao Tong University), Alex Smola (Amazon Web Services)
OptimizationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes Auto-CoT, which utilizes large language models (LLMs) to automatically generate chain-of-thought (CoT) examples, achieving a few-shot prompting that does not rely on manual design, thereby providing performance comparable to manually crafted examples in reasoning tasks.
Automating Nearest Neighbor Search Configuration with Constrained Optimization
Philip Sun (Google Research), Sanjiv Kumar (Google Research)
RetrievalOptimizationTabular
🎯 What it does: This paper proposes a constraint optimization-based quantization ANN hyperparameter tuning method that can automatically generate parameter configurations close to the Pareto front by only specifying the desired search cost or recall rate.
Autoregressive Conditional Neural Processes
Wessel Bruinsma, Richard E Turner
Convolutional Neural NetworkTime Series
🎯 What it does: This paper proposes deploying existing Conditional Neural Processes (CNP) in an autoregressive (AR) manner to construct joint predictive distributions and generate coherent function samples without changing the model architecture and training process.
AutoTransfer: AutoML with Knowledge Transfer - An Application to Graph Neural Networks
Kaidi Cao (Stanford University), Jure Leskovec (Stanford University)
OptimizationHyperparameter SearchGraph Neural NetworkGraph
🎯 What it does: This paper proposes AUTOTRANSFER, which utilizes a task-model repository and Fisher Information Matrix-driven task embeddings to achieve cross-task knowledge transfer in graph neural network AutoML searches, significantly reducing search costs.
Average Sensitivity of Decision Tree Learning
Satoshi Hara (Osaka University), Yuichi Yoshida (National Institute of Informatics)
ClassificationOptimizationTabular
🎯 What it does: A decision tree learning algorithm that is stable against training data perturbations has been designed, along with theoretical proofs and experimental validation.
Avoiding spurious correlations via logit correction
Sheng Liu (New York University), Carlos Fernandez-Granda (New York University)
ClassificationData-Centric LearningImage
🎯 What it does: This paper proposes a logit-corrected loss function (LC) and a Group MixUp data augmentation method to suppress spurious correlations in samples during training, thereby improving the balanced accuracy of the model across different subgroups.
Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning
Beren Millidge (University of Oxford), Rafal Bogacz (University of Oxford)
Contrastive LearningImage
🎯 What it does: This paper proposes a unified theory that explains how the Energy-Based Model (EBM) approximates backpropagation in the limit of infinitesimal inference, and derives a new algorithm called PC-Nudge.
Backpropagation through Combinatorial Algorithms: Identity with Projection Works
Subham Sekhar Sahoo (Cornell University), Georg Martius (Max Planck Institute for Intelligent Systems)
RetrievalOptimizationImage
🎯 What it does: A gradient substitution method is proposed that treats discrete combination operators as negative identity mappings during backpropagation, and improves stability through projection of the cost vector and noise regularization.
Backstepping Temporal Difference Learning
Han-Dong Lim (Korea Advanced Institute of Science and Technology), Donghwan Lee (Korea Advanced Institute of Science and Technology)
Reinforcement LearningTabularOrdinary Differential Equation
🎯 What it does: This paper proposes an offline temporal difference (TD) learning framework based on backstepping control theory, deriving a new offline TD algorithm—Backstepping TD (BTD)—as well as single time-scale variants of algorithms such as TDC and TDC++. Its convergence is proven.
Bag of Tricks for Unsupervised Text-to-Speech
Yi Ren (SEA AI Lab), Shuicheng YAN
GenerationData SynthesisTransformerAuto EncoderTextAudio
🎯 What it does: This paper studies a complete unsupervised text-to-speech (TTS) process, completing TTS training using low-quality, unpaired speech and text data.
BALTO: fast tensor program optimization with diversity-based active learning
Jun Bi (University of Science and Technology of China), Yunji Chen (SKL of Processors, ICT, CAS)
OptimizationTabular
🎯 What it does: This paper presents BALTO, a fast tensor program optimization method that combines bias diversity active learning.
Basic Binary Convolution Unit for Binarized Image Restoration Network
Bin Xia (Tsinghua University), Luc Van Gool (ETH Zurich)
RestorationSuper ResolutionConvolutional Neural NetworkImage
🎯 What it does: This paper studies a binarized image restoration network and proposes a Basic Binarized Convolution Unit (BBCU). It designs different variants for the four modules of the network: head, body, upsampling, and tail, achieving an efficient and lightweight image restoration model.
Batch Multivalid Conformal Prediction
Christopher Jung, Aaron Roth
Tabular
🎯 What it does: Two batch multivalid conformal prediction algorithms, BatchGCP and BatchMVP, are proposed, which can achieve conditional coverage guarantees for any group set on exchangeable data and maintain calibration at the threshold;
BAYES RISK CTC: CONTROLLABLE CTC ALIGNMENT IN SEQUENCE-TO-SEQUENCE TASKS
Jinchuan Tian (Tencent AI Lab), Shinji Watanabe (Carnegie Mellon University)
RecognitionOptimizationComputational EfficiencySequentialAudio
🎯 What it does: A Bayes Risk CTC (BRCTC) criterion is proposed, which controls the alignment prediction of CTC by assigning risk values to different paths, achieving controllable alignment.
Bayes-MIL: A New Probabilistic Perspective on Attention-based Multiple Instance Learning for Whole Slide Images
Yufei CUI, Antoni B. Chan (City University of Hong Kong)
ClassificationExplainability and InterpretabilityConvolutional Neural NetworkImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a multi-instance learning framework based on a Bayesian probability perspective (Bayes-MIL) for weakly supervised diagnosis of whole slide images (WSI), and interprets attention weights through model uncertainty.
Bayesian Oracle for bounding information gain in neural encoding models
Konstantin-Klemens Lurz (University of Tübingen), Fabian H. Sinz (University Göttingen)
Convolutional Neural NetworkImage
🎯 What it does: This paper studies how to compute interpretable evaluation metrics NInGa for likelihood-based neural coding models and proposes a Bayesian gold standard estimation based on posterior predictive distribution to address the issue of overconfidence in upper bound estimation under sparse low-repetition data. It also derives that posterior prediction under zero-inflated distribution requires only one-dimensional integration and establishes a linear relationship between NInGa and commonly used metrics (FEVE, correlation coefficient).
BC-IRL: Learning Generalizable Reward Functions from Demonstrations
Andrew Szot (Meta AI), Franziska Meier (Meta AI)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: A new inverse reinforcement learning method called BC-IRL is proposed, which utilizes behavior cloning loss as the outer objective and employs gradient-based bi-level optimization to learn a generalizable reward function using reinforcement learning in the inner loop, and based on this, BC-IRL-PPO is implemented.
Become a Proficient Player with Limited Data through Watching Pure Videos
Weirui Ye (Tsinghua University), Yang Gao (Tsinghua University)
Convolutional Neural NetworkReinforcement LearningVideo
🎯 What it does: The paper proposes a framework that utilizes a pure video (without action labels) pre-training model to enhance the sample efficiency of models based on reinforcement learning.
Behavior Prior Representation learning for Offline Reinforcement Learning
Hongyu Zang (Beijing Institute of Technology), Romain Laroche (Mila)
Representation LearningReinforcement LearningAuto EncoderContrastive LearningTabularSequential
🎯 What it does: The Behavior Prior Representation (BPR) method is proposed, which learns state representations by behavior cloning the behavior policy from offline data, and applies any offline RL algorithm for policy learning after freezing that representation.
Behavior Proximal Policy Optimization
Zifeng Zhuang (Zhejiang University), Yilang Guo (Beijing Jiaotong University)
Reinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes a BPPO algorithm for offline reinforcement learning based on PPO, which utilizes offline data to monotonically improve the behavior policy, thus achieving offline training without additional constraints.
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition
Jianhao Ma (University of Michigan), Salar Fattahi (University of Michigan)
OptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: By constructing orthogonal basis function decompositions of the solution trajectories of gradient descent, a general analytical framework is proposed along with a convergence proof. This framework is then applied to symmetric matrix decomposition, orthogonal symmetric tensor decomposition, and the training of various deep neural networks.
Benchmarking Constraint Inference in Inverse Reinforcement Learning
Guiliang Liu (Chinese University of Hong Kong), Pascal Poupart (University of Waterloo)
Autonomous DrivingRobotic IntelligenceReinforcement LearningSequentialBenchmark
🎯 What it does: This paper proposes a benchmark framework for Inverse Constraint Reinforcement Learning (ICRL) and builds reproducible experimental environments for two real tasks: robot control and autonomous driving. It also introduces a variational method, VICRL, to learn the posterior distribution of constraints. The performance of different algorithms in constraint inference and execution is evaluated by comparing various ICRL baselines.
Benchmarking Offline Reinforcement Learning on Real-Robot Hardware
Nico Gürtler (Max Planck Institute for Intelligent Systems), Georg Martius (Max Planck Institute for Intelligent Systems)
Robotic IntelligenceReinforcement LearningBenchmark
🎯 What it does: A benchmark dataset for offline reinforcement learning on the TriFinger platform was constructed, and various offline RL algorithms were evaluated in both real robots and the PyBullet simulation environment.
Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models
Kaiyue Wen (Institute for Interdisciplinary Information Sciences Tsinghua University), Jingzhao Zhang (Institute for Interdisciplinary Information Sciences Tsinghua University)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: The study investigates the phenomenon of 'benign overfitting' in practical classification tasks, particularly in ResNet models on ImageNet and CIFAR10. It proposes that under mild over-parameterization (p ≈ n), label noise can lead to the failure of overfitting, and this conclusion is supported by theoretical proof and experimental validation.
Better Generative Replay for Continual Federated Learning
Daiqing Qi (University of Virginia), Sheng Li (University of Virginia)
Federated LearningConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: Proposes the FedCIL framework to address the problem of federated incremental learning for clients continuously learning new categories, and implements memoryless forgetting suppression through generative replay.
Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation
Martin Zong (SenseTime Research), Wanli Ouyang (Shanghai AI Lab)
ClassificationObject DetectionKnowledge DistillationTransformerImage
🎯 What it does: A Dynamic Prior Knowledge (DPK) framework is proposed, utilizing teacher features as input to mix with students and adjusting the ratio through dynamic masking for knowledge distillation.
Betty: An Automatic Differentiation Library for Multilevel Optimization
Sang Keun Choe (Carnegie Mellon University), Eric Xing (Carnegie Mellon University)
Domain AdaptationOptimizationTransformerImageText
🎯 What it does: The BETTY automatic differentiation library has been developed to specifically address the high complexity and implementation difficulties of gradient computation in multi-layer optimization (MLO), enabling scalable training for large-scale MLO.
BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection
Zehui Chen (University of Science and Technology of China), Feng Zhao (University of Science and Technology of China)
Object DetectionAutonomous DrivingKnowledge DistillationTransformerContrastive LearningMultimodalityPoint Cloud
🎯 What it does: Utilizing BEV space to achieve cross-modal knowledge distillation, with the LiDAR detector as the teacher and the multi-view camera detector as the student, feature alignment is realized through a unified BEV representation for distillation of the student.
Beyond calibration: estimating the grouping loss of modern neural networks
Alexandre Perez-Lebel (Inria), Gael Varoquaux
ClassificationImageText
🎯 What it does: A method for estimating the Grouping Loss of modern neural networks is proposed, constructing a theoretical decomposition of the Grouping Loss and an estimable lower bound.
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD
Konstantinos Nikolakakis, Dionysios Kalogerias
Optimization
🎯 What it does: This paper studies the generalization error and excess risk bounds of full-batch gradient descent (GD) on smooth (possibly non-Lipschitz, non-convex) losses.
Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradients
Zhongkai Hao (Tsinghua University), Ze Cheng (Bosch China Investment Ltd)
OptimizationTabularPhysics Related
🎯 What it does: A dual-layer optimization framework is proposed, using PINN to solve the lower-level PDE constraints, while the upper-level supergradient is solved using the Broyden method, thereby achieving PDE-constrained optimization.
Bias Propagation in Federated Learning
Hongyan Chang (National University of Singapore), Reza Shokri (National University of Singapore)
Federated LearningTabular
🎯 What it does: This paper reveals how federated learning (FL) systems propagate group fairness biases among different participants through experimental and theoretical analysis, and provides specific manifestations of these biases in model parameters.
Bidirectional Language Models Are Also Few-shot Learners
Ajay Patel (University of Pennsylvania), Chris Callison-Burch (University of Pennsylvania)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes SAP technology, utilizing the bidirectional model mT5 to achieve few-shot and zero-shot task inference without fine-tuning;
BigVGAN: A Universal Neural Vocoder with Large-Scale Training
Sang-gil Lee (Seoul National University), Sungroh Yoon (Seoul National University)
GenerationData SynthesisGenerative Adversarial NetworkAudio
🎯 What it does: This paper presents a large-scale trained universal neural vocoder, BigVGAN, which can achieve zero-shot high-fidelity audio synthesis in unseen speaker, language, recording environment, and even musical contexts.
Binding Language Models in Symbolic Languages
Zhoujun Cheng (University of Hong Kong), Tao Yu (University of Hong Kong)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringMultimodalityTabular
🎯 What it does: The BINDER framework is proposed, which maps natural language input to executable programs and inserts a unified language model API (such as Codex) to extend the functionality of existing programming languages (SQL/Python), achieving untrained neural-symbolic integration.
Bispectral Neural Networks
Sophia Sanborn (University of California), Christopher J. Hillar
ClassificationExplainability and InterpretabilityRepresentation LearningImage
🎯 What it does: Designed and implemented Bispectral Neural Networks (BNNs), a network that achieves complete invariant representation by learning the reversible Fourier transform and third-order invariance (bispectrum), capable of automatically learning the corresponding group structure, irreducible representations, and the Cayley table of the group from data.
Bit-Pruning: A Sparse Multiplication-Less Dot-Product
Yusuke Sekikawa (DENSO IT LABORATORY INC), Shingo Yashima (DENSO IT LABORATORY INC)
Computational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper proposes Bit-Pruning, a framework for achieving multiply-less sparse dot products through bit-wise pruning during the training phase;
Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts
Amrith Setlur (Carnegie Mellon University), Sergey Levine (University of California Berkeley)
Domain AdaptationOptimizationConvolutional Neural NetworkTransformerGenerative Adversarial NetworkImageText
🎯 What it does: A bit-rate constrained distributionally robust optimization method (BR-DRO) is proposed, which can achieve robustness against unknown group shifts and label noise without relying on the labels of the training sample group.
Block and Subword-Scaling Floating-Point (BSFP) : An Efficient Non-Uniform Quantization For Low Precision Inference
Yun-Chen Lo (National Tsing Hua University), Ren-Shuo Liu (National Tsing Hua University)
ClassificationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a non-uniform quantization method based on Block and Subword Floating Point (BSFP) for low-precision inference of neural network weight vectors.
Blurring Diffusion Models
Emiel Hoogeboom (Google Research), Tim Salimans (Google Research)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: A diffusion model that combines fuzziness (thermal diffusion) with Gaussian noise is proposed, called Blurring Diffusion Models, providing a probabilistic framework under anisotropic noise in the frequency domain.
Boosting Adversarial Transferability using Dynamic Cues
Muzammal Naseer (Mohamed bin Zayed University of AI), Fahad Khan
Domain AdaptationAdversarial AttackTransformerPrompt EngineeringImageVideo
🎯 What it does: This paper proposes a method for learning temporal prompts on frozen image Vision Transformers, enabling image models that originally only process spatial information to perceive the temporal dynamics of videos, thereby generating adversarial perturbations with higher transferability to black-box video models.
Boosting Causal Discovery via Adaptive Sample Reweighting
An Zhang (National University of Singapore), Tat-Seng Chua (National University of Singapore)
TabularBiomedical Data
🎯 What it does: A model-agnostic adaptive sample weighting framework called ReScore is proposed to enhance the performance of differentiable causal structure learning methods.
Boosting Multiagent Reinforcement Learning via Permutation Invariant and Permutation Equivariant Networks
Jianye HAO, Zhen Wang (Northwestern Polytechnical University)
Reinforcement Learning
🎯 What it does: Two network structures utilizing permutation invariance (PI) and permutation equivariance (PE) are proposed (Dynamic Permutation Network DPN and Hypernetwork HPN), which can significantly compress the state-action space and improve sample efficiency by minimally modifying the feedforward layer of the existing MARL algorithm.
Boosting the Cycle Counting Power of Graph Neural Networks with I$^2$-GNNs
Yinan Huang (Peking University), Muhan Zhang (Peking University)
Graph Neural NetworkGraph
🎯 What it does: This study enhances the capability of subgraph MPNN in counting cycles and paths at the node level, proposing I2-GNNs that significantly improve counting accuracy by assigning unique identifiers to the root node and its neighbors.
Bort: Towards Explainable Neural Networks with Bounded Orthogonal Constraint
Borui Zhang (Tsinghua University), Jiwen Lu (Tsinghua University)
OptimizationExplainability and InterpretabilityConvolutional Neural NetworkTransformerImage
🎯 What it does: A novel optimizer Bort is proposed, which improves the interpretability and reversibility of neural networks by applying bounded orthogonal constraints to the network weights.
Brain-like representational straightening of natural movies in robust feedforward neural networks
Tahereh Toosi (Columbia University), Elias Issa
Representation LearningAdversarial AttackConvolutional Neural NetworkVideo
🎯 What it does: This study investigates the changes in representation curvature of robust feedforward neural networks in natural movies and explores its correspondence with neural activity in the brain's V1 area.
BrainBERT: Self-supervised representation learning for intracranial recordings
Christopher Wang (Massachusetts Institute of Technology), Andrei Barbu (Massachusetts Institute of Technology)
Representation LearningTransformerTime SeriesBiomedical Data
🎯 What it does: Developed BrainBERT, a self-supervised model based on Transformer that converts SEEG brain recordings into context-aware embeddings, which can then be quickly and accurately used by a linear decoder to complete various neural decoding tasks;
Breaking Correlation Shift via Conditional Invariant Regularizer
Mingyang Yi (University of Chinese Academy of Sciences), Zhi-Ming Ma (Academy of Mathematics and Systems Science, Chinese Academy of Sciences)
Domain AdaptationOptimizationText
🎯 What it does: The paper proposes a method to alleviate the out-of-distribution (OOD) generalization problem caused by correlation shift through the conditional independence of a regularized model, defines and estimates the Conditional Spurious Variation (CSV), and provides corresponding training algorithms (RCSV/RCSV U).
Bridge the Inference Gaps of Neural Processes via Expectation Maximization
Qi Wang (University of Amsterdam), Herke van Hoof (University of Amsterdam)
Data SynthesisOptimizationMeta LearningImageTabular
🎯 What it does: This paper proposes the Self-normalized Importance weighted Neural Process (SI-NP), which corrects the suboptimal inference problem of traditional Neural Processes through variational EM iterations, improving the quality of function distribution learning.
Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes
Zecheng Hao (Peking University), Zhaofei Yu (Peking University)
Spiking Neural NetworkImage
🎯 What it does: The paper proposes a method to calibrate offset spikes by adjusting the initial membrane potential, achieving high-precision and low-latency conversion from ANN to SNN.
Bridging the Gap to Real-World Object-Centric Learning
Maximilian Seitzer (Max Planck Institute for Intelligent Systems), Francesco Locatello (Amazon Web Services)
Object DetectionSegmentationKnowledge DistillationTransformerContrastive LearningImage
🎯 What it does: The DINOSAUR model is proposed, which achieves unsupervised object-centric learning through the reconstruction of self-supervised features, enabling object segmentation on real images.
Broken Neural Scaling Laws
Ethan Caballero (Mila), David Krueger (University of Cambridge)
GenerationReinforcement LearningImageTextBenchmark
🎯 What it does: This paper proposes and validates a smooth piecewise power law function (Broken Neural Scaling Law, BNSL) to accurately model and extrapolate the performance curves of various neural network architectures across different tasks (vision, language, reinforcement learning, etc.) as their scale changes.
BSTT: A Bayesian Spatial-Temporal Transformer for Sleep Staging
Yuchen Liu (Chinese Academy of Sciences), Ziyu Jia (Chinese Academy of Sciences)
ClassificationExplainability and InterpretabilityTransformerTime SeriesBiomedical Data
🎯 What it does: A model combining Bayesian relational reasoning and Transformer, called BSTT, is proposed for automatic sleep staging, along with an interpretable spatial-temporal relationship graph.
Budgeted Training for Vision Transformer
zhuofan xia, Gao Huang (Alibaba Group)
ClassificationOptimizationTransformerImage
🎯 What it does: This paper proposes a framework for dynamically activating the visual Transformer structure to adapt to any training budget;
Building a Subspace of Policies for Scalable Continual Learning
Jean-Baptiste Gaya (CNRS ISIR Sorbonne University), Roberta Raileanu (Meta AI Research)
Reinforcement Learning
🎯 What it does: An adaptive expanded strategy subspace method CSP is proposed for continuous reinforcement learning;
Building Normalizing Flows with Stochastic Interpolants
Michael Samuel Albergo, Eric Vanden-Eijnden (New York University)
GenerationOptimizationScore-based ModelFlow-based ModelImageTabularOrdinary Differential Equation
🎯 What it does: A continuous time normalization flow method based on random interpolators is proposed, which can directly obtain the velocity field through a simple quadratic loss training, thus achieving probability flow mapping between any two distributions.
Calibrating Sequence likelihood Improves Conditional Language Generation
Yao Zhao (Google Research), Peter J Liu
GenerationTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a three-stage training process—pre-training, fine-tuning, and sequence likelihood calibration (SLiC)—to improve the quality of conditional language generation by calibrating the candidate sequences generated by the model after fine-tuning.
Calibrating the Rigged Lottery: Making All Tickets Reliable
Bowen Lei (Texas A&M University), Bani Mallick (Texas A&M University)
OptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This study investigates the reliability issues of sparse training models and proposes the CigL method to generate more reliable sparse networks with comparable performance.
Calibrating Transformers via Sparse Gaussian Processes
Wenlong Chen (Imperial College London), Yingzhen Li (Imperial College London)
TransformerImageTextGraph
🎯 What it does: By replacing the softmax in the multi-head attention of the Transformer with a kernel function and utilizing Sparse Variational Gaussian Processes (SVGP) for Bayesian inference in the attention output space, we achieve uncertainty quantification and calibration for the Transformer.
Calibration Matters: Tackling Maximization Bias in Large-scale Advertising Recommendation Systems
Yewen Fan (Carnegie Mellon University), Kun Zhang (Mohamed bin Zayed University of Artificial Intelligence)
Recommendation SystemTabular
🎯 What it does: This paper addresses the issue of maximization bias in advertising recommendation systems and proposes a variance-adjusted debiasing calibration method (VAD) that implements linear adjustments to prediction results to enhance calibration effectiveness.
Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectories
Li-Cheng Lan (University of California), Cho-Jui Hsieh (University of California)
Reinforcement Learning
🎯 What it does: This paper proposes a 'Relay-Evaluation' assessment framework to test the generalization ability of reinforcement learning (RL) agents in a fixed environment regarding controllable states (i.e., states with high-reward strategies) and points out that many mainstream algorithms exhibit a high failure rate in this evaluation; it then introduces the Self-Trajectory Augmentation (STA) method, which explores starting from high-value states in its historical trajectory during training, significantly improving the success rate under Relay-Evaluation without reducing conventional testing performance.
Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study
Mingxu Tao (Peking University), Dongyan Zhao (Peking University)
ClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This study investigates the encoding capability of BERT in task-incremental learning through probing and analyzes the topological structure of the representation subspace, demonstrating that BERT can retain knowledge of old tasks over the long term even without experience replay.
Can CNNs Be More Robust Than Transformers?
Zeyu Wang (University of California Santa Cruz), Cihang Xie (University of California Santa Cruz)
ClassificationAnomaly DetectionKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes three simple design improvements for CNNs through the disassembly and experimentation of the Vision Transformer structure: patchifying input, increasing convolution kernel size, and reducing normalization and activation layers, thereby significantly enhancing the robustness of CNNs against out-of-distribution data.
Can discrete information extraction prompts generalize across language models?
Nathanaël Carraz Rakotonirina (Universitat Pompeu Fabra), Marco Baroni (Universitat Pompeu Fabra)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper studies the transfer performance of using AutoPrompt to automatically generate discrete prompts across different pre-trained language models, and proposes the simultaneous use of multiple models (generator + evaluator) during prompt induction to enhance the cross-model generality of prompts.
Can Neural Networks Learn Implicit Logic from Physical Reasoning?
Aaron Traylor (Brown University), Ellie Pavlick (Brown University)
Object DetectionRecurrent Neural NetworkVideoPhysics Related
🎯 What it does: Evaluate whether neural networks can learn implicit logic (negation and disjunction) from physical reasoning training.
Can We Faithfully Represent Absence States to Compute Shapley Values on a DNN?
Jie Ren (Shanghai Jiao Tong University), Quanshi Zhang (Shanghai Jiao Tong University)
OptimizationExplainability and InterpretabilityConvolutional Neural NetworkImageTabular
🎯 What it does: This paper proposes a method to evaluate and learn optimal baseline values through interpretable causal patterns to enhance the credibility of Shapley value explanations for deep neural networks.
Can We Find Nash Equilibria at a Linear Rate in Markov Games?
Zhuoqing Song (Fudan University), Zhuoran Yang (Yale University)
OptimizationReinforcement Learning
🎯 What it does: This paper proposes a decentralized, symmetric, and rational strategy optimization meta-algorithm called Homotopy-PO, which can converge to Nash equilibrium at a global linear rate in two-player zero-sum discounted Markov games. The algorithm switches homotopically between a globally slow-converging sub-algorithm (Averaging OGDA) and a locally fast-converging sub-algorithm (OGDA), using the global slow algorithm as a guide, while the local fast algorithm achieves linear convergence after entering a 'favorable neighborhood'.
Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries
Yuxin Wen (University of Maryland), Tom Goldstein (University of Maryland)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: By optimizing adversarial queries (canary) for the Shadow model, the effectiveness of black-box membership inference attacks is enhanced.
CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning
Samuel Maddock (University of Warwick), Pierre Stock (Meta AI)
Federated LearningSafty and PrivacyAdversarial AttackImageText
🎯 What it does: This paper proposes an empirical attack method called CANIFE, which evaluates model privacy leakage in federated learning through the careful design of 'suspicious samples' (canary), capable of measuring privacy leakage in each round under real threat models.