ICLR 2023 Papers — Page 5
International Conference on Learning Representations · 1573 papers
Distilling Model Failures as Directions in Latent Space
Saachi Jain (Massachusetts Institute of Technology), Aleksander Madry (Massachusetts Institute of Technology)
ClassificationGenerationData SynthesisExplainability and InterpretabilityVision Language ModelDiffusion modelImageMagnetic Resonance Imaging
🎯 What it does: Automatically extract error patterns from the trained model, transform them into directions in the shared CLIP space, and thus achieve detection, explanation (automatically generate explanatory titles), and intervention (incremental training with targeted synthetic data generated by diffusion models) for difficult subgroups.
Distributed Differential Privacy in Multi-Armed Bandits
Sayak Ray Chowdhury (Microsoft Research), Xingyu Zhou (Wayne State University)
Safty and PrivacyReinforcement LearningTabular
🎯 What it does: Under the distributed differential privacy model, a multi-armed bandit algorithm based on successful elimination has been designed, achieving the same asymptotic optimal return rate as pure DP and the central model.
Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
Ali Ramezani-Kebrya (University of Oslo), Volkan Cevher (EPFL)
GenerationOptimizationGenerative Adversarial NetworkImage
🎯 What it does: A universal unbiased quantization extra gradient algorithm (Q-GenX) for multi-GPU distributed environments is proposed, which can uniformly handle different VI solvers and significantly reduce communication overhead.
Distributional Meta-Gradient Reinforcement Learning
Haiyan Yin (Sea AI Lab), Zhongwen Xu (Sea AI Lab)
Meta LearningReinforcement LearningTabularBenchmark
🎯 What it does: A distributed meta-gradient reinforcement learning algorithm (DrMG) is proposed, which achieves adaptive modeling of return distributions by using quantized distributed returns within the Actor-Critic framework and learning distribution parameters at the meta-optimization layer.
Distributionally Robust Post-hoc Classifiers under Prior Shifts
Jiaheng Wei (University of California Santa Cruz), Abhishek Kumar (Google Research)
ClassificationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A lightweight post-processing method called DROPS is proposed, which uses the predicted probabilities of a trained model for learnable scaling to enhance robustness against class/group prior variations.
Distributionally Robust Recourse Action
Duy Nguyen (VinAI Research), Viet Anh Nguyen (Chinese University of Hong Kong)
OptimizationExplainability and InterpretabilityTabularFinance Related
🎯 What it does: A Distributed Robust Backtracking Action (DiRRAc) framework is proposed to generate interpretable return actions with a high probability of success in the context of model parameters changing over time.
Diversify and Disambiguate: Out-of-Distribution Robustness via Disagreement
Yoonho Lee (Stanford University), Chelsea Finn (Stanford University)
Domain AdaptationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A two-stage DivDis framework is proposed, where a multi-head network is first trained under the source distribution to generate maximum inconsistency among heads on the target unlabeled data, and then a small number of labeled samples are used to disambiguate the heads, ultimately selecting the optimal head for predictions on the target distribution.
Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors
Jianfei Yang (Nanyang Technological University), Yang You (National University of Singapore)
Domain AdaptationKnowledge DistillationImageBenchmark
🎯 What it does: A method called BETA is proposed to suppress confirmation bias in black-box predictor domain adaptation by dividing the target domain into easy-to-adapt and hard-to-adapt subdomains and employing a mutual distillation dual network;
DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Images
Bing WANG, Bo Yang (Hong Kong Polytechnic University)
Object DetectionSegmentationGenerationNeural Radiance FieldImage
🎯 What it does: This paper proposes DM-NeRF, which utilizes the implicit representation of NeRF to complete 3D scene reconstruction, object decomposition, and editable rendering solely from 2D images.
Do We Really Need Complicated Model Architectures For Temporal Networks?
Weilin Cong (Penn State), Mehrdad Mahdavi (Penn State)
Recurrent Neural NetworkGraph Neural NetworkGraphTime Series
🎯 What it does: This paper proposes GraphMixer, which uses MLP and mean pooling to construct a minimal architecture for temporal graph link prediction tasks.
DocPrompting: Generating Code by Retrieving the Docs
Shuyan Zhou (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
GenerationRetrievalAI Code AssistantTransformerPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: The DocPrompting method is proposed, which involves first retrieving relevant documents in the process of generating code from natural language, and then inputting the retrieved documents along with the intent into the generation model, allowing the model to use unseen libraries or functions during testing.
Does Deep Learning Learn to Abstract? A Systematic Probing Framework
Shengnan An (Xi'an Jiaotong University), Jian-Guang Lou (Microsoft Corporation)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A systematic detection framework from the perspective of transferability is proposed to examine whether deep learning models possess abstract capabilities. The performance of T5 and GPT2 in learning abstract concepts from concrete instances and transferring them to new tasks is validated through syntactic translation probes.
Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?
Lirui Wang (Massachusetts Institute of Technology), Russ Tedrake (Massachusetts Institute of Technology)
Federated LearningRepresentation LearningContrastive LearningImage
🎯 What it does: This paper studies how to efficiently learn general representations through self-supervised learning (SSL) in a decentralized and non-IID unlabeled data environment, and explores its feasibility and advantages within a federated learning framework.
Does Zero-Shot Reinforcement Learning Exist?
Ahmed Touati (Meta AI Research), Yann Ollivier (Meta AI Research)
Reinforcement LearningContrastive LearningBenchmark
🎯 What it does: This paper proposes and systematically evaluates a zero-shot reinforcement learning method, primarily based on Success Features (SF) and Forward-Backward (FB) representations, aimed at training an agent that can immediately execute any reward task without further planning.
Domain Generalisation via Domain Adaptation: An Adversarial Fourier Amplitude Approach
Minyoung Kim (Samsung AI Center), Timothy Hospedales
Domain AdaptationConvolutional Neural NetworkGenerative Adversarial NetworkImageBenchmark
🎯 What it does: By treating domain generalization as a domain adaptation task, we use adversarially generated Fourier magnitude maps to synthesize the worst target domain, and train the model on this domain using Maximum Classifier Discrepancy (MCD) to enhance the model's robustness to unknown target domains.
Domain Generalization via Heckman-type Selection Models
Hyungu Kahng (Korea University), Judy Zhong (New York University School of Medicine)
Domain AdaptationTabularBenchmark
🎯 What it does: Modeling the domain generalization problem as a non-random sample selection problem, jointly learning the prediction model and the domain selection model, aiming to achieve more robust predictions against the true population distribution.
Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptation
Zihao Xu (Rutgers University), Hao Wang (Hong Kong University of Science and Technology)
Domain AdaptationGenerative Adversarial NetworkTabular
🎯 What it does: A variational inference-based domain index inference framework (VDI) is proposed, which can learn interpretable continuous domain indices from multi-domain data under the condition of only given domain identities;
Don’t fear the unlabelled: safe semi-supervised learning via debiasing
Hugo Schmutz (Universite Cote d'Azur), Pierre-Alexandre Mattei (Universite Cote d'Azur)
ClassificationData-Centric LearningImage
🎯 What it does: A debiased semi-supervised learning framework, DeSSL, is proposed, which incorporates control variables into the traditional SSL risk estimation, ensuring that the risk estimation is unbiased under the MCAR assumption and has theoretical guarantees.
Don’t forget the nullspace! Nullspace occupancy as a mechanism for out of distribution failure
Daksh Idnani (Meta), Shanmukha Ramakrishna Vedantam
Domain AdaptationImage
🎯 What it does: This paper proposes a new out-of-distribution (OoD) generalization failure mechanism called 'nullspace occupancy' and eliminates this failure by learning the optimal projection basis in the training feature space.
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Shuaichen Chang (Ohio State University), Bing Xiang (Amazon Web Services AI Labs)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: To address the robustness of text-to-SQL models, the authors constructed a diagnostic evaluation benchmark named Dr.Spider, which designed 17 task-specific perturbations at three levels: database, natural language questions, and SQL statements, covering both semantic preservation and semantic alteration types of perturbations.
Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
Albert Qiaochu Jiang, Yuhuai Wu (Google Research)
Large Language ModelText
🎯 What it does: This paper proposes the Draft, Sketch, and Prove (DSP) method, which transforms informal mathematical proofs into formal proof sketches and utilizes automated provers to complete the missing reasoning steps.
DreamFusion: Text-to-3D using 2D Diffusion
Ben Poole (Google Research), Ben Mildenhall (Google Research)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelNeural Radiance FieldImageText
🎯 What it does: A text-to-3D (NeRF) generation method based on a pre-trained 2D diffusion model is proposed—DreamFusion;
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training
Joya Chen (National University of Singapore), Angela Yao (National University of Singapore)
ClassificationObject DetectionSegmentationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: DropIT reduces GPU memory usage and approximates gradients by dropping intermediate tensors (activations) based on absolute minimum values or randomly during training, and then filling in the missing elements with zeros during backpropagation.
Dual Algorithmic Reasoning
Danilo Numeroso (University of Pisa), Petar Veličković (DeepMind)
Graph Neural NetworkGraph
🎯 What it does: A Dual Algorithmic Reasoning (DAR) framework is proposed, which enhances the performance of neural algorithmic reasoners by simultaneously learning the maximum flow and its dual minimum cut.
Dual Diffusion Implicit Bridges for Image-to-Image Translation
Xuan Su (Stanford University), Stefano Ermon (Stanford University)
Image TranslationFederated LearningSafty and PrivacyDiffusion modelScore-based ModelImageOrdinary Differential Equation
🎯 What it does: Proposes Dual Diffusion Implicit Bridges (DDIBs), which utilize two independently trained diffusion models to encode source domain images into latent space, and then decode using the target domain model to obtain target domain images, completing unpaired image translation.
Dual Student Networks for Data-Free Model Stealing
James Beetham (University of Central Florida), Mubarak Shah (University of Western Australia)
ClassificationData SynthesisKnowledge DistillationAdversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: The Dual Student method is proposed, which jointly trains two student models to achieve data-free model stealing in a black-box environment without data or model parameters.
DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulation
Yan Zhao (Peking University), Hao Dong (Peking University)
Robotic IntelligenceAuto EncoderPoint CloudMesh
🎯 What it does: This paper presents the DualAfford framework, which learns visually executable properties for two-handed collaboration to perform tasks such as pushing, rotating, inverting, and grasping various 3D objects.
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
Pan Lu (University of California), Ashwin Kalyan (Allen Institute for AI)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTabularChain-of-Thought
🎯 What it does: A new dataset called TABMWP is proposed, and a dynamic prompt learning method based on policy gradient, named PROMPTPG, is developed on this dataset.
Dynamic Update-to-Data Ratio: Minimizing World Model Overfitting
Nicolai Dorka (University of Freiburg), Wolfram Burgard (University of Technology Nuremberg)
Reinforcement LearningWorld ModelImage
🎯 What it does: A dynamic update-to-data ratio (DUTD) method is proposed and implemented for world model training in model-based reinforcement learning. It automatically detects underfitting and overfitting by monitoring validation set errors and adjusts the ratio of training steps to data steps accordingly.
DynaMS: Dyanmic Margin Selection for Efficient Deep Learning
Jiaxing Wang (JD.com), Weipeng Yan (JD.com)
ClassificationComputational EfficiencyImage
🎯 What it does: A dynamic data subset selection method called DynaMS is proposed, which dynamically selects training samples based on the classification margin metric, thereby reducing computational load and improving model performance.
DySR: Adaptive Super-Resolution via Algorithm and System Co-design
Syed Zawad (University of Nevada), Feng Yan (University of Houston)
Super ResolutionOptimizationNeural Architecture SearchImageVideo
🎯 What it does: This paper proposes an adaptive super-resolution framework DySR based on algorithm and system co-design, which can dynamically switch subgraphs on mobile devices according to real-time computing and memory resources, maximizing super-resolution quality while maintaining frame rates.
E-CRF: Embedded Conditional Random Field for Boundary-caused Class Weights Confusion in Semantic Segmentation
Jie Zhu (Peking University), Leye Wang (Peking University)
SegmentationConvolutional Neural NetworkImage
🎯 What it does: To address the class weight confusion (BCWC) problem in semantic segmentation, an Embedded Conditional Random Field (E-CRF) model is proposed, which integrates the CRF mechanism with CNN networks to enhance class weight discrimination and improve boundary pixel prediction.
E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking
Yangtian Zhang (Mila Quebec AI Institute), Jian Tang (HEC Montreal)
Drug DiscoveryProtein Structure PredictionGraph Neural NetworkTransformerBiomedical Data
🎯 What it does: An end-to-end E(3) equivariant network E3Bind has been developed for protein-ligand blind self-docking, capable of directly predicting the coordinates, orientation, and conformation of the ligand without the need for two-stage distance-coordinate transformation or post-processing.
EA-HAS-Bench: Energy-aware Hyperparameter and Architecture Search Benchmark
Shuguang Dou (Tongji University), Dongsheng Li (Microsoft Research Asia)
Computational EfficiencyHyperparameter SearchConvolutional Neural NetworkImageBenchmark
🎯 What it does: A joint architecture and hyperparameter search benchmark for energy consumption, EA‑HAS‑Bench, is proposed, providing complete energy consumption and performance data.
EAGLE: Large-scale Learning of Turbulent Fluid Dynamics with Mesh Transformers
Steeven JANNY, Christian Wolf (Naver Labs Europe)
Graph Neural NetworkTransformerMeshGraph
🎯 What it does: A large-scale two-dimensional unsteady fluid dynamics dataset called EAGLE is proposed, along with a long-range attention model based on grid transformers, aimed at predicting airflow pressure and velocity for drones over various terrains.
Easy Differentially Private Linear Regression
Kareem Amin (Google), Sergei Vassilvitskii (Google)
Safty and PrivacyComputational EfficiencySupervised Fine-TuningTabular
🎯 What it does: A differentially private linear regression algorithm called TukeyEM is proposed, which avoids the dependence on data range and hyperparameters found in traditional methods by using Tukey depth and the exponential mechanism.
Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis
Hao Tang (ETH Zurich), Luc Van Gool (ETH Zurich)
GenerationData SynthesisGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: This paper proposes ECGAN, a generative adversarial network that combines edge guidance and contrastive learning for semantic image synthesis.
Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks
Bowen Jin (University of Illinois), Jiawei Han (University of Illinois)
Representation LearningGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: For text-edge networks, the Edgeformers framework is proposed, which injects virtual node tokens into the Transformer layers to deeply couple graph structures with text information, achieving edge and node representation learning.
Editing models with task arithmetic
Gabriel Ilharco (University of Washington), Ali Farhadi (University of Washington)
ClassificationOptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: Proposes the Task Vector and edits the behavior of pre-trained models through vector operations (addition, subtraction, analogy).
Effective passive membership inference attacks in federated learning against overparameterized models
Jiacheng Li (Purdue University), Bruno Ribeiro (Purdue University)
Federated LearningSafty and PrivacyAdversarial AttackBiomedical Data
🎯 What it does: This paper proposes a passive member inference attack on over-parameterized models in federated learning, utilizing gradient orthogonality for inference.
Effective Self-supervised Pre-training on Low-compute Networks without Distillation
Fuwen Tan (Samsung Artificial Intelligence Cambridge), Brais Martinez (Samsung Artificial Intelligence Cambridge)
Object DetectionSegmentationKnowledge DistillationRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This study investigates the feasibility of self-supervised learning on low-computation resource networks, proposing a view sampling and loss re-weighting strategy tailored to model capacity, and validating its effectiveness across various models and downstream tasks.
Effectively Modeling Time Series with Simple Discrete State Spaces
Michael Zhang (Stanford University), Christopher Re (Stanford University)
ClassificationOptimizationComputational EfficiencyRecurrent Neural NetworkTransformerTime SeriesSequentialElectrocardiogramBenchmarkAudio
🎯 What it does: SPACETIME is proposed, a state space time series model based on the adjoint matrix, designed for efficient and expressive long-range dependency time series forecasting and classification.
Effects of Graph Convolutions in Multi-layer Networks
Aseem Baranwal (University of Waterloo), Aukosh Jagannath (University of Waterloo)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This study investigates the impact of graph convolution on node classification in multilayer networks, providing theoretical thresholds and experimental validation.
Efficient approximation of neural population structure and correlations with probabilistic circuits
Koosha Khalvati (Allen Institute), Michael A Buice
Time Series
🎯 What it does: An efficient framework based on Sum-Product Networks (SPN) has been developed to capture high-order correlations of large-scale neuronal populations and approximate their joint distribution.
Efficient Attention via Control Variates
Lin Zheng (University of Hong Kong), Lingpeng Kong (University of Hong Kong)
ClassificationOptimizationComputational EfficiencyTransformerImageText
🎯 What it does: This paper proposes an efficient attention mechanism based on control variables, called EVA, to improve Random Feature Attention (RFA) and reduce its approximation error compared to the original softmax attention.
Efficient Certified Training and Robustness Verification of Neural ODEs
Mustafa Zeqiri (ETH Zurich), Martin Vechev (ETH Zurich)
ClassificationOptimizationImageTime SeriesOrdinary Differential Equation
🎯 What it does: This paper proposes a framework called GAINS for efficient certified training and robustness verification of high-dimensional Neural Ordinary Differential Equations (NODEs), addressing the issue of continuous step size abstraction caused by traditional adaptive ODE solvers.
Efficient Conditionally Invariant Representation Learning
Roman Pogodin (University College London), Arthur Gretton (University College London)
Domain AdaptationRepresentation LearningImage
🎯 What it does: A kernel-based conditional independence measure called CIRCE is proposed, which is used as a regularization term to learn a representation φ(X) that is conditionally invariant to the given metadata Z, thereby addressing scenarios such as fairness, domain-invariant learning, and causal representation learning.
Efficient Deep Reinforcement Learning Requires Regulating Overfitting
Qiyang Li (University of California Berkeley), Sergey Levine (University of California Berkeley)
Reinforcement Learning
🎯 What it does: Through systematic experimental analysis, it was found that high validation TD error is the main bottleneck limiting the performance of efficient deep reinforcement learning, and the AVTD online model selection method is proposed to automatically select the most suitable regularization strategy based on validation TD error;
Efficient Discrete Multi Marginal Optimal Transport Regularization
Ronak Mehta (University of Wisconsin Madison), Vikas Singh (University of Wisconsin Madison)
Image TranslationOptimizationImageTabular
🎯 What it does: This paper proposes an efficient Discrete Multi-Marginal Optimal Transport (MMOT) regularization method called DEMD, which is integrated into a deep learning framework for tasks such as fairness, discernible representation, and multi-domain image translation.
Efficient Edge Inference by Selective Query
Anil Kag (Boston University), Venkatesh Saligrama (Boston University)
ClassificationComputational EfficiencyNeural Architecture SearchImageText
🎯 What it does: This paper proposes an end-to-end hybrid learning framework that queries the routing model on edge devices only for hard examples that are correctly classified in the cloud, significantly reducing overall inference latency while maintaining high accuracy.
Efficient Federated Domain Translation
Zeyu Zhou (Purdue University), David I. Inouye (Purdue University)
Image TranslationDomain AdaptationFederated LearningAuto EncoderImage
🎯 What it does: A domain translation method for federated learning called FedINB is proposed, which can generate pseudo data when clients only have data from a single domain, thereby addressing the differences in non-IID conditional distributions and improving domain generalization performance.
Efficient Model Updates for Approximate Unlearning of Graph-Structured Data
Eli Chien (University of Illinois), Olgica Milenkovic (University of Illinois)
OptimizationSafty and PrivacyComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: An approximate graph model degradation method based on SGC and GPR is proposed, with theoretical guarantees provided.
Efficient Offline Policy Optimization with a Learned Model
Zichen Liu (Sea AI Lab), Zhongwen Xu (Sea AI Lab)
OptimizationReinforcement LearningTabularSequential
🎯 What it does: A lightweight single-step model-based offline reinforcement learning algorithm, ROSMO, is proposed to replace the high-cost MCTS of MuZero Unplugged for policy improvement.
Efficient Planning in a Compact Latent Action Space
zhengyao jiang, Yuandong Tian (Meta AI)
OptimizationRobotic IntelligenceTransformerReinforcement LearningAuto EncoderTabular
🎯 What it does: This paper proposes the Trajectory Autoencoding Planner (TAP), an offline reinforcement learning method that learns a low-dimensional discrete latent action space and plans within this space.
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time
Anand Subramoney (Institute for Neural Computation, Ruhr University Bochum), David Kappel (Institute for Neural Computation, Ruhr University Bochum)
Recurrent Neural NetworkSequentialOrdinary Differential Equation
🎯 What it does: A GRU-based event-driven RNN, called EGRU, is proposed, which achieves activity sparsity through event communication triggered by thresholds, allowing the forward and backward propagation computations to scale linearly with the number of events.
Efficiently Computing Nash Equilibria in Adversarial Team Markov Games
Fivos Kalogiannis (University of California Irvine), Stelios Andrew Stavroulakis (University of California Irvine)
OptimizationReinforcement Learning
🎯 What it does: This paper proposes a new algorithm that can efficiently compute approximate Nash equilibria in infinite-horizon adversarial team Markov games.
Efficiently Controlling Multiple Risks with Pareto Testing
Bracha Laufer-Goldshtein (Massachusetts Institute of Technology), Tommi S. Jaakkola
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a two-stage method called Pareto Testing, which simultaneously optimizes other objectives under multiple risk constraints to achieve reliable acceleration and performance control of models.
Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement
Chongyi Li (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)
RestorationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a super high-resolution low-light image enhancement network UHDFour that embeds Fourier transform in the frequency domain and contributes the first real UHD low-light enhancement dataset UHD-LL.
Emergence of Maps in the Memories of Blind Navigation Agents
Erik Wijmans (Georgia Institute of Technology), Dhruv Batra (Georgia Institute of Technology)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningSequential
🎯 What it does: Using a minimal perception system that only provides self-motion information (GPS + Compass) without visual, collision, or other sensors, an LSTM agent is trained using reinforcement learning to complete the PointGoal navigation task, and the emergence and utilization of its implicit map are studied.
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
Kenneth Li (Harvard University), Martin Wattenberg (Harvard University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextSequential
🎯 What it does: This study investigates a variant of GPT that predicts legal moves in Othello (Reversi) without prior rules. It finds that nonlinear board state representations emerge in its internal activations and verifies that this representation has a causal impact on model predictions through activation interventions. Consequently, it proposes an intervention-based latent saliency map to explain model decisions.
Empowering Graph Representation Learning with Test-Time Graph Transformation
Wei Jin (Michigan State University), Neil Shah (Snap Inc.)
OptimizationRepresentation LearningAdversarial AttackGraph Neural NetworkContrastive LearningGraph
🎯 What it does: Proposes the GTRANS framework, which utilizes gradient descent to perturb graph structures and node features during testing, enhancing the generalization and robustness of pre-trained GNNs under suboptimal data.
Empowering Networks With Scale and Rotation Equivariance Using A Similarity Convolution
Zikai Sun (Chinese University of Hong Kong), Thierry Blu (Chinese University of Hong Kong)
ClassificationRecognitionComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A new convolution operation called SimConv is proposed, which endows CNNs with continuous equivariance to translation, rotation, and scale, enhancing robustness to deformed inputs.
Encoding Recurrence into Transformers
Feiqing Huang (University of Hong Kong), Guodong Li (Huawei Noah's Ark Lab)
Recurrent Neural NetworkTransformerTextTime SeriesSequential
🎯 What it does: This study investigates how to encode the recursive characteristics of RNNs into Transformers, proposing the Self-Attention with Recurrence (RSA) module.
Energy-based Out-of-Distribution Detection for Graph Neural Networks
Qitian Wu (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Anomaly DetectionGraph Neural NetworkSupervised Fine-TuningGraph
🎯 What it does: A method for out-of-distribution (OOD) detection based on energy functions in graph neural networks (GNN), named GNNSAFE, is proposed, which can obtain an OOD discriminator using only a standard supervised trained GNN classifier.
Energy-Based Test Sample Adaptation for Domain Generalization
Zehao Xiao (University of Amsterdam), Cees G. M. Snoek (University of Amsterdam)
Domain AdaptationImageTextStochastic Differential Equation
🎯 What it does: This paper proposes an energy model-based adaptive method for test samples, aimed at domain generalization.
Energy-Inspired Self-Supervised Pretraining for Vision Models
Ze Wang (Purdue University), Qiang Qiu (Purdue University)
RestorationRepresentation LearningContrastive LearningImage
🎯 What it does: A self-supervised visual pre-training framework based on an energy model is proposed, utilizing the forward propagation of the network to learn the energy function, and implementing conditional sampling/image recovery through backward propagation, merging the encoder-decoder structure into a single network;
Enhancing Meta Learning via Multi-Objective Soft Improvement Functions
Runsheng Yu (Hong Kong University of Science and Technology), James Kwok
Meta LearningReinforcement LearningImage
🎯 What it does: Modeling meta-learning as a multi-objective optimization problem, treating each task as an objective, and proposing a scalable gradient descent method (SIMOL) to solve this problem.
Enhancing the Inductive Biases of Graph Neural ODE for Modeling Physical Systems
Suresh Bishnoi (Indian Institute of Technology Delhi), N M Anoop Krishnan
Graph Neural NetworkGraphTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: A neural ODE model called GNODE based on graph neural networks is proposed, which incorporates various physical prior biases (explicit constraints, separation of internal and external forces, Newton's third law) to learn the dynamics of particle systems.
Ensuring DNN Solution Feasibility for Optimization Problems with Linear Constraints
Tianyu Zhao (Lenovo Machine Intelligence Center), Steven Low
OptimizationTabular
🎯 What it does: This paper proposes a 'preventive learning' framework that utilizes constraint calibration to ensure that the solutions of deep neural networks (DNNs) always satisfy constraints when solving linear constrained optimization problems, without the need for post-processing.
EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Data
Michael Crawshaw (George Mason University), Mingrui Liu (Shanghai Jiao Tong University)
OptimizationFederated LearningImageText
🎯 What it does: EPISODE is proposed, a gradient clipping algorithm for heterogeneous data in federated learning that is non-convex and satisfies relaxed smoothness ((L,L0,1)-smoothness);
Equal Improvability: A New Fairness Notion Considering the Long-term Impact
Ozgur Guldogan (University of California), Kangwook Lee (University of Wisconsin)
TabularFinance Related
🎯 What it does: The concept of Equal Improvability (EI) fairness is proposed, along with three methods for implementing this concept through fair regularization; its effectiveness is validated in both static and dynamic scenarios.
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs
Yi-Lun Liao (Massachusetts Institute of Technology), Tess Smidt (Massachusetts Institute of Technology)
Graph Neural NetworkTransformerGraphBenchmarkPhysics Related
🎯 What it does: This paper proposes Equiformer, a transformation-based equivariant graph neural network for predicting quantum properties of 3D atomic graphs.
EquiMod: An Equivariance Module to Improve Visual Instance Discrimination
Alexandre DEVILLERS, Mathieu Lefort (University Lyon)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: A general EquiMod module is proposed, which combines existing visual self-supervised instance discrimination models to learn the equivariance of data augmentation.
Equivariance-aware Architectural Optimization of Neural Networks
Kaitlin Maile (University of Toulouse), Patrick Forré (University of Amsterdam)
ClassificationOptimizationNeural Architecture SearchConvolutional Neural NetworkImage
🎯 What it does: This paper implements dynamic adjustment of the network's equivariance constraints during training through two new network structures—equivariance relaxation morphism and [G]-mixed equivariant layer—and proposes two neural architecture search (NAS) methods, EquiNAS E (evolutionary) and EquiNAS D (differentiable), to find the optimal equivariance configuration and network weights.
Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning
Hyunwoo Ryu (Yonsei University), Jongeun Choi (Yonsei University)
Robotic IntelligenceTransformerReinforcement LearningImage
🎯 What it does: Equivariant Descriptor Fields (EDFs) are proposed, an end-to-end energy model that utilizes SE(3) symmetry for visual robot manipulation learning with only 5 to 10 demonstration examples;
Equivariant Energy-Guided SDE for Inverse Molecular Design
Fan Bao (Tsinghua University), Jun Zhu (Tsinghua University)
GenerationDrug DiscoveryGraph Neural NetworkDiffusion modelGraphStochastic Differential Equation
🎯 What it does: The EEGSDE framework is proposed, achieving controllable 3D molecular generation through energy-guided equivariant SDE.
Equivariant Hypergraph Diffusion Neural Operators
Peihao Wang (University of Texas at Austin), Pan Li (Georgia Tech)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes ED-HNN, a equivariant hypergraph neural network based on transferable hypergraph diffusion, designed to efficiently capture high-order relationships in hypergraphs and perform node classification tasks.
Equivariant Shape-Conditioned Generation of 3D Molecules for Ligand-Based Drug Design
Keir Adams (Massachusetts Institute of Technology), Connor W. Coley (Massachusetts Institute of Technology)
GenerationDrug DiscoveryGraph Neural NetworkAuto EncoderGraph
🎯 What it does: A shape-based 3D molecular generation framework called SQUID is proposed, which can automatically generate drug-like molecules with diverse chemical structures under given 3D shape constraints.
ERL-Re$^2$: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representation
Jianye HAO, Zhaopeng Meng (Tianjin University)
OptimizationReinforcement LearningSequentialBenchmark
🎯 What it does: Proposes ERL-Re 2, which integrates evolutionary algorithms with reinforcement learning, utilizing shared nonlinear state representations and individual linear policy representations for efficient policy optimization.
Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning
Fahad Sarfraz (NavInfo Europe), Bahram Zonooz (NavInfo Europe)
ClassificationRepresentation LearningConvolutional Neural NetworkImageSequential
🎯 What it does: This paper proposes an experience replay framework based on error sensitivity modulation, ESMER, which utilizes dual-mode memory (short-term experience buffer and long-term semantic model) and error history memory to alleviate representation drift and catastrophic forgetting in continual learning.
ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret
Stephen Marcus McAleer, Tuomas Sandholm (Carnegie Mellon University)
Reinforcement Learning
🎯 What it does: The ESCHER algorithm is proposed, which estimates adversarial avoidance using a fixed sampling strategy and historical value networks without the use of importance sampling, achieving effective learning in large incomplete information games.
ESD: Expected Squared Difference as a Tuning-Free Trainable Calibration Measure
Hee Suk Yoon (Korea Advanced Institute of Science and Technology), Chang D. Yoo (Korea Advanced Institute of Science and Technology)
ClassificationOptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: A trainable calibration loss without internal hyperparameters is proposed - Expected Squared Difference (ESD), which can be optimized simultaneously with negative log-likelihood loss;
Estimating individual treatment effects under unobserved confounding using binary instruments
Dennis Frauen (Munich Center for Machine Learning), Stefan Feuerriegel (Munich Center for Machine Learning)
Biomedical DataElectronic Health Records
🎯 What it does: A multiple robust machine learning framework MRIV is proposed to estimate the conditional average treatment effect (CATE) in the presence of unobservable confounding and only binary instrumental variables.
EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model
Yifu Yuan (Tianjin University), Changjie Fan (Netease Inc)
Robotic IntelligenceReinforcement LearningTabularBenchmark
🎯 What it does: Proposes the EUCLID framework, which jointly trains multi-head dynamic models and policies during the unsupervised pre-training phase, and utilizes the pre-trained model for model predictive planning during the fine-tuning phase, significantly improving sample efficiency for downstream tasks.
Eva: Practical Second-order Optimization with Kronecker-vectorized Approximation
Lin Zhang (Hong Kong University of Science and Technology), Bo Li (Hong Kong University of Science and Technology)
OptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: A second-order optimizer named Eva is proposed, which constructs the curvature matrix using Kronecker-vectorization approximation and implements gradient preconditioning without explicit inverse matrices using the Sherman-Morrison formula;
EVA3D: Compositional 3D Human Generation from 2D Image Collections
Fangzhou Hong (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
GenerationData SynthesisPose EstimationNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: We propose EVA3D, a human generation model based on compositional NeRF, capable of learning to generate high-resolution (512×256) 3D humans from a collection of 2D images.
Evaluating Long-Term Memory in 3D Mazes
Jurgis Pašukonis, Danijar Hafner (DeepMind)
Representation LearningReinforcement LearningSequentialBenchmark
🎯 What it does: A 3D maze benchmark specifically designed to measure the long-term memory capabilities of reinforcement learning agents, called Memory Maze, has been designed and evaluated, providing human baselines, offline datasets, and representation learning probes.
Evaluating Representations with Readout Model Switching
Yazhe Li (DeepMind), Marcus Hutter
Representation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: This paper proposes a representation evaluation framework based on the Minimum Description Length (MDL) principle, which dynamically selects the most suitable decoder by switching between different reading models, thereby unifying the evaluation of the quality of pre-trained representations in downstream tasks.
EVC: Towards Real-Time Neural Image Compression with Mask Decay
Wang Guo-Hua, Yan Lu (Microsoft Research Asia)
CompressionKnowledge DistillationRepresentation LearningConvolutional Neural NetworkImageVideo
🎯 What it does: Designed and implemented an efficient single-model variable bitrate image compression scheme that balances real-time performance with excellent rate-distortion performance.
Everybody Needs Good Neighbours: An Unsupervised Locality-based Method for Bias Mitigation
Xudong Han (University of Melbourne), Trevor Cohn (University of Melbourne)
Tabular
🎯 What it does: An unsupervised neighborhood-based proxy labeling method, ULPL, is proposed for bias mitigation in models without protected attribute labels.
Evidential Uncertainty and Diversity Guided Active Learning for Scene Graph Generation
Shuzhou Sun (University of Oulu), Li Liu (National University of Defense Technology)
Object DetectionGenerationGraph Neural NetworkReinforcement LearningImage
🎯 What it does: A hybrid active learning framework for scene graph generation (EDAL) is proposed, which reduces the cost of relationship annotation and improves model performance through active sampling.
Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systems
Zhong Yi Wan (Google Research), Fei Sha (Google Research)
Time SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: By constructing a spatiotemporal continuous hypernetwork framework, a surrogate model for low-dimensional smooth latent dynamics suitable for governing transport equations was learned.
Evolving Populations of Diverse RL Agents with MAP-Elites
Thomas PIERROT, Arthur Flajolet (InstaDeep)
Robotic IntelligenceReinforcement LearningAgentic AI
🎯 What it does: A PBT-MAP-ELITES framework is proposed that combines a complete RL agent (including policy parameters, other learnable parameters, and hyperparameters) with MAP-ELITES, aiming to achieve both high quality and diversity simultaneously.
Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods
Shunta Akiyama (University of Tokyo), Taiji Suzuki (University of Tokyo)
OptimizationKnowledge DistillationTabular
🎯 What it does: This study investigates the excess risk of a two-layer ReLU neural network in a teacher-student regression model, particularly how the student network learns the unknown teacher network through its output.
Explaining RL Decisions with Trajectories
Shripad Vilasrao Deshmukh (Adobe), Jayakumar Subramanian (Adobe)
Explainability and InterpretabilityRecurrent Neural NetworkTransformerReinforcement LearningSequential
🎯 What it does: This paper proposes a trajectory-based reinforcement learning decision explanation framework, which encodes trajectories from offline RL data into sequence embeddings and clusters them, then trains an explanation policy on the complement data to attribute the decisions of the original policy using action distance and data embedding distance.
Explaining Temporal Graph Models through an Explorer-Navigator Framework
Wenwen Xia (Shanghai Jiao Tong University), Dongsheng Li (Microsoft Research Asia)
Explainability and InterpretabilityGraph Neural NetworkReinforcement LearningGraphTime Series
🎯 What it does: This paper proposes an instance-level interpreter for temporal graph neural networks, which can automatically find the subset of historical events that has the greatest impact on the predicted outcome for a given event.
Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation
Jie Yang (International Digital Economy Academy), Lei Zhang (International Digital Economy Academy)
Object DetectionPose EstimationTransformerImage
🎯 What it does: This paper proposes ED-Pose, a fully end-to-end single-stage multi-person pose estimation framework that utilizes explicit box detection to unify global (human-level) and local (joint-level) information.
Explicitly Minimizing the Blur Error of Variational Autoencoders
Gustav Bredell (ETH Zurich), Ender Konukoglu (ETH Zurich)
RestorationGenerationAuto EncoderImageMagnetic Resonance Imaging
🎯 What it does: Proposes an explicit minimization of blur error in the reconstruction loss of the Variational Autoencoder (VAE) to make generated images sharper.
Exploring Active 3D Object Detection from a Generalization Perspective
Yadan Luo (University of Queensland), Mahsa Baktashmotlagh (University of Queensland)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: An active learning framework CRB is proposed in LiDAR 3D object detection, aiming to achieve near fully supervised detection performance with extremely low annotation costs.
Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness
Yuancheng Xu (University of Maryland), Furong Huang (University of Maryland)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: The study investigates the dynamics of decision boundaries during adversarial training and proposes a dynamic awareness robust training method based on boundary speed (DyART), which directly optimizes the boundary distance in the input space of samples to enhance robustness.