ICLR 2023 Papers — Page 8
International Conference on Learning Representations · 1573 papers
Instance-wise Batch Label Restoration via Gradients in Federated Learning
Kailang Ma (Beihang University), Jianwei Liu (Beihang University)
ClassificationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: In the context of federated learning, the recovery of instance-level labels for batch samples is achieved by analyzing the shared gradients.
Integrating Symmetry into Differentiable Planning with Steerable Convolutions
Linfeng Zhao (Northeastern University), Lawson L.S. Wong (Northeastern University)
OptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: Proposes the Symmetric Planning framework, integrating symmetry into differentiable planning by viewing value iteration as steerable convolution and implementing it using steerable convolutional networks.
Interaction-Based Disentanglement of Entities for Object-Centric World Models
Akihiro Nakano (University of Tokyo), Yutaka Matsuo (University of Tokyo)
Object TrackingOptimizationRecurrent Neural NetworkAuto EncoderWorld ModelVideo
🎯 What it does: This paper studies an unsupervised interactive entity decoupling model STEDIE, which can decompose video sequences into object-level global and relational features, achieving decoupling of object space and time, and is used for downstream planning and causal understanding.
Interactive Portrait Harmonization
Jeya Maria Jose Valanarasu (Johns Hopkins University), Vishal Patel
Image HarmonizationConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: This paper proposes an interactive portrait harmonization framework that allows users to select any reference area in the background to guide the matching of tones and brightness between the foreground and background.
Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation
David Lipshutz (Flatiron Institute), Dmitri Chklovskii (Flatiron Institute)
Recurrent Neural NetworkTabular
🎯 What it does: Two types of linear recurrent neural networks were constructed—one using direct lateral recurrent connections and the other using indirect recurrent connections through local inhibitory interneurons—to study their learning dynamics in the ZCA statistical whitening task.
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small
Kevin Ro Wang (Redwood Research), Jacob Steinhardt (University of California Berkeley)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Conducted a mechanistic interpretability study on GPT-2 small, reverse-engineering a circuit composed of 28 attention heads that implement the Indirect Object Identification (IOI) task.
Interpretability with full complexity by constraining feature information
Kieran A Murphy, Danielle Bassett
Explainability and InterpretabilityAuto EncoderTabular
🎯 What it does: This paper proposes to constrain the feature information flow by adding an information bottleneck to each feature, thereby achieving interpretability while keeping the model complexity unrestricted.
Interpretable Debiasing of Vectorized Language Representations with Iterative Orthogonalization
Prince Osei Aboagye (University of Utah), Jeff Phillips
Explainability and InterpretabilityText
🎯 What it does: An iterative subspace correction method (ISR) is proposed, which alleviates bias and retains original information by gradually orthogonalizing multiple pairs of conceptual subspaces in word vectors.
Interpretable Geometric Deep Learning via Learnable Randomness Injection
Siqi Miao (Georgia Institute of Technology), Pan Li (Purdue University)
Explainability and InterpretabilityGraph Neural NetworkGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes a learnable randomness injection (LRI) mechanism that allows geometric deep learning models to directly generate interpretable point importance scores while maintaining high predictive performance.
Interpretations of Domain Adaptations via Layer Variational Analysis
Huan-Hsin Tseng (Academia Sinica), Yu Tsao (Academia Sinica)
RestorationDomain AdaptationOptimizationConvolutional Neural NetworkRecurrent Neural NetworkImageTime SeriesAudio
🎯 What it does: A theoretical framework based on Layer Variational Analysis (LVA) is proposed to explain and implement transfer learning and domain adaptation in deep networks, and an optimal first-order weight update formula is provided through this framework; its effectiveness is validated on three types of tasks (1D time series regression, speech enhancement, and image deblurring).
Investigating Multi-task Pretraining and Generalization in Reinforcement Learning
Adrien Ali Taiga (MILA, Université de Montréal), Marc G Bellemare
Reinforcement LearningVideo
🎯 What it does: Using IMPALA for multi-task pre-training and fine-tuning on different modes/difficulty variants of Atari 2600, studying its zero-shot and sample efficiency generalization capabilities.
Is a Caption Worth a Thousand Images? A Study on Representation Learning
Shibani Santurkar (Stanford University), Tatsunori Hashimoto (Stanford University)
Representation LearningConvolutional Neural NetworkTransformerContrastive LearningImageTextMultimodality
🎯 What it does: This study investigates the value of language supervision in visual representation learning, comparing the transfer performance of CLIP and image-only supervised SimCLR under different data conditions.
Is Adversarial Training Really a Silver Bullet for Mitigating Data Poisoning?
Rui Wen (CISPA Helmholtz Center for Information Security), Yang Zhang (CISPA Helmholtz Center for Information Security)
Adversarial AttackData-Centric LearningConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: Proposes the EntF method, which reduces the clean set accuracy of adversarial training models by inducing feature entanglement of different category samples in the feature space.
Is Attention All That NeRF Needs?
Mukund Varma T (Indian Institute of Technology Madras), Zhangyang Wang (University of Texas at Austin)
GenerationData SynthesisDepth EstimationTransformerNeural Radiance FieldImage
🎯 What it does: This paper proposes the Generalizable NeRF Transformer (GNT), a fully Transformer architecture that can instantly infer new views from multi-view images without the need for scene-by-scene optimization.
Is Conditional Generative Modeling all you need for Decision Making?
Anurag Ajay (Massachusetts Institute of Technology), Pulkit Agrawal (Massachusetts Institute of Technology)
GenerationOptimizationRobotic IntelligenceReinforcement LearningDiffusion modelTabular
🎯 What it does: This paper proposes transforming offline decision-making problems into conditional generative modeling, directly generating optimal trajectories by training diffusion models (Decision Diffuser) with return, constraint, or skill conditions, thus avoiding the value function estimation and dynamic programming of traditional reinforcement learning.
Is Forgetting Less a Good Inductive Bias for Forward Transfer?
Jiefeng Chen (University of Wisconsin Madison), Arslan Chaudhry (DeepMind)
Representation LearningConvolutional Neural NetworkImage
🎯 What it does: A new forward transfer evaluation method is proposed, which performs k-shot linear probing after fixing feature representations during the continual learning process, independent of any weight preservation or regularization during continual learning.
Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Function
Ruijie Zheng (University of Maryland), Furong Huang (University of Maryland)
Reinforcement LearningSequential
🎯 What it does: This paper studies how the integration of probabilistic dynamic models in model-based reinforcement learning can enhance performance, and proposes that using a single deterministic model can achieve the same or even better performance by leveraging the Lipschitz property of the regularized value function.
Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
Rajkumar Ramamurthy (Fraunhofer IAIS), Yejin Choi (Paul G. Allen School of Computer Science, University of Washington)
GenerationOptimizationTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: An open-source RL4LMs library, GRUE evaluation benchmark, and NLPO algorithm are proposed for aligning large language models (LLMs) with human preferences through reinforcement learning (RL);
IS SYNTHETIC DATA FROM GENERATIVE MODELS READY FOR IMAGE RECOGNITION?
Ruifei He (University of Hong Kong), XIAOJUAN QI
ClassificationRecognitionData SynthesisTransformerDiffusion modelContrastive LearningImage
🎯 What it does: This paper systematically evaluates the effectiveness of synthetic images generated by the current state-of-the-art text-to-image generation model (GLIDE) in image recognition tasks, covering three scenarios: zero-shot, few-shot learning, and large-scale pre-training, and proposes various strategies to enhance the quality and diversity of synthetic data.
Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification
Takashi Ishida (University of Tokyo), Masashi Sugiyama (RIKEN)
ClassificationTransformerImage
🎯 What it does: A method for directly estimating Bayesian error in binary classification problems is proposed, aimed at evaluating classifier performance and detecting overfitting in the test set.
ISAAC Newton: Input-based Approximate Curvature for Newton's Method
Felix Petersen (Stanford University), Oliver Deussen (University of Michigan)
OptimizationConvolutional Neural NetworkTransformerSupervised Fine-TuningImageText
🎯 What it does: This paper proposes the ISAAC (Input-baSed ApproximAte Curvature) algorithm, which utilizes second-order information from layer inputs to condition the gradient, thereby achieving efficient approximate Newton updates.
ISS: Image as Stepping Stone for Text-Guided 3D Shape Generation
Zhengzhe Liu (Chinese University of Hong Kong), Chi-Wing Fu (Chinese University of Hong Kong)
GenerationData SynthesisContrastive LearningImageTextMesh
🎯 What it does: A text-guided 3D shape generation framework ISS based on images as intermediaries is proposed, achieving the generation from text to 3D shapes through two-stage feature space alignment.
Iterative Circuit Repair Against Formal Specifications
Matthias Cosler (CISPA Helmholtz Center for Information Security), Bernd Finkbeiner (CISPA Helmholtz Center for Information Security)
TransformerSequential
🎯 What it does: A deep learning method is proposed that utilizes Transformer to repair sequential circuits to meet given Linear Temporal Logic (LTL) specifications.
Iterative Patch Selection for High-Resolution Image Recognition
Benjamin Bergner (Hasso Plattner Institute for Digital Engineering), Aravindh Mahendran (Google Research)
ClassificationRecognitionTransformerContrastive LearningImage
🎯 What it does: An Iterative Patch Selection (IPS) method is proposed to select the most discriminative patches in high-resolution images without using gradients, and aggregates the selected patches using a cross-attention Transformer, enabling training and inference of large images within limited GPU memory.
Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks
Shuai Zhang (Rensselaer Polytechnic Institute), Miao Liu (IBM Research)
Graph Neural NetworkGraph
🎯 What it does: Theoretical analysis of joint edge sampling and model pruning for Graph Neural Networks (GNN) is conducted, proving that this joint sparse learning can achieve zero generalization error under a two-layer GNN, and providing upper bounds for sample complexity and convergence rate.
Jointly Learning Visual and Auditory Speech Representations from Raw Data
Alexandros Haliassos (Imperial College London), Maja Pantic (Imperial College London)
RecognitionRepresentation LearningTransformerContrastive LearningVideoMultimodalityAudio
🎯 What it does: Proposes the RAVEn framework, which uses self-supervised cross-modal learning to jointly learn visual and auditory speech representations from raw video and audio, and fine-tunes on visual and auditory speech recognition tasks.
Kernel Neural Optimal Transport
Alexander Korotin (Skolkovo Institute of Science and Technology), Evgeny Burnaev (Skolkovo Institute of Science and Technology)
Image TranslationOptimizationImage
🎯 What it does: This paper proposes a neural optimal transport (NOT) method based on kernel weak quadratic cost to learn one-to-one or one-to-many stochastic transport plans.
kNN-Diffusion: Image Generation via Large-Scale Retrieval
Shelly Sheynin (Meta AI), Yaniv Taigman (Meta AI)
GenerationRetrievalDiffusion modelImageMultimodality
🎯 What it does: A text-to-image generation framework based on the combination of large-scale retrieval (kNN) and diffusion models is proposed, capable of achieving cross-domain generation and local semantic editing using only image data (without text labels) for training.
KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLP
Yufei Wang (Macquarie University), Daxin Jiang (Microsoft Corporation)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Developed KnowDA, a Seq2Seq generation model based on multi-task pre-training (KoMT) for data augmentation in low-resource NLP tasks;
Knowledge Distillation based Degradation Estimation for Blind Super-Resolution
Bin Xia (Tsinghua University), Luc Van Gool (ETH Zurich)
RestorationSuper ResolutionKnowledge DistillationConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a blind super-resolution network based on knowledge distillation, KDSR, which first learns implicit degradation representations using a teacher network, and then extracts the same representations using a student network solely from low-resolution images, employing dynamic convolutions generated by IDR for efficient super-resolution reconstruction.
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
Xiaoman Pan (Tencent AI Lab), Jianshu Chen (Tencent AI Lab)
TransformerLarge Language ModelMixture of ExpertsTextRetrieval-Augmented Generation
🎯 What it does: A semi-parametric language model KiC is constructed, which combines external knowledge bases with text-to-text Transformers, supporting various types of knowledge and dynamically selecting them for each instance.
Koopman Neural Operator Forecaster for Time-series with Temporal Distributional Shifts
Rui Wang (University of California San Diego), Rose Yu (University of California San Diego)
TransformerTime SeriesSequentialFinance Related
🎯 What it does: A deep sequence model utilizing Koopman theory, called the Koopman Neural Forecaster (KNF), is proposed for predicting non-stationary time series with temporal distribution drift.
KwikBucks: Correlation Clustering with Cheap-Weak and Expensive-Strong Signals
Sandeep Silwal (Massachusetts Institute of Technology), Seyed Mehran Kazemi (Google Research)
Recommendation SystemOptimizationGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: This paper proposes an algorithm called KwikBucks, which utilizes a cheap weak similarity oracle to guide expensive strong similarity oracle queries within a budget-constrained Budgeted Clustering (BCC) framework.
Label Propagation with Weak Supervision
Rattana Pukdee (Carnegie Mellon University), Nina Balcan
ClassificationGraph Neural NetworkImageTextBenchmark
🎯 What it does: In traditional label propagation (LPA), prior information (such as weak labeler predictions) is incorporated to provide new error bounds, and a method is proposed to integrate multi-source noise information into the graph by adding 'dongle' nodes, ultimately generating more accurate pseudo-labels in weakly supervised scenarios.
Label-free Concept Bottleneck Models
Tuomas Oikarinen (University of California San Diego), Tsui-Wei Weng (University of California San Diego)
Explainability and InterpretabilityLarge Language ModelContrastive LearningImage
🎯 What it does: An automated concept bottleneck model framework that does not require concept labels is proposed, which can transform any network into an interpretable CBM;
Language Modelling with Pixels
Phillip Rust (University of Copenhagen), Desmond Elliott (University of Copenhagen)
TransformerLarge Language ModelAuto EncoderText
🎯 What it does: PIXEL is proposed, a language model that renders text as images and uses Vision Transformer for masked autoencoding, bypassing the traditional vocabulary bottleneck.
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
Abulhair Saparov (New York University), He He (New York University)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes a QA dataset PRONTOQA based on compositional ontology and conducts a systematic formal analysis of the reasoning capabilities of large language models (LLMs) through chain-of-thought (CoT) reasoning.
Language models are multilingual chain-of-thought reasoners
Freda Shi (Google Research), Jason Wei (Google Research)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: This paper studies multilingual chain-of-thought reasoning, proposing and constructing the MGSM benchmark to evaluate the arithmetic reasoning capabilities of large language models in ten different languages.
Language Models are Realistic Tabular Data Generators
Vadim Borisov (University of Tuebingen), Gjergji Kasneci (Technical University of Munich)
GenerationData SynthesisTransformerLarge Language ModelTabular
🎯 What it does: A method is proposed for text encoding of tabular data and generating high-quality synthetic tabular data using Transformer-based LLMs (such as GPT-2).
Language Models Can Teach Themselves to Program Better
Patrick Haluptzok (Microsoft Research), Adam Tauman Kalai (Microsoft Research)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Using a language model to self-generate programming problems and answers, verifying correctness through a Python interpreter, and then fine-tuning the model to significantly improve its performance on programming problems.
Large Language Models are Human-Level Prompt Engineers
Yongchao Zhou (University of Toronto), Jimmy Ba (University of Toronto)
OptimizationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Using large language models to automatically generate and select natural language prompts (Instructions), treating prompt engineering as a black-box optimization and program synthesis problem;
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations
Polina Kirichenko (New York University), Andrew Gordon Wilson (New York University)
Domain AdaptationImage
🎯 What it does: This paper proposes a method (DFR) to retrain only the last layer of a pre-trained neural network to improve robustness against spurious correlations and domain shifts.
Latent Bottlenecked Attentive Neural Processes
Leo Feng (Mila Université de Montréal), Mohamed Osama Ahmed (Borealis AI)
TransformerImageTime Series
🎯 What it does: This paper proposes Latent Bottlenecked Attentive Neural Processes (LBANPs), a neural process model that maintains high performance while having query complexity independent of the number of context points.
Latent Graph Inference using Product Manifolds
Haitz Sáez de Ocáriz Borde (University of Oxford), Pietro Lio (University of Cambridge)
Anomaly DetectionRepresentation LearningGraph Neural NetworkGraphMagnetic Resonance Imaging
🎯 What it does: A latent graph inference method based on product manifolds (dDGM-) is proposed, which can adaptively learn graph structures during training and map node features into Riemannian geometric spaces with variable curvature.
Latent Neural ODEs with Sparse Bayesian Multiple Shooting
Valerii Iakovlev (Aalto University), Harri Lähdesmäki (University of Tübingen)
TransformerTime SeriesSequentialPhysics RelatedOrdinary Differential Equation
🎯 What it does: Proposes a sparse Bayesian multi-shooting latent neural ODE method to address the challenge of training on long trajectories.
Latent State Marginalization as a Low-cost Approach for Improving Exploration
Dinghuai Zhang (Mila, University de Montreal), Ricky T. Q. Chen (Meta AI)
Reinforcement LearningWorld ModelSequential
🎯 What it does: Introduce latent variable policies within the maximum entropy reinforcement learning framework, and achieve better exploration and robustness through a low-cost latent variable marginalization method.
Latent Variable Representation for Reinforcement Learning
Tongzheng Ren (UT Austin), Bo Dai (Google Research)
Reinforcement Learning
🎯 What it does: Proposes a linear MDP representation based on latent variable models (LV-Rep) and designs a feasible RL algorithm based on this;
LAVA: Data Valuation without Pre-Specified Learning Algorithms
Hoang Anh Just (Virginia Tech), Ruoxi Jia (Virginia Tech)
Computational EfficiencyAdversarial AttackData-Centric LearningImage
🎯 What it does: A learning algorithm-independent data value assessment framework LAVA is proposed, utilizing category-based Wasserstein distance as a proxy for validating performance, and assigning values to individual samples through sensitivity analysis of OT distance.
Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations
Ziyu Jiang (Texas A&M University), Zhangyang Wang (University of Texas at Austin)
Representation LearningTransformerContrastive LearningImage
🎯 What it does: Proposes a Layer Grafted Pre-training framework that allocates Mask Image Modeling (MIM) and Contrastive Learning (CL) hierarchically, pre-training the lower layers with MIM and the higher layers with CL;
LDMIC: Learning-based Distributed Multi-view Image Coding
Xinjie Zhang (Hong Kong University of Science and Technology), Jun Zhang (Hong Kong University of Science and Technology)
CompressionImage
🎯 What it does: A learning-driven distributed multi-view image compression framework LDMIC is proposed, which adopts an independent encoding and joint decoding mode, utilizing cross-view attention to achieve global mutual view information fusion.
Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection
Jiajun Fan (Tsinghua University), Shu-Tao Xia (Tsinghua University)
Reinforcement LearningVideoBenchmark
🎯 What it does: A generalizable learnable behavior control framework (LBC) is proposed, which expands the behavior selection space through mixed behavior mapping and achieves sample-efficient Atari game learning in a distributed offline actor-critic method.
Learnable Graph Convolutional Attention Networks
Adrián Javaloy (Saarland University), Isabel Valera (Max Planck Institute for Software Systems)
ClassificationGraph Neural NetworkGraph
🎯 What it does: A learnable graph convolutional attention network (L-CAT) is proposed, which automatically switches between GCN, GAT, and the newly proposed CAT at each layer by learning interpolation parameters, thus eliminating the need for manual cross-validation to select layer types.
Learnable Topological Features For Phylogenetic Inference via Graph Neural Networks
Cheng Zhang (Peking University)
Graph Neural NetworkGraph
🎯 What it does: A learnable topological feature based on graph neural networks is proposed for the systematic representation of tree structures without domain-specific knowledge.
Learned Index with Dynamic $\epsilon$
Daoyuan Chen (Alibaba Group), Jingren Zhou (Alibaba Group)
TabularTime Series
🎯 What it does: This paper proposes a learning index framework that can dynamically adjust the error upper bound ε. Through theoretical derivation, it connects ε with local data features (mean µ, variance σ), and automatically selects an appropriate ε based on local distribution during learning in each linear segment, thereby enhancing the space-time trade-off.
Learning a Data-Driven Policy Network for Pre-Training Automated Feature Engineering
Liyao Li (Zhejiang University), Junbo Zhao (Zhejiang University)
Data-Centric LearningTransformerReinforcement LearningTabular
🎯 What it does: This paper proposes the FETCH framework, which automates feature engineering based on reinforcement learning and constructs a data-driven feature search process by directly using the raw dataset as the MDP state.
Learning About Progress From Experts
Jake Bruce (DeepMind), Rob Fergus (DeepMind)
Reinforcement LearningSequential
🎯 What it does: A progress regression model is trained using expert demonstrations (only observations, no actions, no rewards) to extract monotonic progress information during the game process, and this progress is used as auxiliary rewards to drive agents to efficiently explore and complete tasks in sparse reward environments like NetHack.
Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward
Zihan Zhou (University of Toronto), Animesh Garg (University of Toronto)
Reinforcement Learning
🎯 What it does: Proposes the SEA algorithm, which explores achievement-based environments with sparse rewards through a multi-stage process;
Learning Adversarial Linear Mixture Markov Decision Processes with Bandit Feedback and Unknown Transition
Canzhe Zhao (Shanghai Jiao Tong University), Shuai Li (Shanghai Jiao Tong University)
Reinforcement Learning
🎯 What it does: The study investigates reinforcement learning in linear mixed Markov decision processes (MDP) under unknown transitions, adversarial losses, and bandit feedback, and proposes a feasible algorithm LSUOB-REPS;
Learning Continuous Normalizing Flows For Faster Convergence To Target Distribution via Ascent Regularizations
Shuangshuang Chen (Royal Institute of Technology), Mårten Björkman (Royal Institute of Technology)
Flow-based ModelImageTabularOrdinary Differential Equation
🎯 What it does: A new type of continuous normalizing flow (ACNF) and its ascent regularization implementation are proposed, aimed at accelerating the convergence from the base distribution to the target distribution, and applied to density estimation, unbiased sampling, and variational inference.
Learning Controllable Adaptive Simulation for Multi-resolution Physics
Tailin Wu (Stanford University), Jure Leskovec (Stanford University)
OptimizationGraph Neural NetworkReinforcement LearningMeshPhysics Related
🎯 What it does: This paper proposes a fully deep learning-based adaptive physical simulation framework called LAMP, which can automatically refine or coarsen the grid at each time step based on the system dynamics while simultaneously learning the physical evolution model.
Learning Cut Selection for Mixed-Integer Linear Programming via Hierarchical Sequence Model
Zhihai Wang (University of Science and Technology of China), Feng Wu (University of Science and Technology of China)
OptimizationRecurrent Neural NetworkReinforcement Learning
🎯 What it does: This study investigates the cutting plane selection problem in Mixed Integer Linear Programming (MILP) and proposes a hierarchical sequence model based on reinforcement learning that can simultaneously learn which cutting planes to select, how many to select, and the order in which to add them.
Learning differentiable solvers for systems with hard constraints
Geoffrey Négiar (University of California), Aditi Krishnapriyan
Physics RelatedOrdinary Differential Equation
🎯 What it does: A differentiable PDE constraint layer has been developed to rigidly enforce the satisfaction of partial differential equation constraints within neural networks, thereby learning the mapping from PDE parameters to solutions in an unsupervised setting.
Learning Diffusion Bridges on Constrained Domains
Xingchao Liu (University of Texas at Austin), qiang liu
GenerationData SynthesisDiffusion modelImageTabular
🎯 What it does: A general framework based on Doob h-transformation is proposed to learn diffusion generative models on constrained domains.
Learning Domain-Agnostic Representation for Disease Diagnosis
Churan Wang (Peking University), Yizhou Wang (Fudan University)
Domain AdaptationRepresentation LearningGraph Neural NetworkAuto EncoderImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a domain-agnostic representation learning framework (DarMo) that decouples lesion-related features from central effects through structural causal models and variational autoencoders to achieve robustness in multi-center medical image diagnosis.
Learning Fair Graph Representations via Automated Data Augmentations
Hongyi Ling (Texas A&M University), Na Zou (Texas A&M University)
Representation LearningAdversarial AttackData-Centric LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: Learn fair graph representations through an automated graph data augmentation method (Graphair) to reduce biases caused by sensitive attributes.
Learning Fast and Slow for Online Time Series Forecasting
Quang Pham (Institute for Infocomm Research Agency for Science Technology and Research), Steven Hoi
Recurrent Neural NetworkTime Series
🎯 What it does: This study investigates fast and slow learning in online time series forecasting, proposing the FSNet framework to enhance the adaptability and memory capabilities of deep networks in data streams.
Learning Group Importance using the Differentiable Hypergeometric Distribution
Thomas M. Sutter (ETH Zurich), Julia E Vogt
Image
🎯 What it does: A differentiable hypergeometric distribution is proposed for end-to-end learning of subgroup importance (such as cluster size and number of shared factors).
Learning Harmonic Molecular Representations on Riemannian Manifold
Yiqun Wang (ByteDance Research), Hao Zhou (Institute for AI Industry Research Tsinghua University)
Representation LearningDrug DiscoveryGraph Neural NetworkMeshGraph
🎯 What it does: A multi-resolution molecular representation framework HMR based on molecular surface Laplace-Beltrami eigenfunctions is proposed, and a harmonic information propagation and functional mapping mechanism is designed based on this framework.
Learning Heterogeneous Interaction Strengths by Trajectory Prediction with Graph Neural Network
Seungwoong Ha (KAIST), Hawoong Jeong (KAIST)
Graph Neural NetworkTime SeriesSequential
🎯 What it does: This paper proposes an unsupervised trajectory prediction framework called RAIN, which can infer continuous weighted interaction networks and dynamics from multi-agent trajectories and perform future trajectory predictions.
Learning Hierarchical Protein Representations via Complete 3D Graph Networks
Limei Wang (Texas A&M University), Shuiwang Ji (Texas A&M University)
Representation LearningProtein Structure PredictionGraph Neural NetworkGraphBiomedical Data
🎯 What it does: Construct a 3D representation of proteins and design the ProNet hierarchical graph network to achieve protein representation learning at different granularities from amino acids, backbone to all-atom.
Learning Human-Compatible Representations for Case-Based Decision Support
Han Liu (University of Chicago), Chenhao Tan (University of Chicago)
ClassificationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: This study investigates learning human-friendly representations in case-based decision support, combining classification and metric learning to align model representations with human similarity.
Learning Hyper Label Model for Programmatic Weak Supervision
Renzhi Wu (Georgia Tech), Xu Chu (Georgia Tech)
ClassificationGraph Neural NetworkSupervised Fine-TuningTabular
🎯 What it does: A parameter-free hyper label model (Hyper Label Model) is proposed, which can complete label aggregation in a single forward pass and is suitable for programmatic weak supervision scenarios.
Learning in temporally structured environments
Matt Jones (Google Research), Michael Curtis Mozer
OptimizationTime SeriesSequentialStochastic Differential Equation
🎯 What it does: A multi-scale learning optimizer (Variational EKF) is proposed, capable of online learning in a time-varying 1/f noise environment.
Learning Input-agnostic Manipulation Directions in StyleGAN with Text Guidance
Yoonjeon Kim (Korea Advanced Institute of Science and Technology), Eunho Yang (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: This paper proposes a text-guided StyleGAN image editing method based on dictionary learning called Multi2One, which achieves multi-channel interactive editing direction learning and inference while maintaining real-time inference speed.
Learning Iterative Neural Optimizers for Image Steganography
Xiangyu Chen (Cornell University), Kilian Q Weinberger (Cornell University)
Data SynthesisOptimizationConvolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: A learning-based iterative neural optimizer (LISO) is proposed and trained to find low bit error rate and visually natural steganographic images in image steganography.
Learning Kernelized Contextual Bandits in a Distributed and Asynchronous Environment
Chuanhao Li (University of Virginia), Hongning Wang (University of Virginia)
Recommendation SystemOptimizationFederated LearningReinforcement LearningTabular
🎯 What it does: An asynchronous distributed kernelized contextual multi-armed bandit algorithm, Async-KernelUCB, is proposed to address the inefficiency and lack of robustness of previous synchronous communication schemes in large-scale, delayed, and failure-prone environments.
Learning Label Encodings for Deep Regression
Deval Shah (University of British Columbia), Tor M. Aamodt (University of British Columbia)
Pose EstimationAutonomous DrivingConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: An end-to-end regularized label encoding learning method (RLEL) is proposed for binary classification encoding in regression problems.
Learning Language Representations with Logical Inductive Bias
Jianshu Chen (Tencent AI Lab)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Treats language representation learning as a reasoning process based on first-order logic, designing a differentiable FOLNet architecture;
Learning Locality and Isotropy in Dialogue Modeling
Han Wu (City University of Hong Kong), Linqi Song (City University of Hong Kong)
GenerationRetrievalTransformerLarge Language ModelContrastive LearningText
🎯 What it does: A dialogue representation calibration method called SimDRC is proposed, which optimizes the contextual representation of dialogue models using locality and isotropy constraints.
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
Edo Cohen-Karlik (Tel Aviv University), Amir Globerson (Google Research)
Recurrent Neural NetworkSequential
🎯 What it does: This paper studies the implicit extrapolation characteristics of parameterized recurrent neural networks (RNNs) during gradient descent training. It proves that when the training sequence length is greater than twice the dimensionality of the teacher network's state space, gradient descent converges to a low-dimensional state space solution that can extrapolate to longer sequences, and this theory is experimentally validated on linear RNNs and nonlinear GRUs.
Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions
Ansong Ni (Yale University), Jianfeng Gao (Microsoft Research)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a method that allows pre-trained language models to self-sample multiple correct or partially correct solutions during training, using these solutions as multiple objectives for learning, thereby improving the quality of answers in multi-step mathematical reasoning tasks.
Learning MLPs on Graphs: A Unified View of Effectiveness, Robustness, and Efficiency
Yijun Tian (University of Notre Dame), Nitesh Chawla
Computational EfficiencyKnowledge DistillationRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: Train a multi-layer perceptron (MLP) without message passing on graphs through knowledge distillation, enabling it to capture graph structural information and be robust to feature noise.
Learning multi-scale local conditional probability models of images
Zahra Kadkhodaie (New York University), Eero P Simoncelli
RestorationGenerationSuper ResolutionConvolutional Neural NetworkScore-based ModelImage
🎯 What it does: This paper proposes a Markov conditional probability model based on multi-scale wavelet decomposition, utilizing convolutional networks with local receptive fields to learn the conditional scores of images, thereby achieving denoising, super-resolution, and image synthesis.
Learning Multimodal Data Augmentation in Feature Space
Zichang Liu (Rice University), Andrew Gordon Wilson (New York University)
Data SynthesisRepresentation LearningAuto EncoderGenerative Adversarial NetworkMultimodality
🎯 What it does: A general multimodal data augmentation framework named LeMDA is proposed.
Learning Object-Language Alignments for Open-Vocabulary Object Detection
Chuang Lin (Monash University), Jianfei Cai (Monash University)
Object DetectionVision Language ModelContrastive LearningImageText
🎯 What it does: This paper proposes an open vocabulary object detection model VLDet, which directly learns the alignment between image regions and text words from image-text pair data without the need for manual annotation.
Learning on Large-scale Text-attributed Graphs via Variational Inference
Jianan Zhao (Mila - Quebec AI Institute), Jian Tang (Mila - Quebec AI Institute)
Computational EfficiencyKnowledge DistillationRepresentation LearningGraph Neural NetworkLarge Language ModelTextGraph
🎯 What it does: The research focuses on learning node representations on Text Attribute Graphs (TAG) and proposes the GLEM framework, which achieves scalable integration of language models and graph neural networks through variational EM, addressing the computational and memory bottlenecks of traditional end-to-end training.
Learning Probabilistic Topological Representations Using Discrete Morse Theory
Xiaoling Hu (Stony Brook University), Chao Chen (Stony Brook University)
SegmentationBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Directly learning the topological/structural representation of fine structures in images, constructing a discrete Morse theory to obtain a structural space, and learning a probabilistic model in that space, which in turn yields topologically complete segmentation results through sampling during the inference phase.
Learning Proximal Operators to Discover Multiple Optima
Lingxiao Li (Massachusetts Institute of Technology), Justin Solomon (Massachusetts Institute of Technology)
Object DetectionOptimizationImagePoint Cloud
🎯 What it does: Learn and train proximal operators so that for any parameterized non-convex optimization problem, one can quickly obtain multiple local optimal solutions by iterating from a random initial point for only a few steps, and generalize to unseen problems.
Learning Rationalizable Equilibria in Multiplayer Games
Yuanhao Wang (Princeton University), Chi Jin (Salesforce Research)
Reinforcement Learning
🎯 What it does: This paper proposes an efficient algorithm to learn the rationalizable action profiles and coarse correlated equilibria (CCE) and correlated equilibria (CE) in multi-player games.
Learning ReLU networks to high uniform accuracy is intractable
Julius Berner (University of Vienna), Felix Voigtlaender (Catholic University of Eichstätt-Ingolstadt)
Recurrent Neural Network
🎯 What it does: This paper quantifies the number of training samples required by any learning algorithm to guarantee high uniform accuracy on ReLU neural networks with a given architecture, proving that under general assumptions, the required sample size grows exponentially with the network depth and input dimension.
Learning rigid dynamics with face interaction graph networks
Kelsey R Allen, Tobias Pfaff (DeepMind)
Robotic IntelligenceGraph Neural NetworkMesh
🎯 What it does: A graph network model FIGNet is proposed to learn rigid body collisions through interactions between mesh faces.
Learning Simultaneous Navigation and Construction in Grid Worlds
Wenyu Han, Chen Feng (New York University)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningSimultaneous Localization and MappingMesh
🎯 What it does: A new learning task is proposed - Mobile Construction, where agents simultaneously locate and construct target structures in a constantly changing environment within 1/2/3D grid worlds.
Learning Soft Constraints From Constrained Expert Demonstrations
Ashish Gaurav (University of Waterloo), Pascal Poupart (University of Waterloo)
Autonomous DrivingOptimizationReinforcement LearningFlow-based ModelTabularTime Series
🎯 What it does: This paper proposes a method to recover cumulative soft constraints from expert demonstrations through inverse constraint learning under the premise of a known reward function.
Learning Sparse and Low-Rank Priors for Image Recovery via Iterative Reweighted Least Squares Minimization
Stamatios Lefkimmiatis (Huawei Noah's Ark Lab), Iaroslav Sergeevich Koshelev
RestorationSuper ResolutionConvolutional Neural NetworkImageMagnetic Resonance Imaging
🎯 What it does: A general Iterative Reweighted Least Squares (IRLS) algorithm is proposed, combining sparse and low-rank priors (ℓp, Sp norms) for image recovery;
Learning Sparse Group Models Through Boolean Relaxation
Yijie Wang (Indiana University), Jianzhu Ma (Tsinghua University)
OptimizationDrug DiscoveryBiomedical Data
🎯 What it does: A sparse group model learning framework based on Boolean relaxation is proposed, which can directly obtain integer solutions through convex optimization or obtain feasible solutions through randomized rounding while satisfying group sparsity constraints.
Learning Structured Representations by Embedding Class Hierarchy
Siqi Zeng (Carnegie Mellon University), Han Zhao (University of Illinois)
ClassificationDomain AdaptationRepresentation LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a Cophenetic Correlation Coefficient (CPCC) regularizer that utilizes a tree-structured label hierarchy to embed the tree distance of categories into the feature space in supervised learning, facilitating the learning of structured representations.
Learning Symbolic Models for Graph-structured Physical Mechanism
Hongzhi Shi (Tsinghua University), Yong Li (Tsinghua University)
Graph Neural NetworkGraphTabularPhysics Related
🎯 What it does: This study investigates how to automatically learn symbolic models within graph-structured physical mechanisms, proposing a two-stage framework that transforms formula skeleton search into Pareto optimal message passing flow search.
Learning the Positions in CountSketch
Yi Li (Nanyang Technological University), David Woodruff
OptimizationAuto EncoderImageVideoTabular
🎯 What it does: This paper proposes a learning-based sketching method that, in addition to learning the non-zero values of CountSketch as in previous methods, also learns the positions of the non-zero values, thereby achieving lower errors in tasks such as low-rank approximation and second-order optimization.
Learning to Compose Soft Prompts for Compositional Zero-Shot Learning
Nihal V. Nayak (Brown University), Stephen Bach
RecognitionObject DetectionTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: By using learnable soft word embeddings for the attribute and object vocabulary of CLIP, we train composable prompts to enhance the performance of large-scale vision-language models in compositional zero-shot learning.
Learning to CROSS exchange to solve min-max vehicle routing problems
Minjun Kim (Korea Advanced Institute of Science and Technology), Jinkyoo Park (Korea Advanced Institute of Science and Technology)
OptimizationGraph Neural NetworkSupervised Fine-TuningGraph
🎯 What it does: This study investigates the Cross Exchange (NCE) operation based on Graph Neural Networks to efficiently solve various min-max Vehicle Routing Problems (min-max VRP).