International Conference on Machine Learning Β· 421 papers
PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search
Haibin Wang (Alibaba Group), Xiuyu Sun (Alibaba Group)
CodeNeural Architecture SearchTransformerImage
π― What it does: This paper proposes PreNAS, a NAS method that first uses zero-cost proxies to filter high-quality sub-networks and then performs one-shot shared training in a limited subspace.
Probabilistic Imputation for Time-series Classification with Missing Data
SeungHyun Kim (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
CodeClassificationAnomaly DetectionConvolutional Neural NetworkTransformerAuto EncoderTime SeriesBiomedical DataElectronic Health Records
π― What it does: A probability generative framework based on the MNAR assumption, supnotMIWAE, is proposed for classifying missing data in multivariate time series, and the quality of missing value imputation is improved through ObsDropout regularization.
Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models
Alexander Lin (Harvard University), Demba E. Ba
CodeRecommendation SystemOptimizationTabular
π― What it does: This paper proposes a framework called Probabilistic Unrolling to accelerate maximum likelihood estimation using gradient EM in high-dimensional latent Gaussian models (LGM). The core idea is to replace the direct inversion of the covariance matrix with Monte Carlo sampling and to solve the posterior mean and samples using iterative linear solvers (such as conjugate gradient). The iterative process is then backpropagated to obtain more accurate gradients.
Yunlong Hou (National University of Singapore), Zixin Zhong (University of Alberta)
CodeOptimizationReinforcement LearningTabular
π― What it does: Proposed the 'potentially safe at any time' random combination semi-bandit problem, which requires minimizing the loss of cumulative rewards while ensuring that the variance of each step's choice does not exceed a threshold;
π― What it does: A progressive purification method named POP (PrOgressive Purification) is proposed for instance-dependent partial label learning (PLL), which gradually removes incorrect candidate labels based on the current model reliability threshold during each training epoch and retrains the model with the cleaned labels.
π― What it does: This paper conducts theoretical expansion and empirical validation of proper scoring rules in survival analysis, proving the appropriateness of four common rules (Pinball, Logarithmic, Brier, Ranked Probability Score) in survival contexts, and constructs loss functions and evaluation metrics based on these rules.
π― What it does: A continuous learning method is proposed that does not use a replay buffer, utilizing Prototype-Sample Relation Distillation (PRD) to maintain the relative similarity between old class prototypes and new task samples, thereby suppressing forgetting and enhancing adaptability.
Provable Dynamic Fusion for Low-Quality Multimodal Data
Qingyang Zhang (Tianjin University), Xi Peng (Sichuan University)
CodeClassificationRecognitionMultimodality
π― What it does: A dynamic multimodal fusion method QMF based on uncertainty estimation is proposed, and its superiority is proven from the perspective of generalization error theory.
π― What it does: This paper proposes an Approximate Projection-based SchrΓΆdinger Bridge (aIPF) algorithm and provides a convergence analysis; subsequently, the algorithm is applied to probabilistic time series missing value imputation (CSBI), achieving high-quality imputation for randomly missing positions.
π― What it does: A representation learning algorithm utilizing maximum likelihood estimation and optimistic planning is proposed to address the efficient learning and planning problem of low-rank POMDPs.
π― What it does: A continuous sparse perspective pruning framework SpODE based on sparse index ODE is proposed, along with its PSO algorithm. It utilizes the ODE of sparse evolution over time to guide the precise update of the sparse mask, achieving an efficient one-time pruning strategy.
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Stella Biderman (EleutherAI), Oskar van der Wal (Institute for Logic, Language and Computation)
CodeTransformerLarge Language ModelText
π― What it does: A set of 16 decoder-only autoregressive language models (Pythia) ranging in size from 70M to 12B has been constructed and publicly released, along with 154 checkpoints, a complete training data loader, and training code, supporting fine-grained experiments on training dynamics, scaling, bias, memorization, and frequency effects; case studies were also conducted on the impact of gender bias, memorization, and pre-training word frequency on task performance.
Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
Taku Yamagata (University of Bristol), Raul Santos-Rodriguez
CodeTransformerReinforcement LearningSequential
π― What it does: This paper proposes an offline reinforcement learning framework called QDT, which combines Q-learning and Decision Transformer. It utilizes the value backtracking of Q-learning to re-label the return-to-go in offline data, addressing the issues of DT's lack of stitching capability and the instability of Q-learning in sparse/long-horizon tasks.
π― What it does: This paper proposes a GNN-to-MLP knowledge distillation method based on reliable knowledge points, utilizing the invariance of information entropy under noise perturbation to measure the reliability of GNN knowledge, and providing additional supervision for MLP training through reliability sampling.
Quantized Distributed Training of Large Models with Convergence Guarantees
Ilia Markov (Institute of Science and Technology Austria), Dan Alistarh (Institute of Science and Technology Austria)
CodeTransformerLarge Language ModelText
π― What it does: A QSDP (Quantized Sharded Data-Parallel) training framework is proposed, which can quantize weights and gradients while maintaining convergence guarantees, eliminating the communication bottleneck of FSDP and achieving efficient large-scale language model training.
π― What it does: Proposes the RACE framework, which combines evolutionary algorithms and multi-agent reinforcement learning to enhance collaborative learning through asymmetric representation and co-evolution.
π― What it does: A data augmentation method for graph contrastive learning based on randomized Schur complement (rLap) is proposed, utilizing random node elimination and unbiased Clique approximation to preserve the random walk probabilities of the original graph.
π― What it does: This paper discusses the advantages and disadvantages of stochastic and deterministic image recovery methods, proving that stochastic methods have theoretical advantages in consistency, perceptual quality, and robustness.
π― What it does: A backdoor defense method called Reconstructive Neuron Pruning (RNP) is proposed, which combines neuron-level unlearning with filter-level recovering to expose and prune backdoor-related neurons, thereby eliminating backdoor attacks.
π― What it does: A reflection diffusion model is proposed, utilizing reflective stochastic differential equations to model in the data-supported domain;
Regret-Minimizing Double Oracle for Extensive-Form Games
Xiaohang Tang (University College London), Yaodong Yang (Peking University)
CodeOptimizationReinforcement Learning from Human FeedbackReinforcement LearningSequential
π― What it does: A general Regret-Minimizing Double Oracle (RMDO) framework is proposed, and based on this, a Periodic Double Oracle (PDO) is introduced to address the sample complexity issue of the double oracle method in extensive form games (EFG).
Reinforcement Learning from Passive Data via Latent Intentions
Dibya Ghosh (University of California Berkeley), Sergey Levine (University of California Berkeley)
CodeReinforcement LearningVideo
π― What it does: This paper proposes a method to pre-train state representations by learning latent intentions using only passive observation data (without action/reward information), and applies it to downstream reinforcement learning tasks.
Repository-Level Prompt Generation for Large Language Models of Code
Disha Shrivastava (Mila), Daniel Tarlow (McGill University)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: A Repo-Level Prompt Generator (RLPG) framework is proposed, which automatically generates prompts suitable for single-line code completion by utilizing repository structure and cross-file context.
π― What it does: A restorative generative model (RGM) is proposed through the maximum a posteriori (MAP) objective combined with implicit priors, allowing for flexible selection of the denoising process and eliminating the sampling bottleneck of traditional diffusion models.
Rethink DARTS Search Space and Renovate a New Benchmark
Jiuling Zhang (University of Chinese Academy of Sciences), Zhiming Ding (Institute of Software, Chinese Academy of Sciences)
CodeNeural Architecture SearchImageBenchmark
π― What it does: This paper redesigns the DARTS search space, proposing a larger and more challenging LHD space and constructing a multi-condition evaluation benchmark based on it.
π― What it does: This paper studies the model selection problem in offline reinforcement learning, proposing a new estimation method based on Mean Squared Bellman Error (MSBE) and implementing the Supervised Bellman Validation (SBV) algorithm to select the optimal policy from offline data.
π― What it does: This study investigates the backdoor propagation risks of data-free knowledge distillation (data-free KD) when using contaminated teacher models and proposes a pluggable defense framework called ABD.
π― What it does: The study compares the statistical efficiency of generative naive Bayes and discriminative logistic regression classifiers in the linear evaluation of deep pre-trained models, and provides a sample complexity analysis in the case of multiple categories.
Revisiting Domain Randomization via Relaxed State-Adversarial Policy Optimization
Yun-Hsuan Lien (National Yang Ming Chiao Tung University), Yu-Shuen Wang (National Yang Ming Chiao Tung University)
CodeOptimizationReinforcement LearningSequential
π― What it does: This paper proposes a Relaxed Adversarial Policy Optimization (RAPPO) method, which achieves domain randomization by adding adversarial perturbations after each state transition, and simultaneously enhances both average and worst-case rewards through a bi-level optimization approach, addressing the overly conservative issues caused by traditional state adversarial methods.
RGE: A Repulsive Graph Rectification for Node Classification via Influence
Jaeyun Song (Korea Advanced Institute of Science and Technology), Eunho Yang (AITRICS)
CodeClassificationGraph Neural NetworkGraph
π― What it does: This paper proposes a rejection-based edge group elimination method (RGE) that prioritizes the removal of remote edges and performs multiple rounds of retraining to reduce group effect errors, thereby improving node classification performance.
π― What it does: The RLSBench benchmark is proposed, evaluating 13 mainstream domain adaptation methods, exploring their performance in the 'relaxed label shift' scenario where label proportions are mismatched, and introducing a two-step meta-algorithm (pseudo-balanced resampling + post-hoc reweighting) that can be combined with most methods to significantly improve performance.
π― What it does: A one-class classification method based on first-order Lipschitz neural networks learning signature distance functions (SDF) is proposed, using SDF as a normalized metric to achieve provable robustness.
π― What it does: Robust Weight Signatures (RWS) are proposed, achieving lightweight, adjustable, and composable robustness patches by adding RWS to standard model weights.
π― What it does: A coverage verification method based on random points (TARP) is proposed, which uses only posterior sampling to assess the accuracy of generative posterior estimators.
π― What it does: The InforMARL framework is proposed for multi-agent navigation and collision avoidance problems, utilizing graph neural networks to aggregate local neighborhood information, thereby achieving distributed decision-making and effective collaboration under local observations.
π― What it does: A Universal Mini-Batch Consistent (UMBC) set encoder is proposed, along with an unbiased full gradient approximation algorithm, enabling scalable training on large-scale sets.
Scaling Laws for Generative Mixed-Modal Language Models
Armen Aghajanyan (Meta), Luke Zettlemoyer (University of Washington)
CodeGenerationData SynthesisTransformerLarge Language ModelImageTextMultimodalityAudio
π― What it does: This paper studies the scaling law of mixed-modal generative language models through 250+ experiments and derives an extended formula that includes modal synergy and competition.
π― What it does: A post-hoc calibration method based on class-wise loss scaling is proposed, which reduces calibration error and maintains the original model accuracy by balancing the training loss across different classes.
SeedGNN: Graph Neural Network for Supervised Seeded Graph Matching
Liren Yu (Purdue University), Xiaojun Lin (Purdue University)
CodeGraph Neural NetworkGraph
π― What it does: A supervised graph neural network SeedGNN is designed and implemented to match unlabeled graphs under the condition of having only a few seed nodes, and it learns to extract transferable matching knowledge from the training data.
Self-Interpretable Time Series Prediction with Counterfactual Explanations
Jingquan Yan (Rutgers University), Hao Wang (Rutgers University)
CodeExplainability and InterpretabilityAuto EncoderTime SeriesBiomedical Data
π― What it does: A self-explanatory time series forecasting model called CounTS is proposed, which can generate executable and causally constrained actionable counterfactual explanations while maintaining prediction accuracy.
Semi Bandit dynamics in Congestion Games: Convergence to Nash Equilibrium and No-Regret Guarantees.
Ioannis Panageas (University of California Irvine), Volkan Cevher (Ecole Polytechnique Federale de Lausanne)
CodeOptimizationReinforcement LearningGraph
π― What it does: This paper proposes an online gradient descent algorithm with no regret and convergence to Nash equilibrium in a congestion game with semi-bandit feedback (SBGD-CE).
π― What it does: This paper proposes a training objective based on energy scores without the need for a Jacobian determinant, and designs a semi-autoregressive energy flow model that can train reversible networks without calculating the Jacobian determinant.
π― What it does: A semi-offline reinforcement learning framework (semi-offline RL) is proposed, achieving a smooth transition between offline and online while balancing exploration capability and optimization cost.
Sequential Monte Carlo Learning for Time Series Structure Discovery
Feras Saad, Vikash Mansinghka
CodeOptimizationComputational EfficiencyTime Series
π― What it does: This paper proposes a full Bayesian structure learning framework based on Sequential Monte Carlo (SMC) and reversible MCMC for the automatic discovery of higher-order structures in time series models.
Sequential Underspecified Instrument Selection for Cause-Effect Estimation
Elisabeth Ailer (Helmholtz Munich), Niki Kilbertus (Munich Center for Machine Learning)
Code
π― What it does: This paper proposes a method to estimate causal effects through stepwise experiments and instrument variable selection in the case of high-dimensional processing variables where only a limited number of instrumental variables can be randomized.
π― What it does: This paper presents a generalization error upper bound for adaptive Sliced-Wasserstein distance based on PAC-Bayesian theory, and proposes an optimization method (PAC-SW) for learning tangent distributions using this bound.
Shortest Edit Path Crossover: A Theory-driven Solution to the Permutation Problem in Evolutionary Neural Architecture Search
Xin Qiu (Cognizant AI Labs), Risto Miikkulainen (Cognizant AI Labs)
CodeOptimizationNeural Architecture SearchGraph
π― What it does: This paper proposes a crossover operator based on the Shortest Edit Path (SEP) to address the permutation problem in Neural Architecture Search (NAS).
π― What it does: An Automatic Feature Reweighting (AFR) method is proposed, which enhances robustness to spurious features by weighting misclassified samples and fine-tuning only the last layer of a model trained using standard ERM.
π― What it does: This paper proposes a time-consistent simplified representation learning method (TCRL), which trains the encoder and hidden dynamic model using only the hidden state consistency loss, and directly trains the policy and value function on this representation, combining both model-based and model-free learning modes.
SLAMB: Accelerated Large Batch Training with Sparse Communication
Hang Xu (King Abdullah University of Science and Technology), Panos Kalnis (King Abdullah University of Science and Technology)
CodeOptimizationTransformerImageText
π― What it does: An optimizer named SLAMB is proposed, which combines sparse communication with large-batch training, aiming to achieve efficient distributed training on large-scale GPU clusters.
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Guangxuan Xiao (Massachusetts Institute of Technology), Song Han (Massachusetts Institute of Technology)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A post-training quantization method called SmoothQuant is proposed, which can deploy large language models (such as OPT, BLOOM, GLMβ130B, MTβNLG 530B) in INT8 weight + activation (W8A8) format without retraining the model, while maintaining accuracy close to FP16, significantly reducing memory usage and inference latency.
SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process
Zichong Li (University of Science and Technology of China), Hongyuan Zha (Chinese University of Hong Kong)
CodeTransformerScore-based ModelTextTabularTime SeriesSequentialBiomedical DataFinance Related
π― What it does: A Transformer Hawkes process model based on score matching (SMURF-THP) is proposed to quantify the uncertainty of event arrival times.
Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning
Seungwoong Ha (Korea Advanced Institute of Science and Technology), Hawoong Jeong (Korea Advanced Institute of Science and Technology)
CodeOptimizationReinforcement Learning
π― What it does: This study uses a deep reinforcement learning model to optimize individual social learning strategies (SLS) in cooperative games and simulates the spontaneous emergence and evolution of social learning through a multidimensional NK landscape.
Solving High-Dimensional PDEs with Latent Spectral Models
Haixu Wu (Tsinghua University), Mingsheng Long (Tsinghua University)
CodeTransformerPoint CloudMeshBenchmarkPhysics Related
π― What it does: Latent Spectral Models (LSM) are proposed, which compress high-dimensional coordinate space into latent space through an attention-based hierarchical projection network, and then approximate the input-output mapping in the latent space using neural spectral blocks, thereby efficiently solving high-dimensional PDEs.
Spatial Implicit Neural Representations for Global-Scale Species Mapping
Elijah Cole (California Institute of Technology), Oisin Mac Aodha (University of Edinburgh)
CodeTabularBenchmarkAgriculture Related
π― What it does: This paper utilizes Spatial Implicit Neural Representation (SINR) to jointly estimate the distribution range of approximately 47,000 species globally from a large amount of presence-only records in the iNaturalist dataset.
Spatial-Temporal Graph Learning with Adversarial Contrastive Adaptation
Qianru Zhang (University of Hong Kong), Ruihua Han (University of Hong Kong)
CodeRecommendation SystemAnomaly DetectionGraph Neural NetworkAuto EncoderContrastive LearningGraphTime Series
π― What it does: A self-supervised spatial-temporal graph learning framework named GraphST is proposed to learn regional embeddings under noisy and sparse urban data, which can be used for tasks such as crime prediction, traffic flow prediction, and housing price prediction.
Liu Ziyin (University of Tokyo), Zihao Wang (Hong Kong University of Science and Technology)
CodeCompressionOptimizationConvolutional Neural NetworkSupervised Fine-TuningImageBiomedical Data
π― What it does: A method is proposed to transform the L1 constraint of non-convex objectives into a differentiable problem through reparameterization (Hadamard multiplication), and directly optimize it using standard SGD (or Adam), referred to as the spred method.
SRATTA: Sample Re-ATTribution Attack of Secure Aggregation in Federated Learning.
Tanguy Marchand (Owkin Inc), Arthur Pignet (Owkin Inc)
CodeFederated LearningSafty and PrivacyAdversarial AttackConvolutional Neural NetworkImageMultimodalityBiomedical Data
π― What it does: In cross-device federated learning, the SRATTA attack is proposed, which can recover and classify samples to each client solely by using the aggregated model after secure aggregation, thereby breaking the privacy protection of secure aggregation.
π― What it does: This paper proposes the StableCFR method, which utilizes unified sampling and Ξ΅-greedy matching to upsample minority groups while maintaining covariate balance, thereby achieving robust heterogeneous treatment effect estimation for underrepresented populations.
π― What it does: A new batch active learning algorithm called VeSSAL is proposed, which is suitable for the application of deep neural networks in streaming environments and can immediately query labels when encountering samples.
π― What it does: Proposes STRIDERNET - a reinforcement learning framework based on graph neural networks to optimize atomic structures in rough energy landscapes;
π― What it does: A framework for training personalized models in a decentralized environment is proposedβStructured Cooperative Learning (SCooL), which automatically generates a cooperation graph through graphical model priors and collaboratively updates local models.
π― What it does: The StyleGAN-T model is proposed, which achieves fast and high-quality large-scale text-to-image synthesis by modifying StyleGAN-XL; multiple improvements have been made in the generator, discriminator, and training process, supporting text alignment, controllable diversity, and super-resolution.
π― What it does: This paper proposes a supervised metric learning loss optimized based on contextual similarity, which directly minimizes the contextual similarity between samples during training, thereby improving the ranking quality of image retrieval.
π― What it does: This paper proposes a Symmetry-Aware Robot Design (SARD) framework that generates controllable and efficient robots by searching for and utilizing symmetry during the robot design process.
π― What it does: Using synthetic images instead of traditional validation sets to achieve model selection (including early stopping, random seed selection, and hyperparameter search)
TabLeak: Tabular Data Leakage in Federated Learning
Mark Vero (ETH Zurich), Martin Vechev (ETH Zurich)
CodeFederated LearningSafty and PrivacyTabular
π― What it does: A reconstruction attack method for tabular data in federated learning, called TabLeak, is proposed, which can recover a large amount of private data from gradient updates under different batch sizes and training protocols.
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Research and prove that there exists a very small subset of parameters (about 0.01%) in fine-tuned language models that can retain over 95% of task performance, and propose a method for locating these 'skill' areas through model grafting.
The Benefits of Model-Based Generalization in Reinforcement Learning
Kenny John Young, JΓΌrgen Schmidhuber (International Institute of Applied Systems Analysis)
CodeReinforcement LearningTabular
π― What it does: Through theoretical proof (Theorem 1.1) and large-scale experiments, this paper evaluates and demonstrates the significant improvement in sampling efficiency of reinforcement learning when using learned models to generate imagined experiences in environments with factorized structures.
The Hessian perspective into the Nature of Convolutional Neural Networks
Sidak Pal Singh (ETH Zurich), Bernhard SchΓΆlkopf (MPI for Intelligent Systems)
CodeConvolutional Neural NetworkImage
π― What it does: This paper theoretically analyzes the structure and rank of the Hessian of the loss function of convolutional neural networks (CNNs), revealing the impact of CNN architecture on the Hessian and providing an upper bound.
π― What it does: A new regularization method called Monge gap is proposed for learning optimal transport mappings under arbitrary costs without imposing strict constraints on the network structure.
The Power of Learned Locally Linear Models for Nonlinear Policy Optimization
Daniel Pfrommer (Massachusetts Institute of Technology), Stephen Tu (Google Brain)
CodeOptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningSequential
π― What it does: This paper proposes a learning-based trajectory optimization algorithm that utilizes local linear models to iteratively estimate dynamics and perform iLQR-style updates in nonlinear control.
π― What it does: This paper studies the impact of entropy and reconstruction on mutual information in Multi-View Self-Supervised Learning (MVSSL), proposing and validating an optimization framework based on the ER (Entropy + Reconstruction) lower bound.
Wael Alghamdi (Harvard University), Lalitha Sankar (Arizona State University)
CodeOptimizationSafty and Privacy
π― What it does: This paper proposes a Saddle-Point Accountant (SPA) based on the saddle point method, which can accurately estimate differential privacy parameters in constant time under large compositions (many iterations);
π― What it does: The study investigates the impact of incorporating out-of-domain (OOD) samples into the training data on the generalization error of the target task, finding that the error exhibits a non-monotonic relationship with the number of OOD samples.
The Wisdom of Hindsight Makes Language Models Better Instruction Followers
Tianjun Zhang (University of California), Joseph E. Gonzalez (University of California)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningContrastive LearningText
π― What it does: A two-stage reward-free learning algorithm based on Hindsight Instruction Relabeling (HIR) is proposed, which achieves alignment by having the language model relabel instructions on its own generated outputs.
π― What it does: A non-convex neural network robustness provable method based on low-rank SDP constraints is proposed, overcoming the traditional LP approximation barrier.
Tighter Information-Theoretic Generalization Bounds from Supersamples
Ziqiao Wang (University of Ottawa), Yongyi Mao (University of Ottawa)
CodeContrastive LearningTabular
π― What it does: Under the Supersample setting, various tighter upper bounds on generalization error are proposed using information-theoretic methods such as loss difference, single loss, and Rademacher sequences.
π― What it does: A Parameterized-(DF) framework is proposed, unifying and extending existing GNN models, achieving more flexible graph representations through learning the decomposition (D) and filtering (F) of graph matrices.
π― What it does: A structured learning optimizer based on mathematical conditions is proposed, with the constraint update rule in the form of a preconditioner + bias.
Towards Controlled Data Augmentations for Active Learning
Jianan Yang (Zhejiang University), Junbo Zhao
CodeClassificationData-Centric LearningImage
π― What it does: A proactive learning framework named CAMPAL is proposed, which can enhance the sample selection effectiveness of active learning through a controllable data augmentation strategy.
Sean Kulinski (Purdue University), David I. Inouye (Purdue University)
CodeDomain AdaptationExplainability and InterpretabilityGenerative Adversarial NetworkImageTextTabular
π― What it does: A distribution shift explanation framework based on interpretable transport maps is proposed, providing k-sparse, k-clustering, and interpretable mappings for image adversarial generation.
π― What it does: An efficient and stable adversarial training method for l1 norm perturbations, Fast-EGl 1, is proposed to address the issue of catastrophic overfitting that traditional methods face under l1 perturbations.
π― What it does: The CREST framework is proposed, which improves the sample efficiency and sustainability of deep learning by segmenting non-convex loss into piecewise quadratic approximations and constructing mini-batch coresets in each sub-region, dynamically updated.
π― What it does: A new graph structure noise assessment metric ESNR is proposed, along with a graph rearrangement framework GPS based on self-supervised graph link prediction, aimed at quantifying and reducing the negative impact of graph structure noise on the performance of Graph Neural Networks (GNNs).
π― What it does: TR0N transforms any pre-trained unconditional generative model (such as GANs, VAEs) into a zero-shot conditional generation framework that can generate samples based on any condition (category, text, image semantics) without the need for additional data or fine-tuning.
Tractable Control for Autoregressive Language Generation
Honghua Zhang (University of California), Guy Van den Broeck (University of California)
CodeGenerationOptimizationKnowledge DistillationRecurrent Neural NetworkLarge Language ModelText
π― What it does: The GeLaTo framework is proposed, which combines solvable probabilistic models (such as HMM) with large-scale autoregressive language models to generate text that meets lexical constraints.
Trainability, Expressivity and Interpretability in Gated Neural ODEs
Timothy Doyeon Kim (Princeton University), Kamesh Krishnamurthy (Princeton University)
CodeExplainability and InterpretabilityComputational EfficiencyRecurrent Neural NetworkTime SeriesSequentialOrdinary Differential EquationAudio
π― What it does: This paper proposes and studies the Gated Neural Ordinary Differential Equation (gnODE), exploring its trainability, expressiveness, and interpretability, and conducting experimental validation on various synthetic and real datasets.
Sung Min Park (Massachusetts Institute of Technology), Aleksander Madry (Massachusetts Institute of Technology)
CodeExplainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkTransformerImageText
π― What it does: A data attribution method named TRAK is proposed, which can efficiently trace the sources of model predictions in large-scale non-convex models.
Transcendental Idealism of Planner: Evaluating Perception from Planning Perspective for Autonomous Driving
Weixin Li, Xiaodong Yang (QCraft)
CodeAutonomous DrivingOptimizationPoint Cloud
π― What it does: A perception assessment framework TIP based on a planning perspective is proposed, utilizing the expected utility maximization theory to map perception errors into Hilbert space and quantify their impact on autonomous driving planning decisions.
π― What it does: In datasets with missing values, an unsupervised missing value imputation method based on distribution matching, called TDM, is proposed;
π― What it does: In ELECTRA-style pre-training, the system analyzes the impact of generator capacity on discriminator performance and proposes to control generator training by completely decoupling the optimizers of the generator and discriminator, significantly reducing the model's sensitivity to generator size and improving downstream task performance.