NeurIPS 2023 Papers — Page 33
Conference on Neural Information Processing Systems · 3218 papers
Winner Takes It All: Training Performant RL Populations for Combinatorial Optimization
Nathan Grinsztajn (InstaDeep), Thomas D Barrett
OptimizationTransformerReinforcement LearningTabular
🎯 What it does: A population-based reinforcement learning method called Poppy is proposed to solve NP-hard combinatorial optimization problems by training a set of complementary strategies to improve the quality of solutions during inference.
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model
Zirui Liu (Rice University), Xia Hu (Rice University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A new unbiased sampling matrix multiplication estimation method WTA-CRS is proposed to significantly reduce activation storage during Transformer training while maintaining gradient unbiasedness.
WITRAN: Water-wave Information Transmission and Recurrent Acceleration Network for Long-range Time Series Forecasting
Yuxin Jia (Beijing Jiaotong University), Huaiyu Wan (Beijing Jiaotong University)
Recurrent Neural NetworkTransformerTime Series
🎯 What it does: A new time series long-short term information transmission framework called WIT (Water-wave Information Transmission) is proposed, and based on this, a Recursive Accelerated Network (RAN) is constructed to efficiently capture global/local associations and long/short cycle repetitive semantics for long/ultra-long sequence prediction.
Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations
Piotr Indyk (Massachusetts Institute of Technology), Haike Xu (Massachusetts Institute of Technology)
Point Cloud
🎯 What it does: This paper studies the worst-case performance of graph-based approximate nearest neighbor search algorithms (DiskANN, HNSW, NSG) and provides both theoretical and experimental analyses.
Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis
Alexander Meulemans (ETH Zürich), Greg Wayne
Reinforcement LearningContrastive Learning
🎯 What it does: This paper proposes a model-based long-term credit allocation method called COCOA (Counterfactual Contribution Analysis), which achieves more accurate policy gradient estimation by inferring the contribution of actions to future rewards.
XAGen: 3D Expressive Human Avatars Generation
Zhongcong Xu (National University of Singapore), Mike Zheng Shou (National University of Singapore)
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: Generate 3D human avatars capable of intricate expressions, hand and facial animations in any posture, and achieve independent control over multi-dimensional attributes such as body posture, facial expressions, jaw posture, and hand gestures.
xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data
Jing Gong (BioMap Research), Le Song (BioMap Research)
Representation LearningDrug DiscoveryTransformerBiomedical Data
🎯 What it does: Developed and trained an efficient and scalable pre-trained model for single-cell RNA sequencing, xTrimoGene, which uses a self-supervised regression task to recover sparse gene expression values, and applies this model to downstream tasks such as cell type annotation, perturbation response prediction, and drug combination prediction.
You Only Condense Once: Two Rules for Pruning Condensed Datasets
Yang He (Agency for Science Technology and Research), Joey Tianyi Zhou (Agency for Science Technology and Research)
Data SynthesisCompressionConvolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: The YOCO (You Only Condense Once) method is proposed, which allows for flexible scaling (trimming) of the synthesized dataset based on different computational resource requirements after a single dataset compression, without the need for an additional compression process.
Your representations are in the network: composable and parallel adaptation for large scale models
Yonatan Dukler (AWS AI Labs), Stefano Soatto (AWS AI Labs)
ClassificationRecognitionDomain AdaptationComputational EfficiencyTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes a lightweight cross-attention adapter named InCA, which enables efficient transfer learning for various downstream visual tasks by training only a small number of modules while keeping the base model frozen.
Zero-One Laws of Graph Neural Networks
Sam Adam-Day (University of Oxford), Ismail Ilkan Ceylan (University of Oxford)
Graph Neural NetworkGraph
🎯 What it does: This study proves that graph neural networks (GNNs) follow the zero-one law on large-scale Erdős–Rényi graphs, revealing the limits of their expressiveness and extrapolation capabilities.
Zero-Regret Performative Prediction Under Inequality Constraints
Wenjing Yan (Hong Kong University of Science and Technology), Xuanyu Cao (Hong Kong University of Science and Technology)
OptimizationReinforcement LearningTabular
🎯 What it does: The research addresses the performative prediction problem under inequality constraints, proposing a robust primal-dual framework and designing an adaptive primal-dual algorithm suitable for location family distributions.
Zero-Shot Anomaly Detection via Batch Normalization
Aodong Li (University of California Irvine), Stephan Mandt (University of California Irvine)
Anomaly DetectionMeta LearningContrastive LearningImageTabular
🎯 What it does: A zero-shot anomaly detection method based on batch normalization and meta-training, ACR, is proposed, which can complete anomaly detection under new distributions without retraining.
Zero-shot causal learning
Hamed Nilforoshan (Stanford University), Jure Leskovec (Stanford University)
Meta LearningDrug DiscoveryTabularBiomedical DataElectronic Health Records
🎯 What it does: A zero-shot causal learning framework, CaML, is proposed, which can predict the causal effects of individuals on new interventions without any historical intervention data.
Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Models
Lin Li (Zhejiang University), Long Chen (The Hong Kong University of Science and Technology)
RecognitionObject DetectionComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageChain-of-Thought
🎯 What it does: Proposes the RECODE method, which utilizes large language models to generate composite descriptive prompts (subject, object, space) to improve zero-shot visual relationship detection in CLIP;
Zero-sum Polymatrix Markov Games: Equilibrium Collapse and Efficient Computation of Nash Equilibria
Fivos Kalogiannis (University of California), Ioannis Panageas (University of California)
OptimizationReinforcement Learning from Human FeedbackGraph Neural Network
🎯 What it does: This paper proposes and studies zero-sum multi-player multi-state polymatrix Markov games, proving that their coarse correlated equilibrium (CCE) can collapse to Nash equilibrium (NE), thus achieving a polynomial-time algorithm for ε-approximate NE.
Zeroth-Order Methods for Nondifferentiable, Nonconvex, and Hierarchical Federated Optimization
Yuyang Qiu (Rutgers University), Farzad Yousefian (Rutgers University)
OptimizationFederated LearningImage
🎯 What it does: This paper studies the handling of non-differentiable non-convex objectives in federated learning, as well as optimization problems with bilevel and min-max structures. It proposes a Federated Zeroth-Order (FedRZO) method based on zeroth-order stochastic smoothing, which can solve approximate critical points of Clarke subgradients without relying on differentiability and L-smoothness.
ZipLM: Inference-Aware Structured Pruning of Language Models
Eldar Kurtic (IST Austria), Dan Alistarh (Neural Magic)
CompressionComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes ZipLM—a reasoning-aware structured pruning method that can batch-generate various compressed BERT/GPT models under given reasoning environments and speed-up targets.
ZoomTrack: Target-aware Non-uniform Resizing for Efficient Visual Tracking
Yutong Kou (Chinese Academy of Sciences), Liang Li (Beijing Institute of Basic Medical Sciences)
Object TrackingOptimizationComputational EfficiencyTransformerVideo
🎯 What it does: This paper proposes a visual tracking method called ZoomTrack based on non-uniform scaling, which enhances tracking accuracy while maintaining a small input size and high speed.