NeurIPS 2024 Papers — Page 22
Conference on Neural Information Processing Systems · 4035 papers
LLM Evaluators Recognize and Favor Their Own Generations
Arjun Panickssery (George Washington University), Shi Feng (New York University)
GenerationSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper studies the relationship between self-preference (preferring self-generated text) and self-recognition (detecting one's own text) in large language models (LLMs), demonstrating that self-recognition ability leads to self-preference, and showing a linear causal relationship between the two through fine-tuning; it discusses the potential risks of this phenomenon for model safety.
LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language
James Requeima (University of Toronto), David Duvenaud (University of Toronto)
Large Language ModelPrompt EngineeringTabularTime Series
🎯 What it does: A method called LLM Processes (LLMP) is proposed, which can directly extract numerical prediction distributions from large language models, supporting zero-shot multi-dimensional regression and text conditioning.
LLM-AutoDA: Large Language Model-Driven Automatic Data Augmentation for Long-tailed Problems
Pengkun Wang (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
Data-Centric LearningTransformerLarge Language ModelReinforcement LearningPrompt EngineeringImage
🎯 What it does: This paper proposes an automated long-tail data augmentation framework based on large language models, LLM-AutoDA, which automatically searches for and generates optimal augmentation strategies for long-tail distributions.
LLM-based Skill Diffusion for Zero-shot Policy Adaptation
Woo Kyung Kim (Sunkyunkwan University), Honguk Woo (Sunkyunkwan University)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelDiffusion modelMultimodalitySequentialChain-of-Thought
🎯 What it does: A LLM-driven Skill Diffusion framework (LDuS) is proposed, achieving zero-shot adaptation of skill strategies in the context of language descriptions.
LLM-Check: Investigating Detection of Hallucinations in Large Language Models
Gaurang Sriramanan (University of Maryland), Soheil Feizi (University of Maryland)
GenerationAnomaly DetectionComputational EfficiencyTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a method called LLM-Check that can detect hallucinations in single generated texts from large language models without training, multi-round generation, or external retrieval.
LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
Qidong Liu (Xi'an Jiaotong University), Xiangyu Zhao (City University of Hong Kong)
Recommendation SystemRecurrent Neural NetworkTransformerLarge Language ModelSequentialRetrieval-Augmented Generation
🎯 What it does: This paper proposes an enhanced framework using large language model (LLM) embeddings (LLM-ESR) that improves the performance of sequential recommendation systems for long-tail users and long-tail items through dual-view modeling and retrieval-augmented self-distillation.
LLMDFA: Analyzing Dataflow in Code with Large Language Models
Chengpeng Wang (Purdue University), Xiangyu Zhang (Purdue University)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: A customizable data flow analysis framework LLMDFA is implemented using large language models (LLM) without the need for compilation;
LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings
Duo Wang (Beihang University), Junjie Wu (Beihang University)
ClassificationRepresentation LearningGraph Neural NetworkLarge Language ModelContrastive LearningGraph
🎯 What it does: The TEA-GLM framework is proposed, utilizing LLM as a zero-shot graph learner across datasets and tasks, aligning GNN representations with LLM token embeddings, and generating a fixed number of graph token embeddings through linear projection, combined with unified instructions to complete graph tasks such as node classification and link prediction.
LLMs Can Evolve Continually on Modality for $\mathbb{X}$-Modal Reasoning
Jiazuo Yu (Dalian University of Technology), Long Chen (The Hong Kong University of Science and Technology)
Large Language ModelMixture of ExpertsImageVideoMultimodalityPoint CloudBenchmarkAudio
🎯 What it does: A flexible and scalable framework named PathWeave is proposed, enabling multimodal large language models (MLLMs) to continuously evolve in multimodal reasoning.
LM-HT SNN: Enhancing the Performance of SNN to ANN Counterpart through Learnable Multi-hierarchical Threshold Model
Zecheng Hao (Peking University), Tiejun Huang (Peking University)
ClassificationSpiking Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A learnable multi-layer isochronous threshold model (LM-HT) is proposed, achieving performance comparable to quantized ANN under extremely low latency through global time-weighted membrane potential and learnable thresholds, and providing a scheme for lossless conversion to a single-threshold LIF model.
Local and Adaptive Mirror Descents in Extensive-Form Games
Côme Fiegel (CREST FairPlay ENSAE Paris), Michal Valko (INRIA)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes a Local Online Mirror Descent (LocalOMD) algorithm under a fixed sampling framework for learning approximately optimal strategies in zero-sum games with incomplete information.
Local Anti-Concentration Class: Logarithmic Regret for Greedy Linear Contextual Bandit
Seok-Jin Kim (Columbia University), Min-hwan Oh (Seoul National University)
Reinforcement Learning
🎯 What it does: This paper studies the performance of the pure greedy strategy (LinGreedy) in linear contextual bandits and proposes a new local anti-concentration (LAC) condition to ensure its effectiveness.
Local Curvature Smoothing with Stein's Identity for Efficient Score Matching
GENKI OSADA (LY Corporation), Takashi Nishide (University of Tsukuba)
GenerationData SynthesisComputational EfficiencyScore-based ModelImageStochastic Differential Equation
🎯 What it does: A local curvature smoothing score matching method (LCSS) utilizing Stein identification is proposed for efficient training of high-dimensional data.
Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPs
Davide Maran (Politecnico di Milano), Marcello Restelli (Politecnico di Milano)
Reinforcement Learning
🎯 What it does: This paper introduces a new class of representations called Locally Linearizable MDPs and designs the CINDERELLA algorithm to achieve sub-linear regret upper bounds for no-regret learning in continuous state-action spaces.
Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning
Minghui Chen (University of British Columbia), Xiaoxiao Li (University of British Columbia)
Federated LearningTransformerSupervised Fine-TuningImage
🎯 What it does: In federated learning, pre-trained models are utilized to reduce communication rounds and enhance global model performance through Local Superior Soups (LSS) via local model interpolation.
Local to Global: Learning Dynamics and Effect of Initialization for Transformers
Ashok Vardhan Makkuva (École Polytechnique Fédérale de Lausanne), Chanakya Ekbote (École Polytechnique Fédérale de Lausanne)
OptimizationTransformerSequential
🎯 What it does: This paper studies the learning dynamics of a single-layer Transformer on first-order Markov chain data, clarifying how initialization and data switching factors determine convergence to global or local minima.
Localize, Understand, Collaborate: Semantic-Aware Dragging via Intention Reasoner
Xing Cui (Beijing University of Posts and Telecommunications), Zhaofeng He (University of California Santa Barbara)
Image TranslationLarge Language ModelVision Language ModelDiffusion modelImageBenchmark
🎯 What it does: This paper proposes a drag-and-drop editing framework named LucidDrag, which enables semantic-aware and controllable image editing based on user-specified drag points in images.
Localized Adaptive Risk Control
Matteo Zecchin (King's College London), Osvaldo Simeone (King's College London)
SegmentationOptimizationImageTime Series
🎯 What it does: Proposed and implemented Localized Adaptive Risk Control (L-ARC), an online calibration method that achieves local risk control of the prediction set by learning variable threshold functions in RKHS.
Localized Zeroth-Order Prompt Optimization
Wenyang Hu (National University of Singapore), Bryan Kian Hsiang Low (National University of Singapore)
OptimizationLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a prompt optimization algorithm ZOPO based on local zero-order optimization, aimed at efficiently searching for locally optimal prompts on black-box LLMs;
Localizing Memorization in SSL Vision Encoders
Wenhao Wang (CISPA Helmholtz Center for Information Security), Franziska Boenisch (CISPA Helmholtz Center for Information Security)
Representation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: Two new measurement methods, LayerMem and UnitMem, are proposed to locate the model's memory of training samples at the layer and unit levels in self-supervised visual encoders.
Locally Private and Robust Multi-Armed Bandits
Xingyu Zhou (Wayne State University), WEI ZHANG
OptimizationSafty and PrivacyReinforcement Learning
🎯 What it does: This paper studies the interaction between Local Differential Privacy (LDP) and robustness (Huber corruption and heavy-tailed rewards) in Multi-Armed Bandits (MAB). It provides high-probability mean estimation error, optimal convergence rates for online MAB, and optimal suboptimality for offline MAB, covering LDP-then-Corruption (LTC), Corruption-then-LDP (CTL), and the most practical C-LDP-C scenario.
Locating What You Need: Towards Adapting Diffusion Models to OOD Concepts In-the-Wild
Jianan Yang (Zhejiang University), Haobo Wang (Zhejiang University)
GenerationData SynthesisDomain AdaptationDiffusion modelImage
🎯 What it does: Through the active learning framework CATOD, high-quality out-of-distribution (OOD) concept images are automatically selected and the adapter is gradually trained, achieving high-fidelity generation of text-to-image models on unknown concepts.
LocCa: Visual Pretraining with Location-aware Captioners
Bo Wan (Google DeepMind), Xiaohua Zhai (Google DeepMind)
Object DetectionSegmentationGenerationTransformerVision Language ModelImageVideoText
🎯 What it does: Using a multi-task generative Encoder-Decoder framework, visual pre-training is performed on images, incorporating position-aware tasks (Automatic Referring Expression for AREF and Grounded Captioning for GCAP), enabling a single model to simultaneously learn global descriptions and local localization.
LoCo: Learning 3D Location-Consistent Image Features with a Memory-Efficient Ranking Loss
Dominik Kloepfer, Dylan Campbell (Australian National University)
SegmentationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: Trained an image feature extractor that maintains similar corresponding features in the same 3D position from different perspectives.
LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment
Juelin Zhu (National University of Defense Technology), Maojun Zhang (National University of Defense Technology)
Pose EstimationOptimizationSafty and PrivacyConvolutional Neural NetworkSimultaneous Localization and MappingImage
🎯 What it does: This paper proposes a drone visual localization method called LoD-Loc, which is based on LoD 3D maps and neural wireframe alignment. It can estimate the complete 6-DoF pose in an end-to-end manner with only rough localization priors.
LoFiT: Localized Fine-tuning on LLM Representations
Fangcong Yin (University of Texas at Austin), Greg Durrett (University of Texas at Austin)
OptimizationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A local fine-tuning method named LOFIT is proposed, which first locates the most important attention heads for specific tasks through learning scaling factors, and then learns the bias vectors of the representations of these heads, achieving efficient tuning of large language models;
Log-concave Sampling from a Convex Body with a Barrier: a Robust and Unified Dikin Walk
Yuzhou Gu (New York University), Lichen Zhang (Massachusetts Institute of Technology)
Optimization
🎯 What it does: This paper proposes a robust Dikin walk sampling framework that can efficiently sample from any L-Lipschitz log-concave distribution on convex bodies.
Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning
Otmane Sakhi (Criteo AI Lab), Nicolas Chopin (CREST)
OptimizationReinforcement LearningTabular
🎯 What it does: A logarithmic smoothing-based IPS estimator (Logarithmic Smoothing, LS) is proposed, and lazy lower bound estimation, policy selection, and learning are implemented in an offline context;
Logical characterizations of recurrent graph neural networks with reals and floats
Veeti Ahvonen (Tampere University), Carsten Lutz (Leipzig University)
Graph Neural Network
🎯 What it does: This paper systematically characterizes the expressive power of recurrent graph neural networks (GNNs) in the realms of real numbers and floating-point numbers using logical methods, providing equivalence with modal logic (GMSC and ω-GML).
Loki: Low-rank Keys for Efficient Sparse Attention
Prajwal Singhania (University of Maryland), Abhinav Bhatele (University of Maryland)
TransformerLarge Language ModelText
🎯 What it does: Loki is proposed, a sparse attention method that utilizes low-dimensional structures of attention keys;
Long-form factuality in large language models
Jerry Wei (Google DeepMind), Quoc V Le
TransformerLarge Language ModelAgentic AITextRetrieval-Augmented Generation
🎯 What it does: The LongFact dataset and SAFE evaluation method are proposed to measure the factual accuracy of large language models in long texts.
Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments
Siddharth Nayak (Massachusetts Institute of Technology), Hamsa Balakrishnan (Massachusetts Institute of Technology)
Robotic IntelligenceTransformerLarge Language ModelVision Language ModelMultimodalityBenchmark
🎯 What it does: A multi-agent long-term planning framework based on language models, LLaMAR, is proposed, which can dynamically plan, correct, and verify subtask execution in partially observable environments.
Long-range Brain Graph Transformer
Shuo Yu (Dalian University of Technology), Feng Xia (RMIT University)
ClassificationGraph Neural NetworkTransformerGraphBiomedical DataMagnetic Resonance ImagingAlzheimer's Disease
🎯 What it does: This paper proposes a brain map Transformer named ALTER, specifically designed to capture long-distance dependencies between brain regions and integrate them with short-distance information, enhancing the diagnostic performance of neurological diseases.
Long-Range Feedback Spiking Network Captures Dynamic and Static Representations of the Visual Cortex under Movie Stimuli
Liwei Huang (Peking University), Yonghong Tian (Peking University)
Representation LearningSpiking Neural NetworkVideo
🎯 What it does: This study investigates the dynamic and static representations of the visual cortex under natural movie stimuli and proposes a long-range feedback pulse neural network (LoRaFB-SNet) that simulates top-down feedback and pulse mechanisms to more closely align with the neural activity of the mouse visual cortex.
Long-range Meta-path Search on Large-scale Heterogeneous Graphs
Chao Li (Huazhong University of Science and Technology), Kun He (Huazhong University of Science and Technology)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes an automated framework for searching long-range meta-paths, called LMSPS, which utilizes evolutionary sampling and sampling evaluation to select effective meta-paths on large-scale heterogeneous graphs, thereby enhancing the effectiveness of graph representation learning.
Long-tailed Object Detection Pretraining: Dynamic Rebalancing Contrastive Learning with Dual Reconstruction
Chen-Long Duan (Nanjing University of Science and Technology), Lin Zhao (Nanjing University of Science and Technology)
Object DetectionAuto EncoderContrastive LearningImage
🎯 What it does: This paper proposes a pre-training framework for long-tail object detection called 2DRCL, which integrates global-local contrastive learning, dynamic rebalancing sampling, and a dual reconstruction mechanism;
Long-Tailed Out-of-Distribution Detection via Normalized Outlier Distribution Adaptation
Wenjun Miao (Beihang University), Xiao Bai (Beihang University)
Domain AdaptationAnomaly DetectionImage
🎯 What it does: Proposes the AdaptOD method to address the distribution shift problem in OOD detection under long-tail scenarios;
Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering
Jie Ma (Xi'an Jiaotong University), Youtian Du (Xi'an Jiaotong University)
ClassificationRecognitionData-Centric LearningConvolutional Neural NetworkVision Language ModelVideoMultimodalityAudio
🎯 What it does: A new dataset for audio-visual question answering (AVQA) called MUSIC-AVQA-R is proposed, along with a multi-view cyclic collaborative debiasing strategy (MCCD) designed to enhance the robustness of the model.
Lookback Prophet Inequalities
Ziyad Benomar (ENSAE Ecole Polytechnique), Vianney Perchet (CREST ENSAE)
🎯 What it does: This paper studies the Prophet Inequality that allows for revisiting rejected candidates, proposing a general model characterized by a decay function, and reducing it to a γ-Profit Inequality that only contains the constant γ.
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate
Anthony Fuller (Carleton University), James R Green
Image TranslationSegmentationTransformerImage
🎯 What it does: A LookHere positional encoding based on a 2D directional attention mask is proposed, enabling the Vision Transformer to directly infer on higher resolution images without fine-tuning.
Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models
Regev Cohen (Verily AI), Daniel Freedman
RestorationSuper ResolutionDiffusion modelImage
🎯 What it does: This paper conducts a theoretical analysis of the 'uncertainty-perception' trade-off in generative image restoration using information theory methods, and validates its effectiveness in super-resolution and image inpainting tasks.
LoQT: Low-Rank Adapters for Quantized Pretraining
Sebastian Bugge Loeschcke (University of Copenhagen), Vésteinn Snæbjarnarson (University of Copenhagen)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The LoQT method is proposed, which combines low-rank adapters with 4-bit NF4 quantization, enabling efficient pre-training and fine-tuning of large models on a single 24GB GPU, significantly reducing memory usage.
LoRA-GA: Low-Rank Adaptation with Gradient Approximation
Shaowen Wang (Tsinghua University), Jian Li (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes LoRA-GA, a non-zero initialization method under the LoRA framework that accelerates convergence and improves performance through gradient approximation.
LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search
Elias Jääsaari (University of Helsinki), Teemu Roos (University of Helsinki)
RetrievalComputational EfficiencyImageTextBenchmark
🎯 What it does: A score calculation method based on Reduced-Rank Regression (RRR) is proposed, improving the query speed and memory usage of clustering-based Approximate Nearest Neighbor (ANN) search, and implementing the open-source library LoRANN.
Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics
Jonas Spinner (Heidelberg University), Johann Brehmer (Qualcomm AI Research)
ClassificationGenerationData SynthesisTransformerFlow-based ModelTabularSequentialPhysics Related
🎯 What it does: A new multipurpose high-energy physics network architecture is proposed—Lorentz-Equivariant Geometric Algebra Transformer (L-GATr), which achieves Lorentz invariance and can handle variable-length sets of particles; based on this, the first Lorentz-invariant continuous normalizing flow generative model is constructed, trained using Riemannian flow matching;
Loss Landscape Characterization of Neural Networks without Over-Parametrization
Rustem Islamov (University of Basel), Aurelien Lucchi (Max Planck Institute for Intelligent Systems)
OptimizationImageMagnetic Resonance Imaging
🎯 What it does: A new αβ-condition is proposed to characterize the loss landscape of deep networks, accommodating saddle points and local minima, and providing convergence guarantees without the need for large-scale over-parameterization.
LoTLIP: Improving Language-Image Pre-training for Long Text Understanding
Wei Wu (University of Science and Technology of China), Zheng-Jun Zha (University of Science and Technology of China)
RetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: To address the issue of language-image pre-training models performing poorly in understanding long texts, this paper re-labels 100M image data with long descriptions and incorporates corner tokens during the pre-training phase to enhance text representation, thereby achieving dual understanding of both long and short texts.
LOVA3: Learning to Visual Question Answering, Asking and Assessment
Hengyuan Zhao, Mike Zheng Shou (National University of Singapore)
TransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the LOVA 3 framework, which trains multimodal large language models to have the ability to answer, ask questions, and evaluate visual questions.
Low Degree Hardness for Broadcasting on Trees
Han Huang (University of Missouri), Elchanan Mossel (Massachusetts Institute of Technology)
🎯 What it does: This paper studies the low-degree polynomial difficulty of the broadcasting on trees problem, proving that below the Kesten-Stigum threshold, any low-degree polynomial is almost uncorrelated with the root state, thus providing a lower bound for algorithms at this threshold.
Low Precision Local Training is Enough for Federated Learning
Zhiwei Li (Fudan University), WEIZHONG ZHANG
Federated LearningComputational EfficiencyImage
🎯 What it does: In federated learning, the authors propose an efficient paradigm: using low precision (e.g., 8-bit) operations and quantization for local training on the client side, and uploading the low precision model to the server; the server only uses high precision computation during aggregation and recovers the high precision global model through a moving average method.
Low-Rank Optimal Transport through Factor Relaxation with Latent Coupling
Peter Halmos (Princeton University), Benjamin Raphael
OptimizationComputational EfficiencyTabularBiomedical Data
🎯 What it does: This paper proposes a low-rank optimal transport algorithm called FRLC based on implicit coupling (LC) decomposition, which can directly solve various objectives such as Wasserstein, Gromov-Wasserstein, and Fused Gromov-Wasserstein, and supports balanced, semi-relaxed, and unbalanced marginal constraints.
Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks
Dmitry Kovalev (Yandex Research), Dmitrii Feoktistov (Moscow State University)
Optimization
🎯 What it does: An optimal algorithm for non-smooth convex decentralized optimization problems under time-varying networks is proposed, along with a matching lower bound, filling a theoretical gap in this field.
Lower Bounds of Uniform Stability in Gradient-Based Bilevel Algorithms for Hyperparameter Optimization
Rongzhen Wang (Renmin University of China), Chongxuan Li (Renmin University of China)
OptimizationHyperparameter Search
🎯 What it does: This paper proposes a stability lower bound analysis method for gradient-based two-layer hyperparameter optimization algorithms and constructs a quadratic example to validate the compactness of the theory.
LP-3DGS: Learning to Prune 3D Gaussian Splatting
Zhaoliang Zhang (Johns Hopkins University), Deliang Fan (Arizona State University)
CompressionOptimizationGaussian SplattingPoint Cloud
🎯 What it does: A learnable binary mask is introduced into the 3D Gaussian Splatting model to automatically find the optimal cropping ratio, improving compression rates without significantly reducing rendering quality.
LRM-Zero: Training Large Reconstruction Models with Synthesized Data
Desai Xie (Adobe Research), Hao Tan (Adobe Research)
RestorationGenerationData SynthesisTransformerGaussian SplattingPoint Cloud
🎯 What it does: This paper presents LRM-Zero, a sparse-view 3D reconstruction model trained solely on the non-semantic synthetic data Zeroverse.
LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
Xiaonan Nie (Peking University), Bin CUI
OptimizationComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsImageText
🎯 What it does: Proposes the LSH-MoE framework, which utilizes local sensitive hashing to compress the all-to-all communication volume in MoE training, reducing communication overhead while maintaining model accuracy.
LT-Defense: Searching-free Backdoor Defense via Exploiting the Long-tailed Effect
Yixiao Xu (Beijing University of Posts and Telecommunications), Zhihong Tian (Guangzhou University)
Anomaly DetectionTransformerLarge Language ModelText
🎯 What it does: A search-free backdoor defense method called LT-Defense is proposed, which utilizes the long-tail effect generated by backdoors to detect backdoors in NLP models on clean samples.
Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models
Yang Jiao (Meituan), Yu-Gang Jiang (Fudan University)
Object DetectionSegmentationPose EstimationTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: A new large-scale multimodal model called Lumen is proposed, which can achieve various vision-centric tasks (object detection, instance segmentation, pose estimation, visual localization, and referential segmentation) while maintaining general language interaction capabilities.
Lumina-Next : Making Lumina-T2X Stronger and Faster with Next-DiT
Le Zhuo (Shanghai AI Laboratory), Peng Gao (Shanghai AI Laboratory)
GenerationData SynthesisTransformerDiffusion modelImageVideoMultimodalityPoint CloudOrdinary Differential EquationAudio
🎯 What it does: Based on Lumina-T2X, Lumina-Next is proposed, improving architecture, context extrapolation, and sampling techniques to achieve stronger and faster zero-copy text-to-image, multimodal, and multilingual generation.
LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes
Zefan Qu (City University of Hong Kong), Rynson W. H. Lau (City University of Hong Kong)
RestorationNeural Radiance FieldImage
🎯 What it does: The LuSh-NeRF model is proposed to recover clear and normally lit NeRF scenes from handheld low-light images with motion blur.
M$^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation
Mingshuang Luo (Chinese Academy of Sciences), Shiguang Shan (Chinese Academy of Sciences)
GenerationData SynthesisPose EstimationTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityAudio
🎯 What it does: A unified multimodal multitask framework M3 GPT is proposed, capable of simultaneously handling understanding and generation tasks for text, music, and actions.
MAC Advice for facility location mechanism design
Zohar Barak (Tel Aviv University), Inbal Talgam-Cohen (Tel Aviv University)
Optimization
🎯 What it does: The research focuses on the design of facility location mechanisms under the 'Majority Approximate Correct (MAC)' prediction model, proposing a provably mechanism that can withstand erroneous predictions. Deterministic and randomized algorithms are designed for single facility, β-balanced multi-facility, and unbalanced dual facility scenarios.
MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems
Bin Lei (University of Minnesota), Caiwen Ding (University of Minnesota)
TransformerLarge Language ModelAgentic AIPrompt EngineeringTextChain-of-Thought
🎯 What it does: A multi-agent conditional mining (MACM) prompting framework is proposed and implemented to enhance the inference capability of large language models in complex mathematical reasoning tasks.
MADiff: Offline Multi-agent Learning with Diffusion Models
Zhengbang Zhu (Shanghai Jiao Tong University), Weinan Zhang (Shanghai Jiao Tong University)
TransformerReinforcement LearningDiffusion modelSequential
🎯 What it does: This paper proposes an offline multi-agent learning framework MADIFF based on an attention diffusion model.
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
Wei Tao (Fudan University), Yu Cheng (Chinese University of Hong Kong)
AI Code AssistantTransformerLarge Language ModelAgentic AIPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: A multi-agent framework named MAGIS is proposed, utilizing LLM (based on GPT-4) to automatically locate, plan, write, and review code in GitHub repositories to address issues.
MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization
Orevaoghene Ahia (University of Washington), Noah A. Smith (Allen Institute for AI)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes MAGNET, a multilingual byte-level language model that achieves fair tokenization through a language script-specific boundary predictor.
Magnet: We Never Know How Text-to-Image Diffusion Models Work, Until We Learn How Vision-Language Models Function
Chenyi Zhuang (Nanjing University of Aeronautics and Astronautics), Pan Gao (Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education)
GenerationTransformerVision Language ModelDiffusion modelImageText
🎯 What it does: This study investigates the impact of the CLIP text encoder on attribute binding and proposes a zero-training Magnet method to correct text embeddings, enhancing Stable Diffusion's attribute binding and image quality under complex prompts.
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
Aozhong Zhang (University at Albany), Penghang Yin (University at Albany)
OptimizationTransformerLarge Language ModelText
🎯 What it does: A preprocessing method called MagR based on α-∞ regularization is proposed to reduce the maximum magnitude of pre-trained LLM weights, thereby improving the effectiveness of post-training quantization (PTQ).
Maia-2: A Unified Model for Human-AI Alignment in Chess
Zhenwei Tang (University of Toronto), Ashton Anderson (University of Toronto)
TransformerReinforcement LearningTabular
🎯 What it does: A unified model, Maia-2, is proposed to capture the decision-making behavior of chess players at different skill levels and achieve higher action matching accuracy and coherence.
Make Continual Learning Stronger via C-Flat
Ang Bian (Sichuan University), Tao Feng (Tsinghua University)
OptimizationImage
🎯 What it does: A pluggable C-Flat optimizer is proposed, which combines zero-order and first-order loss surface flattening to enhance the generalization and anti-forgetting capabilities of continuous learning models.
Make Your LLM Fully Utilize the Context
Shengnan An (Xi'an Jiaotong University), Weizhu Chen (Microsoft)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A novel data-driven training method called IN2 (Information-Intensive Training) is proposed, which combines long-context question-answering data to enable the model to learn that key information can be present at any position, thereby alleviating the 'intermediate information loss' problem in long-context models.
Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusion
Yongyuan Liang (Shanghai Qi Zhi Institute), Huazhe Xu (University of California)
Robotic IntelligenceReinforcement LearningDiffusion modelAuto EncoderContrastive LearningSequential
🎯 What it does: A method is proposed to generate control strategies in the policy parameter space using a single behavior demonstration as a condition through a conditional diffusion model (Make-An-Agent).
Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials
Ye Fang (Fudan University), Dahua Lin (Chinese University of Hong Kong)
GenerationRetrievalTransformerLarge Language ModelPrompt EngineeringMultimodalityMesh
🎯 What it does: The research proposes the Make-it-Real framework, which utilizes GPT-4V for material recognition and matching of 3D models containing only albedo, and generates complete SVBRDF PBR maps, significantly enhancing the realism of the models.
Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning
Qi Wang (Shanghai Jiao Tong University), Xiaokang Yang (Shanghai Jiao Tong University)
Representation LearningReinforcement LearningWorld ModelImage
🎯 What it does: The CoWorld method is proposed, utilizing an online simulator as an auxiliary domain to enhance representation learning and value estimation in offline visual reinforcement learning through cross-domain state alignment, reward alignment, and min-max value constraints.
MALT Powers Up Adversarial Attacks
Odelia Melamed (Weizmann Institute of Science), Adi Shamir (Weizmann Institute of Science)
Adversarial AttackImage
🎯 What it does: A new targeted attack method called MALT is designed, which selects the target class using gradient information and an approximate linear assumption on multi-class classifiers, and achieves efficient attacks by combining with APGD.
MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection
Haoyang He (Zhejiang University), Lei Xie (Zhejiang University)
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: A multi-class unsupervised anomaly detection framework called MambaAD based on the Mamba state space model has been developed, utilizing a pre-trained CNN encoder and a multi-scale Mamba decoder to achieve image and pixel-level anomaly detection.
MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space
Jiangwei Weng (Nanjing University of Science and Technology), Jun Li (Nanjing University of Science and Technology)
RestorationObject DetectionConvolutional Neural NetworkImage
🎯 What it does: A low-light image enhancement method named MambaLLIE is proposed, based on a global-then-local state space structure.
MambaLRP: Explaining Selective State Space Sequence Models
Farnoush Rezaei Jafari (Technische Universitat Berlin), Oliver Eberle (Technische Universitat Berlin)
Explainability and InterpretabilityTransformerImageText
🎯 What it does: This paper proposes a Layer-wise Relevance Propagation method for the Mamba architecture (MambaLRP) to generate interpretable and reliable explanations for sequence model predictions.
MambaSCI: Efficient Mamba-UNet for Quad-Bayer Patterned Video Snapshot Compressive Imaging
Zhenghao Pan (Harbin Institute of Technology), Yong Xu (Harbin Institute of Technology)
RestorationCompressionConvolutional Neural NetworkVideo
🎯 What it does: This paper proposes a video compression imaging (SCI) reconstruction method called MambaSCI for the fourfold Bayer pattern, addressing the color distortion and desaturation issues caused by traditional Bayer schemes under fourfold Bayer sensors.
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models
Zunnan Xu (Tsinghua University), Xiu Li (Tsinghua University)
GenerationData SynthesisPose EstimationComputational EfficiencyConvolutional Neural NetworkTransformerAuto EncoderMultimodalityTime SeriesSequentialAudio
🎯 What it does: A two-stage complete body gesture synthesis framework called MambaTalk is proposed based on a selective state space model (Mamba).
MambaTree: Tree Topology is All You Need in State Space Model
Yicheng Xiao (Tsinghua University), Ying Shan (Tencent)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkTransformerSupervised Fine-TuningImageTextMultimodality
🎯 What it does: A tree state space model (Tree SSM) and the MambaTree framework are proposed, utilizing input-aware tree topology to achieve feature propagation, thereby breaking the sequential limitations of traditional SSMs and enhancing the modeling capability for long-range dependencies.
MAmmoTH2: Scaling Instructions from the Web
Xiang Yue (Carnegie Mellon University), Wenhu Chen (University of Waterloo)
TransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: This paper mined 10 million high-quality question-answer pairs from public web corpora without manual annotation through a three-step pipeline (recall, extraction, refinement), constructing the WEBINSTRUCT dataset, and trained the MAmmoTH2 series models based on this dataset.
ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation
Cédric Rommel (Valeo), Eduardo Valle (Recod.ai Lab)
Pose EstimationImage
🎯 What it does: A multi-hypothesis 3D human pose reconstruction model called ManiPose is proposed, based on pose manifold constraints, to address the pose inconsistency problem caused by monocular depth ambiguity.
MaNo: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts
RENCHUNZI XIE, Bo An (Skywork AI)
Domain AdaptationImage
🎯 What it does: An unsupervised model accuracy estimation method called MANO is proposed, which measures the performance of OOV samples using the Lp norm of the model output matrix.
Many-Shot In-Context Learning
Rishabh Agarwal (Google DeepMind), Hugo Larochelle (Google DeepMind)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmarkFinance RelatedChain-of-Thought
🎯 What it does: A systematic evaluation of In-Context Learning (ICL) from few-shot to many-shot under a large-scale context window (up to 1M tokens) was conducted across various tasks (translation, summarization, planning, question answering, mathematical reasoning, etc.), and methods of Reinforced ICL and Unsupervised ICL were proposed that do not require manually reasoned text.
Many-shot Jailbreaking
Cem Anil (Anthropic), David Duvenaud
Adversarial AttackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes and evaluates a Many-shot Jailbreaking (MSJ) attack using long context windows, demonstrating its ability to induce harmful responses from models across different tasks.
Marginal Causal Flows for Validation and Inference
Daniel de Vassimon Manela (University of Oxford), Robin J. Evans (University of Oxford)
Flow-based ModelTabular
🎯 What it does: This paper proposes Frugal Flows, a model based on regularized flows that can directly learn from observational data and infer marginal causal effects while generating synthetic data that meets user-specified causal margins (such as ATE, risk difference, risk ratio, or odds ratio).
Markov Equivalence and Consistency in Differentiable Structure Learning
Chang Deng (University of Chicago), Bryon Aragam (Carnegie Mellon University)
Graph
🎯 What it does: This paper proposes a new differentiable structure learning scoring function that combines log-likelihood with non-convex regularization (MCP);
Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flows
Alberto Cabezas (Lancaster University), Christopher Nemeth (Lancaster University)
OptimizationComputational EfficiencyFlow-based ModelOrdinary Differential Equation
🎯 What it does: A new Markov Chain Monte Carlo (MCMC) sampling method based on Continuous Normalizing Flows (CNF) is proposed, called Markovian Flow Matching (MFM), which accelerates sampling by adaptively training the CNF during the sampling process and combining local gradient proposals with globally flow-driven proposals.
Marrying Causal Representation Learning with Dynamical Systems for Science
Dingling Yao (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
Representation LearningContrastive LearningTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper combines causal representation learning (CRL) with dynamic system identification to propose an identifiable neural simulator (identifier). By transforming the parameter estimation problem into a latent variable identification problem in CRL, it achieves identifiable and efficient prediction of specific parameters for time-invariant trajectories.
Masked Hard-Attention Transformers Recognize Exactly the Star-Free Languages
Andy Yang (University of Notre Dame), Dana Angluin (Yale University)
Transformer
🎯 What it does: This paper provides a precise characterization of the expressive power of Transformers using hard attention and attention masking, proving that they can recognize star-free languages, and explores the impact of positional embeddings, strict/non-strict masking, and network depth on expressive capability.
Masked Pre-training Enables Universal Zero-shot Denoiser
Xiaoxiao Ma (University of Science and Technology of China), Huaian Chen (University of Science and Technology of China)
RestorationConvolutional Neural NetworkImageMagnetic Resonance Imaging
🎯 What it does: A zero-shot image denoising framework based on mask pre-training and iterative filling (MPI) is proposed, which utilizes the knowledge of general image distribution obtained from training with random pixel masks on ImageNet, and achieves high-quality denoising through iterative filling on a single noisy image.
MaskFactory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation
Haotian Qian (Zhejiang University), Deng-Ping Fan (Nankai University)
SegmentationGenerationData SynthesisDiffusion modelImage
🎯 What it does: Proposes the MaskFactory two-stage framework, which uses rigid and non-rigid mask editing and multi-condition control to generate a high-quality binary segmentation dataset.
MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
Gongfan Fang (National University of Singapore), Xinchao Wang (National University of Singapore)
TransformerLarge Language ModelText
🎯 What it does: By learning differentiable semi-structured (N:M) sparse masks, sparsification is achieved on large language models (LLMs) while retaining the performance of the original model;
Matching the Statistical Query Lower Bound for $k$-Sparse Parity Problems with Sign Stochastic Gradient Descent
Yiwen Kou (University of California), Sham M. Kakade (Harvard University)
OptimizationTabularStochastic Differential Equation
🎯 What it does: Using a two-layer fully connected neural network and thresholded sign SGD, we learned the k-sparse parity function distributed on a d-dimensional hypercube, achieving a sample complexity of ˜O(d^{k-1}).
MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
Zichun Yu (Carnegie Mellon University), Chenyan Xiong (Carnegie Mellon University)
TransformerLarge Language ModelText
🎯 What it does: A model-aware data selection framework MATES is proposed, which dynamically captures the model's preferences during pre-training and selects the most beneficial data based on data influence;
MatFormer: Nested Transformer for Elastic Inference
Fnu Devvrit, Prateek Jain (Google DeepMind)
RetrievalOptimizationComputational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: This paper proposes a nested Transformer structure called MatFormer, which can extract hundreds of sub-models of varying scales and similar accuracy for free within a single trained general model, enabling flexible inference.
Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods
Yihan Zhang (Institute of Science and Technology Austria), Marco Mondelli
RestorationOptimizationTabular
🎯 What it does: The study addresses the rank-1 matrix denoising problem under double heteroscedastic noise, with the goal of estimating the signal's singular vector.
MatrixNet: Learning over symmetry groups using learned group representations
Lucas Laird (Northeastern University), Robin Walters (Northeastern University)
Recurrent Neural NetworkTransformerSequential
🎯 What it does: Proposes the MatrixNet structure, which learns matrix representations of group elements to complete group-related prediction tasks.
Matryoshka Query Transformer for Large Vision-Language Models
Wenbo Hu (University of California), Kai-Wei Chang (University of California)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: This work proposes the Matryoshka Query Transformer (MQT), which enables flexible adjustment of the number of visual tokens during training and inference in large-scale visual language models.