NeurIPS 2024 Papers — Page 24
Conference on Neural Information Processing Systems · 4035 papers
Model Fusion through Bayesian Optimization in Language Model Fine-Tuning
Chaeyun Jang (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
OptimizationHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a model fusion method based on Bayesian optimization (BOMF), which finely tunes model fusion weights and hyperparameters through multi-objective Bayesian optimization to enhance the performance of pre-trained language models on downstream tasks.
Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks
Jiacong Hu (Zhejiang University), Mingli Song (Zhejiang University)
ClassificationConvolutional Neural NetworkGraph Neural NetworkImage
🎯 What it does: A model decomposition and recombination (MDA) framework is proposed to extract task-relevant components from pre-trained CNNs without the need for retraining, and to recombine these components into new models.
Model Reconstruction Using Counterfactual Explanations: A Perspective From Polytope Theory
Pasan Dissanayake (University of Maryland), Sanghamitra Dutta (University of Maryland)
Explainability and InterpretabilityAdversarial AttackTabular
🎯 What it does: By utilizing one-sided counterfactual explanations to construct a surrogate model, the Counterfactual Clamping Attack (CCA) strategy is proposed to overcome the issue of decision boundary drift and achieve model reconstruction.
Model Sensitivity Aware Continual Learning
Zhenyi Wang (University of Maryland), Heng Huang (University of Maryland)
ClassificationOptimizationImage
🎯 What it does: A continuous learning method based on model parameter sensitivity (MACL) is proposed, which reduces parameter sensitivity by optimizing the worst-case parameter distribution, achieving a balance between retaining old knowledge and performance on new tasks.
Model-based Diffusion for Trajectory Optimization
Chaoyi Pan (Carnegie Mellon University), Guannan Qu (Carnegie Mellon University)
OptimizationRobotic IntelligenceReinforcement LearningDiffusion modelScore-based ModelTime SeriesStochastic Differential Equation
🎯 What it does: A model-based diffusion framework (MBD) is proposed to directly generate and optimize trajectories from dynamic models without the need for pre-trained data.
Model-Based Transfer Learning for Contextual Reinforcement Learning
Jung-Hoon Cho (Massachusetts Institute of Technology), Cathy Wu (Massachusetts Institute of Technology)
Autonomous DrivingOptimizationReinforcement LearningTabular
🎯 What it does: A framework named Model-based Transfer Learning (MBTL) is proposed, which actively selects training tasks using Bayesian optimization in Contextual Reinforcement Learning (CMDP) to enhance generalization performance.
Model-free Low-Rank Reinforcement Learning via Leveraged Entry-wise Matrix Estimation
Stefan Stojanovic (KTH Royal Institute of Technology), Alexandre Proutiere (KTH Royal Institute of Technology)
Reinforcement Learning
🎯 What it does: A model-free low-rank reinforcement learning algorithm LoRa-PI is proposed, which estimates the Q-function using a low-rank structure and solves the optimal policy through approximate policy iteration.
Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems
Amber Hu (Stanford University), Scott Linderman (Stanford University)
Time SeriesBiomedical DataStochastic Differential Equation
🎯 What it does: A Gaussian Process-based Switching Linear Dynamical System (gpSLDS) is proposed and implemented to infer low-dimensional interpretable latent dynamics from high-dimensional neural signals.
MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks
Xingkui Zhu (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
ClassificationOptimizationComputational EfficiencyTransformerMixture of ExpertsImage
🎯 What it does: The MoE Jetpack framework is proposed, which recycles checkpoints of pre-trained dense models into a sparse expert network and achieves efficient fine-tuning through the SpheroMoE layer.
MoEUT: Mixture-of-Experts Universal Transformers
Róbert Csordás (Stanford University), Christopher D Manning
TransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: A general Transformer based on Mixture of Experts (MoE) called MoEUT is proposed, achieving more efficient language model training through shared layer parameters.
MoGenTS: Motion Generation based on Spatial-Temporal Joint Modeling
Weihao Yuan (Alibaba Group), Qixing Huang (University of Texas at Austin)
GenerationPose EstimationConvolutional Neural NetworkTransformerAuto EncoderVideoTime Series
🎯 What it does: Proposes to independently quantify each joint as a vector, constructing a space-time two-dimensional token, and based on this, perform text-driven action generation.
MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability
Yanrui Du (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the MoGU framework, which divides LLM into two modes: usable (Glad Responder) and safe (Unwill Responder) through dynamic routing, and dynamically allocates weights based on the type of instruction at input to achieve a balance between safety and usability.
MoLE: Enhancing Human-centric Text-to-image Diffusion via Mixture of Low-rank Experts
Jie Zhu (Peking University), Jingdong Wang
GenerationData SynthesisMixture of ExpertsDiffusion modelImage
🎯 What it does: A high-quality human-centered image dataset with millions of images was constructed, and the Mixture of Low-rank Experts (MoLE) method was proposed to enhance the generation quality of faces and hands.
Molecule Design by Latent Prompt Transformer
Deqian Kong (University of California Los Angeles), Ying Nian Wu (University of California Los Angeles)
GenerationOptimizationDrug DiscoveryTransformerReinforcement LearningPrompt EngineeringGraphSequential
🎯 What it does: Proposes the Latent Prompt Transformer (LPT), a conditional generative model that jointly generates molecular sequences and properties;
Molecule Generation with Fragment Retrieval Augmentation
Seul Lee (Korea Advanced Institute of Science and Technology), Weili Nie (NVIDIA)
GenerationDrug DiscoveryTransformerLarge Language ModelGraphBenchmarkRetrieval-Augmented Generation
🎯 What it does: A molecular generation framework f-RAG is proposed, which combines hard/soft fragment retrieval with genetic variation, using the pre-trained SAFE-GPT to generate new molecules and dynamically update the fragment vocabulary, enhancing the exploration-exploitation balance.
MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models
Leyang Shen (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
TransformerLarge Language ModelMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: The research addresses the issue of task interference in multimodal settings and proposes the Mixture of Multimodal Experts (MoME), utilizing MoVE on the visual side and MoLE on the language side to construct a general multimodal large language model.
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
Rachel Teo, Tan Minh Nguyen
OptimizationMixture of ExpertsImageText
🎯 What it does: In the Sparse Mixture of Experts model (SMoE), momentum is incorporated to propose MomentumSMoE, which is further extended to AdamSMoE and Robust MomentumSMoE, achieving improvements in model stability and robustness.
MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence
Fuming You (Zhejiang University), Zhou Zhao (Zhejiang University)
GenerationData SynthesisTransformerDiffusion modelAuto EncoderContrastive LearningVideoMultimodalityAudio
🎯 What it does: The MoMu-Diffusion framework is proposed for long-term synchronized bi-directional generation of motion and music.
MonkeySee: Space-time-resolved reconstructions of natural images from macaque multi-unit activity
Lynn Le (Donders Institute for Brain Cognition and Behaviour Radboud University), Umut Güçlü (Donders Institute for Brain Cognition and Behaviour Radboud University)
RestorationGenerationExplainability and InterpretabilityConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: Reconstruct natural images from macaque multi-unit activity using a CNN decoder.
Monoculture in Matching Markets
Kenny Peng (Cornell Tech), Nikhil Garg (Cornell Tech)
Tabular
🎯 What it does: The study examines the impact of algorithm monoculture on the welfare of firms and applicants in two-sided matching markets and proposes a corresponding theoretical model.
MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders
Xueying Jiang (Nanyang Technological University), Shijian Lu (University of Chinese Academy of Sciences)
Object DetectionAutonomous DrivingTransformerAuto EncoderPoint Cloud
🎯 What it does: Proposes MonoMAE, which simulates and learns occlusion information by adaptively masking in the feature space and reconstructing queries, enhancing monocular 3D detection;
Monomial Matrix Group Equivariant Neural Functional Networks
Hoang V. Tran, Tan Minh Nguyen
Convolutional Neural NetworkImage
🎯 What it does: This paper studies the symmetry of weight space in fully connected and convolutional neural networks, proposing the Monomial-NFN (Monomial Neural Function Network) which achieves a more complete modeling of weight space through a unified treatment of weight scaling or sign flipping and permutation symmetry.
Monte Carlo Tree Search based Space Transfer for Black Box Optimization
Shukuan Wang (Nanjing University), Chao Qian (Nanjing University)
OptimizationTabular
🎯 What it does: A search space transfer framework based on Monte Carlo Tree Search (MCTS-transfer) is proposed, which can dynamically partition, select, and update the search space in black-box optimization.
Most Influential Subset Selection: Challenges, Promises, and Beyond
Yuzheng Hu (University of Illinois), Jiaqi Ma
ImageTabular
🎯 What it does: This paper systematically studies the Most Influential Subset Selection (MISS) problem, analyzes the failure mechanisms of traditional greedy heuristic methods based on influence functions, proposes and theoretically proves the advantages of an adaptive greedy algorithm in capturing sample interactions, and conducts experimental validation on various tasks.
MOTE-NAS: Multi-Objective Training-based Estimate for Efficient Neural Architecture Search
Yuming Zhang, Kuo-Chin Fan (National Central University)
OptimizationNeural Architecture SearchImage
🎯 What it does: A multi-objective training estimator (MOTE) is designed for neural architecture search, combining macro landscape terms and micro speed terms to achieve low-cost evaluation by reducing the architecture and dataset, and is integrated with evolutionary search to produce MOTE-NAS and MOTE-NAS-EF;
MoTE: Reconciling Generalization with Specialization for Visual-Language to Video Knowledge Transfer
Minghao Zhu (Tongji University), Qijun Chen (Tongji University)
RecognitionDomain AdaptationKnowledge DistillationTransformerMixture of ExpertsVision Language ModelVideoMultimodality
🎯 What it does: This paper studies how to transfer large visual language models (such as CLIP) to video recognition tasks while achieving a balance between generalization and specialization with efficient computational costs.
Motif-oriented influence maximization for viral marketing in large-scale social networks
Mingyang Zhou (Shenzhen University), Rui Mao (Shenzhen University)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proposes and solves the 'influence maximization based on graph motifs' problem, first transforming the non-submodular objective function into upper and lower bounds of submodular functions, and then designing the LBMOIM algorithm to achieve approximate optimality;
Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation
Yuanhao Zhai (State University of New York at Buffalo), Lijuan Wang (Microsoft)
GenerationData SynthesisComputational EfficiencyKnowledge DistillationAdversarial AttackDiffusion modelVideo
🎯 What it does: Proposes the Motion Consistency Model (MCM), which achieves fast sampling of video diffusion models and frame quality enhancement through a video consistency model and an image discriminator.
Motion Forecasting in Continuous Driving
Nan Song (Fudan University), Li Zhang (Fudan University)
Autonomous DrivingTransformerTime SeriesSequential
🎯 What it does: A motion prediction framework called RealMotion is proposed for continuous driving scenarios, which can recursively aggregate historical scene information and continuously refine trajectory predictions at each frame.
Motion Graph Unleashed: A Novel Approach to Video Prediction
Yiqi Zhong (Microsoft), Ulrich Neumann (University of Southern California)
GenerationData SynthesisAutonomous DrivingGraph Neural NetworkVideo
🎯 What it does: This paper addresses the problem of video prediction and proposes a graph-based motion representation—motion graph—and builds a complete prediction pipeline based on it.
MotionBooth: Motion-Aware Customized Text-to-Video Generation
Jianzong Wu (Peking University), Kai Chen (Shanghai AI Laboratory)
GenerationData SynthesisDiffusion modelImageVideo
🎯 What it does: The MotionBooth framework is proposed to achieve customized video generation for a small number of object images, supporting controllable subject and camera motion.
MotionCraft: Physics-Based Zero-Shot Video Generation
Antonio Montanaro (Politecnico di Torino), Enrico Magli (Politecnico di Torino)
GenerationData SynthesisDiffusion modelOptical FlowVideoBenchmarkPhysics Related
🎯 What it does: This paper proposes a zero-training physics-driven video generation method that deforms the noise latent space of a pre-trained image diffusion model (Stable Diffusion) using optical flow generated from physical simulations, thereby generating videos with complex physical motion dynamics while maintaining scene consistency.
MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting
Ruijie Zhu (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
RestorationGenerationGaussian SplattingOptical FlowVideo
🎯 What it does: Proposes the MotionGS framework, which uses explicit motion priors to guide the deformation of 3D Gaussians in dynamic scenes, achieving high-quality real-time rendering.
MotionTTT: 2D Test-Time-Training Motion Estimation for 3D Motion Corrected MRI
Tobit Klug (Technical University of Munich), Reinhard Heckel (Technical University of Munich)
RestorationOptimizationConvolutional Neural NetworkBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A deep learning method based on test-time training, MotionTTT, is proposed for rigid motion estimation and correction in 3D MRI.
MoVA: Adapting Mixture of Vision Experts to Multimodal Context
Zhuofan Zong (Chinese University of Hong Kong), Yu Liu (SenseTime Research)
TransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: We propose MoVA, a multimodal large language model that achieves adaptive extraction and fusion of multi-task visual information through a two-stage mechanism (first using LLM for context-aware expert routing, followed by fine-grained fusion of selected visual experts with MoV-Adapter).
Moving Off-the-Grid: Scene-Grounded Video Representations
Sjoerd van Steenkiste (Google Research), Thomas Kipf (Google DeepMind)
Object TrackingDepth EstimationRepresentation LearningTransformerVideo
🎯 What it does: Proposes the MooG model, which uses off-the-grid tokens for video representation learning through cross-attention, leveraging a self-supervised task of predicting the next frame.
MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs
Zhongshen Zeng (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
TransformerLarge Language ModelTextBenchmarkPhysics RelatedChain-of-Thought
🎯 What it does: A new process-oriented benchmark, MR-Ben, is proposed to evaluate the meta-reasoning ability of large language models in System-2 (slow thinking) reasoning.
MSA Generation with Seqs2Seqs Pretraining: Advancing Protein Structure Predictions
Le Zhang (Fudan University), Siqi Sun (Fudan University)
Protein Structure PredictionTransformerBiomedical Data
🎯 What it does: This paper proposes a self-supervised generative pre-training model called MSA-Generator based on the seqs2seqs task, aimed at generating virtual MSAs to improve the accuracy of protein structure prediction.
MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training
Bo Chen (Tsinghua University), Le Song (BioMap Research)
Protein Structure PredictionTransformerReinforcement LearningPrompt EngineeringBiomedical Data
🎯 What it does: A protein structure prediction model based on MSA-generated pre-training, called MSAGPT, is proposed. It can enhance the accuracy of structure predictions like AlphaFold2 in low MSA environments lacking rich homologous sequences by generating virtual MSA.
MSPE: Multi-Scale Patch Embedding Prompts Vision Transformers to Any Resolution
Wenzhuo Liu (University of Chinese Academy of Sciences), Cheng-Lin Liu (University of Chinese Academy of Sciences)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: This paper proposes a Multi-Scale Patch Embedding (MSPE) method that enables Vision Transformers to directly handle images of arbitrary resolutions without the need for resizing or interpolation of the input images.
MTGS: A Novel Framework for Multi-Person Temporal Gaze Following and Social Gaze Prediction
Anshul Gupta (Idiap Research Institute), Jean-marc Odobez
RecognitionObject TrackingTransformerVideo
🎯 What it does: A unified framework for multi-temporal multi-person gaze tracking and social gaze prediction is proposed, capable of simultaneously predicting each person's gaze target, entry/exit labels, and social gaze relationships (looking at people, mutual gaze, shared attention);
Multi-Agent Coordination via Multi-Level Communication
Ziluo Ding (Tsinghua University), Zongqing Lu (Peking University)
Robotic IntelligenceReinforcement LearningWorld ModelSequential
🎯 What it does: Proposes the SeqComm multi-level communication framework, allowing multiple agents to achieve more efficient collaboration through asynchronous decision-making and two-stage communication.
Multi-Agent Domain Calibration with a Handful of Offline Data
Tao Jiang (Nanjing University), Yang Yu (Nanjing University)
Domain AdaptationRobotic IntelligenceReinforcement LearningAuto EncoderTabular
🎯 What it does: This paper proposes the Madoc framework, which utilizes a small amount of offline target domain data to perform multi-agent domain calibration of source domain physical parameters, allowing for the direct deployment of policies trained in the source domain without interacting with the target domain.
Multi-Agent Imitation Learning: Value is Easy, Regret is Hard
Jingwu Tang (Carnegie Mellon University), Steven Wu
Reinforcement Learning
🎯 What it does: This study investigates the relationship between value gap and regret gap in multi-agent imitation learning (MAIL), proposing two theoretical algorithms (MALICE and BLADES) that effectively reduce the regret gap under the assumptions of coverage or queryable experts.
Multi-Group Proportional Representation in Retrieval
Alex Oesterling (Harvard University), Flavio Calmon (Harvard University)
RetrievalOptimizationImage
🎯 What it does: This paper proposes the Multi-Group Proportional Representation (MPR) metric to measure the representativeness of search results across intersecting groups, and designs the MOPR algorithm based on this metric, which balances similarity and MPR constraints during retrieval to achieve fairer search results.
Multi-Head Mixture-of-Experts
Xun Wu (Microsoft Research Asia), Furu Wei (Microsoft Research Asia)
TransformerMixture of ExpertsTextMultimodality
🎯 What it does: Designed and implemented a Multi-Head Sparse Expert Mixture Network (MH-MoE), significantly improving expert activation rates by splitting each token into sub-tokens and assigning them to different experts while keeping computational costs unchanged.
Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images
Donghwan Kim (KAIST), Tae-Kyun Kim (KAIST)
RestorationGenerationData SynthesisPose EstimationTransformerDiffusion modelImagePoint Cloud
🎯 What it does: A multi-hypothesis conditional point cloud diffusion model, MHCDIFF, is proposed to recover pixel-aligned and detail-rich 3D human point clouds from a single occluded image.
Multi-Instance Partial-Label Learning with Margin Adjustment
Wei Tang (Southeast University), Min-Ling Zhang (Southeast University)
ClassificationRecognitionConvolutional Neural NetworkImageBiomedical DataBenchmark
🎯 What it does: This paper proposes a new multi-instance partial label learning framework MIPLMA, which addresses the margin violation issues that traditional methods encounter in both instance space and label space.
Multi-Label Learning with Stronger Consistency Guarantees
Anqi Mao (Courant Institute of Mathematical Sciences), Yutao Zhong (Courant Institute of Mathematical Sciences)
ClassificationOptimization
🎯 What it does: This paper systematically studies surrogate loss functions in multi-label learning, proposing a new multi-label Logistic loss and its general comp-sum and constrained loss framework, and provides H-consistency guarantees and Bayes consistency proofs for arbitrary multi-label losses.
Multi-Label Open Set Recognition
Yibo Wang, Min-Ling Zhang (Southeast University)
ClassificationRecognitionTabular
🎯 What it does: A multi-label open set recognition (MLOSR) framework is proposed, and the SLAN method is designed to achieve open set multi-label classification and unknown label recognition.
Multi-language Diversity Benefits Autoformalization
Albert Q. Jiang (University of Cambridge), Mateja Jamnik (University of Cambridge)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A large-scale informal-formal alignment dataset MMA has been constructed across multiple languages and domains, and it has been used to fine-tune LLMs to achieve zero-shot automatic formalization.
Multi-LLM Debate: Framework, Principals, and Interventions
Andrew Estornell (ByteDance Research), Yang Liu (University of California Santa Cruz)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A theoretical framework for multi-LLM debates is proposed, along with three intervention methods (diversity pruning, quality pruning, and fallacy rebuttal) to enhance debate effectiveness.
Multi-modal Transfer Learning between Biological Foundation Models
Juan Jose Garau-Luis (InstaDeep), Guillaume Richard (InstaDeep)
Representation LearningData-Centric LearningTransformerMultimodalityBiomedical Data
🎯 What it does: Developed IsoFormer—a multimodal transfer learning model that connects DNA, RNA, and protein sequences, and is used to predict the expression levels of different RNA transcripts of the same gene across various tissues.
Multi-model Ensemble Conformal Prediction in Dynamic Environments
Erfan Hajihashemi (University of California), Yanning Shen (University of California)
ClassificationDomain AdaptationComputational EfficiencyConvolutional Neural NetworkMixture of ExpertsImage
🎯 What it does: An adaptive synthetic prediction framework using multi-model ensemble (SAMOCP) is proposed for dynamic environments, capable of 'instantaneously' selecting the most suitable learning model to generate a prediction set at each moment while maintaining a preset coverage probability.
Multi-Object 3D Grounding with Dynamic Modules and Language-Informed Spatial Attention
Haomeng Zhang (Purdue University), Raymond A. Yeh (Purdue University)
Object DetectionTransformerContrastive LearningPoint Cloud
🎯 What it does: A two-stage multi-target 3D localization framework called D-LISA is proposed, which achieves multi-target 3D object localization through three main modules: dynamic bounding box selection, dynamic multi-view rendering, and language-aware spatial attention fusion.
Multi-Object Hallucination in Vision Language Models
Xuweiyi Chen (University of Michigan), Joyce Chai (University of Michigan)
RecognitionObject DetectionTransformerVision Language ModelImageMultimodality
🎯 What it does: This paper systematically studies the hallucination phenomenon of large-scale visual language models (LVLM) in recognizing multiple objects by constructing a multi-object recognition evaluation protocol called ROPE, and analyzes various factors that lead to hallucinations.
Multi-Reward Best Policy Identification
Alessio Russo (Ericsson AB), Filippo Vannella (Ericsson Research)
OptimizationReinforcement Learning
🎯 What it does: This paper studies the Multi-Reward Best Policy Identification (MR-BPI) problem, which focuses on how to quickly determine the optimal policy for all rewards with the least samples given a set of rewards R;
Multi-scale Consistency for Robust 3D Registration via Hierarchical Sinkhorn Tree
Chengwei Ren (Tsinghua University), Yue Gao (Tsinghua University)
Object DetectionRetrievalOptimizationSimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper proposes a Hierarchical Sinkhorn Tree (HST) framework that utilizes Multi-Scale Consistency (MSC) for reliable assessment of coarse matching, significantly enhancing the robustness of 3D point cloud registration.
Multi-Scale Representation Learning for Protein Fitness Prediction
Zuobai Zhang (Mila - Quebec AI Institute), Jian Tang (HEC Montreal)
Representation LearningDrug DiscoveryProtein Structure PredictionGraph Neural NetworkLarge Language ModelPoint CloudGraphBiomedical Data
🎯 What it does: A multi-scale protein representation learning framework S3F is proposed, which combines sequence, structure, and surface information to achieve zero-shot protein function fitness prediction.
Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model
Yuheng Shi (City University of Hong Kong), Chang Xu (University of Sydney)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkImage
🎯 What it does: A multi-scale Vision Mamba (MSVMamba) is proposed, which enhances the long-range modeling capability of SSM in visual tasks through multi-scale 2D scanning (MS2D) and convolutional Feed-Forward networks (ConvFFN).
Multi-Stage Predict+Optimize for (Mixed Integer) Linear Programs
Xinyi HU, Peter J. Stuckey (Monash University)
OptimizationTabularFinance Related
🎯 What it does: A multi-stage Predict+Optimize framework is proposed to address optimization problems where parameters are gradually revealed across multiple stages, along with three training methods.
Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction
Zhu Tengjie, Xiaokang Yang (Shanghai Jiao Tong University)
RestorationData SynthesisMesh
🎯 What it does: A multiple Monte Carlo rendering inverse rendering method is proposed, capable of simultaneously reconstructing high-quality geometry, materials, and ambient light, particularly targeting multi-object inter-reflection scenes.
Multi-turn Reinforcement Learning with Preference Human Feedback
Lior Shani (Google), Remi Munos
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: A multi-turn reinforcement learning from preference feedback (MTPO) algorithm is proposed to address the limitations of traditional single-turn RLHF in multi-turn dialogues.
Multi-view Masked Contrastive Representation Learning for Endoscopic Video Analysis
Kai Hu (Xiangtan University), Xieping Gao (Hunan Normal University)
ClassificationObject DetectionSegmentationRepresentation LearningTransformerContrastive LearningVideoBiomedical Data
🎯 What it does: A multi-view occlusion contrastive learning framework (MCRL) is proposed for unsupervised pre-training of endoscopic videos, combining global view aggregation attention-guided pipeline occlusion and local view random pipeline occlusion, integrating the occlusion reconstruction task with contrastive learning to enhance pixel-level and global discriminative capabilities.
Multi-Winner Reconfiguration
Jiehua Chen (Institute of Logic and Computation TU Wien), Sofia Simola (Institute of Logic and Computation TU Wien)
🎯 What it does: This study investigates the committee reconstruction model in multi-winner elections, determining whether there exists a feasible reconstruction path between two committees.
Multiclass Transductive Online Learning
Steve Hanneke (Purdue University), Unique Subedi (University of Michigan)
🎯 What it does: In the framework of multi-class transductive online learning, this paper studies the learnability and error bounds when the label space is infinitely large, proposing two new combinatorial dimensions to characterize learning difficulty and providing corresponding learning algorithms.
Multidimensional Fractional Programming for Normalized Cuts
Yannan Chen (Chinese University of Hong Kong), Kaiming Shen (Chinese University of Hong Kong)
SegmentationOptimizationImage
🎯 What it does: A new NCut (Normalized Cut) clustering algorithm (FPC) is proposed through Multidimensional Fractional Programming. This algorithm simplifies the original 0-1 ratio optimization problem into an iteratively solvable linear search problem through a matrix form quadratic transformation.
Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization
James Oldfield (Queen Mary University of London), Ioannis Patras (Queen Mary University of London)
Mixture of ExpertsVision Language ModelImageText
🎯 What it does: This paper proposes a multi-linear expert mixture layer µ MoE, utilizing tensor decomposition to achieve scalable expert specialization, addressing the training instability and parameter efficiency issues of sparse MoE.
Multilingual Diversity Improves Vision-Language Representations
Thao Nguyen (University of Washington), Ranjay Krishna (University of Washington)
Image TranslationRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper studies the positive impact of multilingual diversity on English visual tasks by translating multilingual image-text pairs into English and re-filtering them.
Multimodal Large Language Models Make Text-to-Image Generative Models Align Better
Xun Wu (Microsoft Research), Furu Wei (Microsoft Research)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: This paper generates a large-scale fine-grained human preference dataset, VisionPrefer, using a multimodal large language model (GPT-4 V), and based on this, trains a reward model, VP-Score, for RLHF fine-tuning of text-to-image generation models, thereby enhancing the adherence to prompts, aesthetics, authenticity, and safety of the generated images.
Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning
Brandon Huang (University of California), Roei Herzig (IBM Research)
ClassificationRecognitionTransformerReinforcement LearningVision Language ModelMultimodality
🎯 What it does: This paper proposes the Multi-modal Task Vector (MTV), which achieves many-shot context learning in a multi-modal setting without being limited by context length by embedding the average activation of multiple examples in the attention heads of large multi-modal models.
MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities
Hao Dong (ETH Zurich), Olga Fink (EPFL)
Anomaly DetectionVideoMultimodalityBenchmark
🎯 What it does: This paper proposes a Multi-modal Out-of-Distribution detection (MultiOOD) framework and establishes the first multi-modal OOD benchmark.
Multiple Physics Pretraining for Spatiotemporal Surrogate Models
Michael McCabe (Flatiron Institute), Shirley Ho (Flatiron Institute)
TransformerTime SeriesPhysics Related
🎯 What it does: What was done: Proposed Multi-Physics Pretraining (MPP), which jointly learns dynamics across various heterogeneous spatiotemporal physical systems through an autoregressive transformer, forming a transferable foundational model.
MultiPull: Detailing Signed Distance Functions by Pulling Multi-Level Queries at Multi-Step
Takeshi Noda (Tsinghua University), Zhizhong Han (Wayne State University)
Point CloudBenchmark
🎯 What it does: By querying points through multi-step stretching and utilizing multi-scale Fourier frequency features, we learn high-resolution implicit SDF to achieve surface reconstruction from coarse to fine.
Multistable Shape from Shading Emerges from Patch Diffusion
Xinran Han, Ko Nishino (Kyoto University)
GenerationDepth EstimationDiffusion modelImage
🎯 What it does: A multimodal monocular shape estimation method based on diffusion models is proposed, which can generate multiple possible surface normal distributions from a single shadow image.
Multistep Distillation of Diffusion Models via Moment Matching
Tim Salimans (Google DeepMind), Emiel Hoogeboom (Google DeepMind)
GenerationKnowledge DistillationDiffusion modelImage
🎯 What it does: This paper studies and implements a multi-step distillation method that distills a high-step diffusion model into a low-step model, significantly accelerating sampling while maintaining or improving image quality.
Multivariate Probabilistic Time Series Forecasting with Correlated Errors
Vincent Zhihao Zheng (McGill University), Lijun Sun (McGill University)
Recurrent Neural NetworkTransformerTime Series
🎯 What it does: A pluggable method is proposed to learn and calibrate the cross-step error covariance in multivariate probabilistic time series forecasting, thereby improving prediction and uncertainty quantification.
Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarking
Gabriel Rioux (Center for Applied Mathematics Cornell University), Youssef Mroueh (IBM Research)
Recommendation SystemOptimizationComputational EfficiencyLarge Language ModelTabularBenchmark
🎯 What it does: A multivariate first-order stochastic dominance (FSD) testing method based on entropy regularization is proposed, along with its statistical inference (CLT, bootstrap) and algorithm implementation. Experiments are then conducted on synthetic data and real LLM multi-metric evaluation data.
Multiview Scene Graph
Juexiao Zhang (New York University), Chen Feng (New York University)
RecognitionObject DetectionTransformerContrastive LearningImage
🎯 What it does: This paper proposes the construction of a Multiview Scene Graph (MSG) using unposed RGB images, forming a global topological scene representation by connecting image nodes from the same location and associating different views of the same object.
MutaPLM: Protein Language Modeling for Mutation Explanation and Engineering
YIZHEN LUO, Zaiqing Nie (Pharmolix Inc.)
Recommendation SystemOptimizationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical DataChain-of-Thought
🎯 What it does: This paper presents MutaPLM, a unified framework based on protein language models for providing interpretable explanations of protein mutations and engineering-recommendable variants.
Mutli-Armed Bandits with Network Interference
Abhineet Agarwal (University of California Berkeley), Justin Whitehouse (Carnegie Mellon University)
Reinforcement LearningGraph
🎯 What it does: This paper proposes two exploration-commitment algorithms based on a sparse network interference model and discrete Fourier analysis for the multi-armed bandit (MAB) problem in the presence of network interference, aiming to achieve low regret.
Mutual Information Estimation via $f$-Divergence and Data Derangements
Nunzio Alexandro Letizia (University of Klagenfurt), Andrea M Tonello (University of Klagenfurt)
OptimizationGenerative Adversarial NetworkImage
🎯 What it does: A mutual information estimation framework based on f-divergence variational representation (f-DIME) is proposed, introducing a derangement sampling strategy to effectively generate marginal distribution samples, significantly reducing estimation variance and improving estimation accuracy.
Mutual Information Estimation via Normalizing Flows
Ivan Butakov (Skolkovo Institute of Science and Technology), Alexey Frolov (Skolkovo Institute of Science and Technology)
Flow-based ModelImage
🎯 What it does: A mutual information (MI) estimation method based on normalizing flows is proposed, which utilizes reversible mappings to map the original data to a latent space that is easier for estimating MI, and provides a closed-form MI calculation formula, offering consistency, non-asymptotic error bounds, and low variance estimates.
MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encoding
Rajesh Jayaram (Google Research), Vahab Mirrokni (Google Research)
RetrievalTextBenchmark
🎯 What it does: MUVERA is proposed, which transforms the multi-vector retrieval problem into a single-vector retrieval problem through Fixed Dimension Encoding (FDE), thereby achieving fast Chamfer similarity search.
MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images
Eunji Hong (Korea Advanced Institute of Science and Technology), Minhyuk Sung (Korea Advanced Institute of Science and Technology)
SegmentationGenerationOptimizationConvolutional Neural NetworkNeural Radiance FieldImage
🎯 What it does: The MV2Cyl method is proposed, which utilizes multi-view RGB images to directly reconstruct 3D shells (i.e., a set of extruded cylinders) through 2D surface and curve segmentation combined with 3D neural fields.
MVGamba: Unify 3D Content Generation as State Space Sequence Modeling
Xuanyu Yi (Nanyang Technological University), Hanwang Zhang (Nanyang Technological University)
GenerationData SynthesisDiffusion modelGaussian SplattingImagePoint Cloud
🎯 What it does: A unified 3D generation framework called MVGamba is proposed, which combines a multi-view diffusion model with an efficient Gaussian reconstructor based on a state space model (SSM) to generate high-quality 3D content in a single forward pass.
MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editing
Chenjie Cao (Fudan University), Yanwei Fu (Fudan University)
RestorationGenerationDiffusion modelOptical FlowImageVideo
🎯 What it does: This paper proposes a multi-view consistent image inpainting framework called MVInpainter, which extends single-view 2D editing (such as removal, insertion, and replacement) to 3D scenes, enabling pose-free multi-view editing.
MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps
Yating Xu (National University of Singapore), Gim Hee Lee (A*STAR)
Object DetectionDepth EstimationConvolutional Neural NetworkGaussian SplattingPoint Cloud
🎯 What it does: A multi-view 3D object detection framework MVSDet based on efficient planar sweeping is proposed.
MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views
Yuedong Chen (Monash University), Jianfei Cai (Monash University)
GenerationData SynthesisDepth EstimationTransformerDiffusion modelGaussian SplattingImageVideoBenchmark
🎯 What it does: This paper presents MVSplat360, a forward synthesis model that generates new 360° panoramic views using only 5 sparse perspectives without the need for scene-by-scene optimization.
N-agent Ad Hoc Teamwork
Caroline Wang (University of Texas at Austin), Peter Stone (University of Texas at Austin)
Reinforcement LearningAgentic AI
🎯 What it does: This paper proposes the N-agent spontaneous team (NAHT) problem, studying how multiple agents can cooperate in the presence of unknown team members.
NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing
Ting-Hsuan Chen (National Yang Ming Chiao Tung University), Yu-Lun Liu (National Yang Ming Chiao Tung University)
SegmentationGenerationDiffusion modelVideo
🎯 What it does: Designed and implemented the NaRCan video editing framework, which generates natural canonical images using a hybrid deformation field and ensures high quality through a diffusion prior, making it directly applicable to tasks such as video editing, dynamic segmentation, and style transfer.
Natural Counterfactuals With Necessary Backtracking
Guang-Yuan Hao (Chinese University of Hong Kong), Kun Zhang (University of California San Diego)
GenerationOptimizationAuto EncoderGenerative Adversarial NetworkTabular
🎯 What it does: The concept of Natural Counterfactuals is proposed, allowing for necessary backtracking during the generation of counterfactuals, and an optimization framework (FIO) based on naturality constraints and minimal backtracking is provided to generate feasible counterfactual instances that are close to the data distribution.
Nature-Inspired Local Propagation
Alessandro Betti (IMT School for Advanced Studies Lucca), Marco Gori (University of Siena)
Recurrent Neural NetworkTime SeriesOrdinary Differential Equation
🎯 What it does: This paper proposes a spatiotemporal local learning framework based on Hamiltonian equations, modeling online learning as a variational problem. It derives a learning rule that can incrementally update weights on temporal data streams and proves that as the signal propagation speed approaches infinity, this rule degenerates into traditional backpropagation.
Navigable Graphs for High-Dimensional Nearest Neighbor Search: Constructions and Limits
Haya Diwan (New York University), Torsten Suel (New York University)
Graph
🎯 What it does: Study the construction and limits of navigable graphs for high-dimensional point sets, providing upper and lower bounds.
Navigating Chemical Space with Latent Flows
Guanghao Wei (Cornell University), Yuanqi Du (Cornell University)
OptimizationDrug DiscoveryFlow-based ModelAuto EncoderGraph
🎯 What it does: This paper proposes the ChemFlow framework, which views the latent space of molecular generation models as a dynamic system of flow, utilizing vector fields to migrate molecular distributions in the latent space, thereby achieving property optimization and structural exploration.
Navigating Extremes: Dynamic Sparsity in Large Output Spaces
Nasib Ullah (Aalto University), Rohit Babbar (University of Bath)
ClassificationTransformerSupervised Fine-TuningText
🎯 What it does: The study applies Dynamic Sparse Training (DST) to extreme multi-label classification (XMC) by using a semi-structured fixed fan-in sparse matrix in the classification layer and introducing auxiliary objectives (meta-classifier or intermediate layer) to improve gradient flow, resulting in significant memory savings during end-to-end training.
Navigating the Effect of Parametrization for Dimensionality Reduction
Haiyang Huang (Duke University), Cynthia Rudin (Duke University)
OptimizationRepresentation LearningContrastive LearningImageText
🎯 What it does: This paper studies the limitations of parametric dimensionality reduction methods in preserving local structure and proposes a new method called ParamRepulsor to address this issue.
Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
ShengYun Peng (Georgia Tech), Duen Horng Chau (Georgia Tech)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the security risks of large language models (LLMs) during the fine-tuning process and proposes assessing security by visualizing the safety landscape of model parameter space.
Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model
Mark Rowland (Google DeepMind), Will Dabney (Google DeepMind)
Reinforcement LearningTabular
🎯 What it does: A new Direct Classification Fixed Point algorithm (DCFP) is proposed for efficiently estimating the return distribution under generative models.
Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity
Qihao Zhou (Fudan University), Luo Luo (Fudan University)
OptimizationTabular
🎯 What it does: A distributed convex-concave minimax optimization algorithm named SVOGS is proposed, which utilizes second-order similarity and variance reduction techniques to achieve near-optimal communication and computational complexity under a finite summation structure.