International Conference on Learning Representations · 1682 papers
MaxCutPool: differentiable feature-aware Maxcut for pooling in graph neural networks
Carlo Abate (Alma Mater Studiorum University of Bologna), Filippo Maria Bianchi (UiT Arctic University of Norway)
CodeGraph Neural NetworkGraph
🎯 What it does: This paper proposes a differentiable, feature-based MaxCut pooling layer called MaxCutPool, aimed at achieving sparse and trainable graph pooling in graph neural networks.
Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Aymane El Firdoussi (Technology Innovation Institute), Hakim Hacid (Technology Innovation Institute)
CodeClassificationData SynthesisText
🎯 What it does: This study investigates the theoretical and experimental impact of using validation mechanisms for synthetic data on model training in high-dimensional environments, conducting a quantitative analysis of the performance of binary classifiers that mix real and synthetic data based on random matrix theory.
Linzheng Chai (Beihang University), Zhoujun Li (Beihang University)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: The first multilingual code evaluation benchmark MCEVAL covering 40 programming languages is proposed, which includes three main tasks: code generation, interpretation, and completion, and a corresponding multilingual instruction dataset MCEVAL-INSTRUCT is constructed.
MCNC: Manifold-Constrained Reparameterization for Neural Compression
Chayne Thrash (Vanderbilt University), Soheil Kolouri (Vanderbilt University)
CodeCompressionTransformerLarge Language ModelImage
🎯 What it does: A model compression method through low-dimensional nonlinear manifold reparameterization is proposed—MCNC, aimed at significantly reducing the number of trainable parameters in large neural networks.
Measuring Non-Adversarial Reproduction of Training Data in Large Language Models
Michael Aerni (ETH Zurich), Florian Tramèr (ETH Zurich)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Evaluate the non-adversarial training data reproduction of large language models under natural prompts and quantify the overlap between generated text and internet content.
🎯 What it does: In knowledge distillation, static pruning of the training set is performed to retain only moderately difficult samples, and the logits of the retained samples are reshaped, thereby achieving training acceleration while maintaining or improving model accuracy.
MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks
Jiacheng Chen (Simon Fraser University), Wenhu Chen (University of Waterloo)
CodeTransformerLarge Language ModelVision Language ModelImageVideoTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Designed the MEGA-BENCH benchmark, covering over 500 real multimodal tasks, supporting various input/output formats, and providing over 40 custom evaluation metrics;
Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis
Jinbin Bai (National University of Singapore), Shuicheng YAN
CodeGenerationData SynthesisTransformerVision Language ModelImageTextMultimodality
🎯 What it does: Proposed the Meissonic model, improving Mask Image Modeling (MIM) to achieve 1024×1024 high-resolution text-to-image generation and enabling zero-shot image editing.
Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold
Lazar Atanackovic (University of Toronto), Kirill Neklyudov (Mila - Quebec AI Institute)
CodeMeta LearningDrug DiscoveryGraph Neural NetworkFlow-based ModelBiomedical Data
🎯 What it does: This paper proposes Meta Flow Matching (MFM), a method for integrating vector fields on the Wasserstein manifold to predict group dynamics under different initial distributions.
Seungyoon Woo (Seoul National University), Gunhee Kim (Seoul National University)
CodeGenerationMeta LearningMixture of ExpertsNeural Radiance FieldImageVideoAudio
🎯 What it does: The Meta-Continual Learning of Neural Fields (MCL-NF) framework is proposed, which combines a modular architecture with optimization-based meta-learning to achieve rapid continual learning of neural fields, and introduces Fisher Information Maximization Loss.
Metalic: Meta-Learning In-Context with Protein Language Models
Jacob Beck (InstaDeep), Paul Duckworth (InstaDeep)
CodeMeta LearningDrug DiscoveryProtein Structure PredictionTransformerBiomedical Data
🎯 What it does: Utilizing a method that combines meta-learning and contextual learning to enhance the performance of low-sample protein fitness prediction.
MetaOOD: Automatic Selection of OOD Detection Models
Yuehan Qin (University of Southern California), Yue Zhao (University of Southern California)
CodeAnomaly DetectionMeta LearningLarge Language ModelImage
🎯 What it does: MetaOOD is proposed, a meta-learning based unsupervised OOD detection model selection framework that can automatically select the most suitable OOD detector without labels.
CodeTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: This paper proposes the MeteoRA framework, which utilizes a full-mode MoE to embed multiple task LoRA adapters into a single LLM, achieving unsupervised task awareness and automatic switching.
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs
Yusu Qian (Apple), Zhe Gan (Apple)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper presents MIA-Bench, a benchmark for multi-modal large language models (MLLMs) designed to evaluate the models' ability to strictly follow complex hierarchical instructions, and based on this, to improve instruction adherence performance through supervised fine-tuning.
MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models
Ziyu Liu (Shanghai Jiao Tong University), Jiaqi Wang (Shanghai Innovation Institute)
CodeRecommendation SystemOptimizationTransformerReinforcement LearningVision Language ModelImageMultimodality
🎯 What it does: Proposes MIA-DPO: a method for multi-image preference alignment of large visual language models by expanding single-image data into multi-image inputs and using attention values to filter generated unlabeled rejection samples.
🎯 What it does: The research directly trains models without backdoors from poisoned data that can recover the original labels, proposing an end-to-end anti-backdoor learning method.
CodeTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes the MIND method, which utilizes a pre-trained LLM to convert raw mathematical text into multi-turn dialogues, and continuously pre-trains a 7B language model on these dialogues to enhance mathematical reasoning and general reasoning capabilities.
MindSearch: Mimicking Human Minds Elicits Deep AI Searcher
Zehui Chen (University of Science and Technology of China), Feng Zhao (University of Science and Technology of China)
CodeRetrievalTransformerLarge Language ModelAgentic AITextRetrieval-Augmented Generation
🎯 What it does: Proposes the MindSearch framework, utilizing LLM multi-agent to achieve cognitive decomposition and hierarchical retrieval of complex problems, improving network information retrieval and integration.
Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramid
Mingxin Huang (South China University of Technology), Xiang Bai (Huazhong University of Science and Technology)
CodeRecognitionSegmentationCompressionTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This paper proposes the Complementary Image Pyramid (CIP) and Scale Compression Mechanism (SCM), which dynamically constructs a multi-scale image pyramid and compresses visual tokens to address the semantic aliasing effect encountered by cropping-based MLLMs when processing high-resolution images, further developing a lightweight MLLM called Mini-Monkey.
CodeKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: This paper proposes the MINIPLM framework, which integrates the knowledge of the teacher LM into the pre-training data distribution through differential sampling to train efficient small language models.
MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models
Jiachun Li (University of Chinese Academy of Sciences), Jun Zhao (University of Chinese Academy of Sciences)
CodeTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: A synthetic dataset named MIRAGE has been constructed to flexibly generate test samples for a comprehensive evaluation of large language models' performance in inductive reasoning (rule induction and deduction).
🎯 What it does: The AlignCLIP framework is proposed, which enhances the cross-modal alignment of CLIP by sharing the parameters of the visual and text encoders and separating the semantic regularization of unimodal embeddings.
Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization
Sascha Marton (University of Mannheim), Heiner Stuckenschmidt (University of Mannheim)
CodeOptimizationExplainability and InterpretabilityReinforcement LearningTabular
🎯 What it does: This paper presents SYMPOL, a method that can directly optimize axis-aligned decision tree (DT) policies within a policy gradient-based online reinforcement learning framework, achieving interpretable policies without information loss.
Mansi Sakarvadia (University of Chicago), Michael W. Mahoney (Lawrence Berkeley National Laboratory)
CodeSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper addresses the issue of language models (LM) potentially 'remembering' training data during inference, leading to privacy or copyright information leakage, and proposes and evaluates various memory mitigation methods.
Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
Guanyu Zhou (Hong Kong University of Science and Technology), Xuming Hu (Hong Kong University of Science and Technology)
CodeGenerationData SynthesisOptimizationTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: The CAUSALMM method is proposed, which utilizes structural causal models and counterfactual reasoning to intervene in the visual and language attention layers of multimodal LLMs, aiming to mitigate hallucinations caused by modality priors and enhance alignment quality.
Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment
Pritam Sarkar (Queen's University), Tomas Pfister (Google Cloud AI Research)
CodeObject DetectionGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A data augmentation-based phrase-level alignment (DPA) method is proposed, which utilizes generated 'hallucination' and 'correct' response pairs to finely constrain the language generated by multimodal large models, significantly reducing object hallucination.
🎯 What it does: A Sharpness-Aware Fine-Tuning (SAFT) method is proposed within a pre-training-fine-tuning framework to reduce parameter interference in multi-task model fusion by finding flat local minima without the need for joint training.
Mitigating Spurious Correlations in Zero-Shot Multimodal Models
Shenyu Lu (Purdue University), Xiaoqian Wang (Purdue University)
CodeClassificationRecognitionImage TranslationTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: This paper proposes a text-prompt-based image embedding translation method (TIE) and its unlabeled version (TIE*), which eliminates pseudo-correlation and enhances group robustness in zero-shot Vision-Language models.
Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN
Pengxiang Li (Dalian University of Technology), Shiwei Liu (University of Oxford)
CodeTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper studies the phenomenon of deep inactivity in large language models and proposes the Mix-LN normalization technique, which balances gradients across all layers, thereby improving model pre-training and downstream performance.
Mixture Compressor for Mixture-of-Experts LLMs Gains More
Wei Huang (Beihang University), XIAOJUAN QI
CodeCompressionOptimizationTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: This paper proposes an untrained mixed compression framework MC, which significantly reduces the storage and activation costs of expert parameters in MoE LLMs through pre-loaded mixed precision quantization and online dynamic pruning.
Mixture-of-Agents Enhances Large Language Model Capabilities
Junlin Wang (Duke University), James Zou (Stanford University)
CodeTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Proposes the Mixture-of-Agents (MoA) framework, which achieves iterative improvement of multiple models on the same task through collaborative reasoning and aggregation of multi-layer LLMs.
MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents
Junpeng Yue (Peking University), Zongqing Lu (Peking University)
CodeRetrievalRobotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningAgentic AIMultimodality
🎯 What it does: This paper proposes a trajectory retriever MART based on a multi-modal large language model (MLLM), which utilizes interactive feedback to generate preference pairs, fine-tunes the MLLM to evaluate trajectory effectiveness, and compresses trajectories through trajectory abstraction to reduce context window usage.
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
Chenxi Wang (Zhejiang University), Huajun Chen (Zhejiang University)
CodeRecognitionGenerationTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This paper analyzes the internal mechanism of hallucination generation in MLLM through experimental analysis and proposes a training-free dynamic correction decoding method, DeCo, to reduce hallucinations.
MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs
Jiarui Zhang (University of Southern California), Filip Ilievski (Vrije Universiteit Amsterdam)
CodeRecognitionObject DetectionComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This study investigates the performance bottlenecks of multimodal large language models in recognizing small visual details and proposes a training-free visual cropping method to enhance accuracy.
MM-EMBED: UNIVERSAL MULTIMODAL RETRIEVAL WITH MULTIMODAL LLMS
Sheng-Chieh Lin (University of Waterloo), Wei Ping (NVIDIA)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningTextMultimodality
🎯 What it does: This study investigates the construction of a general multimodal retriever using a multimodal large language model (MLLM), enhancing retrieval performance through modality-aware hard negative sample mining, continuous fine-tuning, and zero-shot prompt-based re-ranking.
🎯 What it does: This paper proposes a lightweight joint guidance module that utilizes pre-trained unimodal diffusion models (audio and video) to collaboratively generate aligned audio-video pairs without the need for backpropagation on the base model, thus maintaining the model's transferability.
MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?
YiFan Zhang, Rong Jin (Meta AI)
CodeRecognitionAutonomous DrivingTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmark
🎯 What it does: Two fully manually annotated high-resolution multimodal large model evaluation benchmarks, MME-RealWorld and MME-RealWorld-CN, have been proposed, covering 5 real-world scenarios and including 43 sub-tasks, with a total of 29,429 question-answer pairs.
MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models
Peng Xia (University of North Carolina at Chapel Hill), Huaxiu Yao (Stanford University)
CodeGenerationRetrievalConvolutional Neural NetworkVision Language ModelMultimodalityBiomedical DataRetrieval-Augmented Generation
🎯 What it does: This paper proposes MMed-RAG, a multimodal retrieval-augmented generation system aimed at reducing factual hallucinations in medical text generation.
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageVideoMultimodalityPoint CloudBenchmark
🎯 What it does: This paper proposes the MMIU benchmark, which systematically evaluates the performance of large visual language models in multi-image understanding tasks.
MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation
Donggon Jang (KAIST), Daeshik Kim (KAIST)
CodeObject DetectionSegmentationLarge Language ModelSupervised Fine-TuningImageBenchmark
🎯 What it does: A multi-target, multi-granularity (object-level and part-level) reasoning segmentation dataset MMR with 194K entries was constructed, and a multi-target, multi-granularity reasoning segmentation auxiliary model M-SA was proposed;
MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents
Yanqi Dai (Renmin University of China), Zhiwu Lu (Renmin University of China)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: Proposed and implemented the concept and framework of Multi-modal Role-playing Agents (MRPA) with MMRole, constructed the MMRole-Data dataset, and trained the first dedicated MRPA model, MMRole-Agent.
🎯 What it does: This paper created the MMTEB benchmark through large-scale community collaboration, covering over 500 tasks and more than 250 languages, achieving a comprehensive and multilingual evaluation of text embedding models.
Modality-Specialized Synergizers for Interleaved Vision-Language Generalists
Zhiyang Xu (Virginia Tech), Lifu Huang (University of California)
CodeGenerationData SynthesisTransformerSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: Proposed the Modal Specialization Coordinator (MOSS) and the large-scale interactive instruction dataset LEAFINSTRUCT, improving the interleaved generation of multimodal text and images;
Model Equality Testing: Which Model is this API Serving?
Irena Gao (Stanford University), Carlos Guestrin (Stanford University)
CodeLarge Language ModelPrompt EngineeringText
🎯 What it does: A Model Equality Testing framework is proposed, utilizing two-sample testing methods to evaluate whether the output distribution of a black-box LLM API is consistent with that of a reference model.
🎯 What it does: The KnOTS method is proposed, which improves multi-task model merging by aligning LoRA task updates using SVD, and provides a new benchmark for evaluating general models.
Model-Agnostic Knowledge Guided Correction for Improved Neural Surrogate Rollout
Bharat Srikishan (Stevens Institute of Technology), Nikhil Muralidhar (Stevens Institute of Technology)
CodeOptimizationConvolutional Neural NetworkReinforcement LearningSequentialPhysics Related
🎯 What it does: A model-agnostic and simulator-agnostic hybrid method called HyPER is proposed, which intelligently invokes high-cost physical simulators using reinforcement learning to correct autoregressive predictions when errors occur in neural approximation models, significantly reducing cyclical errors.
Model-agnostic meta-learners for estimating heterogeneous treatment effects over time
Dennis Frauen (LMU Munich), Stefan Feuerriegel (LMU Munich)
CodeMeta LearningTransformerTime SeriesBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes various model-agnostic meta-learners for estimating heterogeneous treatment effects (HTE) that change over time, implemented on any machine learning model (such as transformers).
Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning
Baoqi Pei (Zhejiang University), Limin Wang (Zhejiang University)
CodeObject DetectionRepresentation LearningRobotic IntelligenceTransformerLarge Language ModelContrastive LearningVideo
🎯 What it does: Proposed the HOD data generation pipeline and EgoVideo model, combining fine-grained dynamic learning of hand-object interactions to enhance egocentric video representation.
MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
Peng Jin (Peking University), Shuicheng YAN
CodeTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: Proposes the MoE++ framework, which integrates traditional FFN experts with three types of zero-computation experts (Zero, Copy, Constant) to achieve adaptive routing;
🎯 What it does: This paper presents MOFFLOW, a deep generative model that utilizes Riemannian flow matching specifically for predicting the crystal structures of metal-organic frameworks (MOFs);
MoLEx: Mixture of Layer Experts for Fine-tuning with Sparse Upcycling
Rachel Teo, Tan Minh Nguyen
CodeTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsTextBenchmark
🎯 What it does: A Mixture of Layer Experts (MoLEx) method is proposed, which achieves the reuse and exchange of hierarchical information by treating model layers as experts and implementing sparse routing during the fine-tuning of large-scale language models.
MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectra
Liang Wang (Chinese Academy of Sciences)
CodeRepresentation LearningDrug DiscoveryTransformerContrastive LearningMultimodalityGraphPhysics Related
🎯 What it does: Using quantum mechanical energy spectrum information for pre-training 3D molecular representations, the MolSpectra framework is proposed, which includes the SpecFormer multi-spectrum encoder, a mask completion objective, and a contrastive learning strategy.
🎯 What it does: An unsupervised method named Moner is proposed, which utilizes implicit neural representations to reconstruct high-quality MR images without motion artifacts from undersampled radial MRI k-space, without relying on external training data.
Monet: Mixture of Monosemantic Experts for Transformers
Jungwoo Park (Korea University), Jaewoo Kang (Korea University)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: We propose MONET, a sparse mixture of experts Transformer architecture that can scale to 262,144 experts at each layer, aimed at enhancing the interpretability and controllability of LLMs.
Monitoring Latent World States in Language Models with Propositional Probes
Jiahai Feng (University of California Berkeley), Jacob Steinhardt (University of California Berkeley)
CodeTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: The researchers proposed a proposition probe that can decode logical propositions from the internal activations of language models and achieve semantic combinations through binding subspaces.
Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning
Xiaochuan Li (Tsinghua University), Chenyan Xiong (Carnegie Mellon University)
CodeData SynthesisOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A data synthesis framework named MONTESSORI-INSTRUCT is designed to generate synthetic data that aligns with students' learning preferences by optimizing the parameters of a teacher model, thereby enhancing the performance of the student model.
Elizaveta Tennant (University College London), Mirco Musolesi (University College London)
CodeTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper trains LLM agents in the Iterated Prisoner's Dilemma (IPD) using PPO for RL fine-tuning on Gemma2-2b-it, leveraging predefined intrinsic moral rewards (deontological ethics, utilitarianism, and game rewards), and further evaluates their performance in other matrix games as well as their ability to 'forget' existing selfish strategies.
More Experts Than Galaxies: Conditionally-Overlapping Experts with Biologically-Inspired Fixed Routing
Sagi Shaier (University of Colorado Boulder), Matt Jones
CodeClassificationOptimizationTransformerMixture of ExpertsImageText
🎯 What it does: A sparse neural network method named COMET is proposed, which generates input-related sparse masks using fixed random projections and k-winner-take-all, thereby forming exponentially overlapping experts within the network.
More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness
Aaron Jiaxun Li, Himabindu Lakkaraju
CodeReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: The system evaluated the effects of RLHF on five trust indicators and proposed a data attribution method based on influence functions to explain the results.
🎯 What it does: This paper proposes a Dynamic Token Morphing (DTM) method that addresses spatial inconsistency in MIM by aggregating context while maintaining the number of tokens, thereby enhancing the pre-training performance of ViT.
🎯 What it does: This paper proposes the MOS (Model Synergy) framework for online test-time adaptation (TTA) of LiDAR-based 3D object detection, constructing a super model by dynamically selecting and weighting combinations of historical checkpoints, and performing single-step fine-tuning on the current batch using self-supervised pseudo-labels.
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards
Sheng Wang (University of Hong Kong), Chuan Wu (University of Hong Kong)
CodeTransformerSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: This paper proposes a Mixture of Shards (MOS) method, which achieves significant parameter-efficient fine-tuning by combining global sharing with various nearly cost-free differentiation strategies in the LoRA model.
🎯 What it does: A Transformer-based super network called MotherNet is proposed, which can directly generate a small MLP for tabular classification in a single forward pass based on the training set.
MotionClone: Training-Free Motion Cloning for Controllable Video Generation
Pengyang Ling (University of Science and Technology of China), Yi Jin (University of Science and Technology of China)
CodeGenerationData SynthesisDiffusion modelVideo
🎯 What it does: We propose MotionClone, a training-free video motion cloning framework that can transfer motion from a reference video to a new scene, supporting text-to-video and image-to-video.
mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models
Jiabo Ye (Alibaba Group), Jingren Zhou (Alibaba Group)
CodeRecognitionGenerationComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageVideoTextMultimodality
🎯 What it does: Proposes the mPLUG-Owl3 multimodal large language model, which supports long image sequences and video understanding, achieving efficient inference on multimodal tasks.
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
Julie Kallini (Stanford University), Róbert Csordás (Stanford University)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A byte-level language model named MrT5 is proposed, which compresses sequence length by dynamically removing unimportant bytes through the insertion of learnable deletion gates in fixed layers of the encoder.
🎯 What it does: In MTSAM, the authors restructured the SAM architecture by removing the prompt encoder and adding task embeddings, allowing the model to output multiple tasks at once with variable channel numbers, and employed the low-rank tensor decomposition method ToRA for efficient multi-task fine-tuning;
MTU-Bench: A Multi-granularity Tool-Use Benchmark for Large Language Models
Pei Wang (Alibaba Group), Bo Zheng (University of Chinese Academy of Sciences)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the MTU-Bench multi-granularity tool usage benchmark, which includes the training set MTU-Instruct and the evaluation set MTU-Eval, covering single/multiple rounds, single/multiple tools, and OOD scenarios, and provides automatic evaluation metrics that do not require GPT.
Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM
Zheng Wei Lim (University of Melbourne), Trevor Cohn (Google)
CodeKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Using large-scale language models (LLM) to generate multilingual auxiliary translation candidates and integrating them with post-editing tasks to improve the translation quality of low-resource languages.
🎯 What it does: This study investigates a conformal prediction method that utilizes multi-dimensional nonconformity measures, generating multi-dimensional scores through self-ensemble and partitioning cells in a high-dimensional score space to form deformable coverage regions.
Multi-domain Distribution Learning for De Novo Drug Design
Arne Schneuing (Ecole Polytechnique Federale de Lausanne), Bruno Correia (Ecole Polytechnique Federale de Lausanne)
CodeDrug DiscoveryFlow-based ModelGraph
🎯 What it does: A new structured drug design generation model DRUGFLOW (and its extension FLEXFLOW) has been developed, which can simultaneously learn ligand atomic coordinates, atomic types, bond types, and protein side chain conformations, and can adaptively adjust ligand size during the generation process.
Millicent Li (Northeastern University), Patrick Xia (Microsoft)
CodeRetrievalContrastive LearningText
🎯 What it does: A multi-field adaptive retrieval framework (MFAR) is proposed, which can split semi-structured documents into fields and use lexical retrieval (BM25) and dense retrieval (Contriever) separately, and then adaptively weight and fuse the scores of each field based on query conditions.
Multi-Label Node Classification with Label Influence Propagation
Yifei Sun (Zhejiang University), Bingsheng He (Capital One)
CodeClassificationGraph Neural NetworkGraphBiomedical Data
🎯 What it does: This paper proposes a multi-label node classification method based on label influence propagation, LIP, which enhances multi-label node classification performance by analyzing and utilizing the positive and negative influences between labels in the graph.
Multi-Label Test-Time Adaptation with Bound Entropy Minimization
Xiangyu Wu (Nanjing University of Science and Technology), Jianfeng Lu (Nanjing University of Science and Technology)
CodeObject DetectionDomain AdaptationPrompt EngineeringVision Language ModelImage
🎯 What it does: A multi-label test-time adaptation framework ML-TTA is proposed, which utilizes boundary entropy minimization (BEM) and text titles as pseudo-images to simultaneously enhance the confidence of multi-label predictions.
🎯 What it does: Using cross-modal and joint pre-trained Transformer models for brain encoding under multimodal stimuli (film + audio), comparing their alignment performance with unimodal models;
Rhea Sanjay Sukthanker (University of Freiburg), Frank Hutter (University of Technology Nuremberg)
CodeOptimizationNeural Architecture SearchImage
🎯 What it does: A differentiable neural architecture search algorithm is proposed, capable of generating a complete Pareto front for multiple devices and multiple objectives (such as accuracy, latency, and energy consumption) in a single search process;
Multi-Reward as Condition for Instruction-based Image Editing
Xin Gu (ByteDance Inc), Sijie Zhu (ByteDance Inc)
CodeImage TranslationGenerationTransformerLarge Language ModelDiffusion modelImageTextBenchmark
🎯 What it does: This paper improves the training quality of instruction-based image editing models by introducing multi-view reward data as an additional condition and constructs the Real-Edit evaluation benchmark.
🎯 What it does: Designed and implemented a multi-scale fusion (MSF) technique that utilizes image pyramids and cross-scale, intra-scale quantization fusion to enhance the quality of object representations in VAE-guided Object Center Learning (OCL).
🎯 What it does: A self-supervised multimodal speech representation learning framework called CAV2vec is proposed, specifically designed for robust training against audio-visual joint distortion.
🎯 What it does: This paper proposes TaskDiffusion—a multi-task dense prediction framework based on a joint denoising diffusion process, which achieves multi-task collaborative inference through cross-task label encoding and a cross-task diffusion decoder.
Multilevel Generative Samplers for Investigating Critical Phenomena
Ankur Singha (BIFOLD), Shinichi Nakajima (BIFOLD)
CodeGenerationData SynthesisOptimizationSequentialPhysics Related
🎯 What it does: This paper proposes the Renormalization-informed Generative Critical Sampler (RiGCS), which introduces a conditionally generative model with adjustable receptive fields within the multi-layer Monte Carlo + Heat Bath (MLMC-HB) framework to efficiently sample critical two-dimensional Ising systems while capturing long-range and higher-order interactions under scale invariance (SIC).
Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning
Gang Liu (University of Notre Dame), Jie Chen (IBM Research)
CodeOptimizationDrug DiscoveryGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextMultimodalityGraph
🎯 What it does: A multi-modal large language model called Llamole is proposed, capable of alternately generating between text and molecular graphs, achieving controllable reverse molecular design and retrosynthetic planning.
CodeRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderTextMultimodality
🎯 What it does: This study proposes a generative recommendation framework MQL4GRec that unifies multimodal product information (text, images) into a quantitative language mapping, and achieves cross-domain multimodal knowledge transfer through pre-training and fine-tuning.
🎯 What it does: This paper proposes a multimodal unsupervised domain generalization framework MUDG, which retrieves source data in the CLIP visual-text space and pseudo-labels it, followed by diversified sampling and fine-tuning of the retrieved data, thereby achieving better cross-domain performance on unlabeled tasks.
CodeRepresentation LearningMixture of ExpertsMultimodality
🎯 What it does: Proposes the MOMOK framework, which uses relation-guided multimodal knowledge experts (ReMoKE) to learn multi-perspective, relation-aware modality embeddings, and achieves better multimodal knowledge graph completion (MMKGC) through multimodal joint decision-making (MuJoD) and expert information decoupling (ExID).
Naoya Hasegawa (University of Tokyo), Issei Sato (University of Tokyo)
CodeClassificationRecognitionImageTabular
🎯 What it does: Proves that the Multiple Logarithmic Adjustment (MLA) is approximately the optimal decision boundary through Neural Collapse (NC) theory, and validates its effectiveness in long-tail recognition.
🎯 What it does: Systematically evaluate the 3D equivariance of ViT and enhance 3D correspondence through fine-tuning with multi-view alignment loss, significantly improving the performance of tasks such as pose estimation, video tracking, and semantic correspondence.
MuPT: A Generative Symbolic Music Pretrained Transformer
Xingwei Qu (M-A-P), Ge Zhang
CodeGenerationTransformerLarge Language ModelAudio
🎯 What it does: A long-context pre-trained music generation model MuPT based on ABC Notation is proposed, introducing the Synchronized Multi-Track ABC (SMT-ABC) representation and SMS scale law, providing a scalable LLM solution for symbolic music generation.
MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy
Haitian Jiang (New York University), David Wipf (Amazon Web Services)
CodeGraph Neural NetworkGraph
🎯 What it does: This paper presents MuseGNN, a scalable unfolded graph neural network that achieves large-scale graph training by embedding offline subgraph sampling into the energy function.
Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solver
Zhenting Qi (Microsoft Research Asia), Mao Yang (Microsoft Research Asia)
CodeTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: This paper presents rStar, a generative-discriminative process that enhances reasoning performance in small language models (SLMs) through self-play mutual reasoning, without the need for fine-tuning or stronger models.
N-ForGOT: Towards Not-forgetting and Generalization of Open Temporal Graph Learning
Liping Wang (Hong Kong University of Science and Technology), Lei Chen (Hong Kong University of Science and Technology)
CodeOptimizationRepresentation LearningGraph Neural NetworkGraphTime Series
🎯 What it does: Proposes the N-ForGOT framework, which provides a pluggable two-module solution to address the issues of catastrophic forgetting and distribution drift in Open Temporal Graph Learning (OTGL);
Narrowing Information Bottleneck Theory for Multimodal Image-Text Representations Interpretability
Zhiyu Zhu (University of Technology Sydney), Fang Chen (University of Technology Sydney)
CodeExplainability and InterpretabilityTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: In response to the interpretability issues of multimodal image-text models like CLIP, a new Narrowing Information Bottleneck Theory (NIBT) is proposed, which achieves an explanation method that is free of randomness, hyperparameters, and can distinguish negative attributes.
Frederik Pahde (Fraunhofer Heinrich Hertz Institute), Sebastian Lapuschkin (Fraunhofer Heinrich Hertz Institute)
CodeExplainability and InterpretabilityImageBiomedical Data
🎯 What it does: A concept activation vector method based on patterns (Pattern-CAV) is proposed to accurately capture the true direction of concepts in neural networks, addressing the directional bias issue of traditional filter-based methods (Filter-CAV).
Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving
Peidong Li (Zhijia Technology), Dixiao Cui (Zhijia Technology)
CodeAutonomous DrivingTransformerPoint Cloud
🎯 What it does: The SSR framework is proposed, utilizing only 16 navigation-guided sparse BEV tokens to achieve end-to-end autonomous driving perception and planning, eliminating the reliance on traditional perception tasks.
NetFormer: An interpretable model for recovering dynamical connectivity in neuronal population dynamics
Ziyu Lu (University of Washington), Lu Mi (Georgia Institute of Technology)
CodeExplainability and InterpretabilityTransformerMultimodalityTime Series
🎯 What it does: This paper proposes an interpretable model called NetFormer based on a linearized attention mechanism, aimed at recovering time-varying (non-stationary) connectivity structures from neuronal activity sequences.
🎯 What it does: This paper proposes a method for learning mirror mappings using neural networks (NAMM), enabling diffusion models to satisfy arbitrary differentiable constraints while maintaining generation quality.
🎯 What it does: Proposes to estimate scene flow as fitting an ODE for the entire observation sequence and implements an unsupervised method called EulerFlow;