Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering
Jie Ma (Xi'an Jiaotong University), Youtian Du (Xi'an Jiaotong University)
CodeClassificationRecognitionData-Centric LearningConvolutional Neural NetworkVision Language ModelVideoMultimodalityAudio
π― What it does: A new dataset for audio-visual question answering (AVQA) called MUSIC-AVQA-R is proposed, along with a multi-view cyclic collaborative debiasing strategy (MCCD) designed to enhance the robustness of the model.
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate
Anthony Fuller (Carleton University), James R Green
CodeImage TranslationSegmentationTransformerImage
π― What it does: A LookHere positional encoding based on a 2D directional attention mask is proposed, enabling the Vision Transformer to directly infer on higher resolution images without fine-tuning.
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The LoQT method is proposed, which combines low-rank adapters with 4-bit NF4 quantization, enabling efficient pre-training and fine-tuning of large models on a single 24GB GPU, significantly reducing memory usage.
π― What it does: A score calculation method based on Reduced-Rank Regression (RRR) is proposed, improving the query speed and memory usage of clustering-based Approximate Nearest Neighbor (ANN) search, and implementing the open-source library LoRANN.
LOVA3: Learning to Visual Question Answering, Asking and Assessment
Hengyuan Zhao, Mike Zheng Shou (National University of Singapore)
CodeTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
π― What it does: This paper proposes the LOVA 3 framework, which trains multimodal large language models to have the ability to answer, ask questions, and evaluate visual questions.
π― What it does: In federated learning, the authors propose an efficient paradigm: using low precision (e.g., 8-bit) operations and quantization for local training on the client side, and uploading the low precision model to the server; the server only uses high precision computation during aggregation and recovers the high precision global model through a moving average method.
π― What it does: Based on Lumina-T2X, Lumina-Next is proposed, improving architecture, context extrapolation, and sampling techniques to achieve stronger and faster zero-copy text-to-image, multimodal, and multilingual generation.
MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems
Bin Lei (University of Minnesota), Caiwen Ding (University of Minnesota)
CodeTransformerLarge Language ModelAgentic AIPrompt EngineeringTextChain-of-Thought
π― What it does: A multi-agent conditional mining (MACM) prompting framework is proposed and implemented to enhance the inference capability of large language models in complex mathematical reasoning tasks.
Magnet: We Never Know How Text-to-Image Diffusion Models Work, Until We Learn How Vision-Language Models Function
Chenyi Zhuang (Nanjing University of Aeronautics and Astronautics), Pan Gao (Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education)
CodeGenerationTransformerVision Language ModelDiffusion modelImageText
π― What it does: This study investigates the impact of the CLIP text encoder on attribute binding and proposes a zero-training Magnet method to correct text embeddings, enhancing Stable Diffusion's attribute binding and image quality under complex prompts.
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
Aozhong Zhang (University at Albany), Penghang Yin (University at Albany)
CodeOptimizationTransformerLarge Language ModelText
π― What it does: A preprocessing method called MagR based on Ξ±-β regularization is proposed to reduce the maximum magnitude of pre-trained LLM weights, thereby improving the effectiveness of post-training quantization (PTQ).
Ang Bian (Sichuan University), Tao Feng (Tsinghua University)
CodeOptimizationImage
π― What it does: A pluggable C-Flat optimizer is proposed, which combines zero-order and first-order loss surface flattening to enhance the generalization and anti-forgetting capabilities of continuous learning models.
CodeExplainability and InterpretabilityTransformerImageText
π― What it does: This paper proposes a Layer-wise Relevance Propagation method for the Mamba architecture (MambaLRP) to generate interpretable and reliable explanations for sequence model predictions.
π― What it does: This paper proposes a video compression imaging (SCI) reconstruction method called MambaSCI for the fourfold Bayer pattern, addressing the color distortion and desaturation issues caused by traditional Bayer schemes under fourfold Bayer sensors.
π― What it does: A tree state space model (Tree SSM) and the MambaTree framework are proposed, utilizing input-aware tree topology to achieve feature propagation, thereby breaking the sequential limitations of traditional SSMs and enhancing the modeling capability for long-range dependencies.
π― What it does: A multi-hypothesis 3D human pose reconstruction model called ManiPose is proposed, based on pose manifold constraints, to address the pose inconsistency problem caused by monocular depth ambiguity.
MaNo: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts
RENCHUNZI XIE, Bo An (Skywork AI)
CodeDomain AdaptationImage
π― What it does: An unsupervised model accuracy estimation method called MANO is proposed, which measures the performance of OOV samples using the Lp norm of the model output matrix.
Markov Equivalence and Consistency in Differentiable Structure Learning
Chang Deng (University of Chicago), Bryon Aragam (Carnegie Mellon University)
CodeGraph
π― What it does: This paper proposes a new differentiable structure learning scoring function that combines log-likelihood with non-convex regularization (MCP);
π― What it does: A new Markov Chain Monte Carlo (MCMC) sampling method based on Continuous Normalizing Flows (CNF) is proposed, called Markovian Flow Matching (MFM), which accelerates sampling by adaptively training the CNF during the sampling process and combining local gradient proposals with globally flow-driven proposals.
π― What it does: This paper combines causal representation learning (CRL) with dynamic system identification to propose an identifiable neural simulator (identifier). By transforming the parameter estimation problem into a latent variable identification problem in CRL, it achieves identifiable and efficient prediction of specific parameters for time-invariant trajectories.
π― What it does: A zero-shot image denoising framework based on mask pre-training and iterative filling (MPI) is proposed, which utilizes the knowledge of general image distribution obtained from training with random pixel masks on ImageNet, and achieves high-quality denoising through iterative filling on a single noisy image.
MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
Gongfan Fang (National University of Singapore), Xinchao Wang (National University of Singapore)
CodeTransformerLarge Language ModelText
π― What it does: By learning differentiable semi-structured (N:M) sparse masks, sparsification is achieved on large language models (LLMs) while retaining the performance of the original model;
π― What it does: A model-aware data selection framework MATES is proposed, which dynamically captures the model's preferences during pre-training and selects the most beneficial data based on data influence;
π― What it does: A framework based on maximum entropy inverse reinforcement learning (IRL) called DxMI is proposed to improve sample quality in few-step diffusion model generation, and energy learning is achieved without MCMC through joint training of the energy-based model (EBM).
MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making
Yubin Kim (Massachusetts Institute of Technology), Hae Won Park (Massachusetts Institute of Technology)
CodeTransformerLarge Language ModelImageVideoTextMultimodalityBiomedical DataRetrieval-Augmented GenerationChain-of-Thought
π― What it does: An adaptive multi-agent framework called MDAgents is proposed, which utilizes large language models (LLMs) to dynamically construct collaborative structures for individuals, multidisciplinary teams (MDTs), and integrated teams (ICTs) in the medical decision-making process.
Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance
Kai Xiong (Harbin Institute of Technology), Yixin Cao (Fudan University)
CodeTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
π― What it does: This study investigates the abstract reasoning capabilities of large language models (LLMs), proposes a new evaluation metric called AbsAcc, constructs an abstract reasoning dataset based on general facts named AbsR, and designs a 'Mean Learning' (MeanLearn) paradigm that enables LLMs to implicitly learn and apply general facts through both knowledge and reasoning, thereby improving their performance on multiple reasoning and language understanding benchmarks.
Matt MacDermott (Imperial College London), Tom Everitt (Google DeepMind)
CodeOptimizationReinforcement LearningTabular
π― What it does: This paper proposes and implements the Maximum Entropy Goal Orientation (MEG) as a formal metric to measure the degree of goal orientation in causal models (CBN, CID) and Markov Decision Processes (MDP), and provides corresponding computational algorithms.
π― What it does: This paper proposes MADPO, a multi-agent reinforcement learning framework that enhances heterogeneous task exploration by maximizing mutual policy differences under sequential updates.
Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models
Adam Karvonen (Independent), Samuel Marks (Northeastern University)
CodeExplainability and InterpretabilitySupervised Fine-TuningAuto EncoderText
π― What it does: This paper proposes two new evaluation metricsβcoverage and board reconstructionβby training a sparse autoencoder (SAE) on language models of board games (chess and Go). It also introduces the p-annealing training technique, aiming to objectively measure the interpretability and information capture capability of the SAE.
π― What it does: This paper proposes the Multi-Event Causal Discovery (MECD) task and dataset, and designs a Granger-causal based multimodal model VGCM to construct a complete event-level causal graph from long videos.
π― What it does: This study proposes a multi-granularity block Transformer named Medformer, specifically designed for the classification of medical time series data, combining cross-channel blocking, granularity embedding, and a two-stage self-attention mechanism;
MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
Shuyue Stella Li (University of Washington), Yulia Tsvetkov (University of Washington)
CodeTransformerLarge Language ModelPrompt EngineeringTextBiomedical DataBenchmark
π― What it does: An interactive medical diagnosis evaluation framework, MEDIQ, is proposed, simulating the question-and-answer process between doctors and patients, focusing on the active questioning and information gathering capabilities of LLMs.
Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration
Wenjie Fu (Huazhong University of Science and Technology), Tao Jiang (Huazhong University of Science and Technology)
CodeAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper proposes a membership inference attack (MIA) for fine-tuning large language models (LLMs), which generates reference data through self-prompting and utilizes probability variance metrics to identify training records.
π― What it does: Proposes the MeMo framework, which utilizes a single robot training module to create a modular controller that can be quickly transferred to new robots composed of the same assembly parts;
Memorize What Matters: Emergent Scene Decomposition from Multitraverse
Yiming Li (New York University), Jose M. Alvarez (NVIDIA)
CodeSegmentationAutonomous DrivingKnowledge DistillationRobotic IntelligenceGaussian SplattingSimultaneous Localization and MappingVideoBenchmark
π― What it does: This paper proposes a 3D Gaussian Mapping (3DGM) framework that utilizes RGB videos from multiple driving perspectives for self-supervised 3D environment mapping (EnvGS) and 2D instantaneous object segmentation (EmerSeg), achieving robot memory and localization capabilities without LiDAR and manual annotations.
Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization
Qianli Shen (National University of Singapore), Kenji Kawaguchi (National University of Singapore)
CodeOptimizationKnowledge DistillationLarge Language ModelImageText
π― What it does: A novel algorithm named (FG)UΒ² is proposed, which combines forward gradient expansion and forward gradient methods to address memory and approximation issues in large-scale bilevel optimization.
Memory-Efficient LLM Training with Online Subspace Descent
Kaizhao Liang (University of Texas at Austin), qiang liu
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: An Online Subspace Descent algorithm is proposed, which updates the projection matrix using online PCA, avoiding the SVD computation in traditional low-rank training methods, thus achieving memory-efficient LLM pre-training.
MemVLT: Vision-Language Tracking with Adaptive Memory-based Prompts
Xiaokun Feng (University of Chinese Academy of Sciences), Kaiqi Huang (University of Chinese Academy of Sciences)
CodeObject TrackingTransformerVision Language ModelMultimodality
π― What it does: A MemVLT visual-language tracker is proposed, which utilizes a memory mechanism to adaptively generate multimodal prompts for tracking targets that change dynamically over time.
MeshXL: Neural Coordinate Field for Generative 3D Foundation Models
Sijin Chen (Tencent PCG), Tao Chen (ShanghaiTech University)
CodeGenerationData SynthesisTransformerLarge Language ModelMultimodalityMesh
π― What it does: The study proposes MeshXL, which utilizes Neural Coordinate Field (NeurCF) to map explicit coordinates of 3D meshes to implicit neural embeddings, and directly generates high-quality meshes through an autoregressive Transformer (an OPT variant);
π― What it does: This paper proposes a Transformer-based few-shot behavior cloning framework that can generalize to unseen robot shapes and tasks with only five reward-free demonstrations provided.
π― What it does: This paper proposes Meta-DT, a Transformer-based offline Meta-RL framework that utilizes a world model to decompose task information and achieve task generalization through self-guided complementary prompts.
Meta-Learning Universal Priors Using Non-Injective Change of Variables
Yilang Zhang (University of Minnesota), Georgios B. Giannakis (University of Minnesota)
CodeMeta LearningImage
π― What it does: This paper proposes a method that utilizes non-injective transformations (NCoV) to learn a more expressive prior distribution, thereby enhancing the performance of meta-learning in few-shot tasks.
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
Yuhong Chou (Hong Kong Polytechnic University), Guoqi Li (Institute of Automation Chinese Academy of Sciences)
CodeRetrievalOptimizationTransformerImageText
π― What it does: This paper studies and proposes MetaLA, unifying LinFormer/SSM/LinRNN into a general linear attention mechanism, and constructs a model that achieves the optimal linear approximation of softmax attention, validating its effectiveness across various tasks.
π― What it does: A meta-learning framework called MetaUAS is proposed, which can achieve general anomaly segmentation based on a single normal image prompt without relying on language prompts or target anomaly datasets.
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models
Byung-Kwan Lee (KAIST), Yong Man Ro (KAIST)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: An efficient LLVM model called Meteor is proposed, which integrates long-form multifaceted reasoning (rationale) through the Mamba architecture and a 'traversal reasoning' mechanism, achieving image understanding and multi-domain reasoning capabilities.
Metric Flow Matching for Smooth Interpolations on the Data Manifold
Kacper Kapusniak (University of Oxford), Francesco Di Giovanni (University of Oxford)
CodeData SynthesisOptimizationFlow-based ModelImagePoint CloudBiomedical Data
π― What it does: A data-driven Riemannian metric-based method for simulation-free conditional flow matching (Metric Flow Matching, MFM) is proposed, which generates more natural probability paths by minimizing the kinetic energy of the interpolation path in the metric, keeping the interpolation on the data manifold.
π― What it does: By generating human figures from input images during testing and using the real size of the human figures as a scale prior, unsupervised zero-shot monocular metric depth estimation is achieved.
π― What it does: This paper proposes a multi-scale diversity assessment method based on the size of metric spaces (magnitude) to measure the intrinsic diversity of potential representations and detect mode collapse and loss in generative models.
MG-Net: Learn to Customize QAOA with Circuit Depth Awareness
Yang Qian (University of Sydney), Dacheng Tao (Nanyang Technological University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: This paper proposes MG-Net, an end-to-end deep learning framework for dynamically generating optimal mixed Hamiltonians for a given problem and circuit depth, thereby improving the approximation ratio of QAOA.
π― What it does: This paper proposes an adaptive optimizer named MICROADAM, which significantly reduces the memory footprint of the optimizer's state by compressing before the gradient enters the Adam state and using an error feedback mechanism, while maintaining theoretical convergence guarantees.
π― What it does: This paper proposes the MimicTalk framework, which utilizes a pre-trained 3D crowd-agnostic NeRF model for personalized talking face generation. It achieves high-quality, fast, and expressive video generation through static-dynamic hybrid adaptation and context-stylized audio to motion model.
π― What it does: For cross-domain few-shot classification, we propose the Contrastive Prototype-Image Adaptation (CoPA) method, which learns independent representation transformations for prototypes and image instances on a frozen pre-trained backbone.
π― What it does: By mapping the representations of multilingual pre-trained models to the representation space of large language models (LLMs) and incrementally fusing the input based on this, MindMerger is proposed to enhance the performance of LLMs in non-English multilingual reasoning and language understanding tasks.
Mini-Sequence Transformers: Optimizing Intermediate Memory for Long Sequences Training
Cheng Luo (California Institute of Technology), Anima Anandkumar (California Institute of Technology)
CodeTransformerLarge Language ModelText
π― What it does: By splitting the MLP and LM-Head modules of the Transformer into mini-sequences, the memory usage of intermediate activations is significantly reduced, enabling the training of very long sequences of LLM on a single GPU.
Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization
Zheyi Fan (Academy of Mathematics and Systems Science, Chinese Academy of Sciences), Qingpei Hu (Academy of Mathematics and Systems Science, Chinese Academy of Sciences)
CodeOptimizationTabular
π― What it does: This paper proposes two local Bayesian optimization algorithms based on Gaussian processes: MinUCB and LA-MinUCB, which achieve more efficient local search by minimizing UCB.
π― What it does: This paper studies the extension of mirror descent and preconditioned gradient descent in Wasserstein space, provides discrete-time convergence theory, and verifies convergence on various objective functions. It subsequently demonstrates significant acceleration effects through preconditioned gradient descent on single-cell biology datasets.
π― What it does: A method for actively defending against backdoor attacks is proposed, which involves injecting a defensive backdoor (PDB) during the training phase.
π― What it does: A method called DrilDICE is proposed for offline imitation learning, designed to address the covariate shift problem where expert demonstration data deviates from the stationary distribution in state distribution, by creating a distributionally robust behavior cloning objective.
Mitigating Object Hallucination via Concentric Causal Attention
Yun Xing (Nanyang Technological University), Shijian Lu (Nanyang Technological University)
CodeRecognitionObject DetectionTransformerVision Language ModelImageTextMultimodality
π― What it does: This paper proposes the Concentric Causal Attention (CCA) method, which reduces the relative distance between visual tokens and instruction tokens by reordering visual tokens into concentric hierarchies and rewriting the attention mask, thereby suppressing object hallucinations in large visual language models (LVLMs);
π― What it does: Proposes the Ξ»-difference metric to detect and mitigate partial observability in sequential decision-making, incorporating it as an auxiliary loss in Recurrent PPO.
π― What it does: This paper proposes a method based on Adversarial Meta-Tuning (AMT), which enhances robust generalization on out-of-distribution (OOD) tasks by introducing low-rank adapters (LoRA) on a pre-trained Vision Transformer and applying adversarial perturbations on query samples and their singular values/vectors.
Mixture of Experts Meets Prompt-Based Continual Learning
Minh Le (VinAI Research), Nhat Ho (University of Texas at Austin)
CodeTransformerPrompt EngineeringMixture of ExpertsImage
π― What it does: This paper studies the relationship between prefix tuning and self-attention, viewing it as a special type of expert model, and based on this, proposes a Nonlinear Residual Gate (NoRGa) to enhance the performance of prompt-based continual learning.
Mixture of In-Context Experts Enhance LLMs' Long Context Awareness
Hongzhan Lin (Renmin University of China), Rui Yan (Renmin University of China)
CodeTransformerLarge Language ModelMixture of ExpertsTextBenchmark
π― What it does: This paper proposes the Mixture of In-Context Experts (MoICE) plugin, which enhances the LLM's ability to perceive long text contexts by dynamically selecting multiple RoPE angles (considered as experts) through routers added to each attention head.
Li Ma (Shanghai Jiao Tong University), Jiliang Tang (Michigan State University)
CodeGraph Neural NetworkMixture of ExpertsGraph
π― What it does: This paper proposes a hybrid expert model for graph link prediction called Link-MoE, which can dynamically select the most suitable GNN predictor based on various heuristic features of node pairs.
CodeRecognitionSegmentationTransformerMixture of ExpertsMultimodalityAudio
π― What it does: Proposes AVMoE - an audio-visual learning framework that injects unimodal and cross-modal Adapters into a frozen pre-trained model and dynamically allocates weights through Mixture of Experts (MoE).
MMSite: A Multi-modal Framework for the Identification of Active Sites in Proteins
Song Ouyang (Wuhan University), Bo Du (Wuhan University)
CodeRecognitionProtein Structure PredictionTransformerPrompt EngineeringMultimodalityBiomedical Data
π― What it does: A multi-modal framework called MMSite is proposed, which jointly identifies protein active sites using protein sequences and multi-attribute text descriptions.
Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration
Junyang Wang (Beijing Jiaotong University), Jitao Sang (Beijing Jiaotong University)
CodeLarge Language ModelAgentic AIVision Language ModelTextMultimodality
π― What it does: A multi-agent architecture called Mobile-Agent-v2 has been designed and implemented to perform complex UI operation tasks on mobile devices, with three main agentsβplanning, decision-making, and reflectionβcollaborating to navigate task progress and focal content.
Mobility-LLM: Learning Visiting Intentions and Travel Preference from Human Mobility Data with Large Language Models
Letian Gong (Beijing Jiaotong University), Huaiyu Wan (Beijing Jiaotong University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTime SeriesSequential
π― What it does: A unified framework called Mobility-LLM is proposed, which utilizes large language models (LLM) to learn from check-in sequences, extracting access intentions and travel preferences to accomplish three types of tasks: location prediction, trajectory user association, and time prediction.
Model Based Inference of Synaptic Plasticity Rules
Yash Mehta (Howard Hughes Medical Institute), Jan Funke (Howard Hughes Medical Institute)
CodeTime Series
π― What it does: A gradient descent-based parameterized plasticity rule inference method has been developed, which can automatically learn synaptic plasticity functions from neural or behavioral trajectories and has been validated on simulations and fruit fly behavioral data.
Model Fusion through Bayesian Optimization in Language Model Fine-Tuning
Chaeyun Jang (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
CodeOptimizationHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a model fusion method based on Bayesian optimization (BOMF), which finely tunes model fusion weights and hyperparameters through multi-objective Bayesian optimization to enhance the performance of pre-trained language models on downstream tasks.
π― What it does: A framework named Model-based Transfer Learning (MBTL) is proposed, which actively selects training tasks using Bayesian optimization in Contextual Reinforcement Learning (CMDP) to enhance generalization performance.
π― What it does: A Gaussian Process-based Switching Linear Dynamical System (gpSLDS) is proposed and implemented to infer low-dimensional interpretable latent dynamics from high-dimensional neural signals.
MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks
Xingkui Zhu (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
CodeClassificationOptimizationComputational EfficiencyTransformerMixture of ExpertsImage
π― What it does: The MoE Jetpack framework is proposed, which recycles checkpoints of pre-trained dense models into a sparse expert network and achieves efficient fine-tuning through the SpheroMoE layer.
RΓ³bert CsordΓ‘s (Stanford University), Christopher D Manning
CodeTransformerLarge Language ModelMixture of ExpertsText
π― What it does: A general Transformer based on Mixture of Experts (MoE) called MoEUT is proposed, achieving more efficient language model training through shared layer parameters.
MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability
Yanrui Du (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes the MoGU framework, which divides LLM into two modes: usable (Glad Responder) and safe (Unwill Responder) through dynamic routing, and dynamically allocates weights based on the type of instruction at input to achieve a balance between safety and usability.
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
Rachel Teo, Tan Minh Nguyen
CodeOptimizationMixture of ExpertsImageText
π― What it does: In the Sparse Mixture of Experts model (SMoE), momentum is incorporated to propose MomentumSMoE, which is further extended to AdamSMoE and Robust MomentumSMoE, achieving improvements in model stability and robustness.
MonkeySee: Space-time-resolved reconstructions of natural images from macaque multi-unit activity
Lynn Le (Donders Institute for Brain Cognition and Behaviour Radboud University), Umut GΓΌΓ§lΓΌ (Donders Institute for Brain Cognition and Behaviour Radboud University)
CodeRestorationGenerationExplainability and InterpretabilityConvolutional Neural NetworkGenerative Adversarial NetworkImage
π― What it does: Reconstruct natural images from macaque multi-unit activity using a CNN decoder.
π― What it does: The study examines the impact of algorithm monoculture on the welfare of firms and applicants in two-sided matching markets and proposes a corresponding theoretical model.
Monomial Matrix Group Equivariant Neural Functional Networks
Hoang V. Tran, Tan Minh Nguyen
CodeConvolutional Neural NetworkImage
π― What it does: This paper studies the symmetry of weight space in fully connected and convolutional neural networks, proposing the Monomial-NFN (Monomial Neural Function Network) which achieves a more complete modeling of weight space through a unified treatment of weight scaling or sign flipping and permutation symmetry.
Monte Carlo Tree Search based Space Transfer for Black Box Optimization
Shukuan Wang (Nanjing University), Chao Qian (Nanjing University)
CodeOptimizationTabular
π― What it does: A search space transfer framework based on Monte Carlo Tree Search (MCTS-transfer) is proposed, which can dynamically partition, select, and update the search space in black-box optimization.
Most Influential Subset Selection: Challenges, Promises, and Beyond
Yuzheng Hu (University of Illinois), Jiaqi Ma
CodeImageTabular
π― What it does: This paper systematically studies the Most Influential Subset Selection (MISS) problem, analyzes the failure mechanisms of traditional greedy heuristic methods based on influence functions, proposes and theoretically proves the advantages of an adaptive greedy algorithm in capturing sample interactions, and conducts experimental validation on various tasks.
CodeRecognitionDomain AdaptationKnowledge DistillationTransformerMixture of ExpertsVision Language ModelVideoMultimodality
π― What it does: This paper studies how to transfer large visual language models (such as CLIP) to video recognition tasks while achieving a balance between generalization and specialization with efficient computational costs.
π― What it does: A motion prediction framework called RealMotion is proposed for continuous driving scenarios, which can recursively aggregate historical scene information and continuously refine trajectory predictions at each frame.
π― What it does: The MotionBooth framework is proposed to achieve customized video generation for a small number of object images, supporting controllable subject and camera motion.
MoVA: Adapting Mixture of Vision Experts to Multimodal Context
Zhuofan Zong (Chinese University of Hong Kong), Yu Liu (SenseTime Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelImageTextMultimodality
π― What it does: We propose MoVA, a multimodal large language model that achieves adaptive extraction and fusion of multi-task visual information through a two-stage mechanism (first using LLM for context-aware expert routing, followed by fine-grained fusion of selected visual experts with MoV-Adapter).
MSA Generation with Seqs2Seqs Pretraining: Advancing Protein Structure Predictions
Le Zhang (Fudan University), Siqi Sun (Fudan University)
CodeProtein Structure PredictionTransformerBiomedical Data
π― What it does: This paper proposes a self-supervised generative pre-training model called MSA-Generator based on the seqs2seqs task, aimed at generating virtual MSAs to improve the accuracy of protein structure prediction.
MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training
Bo Chen (Tsinghua University), Le Song (BioMap Research)
CodeProtein Structure PredictionTransformerReinforcement LearningPrompt EngineeringBiomedical Data
π― What it does: A protein structure prediction model based on MSA-generated pre-training, called MSAGPT, is proposed. It can enhance the accuracy of structure predictions like AlphaFold2 in low MSA environments lacking rich homologous sequences by generating virtual MSA.
π― What it does: This paper proposes a new multi-instance partial label learning framework MIPLMA, which addresses the margin violation issues that traditional methods encounter in both instance space and label space.
Multi-model Ensemble Conformal Prediction in Dynamic Environments
Erfan Hajihashemi (University of California), Yanning Shen (University of California)
CodeClassificationDomain AdaptationComputational EfficiencyConvolutional Neural NetworkMixture of ExpertsImage
π― What it does: An adaptive synthetic prediction framework using multi-model ensemble (SAMOCP) is proposed for dynamic environments, capable of 'instantaneously' selecting the most suitable learning model to generate a prediction set at each moment while maintaining a preset coverage probability.
π― What it does: A two-stage multi-target 3D localization framework called D-LISA is proposed, which achieves multi-target 3D object localization through three main modules: dynamic bounding box selection, dynamic multi-view rendering, and language-aware spatial attention fusion.
π― What it does: This paper studies the Multi-Reward Best Policy Identification (MR-BPI) problem, which focuses on how to quickly determine the optimal policy for all rewards with the least samples given a set of rewards R;
CodeRepresentation LearningDrug DiscoveryProtein Structure PredictionGraph Neural NetworkLarge Language ModelPoint CloudGraphBiomedical Data
π― What it does: A multi-scale protein representation learning framework S3F is proposed, which combines sequence, structure, and surface information to achieve zero-shot protein function fitness prediction.
π― What it does: A multi-scale Vision Mamba (MSVMamba) is proposed, which enhances the long-range modeling capability of SSM in visual tasks through multi-scale 2D scanning (MS2D) and convolutional Feed-Forward networks (ConvFFN).