π― What it does: Aiming at post-training quantization (PTQ) for Diffusion Transformers (DiTs), the VETA-DiT framework is proposed to achieve efficient inference with 4-bit weights/activations while maintaining generation quality.
π― What it does: This paper proposes a video tokenizer called VFRTok based on the Duration-Proportional Information Assumption, which supports variable frame rate encoding and decoding, and enhances content modeling through Partial RoPE.
VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold
Dominic Rosario Maggio (Massachusetts Institute of Technology), Luca Carlone (Massachusetts Institute of Technology)
CodeOptimizationSimultaneous Localization and MappingPoint Cloud
π― What it does: Using an uncalibrated monocular RGB camera, combined with VGGT's feedforward scene reconstruction, a dense 3D map is incrementally constructed by subgraphs, and global consistency is achieved through 15-degree-of-freedom projection homotopy (SL(4)) optimization between subgraphs;
Kaituo Feng (Chinese University of Hong Kong), Xiangyu Yue (Chinese University of Hong Kong)
CodeSupervised Fine-TuningReinforcement LearningVision Language ModelImageVideoMultimodality
π― What it does: This paper presents Video-R1, a multimodal large language model based on a rule reinforcement learning (R1) framework, specifically designed to enhance video reasoning capabilities.
Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension
Yongdong Luo (Xiamen University), Rongrong Ji (Xiamen University)
CodeObject DetectionRetrievalTransformerLarge Language ModelVision Language ModelVideoTextRetrieval-Augmented Generation
π― What it does: A no-training, pluggable retrieval-augmented long video understanding framework called Video-RAG has been developed, which enhances long video reasoning capabilities by utilizing OCR, ASR, and object detection to generate visually aligned auxiliary text fed into existing large language models.
VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perception
Ziang Yan (Zhejiang University), Yi Wang (Shanghai AI Laboratory)
CodeOptimizationLarge Language ModelReinforcement LearningVision Language ModelVideoMultimodalityBenchmark
π― What it does: Proposes Visual Test-Time Scaling (VTTS), which enhances the reasoning ability of multimodal large language models (MLLMs) through iterative visual perception during the inference process.
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding
Zongxia Li (University of Maryland), Jordan Lee Boyd-Graber (University of Maryland)
CodeData SynthesisAnomaly DetectionTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelVideoTextMultimodality
π― What it does: The VideoHallu dataset is proposed to test the physical and common sense understanding of VLM using synthetic videos, and RL fine-tuning is used to enhance its ability to recognize abnormal scenes.
VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning
Qi Wang (Beijing Institute of Technology), Tianfei Zhou (Beijing Institute of Technology)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelVideoMultimodalityChain-of-Thought
π― What it does: A framework called VIDEORFT is proposed to cultivate video reasoning capabilities by implementing Reinforcement Fine-Tuning (RFT) on a multimodal large language model (MLLM);
π― What it does: A behavior cloning framework based on latent representations, BCV-LR, is proposed, which can achieve sample-efficient visual policy learning in a reward-free environment using only unlabeled videos.
VIKING: Deep variational inference with stochastic projections
Samuel G. Fadel (Technical University of Denmark), SΓΈren Hauberg (Technical University of Denmark)
CodeOptimizationImage
π― What it does: A variational inference framework VIKING is proposed for over-parameterized deep networks, which divides the parameter space into the kernel of Fisher-Rao metric (the directions invariant to training data) and the image (the directions that change the loss), and describes the two types of uncertainty using fully correlated Gaussian posteriors.
π― What it does: This paper proposes the VIPAMIN visual prompt initialization method, which improves the visual prompt tuning of self-supervised models through attention-guided matching and orthogonal projection.
Virus Infection Attack on LLMs: Your Poisoning Can Spread "VIA" Synthetic Data
Zi Liang (Hong Kong Polytechnic University), Haibo Hu (Hong Kong Polytechnic University)
CodeData SynthesisAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper systematically evaluates the security risks of data contamination and backdoor attacks when using synthetic data in LLM training, and proposes the Virus Infection Attack (VIA) framework, which allows maliciously injected content to spread to downstream models through synthetic data.
π― What it does: This paper proposes an image tokenizer VFMTok based on a frozen Vision Foundation Model (VFM), utilizing region-adaptive quantization and semantic reconstruction objectives to achieve efficient image reconstruction and autoregressive (AR) image generation.
Cheng Shi (Sun Yat-sen University), Sibei Yang (Sun Yat-sen University)
CodeTransformerLarge Language ModelVision Language ModelImageMultimodality
π― What it does: This study investigates the Visual Function Layer (VFL) in multimodal large language models (MLLMs), locating the distribution of different visual functions across network layers through visual token swapping and dropping techniques. Based on these findings, targeted LoRA (VFL-LoRA) and data selection methods (VFL-Select) are proposed to enhance the parameter efficiency and data utilization of the model.
Nicholas Jiang, Yossi Gandelsman (University of California Berkeley)
CodeObject DetectionSegmentationExplainability and InterpretabilityTransformerImageMultimodality
π― What it does: This paper analyzes the high-norm outlier tokens in Vision Transformers and finds that a small number of sparse neurons (register neurons) in the feedforward layer are responsible for generating these outlier tokens. The authors propose a 'test-time registers' method that does not require retraining the model: during inference, these high activation values are moved to additional register tokens, thereby eliminating noise in the attention maps and enhancing interpretability.
Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
Yulu Qin (Boston University), Najoung Kim (Boston University)
CodeTransformerVision Language ModelText
π― What it does: A pure text question-answer dataset, TaxonomiGQA, was constructed to specifically examine the performance of models in scenarios requiring hierarchical classification knowledge, and this dataset was compared with the VLM-LLM minimal pair;
VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
Senqiao Yang (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
CodeComputational EfficiencyTransformerReinforcement LearningVision Language ModelImageMultimodality
π― What it does: This paper proposes VisionThink, a method that dynamically decides whether to use high-resolution images to answer visual questions through reinforcement learning, significantly reducing the consumption of visual tokens while maintaining performance.
ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding
Jialiang Kang (Peking University), Xinghao Chen (Huawei Noah's Ark Lab)
CodeData SynthesisComputational EfficiencyTransformerVision Language ModelImageTextMultimodality
π― What it does: This paper proposes Vision-Aware Speculative Decoding (ViSpec), which accelerates inference for visual language models by compressing image embeddings through a lightweight visual adapter and injecting global visual features.
Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection
Chanhyeong Yang (Korea University), Hyunwoo J. Kim (Korea Advanced Institute of Science and Technology)
CodeObject DetectionTransformerPrompt EngineeringVision Language ModelImage
π― What it does: In zero-shot human-object interaction detection, the VDRP framework is proposed, which enhances the model's adaptability to visual differences of the same verb and its ability to distinguish similar interactions through visual diversity perception prompts and region perception prompts.
Changdae Oh (University of Wisconsin Madison), Sharon Li (University of Wisconsin Madison)
CodeObject DetectionDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: In the instruction tuning of multimodal large language models (MLLM), a Visual Instruction Bottleneck Tuning (Vittle) module is introduced, applying regularization in the internal representation layer through the Information Bottleneck (IB) principle to enhance the model's robustness against distribution shifts and input perturbations.
VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
Chaoyou Fu (Nanjing University), Ran He
CodeTransformerLarge Language ModelVision Language ModelImageVideoTextMultimodalityAudio
π― What it does: This paper presents VITA-1.5, a multimodal large language model capable of visual, text, and speech interaction, achieving seamless integration of vision and speech through a three-stage training method.
VITA-Audio: Fast Interleaved Audio-Text Token Generation for Efficient Large Speech-Language Model
Zuwei Long (Tencent Youtu Lab), Xing Sun (Xiamen University)
CodeGenerationComputational EfficiencyTransformerLarge Language ModelMultimodalityAudio
π― What it does: This paper proposes the VITA-Audio end-to-end large-scale speech model, which utilizes a lightweight multimodal cross-token prediction (MCTP) module to generate audio in a single forward pass, significantly reducing the initial audio generation latency.
VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
Shufan Shen (Institute of Computing Technology, Chinese Academy of Sciences), Shuhui Wang (Institute of Computing Technology, Chinese Academy of Sciences)
CodeExplainability and InterpretabilityRepresentation LearningVision Language ModelAuto EncoderContrastive LearningMultimodality
π― What it does: Designed and trained a sparse autoencoder VL-SAE for a unified concept set to explain and enhance the alignment mechanism of visual-language models.
Zhanhui Zhou (University of California Berkeley), Chaochao Lu (University of Illinois Urbana-Champaign)
CodeGenerationAdversarial AttackTransformerSupervised Fine-TuningVision Language ModelImage
π― What it does: This paper studies and verifies the ability of visual language models (VLM) to learn and generate corresponding text by aggregating image fragments scattered across different training samples, referred to as visual stitching, and explores its potential threats to model security.
π― What it does: The paper proposes an end-to-end adjustable recurrent neural network (e-nmRNN) based on neural modulation volume transmission, achieving online, targeted credit allocation through contextual factorization, and is evaluated on tasks such as sequence-to-sequence, meta-learning, multi-tasking, and reinforcement learning.
π― What it does: The VORTA framework is proposed, utilizing dynamic routing sparse attention to accelerate the generation process of video diffusion transformers.
π― What it does: This paper proposes VQ-Seg, a semi-supervised medical image segmentation framework that utilizes vector quantization (VQ) and quantized perturbation to replace traditional dropout, enhancing segmentation performance through a dual-branch shared Post-VQ space and semantic alignment with the base model.
VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning
Qiuchen Wang (MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China), Feng Zhao (MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China)
CodeRetrievalTransformerReinforcement LearningVision Language ModelMultimodalityRetrieval-Augmented Generation
π― What it does: Developed VRAG-RL, a retrieval-augmented generation framework that achieves multi-round visual perception and retrieval reasoning through reinforcement learning training of visual language models.
VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning
Wenhao Li (Shandong University), Yilong Yin (Shandong University)
CodeClassificationRecognitionGenerationMeta LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: This paper proposes the use of large language models to generate cross-modal prompts for vision and text, and achieves few-shot learning through geometric alignment.
VTON-VLLM: Aligning Virtual Try-On Models with Human Preferences
Siqi Wan (University of Science and Technology of China), Tao Mei (HiDream.ai)
CodeGenerationRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelImageTextBenchmarkChain-of-Thought
π― What it does: A visual large language model VTON-VLLM based on human feedback fine-tuning is proposed to evaluate and guide virtual try-on (VTON) models to better align with user preferences.
π― What it does: This paper studies a text-to-3D generation framework called TraCe based on the SchrΓΆdinger bridge, explicitly constructing and learning the optimal transport trajectory from the current rendering to the text alignment target to achieve high-quality 3D asset generation.
Walking the Tightrope: Autonomous Disentangling Beneficial and Detrimental Drifts in Non-Stationary Custom-Tuning
Xiaoyu Yang (University of Technology Sydney), En Yu (University of Technology Sydney)
CodeOptimizationTransformerLarge Language ModelReinforcement LearningVision Language ModelMultimodalityBiomedical DataElectronic Health RecordsChain-of-Thought
π― What it does: This paper proposes the Counterfactual Preference Optimization (CPO) method, which eliminates concept drift in non-stationary environments and enhances diagnostic reliability through causal counterfactual interventions on the Chain-of-Thought (CoT) generation path during reinforcement fine-tuning (RFT) of multimodal large language models.
π― What it does: A state space model called WaLRUS based on wavelet frames is proposed for online function approximation and compressed representation of long sequences.
π― What it does: A theoretical analysis of the sampling error of Critical Damping Langevin Diffusion (CLD) under Wasserstein distance is proposed, introducing a noise smoothing hyperparameter to enhance sampling performance.
Kaicheng Zhang (Zhejiang University), Yidong Zhou (University of California)
CodeDomain AdaptationTabularBiomedical Data
π― What it does: A transfer learning framework for regression of distributed responses in Wasserstein space (WaTL) and its adaptive version (AWaTL) is proposed, which improves the prediction accuracy of the target domain distribution by combining source domain information with target domain samples.
CodeGenerationData SynthesisLarge Language ModelSupervised Fine-TuningImageMultimodalityAudio
π― What it does: A generation time zero watermarking method for autoregressive image generation models is proposed, addressing insufficient reverse cycle consistency and achieving robust detection.
Wavelet Canonical Coherence for Nonstationary Signals
Haibo Wu (King Abdullah University of Science and Technology), Hernando Ombao
CodeTime SeriesSequential
π― What it does: A Wavelet Canonical Coherence (WaveCanCoh) framework based on multivariate locally variable waveforms is proposed to estimate time-varying scale-specific co-movement relationships among non-stationary multivariate signal groups.
Satoshi Noguchi (Japan Agency for Marine-Earth Science and Technology), Yoshinobu Kawahara (RIKEN)
CodeTransformerDiffusion modelImageTextGraph
π― What it does: This paper studies the oversmoothing problem of Transformers and proposes the Wavy Transformer, which uses attention layers based on second-order wave dynamics and corresponding physically consistent layer normalization and feedforward networks, aiming to alleviate oversmoothing in deep models.
WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization
Jiahao Wen (Shanghai University), Zhedong Zheng (University of Macau)
CodeRepresentation LearningTransformerLarge Language ModelVision Language ModelImageTextMultimodalityChain-of-Thought
π― What it does: This paper proposes WeatherPrompt, which combines training-free weather description with chain reasoning and dynamic text gating for multimodal representation learning to achieve robustness in drone visual localization under all weather conditions.
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITextChain-of-Thought
π― What it does: An end-to-end web information retrieval agent system called WebDancer is proposed, which includes four stages: data synthesis, trajectory sampling, SFT cold start, and RL enhancement.
WebThinker: Empowering Large Reasoning Models with Deep Research Capability
Xiaoxi Li (Renmin University of China), Zhicheng Dou (Renmin University of China)
CodeTransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: WebThinker has been developed, a deep research agent based on large reasoning models, capable of autonomously searching the web, browsing web pages, and writing research reports in real-time during the reasoning process.
π― What it does: The study investigates how to infer the implicit state of an agent's partial knowledge of the environment through observing navigation trajectories and presents a Bayesian inference method.
What Moves the Eyes: Doubling Mechanistic Model Performance Using Deep Networks to Discover and Test Cognitive Hypotheses
Federico D'Agostino (University of TΓΌbingen), Matthias Kuemmerer
CodeExplainability and InterpretabilityImage
π― What it does: By comparing the high-performance deep network DeepGaze III with the interpretable mechanistic model SceneWalk, the differences in predicting eye movement scan paths (i.e., 'controversial eye movements') are identified. Based on these differences, mechanisms such as time-varying temperature scaling, eye movement inertia, and directional bias are gradually introduced, ultimately significantly improving the predictive performance of SceneWalk.
What's Producible May Not Be Reachable: Measuring the Steerability of Generative Models
Keyon Vafa (Harvard University), Sendhil Mullainathan (Massachusetts Institute of Technology)
CodeGenerationData SynthesisOptimizationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelImageTextBenchmark
π― What it does: This paper proposes a mathematical framework and benchmark tasks for evaluating the steerability of generative models, and assesses the steerability of text-to-image and large language models through large-scale user studies, pointing out their common deficiencies and providing improvement suggestions.
When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery
Dominik Meier (Cornell Tech), Kyra Gan (Cornell Tech)
CodeDiffusion modelTabular
π― What it does: A bivariate causal discovery method based on diffusion models, BiDD, is proposed to address the issue of unobserved mediating variables that cause traditional ANM methods to fail.
When Can Model-Free Reinforcement Learning be Enough for Thinking?
Josiah P. Hanna (University of Wisconsin), Nicholas E. Corrado (University of Wisconsin)
CodeTransformerLarge Language ModelReinforcement LearningTextSequentialChain-of-Thought
π― What it does: The paper theoretically analyzes when model-agnostic reinforcement learning exhibits 'thinking' behavior by introducing the Thought Markov Decision Process (Thought MDP) model under model-agnostic conditions, and validates this theory using a language model (LLM) and a simple grid world experiment.
When Causal Dynamics Matter: Adapting Causal Strategies through Meta-Aware Interventions
Moritz Willig (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
CodeDrug DiscoveryTabularTime Series
π― What it does: This paper proposes the Meta-Causal Model (MCM) and Meta-Causal Analysis (MCA) framework to design and evaluate intervention strategies in dynamic environments where causal relationships evolve over time. It demonstrates the application effects in medical and judicial cases through Direct MCM, Linearized Meta-Causal Dynamics (LMCD) algorithm, and sMCATE metrics.
π― What it does: Two embedding fusion frameworks based on the Kronecker product, KrossFuse and its scalable version RP-KrossFuse, are proposed, which can unify cross-modal and single-modal embeddings in the same representation space without training.
When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
Weixiang Zhao (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: By identifying and removing language-specific subspaces in the activation space of LLMs, the decoupling of language and reasoning is achieved, significantly improving multilingual reasoning performance.
When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration
Quan Shi (Princeton University), Karthik R Narasimhan
CodeAI Code AssistantTransformerLarge Language ModelText
π― What it does: This study proposes the KITE framework and conducts a two-phase human-machine collaboration experiment with 118 participants to systematically evaluate the knowledge transfer capability of LLMs in coding and mathematical tasks.
π― What it does: This paper proposes the Multi-Moment Retrieval (MMR) task, constructs the QV-MΒ² dataset annotated by humans, and designs the FlashMMR framework based on this.
When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding
Yan Shu (University of Trento), Nicu Sebe (University of Trento)
CodeRecognitionTransformerLarge Language ModelTextMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper addresses the issue of 'semantic illusion' in large multimodal models during scene text recognition and understanding, proposing a plug-and-play error correction framework that requires no training and can be applied only during the inference phase, and based on this, a new evaluation benchmark is created.
Who You Are Matters: Bridging Interests and Social Roles via LLM-Enhanced Logic Recommendation
Qing Yu (Wuhan University), Lixin Zou (Wuhan University)
CodeRecommendation SystemKnowledge DistillationTransformerLarge Language ModelContrastive LearningVideoTextMultimodality
π― What it does: Utilizing multi-modal LLM to extract user roles and project topic tags, and using LLM to infer user-project behavior logic, a TagCF framework is constructed to enhance recommendation performance.
Why 1 + 1 < 1 in Visual Token Pruning: Beyond Naive Integration via Multi-Objective Balanced Covering
Yangfu Li (East China Normal University), Yue Lu (East China Normal University)
CodeCompressionOptimizationTransformerVision Language ModelMultimodality
π― What it does: This paper proposes a training-independent visual token pruning method called MoB, which dynamically allocates retained tokens based on multi-objective balanced coverage theory to simultaneously meet the needs of visual retention and prompt alignment.
Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Yiyou Sun (University of California), Dawn Song (University of California)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: A framework based on subsequence association is proposed to systematically track and explain the hallucination phenomenon of large language models.
Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
Sungmin Cha (New York University), Kyunghyun Cho (Genentech)
CodeGenerationKnowledge DistillationTransformerLarge Language ModelText
π― What it does: This paper explores the precision-recall trade-off of knowledge distillation in generative models through simulations of Gaussian mixture models and large-scale language model experiments.
π― What it does: A new discrete diffusion model design space called Schedule Conditioned Diffusion (SCUD) is proposed, which enhances model fitting by embedding the jump time distribution of the forward process into the backward process.
CodeOptimizationAI Code AssistantTransformerLarge Language ModelReinforcement LearningTextBenchmark
π― What it does: Proposes the Adaptive Branching Monte Carlo Tree Search (AB-MCTS) framework, which utilizes the diversity of LLMs and external feedback to dynamically decide whether to 'expand width' or 'dig deep' during inference, thereby improving the quality of task responses.
WISA: World simulator assistant for physics-aware text-to-video generation
Jing Wang (Sun Yat-Sen University), Xiaodan Liang (University of Science and Technology Beijing)
CodeGenerationData SynthesisTransformerMixture of ExpertsDiffusion modelVideoTextPhysics Related
π― What it does: The WISA framework is proposed, which guides the generation of videos that comply with physical laws by breaking down physical principles into textual descriptions, qualitative categories, and quantitative attributes, and constructs an 80K physical video dataset WISA-80K.
Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting
Chenchen Tan (Monash University), Longxiang Gao (Qilu University of Technology)
CodeTransformerLarge Language ModelText
π― What it does: Proposes the Attention-Shifting framework to achieve selective forgetting in LLMs, reducing access to sensitive knowledge without compromising overall performance.
π― What it does: This paper proposes and implements CERMIC, a pluggable multi-agent contextual calibration module designed to enhance intrinsic curiosity exploration in sparse reward, partially observable, and non-communicative MARL scenarios.
π― What it does: This paper proposes an OOD detection method called X-Maha based on multi-layer feature fusion of Transformer, utilizing the total variance of features at each layer as importance weights, and calculating the Mahalanobis distance as the OOD score after mixing features.
π― What it does: A multi-agent control framework XVerse based on DiT text stream modulation is proposed, which can achieve fine-grained, editable generation of multiple agents' identities, poses, styles, lighting, and other semantic attributes while maintaining the integrity of the image structure.
π― What it does: In the YOLO series, an attention mechanism is incorporated to propose the YOLOv12 framework, achieving efficient real-time object detection through modules such as Area Attention and R-ELAN.
π― What it does: A parameter-free self-enhancing plugin DCBoost is proposed, which can improve the global structure and clustering performance of existing deep clustering models.
You Only Communicate Once: One-shot Federated Low-Rank Adaptation of MLLM
Binqian Xu (National University of Singapore), Xiangbo Shu (Institute of High-Performance Computing)
CodeFederated LearningLarge Language ModelMultimodality
π― What it does: A truly one-shot federated low-rank adaptation (OFL) scheme called YOCO is proposed, which guides the local LoRA training of a multimodal large language model (MLLM) using implicit global supervision, thus completing model adaptation in a single round of communication.
Zebra-Llama: Towards Extremely Efficient Hybrid Models
Mingyu Yang (Advanced Micro Devices), Emad Barsoum (Advanced Micro Devices)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a method to quickly construct a low-memory, high-efficiency hybrid model (Zebra-Llama) from existing large language models (such as Llama, Qwen), achieving almost no performance loss in knowledge transfer through fine initialization and hierarchical distillation.
π― What it does: A zero-shot cross-subject brain visual decoding framework called ZEBRA is proposed, which reconstructs images from fMRI without the need for adaptation to new subjects.
Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts
James Chapman (University of California, Los Angeles), Guido Montufar
CodeReinforcement Learning
π― What it does: Proposes a Contextualized Bellman Equation (CEBE) and a Sample Enhancement method (CSE) to improve the zero-shot generalization ability of DRL under limited training contexts.
π― What it does: This study investigates the implicit regularization of zero-order optimization (two-point estimators) under convex smooth functions, proving that it tends towards flat local minima with minimal Hessian trace;
π― What it does: A point cloud pre-training framework called ZigzagPointMamba is proposed, which combines a zigzag scanning path and a semantic semi-twin mask strategy to improve self-supervised learning.
CodeCompressionComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The zip2zip framework is proposed, which utilizes online LZW compression to dynamically expand the vocabulary during inference, achieving adaptive tokenization for large language models.
Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
Xudong Li (Xiamen University), Rongrong Ji (Xiamen University)
CodeOptimizationTransformerImage
π― What it does: This paper studies a multi-level preference optimization framework called CcDPO, aimed at enhancing multi-image understanding capabilities.