π― What it does: A novel unsupervised content-style separation method V3 is proposed, which learns interpretable symbolic representations using variance and invariance constraints.
π― What it does: This paper proposes CAMeLU, a Transformer framework that reformulates unsupervised meta-learning as context learning, capable of performing few-shot classification in a single forward inference.
Unsupervised Multiple Kernel Learning for Graphs via Ordinality Preservation
Yan Sun (National University of Singapore), Stanley Kok (National University of Singapore)
CodeGraph Neural NetworkGraph
π― What it does: This paper proposes an unsupervised multiple kernel learning framework UMKL-G, which automatically combines multiple graph kernels while preserving the topological structure of the graphs, thereby enhancing graph-level clustering performance without the need for labels or predefined neighbors.
π― What it does: A dual-value forward-backward representation (DVFB) framework is proposed, utilizing parallel training of exploration value and skill value to achieve zero-shot generalization and fine-tuning adaptation in unsupervised online reinforcement learning.
Unveiling the Magic of Code Reasoning through Hypothesis Decomposition and Amendment
Yuze Zhao (University of Science and Technology of China), Enhong Chen (University of Science and Technology of China)
CodeAI Code AssistantTransformerLarge Language ModelReinforcement LearningTextBenchmark
π― What it does: This paper proposes a code reasoning task that unifies three types of logical reasoningβinductive, deductive, and abductiveβinto a code format. Based on this, a reflective hypothesis decomposition and revision (RHDA) process is designed, utilizing LLMs and external execution tools (compilers, VirtualHome) to achieve multi-round reasoning, verification, and improvement, significantly enhancing the reasoning performance of LLMs.
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs
Aldo Pareja (Red Hat AI Innovation), Akash Srivastava (Red Hat AI Innovation)
CodeOptimizationHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Conducted systematic experiments on supervised instruction fine-tuning of open-source LLMs in the 3Bβ7B scale, systematically evaluating the impact of hyperparameters such as batch size, learning rate, warmup, and training strategies (staged vs. stacked) on final performance.
Utilitarian Algorithm Configuration for Infinite Parameter Spaces
Devon R. Graham (University of British Columbia), Kevin Leyton-Brown (University of British Columbia)
CodeOptimizationHyperparameter SearchTabular
π― What it does: This paper proposes COUP (Continuous, Optimistic Utilitarian Procrastination) β a novel framework for configurability algorithms that can operate in an infinite parameter space. It demonstrates that COUP is faster with finite configuration sets and can effectively explore and provide approximate optimal guarantees in infinite sets.
Afra Amini (ETH Zurich), Ryan Cotterell (ETH Zurich)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: By deriving the distribution generated by Best-of-N (BoN) and using the inverse KL variational method (vBoN) to fine-tune the language model, it approximates the high-reward outputs of BoN during single sampling, thus achieving efficient aligned inference.
π― What it does: A posterior sampling method MGPS based on Denoising Diffusion Models is proposed, which improves the estimation of the guiding term for observations using midpoint decomposition.
Daniel M. Steinberg (Data61 CSIRO), Edwin V. Bonilla (Data61 CSIRO)
CodeOptimizationDrug DiscoveryRecurrent Neural NetworkTransformerReinforcement LearningSequentialBiomedical Data
π― What it does: A method called Variational Search Distributions (VSD) is proposed for sequentially learning and generating generative models of rare target classes (such as highly active protein/nucleic acid sequences) in high-dimensional discrete combinatorial design spaces, and for batch evaluation through black-box experiments/simulations.
Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only
Jihan Yao (University of Washington), Yulia Tsvetkov (University of Washington)
CodeTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBiomedical Data
π― What it does: In task scenarios lacking reliable correct answers, this study explores the use of LLMs to generate incorrect answers and improve the quality and calibration of model answers through 'wrong-over-wrong' preference alignment.
VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Tianyu Zhang (Mila), Yoshua Bengio (Mila)
CodeRestorationData SynthesisTransformerSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: The task of Visual Text Recovery (VCR) is proposed, and the corresponding dataset VCR-WIKI is constructed, exploring pixel-level recovery of occluded text in images and complex reasoning of visual context.
Vector-ICL: In-context Learning with Continuous Vector Representations
Yufan Zhuang (University of California San Diego), Jianfeng Gao (Microsoft Research)
CodeClassificationGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextMultimodalityTime SeriesMagnetic Resonance Imaging
π― What it does: The Vector-ICL method is proposed, which projects arbitrary continuous vectors into 'box tokens' that can be directly processed by LLMs, enabling cross-modal contextual learning.
π― What it does: Proposes the VEDIT framework, utilizing the latent embeddings of a frozen visual encoder combined with a diffusion transformer for step prediction and planning.
Vertical Federated Learning with Missing Features During Training and Inference
Pedro Valdeira (Carnegie Mellon University), Yuejie Chi (Carnegie Mellon University)
CodeFederated LearningTabularBiomedical Data
π― What it does: This paper proposes LASER-VFL, a vertical federated learning method capable of handling arbitrary missing feature blocks during both training and inference, avoiding data waste and achieving scalability.
VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models
Lisa Dunlap (University of California Berkeley), Joseph E. Gonzalez
CodeRecommendation SystemExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: VibeCheck has been developed, a method for automatically discovering and quantifying interpretable and measurable 'vibes' in the outputs of large language models, helping to compare subtle differences between models.
ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler
Serin Yang (KAIST), Jong Chul Ye (KAIST)
CodeGenerationData SynthesisDiffusion modelVideo
π― What it does: The ViBiDSampler keyframe interpolation method is proposed, utilizing bidirectional sampling and advanced manifold guidance to generate high-quality, temporally coherent video frames.
π― What it does: A video imitation model called VidIT based on autoregressive Transformer is proposed, which can generate new videos consistent with the demonstration semantics in a zero-shot scenario by watching demonstration videos.
VideoPhy: Evaluating Physical Commonsense for Video Generation
Hritik Bansal (University of California Los Angeles), Aditya Grover (University of California Los Angeles)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVideoTextMultimodalityBenchmarkPhysics Related
π― What it does: The VIDEOPHY benchmark was constructed to evaluate the semantic adherence and physical common sense of text-to-video models using human annotations, and an automatic evaluator, VIDEOCON-PHYSICS, was proposed.
VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking
Runyi Hu (Nanyang Technological University), Tianwei Zhang (Nanyang Technological University)
CodeGenerationData SynthesisSafty and PrivacyDiffusion modelVideo
π― What it does: Achieve untrained watermark embedding in the diffusion-based video generation process, and implement a complete framework VIDEOSHIELD for watermark extraction and spatiotemporal tampering localization through DDIM inversion.
VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation
Yecheng Wu (Tsinghua University), Yao Lu (NVIDIA)
CodeRecognitionGenerationTransformerLarge Language ModelVision Language ModelContrastive LearningImageVideoTextMultimodality
π― What it does: We propose VILA-U, a unified visual language model capable of predicting the next token in both visual and textual modalities under the same autoregressive framework, achieving both visual understanding and visual generation.
Vision and Language Synergy for Rehearsal Free Continual Learning
Muhammad Anwar Ma'sum (University of South Australia), Ryszard Kowalczyk (University of South Australia)
CodeGenerationTransformerPrompt EngineeringImage
π― What it does: A replay-free continual learning framework LEAPGen is proposed, which utilizes language descriptions to generate prompts for adapting to new tasks.
π― What it does: Train convolutional networks on synthetic images to estimate the spatial latent variables of objects (such as position, pose, distance, etc.) and evaluate their alignment with neural data from the monkey visual pathway.
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
Yuchen Duan (Chinese University of Hong Kong), Wenhai Wang (Chinese University of Hong Kong)
CodeClassificationObject DetectionSegmentationComputational EfficiencyTransformerVision Language ModelImage
π― What it does: This paper proposes Vision-RWKV (VRWKV), a visual encoder that employs a linear attention mechanism, capable of maintaining global perception while reducing computational complexity, supporting high-resolution image processing and sparse input.
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents
Shi Yu (Tsinghua University), Maosong Sun (Tsinghua University)
CodeGenerationRetrievalTransformerVision Language ModelMultimodalityRetrieval-Augmented Generation
π― What it does: A visual retrieval-augmented generation system, VisRAG, has been established, utilizing VLM to directly retrieve and generate multimodal documents, eliminating the need for OCR parsing.
π― What it does: The VISUAL-O1 framework is proposed, utilizing multi-modal multi-turn chain reasoning to help the model eliminate instruction ambiguity in visual contexts and generate experiences during the reasoning process;
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
Xiao Liu (Tsinghua University), Jie Tang
CodeRobotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper proposes VisualAgentBench (VAB), a unified multi-domain evaluation benchmark for training and assessing the visual foundational agent capabilities of large multimodal models (LMM) in diverse real-world environments.
Seulki Park (University of Michigan), Jonathan Huang (Scaled Foundations)
CodeClassificationSegmentationTransformerImage
π― What it does: Proposes the H-CAST method, which achieves hierarchical classification with visual and semantic unity through fine-grained to coarse-grained visual segmentation and internal consistency.
VLAS: Vision-Language-Action Model with Speech Instructions for Customized Robot Manipulation
Wei Zhao (Westlake University), Donglin Wang (Westlake University)
CodeRobotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelTextMultimodalityRetrieval-Augmented GenerationAudio
π― What it does: The paper proposes an end-to-end Visual Language Action Model (VLAS) that can directly process voice commands and generate robot actions.
VLMaterial: Procedural Material Generation with Large Vision-Language Models
Beichen Li (Massachusetts Institute of Technology), Wojciech Matusik (Massachusetts Institute of Technology)
CodeGenerationData SynthesisTransformerLarge Language ModelVision Language ModelImageMultimodality
π― What it does: Utilize large-scale visual-language models (VLM) to generate Blender procedural material programs for input images, supporting complete editing;
π― What it does: A human visual-tactile dataset VTDexManip was constructed, and based on this dataset, six complex multi-finger robotic grasping tasks were simulated in Isaac Gym to establish benchmarks, comparing 18 pre-trained and non-pre-trained methods to verify the improvement of visual-tactile joint pre-training on robotic grasping performance.
π― What it does: A multi-target long-term reinforcement learning environment VVC-Gym based on fixed-wing drone velocity vector control is proposed, and a multi-quality demonstration set is generated through a PID controller and IRPO, providing benchmark experiments.
Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations
Katie Matton (Massachusetts Institute of Technology), Emre Kiciman
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Evaluate whether the natural language explanations generated by large language models are authentic and trustworthy, and propose a new measurement method;
Ward: Provable RAG Dataset Inference via LLM Watermarks
Nikola JovanoviΔ (ETH Zurich), Martin Vechev (ETH Zurich)
CodeGenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: A black-box method called WARD is proposed for detecting unauthorized use of datasets in Retrieval-Augmented Generation (RAG), providing strict statistical guarantees.
Wasserstein Distances, Neuronal Entanglement, and Sparsity
Shashata Sawmya (MIT), Nir N Shavit
CodeLarge Language ModelMixture of ExpertsText
π― What it does: This study investigates the heterogeneity of neurons (measured by Wasserstein distance) and proposes a Sparse Expansion framework that decomposes the distribution of neurons into experts through clustering inputs while maintaining sparsity, thereby enhancing the sparsification performance of large language models.
π― What it does: A cover gap upper bound based on Wasserstein distance is proposed, and the Wasserstein-regularized Conformal Prediction (WR-CP) algorithm is designed in the multi-source domain generalization scenario, using importance weighting and regularized representation learning to simultaneously reduce errors caused by covariate and concept shifts, generating more accurate and compact prediction sets.
Wavelet-based Positional Representation for Long Context
Yui Oka (NTT Corporation), Kuniko Saito (NTT Corporation)
CodeTransformerLarge Language ModelText
π― What it does: This paper proposes a multi-scale position representation method based on wavelet transform, which can achieve effective long-context position encoding and extrapolation in Transformers.
π― What it does: Proposes WavTokenizer, which compresses speech, music, and general audio into only 40 or 75 discrete tokens under a single-layer quantizer, achieving high-quality reconstruction at extremely low bit rates.
Weak-to-Strong Generalization Through the Data-Centric Lens
Changho Shin (University of Wisconsin Madison), Frederic Sala (University of Wisconsin Madison)
CodeData-Centric LearningLarge Language ModelText
π― What it does: This paper proposes and validates a data center mechanismβoverlap density, explaining how weak models can assist strong models in learning difficult-to-recognize patterns by labeling overlapping samples. It also provides an overlapping sample detection algorithm and a UCB-based multi-source data selection strategy. Experiments are then conducted on large language models, weak supervision, and synthetic data to verify the positive correlation between overlap density and weak-strong generalization performance.
CodeOptimizationReinforcement Learning from Human FeedbackReinforcement LearningText
π― What it does: Proposes the WSPO (Weak-to-Strong Preference Optimization) method, which utilizes the differences in probability distributions before and after alignment of the weak model to guide the alignment of the strong model, thereby transferring and amplifying the alignment capability of the weak model to the strong model.
Weakly Supervised Video Scene Graph Generation via Natural Language Supervision
Kibum Kim (KAIST), Chanyoung Park (KAIST)
CodeObject DetectionGenerationTransformerLarge Language ModelVision Language ModelVideoText
π― What it does: A weakly supervised video scene graph generation framework, NL-VSGG, is proposed, which uses only video subtitles as weak supervision to train a VidSGG model without manual annotations.
Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors
Peiran Xu (Wangxuan Institute of Computer Technology Peking University), Yadong MU
CodeObject DetectionSegmentationTransformerVision Language ModelContrastive LearningImage
π― What it does: A pseudo-fully supervised training framework for weakly supervised functional localization tasks is proposed, utilizing visual foundation models to generate and refine pseudo-labels, achieving accurate affordance heatmap predictions.
π― What it does: We propose and implement WeatherGFM, the first weather general foundation model capable of uniformly handling 12 tasks including weather forecasting, rainfall, radar, satellite image super-resolution, image transformation, and post-processing.
CodeLarge Language ModelReinforcement LearningWorld ModelText
π― What it does: By incorporating a world model into an LLM-driven web navigation agent, the environmental state after actions is simulated to improve decision-making, and a transfer-focused observation abstraction method is proposed.
CodeTransformerLarge Language ModelReinforcement LearningAgentic AIText
π― What it does: A self-evolving online course reinforcement learning framework named WEBRL has been developed to train web agents based on open-source large language models (LLMs), significantly improving their success rate on the WebArena tasks.
Weighted-Reward Preference Optimization for Implicit Model Fusion
Ziyi Yang (Sun Yat-sen University), Xiaojun Quan (Sun Yat-sen University)
CodeRecommendation SystemOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: A method for implicit model fusion based on weighted reward preference optimization (WRPO) is proposed, which achieves the transfer of multi-source LLM knowledge by dynamically balancing the preference information of the source model and the target model.
What Are Good Positional Encodings for Directed Graphs?
Yinan Huang (Georgia Institute of Technology), Pan Li (Georgia Institute of Technology)
CodeGraph Neural NetworkGraph
π― What it does: This paper studies the position information encoding (PE) of directed graphs, proposing the Walk Profile concept that captures bidirectional relationships, and designing a Multi-q Magnetic Laplacian PE (Multiβq MagβPE) and a basis-invariant stable PE framework based on complex features to enhance the representational capability on directed graphs.
π― What it does: A hierarchical prototype network HComP-Net is proposed, which utilizes image learning to evolve hierarchical features and achieve generalized localization for unseen species.
What is Wrong with Perplexity for Long-context Language Modeling?
Lizhe Fang (Peking University), Yisen Wang (Peking University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This study investigates why traditional perplexity (PPL) cannot measure the performance of large language models on long-context tasks and proposes a new evaluation metric, LongPPL, and training loss, LongCE.
π― What it does: This paper presents GenPercept, a single-step deterministic fine-grained perception framework that utilizes a pre-trained diffusion model (Stable Diffusion);
What should a neuron aim for? Designing local objective functions based on information theory
Andreas Christian Schneider, Michael Wibral (University of GΓΆttingen)
CodeClassificationOptimizationExplainability and InterpretabilityHyperparameter SearchImage
π― What it does: Proposed a local learning objective based on PID information decomposition, designed interpretable infomorphic neurons, and achieved self-organizing classification tasks using three types of inputs: feedforward, contextual, and lateral.
π― What it does: This paper proposes a task of 'discovering novel domains in fine-tuning datasets through generation' and introduces the Contrastive Generative Exploration (CGE) method to achieve this task.
When Attention Sink Emerges in Language Models: An Empirical View
Xiangming Gu (Sea AI Lab), Min Lin (Sea AI Lab)
CodeTransformerLarge Language ModelText
π― What it does: This paper systematically studies the phenomenon of 'attention sink' in autoregressive language models, clarifying the conditions, mechanisms, and its relationship with model design, and proposes that changing the attention normalization method (such as using sigmoid attention) can eliminate this phenomenon.
When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach
Qian Chen (Chinese University of Hong Kong), Qingjiang Shi (Tongji University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: This paper addresses the issue of symmetry in Integer Linear Programming (ILP) and investigates the problem of Graph Neural Networks (GNN) being unable to distinguish symmetric variables when predicting the optimal solution of ILP. It proposes an orbit-based feature enhancement method to solve this problem.
π― What it does: By placing LLMs in a telephone game-style transmission chain, the evolution of text toxicity, positivity, difficulty, and length during multi-round interactions reveals the phenomenon of LLM cultural attractors.
When Prompt Engineering Meets Software Engineering: CNL-P as Natural and Robust "APIs'' for Human-AI Interaction
Zhenchang Xing (CSIRO Data61), Chenhua Liu (Jiangxi Normal University)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
π― What it does: CNL-P is proposed, a controlled natural language framework that embeds core software engineering principles (modularity, abstraction, encapsulation, separation of concerns) into prompt syntax to enhance the readability, maintainability, and executability of LLM prompts.
When Selection Meets Intervention: Additional Complexities in Causal Discovery
Haoyue Dai (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
CodeTabularSequential
π― What it does: This paper proposes a new framework for causal discovery under the presence of selection bias, designing the 'Intervention Twin Graph' and providing corresponding criteria for Markovianity and equivalence.
Why Does the Effective Context Length of LLMs Fall Short?
Chenxin An (University of Hong Kong), Lingpeng Kong (University of Hong Kong)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: This paper analyzes the root cause of the insufficient effective context length of large language models, finding that the position frequency distribution during the pre-training phase is severely left-skewed, leading to inadequate training for long-distance dependencies. To address this issue, the STRING (ShifTed Rotary position embeddING) technique is proposed, which compensates for this defect by shifting high-frequency positions to low-frequency positions during the inference phase, without requiring additional training.
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Haipeng Luo (Shenzhen International Graduate School Tsinghua University), Dongmei Zhang (Microsoft Corporation)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
π― What it does: The WizardMath model is introduced to enhance the mathematical reasoning capabilities of large language models using the RLEIF method.
Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers
Omer Sahin Tas (FZI Research Center for Information Technology), Royden Wagner (Karlsruhe Institute of Technology)
CodeAutonomous DrivingExplainability and InterpretabilityTransformerAuto EncoderTime Series
π― What it does: An interpretability analysis of the hidden states of the motion Transformer model is conducted, using linear probes to examine the phenomenon of neural collapse, and fitting control vectors based on the differences in opposing features; during inference, the control vector is added to the hidden states to achieve controllable modifications of the predicted trajectories without the need for fine-tuning.
WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models
Shengda Fan (Renmin University of China), Maosong Sun (Tsinghua University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: A large-scale WorkflowBench dataset was constructed, and WorkflowLlama was fine-tuned on it to enhance the performance of LLMs in workflow orchestration.
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale
Haoran Xu (Microsoft), Huda Khayrallah (Amazon)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: X-ALMA is a multilingual translation LLM designed for 50 languages, capable of maintaining high-quality translations across all languages, regardless of resource abundance.
π― What it does: Proposes the X-DRIVE dual-branch latent diffusion model, which jointly generates aligned LiDAR point clouds and multi-view camera images, and supports text and 3D bounding box control.
CodeClassificationAnomaly DetectionExplainability and InterpretabilityGraph Neural NetworkTransformerSupervised Fine-TuningTime SeriesBiomedical DataAlzheimer's Disease
π― What it does: This paper proposes a model called XAIguiFormer that utilizes explainable artificial intelligence (XAI) to guide Transformers in identifying brain diseases using multi-frequency EEG connectivity graphs.
xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
Qingchen Yu (Institute for Advanced Algorithms Research), Ding Chen (Institute for Advanced Algorithms Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Developed xFinder, an evaluator specifically designed for answer extraction and matching in LLM evaluation, replacing traditional regular expression extraction methods.
π― What it does: This paper constructs the XLAND-100B large-scale multi-task dataset and provides tools and a smaller version to support and evaluate in-context reinforcement learning; it also experiments with common AD and DPT baselines on this dataset.
π― What it does: This paper proposes the Retriever-Dictionary (RD) module, which pre-encodes dataset knowledge into a dictionary and dynamically generates coefficients through a retriever, embedding it into YOLO and other detection models to enhance detection, segmentation, and classification performance.
π― What it does: A self-coherent diffusion GAN model named YOSO is proposed and trained, achieving one-step (single-step) high-quality image generation, and enabling one-step text-to-image generation through fine-tuning of the pre-trained model.
Youku Dense Caption: A Large-scale Chinese Video Dense Caption Dataset and Benchmarks
Zixuan Xiong (Shenzhen International Graduate School Tsinghua University), Hai-Tao Zheng (Shenzhen International Graduate School Tsinghua University)
CodeGenerationRetrievalTransformerSupervised Fine-TuningVision Language ModelVideoTextMultimodalityBenchmark
π― What it does: The first large-scale high-quality dense video caption dataset in China, Youku Dense Caption, has been proposed and released, and multimodal benchmarks for retrieval, localization, and generation have been constructed based on it.
π― What it does: This paper proposes RADD (Reparameterized Absorbing Discrete Diffusion), a time-condition-free, cache-accelerated absorbing discrete diffusion model, and unifies it with any autoregressive model (AO-ARM) in terms of training objectives.
Your Mixture-of-Experts LLM Is Secretly an Embedding Model for Free
Ziyue Li (University of Maryland), Tianyi Zhou (University of Maryland)
CodeLarge Language ModelMixture of ExpertsTextBenchmark
π― What it does: This paper studies the routing weights (RW) of the Mixture-of-Experts language model as an untrained embedding representation, and combines RW with hidden states (HS) to propose the MOEE (MoE Embedding) method;
Your Weak LLM is Secretly a Strong Teacher for Alignment
Leitian Tao (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: Explores the use of weak LLMs (125M level) as teachers for language model alignment, verifying that their feedback quality is comparable to or even better than human feedback.
CodeTransformerLarge Language ModelReinforcement LearningTime Series
π― What it does: This paper proposes the Disentangled In-Context Learning (DICL) framework, which utilizes pre-trained large language models (LLMs) for zero-shot dynamics prediction in Markov Decision Processes (MDPs) within continuous state spaces, and validates its effectiveness in two application scenarios: policy evaluation and data-augmented offline reinforcement learning (DICL-SAC).
CodeClassificationGenerationExplainability and InterpretabilityConvolutional Neural NetworkTransformerPrompt EngineeringContrastive LearningImage
π― What it does: A zero-shot, trustworthy natural language explanation (NLE) method is proposed, which can explain any visual classifier while supporting zero-shot image classification, concept discovery, and image caption generation.
π― What it does: This study investigates the performance of zero-shot learning in scenarios with scarce samples and proposes the ZeroDiff framework to enhance visual-semantic associations, thereby generating more reliable features for unseen categories.
Zeroth-Order Fine-Tuning of LLMs with Transferable Static Sparsity
Wentao Guo (Princeton University), Zhaozhuo Xu (University of Minnesota)
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The paper proposes the SensZOQ framework, which utilizes zero-order optimization combined with transferable static sparse parameters and quantization techniques to achieve efficient personalized fine-tuning of large language models (such as Llama2-7B) on memory-constrained devices like mobile phones.
ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Models
Seonghwan Park (POSTECH), Namhoon Lee (POSTECH)
CodeOptimizationComputational EfficiencyPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: Aiming at prompt tuning for black-box visual language models, a low-dimensional zero-order gradient method (ZIP) is proposed, significantly reducing the number of queries and improving performance.