International Conference on Learning Representations Β· 1682 papers
Enhancing Document Understanding with Group Position Embedding: A Novel Approach to Incorporate Layout Information
Yuke Zhu (MYbank), Sheng Guo (MYbank)
CodeTransformerLarge Language ModelTextMultimodalityBenchmark
π― What it does: This paper proposes Group Position Embedding (GPE), which allocates different dimensions of positional information by group in multi-head attention, allowing large language models to learn document layout without changing the network structure or input format.
π― What it does: A self-supervised latent world model (LAW) is proposed, which predicts future latent features based on the current scene's latent features and the ego vehicle's trajectory, and jointly optimizes scene representation and trajectory prediction for end-to-end driving.
Enhancing Language Model Agents using Diversity of Thoughts
Vijay Lingam (Amazon), Anoop Deoras (Amazon)
CodeOptimizationAI Code AssistantTransformerLarge Language ModelReinforcement LearningAgentic AIText
π― What it does: A new framework is proposed to enhance the performance of language model agents through diversified reflection and cross-task memoryβDiversity of Thoughts (DoT);
π― What it does: This study proposes a new regularization technique that enhances the robust fairness of deep neural networks through spectral regularization, particularly addressing the issue of robust accuracy discrepancies across different classes.
Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-negative Decision Layer
Xinyue Hu (Xidian University), Mingyuan Zhou (University of Texas at Austin)
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkTransformerImage
π― What it does: A Bayesian Non-negative Decision Layer (BNDL) is proposed, introducing a sparse, non-negative probabilistic generative model in the final layer of deep networks to enhance uncertainty estimation and interpretability.
Yinghao Li (Amazon Web Service), MohamadAli Torkamani (Amazon Web Service)
CodeLarge Language ModelMixture of ExpertsText
π― What it does: The ELREA framework is proposed, which generates a low-rank expert adapter ensemble through gradient direction clustering to address gradient conflicts in large model fine-tuning.
π― What it does: Proposes Adaptive Feature Aggregation (AFA), which dynamically fuses features in a multi-model UNet structure, adjusting the contributions of each model based on prompts, noise, steps, and spatial locations to enhance generation quality and contextual consistency.
π― What it does: A distance-based intra-episode reward mechanism named ETD is proposed, which drives the agent's exploration by learning distances in the context of Markov decision processes;
Yaniv Oren (Delft University of Technology), Wendelin Boehmer
CodeReinforcement LearningSequential
π― What it does: This paper proposes Epistemic Monte Carlo Tree Search (EMCTS), which incorporates the ontological uncertainty generated by learning models into MCTS to achieve deep exploration.
eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channels
Alexander DeRieux (Virginia Tech), Walid Saad (Virginia Tech)
CodeReinforcement LearningPhysics Related
π― What it does: A distributed actor-critic framework based on quantum entanglement, eQMARL, is proposed for cooperation and learning in multi-agent reinforcement learning.
Royina Karegoudra Jayanth (University of Pennsylvania), Daniel Gehrig (University of Pennsylvania)
CodeAutonomous DrivingRobotic IntelligenceSimultaneous Localization and MappingTime Series
π― What it does: A method for Neural Inertial Odometry (EqNIO) based on an equivariant framework is proposed. This method first projects IMU data into a learned, gravity-aligned, and equivariant 'canonical frame', then trains a neural displacement prior within this framework, and finally maps the predicted results back to the original frame, achieving complete invariance to changes in IMU orientation.
π― What it does: Research and improve the performance of graph diffusion models in graph-to-graph translation tasks (taking chemical reaction prediction as an example).
π― What it does: A new self-supervised learning framework is proposedβEquivariant Masked Position Prediction (EMPP), which better captures quantum mechanical features by masking atomic 3D positions and predicting their coordinates using information from neighboring atoms.
Equivariant Neural Functional Networks for Transformers
Hoang V. Tran, Tan Minh Nguyen
CodeTransformerImageTextBenchmark
π― What it does: This paper systematically studies the Neural Functional Network (NFN) of the Transformer by analyzing the maximum symmetric group of multi-head attention and constructing the weight space and group action. It proposes the Transformer-NFN, a polynomial NFN that is equivariant to this group action, and releases the Small Transformer Zoo dataset containing over 12,500 Transformer checkpoints.
Erasing Concept Combination from Text-to-Image Diffusion Model
hongyi nie, Yatao Bian (Tencent AI Lab)
CodeGenerationData SynthesisLarge Language ModelDiffusion modelImageText
π― What it does: This paper proposes a new Concept Combination Elimination (CCE) framework that can eliminate inappropriate content concept combinations in text-to-image diffusion models without compromising the quality of individual concept generation.
π― What it does: This paper proposes a sentence embedding method called ESE (Espresso Sentence Embeddings) that can simultaneously achieve scalability in model depth and embedding dimensions.
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time
Yi Ding (Purdue University), Ruqi Zhang (Purdue University)
CodeSafty and PrivacyTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: A two-stage inference-time safety alignment framework named ETA is proposed, which first assesses the safety of input and output through visual and textual evaluations, and then achieves safe and useful responses through shallow interventions (interference prefixes) and deep searches (sentence-level best-of-N).
Evaluating Large Language Models through Role-Guide and Self-Reflection: A Comparative Study
Lili Zhao (University of Science and Technology of China), Shijin Wang (iFLYTEK Co. Ltd)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper proposes the RoSe (Role-guided and Self-reflection) strategy, which evaluates self-awareness and self-correction in large language models (LLMs) and extracts high-quality data from closed-source LLMs through dual calibration (accuracy and confidence) for fine-tuning open-source LLMs.
Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective
Xiangru Zhu (Fudan University), Xiaoxiao Xu (Renmin University of China)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningImageTextBenchmark
π― What it does: This paper studies the causal relationship between word order changes and semantic differences in text-to-image (T2I) synthesis, and proposes a new SemVarEffect metric and SemVarBench benchmark to systematically evaluate the model's ability to capture semantic changes.
Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models
Jingcheng Deng (Institute of Computing Technology, Chinese Academy of Sciences), Xueqi Cheng (Institute of Computing Technology, Chinese Academy of Sciences)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: An editing method for unstructured knowledge called UnKE has been proposed, along with the release of the UnKEBench benchmark.
Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles
Buu Phan (University of Toronto), Karen Ullrich (Meta AI)
CodeGenerationOptimizationTransformerLarge Language ModelText
π― What it does: This study investigates the impact of tokenization on language models, introduces the concept of 'tokenization bias', and presents the Byte-Token Representation Lemma. Based on this theory, an O(1) algorithm is designed to convert any pre-trained tokenized language model into an equivalent byte-level model without the need for retraining, and it is applied in the Fill-in-the-Middle (FIM) task and model integration.
Exact Certification of (Graph) Neural Networks Against Label Poisoning
Mahalakshmi Sabanayagam (Technical University of Munich), Debarghya Ghoshdastidar (Technical University of Munich)
CodeGraph Neural NetworkGraph
π― What it does: An exact robustness proof against label flipping attacks on Graph Neural Networks (GNN) is proposed, providing both sample-level certificates and collective certificates.
Exact Computation of Any-Order Shapley Interactions for Graph Neural Networks
Maximilian Muschalik (Ludwig Maximilian University of Munich), Barbara Hammer (Bielefeld University)
CodeExplainability and InterpretabilityComputational EfficiencyGraph Neural NetworkGraph
π― What it does: This paper proposes a precise calculation method for Shapley interactions (SIs) in graph prediction tasks of Graph Neural Networks (GNN), called GraphSHAP-IQ, and visualizes node-level interactions in the form of SI-Graph.
Examining Alignment of Large Language Models through Representative Heuristics: the case of political stereotypes
Sullam Jeoung (University of Illinois at Urbana-Champaign), Jana Diesner (Technical University of Munich)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: From the perspective of representativeness heuristics, this paper systematically evaluates whether the outputs of large language models (LLMs) align with human values on political issues, quantitatively analyzing the extent and conditions of their deviation from empirical positions.
Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting
Wei Chen (Hong Kong University of Science and Technology), Yuxuan Liang (Hong Kong University of Science and Technology)
CodeGraph Neural NetworkPrompt EngineeringGraphTime Series
π― What it does: A continuous spatiotemporal graph prediction framework EAC based on prompt learning is proposed to address the issues of model expansion and catastrophic forgetting caused by the addition of new sensors.
Explain Yourself, Briefly! Self-Explaining Neural Networks with Concise Sufficient Reasons
Shahaf Bassan (IBM Research), Shlomit Gur (IBM Research)
CodeExplainability and InterpretabilityConvolutional Neural NetworkTransformerSupervised Fine-TuningImageText
π― What it does: A self-supervised training framework SST is proposed, allowing neural networks to directly provide minimal sufficient reasons in their outputs, thus avoiding post-hoc calculations.
Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation
Itamar Zimerman (Tel Aviv University), Lior Wolf (Tel Aviv University)
CodeExplainability and InterpretabilityRecurrent Neural NetworkTransformerImageText
π― What it does: A unified implicit attention representation is proposed, which explains and visualizes the internal mechanisms of modern gated linear RNNs (such as Mamba, RWKV, Griffin, etc.) and Transformers, and develops new interpretability methods based on this representation.
Explanations of GNN on Evolving Graphs via Axiomatic Layer edges
Yazheng Liu (Hong Kong University of Science and Technology), Sihong Xie (Hong Kong University of Science and Technology)
CodeOptimizationExplainability and InterpretabilityGraph Neural NetworkGraph
π― What it does: This study investigates methods for explaining GNN predictions on evolving graphs with continuously changing edge weights, proposing a layer-edge-based explanation framework.
Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning
Melanie Sclar (University of Washington), Asli Celikyilmaz (Meta)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A framework called EXPLORETOM driven by A* search has been constructed to generate diverse and challenging Theory of Mind (ToM) stories and questions on a large scale.
Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View
Xuan Liu (Hong Kong Polytechnic University), Quanyan Zhu (New York University)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: The CogMir framework is proposed, utilizing the systematic illusion properties of LLMs to simulate human cognitive biases and assess the rationality and prosocial decision-making of LLM agents in social contexts.
Exploring the Design Space of Visual Context Representation in Video MLLMs
Yifan Du (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeOptimizationRepresentation LearningTransformerLarge Language ModelVision Language ModelVideoMultimodality
π― What it does: This paper systematically studies the visual context representation in video multimodal large language models (Video-MLLM), specifically how to select the number of video frames and the number of visual embeddings per frame under a fixed context window, modeling it as a constrained optimization problem to derive the optimal allocation.
π― What it does: A scalable communication protocol called ExpoComm based on exponential topology is proposed for large-scale cooperative reinforcement learning.
F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI
Xu Zheng (Florida International University), Dongsheng Luo (Florida International University)
CodeExplainability and InterpretabilitySupervised Fine-TuningImageTextTime Series
π― What it does: A framework called F-Fidelity based on fine-grained tuning and random masking is proposed to evaluate the trustworthiness of explainable AI.
π― What it does: FaceShot proposes a training-free portrait animation framework that can match any character with any driving video to generate realistic animations.
Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning
Mingyang Chen (Baichuan Inc.), weipeng chen
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper proposes the BUTTON method, which generates 8,000 multi-turn function call instruction tuning data through bottom-up task construction and top-down trajectory generation.
Yicong Li (Dalian University of Technology), Feng Xia (RMIT University)
CodeExplainability and InterpretabilityTabularBiomedical DataElectronic Health Records
π― What it does: This paper proposes AGAIN, which utilizes factor graphs to encode logical rules, identifying and correcting logical errors in concept explanations during the inference phase, thereby generating understandable explanations under unknown disturbances.
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
Yifei Ming (Salesforce AI Research), Shafiq Joty (Salesforce AI Research)
CodeGenerationRetrievalTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
π― What it does: The FaithEval benchmark is proposed to evaluate the fidelity of large language models in retrieval-augmented generation (RAG) scenarios, constructing three types of tasks: unanswerable, inconsistent, and counterfactual.
CodeAnomaly DetectionExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningImageMultimodality
π― What it does: Proposes a multimodal explainable image forgery detection and localization framework called FakeShield, which integrates detection, localization, and textual explanation;
Fantastic Copyrighted Beasts and How (Not) to Generate Them
Luxi He (Princeton University), Peter Henderson (Princeton University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageVideoText
π― What it does: This study investigates the risks of text-to-image/video generation models when generating copyrighted characters, proposing an evaluation framework that balances copyright protection and user intent, systematically identifying indirect anchors, and assessing and improving existing mitigation strategies (prompt rewriting, negative prompting).
Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them
Anh Tuan Bui (Monash University), Dinh Phung (Monash University)
CodeGenerationOptimizationDiffusion modelImage
π― What it does: Proposes the Adaptive Guided Erasure (AGE) method, which dynamically selects target concepts and fine-tunes the diffusion model under a minimax optimization framework to efficiently eliminate undesirable concepts while maximizing the retention of other concepts.
Fast and Slow Streams for Online Time Series Forecasting Without Information Leakage
Ying-yee Ava Lau (Hong Kong University of Science and Technology), Dit-Yan Yeung (Hong Kong University of Science and Technology)
CodeTime Series
π― What it does: A dual-stream online time series forecasting framework (DSOF) is proposed, redefining the online time series forecasting task and eliminating the information leakage problem.
Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generation
Kim Yong Tan (Nanyang Technological University), Yew-Soon Ong (Nanyang Technological University)
CodeGenerationOptimizationDrug DiscoveryDiffusion modelImageBiomedical Data
π― What it does: This paper proposes an online, query-efficient black-box target generation algorithm called Fast Direct, which utilizes a pre-trained diffusion model to guide noise sequences through pseudo-targets during inference, enabling the generation of samples that meet specific objectives (such as image alignment or molecular binding affinity) within a limited query budget.
Johannes Hertrich (University Paris Dauphine PSL), Michael Quellmalz (Technische Universitat Berlin)
CodeOptimizationComputational EfficiencyImage
π― What it does: A fast kernel summation method based on random projection and one-dimensional kernel summationβslicingβis proposed, and quasi-Monte Carlo (QMC) design on the sphere is introduced to improve the accuracy of slicing, further deriving error upper bounds and variance analysis.
π― What it does: This paper proposes the use of the natural evolution of parameters during the RBM training process (trajectory annealing) to achieve efficient log-likelihood estimation and sampling, and addresses the initialization problem of highly structured data through low-rank RBM pre-training.
Fast unsupervised ground metric learning with tree-Wasserstein distance
Kira Michaela DΓΌsterwald (University College London), Makoto Yamada (Okinawa Institute of Science and Technology)
CodeOptimizationComputational EfficiencyBiomedical Data
π― What it does: A tree-Wasserstein distance-based unsupervised benchmark metric learning method, Tree-WSV, is proposed to quickly estimate the distance between samples and features.
π― What it does: FasterCache is proposed, a training-independent acceleration strategy to enhance the inference speed of video diffusion models while maintaining high-quality generation.
Fat-to-Thin Policy Optimization: Offline Reinforcement Learning with Sparse Policies
Lingwei Zhu (University of Tokyo), Yukie Nagai (University of Tokyo)
CodeOptimizationReinforcement LearningBiomedical Data
π― What it does: This paper proposes an offline reinforcement learning algorithm named FtTPO, which learns sparse continuous policies from log data using a two-stage fat-to-thin strategy.
Federated Domain Generalization with Data-free On-server Matching Gradient
Trong Binh Nguyen, Won-Joo Hwang
CodeDomain AdaptationFederated LearningImage
π― What it does: This paper proposes FedOMG, an algorithm that utilizes local gradients for gradient matching on the federated learning server to achieve a domain-invariant global model, thereby addressing the Federated Domain Generalization (FDG) problem.
π― What it does: This paper defines the problem of Federated Few-Shot Class Incremental Learning (FFSCIL) and proposes a Unified Optimization Prototype Prompt (UOPP) model to address catastrophic forgetting, overfitting, and prototype bias while maintaining data privacy.
Federated Residual Low-Rank Adaptation of Large Language Models
Yunlu Yan (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
CodeFederated LearningTransformerLarge Language ModelText
π― What it does: In response to the data heterogeneity of large-scale language models in federated learning, a Federated Residual LoRa Adaptation (FRLoRA) method based on low-rank residual updates is proposed.
π― What it does: In federated learning, FedLWS is proposed, which utilizes adaptive hierarchical weight shrinkage on the aggregated global model at the server side to enhance the model's generalization performance.
Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimization
Xi Lin (City University of Hong Kong), Qingfu Zhang (City University of Hong Kong)
CodeOptimizationTabular
π― What it does: This paper proposes a method based on Tchebycheff set scalarization (TCH-Set) and its smoothed version (STCH-Set) to find only a few (e.g., 5) solutions in multi-objective optimization to cover a large number of objectives (e.g., 100+).
π― What it does: Proposes the Few-Class Arena benchmark and SimSS difficulty measurement for evaluating the performance of visual models in few-class scenarios.
Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning
Yujian Liu (University of California Santa Barbara), Yang Zhang (MIT IBM Watson AI Lab)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
π― What it does: A fine-tuning strategy named PREREQ-TUNE is designed and implemented, which first learns the prerequisite knowledge required for the task through knowledge LoRA, then freezes that LoRA and uses skill LoRA to learn the skills of the task itself, thereby separating the learning of knowledge and skills, reducing the probability of hallucinations in LLMs.
Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
Keisuke Kamahori (University of Washington), Baris Kasikci (University of Washington)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsText
π― What it does: A hybrid CPU-GPU inference system named Fiddler is proposed, specifically designed for Mixture-of-Experts (MoE) language models in resource-constrained environments. It dynamically allocates expert layers to CPU or GPU to minimize inference latency and efficiently utilize limited GPU memory.
Filtered not Mixed: Filtering-Based Online Gating for Mixture of Large Language Models
Raeid Saqur (University of Toronto), Frank Rudzicz (Dalhousie University)
CodeLarge Language ModelMixture of ExpertsTime SeriesFinance Related
π― What it does: An online gating mechanism MoE-F based on continuous-time filters is proposed, capable of dynamically combining multiple pre-trained LLM experts for time series prediction.
Find A Winning Sign: Sign Is All We Need to Win the Lottery
Junghun Oh (Seoul National University), Kyoung Mu Lee (Seoul National University)
CodeOptimizationConvolutional Neural NetworkImage
π― What it does: An improved learning rate reset (LRR) method named AWS is proposed to find sparse subnetworks that can be trained from any random initialization to performance comparable to that of fully parameterized networks, and it is demonstrated that the signed mask can transfer the generalization potential of the subnetwork to new initialized networks.
Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment
Chenhang Cui (National University of Singapore), Tat-Seng Chua (National University of Singapore)
CodeGenerationRecommendation SystemOptimizationTransformerReinforcement LearningVision Language ModelMultimodality
π― What it does: Proposes the FiSAO self-alignment method, which utilizes the visual encoder of VLLM to perform fine-grained scoring for each generated token and fine-tunes the model through reinforcement learning to enhance visual-language alignment;
Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design
Chenyu Wang (Massachusetts Institute of Technology), Tommaso Biancalani (Genentech)
CodeOptimizationDrug DiscoveryReinforcement LearningDiffusion modelSequentialBiomedical Data
π― What it does: Reward optimization fine-tuning of pre-trained discrete diffusion models is performed, proposing the DRAKES algorithm, which enables the model to generate sequences that meet task objectives while maintaining a natural distribution.
Fine-tuning with Reserved Majority for Noise Reduction
Shuyang Jiang (Fudan University), Yu Wang (Shanghai Jiao Tong University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This study investigates the redundancy of LoRA parameters and proposes the PREFT framework and NORM method to eliminate noise and enhance PEFT effectiveness.
FlashMask: Efficient and Rich Mask Extension of FlashAttention
Guoxia Wang (Baidu Inc.), Haifeng Wang (Baidu Inc.)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes FLASHMASK, which extends FlashAttention by introducing a column-sparse mask representation that supports more complex masking patterns and achieves linear memory complexity.
π― What it does: The FlashRNN library is proposed, which aims for efficient implementation of traditional RNNs (LSTM, GRU, sLSTM) on GPUs by integrating matrix multiplication and activation operations, supporting multi-head parallelism, providing implementations in both CUDA and Triton, and using the integer constraint solver ConstrINT for automatic tuning.
Flat Reward in Policy Parameter Space Implies Robust Reinforcement Learning
Hyun Kyu Lee (Ulsan National Institute of Science and Technology), Sung Whan Yoon (Ulsan National Institute of Science and Technology)
CodeReinforcement LearningSequential
π― What it does: This paper studies the relationship between the flatness of the reward function in reinforcement learning and robustness, proving that a flatter reward landscape can enhance robustness to action, transition probability, and reward perturbations, and validates this theory through algorithms such as SAM+PPO.
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper presents FlexCAD, a unified controllable CAD generation model that utilizes LLM to achieve multi-level control of CAD generation (such as CAD, sketch-extrusion, sketch, face, loop, curve, etc.) through structured text representation and hierarchical-aware masking.
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
Xunhao Lai (Peking University), Xun Zhou (ByteDance Inc)
CodeComputational EfficiencyTransformerLarge Language ModelTextBenchmark
π― What it does: Introducing a mechanism called FlexPrefill that dynamically adjusts the sparse attention pattern during the pre-filling phase of long sequence reasoning;
Boye Niu (Sydney AI Centre, University of Sydney), Tongliang Liu (Sydney AI Centre, University of Sydney)
CodeLarge Language ModelAgentic AITextMultimodality
π― What it does: This paper proposes a multi-agent framework called Flow, based on large language models, which can dynamically adjust workflows during task execution, achieving task parallelization and fault tolerance.
π― What it does: This paper proposes the Social Dynamics Adaptation (SDA) model, which utilizes human trajectory information available during training to assist robots in social navigation, and infers social dynamics during deployment through state-action history.
Forewarned is Forearmed: Harnessing LLMs for Data Synthesis via Failure-induced Exploration
Qintong Li (Hong Kong University), Lingpeng Kong (Hong Kong University)
CodeData SynthesisSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: A data synthesis framework named REVERSEGEN is proposed, which generates targeted training samples using the failure cases of the target model, thereby improving the model's performance on tasks related to safety, honesty, and mathematical reasoning.
Forget the Data and Fine-Tuning! Just Fold the Network to Compress
Dong Wang (Graz University of Technology), Olga Saukh (ETH Zurich)
CodeCompressionConvolutional Neural NetworkImage
π― What it does: Without using training data and fine-tuning, a model folding technique is proposed to compress the model by clustering and merging similar channels within the same network.
Fourier Sliced-Wasserstein Embedding for Multisets and Measures
Tal Amir (Technion Israel Institute of Technology), Nadav Dym (Technion Israel Institute of Technology)
CodeOptimizationPoint CloudBiomedical Data
π― What it does: A Fourier Sliced-Wasserstein (FSW) embedding method is proposed, which embeds finite-support multisets and measures into Euclidean space, approximately preserving the sliced Wasserstein distance.
π― What it does: An interactive frame interpolation framework called Framer has been developed, allowing users to control smooth transitions between two frames by specifying keypoint trajectories.
π― What it does: This paper proposes a frequency-aware cascading sampling framework named FreCaS, which efficiently generates high-resolution images (e.g., 4096Γ4096) using existing diffusion models without the need for retraining.
FreDF: Learning to Forecast in the Frequency Domain
Hao Wang (Zhejiang University), Dacheng Tao (Nanyang Technological University)
CodeTransformerTime Series
π― What it does: This study investigates the learning target bias caused by label autocorrelation in direct forecasting (DF) methods and proposes the FreDF framework for frequency domain learning prediction, compatible with any forecasting model.
π― What it does: Proposes the Free Hunch framework, achieving covariance estimation for diffusion model denoisers with no additional training and low computational cost;
FreeCG: Free the Design Space of Clebsch-Gordan Transform for Machine Learning Force Fields
Shihao Shao (Peking University), Qinghua Cui (Peking University)
CodeGraph Neural NetworkTransformerGraphPhysics Related
π― What it does: A geometric neural network named FreeCG is proposed, which constructs abstract edges that can be freely designed using variable ClebschβGordan (CG) transformations, significantly enhancing the expressive power of machine learning force fields (MLFF).
π― What it does: Designed and implemented FreqPrior, a technique that improves the noise prior of video diffusion models through frequency domain noise filtering.
π― What it does: A frequency-domain based self-supervised learning framework called FOLK is proposed, utilizing adaptive frequency masking and teacher-student knowledge distillation for pre-training.
π― What it does: The FreSh method is proposed, which automatically selects the best embedding hyperparameters by matching the frequency spectrum of the model's initial output with the target signal, avoiding the heavy grid search.
From an LLM Swarm to a PDDL-empowered Hive: Planning Self-executed Instructions in a Multi-modal Jungle
Kaustubh Vyas (Huawei Technologies), Jeff Z. Pan (University of Edinburgh)
CodeRecommendation SystemOptimizationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextMultimodalityBenchmarkAudio
π― What it does: The HIVE system is proposed, which constructs a model capability knowledge graph (C-KG). It combines LLM and PDDL planning to automatically parse multimodal user instructions, select appropriate model sets, and execute them according to an interpretable action plan, supporting user-defined constraints.
From Attention to Activation: Unraveling the Enigmas of Large Language Models
Prannay Kaul (Huawei), Jiankang Deng (Huawei)
CodeOptimizationTransformerLarge Language ModelText
π― What it does: This study investigates two phenomena in autoregressive Transformers: the dominance of the first token's attention and the abnormal activation of hidden states, and proposes corresponding solutions.
From Commands to Prompts: LLM-based Semantic File System for AIOS
Zeru Shi (Rutgers University), Yongfeng Zhang (Rutgers University)
CodeRetrievalOptimizationTransformerLarge Language ModelPrompt EngineeringTextMultimodality
π― What it does: Designed and implemented a Large-scale Language Model-based Semantic File System (LSFS), achieving a closed loop from natural language prompts to file retrieval, updates, rollbacks, and other operations through the construction of a semantic index, vector database, and scalable syscall and API layers.
π― What it does: This paper proposes AdaQTransform, a post-training quantization (PTQ) method that decouples the weight quantization step size and applies adaptive linear transformations to the quantized output, making the quantized model closer to the FP32 output.
From GNNs to Trees: Multi-Granular Interpretability for Graph Neural Networks
Jie Yang (Zhejiang University), Shunyu Liu (Nanyang Technological University)
CodeExplainability and InterpretabilityGraph Neural NetworkGraph
π― What it does: Transforming ordinary GNNs into a hierarchical tree structure to achieve interpretable graph classification at multiple levels (multi-granularity).
From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs
Alireza Rezazadeh (Center for Advanced AI), Yujia Bao (Center for Advanced AI)
CodeLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: This paper presents MemTree, a long-term memory representation method based on a dynamic tree structure that can update, aggregate, and retrieve information in real-time during large language model dialogues or document question-answering.
From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question-Answering
Nathaniel Weir (Johns Hopkins University), Peter Clark (Allen Institute for AI)
CodeOptimizationKnowledge DistillationTransformerLarge Language ModelText
π― What it does: Through a problem-driven process, the internal knowledge of the language model is distilled into interpretable 'micro-theories' (a set of core knowledge sentences), and their ability to support answers to topics is tested using a text entailment mechanism.
From Promise to Practice: Realizing High-performance Decentralized Training
Zesen Wang (KTH Royal Institute of Technology), Mikael Johansson (KTH Royal Institute of Technology)
CodeOptimizationTransformerImageText
π― What it does: Implement and evaluate decentralized training on multi-node GPU clusters, propose a decentralized Adam variant that covers communication and computation along with an accumulation mechanism, and construct a runtime model to guide topology and parameter configuration.
From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation
Nikita Kotelevskii (Mohammed Bin Zayed University of Artificial Intelligence), Maxim Panov (Mohammed Bin Zayed University of Artificial Intelligence)
CodeClassificationAnomaly DetectionImage
π― What it does: A prediction uncertainty framework based on point risk decomposition and strict proper scoring rules is proposed, unifying various uncertainty measures that can be obtained through Bayesian estimation.
π― What it does: A generative feedback model called GenRe is proposed, aimed at jointly optimizing effective algorithmic feedback methods to help individuals adversely affected by automated model decisions improve their features, thereby achieving more favorable outcomes.
π― What it does: Study the impact of Chain of Thought (CoT) on the sample efficiency and attention sparsity of Transformers in learning parity functions.
From Tokens to Words: On the Inner Lexicon of LLMs
Guy Kaplan (Hebrew University of Jerusalem), Roy Schwartz (Hebrew University of Jerusalem)
CodeTransformerLarge Language ModelTextBiomedical Data
π― What it does: This study investigates how large language models internally reorganize subword sequences into complete words and explores the hierarchical mechanisms and internal vocabulary involved in this process.
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
Jiahui Gao (University of Hong Kong), Lingpeng Kong (University of Hong Kong)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityBenchmark
π― What it does: This paper proposes G-LLAVA, a multimodal large language model for geometric problems, which utilizes a text LLM to automatically generate a high-quality geometric dataset Geo170K, thereby enhancing the model's understanding and reasoning capabilities regarding geometric shapes.
π― What it does: This paper studies a Gap Preserving Distillation (GPD) method, which utilizes a dynamic teacher constructed from the student model and enforces parameter sharing to maintain an appropriate performance gap between the student and teacher, thereby significantly enhancing the effectiveness of knowledge distillation.
Gated Delta Networks: Improving Mamba2 with Delta Rule
Songlin Yang (Massachusetts Institute of Technology), Ali Hatamizadeh (NVIDIA)
CodeTransformerText
π― What it does: This paper proposes a new linear TransformerβGated DeltaNet, which constructs an efficient memory management mechanism by integrating gated decay with the Delta update rule, and provides a hardware-friendly chunked parallel training algorithm.
π― What it does: The research focuses on supervised facial expression detection, constructing a dual-branch framework and introducing Gaussian Instance Adaptive Intensity Modeling to achieve soft pseudo-labeling, further enhancing feature discrimination through intensity-aware contrastive learning.