International Conference on Learning Representations · 1682 papers
Real-time design of architectural structures with differentiable mechanics and neural networks
Rafael Pastrana (Princeton University), Ryan P Adams
CodeOptimizationMeshPhysics Related
🎯 What it does: This study investigates a model that combines neural networks with a differentiable mechanical simulator, capable of inferring the shape approximations of architectural structures (such as stone arch shells and cable towers) in real-time while maintaining mechanical integrity.
🎯 What it does: A unified Partial Label Learning (PLL) evaluation benchmark PLENCH has been constructed, new model selection criteria have been proposed, and a human-annotated image dataset PLCIFAR10 has been created.
Reasoning of Large Language Models over Knowledge Graphs with Super-Relations
Song Wang (University of Virginia), Yada Zhu (IBM)
CodeRetrievalTransformerLarge Language ModelGraph
🎯 What it does: The ReKnoS framework is proposed, utilizing super-relations for multi-path parallel reasoning on knowledge graphs, significantly expanding the search space and improving retrieval success rates.
Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval
Pengcheng Jiang (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
CodeClassificationRecommendation SystemGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTabularBiomedical DataElectronic Health RecordsRetrieval-Augmented Generation
🎯 What it does: Proposes the KARE framework, which combines knowledge graph community retrieval with LLM reasoning to enhance the accuracy of medical predictions.
ReAttention: Training-Free Infinite Context with Finite Attention Scope
Xiaoran Liu (Fudan University), Xipeng Qiu (Fudan University)
CodeTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: A training-independent ReAttention method is proposed, which allows LLMs to handle infinite-length contexts under a limited attention window by performing position-independent top-k cache selection before self-attention.
🎯 What it does: This paper proposes a framework based on force-aware graph re-binding (REBIND) to predict the ground state three-dimensional conformation of molecules from two-dimensional molecular graphs.
🎯 What it does: A text-to-3D generation method called RecDreamer is proposed, which eliminates the Multi-Face Janus geometric inconsistency problem through uniform distribution correction.
🎯 What it does: The first industrial-grade end-to-end recommendation dataset, RecFlow, has been constructed and made public. It collects samples from six stages, including retrieval and ranking, and experiments have been conducted on retrieval, coarse ranking, and ranking stages using this dataset.
ReCogLab: a framework testing relational reasoning & cognitive hypotheses on LLMs
Andrew Liu (Google Deepmind), Kenneth Marino (Google Deepmind)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: A configurable generative framework, ReCogLab, is proposed for automatically constructing relationship reasoning tasks with adjustable difficulty, and systematic experiments are conducted on various models.
Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNs
Steve Azzolin (University of Trento), Andrea Passerini (University of Trento)
CodeExplainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: Systematically evaluate the authenticity metrics of graph neural network explanations, theoretically analyze their limitations, and study the implementation and impact of interpretability under different architectures (regular, interpretable, domain-invariant), proposing improvement methods and validating their effectiveness.
🎯 What it does: A Reconstruction-Guided Policy (RGP) framework is proposed, which constructs agent-wise states using a decision module and a guidance module, maintaining state consistency during training and execution to enhance decision-making performance in multi-agent reinforcement learning under partially observable environments.
Recovering Manifold Structure Using Ollivier Ricci Curvature
Tristan Luca Saidi (Columbia University), Andrew J. Blumberg (Yale University)
CodePoint CloudBiomedical Data
🎯 What it does: The ORC-MANL algorithm is proposed, which identifies and prunes 'shortcut' edges in the nearest neighbor graph by combining Ollivier-Ricci curvature with a graph distance threshold.
Recovery of Causal Graph Involving Latent Variables via Homologous Surrogates
Xiu-Chuan Li (Sydney AI Centre, University of Sydney), Tongliang Liu (Sydney AI Centre, University of Sydney)
CodeGraphTabular
🎯 What it does: Proposes the concept of 'homologous substitute variables' to eliminate the reliance on the pure child assumption in traditional causal discovery, allowing for partial or complete recovery of causal graphs with latent variables under weaker or stronger conditions;
🎯 What it does: The Rectified Diffusion method is proposed, simplifying and generalizing the idea of Rectified Flow. It supports any diffusion model, staged training, and consistency distillation by using pre-collected noise-sample pairs for retraining.
ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
ZhongXiang Sun, Han Li (Kuaishou Technology Co., Ltd.)
CodeRetrievalExplainability and InterpretabilityTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a mechanism-interpretable RAG hallucination detection method called ReDeEP, and based on this, designs AARF to mitigate hallucinations.
Yanwen Huang (Peking University), Yanyan Lan (Tsinghua University)
CodeDrug DiscoveryConvolutional Neural NetworkGraph Neural NetworkBiomedical Data
🎯 What it does: Redefine the bioactivity prediction task, construct a large-scale structural small molecule-protein interaction dataset SIU, and propose Pearson/Spearman evaluation metrics grouped by protein pockets.
Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering
Sheng Liu (Stanford University), James Zou (Stanford University)
CodeObject DetectionGenerationTransformerVision Language ModelImageTextMultimodality
🎯 What it does: A method for intervention in the latent space during inference (VTI) is proposed in large-scale vision-language models, which reduces hallucination generation by applying pre-computed offset vectors to the visual and text hidden layers.
CodeRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: A no-training fingerprint method based on representation layer similarity, REEF, is proposed to identify whether an open-source LLM is a subsequent development of the target model.
RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
Dhruv Gautam (University of California Berkeley), Roshanak Zilouchian Moghaddam (Microsoft)
CodeAI Code AssistantTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: A benchmark called RefactorBench is proposed to evaluate the reasoning and state-awareness capabilities of language model agents in multi-file code refactoring.
🎯 What it does: In response to backdoor attacks on third-party deep learning models, a backdoor defense method called REFINE is proposed, which is based on model reprogramming and does not require reverse engineering of triggers.
Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generation
Jihyo Kim (Seoul National University of Science and Technology), Sangheum Hwang (Seoul National University of Science and Technology)
CodeAnomaly DetectionTransformerPrompt EngineeringVision Language ModelImageMultimodalityBenchmark
🎯 What it does: This study investigates the performance of visual language large models in out-of-distribution detection (OoDD) and proposes a self-guided prompting (ReGuide) method to enhance their OoDD capabilities.
Reframing Structure-Based Drug Design Model Evaluation via Metrics Correlated to Practical Needs
Bowen Gao (Tsinghua University), Yanyan Lan (Beijing Academy of Artificial Intelligence)
CodeDrug DiscoveryBiomedical Data
🎯 What it does: A framework for evaluating structure-based drug design (SBDD) models based on practical needs is proposed, and various mainstream generative models are systematically evaluated within this framework.
Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF
Zhaolin Gao (Cornell University), Wen Sun (Princeton University)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes a multi-round RLHF method called REFUEL, which utilizes a single model to regress relative future rewards, achieving online alignment and addressing the covariance shift problem encountered in traditional single-round RLHF in multi-round dialogues.
Regulatory DNA Sequence Design with Reinforcement Learning
Zhao Yang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeOptimizationReinforcement LearningBiomedical Data
🎯 What it does: A design method for cis-regulatory elements (CRE) called TACO is proposed, which combines reinforcement learning and autoregressive DNA generation models, capable of generating highly functional and diverse promoter and enhancer sequences from scratch.
Reinforcement Learning for Control of Non-Markovian Cellular Population Dynamics
Josiah C Kratz, Jacob Adamczyk (University of Massachusetts)
CodeDrug DiscoveryReinforcement LearningBiomedical Data
🎯 What it does: The study uses reinforcement learning to control drug dosage in non-Markovian cell populations, proposing a memory-enabled model to obtain the optimal strategy.
Reinforcement Learning from Imperfect Corrective Actions and Proxy Rewards
Zhaohui JIANG, Changjie Fan (NetEase)
CodeReinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: This paper studies a novel reinforcement learning framework called ICoPro, which utilizes dual signals of imperfect agent rewards and human corrective actions to enhance the alignment of agents with human preferences and sample efficiency.
🎯 What it does: A reinforcement learning algorithm named SEQUOIA is proposed, which combines deep Q-networks with mixed-integer programming to directly search for the maximum expected return in the combinatorial action space at each step.
Relation-Aware Diffusion for Heterogeneous Graphs with Partially Observed Features
Daeho Um (Samsung Electronics), Seong Jin Ahn (KAIST)
CodeGraph Neural NetworkGraphBiomedical Data
🎯 What it does: The HetGFD method is proposed for feature diffusion-based missing value imputation in heterogeneous graphs by utilizing virtual features and edge type heterogeneity.
RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data
Maxwell A Xu, Shirley You Ren
CodeClassificationRecognitionRepresentation LearningConvolutional Neural NetworkContrastive LearningTime Series
🎯 What it does: The RelCon framework is proposed, which uses a learnable motif distance and relative contrastive loss for self-supervised pre-training on wearable accelerometer data to build a foundational model for movement.
Reliable and Diverse Evaluation of LLM Medical Knowledge Mastery
Yuxuan Zhou (Tsinghua University), Ji Wu (Tsinghua University)
CodeTransformerLarge Language ModelPrompt EngineeringTextBiomedical Data
🎯 What it does: The PretexEval framework is proposed, which dynamically generates reliable and diverse evaluation samples from medical knowledge bases using predicate equivalence transformation and prototype paraphrasing techniques, aimed at assessing large language models' mastery of medical factual knowledge.
ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
Ziteng Wang (Tsinghua University), Jianfei Chen (Tsinghua University)
CodeMixture of ExpertsText
🎯 What it does: This paper proposes ReMoE, a fully differentiable Mixture-of-Experts model using ReLU routing, serving as a direct alternative to TopK+Softmax routing.
Remove Symmetries to Control Model Expressivity and Improve Optimization
Liu Ziyin (Massachusetts Institute of Technology), Isaac L. Chuang (Massachusetts Institute of Technology)
CodeOptimizationReinforcement LearningImage
🎯 What it does: By adding random static bias to the loss function and combining it with weight decay, the reflection symmetry in the network is eliminated, avoiding low capacity traps and improving optimization results.
RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph
Siru Ouyang (University of Illinois), Dong Yu (Tencent)
CodeAI Code AssistantGraph Neural NetworkLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: The REPOGRAPH plugin module is proposed to construct a repository-level code graph (line-level dependency graph) and provide it as a subgraph for retrieval to LLMs, enhancing the effectiveness of AI software engineering tasks.
🎯 What it does: A comprehensive benchmark for representation similarity measurement, ReSi, has been constructed, which includes six designed tests, 24 types of similarity measures, 14 different domain neural network models (graph, language, visual), and seven publicly available datasets, with open code for community use.
🎯 What it does: A new model for implementing residual deep Gaussian processes on Riemannian manifolds is proposed, along with corresponding Bayesian inference methods.
Tim Lawson (University of Bristol), Laurence Aitchison (University of Bristol)
CodeTransformerAuto Encoder
🎯 What it does: A multi-layer sparse autoencoder (MLSAE) trained on the residual flow of all layers of the Transformer is proposed and trained to analyze the flow of information between layers.
🎯 What it does: A 'resolution attack' framework based on diffusion models is proposed and implemented, capable of automatically generating images that display one semantic at high resolution and another semantic at low resolution, thereby misleading classifiers and human observers;
🎯 What it does: A new gradient propagation method called the Rotation Trick is proposed in VQ-VAE. This method involves rotating and scaling the encoder output with the nearest codebook vector during forward propagation, while keeping the angle between the two unchanged during backpropagation, allowing the gradient to bypass the non-differentiable vector quantization layer without losing information.
RESuM: A Rare Event Surrogate Model for Physics Detector Design
Ann-Kathrin Schuetz (Lawrence Berkeley National Laboratory), Aobo Li (University of California San Diego)
CodeOptimizationTabularPhysics Related
🎯 What it does: Proposed and implemented the RESuM rare event surrogate model to optimize the neutron shield design in the LEGEND experiment, significantly reducing the background rate.
Rethinking and Improving Autoformalization: Towards a Faithful Metric and a Dependency Retrieval-based Approach
Qi Liu (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a credible automatic evaluation metric BEq and a retrieval-enhanced statement automation method RAutoformalizer, addressing the issues of traditional evaluations being imprecise and lacking contextual information.
Rethinking Classifier Re-Training in Long-Tailed Recognition: Label Over-Smooth Can Balance
Siyu Sun (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeClassificationRecognitionImage
🎯 What it does: This paper reevaluates classifier retraining methods in long-tail recognition and proposes new evaluation metrics and a Label Over-Smoothing (LOS) method based on logits magnitude balancing.
Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words
Gouki Minegishi (University of Tokyo), Yutaka Matsuo (University of Tokyo)
CodeRepresentation LearningAuto EncoderText
🎯 What it does: This paper proposes Poly-Semantic Evaluation (PS-Eval) to assess whether Sparse Autoencoders (SAE) can decompose the polysemous activations of LLMs into monosemantic features.
Lizhe Fang (Peking University), Yisen Wang (Peking University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This work proposes a new method for Invariant In-context Learning (InvICL), aimed at addressing the issue of large language models being sensitive to the order of context examples during ICL, while achieving model order invariance under the premise of information leakage prevention and context interdependence.
Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond
Qizhou Wang (Hong Kong Baptist University), Kilian Q Weinberger
CodeTransformerLarge Language ModelText
🎯 What it does: This paper proposes a tool for evaluating the learning objectives of LLMs from the perspective of gradients—G-effect. Based on this, it analyzes existing learning methods and proposes improved new objectives such as WGA, TNPO, and WTNPO. Experiments are conducted on the TOFU dataset with various LLMs, demonstrating their advantages in removing target knowledge while maintaining model integrity.
🎯 What it does: A multi-instance learning framework called PSMIL is proposed and implemented, which aligns and pools in probability space, addressing the issue of instance representation drift in traditional feature space.
🎯 What it does: This paper proposes a 'weight embedding' method that directly learns weight-specific representations within a single-objective network, thereby efficiently solving multi-objective combinatorial optimization problems.
Rethinking the generalization of drug target affinity prediction algorithms via similarity aware evaluation
Chenbin Zhang (MoleculeMind), Shaoting Zhang (Shanghai AI Laboratory)
CodeDrug DiscoveryConvolutional Neural NetworkRecurrent Neural NetworkGraph Neural NetworkTransformerBiomedical Data
🎯 What it does: This paper proposes a similarity-based evaluation framework (SAE) that can partition the test set of drug-target affinity prediction models according to the expected similarity distribution (such as uniform or simulating external data distribution), thereby more accurately assessing the model's performance on low-similarity samples.
Rethinking Visual Counterfactual Explanations Through Region Constraint
Bartlomiej Sobieski (University of Warsaw), Przemyslaw Biecek (University of Warsaw)
CodeImage TranslationExplainability and InterpretabilityScore-based ModelImage
🎯 What it does: This paper proposes Region-Constrained Visual Counterfactual Explanations (RVCE) and its generation method RCSB, which can modify images within a specified region to cause changes in classifier predictions, thereby providing more interpretable counterfactuals.
🎯 What it does: A method called Reti-Diff, which combines a latent diffusion model based on Retinex theory with Transformer, is proposed to achieve high-quality image enhancement and color correction in various degradation scenarios such as low light, underwater, and backlight.
🎯 What it does: A retrieval-enhanced diffusion framework called RADAb is proposed, which guides antibody sequence generation using structurally homologous fragments, achieving antibody design and optimization under structural constraints.
Retrieval Head Mechanistically Explains Long-Context Factuality
Wenhao Wu (Peking University), Yao Fu (University of Edinburgh)
CodeRetrievalTransformerLarge Language ModelText
🎯 What it does: This paper studies the internal mechanisms of Transformer long-context language models and proposes and examines a class of attention heads specifically responsible for information retrieval, referred to as retrieval heads.
RetroInText: A Multimodal Large Language Model Enhanced Framework for Retrosynthetic Planning via In-Context Representation Learning
Chenglong Kang (Central South University), Fei Guo (Central South University)
CodeRepresentation LearningDrug DiscoveryTransformerLarge Language ModelPrompt EngineeringTextMultimodality
🎯 What it does: This paper presents RetroInText, a multimodal large language model framework that achieves more accurate single-step and multi-step retrosynthetic predictions by using ChatGPT to generate text descriptions in multi-step retrosynthetic pathways and integrating them with molecular 2D/3D representations.
Revealing and Mitigating Over-Attention in Knowledge Editing
Pinzheng Wang (Soochow University), Min Zhang (Soochow University)
CodeTransformerLarge Language ModelText
🎯 What it does: This paper studies the issue of specificity failure that arises after knowledge editing in large language models and proposes a new regularization method called SADR to mitigate this problem.
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)
Leander Girrbach (Technical University of Munich), Zeynep Akata (Technical University of Munich)
CodePrompt EngineeringImageMultimodality
🎯 What it does: Evaluated the performance of 22 open-source visual language assistants in terms of gender bias and attempted various debiasing techniques.
🎯 What it does: This paper proposes a DNA foundational model called ConvNova based on convolutional neural networks, achieving optimal or near-optimal performance on multiple DNA prediction benchmarks, demonstrating the competitiveness of CNNs in DNA modeling.
🎯 What it does: This paper proposes the use of nonlinear Bézier surfaces to achieve neural network mode connectivity, constructing a multi-dimensional low-loss high-precision parameter space.
Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later
Han-Jia Ye (Nanjing University), Wei-Lun Chao (Ohio State University)
CodeClassificationOptimizationTabular
🎯 What it does: Improved the classic nearest neighbor method NCA by incorporating deep learning techniques, constructing MODERNNCA as a powerful deep learning baseline for tabular data.
Jinwoo Kim (Korea Advanced Institute of Science and Technology), Seunghoon Hong (Korea Advanced Institute of Science and Technology)
CodeGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: This paper proposes and systematically analyzes Random Walk Neural Networks (RWNNs), which generate readable sequences through random walks and are read by deep networks to achieve graph/vertex-level predictions.
Revisiting Source-Free Domain Adaptation: a New Perspective via Uncertainty Control
Gezheng Xu (University of Western Ontario), Grace Yi
CodeDomain AdaptationContrastive LearningImage
🎯 What it does: The UCon-SFDA method is proposed, which enhances the robustness of model self-supervised learning in source-free domain adaptation tasks through theoretical analysis of negative sample sampling errors and positive sample prediction uncertainties, designing two uncertainty control strategies: negative sample dispersion control and partial label learning for positive samples.
CodeGenerationData SynthesisTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: A large-scale, fine-grained text-to-image (T2I) evaluation suite called Gecko has been constructed, covering 2,000 diverse prompts, four annotation templates, four mainstream models, and collecting approximately 100,000 human ratings.
REvolve: Reward Evolution with Large Language Models using Human Feedback
RISHI HAZRA, Pedro Zuidberg Dos Martires (Orebro University)
CodeAutonomous DrivingOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVideo
🎯 What it does: A reward function design framework REvolve based on evolutionary algorithms and large language models is proposed, which utilizes human feedback to guide the evolution of the reward function and generates interpretable Python code rewards.
Yannick Metz (University of Konstanz), Mennatallah El-Assady (ETH Zurich)
CodeAutonomous DrivingReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: This study investigates various types of human feedback (evaluation, comparison, demonstration, correction, description, and descriptive preference) for reward learning and implements a unified simulation generation framework.
🎯 What it does: This paper first constructs a pixel-level paired dataset of events and RAW images, and evaluates and validates the event-guided image signal processing (ISP) pipeline based on this dataset.
Risk-Sensitive Diffusion: Robustly Optimizing Diffusion Models with Noisy Samples
Yangming Li (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeGenerationOptimizationDiffusion modelScore-based ModelTabularTime SeriesBiomedical DataElectronic Health RecordsStochastic Differential Equation
🎯 What it does: This paper proposes a robust method for training diffusion models on non-image data in the presence of noise, which pairs each noisy sample with its risk vector and introduces a risk-sensitive SDE. By incorporating risk information into the diffusion process, it mitigates the negative impact of noise, achieving robust optimization and sampling of the diffusion model.
Risk-Sensitive Variational Actor-Critic: A Model-Based Approach
Alonso Granados (University of Arizona), Jason Pacheco (University of Arizona)
CodeReinforcement Learning
🎯 What it does: A risk-sensitive actor-critic algorithm based on variational inference, rsV AC, has been developed, capable of simultaneously learning risk-averse and risk-seeking strategies in environments with unknown transition dynamics and stochastic rewards.
RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style
Yantao Liu (Fudan University), Juanzi Li (Tsinghua University)
CodeLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: The RM-BENCH benchmark is proposed to evaluate the ability of reward models to distinguish subtle content differences and resist style bias, demonstrating a high correlation with the performance of policy models.
CodeTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: The RMB benchmark is proposed to evaluate the effectiveness of reward models in LLM alignment, covering 49 fine-grained real-world scenarios, and providing pairwise and Best-of-N evaluations.
🎯 What it does: This paper proposes a real-time multipurpose segmentation model RMP-SAM, capable of performing image panoptic segmentation, video instance segmentation, and interactive segmentation all at once;
🎯 What it does: A robust radar-camera fusion 3D object detection framework, RobuRCDet, is proposed, focusing on improving detection performance in harsh environments and noisy conditions.
🎯 What it does: This paper proposes a robust barycenter estimation method based on Semi-Unbalanced Optimal Transport (SUOT), which maintains good performance even when the data contains outliers or class imbalance.
Robust Function-Calling for On-Device Language Model via Function Masking
Qiqiang Lin (OPPO Research Institute), Weinan Zhang (Shanghai Jiao Tong University)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: To address the robustness issue of language models when executing external function calls, a series of lightweight models called Hammer has been proposed, specifically designed for on-device function calls.
🎯 What it does: A robust model for one-time denoising and classification is designed by aligning the representation of noisy and clean samples on the probabilistic flow trajectory.
Robust Root Cause Diagnosis using In-Distribution Interventions
Lokesh Nagalapatti (Indian Institute of Technology Bombay), Amit Sharma (Microsoft Research India)
CodeAnomaly DetectionTabular
🎯 What it does: The In-Distribution Interventions (IDI) method is proposed, which uses interventions conducted only within the training distribution to evaluate the fix conditions of root cause nodes, thereby achieving root cause diagnosis for cloud services and industrial system anomalies.
CodeMeta LearningDrug DiscoveryAuto EncoderBiomedical Data
🎯 What it does: In the scenario of simulation-based inference (SBI) with missing data, a model called RISE is proposed to jointly learn missing value imputation and posterior estimation, achieving robust inference against varying missing rates.
Robust Transfer of Safety-Constrained Reinforcement Learning Agents
Markel Zubia (Ruhr University Bochum), Nils Jansen (Radboud University Nijmegen)
CodeSafty and PrivacyReinforcement Learning
🎯 What it does: This paper studies a method for training safe agents in the source environment and robustly transferring them to the target environment, ensuring that safety constraints are met in the target environment.
🎯 What it does: A new benchmark W-Bench is proposed for four types of editing tasks: image reconstruction, global editing, local editing, and image-to-video generation, and a robust watermark model VINE is designed based on spectral analysis and generative priors.
Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis
Hyun woo Lee, Hyunju Kim (Korea Institute of Energy Technology)
CodeOptimizationConvolutional Neural NetworkImagePhysics Related
🎯 What it does: A weight initialization method based on fixed point analysis of tanh(ax) is proposed, aimed at keeping the activation values of deep tanh networks from vanishing or saturating, without the need for batch normalization or layer normalization.
🎯 What it does: This paper proposes a reprogramming method that does not change the parameters of the trained deep model to enhance its robustness against adversarial or noise perturbations.
RocketEval: Efficient automated LLM evaluation via grading checklist
Tianjun Wei (City University of Hong Kong), Jianghong Ma (Harbin Institute of Technology Shenzhen)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: RocketEval is proposed, an automated evaluation framework that utilizes lightweight LLMs as judges, transforming assessments into multi-dimensional question-and-answer formats using instance-specific checklists.
Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions
Zhihao He (Shanghai Jiao Tong University), Weiyao Lin (Ant Group)
CodeGenerationComputational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper introduces Rodimus (a pure recursive linear attention model) and Rodimus+ (a hybrid model combining sliding window shared key attention), achieving O(1) word generation complexity and outperforming existing LLMs on multiple benchmarks.
Root Cause Analysis of Anomalies in Multivariate Time Series through Granger Causal Discovery
Xiao Han (Utah State University), Shuhan Yuan (Utah State University)
CodeAnomaly DetectionAuto EncoderTime Series
🎯 What it does: This paper proposes an end-to-end framework named AERCA, which combines Granger causality discovery with root cause analysis by inferring the distribution of exogenous variables through an encoder and detecting anomalous root causes during deployment.
Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models
Qiong Wu (Xiamen University), Rongrong Ji (Xiamen University)
CodeComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsVision Language ModelMultimodality
🎯 What it does: This paper proposes the RoE (Routing Experts) method, treating the pre-trained multimodal large language model (MLLM) as a sparse expert network, achieving example-dependent computational paths through dynamic hierarchical routing, significantly improving inference efficiency.
🎯 What it does: A row-level Top-k selection algorithm specifically designed for GPUs, RTop-K, is proposed, which implements efficient parallel row-level Top-k selection using binary search and supports an early stopping mechanism.
RuAG: Learned-rule-augmented Generation for Large Language Models
Yudi Zhang (Eindhoven University of Technology), Qi Zhang (Microsoft)
CodeGenerationAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringTextTime SeriesRetrieval-Augmented Generation
🎯 What it does: This paper proposes a method for automatically generating and refining first-order logic rules (RuAG) using large language models (LLMs). It compresses offline data into interpretable rules and then injects these rules into LLM prompts in natural language to enhance the reasoning and generation performance of LLMs across multiple tasks.
S4M: S4 for multivariate time series forecasting with Missing values
Peng Jing, Xiaoxiao Li (University of British Columbia)
CodeRecurrent Neural NetworkTransformerTime Series
🎯 What it does: An end-to-end temporal prediction framework S4M is proposed, which can directly handle block missing values in multivariate time series.
CodeComputational EfficiencyTransformerLarge Language ModelImageVideoTextMultimodality
🎯 What it does: This paper proposes SageAttention, an attention acceleration method based on INT8 quantization, achieving efficient inference through smoothing the K matrix, using FP16 accumulators, and adaptive quantization.
SAGEPhos: Sage Bio-Coupled and Augmented Fusion for Phosphorylation Site Detection
Jingjie Zhang (Chinese University of Hong Kong), Chunbin Gu (Chinese University of Hong Kong)
CodeDrug DiscoveryGraph Neural NetworkTransformerBiomedical Data
🎯 What it does: A kinase-substrate dual-modal fusion framework SAGEPhos based on structural information is proposed for phosphorylation site prediction.
🎯 What it does: We propose SAM 2, a unified Promptable visual segmentation model that can perform interactive segmentation on single-frame images and real-time streaming processing in videos, supporting point, box, and mask prompts.
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
Liliang Ren (Microsoft), Weizhu Chen (Microsoft)
CodeGenerationRetrievalComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes SAMBA, a linear-time long sequence language model that interleaves the Mamba state space model with sliding window attention (SWA) at a hierarchical level.
🎯 What it does: A scalable data value assessment method called SAVA is proposed, which performs calculations based on hierarchical optimal transport (OT) at the batch level to address the memory bottleneck issue of the original LAVA on large-scale datasets.
🎯 What it does: A scalable and verifiable graph unlearning method, ScaleGUN, is proposed, which can achieve efficient and verifiable graph neural network data deletion at the billion-edge level.
Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video
Xiaohao Xu (University of Michigan), Xiaonan Huang (University of Michigan)
CodeData SynthesisPose EstimationDepth EstimationOptimizationGaussian SplattingSimultaneous Localization and MappingVideoBenchmark
🎯 What it does: This study proposes a scalable noise data synthesis pipeline, the Robust-Ego3D benchmark, and the corresponding CorrGS method for robust self-motion estimation and high-quality 3D reconstruction in noisy videos.
Scalable Influence and Fact Tracing for Large Language Model Pretraining
Tyler A. Chang (Google DeepMind), Ian Tenney (Google DeepMind)
CodeRetrievalOptimizationTransformerLarge Language ModelText
🎯 What it does: A gradient-based influence method named TrackStar is proposed to efficiently retrieve training samples that significantly impact model predictions during the pre-training phase of large language models (8B parameters) and to perform 'fact tracking' for factual reasoning tasks.
🎯 What it does: This paper proposes the Scalable Mechanistic Neural Network (S-MNN), which improves the original Mechanistic Neural Network (MNN) to efficiently model long time series while balancing prediction and equation discovery.