NeurIPS 2024 Papers — Page 19
Conference on Neural Information Processing Systems · 4035 papers
Infusing Self-Consistency into Density Functional Theory Hamiltonian Prediction via Deep Equilibrium Models
Zun Wang (Microsoft Research AI4Science), Bin Shao (Microsoft Research AI4Science)
Graph Neural NetworkGraphPhysics Related
🎯 What it does: In this study, the authors propose a DFT Hamiltonian prediction architecture called DEQHNet based on a deep equilibrium model, which can achieve self-consistent Hamiltonian solving without relying on traditional DFT iterations.
Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing
Zhongwang Zhang (Shanghai Jiao Tong University), Zhi-Qin John Xu (Shanghai Jiao Tong University)
TransformerTabular
🎯 What it does: This study investigates the learning behavior of Transformers in combinatorial tasks, exploring the impact of parameter initialization scale on inferential and symmetric solutions, and provides a tunable scheme for the initialization rate γ.
Initializing Services in Interactive ML Systems for Diverse Users
Avinandan Bose (University of Washington), Maryam Fazel (University of Washington)
Recommendation SystemOptimizationTabular
🎯 What it does: An initialization algorithm based on adaptive user sampling (AcQUIRE) is proposed for multi-service machine learning systems, which quickly determines a set of service parameters to minimize the total loss of all users under the conditions of only obtaining bandit feedback and a non-convex objective function.
Initializing Variable-sized Vision Transformers from Learngene with Learnable Transformation
Shiyu Xia (Southeast University), Xin Geng (Southeast University)
Knowledge DistillationRepresentation LearningTransformerImage
🎯 What it does: The LeTs method is proposed, which transforms Learngene through learnable width and depth transformation matrices, enabling the initialization of multi-size ViT models with a single training session.
Injecting Undetectable Backdoors in Obfuscated Neural Networks and Language Models
Alkis Kalavasis (Yale University), Manolis Zampetakis (Yale University)
Safty and PrivacyAdversarial AttackText
🎯 What it does: This paper studies how to implant undetectable and non-reproducible backdoors in neural networks and language models after equivalent indistinguishability obfuscation (iO), thereby achieving covert manipulation of model outputs.
Input-to-State Stable Coupled Oscillator Networks for Closed-form Model-based Control in Latent Space
Maximilian Stölzle (Delft University of Technology), Cosimo Della Santina (Delft University of Technology)
OptimizationRobotic IntelligenceRecurrent Neural NetworkAuto EncoderImageOrdinary Differential Equation
🎯 What it does: This paper proposes an input-state stable Coupled Oscillator Network (CON) for learning and controlling the dynamics of physical systems in a low-dimensional latent space.
Instance-adaptive Zero-shot Chain-of-Thought Prompting
Xiaosong Yuan (Jilin University), Jieping Ye (Alibaba Cloud Computing)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes the Instance-Adaptive Zero-shot Chain of Thought (IAP) method, which dynamically selects the most suitable prompts for each question through significant analysis of information flow, thereby improving the reasoning accuracy of large language models.
Instance-Optimal Private Density Estimation in the Wasserstein Distance
Vitaly Feldman (Apple), Kunal Talwar (Apple)
OptimizationSafty and Privacy
🎯 What it does: The research addresses the problem of differential privacy density estimation under Wasserstein distance and proposes an instance-optimal algorithm.
Instance-Specific Asymmetric Sensitivity in Differential Privacy
David Durfee (Mozilla)
Safty and PrivacyImageTabular
🎯 What it does: A new differential privacy estimation framework is proposed - the Asymmetric Sensitivity Mechanism.
InstructG2I: Synthesizing Images from Multimodal Attributed Graphs
Bowen Jin (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
GenerationData SynthesisGraph Neural NetworkTransformerDiffusion modelMultimodalityGraph
🎯 What it does: Proposes the Graph2Image task and designs the INSTRUCTG2I model to achieve image generation based on multimodal attribute graphs.
Instruction Tuning With Loss Over Instructions
Zhengyan Shi (University College London), Aldo Lipani (University College London)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the Instruction Modelling (IM) method, which calculates the loss for both the instruction and response parts during instruction tuning, rather than only for the response part, thereby improving the performance of language models on various NLP tasks and open-ended generation tasks.
Instruction-Guided Visual Masking
Jinliang Zheng (Tsinghua University), Xianyuan Zhan
Object DetectionSegmentationTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelImageMultimodality
🎯 What it does: A command-guided visual mask (IVM) model is proposed, which significantly enhances multimodal instruction-following performance by automatically generating visual masks to eliminate areas in images that are irrelevant to the instructions.
Instructor-inspired Machine Learning for Robust Molecular Property Prediction
Fang Wu (Stanford University), Stan Z. Li (Westlake University)
Drug DiscoveryGraph Neural NetworkTransformerSupervised Fine-TuningGraphTabularBiomedical Data
🎯 What it does: This paper proposes InstructMol, a semi-supervised framework based on instruction learning that generates pseudo-labels using a target molecular model, and then evaluates the reliability of these pseudo-labels with an additional guiding model, helping the target model better utilize vast amounts of unlabeled molecular data.
Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentation
Arvind Murari Vepa (University of California Los Angeles), Yizhou Sun (University of California Los Angeles)
SegmentationConvolutional Neural NetworkContrastive LearningImageVideoBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a slice-level active learning framework that combines deep metric learning with Coreset for 3D medical image segmentation.
Integrating GNN and Neural ODEs for Estimating Non-Reciprocal Two-Body Interactions in Mixed-Species Collective Motion
Masahito Uwamichi (University of Tokyo), Satoshi Sawai (University of Tokyo)
Graph Neural NetworkGraphTime SeriesStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a deep learning framework that combines graph neural networks with neural ODEs to estimate two-body non-reciprocal interactions from the collective motion trajectories of mixed species.
Integrating Suboptimal Human Knowledge with Hierarchical Reinforcement Learning for Large-Scale Multiagent Systems
Dingbang Liu (University of Wollongong), Guoxin Su (University of Wollongong)
Graph Neural NetworkReinforcement LearningGraph
🎯 What it does: A hierarchical structure-based multi-agent reinforcement learning framework (hhk-MARL) is proposed, which significantly enhances the learning efficiency and final performance of large-scale multi-agent systems by combining abstract human sub-optimal knowledge represented by fuzzy logic with group control in a graph structure.
Interaction-Force Transport Gradient Flows
Egor Gladin (Humboldt University of Berlin), Jia-Jie Zhu (Weierstrass Institute for Applied Analysis and Stochastics)
OptimizationTabular
🎯 What it does: A new geometry of Interactive Force Transmission (IFT) gradient flow is proposed, combining unbalanced optimal transport of Wasserstein and MMD tensors, and a particle optimization algorithm based on JKO splitting is provided, proving global exponential convergence under MMD and KL energy.
Interactive Deep Clustering via Value Mining
Honglin Liu (Sichuan University), Xi Peng (Sichuan University)
Representation LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: In the task of unlabeled deep clustering, this paper proposes an Interactive Deep Clustering (IDC) framework that utilizes a small amount of human interaction to correct the misallocations of a pre-trained clustering model at the cluster boundaries.
InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint
Zhenzhi Wang (Chinese University of Hong Kong), Bo Dai (University of Hong Kong)
GenerationPose EstimationOptimizationLarge Language ModelDiffusion modelVideo
🎯 What it does: This paper studies a zero-shot multi-person interaction generation method called InterControl, based on a single-person motion generation model, which can achieve interactive actions for any number of people by precisely controlling the position of each joint.
InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction
Sirui Xu (University of Illinois Urbana-Champaign), Liangyan Gui
GenerationData SynthesisPose EstimationTransformerLarge Language ModelWorld ModelTextRetrieval-Augmented Generation
🎯 What it does: Using LLM and text-motion models to generate semantically aligned human actions, and then retrieving initial object poses and predicting object dynamics based on contact vertices in a world model, thereby achieving zero-shot text-guided 3D human-computer interaction sequence generation.
Interfacing Foundation Models' Embeddings
Xueyan Zou, Lijuan Wang
Object DetectionSegmentationRetrievalTransformerVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: The FIND interface is proposed, which utilizes embeddings generated by foundational models (Vision and Language) to uniformly handle multimodal tasks such as segmentation, localization, and retrieval in a lightweight Transformer, achieving cross-retrieval and segmentation from pixel-level to image-level under the same model weights.
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD
Xiaoyi Dong (Shanghai Artificial Intelligence Laboratory), Jiaqi Wang (Shanghai Artificial Intelligence Laboratory)
RecognitionOptimizationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: Developed InternLM‑XComposer2‑4KHD, a multimodal large model that supports resolutions from 336 pixels to 4K HD (3840×1600) and even higher, achieving significant improvements on 16 benchmarks including OCR and visual reasoning.
Interpolating Item and User Fairness in Multi-Sided Recommendations
Qinyi Chen (Massachusetts Institute of Technology), Djallel Bouneffouf (IBM Research)
Recommendation SystemOptimizationTabular
🎯 What it does: A multi-party fair recommendation framework called Problem (FAIR) is proposed, and an online fair recommendation algorithm FORM is designed, balancing platform revenue with fairness for items and users.
Interpret Your Decision: Logical Reasoning Regularization for Generalization in Visual Classification
Zhaorui Tan (Xi'an-Jiaotong Liverpool University), Kaizhu Huang (Duke Kunshan University)
ClassificationDomain AdaptationExplainability and InterpretabilityConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a logic-based regularization method called L-Reg to improve the generalization performance of visual classification models in multi-domain generalization, multi-object discovery, and their combined scenarios.
Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents
Quentin Delfosse (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
Explainability and InterpretabilityKnowledge DistillationReinforcement LearningVideo
🎯 What it does: Designed and evaluated an interpretable reinforcement learning agent called Successive Concept Bottleneck Agents (SCoBots), which achieves interpretable decision-making processes in RL tasks through multi-layer concept bottlenecks.
Interpretable Concept-Based Memory Reasoning
David Debot (KU Leuven), Giuseppe Marra (KU Leuven)
ClassificationExplainability and InterpretabilityImage
🎯 What it does: An interpretable and verifiable concept-based model CMR is proposed, which selects rules from a learnable logic rule memory through a neural selection mechanism and performs symbolic evaluation, achieving globally interpretable task predictions.
Interpretable Generalized Additive Models for Datasets with Missing Values
Hayden McTavish (Duke University), Cynthia Rudin (Duke University)
OptimizationExplainability and InterpretabilityTabularBiomedical Data
🎯 What it does: This paper proposes a sparse interpretable Generalized Additive Model (M-GAM) that directly embeds missing indicators and their interaction terms into the model to handle missing data, rather than the traditional approach of filling in data before prediction.
Interpretable Image Classification with Adaptive Prototype-based Vision Transformers
Chiyu Ma (Dartmouth), Chaofan Chen (Maine)
ClassificationExplainability and InterpretabilityTransformerImage
🎯 What it does: This paper proposes ProtoViT, an interpretable image classification method that combines Vision Transformer with deformable prototypes, utilizing case-based reasoning to provide 'looks like...' explanations.
Interpretable Lightweight Transformer via Unrolling of Learned Graph Smoothness Priors
VIET HO TAM THUC DO, Philip Chou
Image TranslationRestorationExplainability and InterpretabilityComputational EfficiencyGraph Neural NetworkTransformerImage
🎯 What it does: Construct a interpretable and lightweight Transformer network, achieving efficient recovery of image interpolation and colorization tasks through the iterative optimization algorithm of minimizing Graph Laplacian Regularization (GLR) or Graph Total Variation (GTV).
Interpretable Mesomorphic Networks for Tabular Data
Arlind Kadra (University of Freiburg), Josif Grabocka (University of Technology Nuremberg)
ClassificationExplainability and InterpretabilityTabularBenchmark
🎯 What it does: A new interpretable deep neural network is proposed - IMN (Interpretable Mesomorphic Neural Networks), which generates linear explanation models for each sample through a hypernetwork, achieving instance-level interpretability while maintaining the accuracy of deep learning.
Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge
Fawaz Sammani (Vrije Universiteit Brussel), Nikos Deligiannis (Vrije Universiteit Brussel)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A multimodal explanation method based on text concepts is proposed, utilizing the 'information channel' shared by visual and language encoders to explain the zero-shot image classification of the CLIP model; at the same time, shared knowledge between visual and language encoders is defined and quantified, analyzing its impact on model performance.
Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)
Usha Bhalla (Harvard University), Himabindu Lakkaraju (Harvard University)
RetrievalExplainability and InterpretabilityContrastive LearningImageText
🎯 What it does: This paper proposes a training-free, task-agnostic sparse linear concept embedding method called SpLiCE, which converts the high-dimensional dense vectors of CLIP into non-negative sparse linear combinations of interpretable concept vectors.
Interpreting Learned Feedback Patterns in Large Language Models
Luke Marks (Apart Research), Fazl Barez (University of Oxford)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: By training a detector to predict the implicit feedback signals learned from RLHF, we measure the consistency of the learning feedback patterns of LLMs with human feedback.
Interpreting the Weight Space of Customized Diffusion Models
Amil Dravid, Kfir Aberman
GenerationData SynthesisExplainability and InterpretabilitySupervised Fine-TuningDiffusion modelImage
🎯 What it does: Proposed and implemented the 'weights2weights (w2w)' space, constructing an interpretable subspace with model weights to achieve sampling, editing, and single-image inversion of customized diffusion models;
Intervention and Conditioning in Causal Bayesian Networks
sainyam galhotra, Joseph Halpern
🎯 What it does: This paper studies the probability calculation problem of interventions and conditioning in Causal Bayesian Networks (CBN). It proposes a unique probability assessment method for intervention formulas (including sufficiency and necessity probabilities) under the assumption of mechanism independence and proves that these probabilities can be estimated from observational data.
Interventional Causal Discovery in a Mixture of DAGs
Burak Varıcı (Carnegie Mellon University), Ali Tajer (Rensselaer Polytechnic Institute)
Graph
🎯 What it does: Utilizing intervention data for causal relationship learning on mixed DAG models (multiple coexisting directed acyclic graphs), this paper presents both theoretical and algorithmic components: it first provides the necessary and sufficient conditions for the size of interventions required to learn the true edges; then it designs the CADIM algorithm, which approximates the optimal intervention size within O(n²) interventions and can simultaneously identify and orient all true edges.
Interventionally Consistent Surrogates for Complex Simulation Models
Joel Dyer (University of Oxford), Michael J. Wooldridge
Recurrent Neural NetworkTime SeriesOrdinary Differential Equation
🎯 What it does: A framework for proxy modeling is proposed that utilizes causal abstraction theory learning and complex simulation models to maintain consistency under target interventions, aiming to reduce the high-cost simulation time while ensuring that policy experiments conducted on the proxy model align with the results of the original model.
IntraMix: Intra-Class Mixup Generation for Accurate Labels and Neighbors
Shenghe Zheng (Harbin Institute of Technology), Xianglong Liu (Harbin Institute of Technology)
Graph Neural NetworkSupervised Fine-TuningGraphPhysics Related
🎯 What it does: A novel data augmentation framework called IntraMix is proposed in graph neural networks to address the two major challenges of label scarcity and incomplete neighbors.
Intrinsic Robustness of Prophet Inequality to Strategic Reward Signaling
Wei Tang (Chinese University of Hong Kong), Derek Zhu (University of Chicago)
🎯 What it does: This paper studies the intrinsic robustness of the prophet inequality under strategic reward signals and explores the performance of simple threshold stopping strategies in the face of selective information disclosure by self-interested players.
Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting
Bong Gyun Kang (Seoul National University), Sungroh Yoon (Seoul National University)
OptimizationTransformerTime Series
🎯 What it does: Proposes a Spectral Attention mechanism that utilizes exponential moving average and multi-frequency attention in time series forecasting, maintaining temporal correlation between samples and achieving gradient backpropagation across steps;
Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity
Kaiqu Liang (Princeton University), Jaime Fernández Fisac (Princeton University)
Safty and PrivacyRobotic IntelligenceTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: In response to execution errors caused by inaccurate reasoning, task ambiguity, and safety risks in large language models for robotic planning, this paper proposes an 'Introspective Planning' framework. It constructs a knowledge base containing examples of posterior reasoning and retrieves similar cases during reasoning, guiding the model to first self-assess the compliance and safety of the plan before making decisions. Subsequently, this framework is combined with conformal prediction calibrated by quantiles to ensure statistical success rates and reduce excessive inquiries.
Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level
Runlin Lei (Renmin University of China), Zhewei Wei (Renmin University of China)
Explainability and InterpretabilityAdversarial AttackGraph Neural NetworkLarge Language ModelTextGraph
🎯 What it does: The study conducts text-level graph injection attacks (GIA) on Text Attribute Graphs (TAG) and proposes and evaluates three attack designs: ITGIA, VTGIA, and WTGIA.
Invariant subspaces and PCA in nearly matrix multiplication time
Aleksandros Sobczyk (IBM Research and ETH Zurich), Mathieu Luisier (ETH Zurich)
OptimizationComputational Efficiency
🎯 What it does: The paper proposes an algorithm for calculating the spectral projector of the Hermitian stationary generalized eigenvalue problem using approximate matrix multiplication time under a floating-point model, and applies it to PCA and Block-Krylov PCA.
Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation
Keqiang Yan (Texas A&M University), Shuiwang Ji (University of Illinois Urbana-Champaign)
GenerationData SynthesisTransformerLarge Language ModelGraph
🎯 What it does: A method called Mat2Seq has been developed to convert crystal structures into a unique and complete 1D sequence, and based on this, new crystal structures are generated using language models.
Inverse Factorized Soft Q-Learning for Cooperative Multi-agent Imitation Learning
The Viet Bui (Singapore Management University), Thanh Hong Nguyen (University of Oregon)
Reinforcement Learning
🎯 What it does: This paper proposes a Multi-Agent Inverse Soft Q Learning method (MIFQ), which achieves centralized training and decentralized execution through a hybrid network and hypernetwork for imitation learning.
Inverse M-Kernels for Linear Universal Approximators of Non-Negative Functions
Hideaki Kim (NTT Corporation)
OptimizationTime Series
🎯 What it does: The concept of the inverse M-kernel is proposed, and a linear non-negative approximator is constructed in a one-dimensional input space, proving that this approximator can achieve universal approximation of non-negative functions.
Inversion-based Latent Bayesian Optimization
Jaewon Chu (Korea University), Hyunwoo J. Kim (Korea University)
OptimizationDrug DiscoveryAuto EncoderTabular
🎯 What it does: This paper proposes a latent Bayesian optimization framework based on an inverse decoder, InvBO, which addresses the alignment issues caused by reconstruction errors in traditional LBO and improves the selection of trust region anchor points.
InversionView: A General-Purpose Method for Reading Information from Neural Activations
Xinting Huang (Saarland University), Michael Hahn (Saarland University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: The InversionView method is proposed, which visualizes and explains the information encoded in neural network activations by recovering inputs from activation vectors through training a conditional decoder.
Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps
Nikita Starodubcev (Yandex Research), Dmitry Baranchuk (Yandex Research)
GenerationKnowledge DistillationDiffusion modelImageTextOrdinary Differential Equation
🎯 What it does: Proposes an Inverse Consistency Distillation (iCD) framework that can complete text-guided image generation and image encoding/inversion in 3-4 steps;
Invisible Image Watermarks Are Provably Removable Using Generative AI
Xuandong Zhao (University of California Berkeley), Lei Li (Carnegie Mellon University)
RestorationGenerationAdversarial AttackDiffusion modelAuto EncoderImage
🎯 What it does: This paper studies a new regeneration attack, which adds random noise to the image embedding space and utilizes generative/denoising models to reconstruct the image, thereby removing invisible pixel-level watermarks while maintaining image quality.
IODA: Instance-Guided One-shot Domain Adaptation for Super-Resolution
Zaizuo Tang (Nanjing University), Yu-Bin Yang (Nanjing University)
RestorationSuper ResolutionDomain AdaptationContrastive LearningImage
🎯 What it does: This paper proposes a one-time domain adaptation method for super-resolution networks, called IODA, which utilizes only a single unannotated low-resolution image from the target domain.
IPM-LSTM: A Learning-Based Interior Point Method for Solving Nonlinear Programs
Xi Gao (Xi'an Jiaotong University), Qingjiang Shi (Tongji University)
OptimizationRecurrent Neural NetworkSupervised Fine-TuningTabular
🎯 What it does: This paper proposes a learning-based interior point method (IPM) called IPM-LSTM by replacing the steps of solving linear equations in traditional interior point methods with a trained long short-term memory network (LSTM) to approximate solutions. It constructs a two-stage framework: first, using IPM-LSTM to obtain high-quality approximate primal-dual solutions, and then using these as a warm-start to initiate a standard IPM solver.
IPO: Interpretable Prompt Optimization for Vision-Language Models
Yingjun Du (University of Amsterdam), Cees G. M. Snoek (University of Amsterdam)
OptimizationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes an interpretable prompt optimization method (IPO) that utilizes large language models (LLMs) to dynamically generate and optimize text prompts for visual-language models (such as CLIP), replacing traditional gradient descent methods, and combines multimodal models to generate image descriptions, thereby enhancing the model's generalization ability to new categories.
IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering
Ruosen Li (University of Texas at Dallas), Xinya Du (University of Texas at Dallas)
Large Language ModelText
🎯 What it does: An automatic evaluation framework for human-computer interaction question-answering systems, IQA-EVAL, is proposed, utilizing large language models (LLMs) as evaluation agents (LEA) to simulate human behavior and assess interactions.
IR-CM: The Fast and General-purpose Image Restoration Method Based on Consistency Model
Xiaoxuan Gong (Huazhong University of Science and Technology), Jie Ma (Huazhong University of Science and Technology)
RestorationImageStochastic Differential Equation
🎯 What it does: A multi-task rapid image restoration method IR-CM based on a consistency model is proposed, achieving first-order or low-order sampling inference.
IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons
Dan Shi (Tianjin University), Deyi Xiong (Tianjin University)
GenerationOptimizationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This study investigates the generation issues of LLMs when faced with conflicts between context and pre-trained knowledge, proposing the IRCAN framework to enhance the model's fidelity to new information by identifying and amplifying context-sensitive neurons.
Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
Jiayu Wang (University of Wisconsin Madison), Neel Joshi (Microsoft Research)
TransformerLarge Language ModelVision Language ModelTextMultimodalityBenchmark
🎯 What it does: Four types of multimodal spatial reasoning tasks (Spatial-Map, Maze-Nav, Spatial-Grid, Spatial-Real) were constructed, and a systematic evaluation of the performance of LLMs and VLMs under text, visual, and bimodal inputs was conducted.
Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
Dylan J Foster, Dipendra Misra (Microsoft Research)
Reinforcement LearningSequential
🎯 What it does: This paper conducts a new theoretical analysis of the log-loss used in Behavior Cloning (BC) to study the relationship between offline and online imitation learning (IL) in terms of sample complexity and time horizon.
Is Cross-validation the Gold Standard to Estimate Out-of-sample Model Performance?
Garud Iyengar (Columbia University), Tianyu Wang (Columbia University)
OptimizationComputational EfficiencyData-Centric LearningTabularFinance Related
🎯 What it does: This paper discusses the statistical performance of models evaluated using cross-validation versus direct evaluation with the training set, proving that in most non-parametric models, simple 'plug-in' evaluation is more robust compared to K-fold cross-validation.
Is Knowledge Power? On the (Im)possibility of Learning from Strategic Interactions
Nivasini Ananthakrishnan (University of California Berkeley), Kunhe Yang (University of California Berkeley)
🎯 What it does: The study investigates whether players can learn and achieve their Stackelberg value solely through strategic interaction in repeated games with complete or partial information asymmetry.
Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?
Yang Dai (National University of Defense Technology), Li Shen (Shenzhen Campus of Sun Yat-sen University)
OptimizationTransformerReinforcement LearningSequential
🎯 What it does: A Transformer-style Decision Mamba (DeMa) model is proposed and evaluated, demonstrating its compatibility with offline reinforcement learning tasks for trajectory optimization.
Is Multiple Object Tracking a Matter of Specialization?
Gianluca Mancusi (University of Modena and Reggio Emilia), Simone Calderara (University of Modena and Reggio Emilia)
Object TrackingDomain AdaptationTransformerSupervised Fine-TuningVideo
🎯 What it does: The PASTA framework is proposed, which splits the Transformer-based multi-object tracker into parameter-efficient fine-tuning modules specialized by scene attributes (such as lighting, viewpoint, density, position, camera motion), achieving a composable model with no additional inference time.
Is O(log N) practical? Near-Equivalence Between Delay Robustness and Bounded Regret in Bandits and RL
Enoch H. Kang (University of Washington), Panganamala Kumar
OptimizationReinforcement Learning
🎯 What it does: This paper studies the impact of delayed anonymous rewards on the robustness of consistency algorithms in interactive decision-making, revealing that a Graves-Lai constant of zero is a necessary and sufficient condition for achieving bounded returns while being robust to delayed model errors.
Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Models.
Athanasios Tragakis (University of Glasgow), Daniele Faccio (University of Glasgow)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: Proposes the Pixelsmith framework, which can generate any ultra-high-resolution text-to-image content using a single GPU;
Is Programming by Example Solved by LLMs?
Wen-Ding Li (Cornell University), Kevin Ellis (Cornell University)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper studies the effectiveness of large language models (LLM) in programming-by-example (PBE) tasks, proposing self-supervised data generation and fine-tuning of LLMs, further introducing adaptation mechanisms to enhance the generalization ability of discrete distributions, and conducting experiments in three typical domains: list functions, text editing, and LOGO graphics.
Is Score Matching Suitable for Estimating Point Processes?
Haoqun Cao (Renmin University of China), Feng Zhou (Renmin University of China)
Score-based ModelPoint Cloud
🎯 What it does: This paper proposes a Weighted Score Matching (WSM) and Autoregressive Weighted Score Matching (AWSM) method for point process parameter estimation, demonstrating their consistency and convergence.
Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization
Wei Liu (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
Explainability and InterpretabilityTabular
🎯 What it does: This paper proposes a new interpretability criterion - Maximum Residual Difference (MRD), which allows for the efficient extraction of causal explanation subsets even in datasets with spurious correlated features.
Is Value Learning Really the Main Bottleneck in Offline RL?
Seohong Park (University of California Berkeley), Aviral Kumar (Carnegie Mellon University)
Reinforcement Learning
🎯 What it does: This paper conducts systematic experiments and data scaling studies, breaking down the bottlenecks of offline reinforcement learning into value function learning, policy extraction, and generalization during testing, and quantifies their impact on performance.
Is Your LiDAR Placement Optimized for 3D Scene Understanding?
Ye Li (University of Michigan), Xiaonan Huang (University of Michigan)
Autonomous DrivingOptimizationPoint Cloud
🎯 What it does: This paper proposes a full-cycle multi-radar placement evaluation and optimization framework called Place3D, which combines semantic occupancy grid evaluation metrics, CMA-ES optimization, and large-scale simulation data generation to systematically study multi-radar arrangements in autonomous driving.
Iteration Head: A Mechanistic Study of Chain-of-Thought
Vivien Cabannes (Meta AI), Julia Kempe (New York University)
OptimizationComputational EfficiencyTransformerSequentialChain-of-Thought
🎯 What it does: This study investigates the reasoning of Transformers in iterative tasks through Chain of Thought (CoT), revealing and validating the mechanism of the 'iteration head' and exploring the impact of data preprocessing on model learning.
Iterative Methods via Locally Evolving Set Process
Baojian Zhou (Fudan University), Yanghua Xiao (Fudan University)
OptimizationComputational EfficiencyGraph
🎯 What it does: This paper proposes a framework based on a local evolution set process that can localize standard iterative solvers (such as Gauss-Seidel, gradient descent, Chebyshev, Heavy-Ball) to achieve efficient solutions for the approximate personalized PageRank (PPR) problem.
Iterative Reasoning Preference Optimization
Richard Yuanzhe Pang (Meta FAIR), Jason E Weston
OptimizationTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: By iteratively generating multiple Chain-of-Thought (CoT) schemes, constructing correct/incorrect answer pairs, and fine-tuning the model using an improved DPO+NLL loss, the reasoning ability of the LLM is enhanced.
Iteratively Refined Behavior Regularization for Offline Reinforcement Learning
Yi Ma (Shanxi University), Chenjun Xiao (Chinese University of Hongkong)
Reinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes an offline reinforcement learning algorithm called Conservative Policy Iteration (CPI) that iteratively refines behavior regularization. By continuously updating the reference policy in behavior regularization, the learning process remains within the support of the data, theoretically converging to the optimal policy within the sample in tabular environments. Experiments demonstrate that it outperforms existing methods in various offline control tasks.
Iteratively Refined Early Interaction Alignment for Subgraph Matching based Graph Retrieval
Ashwin Ramachandran (University of California San Diego), Abir De (Indian Institute of Technology Bombay)
RetrievalGraph Neural NetworkGraph
🎯 What it does: A new early interaction network called IsoNet++ is proposed for graph retrieval based on subgraph isomorphism, aiming to improve retrieval performance through iterative refinement of alignment.
iVideoGPT: Interactive VideoGPTs are Scalable World Models
Jialong Wu (Tsinghua University), Mingsheng Long (Tsinghua University)
Robotic IntelligenceTransformerReinforcement LearningWorld ModelVideo
🎯 What it does: Designed and trained a scalable interactive video GPT model iVideoGPT for learning large-scale visual world models and achieving action-conditioned video prediction, visual planning, and model-based reinforcement learning in downstream tasks.
IWBVT: Instance Weighting-based Bias-Variance Trade-off for Crowdsourcing
Wenjun Zhang (China University of Geosciences), Chaoqun Li (China University of Geosciences)
ClassificationOptimizationTabular
🎯 What it does: This paper proposes an instance-weighted bias-variance trade-off method (IWBVT) as a post-processing step for existing label fusion and noise correction algorithms, enhancing the predictive quality of models trained on crowdsourced data.
Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters
Haibo Jin (University of Illinois at Urbana-Champaign), Haohan Wang (University of Illinois at Urbana-Champaign)
OptimizationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A new red team benchmark called JAMBench is proposed to trigger content moderation defenses in large language models, along with a jailbreak method named JAM that effectively bypasses security guards at the input and output layers.
JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models
Kun Zhou (Renmin University of China), Ji-Rong Wen (Renmin University of China)
Data SynthesisComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper trains a lightweight LLM with a scale of 7B, using it to generate approximately 4.6B high-quality math question-answer pairs, which are then used to pre-train the JiuZhang3.0 model, enhancing its mathematical reasoning capabilities.
John Ellipsoids via Lazy Updates
David Woodruff, Taisuke Yasuda (Voleon Group)
OptimizationComputational Efficiency
🎯 What it does: An algorithm utilizing lazy updates and fast matrix multiplication has been developed, capable of efficiently approximating the John ellipsoid (minimum enclosing ellipse) of an n-dimensional point set in approximately O(ε⁻¹ n d log(n/d)) time.
Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learning
Divyam Madaan (New York University), Kyunghyun Cho (Genentech)
GenerationRepresentation LearningMixture of ExpertsMultimodalityBiomedical DataMagnetic Resonance ImagingElectronic Health Records
🎯 What it does: A framework called I2M2 is proposed, which simultaneously captures cross-modal and single-modal dependencies, addressing the issue of performance fluctuations caused by focusing solely on one type of dependency in multimodal learning.
Just Add $100 More: Augmenting Pseudo-LiDAR Point Cloud for Resolving Class-imbalance Problem
Mincheol Chang (Korea University), Namil Kim (NAVER LABS)
Object DetectionDomain AdaptationAutonomous DrivingNeural Radiance FieldGenerative Adversarial NetworkGaussian SplattingPoint Cloud
🎯 What it does: By generating pseudo LiDAR point clouds from small models and network videos, and inserting them into the scene during training, the issue of class imbalance in 3D object detection is addressed.
Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling
Jiatao Gu (Apple), Joshua M. Susskind (Apple)
GenerationData SynthesisLarge Language ModelDiffusion modelImageText
🎯 What it does: Proposes Kaleido Diffusion, which enhances the diversity of conditional diffusion models through autoregressive latent priors.
Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learning
Xinran Li (Hong Kong University of Science and Technology), Jun Zhang (Hong Kong University of Science and Technology)
Reinforcement Learning
🎯 What it does: This paper proposes a learnable mask-based adaptive partial parameter sharing mechanism (Kaleidoscope) for multi-agent reinforcement learning, which achieves diversity in agent policies while maintaining sample efficiency.
KALM: Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts
Jing-Cheng Pang (Nanjing University), Yang Yu (Nanjing University)
Robotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringMultimodality
🎯 What it does: By transforming the knowledge of large language models into virtual trajectories, we combine offline reinforcement learning to train robotic agents with low-level control capabilities.
Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting
Fangcheng Liu (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: A self-inference decoding framework named Kangaroo is proposed, which accelerates the inference of large language models through a dual early-exit strategy without the need for additional training of an independent draft model.
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
Kaifeng Lyu (Princeton University), Sanjeev Arora (Princeton University)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper discusses how to maintain safety when fine-tuning aligned large language models by changing prompt templates, proposing and validating the 'Pure Tuning, Safe Testing (PTST)' strategy.
Kermut: Composite kernel regression for protein variant effects
Peter Mørch Groth (University of Copenhagen), Wouter Boomsma (University of Copenhagen)
Protein Structure PredictionBiomedical DataBenchmark
🎯 What it does: A method for predicting the effects of protein variants based on Gaussian processes, called Kermut, is proposed, along with a novel composite kernel function.
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
Alexander V Nikitin, Pekka Marttinen (Aalto University)
GenerationOptimizationTransformerLarge Language ModelText
🎯 What it does: Proposes the Kernel Language Entropy (KLE) method, which uses semantic kernels and von Neumann entropy to finely quantify the semantic uncertainty of LLM-generated texts.
Kernel PCA for Out-of-Distribution Detection
Kun Fang (Shanghai Jiao Tong University), JIE YANG
Anomaly DetectionConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: Using kernel principal component analysis (KPCA) to analyze the reconstruction error of features obtained from DNN training, two task-specific kernels (cosine kernel and cosine-Gaussian kernel) are constructed, and large-scale efficient out-of-distribution (OoD) detection is achieved through explicit feature mapping.
Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithm
Sattar Vakili (MediaTek Research), Julia Olkhovskaya (TU Delft)
Reinforcement Learning
🎯 What it does: An optimistic algorithm KUCB-RL based on kernel ridge regression is proposed in the context of infinite average reward RL.
Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features
Chengkai Hou (Shanghai Qizhi Institute), Huazhe Xu (Tsinghua University)
Object DetectionPose EstimationAuto EncoderPoint Cloud
🎯 What it does: Proposes Key-Grid, an unsupervised 3D keypoint detection method that can handle both rigid and deformable objects, implemented through an autoencoder framework.
KFNN: K-Free Nearest Neighbor For Crowdsourcing
Wenjun Zhang (China University of Geosciences), Chaoqun Li (China University of Geosciences)
ClassificationTabular
🎯 What it does: This paper proposes a K-free nearest neighbor label integration algorithm (KFNN) to automatically determine the number of neighbors for each sample in crowdsourced data and to infer true labels by integrating attribute and noise label information.
KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge
Pengcheng Jiang (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
Graph Neural NetworkLarge Language ModelSupervised Fine-TuningGraph
🎯 What it does: Proposes the KG-FIT framework, which combines the open-world knowledge of LLM with KG embeddings, enhancing graph embeddings through hierarchical clustering and text embedding fine-tuning.
KnowGPT: Knowledge Graph based Prompting for Large Language Models
Qinggang Zhang (Hong Kong Polytechnic University), Xiao Huang (Hong Kong Polytechnic University)
OptimizationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: The KnowGPT framework is proposed, which extracts the most informative reasoning paths from knowledge graphs through reinforcement learning and automatically constructs hard prompts tailored to questions using multi-armed bandits, thereby improving the answer quality of closed-source LLMs.
Knowledge Circuits in Pretrained Transformers
Yunzhi Yao (Zhejiang University), Huajun Chen (Zhejiang University)
TransformerLarge Language ModelText
🎯 What it does: This study discovers and analyzes 'knowledge circuits' in the computation graph of Transformers using a circuit theory-based approach, revealing how the model stores and expresses knowledge such as facts and biases internally.
Knowledge Composition using Task Vectors with Learned Anisotropic Scaling
Frederic Z. Zhang (Australian Institute for Machine Learning), Ehsan Abbasnejad (Australian Institute for Machine Learning)
ClassificationRecognitionDomain AdaptationTransformerContrastive LearningImage
🎯 What it does: This paper proposes a method called aTLAS (Adaptive Task Vector Learning through Non-Proportional Scaling) to efficiently combine knowledge across different tasks, enabling transfer learning and low-data learning.
Knowledge Graph Completion by Intermediate Variables Regularization
Changyi Xiao (Fudan University), Yixin Cao (Fudan University)
Knowledge DistillationGraph
🎯 What it does: A general form of the TDB model is proposed, and based on this, Intermediate Variable Regularization (IVR) is introduced to alleviate the overfitting problem in knowledge graph completion.
Knowledge-Empowered Dynamic Graph Network for Irregularly Sampled Medical Time Series
Yicheng Luo (South China University of Technology), Qianli Ma (South China University of Technology)
Recurrent Neural NetworkGraph Neural NetworkLarge Language ModelTime SeriesBiomedical Data
🎯 What it does: A knowledge-enhanced dynamic graph network (KEDGN) is proposed to handle irregularly sampled medical time series (ISMTS).
KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis
Youngwan Lee (Electronics and Telecommunications Research Institute), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisComputational EfficiencyKnowledge DistillationDiffusion modelImageText
🎯 What it does: A lightweight text-to-image model KOALA is constructed, compressing the U-Net of SDXL and achieving efficient generation through knowledge distillation.
KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension
Jie Yang (Sun Yat-sen University), Ruimao Zhang (Sun Yat-sen University)
RecognitionObject DetectionTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: This paper studies the task of semantic keypoint understanding and proposes a unified model KptLLM, which employs a recognition-detection strategy to accomplish semantic interpretation of keypoints, visual prompt keypoint localization, and text prompt keypoint localization in three major scenarios.