🎯 What it does: A completely unsupervised group recommendation framework ITI (Identify Then Recommend) is proposed, which first automatically identifies user groups in the user embedding space and then performs group recommendation through self-supervised pretext tasks.
Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning
Dan Braun (Apollo Research), Lee Sharkey (ML Alignment and Theory Scholars University of Queensland)
CodeExplainability and InterpretabilityComputational EfficiencyPrompt EngineeringAuto EncoderText
🎯 What it does: This paper proposes and implements an end-to-end sparse dictionary learning (e2e SAE) method to train sparse autoencoders, enabling the learned features to have functional importance in network performance.
Identifying General Mechanism Shifts in Linear Causal Representations
Tianyu Chen (University of Texas at Austin), Pradeep Kumar Ravikumar
CodeRepresentation LearningTabular
🎯 What it does: This study considers the setting of linear causal representation learning, aiming to identify changes in potential causal mechanisms across multiple environments, particularly in the context of imperfect interventions to identify changes between latent factors.
IDGen: Item Discrimination Induced Prompt Generation for LLM Evaluation
Fan Lin (Tencent), Yu Zhang (SouthEast University)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: A framework for prompt generation based on Item Discrimination (ID) theory has been designed and implemented to dynamically and sustainably produce high-discriminative, appropriately difficult LLM evaluation data.
🎯 What it does: This paper studies the impact of weight initialization and training cycles on the adversarial robustness of Graph Neural Networks (GNNs) and Deep Neural Networks (DNNs).
🎯 What it does: The IF-Font framework is proposed, transforming font generation into an autoregressive token prediction task based on Ideographic Description Sequences (IDS), achieving high-quality reproduction of font styles.
🎯 What it does: A non-inverse rendering 3D reconstruction method based on diffusion models, IllumiNeRF, is proposed, which can generate new views that can be rendered under any target lighting from a set of observed images in unknown lighting conditions.
🎯 What it does: An unsupervised image restoration method called aSeqDIP is proposed, which does not require a pre-trained model and utilizes the network structure itself. This method achieves gradual denoising and reconstruction by updating network weights in stages and adaptively updating the input.
🎯 What it does: This study investigates how to transfer knowledge from image understanding (IU) models to image generation (IG) by training an image tokenizer through feature reconstruction, thereby improving generation quality.
🎯 What it does: A unified conditional framework IMAGPose is proposed for pose-guided human image generation, capable of generating multiple target images with different poses at once, and can also generate target images using multi-view source images.
ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images
Timing Yang (Shanghai Qi Zhi Institute), Li Yi (Shanghai AI Lab)
CodeObject DetectionDepth EstimationLarge Language ModelSupervised Fine-TuningContrastive LearningImagePoint Cloud
🎯 What it does: This study proposes the ImOV3D framework, which generates pseudo 3D point clouds and pseudo 3D annotations from 2D images through depth estimation and rendering, and trains an open vocabulary 3D object detection model using pseudo-multimodal representation (image-point cloud), significantly reducing the modality gap between training and testing.
Zhenxiong Tan (National University of Singapore), Xinchao Wang (National University of Singapore)
CodeReinforcement Learning
🎯 What it does: A controllable context version of Procgen, C-Procgen, was constructed, and a fine-grained analysis of the learning process under multi-level training revealed the existence of implicit curriculum learning.
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs
Mustafa Shukor (Sorbonne University), Matthieu Cord (Valeo)
CodeCompressionComputational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelImageVideoMultimodalityAudio
🎯 What it does: This study investigates the internal representations of frozen large language models when receiving multimodal inputs such as images, videos, and audio, revealing different spatial distributions of perception and text tokens within the model, a high overlap in weight activations, and an implicit multimodal alignment phenomenon (IMA) that naturally emerges during training and inference.
Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses
Xiaosen Zheng (Sea AI Lab), Min Lin (Sea AI Lab)
CodeAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes an improved few-shot jailbreak attack method—I-FSJ, which can achieve a high success rate of jailbreak with a very low number of queries (1-8 times) on open-source aligned LLMs with limited context windows (≤8192).
Improved Generation of Adversarial Examples Against Safety-aligned LLMs
Qizhang Li (Harbin Institute of Technology), Hao Chen (UC Davis)
CodeAdversarial AttackLarge Language ModelPrompt EngineeringText
🎯 What it does: Conduct white-box attacks on large language models for safety alignment, proposing an improved gradient-based adversarial example generation method.
🎯 What it does: A soft label distillation method based on category temperature adjustment is proposed to improve the robustness and fairness of deep networks under adversarial attacks.
Improving Deep Learning Optimization through Constrained Parameter Regularization
Jörg K.H. Franke (University of Freiburg), Frank Hutter (University of Freiburg)
CodeClassificationSegmentationOptimizationImageTextBiomedical Data
🎯 What it does: Proposes Constrained Parameter Regularization (CPR), transforming regularization into a constrained optimization problem for each parameter matrix, and achieving adaptive regularization strength through the augmented Lagrangian method.
Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn
Hongyao Tang (Mila - Quebec Artificial Intelligence Institute University of Montreal), Glen Berseth (Mila - Quebec Artificial Intelligence Institute University of Montreal)
CodeReinforcement Learning
🎯 What it does: This study investigates the churn (output drift) of value networks and policy networks in deep reinforcement learning and its chain amplification effect, proposing a pluggable regularization method called CHAIN to mitigate churn and enhance learning performance.
Improving Equivariant Model Training via Constraint Relaxation
Stefanos Pertigkiozoglou (University of Pennsylvania), Kostas Daniilidis (University of Pennsylvania)
CodeOptimizationGraph Neural NetworkPoint Cloud
🎯 What it does: A training framework is proposed that relaxes the equivariance constraints during the training phase and restores equivariance during the inference phase to enhance the optimization performance of equivariant neural networks.
Improving Generalization and Convergence by Enhancing Implicit Regularization
Mingze Wang (Peking University), Lei Wu (Peking University)
CodeOptimizationComputational EfficiencyConvolutional Neural NetworkTransformerLarge Language ModelImageText
🎯 What it does: An Implicit Regularization Enhancement (IRE) framework is proposed, which accelerates gradient updates in flat directions during training, leading to faster convergence to flat minima, thereby enhancing the model's generalization ability and training speed.
CodeFederated LearningSafty and PrivacyImageStochastic Differential Equation
🎯 What it does: A posterior inference framework FedMDMI is proposed in federated learning through model-data mutual information regularization to enhance the model's generalization ability and uncertainty estimation.
🎯 What it does: This paper identifies the representation density problem in non-Gloss sign language translation and proposes a lightweight contrastive learning strategy, SignCL, to reduce feature density and improve translation performance.
🎯 What it does: This paper addresses the hyperparameter optimization of Gaussian processes (GP) for large datasets and proposes three techniques to improve the linear system solver, significantly enhancing computational efficiency.
🎯 What it does: A method is proposed to enhance the model's robustness to various image distortions (noise, lighting, compression, etc.) by multiplying the network weights by random multiplicative noise (DAMP) during training.
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerAuto EncoderText
🎯 What it does: A Gated Sparse Autoencoder (Gated SAE) is proposed for efficiently and sparsely learning interpretable features in language model activations.
🎯 What it does: This paper proposes a new temporal walk matrix projection model TPNet, which combines a time-decayed temporal walk matrix with random feature projection to implicitly maintain node representations for efficient relative encoding.
🎯 What it does: This paper proposes a Deep Fourier Shifting operator based on Fourier cycles to replace traditional convolution units, aiming to enhance the performance of low-level image restoration tasks.
🎯 What it does: This paper proposes Gaussian Neighborhood Minimization Prompt Tuning (GNM-PT), which uses Gaussian neighborhood mean loss in Visual Prompt Tuning (VPT) to smooth the loss landscape and enhance the generalization performance of long-tail visual recognition.
In-Context Symmetries: Self-Supervised Learning through Contextual World Models
Sharut Gupta (Massachusetts Institute of Technology), Stefanie Jegelka (Massachusetts Institute of Technology)
CodeClassificationRetrievalRepresentation LearningTransformerContrastive LearningImageTabularBiomedical Data
🎯 What it does: A self-supervised learning framework called CONTEXTSSL is proposed, which can adaptively achieve equivariant or invariant representations for different transformations based on task requirements through a small amount of contextual memory.
Incentivizing Quality Text Generation via Statistical Contracts
Eden Saig (Technion Israel Institute of Technology), Inbal Talgam-Cohen (Tel Aviv University)
CodeGenerationOptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A 'pay-for-performance' scheme based on contract design is proposed to incentivize large language models (LLMs) to produce high-quality text and avoid moral hazards.
Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques
Manh Cuong Dao (Hanoi University of Science and Technology), Trong Nghia Hoang (Washington State University)
CodeOptimization
🎯 What it does: This paper proposes a model-agnostic regularization method based on model sharpness (IGNITE) to improve the training of surrogate models for offline optimizers, thereby enhancing the performance of offline optimization tasks.
🎯 What it does: A meta-learning framework that integrates test-time optimization into the training process is proposed, utilizing a dual network structure to unify training and testing objectives, thereby improving the accuracy of human mesh reconstruction from a single image.
INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness
Hung Le (Salesforce Research), Silvio Savarese (Salesforce Research)
CodeSafty and PrivacyAI Code AssistantTransformerLarge Language ModelAgentic AIText
🎯 What it does: A framework named INDICT is proposed, which utilizes two internal critics (security critic and practicality critic) for collaborative review and improvement of code generated by large language models;
🎯 What it does: A model named Switching Recurrent Neural Network (SRNN) is proposed and implemented to automatically identify discrete states from neural time series data and reconstruct nonlinear neural dynamics.
🎯 What it does: This paper utilizes representations obtained from regularized temporal contrastive learning and proves that these representations form a Gauss-Markov chain under certain assumptions, allowing for predictions of future states, inferences of intermediate states, and reasoning tasks such as path planning through simple methods like linear interpolation or low-dimensional matrix inversion.
🎯 What it does: A neural SDF learning method that combines data-driven priors and overfitting strategies is proposed, using local noise-noise statistical inference to fine-tune the prior on a single noisy point cloud, thereby recovering high-quality implicit surfaces without the need for signed distance supervision, clean point clouds, or normal information.
Inferring stochastic low-rank recurrent neural networks from neural data
Matthijs Pals (University of Tübingen), Jakob H. Macke (Max Planck Institute for Intelligent Systems)
CodeGenerationExplainability and InterpretabilityRecurrent Neural NetworkTime SeriesSequentialBiomedical DataElectrocardiogramStochastic Differential Equation
🎯 What it does: Using the variational sequential Monte Carlo (SMC) method to fit recurrent neural networks (RNNs) with low-rank structure and stochastic transitions, generating interpretable generative models that can match the variability of neural data.
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
Chaojun Xiao (Tsinghua University), Maosong Sun (Tsinghua University)
CodeTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed a training-free InfLLM method that handles ultra-long sequences through sliding window attention combined with external context memory.
InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling
Yuchun Miao (Wuhan University), Dacheng Tao (Nanyang Technological University)
CodeOptimizationReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: Proposes the InfoRM information bottleneck reward model to address the issue of reward hacking in RLHF, and designs the Cluster Separation Index (CSI) metric for real-time detection and prevention of over-optimization.
Information Re-Organization Improves Reasoning in Large Language Models
Xiaoxia Cheng (Zhejiang University), Weiming Lu (Zhejiang University)
CodeTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Reorganize information from the context, first extract logical relationships, then prune noise, and use the reorganized information for reasoning.
Infusing Self-Consistency into Density Functional Theory Hamiltonian Prediction via Deep Equilibrium Models
Zun Wang (Microsoft Research AI4Science), Bin Shao (Microsoft Research AI4Science)
CodeGraph Neural NetworkGraphPhysics Related
🎯 What it does: In this study, the authors propose a DFT Hamiltonian prediction architecture called DEQHNet based on a deep equilibrium model, which can achieve self-consistent Hamiltonian solving without relying on traditional DFT iterations.
🎯 What it does: This paper proposes an input-state stable Coupled Oscillator Network (CON) for learning and controlling the dynamics of physical systems in a low-dimensional latent space.
Zhengyan Shi (University College London), Aldo Lipani (University College London)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the Instruction Modelling (IM) method, which calculates the loss for both the instruction and response parts during instruction tuning, rather than only for the response part, thereby improving the performance of language models on various NLP tasks and open-ended generation tasks.
CodeObject DetectionSegmentationTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelImageMultimodality
🎯 What it does: A command-guided visual mask (IVM) model is proposed, which significantly enhances multimodal instruction-following performance by automatically generating visual masks to eliminate areas in images that are irrelevant to the instructions.
🎯 What it does: This paper proposes a slice-level active learning framework that combines deep metric learning with Coreset for 3D medical image segmentation.
Egor Gladin (Humboldt University of Berlin), Jia-Jie Zhu (Weierstrass Institute for Applied Analysis and Stochastics)
CodeOptimizationTabular
🎯 What it does: A new geometry of Interactive Force Transmission (IFT) gradient flow is proposed, combining unbalanced optimal transport of Wasserstein and MMD tensors, and a particle optimization algorithm based on JKO splitting is provided, proving global exponential convergence under MMD and KL energy.
InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint
Zhenzhi Wang (Chinese University of Hong Kong), Bo Dai (University of Hong Kong)
CodeGenerationPose EstimationOptimizationLarge Language ModelDiffusion modelVideo
🎯 What it does: This paper studies a zero-shot multi-person interaction generation method called InterControl, based on a single-person motion generation model, which can achieve interactive actions for any number of people by precisely controlling the position of each joint.
CodeRecognitionOptimizationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: Developed InternLM‑XComposer2‑4KHD, a multimodal large model that supports resolutions from 336 pixels to 4K HD (3840×1600) and even higher, achieving significant improvements on 16 benchmarks including OCR and visual reasoning.
Interpret Your Decision: Logical Reasoning Regularization for Generalization in Visual Classification
Zhaorui Tan (Xi'an-Jiaotong Liverpool University), Kaizhu Huang (Duke Kunshan University)
CodeClassificationDomain AdaptationExplainability and InterpretabilityConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a logic-based regularization method called L-Reg to improve the generalization performance of visual classification models in multi-domain generalization, multi-object discovery, and their combined scenarios.
Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents
Quentin Delfosse (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
CodeExplainability and InterpretabilityKnowledge DistillationReinforcement LearningVideo
🎯 What it does: Designed and evaluated an interpretable reinforcement learning agent called Successive Concept Bottleneck Agents (SCoBots), which achieves interpretable decision-making processes in RL tasks through multi-layer concept bottlenecks.
David Debot (KU Leuven), Giuseppe Marra (KU Leuven)
CodeClassificationExplainability and InterpretabilityImage
🎯 What it does: An interpretable and verifiable concept-based model CMR is proposed, which selects rules from a learnable logic rule memory through a neural selection mechanism and performs symbolic evaluation, achieving globally interpretable task predictions.
CodeOptimizationExplainability and InterpretabilityTabularBiomedical Data
🎯 What it does: This paper proposes a sparse interpretable Generalized Additive Model (M-GAM) that directly embeds missing indicators and their interaction terms into the model to handle missing data, rather than the traditional approach of filling in data before prediction.
Interpretable Image Classification with Adaptive Prototype-based Vision Transformers
Chiyu Ma (Dartmouth), Chaofan Chen (Maine)
CodeClassificationExplainability and InterpretabilityTransformerImage
🎯 What it does: This paper proposes ProtoViT, an interpretable image classification method that combines Vision Transformer with deformable prototypes, utilizing case-based reasoning to provide 'looks like...' explanations.
Interpretable Mesomorphic Networks for Tabular Data
Arlind Kadra (University of Freiburg), Josif Grabocka (University of Technology Nuremberg)
CodeClassificationExplainability and InterpretabilityTabularBenchmark
🎯 What it does: A new interpretable deep neural network is proposed - IMN (Interpretable Mesomorphic Neural Networks), which generates linear explanation models for each sample through a hypernetwork, achieving instance-level interpretability while maintaining the accuracy of deep learning.
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A multimodal explanation method based on text concepts is proposed, utilizing the 'information channel' shared by visual and language encoders to explain the zero-shot image classification of the CLIP model; at the same time, shared knowledge between visual and language encoders is defined and quantified, analyzing its impact on model performance.
CodeRetrievalExplainability and InterpretabilityContrastive LearningImageText
🎯 What it does: This paper proposes a training-free, task-agnostic sparse linear concept embedding method called SpLiCE, which converts the high-dimensional dense vectors of CLIP into non-negative sparse linear combinations of interpretable concept vectors.
Interpreting Learned Feedback Patterns in Large Language Models
Luke Marks (Apart Research), Fazl Barez (University of Oxford)
CodeExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: By training a detector to predict the implicit feedback signals learned from RLHF, we measure the consistency of the learning feedback patterns of LLMs with human feedback.
Interpreting the Weight Space of Customized Diffusion Models
Amil Dravid, Kfir Aberman
CodeGenerationData SynthesisExplainability and InterpretabilitySupervised Fine-TuningDiffusion modelImage
🎯 What it does: Proposed and implemented the 'weights2weights (w2w)' space, constructing an interpretable subspace with model weights to achieve sampling, editing, and single-image inversion of customized diffusion models;
IntraMix: Intra-Class Mixup Generation for Accurate Labels and Neighbors
Shenghe Zheng (Harbin Institute of Technology), Xianglong Liu (Harbin Institute of Technology)
CodeGraph Neural NetworkSupervised Fine-TuningGraphPhysics Related
🎯 What it does: A novel data augmentation framework called IntraMix is proposed in graph neural networks to address the two major challenges of label scarcity and incomplete neighbors.
Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting
Bong Gyun Kang (Seoul National University), Sungroh Yoon (Seoul National University)
CodeOptimizationTransformerTime Series
🎯 What it does: Proposes a Spectral Attention mechanism that utilizes exponential moving average and multi-frequency attention in time series forecasting, maintaining temporal correlation between samples and achieving gradient backpropagation across steps;
Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level
Runlin Lei (Renmin University of China), Zhewei Wei (Renmin University of China)
CodeExplainability and InterpretabilityAdversarial AttackGraph Neural NetworkLarge Language ModelTextGraph
🎯 What it does: The study conducts text-level graph injection attacks (GIA) on Text Attribute Graphs (TAG) and proposes and evaluates three attack designs: ITGIA, VTGIA, and WTGIA.
Inverse M-Kernels for Linear Universal Approximators of Non-Negative Functions
Hideaki Kim (NTT Corporation)
CodeOptimizationTime Series
🎯 What it does: The concept of the inverse M-kernel is proposed, and a linear non-negative approximator is constructed in a one-dimensional input space, proving that this approximator can achieve universal approximation of non-negative functions.
Jaewon Chu (Korea University), Hyunwoo J. Kim (Korea University)
CodeOptimizationDrug DiscoveryAuto EncoderTabular
🎯 What it does: This paper proposes a latent Bayesian optimization framework based on an inverse decoder, InvBO, which addresses the alignment issues caused by reconstruction errors in traditional LBO and improves the selection of trust region anchor points.
InversionView: A General-Purpose Method for Reading Information from Neural Activations
Xinting Huang (Saarland University), Michael Hahn (Saarland University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: The InversionView method is proposed, which visualizes and explains the information encoded in neural network activations by recovering inputs from activation vectors through training a conditional decoder.
🎯 What it does: This paper studies a new regeneration attack, which adds random noise to the image embedding space and utilizes generative/denoising models to reconstruct the image, thereby removing invisible pixel-level watermarks while maintaining image quality.
🎯 What it does: This paper proposes a one-time domain adaptation method for super-resolution networks, called IODA, which utilizes only a single unannotated low-resolution image from the target domain.
🎯 What it does: This paper proposes a learning-based interior point method (IPM) called IPM-LSTM by replacing the steps of solving linear equations in traditional interior point methods with a trained long short-term memory network (LSTM) to approximate solutions. It constructs a two-stage framework: first, using IPM-LSTM to obtain high-quality approximate primal-dual solutions, and then using these as a warm-start to initiate a standard IPM solver.
IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering
Ruosen Li (University of Texas at Dallas), Xinya Du (University of Texas at Dallas)
CodeLarge Language ModelText
🎯 What it does: An automatic evaluation framework for human-computer interaction question-answering systems, IQA-EVAL, is proposed, utilizing large language models (LLMs) as evaluation agents (LEA) to simulate human behavior and assess interactions.
🎯 What it does: A multi-task rapid image restoration method IR-CM based on a consistency model is proposed, achieving first-order or low-order sampling inference.
IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons
Dan Shi (Tianjin University), Deyi Xiong (Tianjin University)
CodeGenerationOptimizationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This study investigates the generation issues of LLMs when faced with conflicts between context and pre-trained knowledge, proposing the IRCAN framework to enhance the model's fidelity to new information by identifying and amplifying context-sensitive neurons.
Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
Jiayu Wang (University of Wisconsin Madison), Neel Joshi (Microsoft Research)
CodeTransformerLarge Language ModelVision Language ModelTextMultimodalityBenchmark
🎯 What it does: Four types of multimodal spatial reasoning tasks (Spatial-Map, Maze-Nav, Spatial-Grid, Spatial-Real) were constructed, and a systematic evaluation of the performance of LLMs and VLMs under text, visual, and bimodal inputs was conducted.
Is Score Matching Suitable for Estimating Point Processes?
Haoqun Cao (Renmin University of China), Feng Zhou (Renmin University of China)
CodeScore-based ModelPoint Cloud
🎯 What it does: This paper proposes a Weighted Score Matching (WSM) and Autoregressive Weighted Score Matching (AWSM) method for point process parameter estimation, demonstrating their consistency and convergence.
Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization
Wei Liu (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
CodeExplainability and InterpretabilityTabular
🎯 What it does: This paper proposes a new interpretability criterion - Maximum Residual Difference (MRD), which allows for the efficient extraction of causal explanation subsets even in datasets with spurious correlated features.
Is Your LiDAR Placement Optimized for 3D Scene Understanding?
Ye Li (University of Michigan), Xiaonan Huang (University of Michigan)
CodeAutonomous DrivingOptimizationPoint Cloud
🎯 What it does: This paper proposes a full-cycle multi-radar placement evaluation and optimization framework called Place3D, which combines semantic occupancy grid evaluation metrics, CMA-ES optimization, and large-scale simulation data generation to systematically study multi-radar arrangements in autonomous driving.
🎯 What it does: This study investigates the reasoning of Transformers in iterative tasks through Chain of Thought (CoT), revealing and validating the mechanism of the 'iteration head' and exploring the impact of data preprocessing on model learning.
🎯 What it does: This paper proposes a framework based on a local evolution set process that can localize standard iterative solvers (such as Gauss-Seidel, gradient descent, Chebyshev, Heavy-Ball) to achieve efficient solutions for the approximate personalized PageRank (PPR) problem.
Iteratively Refined Early Interaction Alignment for Subgraph Matching based Graph Retrieval
Ashwin Ramachandran (University of California San Diego), Abir De (Indian Institute of Technology Bombay)
CodeRetrievalGraph Neural NetworkGraph
🎯 What it does: A new early interaction network called IsoNet++ is proposed for graph retrieval based on subgraph isomorphism, aiming to improve retrieval performance through iterative refinement of alignment.
IWBVT: Instance Weighting-based Bias-Variance Trade-off for Crowdsourcing
Wenjun Zhang (China University of Geosciences), Chaoqun Li (China University of Geosciences)
CodeClassificationOptimizationTabular
🎯 What it does: This paper proposes an instance-weighted bias-variance trade-off method (IWBVT) as a post-processing step for existing label fusion and noise correction algorithms, enhancing the predictive quality of models trained on crowdsourced data.
JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models
Kun Zhou (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeData SynthesisComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper trains a lightweight LLM with a scale of 7B, using it to generate approximately 4.6B high-quality math question-answer pairs, which are then used to pre-train the JiuZhang3.0 model, enhancing its mathematical reasoning capabilities.
Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learning
Divyam Madaan (New York University), Kyunghyun Cho (Genentech)
CodeGenerationRepresentation LearningMixture of ExpertsMultimodalityBiomedical DataMagnetic Resonance ImagingElectronic Health Records
🎯 What it does: A framework called I2M2 is proposed, which simultaneously captures cross-modal and single-modal dependencies, addressing the issue of performance fluctuations caused by focusing solely on one type of dependency in multimodal learning.
Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learning
Xinran Li (Hong Kong University of Science and Technology), Jun Zhang (Hong Kong University of Science and Technology)
CodeReinforcement Learning
🎯 What it does: This paper proposes a learnable mask-based adaptive partial parameter sharing mechanism (Kaleidoscope) for multi-agent reinforcement learning, which achieves diversity in agent policies while maintaining sample efficiency.
Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting
Fangcheng Liu (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: A self-inference decoding framework named Kangaroo is proposed, which accelerates the inference of large language models through a dual early-exit strategy without the need for additional training of an independent draft model.
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper discusses how to maintain safety when fine-tuning aligned large language models by changing prompt templates, proposing and validating the 'Pure Tuning, Safe Testing (PTST)' strategy.
🎯 What it does: Using kernel principal component analysis (KPCA) to analyze the reconstruction error of features obtained from DNN training, two task-specific kernels (cosine kernel and cosine-Gaussian kernel) are constructed, and large-scale efficient out-of-distribution (OoD) detection is achieved through explicit feature mapping.
Wenjun Zhang (China University of Geosciences), Chaoqun Li (China University of Geosciences)
CodeClassificationTabular
🎯 What it does: This paper proposes a K-free nearest neighbor label integration algorithm (KFNN) to automatically determine the number of neighbors for each sample in crowdsourced data and to infer true labels by integrating attribute and noise label information.
KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge
Pengcheng Jiang (University of Illinois at Urbana-Champaign), Jiawei Han (University of Illinois at Urbana-Champaign)
CodeGraph Neural NetworkLarge Language ModelSupervised Fine-TuningGraph
🎯 What it does: Proposes the KG-FIT framework, which combines the open-world knowledge of LLM with KG embeddings, enhancing graph embeddings through hierarchical clustering and text embedding fine-tuning.
Yunzhi Yao (Zhejiang University), Huajun Chen (Zhejiang University)
CodeTransformerLarge Language ModelText
🎯 What it does: This study discovers and analyzes 'knowledge circuits' in the computation graph of Transformers using a circuit theory-based approach, revealing how the model stores and expresses knowledge such as facts and biases internally.
🎯 What it does: This paper proposes a method called aTLAS (Adaptive Task Vector Learning through Non-Proportional Scaling) to efficiently combine knowledge across different tasks, enabling transfer learning and low-data learning.
Knowledge Graph Completion by Intermediate Variables Regularization
Changyi Xiao (Fudan University), Yixin Cao (Fudan University)
CodeKnowledge DistillationGraph
🎯 What it does: A general form of the TDB model is proposed, and based on this, Intermediate Variable Regularization (IVR) is introduced to alleviate the overfitting problem in knowledge graph completion.
🎯 What it does: A lightweight text-to-image model KOALA is constructed, compressing the U-Net of SDXL and achieving efficient generation through knowledge distillation.
Kraken: Inherently Parallel Transformers For Efficient Multi-Device Inference
Rohan Baskar Prabhakar (Princeton University), David Wentzlaff (Princeton University)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: A Transformer variant named Kraken is designed and implemented, which reduces communication bottlenecks in multi-device inference by introducing fixed parallelism at each layer and performing AllReduce only once at the end of the layer.
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Coleman Richard Charles Hooper (University of California), Amir Gholami (University of California)
CodeRetrievalCompressionTransformerLarge Language ModelText
🎯 What it does: The KVQuant method is proposed, which performs low-precision (≤3bit) quantization on the KV cache in LLM inference to support million-level context length inference.
🎯 What it does: A lightweight test-time adaptation method (L-TTA) is proposed, which achieves rapid adaptation to the target domain by only reconstructing the stem layer of the model.
Botos Csaba (University of Oxford), Adel Bibi (University of Oxford)
CodeContrastive LearningTime SeriesSequential
🎯 What it does: The paper addresses the issue of label delay in online continual learning and presents a new experimental framework and evaluation metrics.
🎯 What it does: A theoretical framework based on Relative Signal Strength (RSS) is proposed to analyze the limits under instance-dependent label noise, and the near-optimality of Noise Ignoring Empirical Risk Minimization (NI-ERM) is validated in both theory and practice.
LACIE: Listener-Aware Finetuning for Calibration in Large Language Models
Elias Stengel-Eskin (University of North Carolina at Chapel Hill), Mohit Bansal (University of North Carolina at Chapel Hill)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes and implements the Listener-Aware Calibration for Implicit and Explicit confidence (LACIE) method, which fine-tunes large language models through a multi-agent speaker-listener game and preference optimization, enabling them to express confidence more accurately when answering questions.