Conference on Neural Information Processing Systems · 1874 papers
Towards Understanding Evolving Patterns in Sequential Data
QIUHAO Zeng, Boyu Wang (Western University)
CodeRecurrent Neural NetworkTransformerAuto EncoderVideoTime SeriesSequentialFinance Related
🎯 What it does: A metric based on mutual information (EVORATE) is proposed to quantitatively assess evolutionary patterns in sequence data, and it is extended to scenarios without correspondence (EVORATE W), thereby helping to determine the suitability of using sequence models, estimate the order of sequences, and perform feature selection.
🎯 What it does: A unified multimodal editing framework called UniKE is proposed, which balances internal knowledge editing and external knowledge retrieval.
Towards Unsupervised Model Selection for Domain Adaptive Object Detection
Hengfu Yu (University of Electronic Science and Technology of China), Lixin Duan (University of Electronic Science and Technology of China)
CodeObject DetectionDomain AdaptationImage
🎯 What it does: An unsupervised model selection method called Detection Adaptation Score (DAS) is proposed, which can select nearly optimal checkpoints for domain adaptive object detection (DAOD) models without target domain labels.
Zhanhao Hu (University of California), David Wagner (University of California)
CodeClassificationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A toxicity detection method utilizing the output of large language models (the logits of the first token) for querying without additional models is proposed—MULI;
🎯 What it does: Utilize instance motion estimation to crop the search space for video point tracking, and combine point tracking with instance segmentation to improve tracking accuracy and achieve zero-shot video instance segmentation.
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning
Yeongbin Seo (Yonsei University), Jinyoung Yeo (Yonsei University)
CodeMeta LearningTransformerLarge Language ModelTextTime SeriesBenchmark
🎯 What it does: This paper proposes the Train-Attention-Augmented Language Model (TAALM), a continual knowledge learning method that predicts token importance through meta-learning and dynamically weights it during training, and establishes a new benchmark called LAMA-CKL.
🎯 What it does: This paper proposes the VISPA algorithm, which combines Gaussian variational inference with low-rank semidefinite programming to address the training problem of binary neural networks (BNN).
CodeProtein Structure PredictionTransformerLarge Language ModelSupervised Fine-TuningBiomedical Data
🎯 What it does: This study investigates how to optimally train protein language models under a fixed computational budget, proposing scaling laws for CLM and MLM, and providing an optimal allocation scheme for model and data size.
Chao Chen (Harbin Institute of Technology), Sihong Xie
CodeExplainability and InterpretabilityAdversarial AttackGraph Neural NetworkMultimodalityTabular
🎯 What it does: A new metric called 'explanation ranking thickness' is proposed to measure the robustness of top-k important features in model explanations (especially gradient-based explanations) against perturbations, and based on this metric, the R2ET training method is designed.
🎯 What it does: A trajectory planning method based on diffusion models (Trajectory Diffusion) is proposed, which generates future trajectory sequences through semantic maps and target information in the ObjectGoal navigation task, guiding the agent to efficiently reach the target.
Trajectory Flow Matching with Applications to Clinical Time Series Modelling
Xi Zhang (McGill University), Alexander Tong (Mila - Quebec AI Institute)
CodeFlow-based ModelTime SeriesBiomedical DataElectronic Health RecordsStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A simulation-agnostic training framework called Trajectory Flow Matching (TFM) is proposed for efficiently learning Neural Stochastic Differential Equations (Neural SDE) and modeling clinical time series.
TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration
Yiwei Guo (Shenzhen Institutes of Advanced Technology), Yali Wang (Shenzhen Institutes of Advanced Technology)
CodeClassificationRecognitionKnowledge DistillationTransformerMixture of ExpertsVision Language ModelImageMultimodality
🎯 What it does: The TransAgent framework is proposed, which significantly enhances generalization ability in low-sample scenarios by transferring knowledge from multimodal expert models to visual-language foundation models like CLIP through the collaboration of multi-source heterogeneous agents.
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This study investigates the transcoder as a sparse approximation of the MLP sublayer in Transformers for fine-grained circuit analysis, validating its interpretability and accuracy on GPT-2 and small Pythia models.
Transfer Q-star : Principled Decoding for LLM Alignment
Souradip Chakraborty (University of Maryland), Furong Huang (University of Maryland)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes Transfer Q*, which utilizes existing aligned baseline models to directly or indirectly transfer the estimation of the optimal Q*, thereby achieving safe alignment of LLMs during the inference phase.
🎯 What it does: Derived the upper bound theory of adversarial sample transferability and proposed an optimizable attack method TPA based on this theory.
🎯 What it does: The research utilizes the publicly available Segment Anything Model (SAM) as a surrogate and proposes a transferable adversarial attack method called UMI-GRAT, which can effectively attack SAM and its fine-tuned downstream models without accessing downstream task data and models.
Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
Deqing Fu (University of Southern California), Vatsal Sharan (University of Southern California)
CodeOptimizationTransformerTabular
🎯 What it does: The study investigates how Transformer achieves linear regression in context learning without parameter updates, finding that its internal implementation is similar to second-order optimization methods.
Transformers on Markov data: Constant depth suffices
Nived Rajaraman (University of California Berkeley), Michael Gastpar (École Polytechnique Fédérale de Lausanne)
CodeTransformerSequential
🎯 What it does: This paper studies the ability of Transformers to learn conditional k-gram models from sequences sampled from k-th order Markov processes through experimental and theoretical analysis, and proves that a constant-layer Transformer can represent this model.
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner
Hanwen Zhong (Beihang University), Yunhong Wang (Beihang University)
CodeClassificationSegmentationOptimizationTransformerMixture of ExpertsImage
🎯 What it does: This paper proposes an efficient multi-task learning framework called EMTAL, which decomposes the pre-trained Vision Transformer into low-rank Mixture-of-Experts and fine-tunes it using LoRA. It then achieves asynchronous task optimization through a Quality Retention (QR) mechanism, and finally employs routing decay for parameter reparameterization, resulting in a unified model with no additional inference cost.
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
Chenyang Le (Shanghai Jiao Tong University), Michael Zeng (Microsoft)
CodeKnowledge DistillationTransformerAudio
🎯 What it does: Develop an end-to-end speech-to-speech translation system, TransVIP, which can directly translate source language speech into target language speech while maintaining the speaker's voice characteristics and timing (speech rate, pauses), suitable for scenarios such as video dubbing.
🎯 What it does: An optimal randomized Clopper-Pearson confidence interval and an adaptive sampling method based on confidence sequences are proposed for statistical estimation in randomized smoothing, significantly reducing the required number of forward propagations.
🎯 What it does: Proposes TreeVI (Tree-structured Variational Inference) and MTreeVI (Tree Mixture Model) to approximate the posterior distribution with instance-level correlations, and implements a parallelizable matrix form reparameterization.
🎯 What it does: Proposes the TSDS framework, which drives the distribution alignment and diversity balance of large-scale candidate data using representative examples, ultimately selecting high-quality training samples for task-specific fine-tuning.
TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
Benjamin Feuer (New York University), Colin White (Abacus.AI)
CodeClassificationOptimizationExplainability and InterpretabilityTransformerPrompt EngineeringTabularBenchmark
🎯 What it does: This paper proposes TuneTables, a context optimization method for soft prompt tuning on the pre-trained TabPFN, aimed at compressing the context of large-scale datasets, enabling the expansion of features, samples, and the number of classes, while supporting multi-objective optimization and interpretability analysis.
TurboHopp: Accelerated Molecule Scaffold Hopping with Consistency Models
Kiwoong Yoo (AIGEN Sciences), Jaewoo Kang (Korea University)
CodeOptimizationDrug DiscoveryReinforcement LearningDiffusion modelBiomedical Data
🎯 What it does: This paper presents TurboHopp, a 3D Scaffold Hopping model based on a consistency model, designed for the rapid generation of active compound scaffolds at protein binding sites.
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
Zhenyi Lu (Huazhong University of Science and Technology), Yu Cheng (Chinese University of Hong Kong)
CodeGenerationOptimizationTransformerMixture of ExpertsText
🎯 What it does: Proposes the Twin-Merging method, which separates shared and exclusive knowledge and dynamically merges them based on input during inference, significantly improving performance in multi-task model merging.
Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning
Shuguang Yu (Shanghai University of Finance and Economics), Chengchun Shi (London School of Economics and Political Science)
CodeReinforcement LearningBiomedical DataElectronic Health Records
🎯 What it does: Proposes a two-way debiasing estimation method that utilizes the two-way unobserved confounding assumption for offline policy evaluation (OPE) debiasing.
🎯 What it does: This paper proposes Typicalness-Aware Learning (TAL), which alleviates the overconfidence problem of deep networks and enhances failure detection performance by dynamically adjusting the logit magnitude.
🎯 What it does: A diffusion Transformer based on U-shaped structure (U-DiT) is proposed, which reduces computational cost and improves generation quality by downsampling visual tokens in self-attention.
🎯 What it does: A unified neural divide-and-conquer framework (UDC) is proposed, capable of solving large-scale combinatorial optimization problems without relying on problem-specific heuristics.
🎯 What it does: A general image embedding learning framework UDON based on multi-teacher online distillation is proposed, which jointly trains domain-specific teachers and a unified student using a shared backbone network, and dynamically samples to accelerate the convergence of hard-to-learn domains.
🎯 What it does: Proposes the Upsampling Diffusion Probabilistic Model (UDPM), which incorporates downsampling and upsampling in the diffusion process, allowing for high-quality image generation in just 3 steps.
Ultrafast classical phylogenetic method beats large protein language models on variant effect prediction
Sebastian Prillo (University of California), Yun S. Song (University of California)
CodeComputational EfficiencyProtein Structure PredictionBiomedical Data
🎯 What it does: Developed a near-linear time method called FastCherries to estimate 'cherry' pairs and branch lengths in MSA, combined with CherryML to directly estimate the LG model and a more granular SiteRM site rate matrix on MSA, avoiding the computational bottleneck of full tree reconstruction.
UMB: Understanding Model Behavior for Open-World Object Detection
Xing Xi (South China University of Technology), Ronghua Luo (South China University of Technology)
CodeObject DetectionTransformerLarge Language ModelVision Language ModelImage
🎯 What it does: A framework named UMB is proposed to understand the model's prediction behavior for unknown categories in the open-world object detection (OWOD) task and to generate textual attribute descriptions for unlabeled objects.
UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models
Jiachen Liang (Institute of Computing Technology, Chinese Academy of Sciences), Xilin Chen (Institute of Computing Technology, Chinese Academy of Sciences)
CodeDomain AdaptationTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: This paper proposes an unsupervised, multi-domain feature calibration method called UMFC, aimed at enhancing the generalization ability of CLIP on multi-domain unlabeled data.
Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in LLMs
Zhiyuan Hu (National University of Singapore), Bryan Hooi (National University of Singapore)
CodeTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: An Uncertainty of Thoughts (UoT) algorithm is designed to enable large language models to actively acquire information by perceiving their own uncertainty and asking questions, thereby improving the completion rate of information retrieval and decision-making tasks.
Uncertainty-aware Fine-tuning of Segmentation Foundation Models
Kangning Liu (New York University), Carlos Fernandez-Granda (New York University)
CodeSegmentationSupervised Fine-TuningImage
🎯 What it does: A fine-tuning framework called SUM is proposed based on uncertainty perception to enhance the segmentation quality of SAM on complex structured images while maintaining its generality.
Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast
Chufan Shi (Tsinghua University), Yu Meng (University of Virginia)
CodeOptimizationTransformerMixture of ExpertsContrastive LearningText
🎯 What it does: This study investigates the potential contributions of inactive experts in the Mixture-of-Experts (MoE) model and proposes a self-contrastive decoding method (SCMoE) that enhances the prediction of the next token by leveraging contrasts from different routing strategies.
🎯 What it does: A recurrent neural network model named ORGaNICs is proposed and analyzed, which achieves neural dynamical stability through Divisive Normalization and can learn in continuous time.
Uncovering Safety Risks of Large Language Models through Concept Activation Vector
Zhihao Xu (Renmin University of China), Xiting Wang (Renmin University of China)
CodeSafty and PrivacyAdversarial AttackLarge Language ModelPrompt EngineeringText
🎯 What it does: This study investigates the security risks of large language models and proposes the Security Concept Activation Vector (SCAV) framework, which enables embedding-level and prompt-level attacks by interpreting the model's security mechanisms.
🎯 What it does: This paper studies the redundancy of graph self-supervised learning models and proposes a fine-tuning framework called SLIDE, which significantly reduces the number of adjustable parameters while maintaining or improving downstream node classification performance.
🎯 What it does: This paper theoretically demonstrates the advantages of Adversarial Collaborative Filtering (ACF) in enhancing the performance and robustness of recommendation systems, and further proposes a user embedding scale adaptive adversarial perturbation magnitude allocation method called PamaCF.
🎯 What it does: This paper studies the training of an unsupervised loss-guided diffusion guidance method and conducts an in-depth analysis of its mechanisms and limitations at both theoretical and experimental levels.
Understanding Hallucinations in Diffusion Models through Mode Interpolation
Sumukh K Aithal (Carnegie Mellon University), J Zico Kolter
CodeGenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper studies the phenomenon of hallucination in diffusion models, particularly the failure modes of mode interpolation, finding that diffusion models perform smooth interpolation between adjacent data modes in the training set, thereby generating samples that are completely outside the support of the original training distribution.
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective
Akiyoshi Tomihari (University of Tokyo), Issei Sato (University of Tokyo)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: This paper analyzes the training dynamics of two-stage fine-tuning (first linear probing LP then fine-tuning FT) based on the Neural Tangent Kernel (NTK) theory, revealing the impact of the linear head norm on feature stability, and addresses the calibration issue caused by large norms through temperature scaling. It also extends the NTK analysis to the Low-Rank Adaptation (LoRA) method, verifying its similarity to standard fine-tuning.
Understanding Multi-Granularity for Open-Vocabulary Part Segmentation
Jiho Choi (KAIST), Hyunjung Shim (KAIST)
CodeObject DetectionSegmentationTransformerVision Language ModelImage
🎯 What it does: A fine-grained part segmentation framework called PartCLIPSeg is proposed for open vocabulary, achieving accurate segmentation of unknown category parts in multi-granularity scenarios.
Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data
Alexander Havrilla, Wenjing Liao (Georgia Institute of Technology)
CodeTransformerLarge Language ModelText
🎯 What it does: This paper presents the statistical estimation and approximation theory of Transformers on low-dimensional manifold data, and based on this, provides theoretical predictions and empirical validation of the scaling laws for LLM models and data.
Understanding the Differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks
Jerome Sieber (ETH Zurich), Antonio Orvieto (ELLIS Institute)
CodeRecurrent Neural NetworkText
🎯 What it does: Proposes a Dynamical Systems Framework (DSF) that unifies attention, State Space Models (SSM), and RNN into a linear recursive form, and conducts theoretical and experimental comparisons based on this framework;
🎯 What it does: This study investigates the equivariance methods in self-supervised learning and reveals their mechanism for improving downstream task performance from an information-theoretic perspective.
🎯 What it does: This study investigates and quantifies the transferability of pre-trained models across different downstream tasks, proposing a 'task-relatedness' metric and demonstrating it as an upper bound for transferability.
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper evaluates the model's utilization of contextual information through N-gram statistical rule approximation for the next word prediction of Transformer LLM.
Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE
Xun Zhu (Tsinghua University), Ji Wu (Tsinghua University)
CodeClassificationRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelMultimodalityBiomedical Data
🎯 What it does: A unified medical multimodal large language model, Uni-Med, has been constructed, supporting six tasks: question answering, visual question answering, report generation, understanding/generating location expressions, and image classification.
UniAudio 1.5: Large Language Model-Driven Audio Codec is A Few-Shot Audio Task Learner
Dongchao Yang (Chinese University of Hong Kong), Helen M. Meng
CodeClassificationCompressionTransformerLarge Language ModelGenerative Adversarial NetworkMultimodalityAudio
🎯 What it does: This paper proposes an audio codec (LLM-Codec) that compresses audio into the vocabulary space of LLMs and combines it with a frozen LLM to achieve cross-modal few-shot audio task learning.
UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation
Hanzhang Zhou (Nanyang Technological University), Kezhi Mao (Nanyang Technological University)
CodeClassificationGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: By providing a mechanistic explanation of the FFN vectors and attention heads within large language models, this paper identifies and masks these internal components to eliminate model bias during the inference phase, thereby improving the robustness of in-context learning (ICL).
🎯 What it does: This paper proposes a unified cross-domain 3D semantic segmentation framework called UniDSeg, which utilizes the prior knowledge of visual foundation models (VFM) to enhance the model's adaptability and generalization ability across different domains.
Unified Lexical Representation for Interpretable Visual-Language Alignment
Yifan Li (Fudan University), Tong He (Amazon Web Services)
CodeRetrievalExplainability and InterpretabilityTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: The LexVLA framework is proposed, unifying vision and language into a sparse vocabulary representation, utilizing DINOv2 and Llama2 to achieve interpretable cross-modal alignment.
🎯 What it does: The Latent Graph Diffusion (LGD) framework is proposed, unifying the generation, regression, and classification tasks of graph data into a generative task; by training a diffusion model in the latent space, it achieves the simultaneous generation of nodes, edges, and graph-level features in one go; and formalizes regression/classification tasks as conditional generation problems, providing theoretical guarantees.
Unifying Homophily and Heterophily for Spectral Graph Neural Networks via Triple Filter Ensembles
Rui Duan (Guangzhou University), Haoran Yang (Tongji University)
CodeClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a spectral graph convolutional neural network named TFE-GNN, which adaptively extracts homophily and heterophily information from graphs through Triple Filter Ensemble (TFE) and utilizes initial features for node classification.
UniMTS: Unified Pre-training for Motion Time Series
Xiyuan Zhang (University of California San Diego), Jingbo Shang (University of California San Diego)
CodeClassificationPose EstimationRepresentation LearningGraph Neural NetworkLarge Language ModelContrastive LearningTextTime Series
🎯 What it does: A UniMTS model that can be uniformly pre-trained was constructed by comparing the full-joint motion sequences generated by physical simulation with text descriptions through contrastive learning.
UNIT: Unifying Image and Text Recognition in One Vision Encoder
Yi Zhu (Huawei Noah's Ark Lab), Hang Xu (Huawei Noah's Ark Lab)
CodeRecognitionTransformerVision Language ModelImageText
🎯 What it does: The UNIT framework is proposed, using a single Vision Transformer encoder to simultaneously perform image recognition and text recognition.
CodeClassificationAnomaly DetectionTransformerPrompt EngineeringTime SeriesBiomedical DataFinance Related
🎯 What it does: A unified multi-task time series model called UNITS is proposed, capable of simultaneously handling various tasks such as forecasting, classification, anomaly detection, and imputation.
Universal Exact Compression of Differentially Private Mechanisms
Yanxiao Liu (Chinese University of Hong Kong), Cheuk Ting Li (Chinese University of Hong Kong)
CodeCompressionSafty and PrivacyTabular
🎯 What it does: A new scheme for compressing different privacy mechanisms is proposed - Poisson Private Representation (PPR), which can accurately simulate any local or global differential privacy mechanism and compress its communication volume.
Universal In-Context Approximation By Prompting Fully Recurrent Models
Aleksandar Petrov (University of Oxford), Adel Bibi (University of Oxford)
CodeRecurrent Neural NetworkPrompt Engineering
🎯 What it does: This study constructs a programming language called LSRL and uses it to prove that various recursive neural networks (RNN, LSTM, GRU, linear RNN, and their gated variants) can approximate any function through in-context prompting, demonstrating that they are universal in-context approximators.
🎯 What it does: A universal neural functional body (UNF) is proposed that can automatically construct permutation equivariant networks in arbitrary weight spaces, and it is used for tasks such as predicting model generalization and training learning optimizers.
Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction
Anton Rodomanov (CISPA Helmholtz Center for Information Security), Sebastian U Stich
CodeOptimization
🎯 What it does: A unified AdaGrad step size adaptive gradient method (basic version UniSgd and accelerated version UniFastSgd) is proposed, suitable for convex composite optimization problems where the main function is approximately smooth and can only be accessed through (possibly biased) stochastic gradient oracles. Convergence analysis is provided under various scenarios (uniform variance, implicit variance reduction, SVRG variance reduction).
🎯 What it does: A 'non-learnable' framework for 3D point clouds is proposed, utilizing category-adaptive multi-transformations (rotation, scaling, distortion, shearing) to generate data that cannot be learned by unauthorized models, and a scheme is provided for recovery of learning through inverse transformations.
CodeGenerationTransformerLarge Language ModelContrastive LearningTextMultimodality
🎯 What it does: This paper proposes a method for generating referential expressions based on a multimodal large language model, utilizing intermediate layer region information for decoding, and filtering high-quality descriptions through cyclic consistency discrimination to reduce object hallucination.
🎯 What it does: A framework for solving inverse problems based on diffusion models is proposed—ProjDiff, which constructs auxiliary variables using the diffusion process to transform noisy observations into equivalent noise samples, and solves a two-variable constrained optimization problem through projected gradient descent.
🎯 What it does: A framework called DiffewS is proposed, which utilizes the Stable Diffusion model (Stable Diffusion 2.1) for few-shot semantic segmentation, generating target segmentation masks directly in the latent space.
Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought
Qiguang Chen (Harbin Institute of Technology), Wanxiang Che (Harbin Institute of Technology)
CodeOptimizationTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Proposed and validated the 'Reasoning Boundary Framework (RBF)' to quantitatively assess and optimize the chain of thought (CoT) capabilities of large language models.
Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback
Hamish Ivison (University of Washington), Hannaneh Hajishirzi (University of Washington)
CodeRecommendation SystemReinforcement Learning from Human FeedbackTransformerReinforcement LearningText
🎯 What it does: This paper explores and compares four core elements of preference-based learning: preference data, learning algorithms, reward models, and strategy training prompts. It also proposes best practices for using PPO with large-scale reward models under the condition of synthesizing high-quality data.
UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation
Ye Sun (Fudan University), Yu-Gang Jiang (Fudan University)
CodeSegmentationOptimizationImage
🎯 What it does: A framework for generating unlearned samples (UnSeg) aimed at image segmentation tasks is proposed, utilizing the pre-trained Segment Anything Model to learn a general noise generator through a dual-layer optimal optimization, allowing any image with a given mask to generate unlearnable noise in a single forward pass, thereby rendering the model ineffective during training.
Unsupervised Discovery of Formulas for Mathematical Constants
Michael Shalyt (Technion Israel Institute of Technology), Ido Kaminer (Technion Israel Institute of Technology)
Code
🎯 What it does: This paper proposes an unsupervised clustering method based on the dynamical characteristics of PCF (Polynomial Continued Fractions) and uses this method to automatically discover and verify hundreds of thousands of formulas, resulting in the introduction of hundreds of new mathematical constant formulas.
🎯 What it does: An unsupervised multimodal image planar transformation estimation framework called AltO is proposed, which utilizes alternating optimization to handle geometric and modal discrepancies separately.
🎯 What it does: Proposes the MADM method, extending unsupervised domain adaptation to multi-modal semantic segmentation; utilizes a pre-trained text-image diffusion model for cross-modal feature extraction and pseudo-label generation;
🎯 What it does: Using untrained neural networks (UNN) and deep image priors, a theoretical analysis framework is proposed, and the SCI-BDVP (Bagged-DVP) algorithm is designed to achieve high-quality video/spectral reconstruction from single-frame compressed imaging.
🎯 What it does: For backdoor defense in the post-training phase, a two-stage method is proposed: first, the neuron weight changes (NWC) are obtained through 'clean unlearning' of the backdoored model, and these changes are used to zero-reset the high-weight-changing sub-weights to eliminate the backdoor; subsequently, an activation-aware fine-tuning (with gradient norm constraints) is performed on the reset model to restore clean accuracy.
Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
Haoang Chi (National University of Defense Technology), Bo Han (Hong Kong Baptist University)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper explores the causal reasoning ability of large language models, demonstrating that they only possess shallow (level-1) causal reasoning based on learned knowledge, rather than deep (level-2) true causal reasoning.
Unveiling LoRA Intrinsic Ranks via Salience Analysis
Wenjun Ke (Southeast University), Yining Li (Southeast University)
CodeSupervised Fine-TuningTextTime Series
🎯 What it does: An adaptive method for dynamically allocating the internal rank of LoRA through significance analysis of intra-time series rank is proposed, called SalientLoRA.
🎯 What it does: This paper derives the self-attention mechanism of the Transformer from the perspective of Kernel Principal Component Analysis (Kernel PCA) and proposes a robust self-attention mechanism called RPC-Attention, which further enhances the model's robustness against interference and attacks.
🎯 What it does: Dynamic analysis of the weight norm variance across different channels in the same layer reveals two distinct variance evolution patterns (IS and DS) for wide and narrow layers during training. A width adaptation strategy is designed based on this pattern, adjusting the width layer by layer in CNN architectures such as VGG and ResNet, which reduces parameters while enhancing performance.
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Proposes the ConBench benchmark to evaluate the consistency of large visual language models (LVLM) in different solution spaces, and systematically analyzes their performance through experiments.
🎯 What it does: A jump connection structure uC based on 2D U-Net is proposed for 3D medical image segmentation, and uC 3DU-Net is constructed accordingly.
UPS: Unified Projection Sharing for Lightweight Single-Image Super-resolution and Beyond
Kun Zhou (Chinese University of Hong Kong Shenzhen), Jiangbo Lu (SmartMore Corporation)
CodeRestorationSuper ResolutionTransformerImage
🎯 What it does: A Unified Projection Sharing (UPS) algorithm is proposed for lightweight single-image super-resolution and other image restoration tasks.
UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction
Yansong Ning (Hong Kong University of Science and Technology), Hao Liu (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextGraphChain-of-Thought
🎯 What it does: This study proposes the UrbanKGent framework, which utilizes large language models (LLMs) along with a custom instruction set, tool calls, and iterative trajectory optimization to achieve automated and low-cost construction of urban knowledge graphs (UrbanKG).
Sophie Greenwood (Cornell Tech), Nikhil Garg (Cornell Tech)
CodeRecommendation SystemOptimizationText
🎯 What it does: This paper studies the trade-off between user fairness and item fairness in recommendation systems, proposing the concept of 'fair price'. It analyzes the loss incurred when item fairness constraints are added while maximizing user fairness, and further explores how fairness constraints can amplify estimation costs in the case of preference misestimation. The authors provide two key phenomena through theoretical derivation and construct an empirical system on the arXiv paper recommendation task to validate the theoretical conclusions.
🎯 What it does: This paper proposes the use of a time-aware graph neural network (DBGNN) to predict the temporal betweenness and closeness of nodes in temporal graphs.
🎯 What it does: This paper proposes a unified generative temporal model: first, temporal signals are mapped to two-dimensional images through reversible transformations (delay embedding or short-time Fourier transform), then a pre-existing visual diffusion model (EDM) is used to generate images, and finally, the inverse transformation is applied to recover the temporal signals, thus achieving generation, interpolation, and extrapolation of short, long, and ultra-long sequences.
Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack
Tiansheng Huang (Georgia Institute of Technology), Ling Liu (Georgia Institute of Technology)
CodeAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes an alignment defense method called Vaccine against LLM fine-tuning-as-a-service attacks, addressing the security failure issue of alignment models caused by a small amount of malicious data during user fine-tuning.
🎯 What it does: A 3D Gaussian Spray (3DGS) uncertainty estimation method based on variational multi-scale representation is proposed, which can provide predictive confidence during rendering and automatically eliminate noise Gaussians.
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
Yang Li (Georgia State University), Shihao Ji (University of Connecticut)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes VB-LoRA, an extremely parameter-efficient fine-tuning method implemented through a vector pool, balancing storage and performance.
🎯 What it does: In the context of class-incremental continual learning, the VQ-Prompt method is proposed, which achieves the discretization and selection of task knowledge through a discrete prompt pool on a fixed pre-trained ViT model.
🎯 What it does: We propose VeLoRA, which utilizes fixed one-dimensional subspace for grouping projection compression of forward activations and roughly reconstructs them during backpropagation, significantly reducing the storage requirements for intermediate activations during large model training.