NeurIPS 2024 Papers — Page 14
Conference on Neural Information Processing Systems · 4035 papers
Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints
Zhenwei Lin (Shanghai University of Finance and Economics), Qi Deng (Shanghai Jiao Tong University)
OptimizationTabular
🎯 What it does: An accelerated primal-dual first-order algorithm is proposed to improve convergence speed using strong convexity constraints, addressing optimization problems with convex objectives constrained by strongly convex functions.
Faster Algorithms for User-Level Private Stochastic Convex Optimization
Andrew Lowy (University of Wisconsin-Madison), Hilal Asi (Apple)
OptimizationFederated LearningSafty and Privacy
🎯 What it does: A new user-level differential privacy stochastic convex optimization algorithm is proposed, significantly reducing the gradient computation while maintaining optimal risk in large-scale settings.
Faster Differentially Private Top-$k$ Selection: A Joint Exponential Mechanism with Pruning
Hao WU (University of Waterloo), Hanwen Zhang (University of Copenhagen)
OptimizationSafty and PrivacyComputational EfficiencyTabular
🎯 What it does: This study investigates the Top-k selection problem under differential privacy and proposes an improved algorithm aimed at identifying the k highest-scoring items from d items.
Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference
Senmao Li (Nankai University), jian Yang
GenerationComputational EfficiencyDiffusion modelImageVideo
🎯 What it does: This paper analyzes that the feature changes of the UNet encoder are minimal, and proposes to reuse encoder features at several time steps and decode in parallel, thereby accelerating the inference of diffusion models;
Faster Local Solvers for Graph Diffusion Equations
Jiahe Bai (Fudan University), Yanghua Xiao (Fudan University)
OptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: A general framework based on local diffusion processes is proposed, which can transform graph diffusion equations (such as Personalized PageRank, Katz centrality, Heat kernel) into local iterative solvers, significantly reducing computational costs.
Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level
Ali Hassani (Georgia Tech), Humphrey Shi (Georgia Tech)
OptimizationComputational EfficiencyReinforcement LearningBenchmark
🎯 What it does: Two types of efficient CUDA kernels are proposed for Neighborhood Attention and multi-dimensional sliding window attention: the GEMM NA kernel based on batch GEMM and the Fused NA kernel, which are integrated into the NATTEN library.
Faster Repeated Evasion Attacks in Tree Ensembles
Lorenzo Cascioli (KU Leuven), Jesse Davis (KU Leuven)
Computational EfficiencyAdversarial AttackTabular
🎯 What it does: This paper proposes an accelerated adversarial sample generation method based on the most frequently disturbed feature subset in tree models, first identifying this subset and then pruning the tree or using mixed search to reduce the search space.
FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification
Jingfeng Yao (Huazhong University of Science and Technology), Xinggang Wang (Huazhong University of Science and Technology)
GenerationComputational EfficiencyTransformerDiffusion modelImage
🎯 What it does: This paper proposes FasterDiT, which improves the training speed of the Diffusion Transformer without architectural changes.
FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model
Xiaobao Wu (Nanyang Technological University), Anh Tuan Luu (Nanyang Technological University)
TransformerText
🎯 What it does: A topic model named FASTopic is proposed, utilizing Dual Semantic-relation Reconstruction (DSR) and Embedding Transport Plan (ETP) to achieve efficient, stable, and transferable topic modeling.
FastSurvival: Hidden Computational Blessings in Training Cox Proportional Hazards Models
Jiachang Liu (Cornell University), Cynthia Rudin (Duke University)
OptimizationComputational EfficiencyTabular
🎯 What it does: Developed a method for optimizing quadratic/cubic surrogate functions with hidden structures using the Cox proportional hazards model, achieving rapid training.
Fearless Stochasticity in Expectation Propagation
Jonathan So (University of Cambridge), Richard E. Turner (University of Cambridge)
OptimizationTabular
🎯 What it does: This paper proposes two new variants of Expectation Propagation (EP) - EPη and EPµ, which maintain numerical stability when using Monte Carlo (MC) sampling for estimation updates and can converge efficiently with just a single sample.
Feature-Level Adversarial Attacks and Ranking Disruption for Visible-Infrared Person Re-identification
Xi Yang (Xidian University), Xinbo Gao (Chongqing University of Posts and Telecommunications)
RecognitionAdversarial AttackConvolutional Neural NetworkImageMultimodality
🎯 What it does: This paper proposes a feature-level adversarial attack and ranking disruption method for visible-infrared person re-identification (VIReID), utilizing universal adversarial perturbations, a frequency-space attention module (FSAM), and an auxiliary quadruple adversarial loss for system security assessment.
FedAvP: Augment Local Data via Shared Policy in Federated Learning
Minui Hong (Seoul National University), Gunhee Kim (Seoul National University)
Federated LearningMeta LearningImage
🎯 What it does: FedAvP is proposed in a federated learning environment, using shared augmentation strategies instead of shared data for local data augmentation.
Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning
Dario Fenoglio (Università della Svizzera italiana), Marc Langheinrich (Università della Svizzera italiana)
Federated LearningSafty and PrivacyTabular
🎯 What it does: This paper proposes Federated Behavioural Planes (FBPs), which visualize and track the behavior of each client in federated learning through two behavioral planes (prediction error plane and counterfactual plane); based on FBPs, it designs Federated Behavioural Shields as a robust aggregation strategy to enhance the detection and defense against malicious clients.
Federated Black-Box Adaptation for Semantic Segmentation
Jay Nitin Paranjape, Vishal M. Patel (Johns Hopkins University)
SegmentationFederated LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: BlackFed is proposed under the federated learning framework, achieving black-box distributed learning for semantic segmentation tasks without gradient or model information exchange.
Federated Ensemble-Directed Offline Reinforcement Learning
Desik Rengarajan (Texas A&M University), Srinivas Shakkottai (Texas A&M University)
Federated LearningReinforcement LearningSequential
🎯 What it does: A federated offline reinforcement learning algorithm named FEDORA is proposed and implemented, which enables multiple clients to collaboratively learn high-quality control policies without sharing data.
Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources
Jiamu Bai (Pennsylvania State University), Yaliang Li (Alibaba Group)
Federated LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes FlexLoRA, an aggregation method for parameter-efficient fine-tuning of large language models in the context of federated learning, addressing the 'bucket effect' caused by the heterogeneity of resources and tasks across different clients.
Federated Graph Learning for Cross-Domain Recommendation
Ziqi Yang (Xiamen University), Xiaoliang Fan (Xiamen University)
Recommendation SystemFederated LearningSafty and PrivacyGraph Neural NetworkGraph
🎯 What it does: A cross-domain recommendation framework FedGCDR based on federated graph learning is proposed, which supports multi-source domain knowledge transfer while preventing privacy leakage and negative transfer.
Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method
Bikang Pan (ShanghaiTech University), Ye Shi (ShanghaiTech University)
Federated LearningTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: The paper proposes a federated learning framework based on the visual language foundation model CLIP, and achieves a combination of global and local prompts through prompt learning (PromptFolio) to balance the model's generalization ability and personalized adaptability.
Federated Learning over Connected Modes
Dennis Grinwald (BIFOLD), Shinichi Nakajima (RIKEN Center for Advanced Intelligence Project)
Federated LearningConvolutional Neural NetworkImage
🎯 What it does: A federated learning method called FLOCO is proposed, which utilizes pattern connectivity to construct a solution simplex and maps clients to sub-regions of the standard simplex through gradient similarity, allowing for local learning within these sub-regions while balancing global model and local personalization.
Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis
Michael Crawshaw (George Mason University), Mingrui Liu (George Mason University)
Federated LearningComputational EfficiencyImage
🎯 What it does: A new algorithm for Federated Learning, Amplified SCAFFOLD, is proposed to address periodic client participation and data heterogeneity, along with theoretical proofs and experimental validation.
Federated Model Heterogeneous Matryoshka Representation Learning
Liping Yi (Nankai University), Xiaoxiao Li (University of British Columbia)
Federated LearningRepresentation LearningImage
🎯 What it does: The FedMRL method is proposed, which utilizes a small homogeneous model shared by the server and heterogeneous models on the clients to achieve adaptive representation fusion and multi-level Matryoshka representation learning, addressing the issues of data, system, and model heterogeneity in model heterogeneous federated learning.
Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning
Tong Yang (Carnegie Mellon University), Yuejie Chi (Carnegie Mellon University)
Federated LearningReinforcement LearningSequential
🎯 What it does: A completely decentralized federated multi-task reinforcement learning framework is proposed, with the design of two algorithms, FedNPG and FedNAC, to learn a global optimal policy without sharing local reward information.
Federated Online Prediction from Experts with Differential Privacy: Separations and Regret Speed-ups
Fengyu Gao (Pennsylvania State University), Jing Yang (Pennsylvania State University)
Recommendation SystemFederated LearningSafty and PrivacyTabularSequential
🎯 What it does: The research addresses the problem of differential privacy online expert prediction in a federated environment, proposing the Fed-DP-OPE-Stoch algorithm for random adversaries and the Fed-SVT algorithm for blind adversaries under realizability constraints.
Federated Transformer: Multi-Party Vertical Federated Learning on Practical Fuzzily Linked Data
Zhaomin Wu (National University of Singapore), Bingsheng He (National University of Singapore)
Federated LearningSafty and PrivacyTransformerTabular
🎯 What it does: Proposes Federated Transformer (FeT), supporting vertical federated learning with multi-party fuzzy associations.
FedGMark: Certifiably Robust Watermarking for Federated Graph Learning
Yuxin Yang (Jilin University), Binghui Wang (Illinois Institute of Technology)
Federated LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes FedGMark, a provably robust backdoor watermarking method for federated graph learning (FedGL) models to protect model ownership.
FedGMKD: An Efficient Prototype Federated Learning Framework through Knowledge Distillation and Discrepancy-Aware Aggregation
Jianqiao Zhang (Aberystwyth University), Jungong Han (Tsinghua University)
Federated LearningKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: Proposes the FedGMKD framework, which achieves efficient prototype federated learning without public data and without a server by utilizing knowledge fusion through clustering and difference-aware aggregation.
FedGTST: Boosting Global Transferability of Federated Models via Statistics Tuning
Evelyn Ma (University of Illinois Urbana-Champaign), Olgica Milenkovic (University of Illinois Urbana-Champaign)
Domain AdaptationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new federated learning algorithm, FedGTST, aimed at enhancing the transfer performance of federated models in the target domain. The algorithm utilizes cross-client Jacobian norm (gradient norm) information at both the client and server sides for statistical tuning to control gradient variance and improve the average gradient norm, thereby directly affecting the loss in the target domain.
FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation
Xiang Liu (National University of Singapore), Jialin Li (National University of Singapore)
OptimizationFederated LearningImage
🎯 What it does: A one-round federated learning framework FedLPA is proposed, utilizing hierarchical posterior aggregation to achieve data-independent global model training.
FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction
Ziwei Li (Ohio State University), Wei-Lun Chao (Ohio State University)
Federated LearningContrastive LearningImageBiomedical Data
🎯 What it does: A neighbor embedding (FEDNE) model is proposed under the federated learning framework to address the visualization problem of distributed data without data sharing.
FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference
Zihan Tan (Wuhan University), Mang Ye (Wuhan University)
Federated LearningGraph Neural NetworkGraph
🎯 What it does: Learning personalized graph neural networks for each client in a federated environment, and achieving cross-domain graph classification through spectral knowledge sharing and preference adjustment.
Feedback control guides credit assignment in recurrent neural networks
Klara Kaleb (Imperial College London), Claudia Clopath (Imperial College London)
Recurrent Neural NetworkTime SeriesSequential
🎯 What it does: This paper studies the rapid online correction and local learning effects in motion adaptation tasks by incorporating feedback control into recurrent neural networks.
FEEL-SNN: Robust Spiking Neural Networks with Frequency Encoding and Evolutionary Leak Factor
Mengting Xu (Zhejiang University), Gang Pan (Zhejiang University)
Adversarial AttackSpiking Neural NetworkImage
🎯 What it does: A robust spiking neural network named FEEL-SNN is designed and implemented, combining frequency encoding (FE) and evolutionary leakage factors (EL) to enhance resistance to adversarial and noise attacks.
Feint Behaviors and Strategies: Formalization, Implementation and Evaluation
Junyu Liu (Brown University), Xiangjun Peng (Chinese University of Hong Kong)
Reinforcement Learning
🎯 What it does: This paper proposes and implements a complete formalization and realization of 'Feint' (deceptive actions) in multiplayer games, including action-level Palindrome-guided template generation, a Dual-Behavior model, and a unified implementation and evaluation within the MARL framework.
FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning
Lisha Chen (Rensselaer Polytechnic Institute), Tianyi Chen (Rensselaer Polytechnic Institute)
OptimizationAudio
🎯 What it does: This paper proposes the FERERO framework, modeling preference-guided multi-objective learning as a constrained vector optimization problem, and designs a single-loop primal algorithm and its stochastic variant, which can adaptively handle relative and absolute preferences, support controlled ascent, and avoid weak optimal solutions.
Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity
Hanlin Gu (Webank), Lixin Fan (Webank)
Federated LearningImageTextTabular
🎯 What it does: The Federated Feature Unlearning framework Ferrari is proposed, which achieves the unlearning of sensitive/backdoor/bias features in federated learning by minimizing feature sensitivity.
Fetch and Forge: Efficient Dataset Condensation for Object Detection
Ding Qi (Tongji University), Cairong Zhao (Tongji University)
Object DetectionData SynthesisCompressionOptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: The DCOD framework is proposed, which divides dataset condensation into two stages: Fetch and Forge, using model inversion techniques to extract and reconstruct synthetic images from the original detection data.
Few-Shot Adversarial Prompt Learning on Vision-Language Models
Yiwei Zhou (Beijing Institute of Technology), Tongliang Liu (University of Sydney)
ClassificationAdversarial AttackTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: This paper proposes a few-shot adversarial prompt learning framework (FAP) that enhances the adversarial robustness of the pre-trained vision-language model CLIP by learning adjustable prompts and adversarial text supervision.
Few-Shot Diffusion Models Escape the Curse of Dimensionality
Ruofeng Yang (Shanghai Jiao Tong University), Shuai Li (Shanghai Jiao Tong University)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelImageStochastic Differential Equation
🎯 What it does: This paper conducts a theoretical analysis to study the approximation error and optimization properties of few-shot diffusion models, and experimentally verifies that high-quality generation can be achieved by fine-tuning only the encoder/decoder.
Few-Shot Task Learning through Inverse Generative Modeling
Aviv Netanyahu (Massachusetts Institute of Technology), Pulkit Agrawal (Massachusetts Institute of Technology)
Autonomous DrivingOptimizationRobotic IntelligenceDiffusion modelMultimodalitySequential
🎯 What it does: Using pre-trained reversible generative models (such as diffusion models) to learn new task concepts by optimizing latent concept vectors with only a few demonstrations, and generating corresponding trajectories when in new environments or in combination with learned concepts.
FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training
Ruihong Yin (University of Amsterdam), Theo Gevers (University of Amsterdam)
RestorationGenerationData SynthesisGaussian SplattingImage
🎯 What it does: Developed a new perspective synthesis method called FewViewGS based on 3D Gaussian Splatting, achieving high-quality rendering of sparse input images through multi-stage training and matching consistency constraints.
FFAM: Feature Factorization Activation Map for Explanation of 3D Detectors
Shuai Liu (Sun Yat-sen University), Kai Huang (Sun Yat-sen University)
Object DetectionAutonomous DrivingExplainability and InterpretabilityPoint Cloud
🎯 What it does: This paper proposes a feature factorization-based activation map (FFAM) method for generating visual explanations for LiDAR-based 3D detectors.
FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
Feijie Wu (Purdue University), Jing Gao (Purdue University)
Federated LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningImageText
🎯 What it does: A model heterogeneous federated learning algorithm named FIARSE is designed and implemented, utilizing importance-aware sub-models to achieve client adaptive sub-model training and global model aggregation.
FIDE: Frequency-Inflated Conditional Diffusion Model for Extreme-Aware Time Series Generation
Asadullah Hill Galib (Michigan State University), Lifeng Luo (Michigan State University)
GenerationData SynthesisAnomaly DetectionDiffusion modelTime SeriesSequentialBiomedical DataElectrocardiogramFinance Related
🎯 What it does: This paper proposes a temporal generation framework FIDE based on diffusion models, specifically designed to better preserve the distribution of extreme values (block extremes).
FIFO-Diffusion: Generating Infinite Videos from Text without Training
Jihwan Kim (Seoul National University), Bohyung Han (Seoul National University)
GenerationData SynthesisDiffusion modelVideoText
🎯 What it does: This paper proposes FIFO-Diffusion, a training-independent inference method that utilizes a pre-trained short video diffusion model to achieve infinite-length video generation through diagonal denoising and a queue mechanism. It also introduces latent space partitioning and lookahead denoising to enhance quality and stability.
Fight Back Against Jailbreaking via Prompt Adversarial Tuning
Yichuan Mo (Peking University), Yisen Wang (Peking University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes Prompt Adversarial Tuning (PAT), which enhances the robustness of large language models against jailbreak attacks by inserting a trained defense prefix (control prompt) before user prompts during inference, while maintaining normal responses to legitimate requests.
FilterNet: Harnessing Frequency Filters for Time Series Forecasting
Kun Yi (North China Institute of Computing Technology), Wei Fan (University of Oxford)
TransformerTime Series
🎯 What it does: A time series prediction framework called FilterNet based on learnable frequency filters is proposed, using instance normalization + FFT + learnable filtering + FFN for modeling.
FINALLY: fast and universal speech enhancement with studio-like quality
Nicholas Babaev (Samsung Research), Pavel Andreev (Samsung Research)
RestorationGenerationGenerative Adversarial NetworkAudio
🎯 What it does: A speech enhancement model named FINALLY is proposed, based on the HiFi++ architecture and incorporating WavLM perceptual loss and multi-stage training, capable of generating high-quality, studio-like clean speech at 48 kHz in a single forward inference.
FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making
Yangyang Yu (Stevens Institute of Technology), Qianqian Xie
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextTime SeriesFinance RelatedAudio
🎯 What it does: This paper presents FINCON, a multi-agent system based on large language models for single stock trading and portfolio management.
Finding good policies in average-reward Markov Decision Processes without prior knowledge
Adrienne Tuynman, Emilie Kaufmann (University of Lille)
OptimizationReinforcement Learning
🎯 What it does: Under the premise of no prior knowledge, a set of optimal policy identification algorithms for average reward Markov Decision Processes (MDP) has been designed, and it has been proven that the traditional optimal bias span (H) cannot be estimated through polynomial samples. Consequently, a Diameter Free Exploration (DFE) algorithm based on diameter estimation is proposed to achieve prior-free PAC optimal policy output. An online version of DFE is also provided, along with an adaptive stopping rule based on value iteration, and a lower bound for online learning is given.
Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models
Dominik Hintersdorf (German Research Center for Artificial Intelligence), Franziska Boenisch (CISPA Helmholtz Center for Information Security)
GenerationData SynthesisAnomaly DetectionTransformerDiffusion modelImageText
🎯 What it does: The NEMO method is proposed to locate memorized neurons in diffusion models and eliminate the model's replication of training samples by deactivating these neurons.
Finding Transformer Circuits With Edge Pruning
Adithya Bhaskar (Princeton University), Danqi Chen (Princeton University)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes a technique called Edge Pruning, which uses gradient optimization to sparsify edges in Transformers, automatically discovering sparse computational subgraphs (Circuits) relevant to specific tasks.
Fine Tuning Out-of-Vocabulary Item Recommendation with User Sequence Imagination
Ruochen Liu (Central South University), Jianxin Wang (Central South University)
Recommendation SystemReinforcement LearningSequential
🎯 What it does: By constructing the User Sequence Imagination (USIM) framework, the embedding of out-of-vocabulary (OOV) items without historical interactions is further optimized, thereby improving the recommendation effectiveness for OOV items.
Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond
Yingcong Li (University of Michigan), Samet Oymak (University of Michigan)
Domain AdaptationOptimizationTabularRetrieval-Augmented Generation
🎯 What it does: The paper studies the optimization and generalization landscape of single-layer linear attention and state space models in context learning, and theoretically and experimentally verifies that both achieve first-order preconditioned gradient descent.
Fine-grained Control of Generative Data Augmentation in IoT Sensing
Tianshi Wang (University of Illinois Urbana-Champaign), Tarek F. Abdelzaher
ClassificationGenerationData SynthesisTransformerDiffusion modelAuto EncoderTime SeriesSequential
🎯 What it does: This paper proposes a fine-grained controllable generative data augmentation method that guides the synthesis of IoT sensor data by constructing a statistical metric space based on STFT spectrograms, thereby enhancing the robustness of downstream models.
Fine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random
Mingming Ha (MYbank, Ant Group), Linxun Chen (MYbank, Ant Group)
Recommendation SystemOptimizationTabular
🎯 What it does: The paper addresses the prediction bias in missing not at random (MNAR) data and proposes a fine-grained dynamic framework to achieve joint optimization of bias and variance.
Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Models
Yifan Zhang (City University of Hong Kong), Junhui Hou (City University of Hong Kong)
Object DetectionSegmentationAutonomous DrivingKnowledge DistillationRepresentation LearningContrastive LearningImagePoint Cloud
🎯 What it does: The OLIVINE method is proposed, which generates weak semantic labels through a Visual Foundation Model (VFM) and conducts fine-grained image-point cloud contrastive learning on LiDAR point clouds to address self-conflict issues and enhance 3D representation learning.
Fine-Tuning is Fine, if Calibrated
Zheda Mai (Ohio State University), Wei-Lun Chao (Ohio State University)
ClassificationDomain AdaptationSupervised Fine-TuningImage
🎯 What it does: The study investigates the behavior of models when fine-tuning a pre-trained classifier using only a subset of classes, finding that features are not forgotten. The main issue is the logit bias for known classes, which leads to a decrease in accuracy for missing classes. It proposes a post-processing calibration (adding a bias factor γ) to restore the performance of missing classes.
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
Yuexiang Zhai (University of California Berkeley), Sergey Levine (University of California Berkeley)
Robotic IntelligenceTransformerReinforcement LearningVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: A framework is proposed that utilizes reinforcement learning to directly fine-tune large-scale visual language models. The model generates chain-of-thought and text actions after receiving a task description and visual input, then parses the actions into the environment and updates the model through reward feedback.
Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients
Youssef Allouah (École Polytechnique Fédérale de Lausanne), Rafael Pinot (Sorbonne Université and Université Paris Cité)
Domain AdaptationFederated LearningAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper studies the incorporation of fine-tuned personalization in federated learning to mitigate the impact of adversarial clients. It proposes an interpolation personalization objective in the presence of Byzantine adversaries and provides the theoretical optimal value of the cooperation degree λ and its relationship with data heterogeneity, adversarial ratio, and sample size.
FineCLIP: Self-distilled Region-based CLIP for Better Fine-grained Understanding
Dong Jing (Renmin University of China), Zhiwu Lu (Renmin University of China)
Object DetectionSegmentationRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes FineCLIP, a visual-language pre-training framework that enhances fine-grained understanding based on CLIP.
FineStyle: Fine-grained Controllable Style Personalization for Text-to-image Models
Gong Zhang (Georgia Tech), Irfan Essa (Google DeepMind)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: This paper studies a fine-grained style personalization method for text-to-image models called FineStyle, which requires only a single style reference image.
First-Explore, then Exploit: Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offs
Ben Norman (University of British Columbia), Jeff Clune (University of British Columbia)
Meta LearningTransformerReinforcement Learning
🎯 What it does: This paper proposes the First-Explore framework, which trains two strategies: explore first and then exploit, to address the exploration challenges faced by traditional cumulative reward Meta-RL.
First-Order Methods for Linearly Constrained Bilevel Optimization
Guy Kornowski (Weizmann Institute of Science), Suvrit Sra (Massachusetts Institute of Technology)
Optimization
🎯 What it does: A first-order linear constrained bilevel optimization algorithm is proposed, with finite-time convergence guarantees.
First-Order Minimax Bilevel Optimization
Yifan Yang (University at Buffalo), Kaiyi Ji (University at Buffalo)
OptimizationMeta LearningImage
🎯 What it does: Two full gradient first-order methods (FOSL and MemCS) are proposed to solve multi-block min-max bilevel optimization problems.
Fisher Flow Matching for Generative Modeling over Discrete Data
Oscar Davis (University of Oxford), Joey Bose
GenerationData SynthesisOptimizationFlow-based ModelTextSequential
🎯 What it does: Fisher-Flow is proposed, a generative model for matching discrete data streams based on the Fisher-Rao metric;
Fixed Confidence Best Arm Identification in the Bayesian Setting
Kyoungseok Jang (Universitá degli Studi di Milano), Kazutoshi Yamazaki (University of Queensland)
🎯 What it does: This study investigates the Bayesian best arm identification problem under fixed confidence, proving that traditional frequentist algorithms perform poorly in Bayesian settings, and proposes an algorithm based on elimination and early stopping to achieve approximately optimal sample complexity.
FLAME : Factuality-Aware Alignment for Large Language Models
Sheng-Chieh Lin (University of Waterloo), Xilun Chen (Meta AI)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: By incorporating factual objectives into the alignment process of large language models, the factual accuracy of the model has been improved.
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
Jay Shah (Colfax Research), Tri Dao (Princeton University)
Transformer
🎯 What it does: A new Attention kernel FLASHATTENTION-3 has been implemented on the Hopper H100 GPU, utilizing asynchronous execution and low-precision FP8 to enhance the speed of attention computation in Transformers.
Flatten Anything: Unsupervised Neural Surface Parameterization
Qijian Zhang (City University of Hong Kong), Ying He (Nanyang Technological University)
Point CloudMesh
🎯 What it does: A fully automatic unsupervised neural surface parameterization model FAM is proposed, capable of achieving global free boundary UV unfolding on arbitrary topologies, any mesh quality, and even unstructured point clouds, directly operating on discrete point sets without manual cutting.
Flaws can be Applause: Unleashing Potential of Segmenting Ambiguous Objects in SAM
Chenxin Li (Chinese University of Hong Kong), Yixuan Yuan (Chinese University of Hong Kong)
SegmentationGenerationAuto EncoderImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: The A-SAM framework is proposed, utilizing conditional variational autoencoders to model the prompts and object fine-grained details of SAM as probability distributions, achieving controllable and diverse generation of ambiguous segmentation results.
Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts
Sukwon Yun (University of North Carolina), Tianlong Chen (University of North Carolina)
ClassificationRecognitionAnomaly DetectionTransformerMixture of ExpertsMultimodalityBiomedical DataAlzheimer's DiseaseElectronic Health Records
🎯 What it does: A multi-modal learning framework named Flex-MoE has been designed and implemented, capable of flexibly handling any combination of missing modalities;
FlexCap: Describe Anything in Images in Controllable Detail
Debidatta Dwibedi (Google Deepmind), Yusuf Aytar (Google Deepmind)
Object DetectionGenerationTransformerLarge Language ModelVision Language ModelImageVideoText
🎯 What it does: FlexCap is proposed, a visual language model that can control the length of descriptions based on specified regions and word counts, capable of generating controllable regional descriptions ranging from short labels to detailed sentences.
Flexible Context-Driven Sensory Processing in Dynamical Vision Models
Lakshmi Narasimhan Govindarajan (Massachusetts Institute of Technology), Ila R Fiete (Massachusetts Institute of Technology)
RecognitionOptimizationExplainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: A trainable, biologically inspired dynamic cortical network (DCnet) with excitatory/inhibitory neurons and low-rank up/down modulation is proposed to solve visual cue-delay search tasks and replicate attention reaction times in psychophysical experiments.
Flexible mapping of abstract domains by grid cells via self-supervised extraction and projection of generalized velocity signals
Abhiram Iyer (Massachusetts Institute of Technology), Ila R Fiete
Auto EncoderSimultaneous Localization and MappingTime Series
🎯 What it does: This paper proposes a self-supervised framework for extracting low-dimensional velocity signals from high-dimensional abstract cognitive spaces, and utilizes the extracted velocities to achieve path integration of grid cells, allowing the same set of grid cells to map multiple non-spatial environments.
Flexible task abstractions emerge in linear networks with fast and bounded units
Kai Jappe Sandbrink (Oxford Brain Mind Institute), Ali Hummos (Massachusetts Institute of Technology)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper demonstrates that task abstraction (gate variables) can spontaneously form by jointly training linear networks with fast, non-negative, and finite activation gates using gradient descent, thereby achieving flexible task switching and combinatorial generalization.
FlexPlanner: Flexible 3D Floorplanning via Deep Reinforcement Learning in Hybrid Action Space with Multi-Modality Representation
Ruizhe Zhong (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationGraph Neural NetworkTransformerReinforcement LearningMultimodalityGraphBenchmark
🎯 What it does: This paper proposes FlexPlanner, a 3D layout planning method based on deep reinforcement learning, which can directly determine the 2D position, hierarchy, and aspect ratio of modules, and complete planning through multimodal representations (visual, graph, sequence).
FlexSBDD: Structure-Based Drug Design with Flexible Protein Modeling
ZAIXI ZHANG, Qi Liu (University of Science and Technology of China)
Drug DiscoveryFlow-based ModelBiomedical DataOrdinary Differential Equation
🎯 What it does: We propose FlexSBDD, a flow-matching based deep generative model that can generate 3D ligand molecules binding to proteins while maintaining the structural flexibility of the proteins.
Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery
Haonan Lin (Xi'an Jiaotong University), Jingdong Wang (Baidu Inc)
ClassificationRecognitionTransformerContrastive LearningImage
🎯 What it does: The FlipClass method is proposed, which dynamically updates the teacher's attention to keep the focus of the teacher and student consistent, thereby improving the task of generalization category discovery.
Flipping-based Policy for Chance-Constrained Markov Decision Processes
Xun Shen (Osaka University), Sebastien Gros (Norwegian University of Science and Technology)
OptimizationSafty and PrivacyReinforcement LearningSequentialBenchmark
🎯 What it does: In the joint probability constrained Markov decision process (CCMDP), a 'flipping' strategy (state-dependent binary stochastic policy) is proposed and theoretically proven to achieve optimality, along with a practical framework for training this strategy using existing safe RL algorithms (such as CPO, PCPO).
FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
Ziyao Wang (University of Maryland), Ang Li (University of Maryland)
Federated LearningSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: FLORA is proposed, a noise-free global update method for heterogeneous low-rank adapter aggregation of large language models using LoRA within a federated learning framework.
Flow Priors for Linear Inverse Problems via Iterative Corrupted Trajectory Matching
Yasi Zhang (University of California Los Angeles), Oscar Leong (University of California Los Angeles)
RestorationSuper ResolutionCompressionFlow-based ModelImageBiomedical DataMagnetic Resonance ImagingOrdinary Differential Equation
🎯 What it does: An iterative algorithm (ICTM) is proposed that utilizes the prior of the flow matching model to perform MAP estimation on linear inverse problems (super-resolution, deblurring, inpainting, compressed sensing), achieving efficient recovery without the need for backpropagation through the ODE solver.
Flow Snapshot Neurons in Action: Deep Neural Networks Generalize to Biological Motion Perception
Shuangpeng Han (Nanyang Technological University), Mengmi Zhang (Nanyang Technological University)
ClassificationRecognitionOptical FlowVideoBenchmark
🎯 What it does: The Motion Perceiver (MP) model is proposed, which utilizes patch-level optical flow relying solely on motion information for action recognition, achieving zero-training generalization in the bio-motion perception (BMP) task under lit conditions.
FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributions
Anuroop Sriram (Meta), Brandon M Wood
GenerationData SynthesisGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningFlow-based ModelText
🎯 What it does: This paper proposes a generative framework called FlowLLM that combines large language models (LLM) with Riemannian Flow Matching (RFM) for the efficient generation of stable and novel crystal materials.
FlowTurbo: Towards Real-time Flow-Based Image Generation with Velocity Refiner
Wenliang Zhao (Tsinghua University), Jiwen Lu (Tsinghua University)
GenerationData SynthesisFlow-based ModelImageTextOrdinary Differential Equation
🎯 What it does: The FlowTurbo framework is proposed to enhance the sampling speed of flow-based generative models while maintaining quality;
FM-Delta: Lossless Compression for Storing Massive Fine-tuned Foundation Models
Wanyi Ning (Beijing University of Posts and Telecommunications), Ce Zhang (University of Chicago)
CompressionSupervised Fine-Tuning
🎯 What it does: A lossless compression method FM-Delta is proposed for large-scale fine-tuning of foundational models in cloud storage, which compresses the complete fine-tuned model by mapping floating-point parameters to unsigned integers and performing range coding on the integer differences.
fMRI predictors based on language models of increasing complexity recover brain left lateralization
Laurent Bonnasse-Gahot (Centre d'Analyse et de Mathématique Sociales), Christophe Pallier (Cognitive Neuroimaging Unit)
TransformerLarge Language ModelTextMagnetic Resonance ImagingAudio
🎯 What it does: Using pre-trained language models of different scales (28 models, ranging from 124M to 14.2B parameters) to encode and model natural language auditory fMRI data, exploring the impact of model complexity on brain signal prediction.
FNP: Fourier Neural Processes for Arbitrary-Resolution Data Assimilation
Kun Chen (Fudan University), LEI BAI
Time Series
🎯 What it does: A method for assimilating observational data of arbitrary resolution based on Fourier Neural Processes (FNP) is proposed, which can directly fuse observations of different resolutions with the background to generate high-precision analysis fields.
Focus On What Matters: Separated Models For Visual-Based RL Generalization
Di Zhang (Tongji University), changjun jiang
Robotic IntelligenceReinforcement LearningImageVideo
🎯 What it does: The SMG method is proposed, which utilizes a separation model and an image reconstruction auxiliary task to enhance the zero-shot generalization performance of visual reinforcement learning (RL).
FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detection
Xinting Liao (Zhejiang University), Xiaolin Zheng (Zhejiang University)
Domain AdaptationAnomaly DetectionFederated LearningScore-based ModelImage
🎯 What it does: This paper proposes FOOGD, a federated learning framework aimed at simultaneously addressing the issues of covariate shift (OOD normalization) and semantic shift (OOD detection) that arise in non-IID real-world data.
Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning
Wang Xinrui, Songcan Chen (Huazhong University of Science and Technology)
ClassificationRecognitionTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes a new online continuous learning framework, NsCE, which addresses the issues of 'ignorance' and 'short-sightedness' that arise in single-channel learning.
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
Zhenyu Zhang (University of Texas at Austin), Zhangyang Wang (University of Texas at Austin)
GenerationRetrievalTransformerLarge Language ModelText
🎯 What it does: Proposes a multi-scale position encoding (Ms-PoE) to address the 'intermediate loss' problem in LLMs with long contexts, achieving improved context utilization without additional training or overhead.
Foundation Inference Models for Markov Jump Processes
David Berghaus (Fraunhofer Institute for Intelligent Analysis and Information Systems), Ramses J Sanchez
Recurrent Neural NetworkTransformerTime SeriesSequential
🎯 What it does: A zero-shot inference framework based on a pre-trained neural recognition model (Foundation Inference Model, FIM) is proposed to recover the transition rate matrix and initial distribution of Markov Jump Processes (MJP) from noisy sparse observations in a discrete bounded state space.
Foundations of Multivariate Distributional Reinforcement Learning
Harley Wiltzer (Mila - Quebec AI Institute McGill University), Mark Rowland (Google DeepMind)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a computable and theoretically provable convergent Multivariate Distributional Reinforcement Learning algorithm, covering both dynamic programming and temporal difference learning paradigms.
FouRA: Fourier Low-Rank Adaptation
Shubhankar Borse (Qualcomm AI Research), Fatih Porikli (Qualcomm AI Research)
GenerationDomain AdaptationTransformerDiffusion modelImageText
🎯 What it does: A low-rank adaptive parameter-efficient fine-tuning method called FouRA has been developed for style transfer and concept editing in text-to-image diffusion models and language models.
Fourier Amplitude and Correlation Loss: Beyond Using L2 Loss for Skillful Precipitation Nowcasting
Chiu-Wai Yan (Hong Kong University of Science and Technology), Wai-Kin Wong (Hong Kong Observatory)
GenerationData SynthesisConvolutional Neural NetworkRecurrent Neural NetworkTransformerImageTime Series
🎯 What it does: This paper proposes a frequency-domain-based loss function—Fourier Amplitude and Correlation Loss (FACL)—to replace the traditional pixel-level MSE, aiming to enhance the clarity and realism of precipitation nowcasting images.
Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion
Yujie Liang, Liang-Jian Deng (University of Electronic Science and Technology of China)
Image TranslationRestorationData SynthesisConvolutional Neural NetworkImage
🎯 What it does: A Fourier-enhanced implicit neural fusion network (FeINFN) is proposed for the fusion of multispectral and hyperspectral images.
Fractal Patterns May Illuminate the Success of Next-Token Prediction
Ibrahim Alabdulmohsin (Google Deepmind), Mostafa Dehghani (Google Deepmind)
Large Language ModelText
🎯 What it does: By normalizing the negative logarithmic probabilities of text sequences and calculating fractal parameters, the self-similarity and long-range dependencies of language are studied;
Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancement
Yanyan Huang (University of Hong Kong), Lequan Yu (University of Hong Kong)
ClassificationDomain AdaptationTransformerVision Language ModelContrastive LearningImageBiomedical Data
🎯 What it does: This paper proposes the CATE framework, which utilizes task-specific concept guidance in the pathology vision-language model to calibrate and enhance the features of general foundational models, thereby improving the performance of multi-instance learning (MIL) in whole slide image (WSI) classification tasks.
Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning
Mengmeng Chen (Beijing University of Posts and Telecommunications), Han Yu (Westlake University)
Federated LearningGraph Neural NetworkTabularBiomedical Data
🎯 What it does: The FedEgoists algorithm is proposed to construct a cooperative alliance in cross-device federated learning that is free of free riders and avoids conflicts of interest.