Conference on Neural Information Processing Systems Β· 1874 papers
DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs
Haokun Lin (University of Chinese Academy of Sciences), Ying Wei (Zhejiang University)
CodeAnomaly DetectionOptimizationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A low-bit quantization method for LLM named DuQuant is proposed, which effectively suppresses large-scale outliers (including ordinary and large-scale outliers) in activations through rotation and permutation transformations, thereby improving the performance of 4/6-bit quantized models.
Dynamic Conditional Optimal Transport through Simulation-Free Flows
Gavin Kerrigan (University of California, Irvine), Padhraic Smyth (University of California, Irvine)
CodeGenerationOptimizationFlow-based ModelTabularTime Series
π― What it does: A theory of dynamic conditional optimal transport is proposed, and based on this, a conditional generative model (COT-FM) without simulation flow matching is constructed.
π― What it does: A dynamic neural regeneration (DNR) iterative training framework is proposed, which enhances the generalization ability of small sample datasets by reinitializing part of the parameters based on a data-aware significance mask at the end of each training generation.
Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation
Wangbo Zhao (National University of Singapore), Yang You (National University of Singapore)
CodeClassificationSegmentationOptimizationComputational EfficiencyTransformerMixture of ExpertsImageVideo
π― What it does: A dynamic tuning framework named DyT is proposed to simultaneously enhance parameter efficiency and inference efficiency during the adaptation process of Vision Transformers (ViT).
π― What it does: This paper proposes an Event-Sequence Diffusion Network, which utilizes high spatiotemporal resolution data from event cameras as conditional input. It generates future event sequences through pre-training, reinforcement learning alignment, and multi-frame enhancement during testing to predict object motion.
π― What it does: An end-to-end multimodal fusion detection framework E2E-MFD is proposed, capable of completing the training and inference of visible-infrared image fusion and object detection in one go.
π― What it does: A 3D medical image segmentation network called E2ENet is proposed, which mainly enhances segmentation accuracy while reducing model size and computational load through Dynamic Sparse Feature Fusion (DSFF) and Restricted Depth-Shift.
EAI: Emotional Decision-Making of LLMs in Strategic Games and Ethical Dilemmas
Mikhail Mozikov (AIRI), Ilya Makarov
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: The paper proposes the EAI framework, which injects emotions into LLMs and evaluates their performance in ethical judgment and game decision-making.
π― What it does: This paper proposes the 'easy-to-difficult generalization' of transferring human supervision from easy tasks to hard tasks, by first training an evaluation model and then using it to assess or guide the generator through reinforcement learning, thereby improving model performance without human supervision on hard tasks.
π― What it does: This paper proposes a scalable Lipschitz constant estimation framework, which breaks down large-scale SDP verification problems into smaller subproblems layer by layer, introducing two algorithms: ECLipsE (using small-scale SDP) and ECLipsE-Fast (using closed-form solutions);
π― What it does: Developed a 10M parameter EEGPT pre-trained Transformer model to learn general and robust EEG representations from multi-task, multi-modal EEG data, achieving state-of-the-art performance on downstream tasks through linear probing.
Effective Exploration Based on the Structural Information Principles
Xianghua Zeng (Beihang University), Angsheng Li (Zhongguancun Laboratory)
CodeReinforcement LearningTabular
π― What it does: A framework called SI2E based on the principle of structural information is proposed for efficient exploration in reinforcement learning.
π― What it does: This paper proposes the Representation Learning Accelerator (RELA) framework, which utilizes publicly available pre-trained models as labelers to generate high-quality efficient data, thereby accelerating the representation learning process.
CodeComputational EfficiencyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: An efficient continuous adversarial training algorithm CAT and CAPO is proposed, enhancing the robustness of LLM against discrete adversarial attacks while maintaining practicality.
Efficient and Private Marginal Reconstruction with Local Non-Negativity
Brett Mullins (University of Massachusetts), Daniel Sheldon (University of Massachusetts)
CodeSafty and PrivacyComputational EfficiencyTabular
π― What it does: This paper proposes a post-processing method based on residual queries, called ReM, for efficiently reconstructing multi-dimensional marginal queries in a differentially private (DP) environment, and extends to Gaussian noise and local non-negativity constraints.
Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes
Andrew Bennett (Morgan Stanley), Kaiwen Wang (Cornell University)
CodeReinforcement LearningBiomedical DataElectronic Health Records
π― What it does: A precise evaluation framework for the best/worst policy value in model-free reinforcement learning under unknown environmental disturbances is proposed.
π― What it does: This paper proposes an approximate Centroid-Linkage Hierarchical Clustering (HAC) algorithm that can complete clustering in sub-quadratic time.
Efficient Combinatorial Optimization via Heat Diffusion
Hengyuan Ma (Institute of Science and Technology for Brain-inspired Intelligence Fudan University), Jianfeng Feng (Institute of Science and Technology for Brain-inspired Intelligence Fudan University)
π― What it does: A combination optimization framework based on thermal diffusion, HeO, is proposed, which utilizes the heat equation to smooth the objective function and introduces a thermal diffusion parameter during the gradient descent process to expand the search receptive field and improve the efficiency of global optimal search.
Efficient Large Multi-modal Models via Visual Context Compression
Jieneng Chen (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageVideoMultimodality
π― What it does: This paper addresses the issue of visual token redundancy in multimodal large language models (MLLMs) by proposing the Visual Context Compressor and the staged training framework LLaVolta, significantly improving training and inference efficiency while maintaining or even enhancing model performance.
π― What it does: A randomized ALS algorithm based on precise leverage score sampling (rTT-ALS) is proposed for Tensor Train decomposition of high-dimensional tensors.
Efficient Lifelong Model Evaluation in an Era of Rapid Progress
Ameya Prabhu (University of TΓΌbingen), Samuel Albanie (University of Cambridge)
CodeComputational EfficiencyImage
π― What it does: A framework called Sort & Search is proposed, which utilizes the evaluation results of existing models for sample sorting and subset sampling, achieving efficiency in lifelong model evaluation.
Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization
Kai Hu (Carnegie Mellon University), Matt Fredrikson (Carnegie Mellon University)
CodeOptimizationAdversarial AttackTransformerLarge Language ModelText
π― What it does: This paper proposes a new token-level attack method called Adaptive Dense-to-Sparse Constrained Optimization (ADC) to break the security defenses of large language models.
Yichao Fu (University of California San Diego), Hao Zhang
CodeGenerationData SynthesisOptimizationComputational EfficiencyTransformerLarge Language ModelReinforcement LearningText
π― What it does: Construct a lightweight generative length ranking predictor and combine it with the vLLM service to dynamically schedule LLM requests based on learned ranking information, significantly reducing HOL blocking, improving chat latency, and increasing synthetic data generation throughput.
Felipe Maia Polo (University of Michigan), Mikhail Yurochkin (IBM Research)
CodeLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Proposes the PromptEval method, which achieves efficient estimation of LLM performance distribution under a large number of prompt templates.
Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters
Yifei Xia (Peking University), Bin CUI
CodeTransformerLarge Language ModelText
π― What it does: This paper proposes LoRA-Inlaid, a service system for large language models (LLMs) that supports multi-tasking, quantization, and the dynamic addition of LoRA adapters.
Efficient Prompt Optimization Through the Lens of Best Arm Identification
Chengshuai Shi (University of Virginia), Cong Shen (University of Virginia)
CodeOptimizationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Automatically optimize prompt words for large language models under a limited budget, proposing the TRIPLE framework that can be directly applied to prompt selection.
π― What it does: A recursive offline reinforcement learning algorithm RESeL is proposed, which uses a context encoder-specific learning rate to enhance the training stability and performance of RNNs in POMDP tasks.
π― What it does: This paper presents BaFormer, a boundary-aware query network based on Transformer, which transforms dense frame-by-frame action segmentation into sparse query classification, achieving efficient temporal action segmentation.
EfficientCAPER: An End-to-End Framework for Fast and Robust Category-Level Articulated Object Pose Estimation
Xinyi Yu (Zhejiang University of Technology), Liu Liu (Hefei University of Technology)
CodePose EstimationPoint Cloud
π― What it does: An end-to-end EfficientCAPER framework has been developed for 6D joint pose estimation of category-level articulated objects, eliminating post-optimization and solving steps.
EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization
Dong HUANG, Jie Zhang
CodeOptimizationComputational EfficiencyAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: The EFFI-LEARNER framework is proposed, which utilizes execution time and memory usage profiles for self-optimization after generating code with LLM, thereby improving code efficiency.
π― What it does: This study investigates rigid body dynamics modeling and proposes the Event-attend Graph ODE (EGODE) framework to jointly simulate rigid body collisions and evolution using continuous ODE and event modules.
π― What it does: This paper proposes einspace, a neural architecture search space based on parameterized probabilistic context-free grammar (PCFG), capable of representing various networks from ResNet, ViT, MLP-Mixer to custom mixed structures;
ElasTST: Towards Robust Varied-Horizon Forecasting with Elastic Time-Series Transformer
Jiawen Zhang (Hong Kong University of Science and Technology), Jia Li (Hong Kong University of Science and Technology)
CodeTransformerTime Series
π― What it does: A non-autoregressive time series Transformer model, ElasTST, has been designed and implemented to maintain robustness under different prediction durations.
π― What it does: An elliptical attention mechanism based on Mahalanobis distance is proposed, improving the traditional self-attention spherical kernel to an elliptical kernel, enhancing representation diversity and robustness.
π― What it does: This paper explores the design space of dataset distillation and proposes the EDC framework to efficiently generate small synthetic datasets.
Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning
Yiming Wang (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
CodeGenerationAnomaly DetectionTransformerLarge Language ModelText
π― What it does: This study proposes an OOD detection method based on embedded trajectory fluctuationsβTV Scoreβspecifically designed for generative language models in mathematical reasoning scenarios.
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
Zebang Cheng (Shenzhen Technology University), Alexander G Hauptmann
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningVideoTextMultimodalityAudio
π― What it does: Designed and trained Emotion-LLaMA to achieve emotion recognition and reasoning through multi-modal inputs of audio, visual, and text.
π― What it does: This paper proposes an active learning framework for 3D molecular graphs, utilizing diversity and uncertainty assessment to achieve sample selection through quadratic programming, combined with new geometric isometric transformations and Bayesian graph neural networks for 3D molecular property prediction.
Empowering Visible-Infrared Person Re-Identification with Large Foundation Models
Zhangyi Hu (Wuhan University), Mang Ye (Wuhan University)
CodeRecognitionRetrievalConvolutional Neural NetworkLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: A visible-infrared person re-identification framework based on large foundational models is proposed, utilizing generative visual language models and large language models to automatically generate text descriptions, enriching infrared modality information and enhancing cross-modal retrieval performance.
EMR-Merging: Tuning-Free High-Performance Model Merging
Chenyu Huang (Fudan University), Wanli Ouyang (Shanghai AI Laboratory)
CodeTransformerTextMultimodality
π― What it does: A new model merging method called EMR-MERGING is proposed, aimed at merging models from different tasks by selecting a unified model and a lightweight task-specific regulator, without the need for additional tuning or data.
π― What it does: This paper proposes an efficient fine-tuning pipeline EMVP for Visual Place Recognition (VPR) based on a Visual Foundation Model (VFM), incorporating a Centroid-Free Probing (CFP) and Dynamic Power Normalization (DPN) module, achieving parameter-efficient and high-performance fine-tuning.
π― What it does: DIVA is proposed, a semi-supervised environment design method based on Quality Diversity (QD), used to generate diverse training tasks in an initially unstructured simulator, thereby training adaptable Meta-RL agents.
π― What it does: This paper studies the internal mechanisms of Token-based Non-Autoregressive Transformers (NAT) for image synthesis and proposes the EfficientNAT (ENAT) model, which significantly improves generation quality and efficiency through spatial decoupling and temporal feature reuse.
π― What it does: An end-to-end learnable clustering intent learning framework ELCRec is proposed to enhance the joint optimization of user intent modeling and behavior representation in recommendation systems.
End-to-End Ontology Learning with Large Language Models
Andy Lo (University of Cambridge), Mateja Jamnik (University of Cambridge)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextGraph
π― What it does: This paper proposes an end-to-end framework called OLLM for constructing ontologies using large language models (LLM), which directly generates ontology subgraphs from documents and aggregates them to form a complete ontology.
π― What it does: An energy-based model (EBM) training method is proposed in discrete and mixed state spaces that does not require MCMC, using energy discrepancy loss and constructing perturbations through discrete heat equations.
π― What it does: An energy-guided continuous entropy regularization OT centroid estimation algorithm is proposed, which can handle arbitrary OT costs and supports constraining centroids on image manifolds generated by pre-trained generative models.
π― What it does: This paper presents AlphaGateau, a reinforcement learning framework based on graph neural networks for playing international chess, which replaces the CNN architecture of AlphaZero and introduces a new GATEAU layer.
Enhancing Diversity in Bayesian Deep Learning via Hyperspherical Energy Minimization of CKA
David Smerkous (Oregon State University), Li Fuxin (Oregon State University)
CodeAnomaly DetectionImage
π― What it does: This paper constructs a new regularization objective by combining centralized kernel alignment (CKA) with spherical energy (HE) at the network function layer to enhance model diversity in Bayesian deep learning, thereby improving uncertainty estimation and anomaly detection performance.
Enhancing Domain Adaptation through Prompt Gradient Alignment
Hoang Phan (New York University), Trung Le (Monash University)
CodeDomain AdaptationPrompt EngineeringImage
π― What it does: A prompt learning-based unsupervised domain adaptation framework is proposed, which unifies the losses of the source and target domains into different objectives through multi-objective optimization, and optimizes shared prompts via gradient alignment and gradient norm regularization to enhance cross-domain generalization ability.
π― What it does: A safe reinforcement learning algorithm ESPO based on dynamic adjustment of sampling size with gradient conflict is proposed, significantly improving sample efficiency while maintaining safety constraints.
π― What it does: A multi-channel visual Transformer named DiChaViT is proposed, aimed at enhancing feature diversity and robustness, particularly for multi-channel image data;
π― What it does: A Hierarchical Distance Structure Encoding (HDSE) is proposed and seamlessly integrated into the attention mechanism of existing graph Transformer models to enhance the modeling capability of multi-level hierarchies and long-range dependencies in graphs.
Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective
Xinhao Yao (Renmin University of China), Yong Liu (Renmin University of China)
CodeTransformerLarge Language ModelText
π― What it does: This study investigates the impact of SVD-based weight pruning on the performance of large language models (LLMs) in in-context learning (ICL) and proposes theoretical explanations and practical algorithms.
Kai Liu (Zhejiang University), Jieping Ye (Alibaba Cloud)
CodeKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Proposes to enhance the cognitive and reasoning abilities of large language models by converting long texts into a three-layer ScopeβAspectβDescription structured representation.
π― What it does: A completely label-free, training-free, and hyperparameter-free framework called Frolic is proposed to enhance the performance of zero-shot visual models like CLIP.
Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration
Yichong Huang (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeTransformerLarge Language ModelTextMultimodalityBenchmark
π― What it does: An untrained multimodal large language model ensemble framework called DEEPEN is proposed, which aligns the probability distributions of different vocabularies using relative representations and fuses them, mapping the aggregated results back to the main model space through inverse transformation search.
π― What it does: A diffusion policy based on entropy regularization is proposed in offline reinforcement learning, incorporating Q-ensemble to enhance the handling of uncertainty and exploration capabilities with offline data.
π― What it does: In molecular conformation generation, a hierarchical equivariant fuzzy diffusion (EBD) model is proposed, which first generates a coarse structure at the fragment level from the molecular graph and then refines it into a complete atomic-level conformation through an equivariant network.
π― What it does: An efficient U-Net based on E(3)ΓSO(3) equivariant convolution has been developed for diffusion MRI fiber deconvolution, significantly reducing computation time and memory consumption.
π― What it does: A single model framework (HyperDM) is proposed that utilizes Bayesian hypernetworks and diffusion models to estimate and separate the epistemic uncertainty and aleatoric uncertainty in machine learning predictions.
Estimating the Hallucination Rate of Generative AI
Andrew Jesson (Columbia University), David Blei (Columbia University)
CodeGenerationLarge Language ModelText
π― What it does: A method for estimating the posterior hallucination rate from a Bayesian perspective is proposed to assess the probability of hallucinations occurring in generative AI during contextual learning.
π― What it does: A generative model based on equivariant flow matching is proposed for generating low-energy three-dimensional conformations of given molecular graphs.
π― What it does: This study investigates whether generative models can implicitly learn and accurately recover the world model of the domain they are trained on (within the framework of deterministic finite automata), proposing new evaluation metrics and conducting experiments on maps, board games, and logic puzzles.
Hamed Shirzad (University of British Columbia), Danica J. Sutherland (University of British Columbia)
CodeGraph Neural NetworkTransformerGraph
π― What it does: A two-stage training method is proposed: first, a narrow network is trained to estimate attention scores, and then based on these scores, the graph is hierarchically sparsified and a wide network is trained, thereby improving the memory efficiency of the Graph Transformer.
π― What it does: This paper proposes Event-3DGS, a method that directly processes event camera data for 3D reconstruction and novel view synthesis using 3D Gaussian splatting (3DGS).
Everyday Object Meets Vision-and-Language Navigation Agent via Backdoor
Keji He (Shandong University), Liang Wang (Pattern Recognition Institute of Automation, Chinese Academy of Sciences)
CodeAdversarial AttackTransformerReinforcement LearningVision Language ModelImageTextMultimodality
π― What it does: A visual language navigation (VLN) backdoor attack method based on real object triggers is proposed, which can operate in normal environments but automatically stops execution when encountering specific object triggers.
EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models
Rui Zhao (National University of Singapore), Mike Zheng Shou
CodeGenerationData SynthesisTransformerVision Language ModelDiffusion modelImageTextMultimodality
π― What it does: Utilizing large-scale visual language models (VLM) as 'directors', we collect data generated by advanced text-to-image models through public APIs to construct a dynamically iterable training set, training the baseline diffusion Transformer (DiT) model to approximate or even surpass the generative capabilities of various advanced text-to-image models.
Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor Generalization
Davide Buffelli (MediaTek Research), Alberto Bernacchia (MediaTek Research)
CodeOptimizationTabular
π― What it does: This paper studies and implements a computable Gauss-Newton (GN) optimization in reversible deep networks (RevMLP), systematically evaluating the training speed and generalization performance of GN for the first time on large-scale practical networks.
Hyun-Young Park (Korea Advanced Institute of Science and Technology), Si-Hyeon Lee (Korea Advanced Institute of Science and Technology)
CodeOptimizationSafty and Privacy
π― What it does: This paper studies the problem of private sampling under local differential privacy, defines the privacy-utility trade-off (PUT), and proposes an optimal private sampling mechanism in the minimax sense.
Jia-Wei Liu (Show Lab), Mike Zheng Shou (Show Lab)
CodeGenerationData SynthesisDiffusion modelVideo
π― What it does: Using sparse appearance videos captured by four 360Β° appearance cameras to generate first-person perspective videos of the same scene and action.
Exogenous Matching: Learning Good Proposals for Tractable Counterfactual Estimation
Yikang Chen (Shanghai Key Laboratory of Trustworthy Computing, East China Normal University), Lili Tian (Shanghai Key Laboratory of Trustworthy Computing, East China Normal University)
CodeFlow-based ModelTabular
π― What it does: A method based on importance sampling called Exogenous Matching is proposed for estimating counterfactual expressions that can be solved in general settings.
Shufan Shen (Institute of Computing Technology, Chinese Academy of Sciences), Shuhui Wang (Institute of Computing Technology, Chinese Academy of Sciences)
π― What it does: A low-memory sparse fine-tuning framework called SNELL is proposed, which combines the low-rank decomposition of LoRA with kernelization and employs a competitive sparse mechanism to achieve efficient parameter fine-tuning on large-scale pre-trained visual models.
Expectation Alignment: Handling Reward Misspecification in the Presence of Expectation Mismatch
Malek Mechergui (Colorado State University), Sarath Sreedharan (Colorado State University)
CodeOptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabular
π― What it does: The Expectation Alignment framework is proposed to represent and address the reward function mis-specification problem through human theoretical attitudes towards agents.
Explaining Datasets in Words: Statistical Models with Natural Language Parameters
Ruiqi Zhong (University of California Berkeley), Jacob Steinhardt (University of California Berkeley)
CodeClassificationOptimizationExplainability and InterpretabilityLarge Language ModelTextTime Series
π― What it does: A statistical model framework using natural language predicates as model parameters has been designed, and a general optimization algorithm has been proposed to learn these predicates through continuous relaxation and iterative refinement, applied to tasks such as clustering, time series, and multi-label classification.
π― What it does: The paper studies the training dynamics of SAM, proposes a third-order continuous-time model (SDE), reveals the importance of the alignment between the perturbation vector and the maximum eigenvector of the Hessian for sharpness regularization, and based on this, proposes the Eigen-SAM algorithm, which explicitly regularizes the maximum eigenvalue.
Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion
Filip Szatkowski (Warsaw University of Technology), Simone Scardapane (Sapienza University of Rome)
CodeComputational EfficiencyTransformerMixture of ExpertsImageText
π― What it does: Transforming dense Transformer models into dynamic Mixture-of-Experts (MoE) to enhance inference efficiency through activation sparsity;
Kazuki Egashira (ETH Zurich), Martin Vechev (ETH Zurich)
CodeSafty and PrivacyAdversarial AttackAI Code AssistantTransformerLarge Language ModelText
π― What it does: This paper studies the potential security vulnerabilities of large language models (LLMs) during the zero-copy quantization process, proposing an attack framework that utilizes quantization differences to mask malicious behavior in full-precision models and activate malicious behavior after quantization. It employs projected gradient descent (PGD) for constrained training of the full-precision model, causing it to exhibit malicious outputs after quantization.
Exploiting Representation Curvature for Boundary Detection in Time Series
Yooju Shin (Korea Advanced Institute of Science and Technology), Jae-Gil Lee (Korea Advanced Institute of Science and Technology)
CodeAnomaly DetectionRepresentation LearningContrastive LearningMultimodalityTime Series
π― What it does: This paper proposes a time series boundary detection method based on trajectory curvature representation called RECURVE. It uses a representation sequence generated by self-supervised representation learning as a curve, calculates the curvature at each time point, and identifies class boundaries based on small curvature values.
π― What it does: An end-to-end framework based on behavioral information, Variational Autoencoder (VAE), and Video Diffusion Model (VDM) is proposed (BeNeDiff) to learn decoupled neural latent subspaces from head-fixed mouse whole-brain calcium imaging data and to explain the behavioral dynamics corresponding to each latent factor by generating behavioral videos.
π― What it does: Proposes the idea of maintaining consistency in graph representation similarity between layers of Graph Neural Networks (GNNs) and introduces a consistency loss;
Exploring Jacobian Inexactness in Second-Order Methods for Variational Inequalities: Lower Bounds, Optimal Algorithms and Quasi-Newton Approximations
Artem Agafonov (Mohamed bin Zayed University of Artificial Intelligence), Dmitry Kamzolov (Mohamed bin Zayed University of Artificial Intelligence)
CodeOptimization
π― What it does: This paper proposes a second-order method for variational inequalities that can achieve global convergence even in the presence of inaccuracies in the Jacobian matrix.
Xiaohong Ji (DP Technology), Weinan E (Peking University)
CodeDrug DiscoveryTransformerGraph
π― What it does: A scalable molecular pre-training model, Uni-Mol2, is proposed and trained, utilizing a dual-track Transformer to jointly model atomic, graph, and geometric information, and is pre-trained on a large scale with 884M 3D molecular structures.
π― What it does: The Ex-MCR method is proposed, utilizing pre-trained unimodal spaces to achieve multimodal contrastive learning with unpaired data through overlapping modalities, extending to unified representations beyond three modalities.
π― What it does: This paper studies attack methods for extracting training data from molecular pre-training models and successfully implements effective data leakage attacks on non-generative molecular pre-training models for the first time.
Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
Chong Ma (ShanghaiTech University), Xiang Li (Massachusetts General Hospital)
CodeClassificationRetrievalRepresentation LearningTransformerContrastive LearningImageTextMultimodalityBiomedical Data
π― What it does: This paper proposes a framework for guiding the multimodal alignment of medical images and text using radiologists' gaze tracking data (EGMA), achieving more precise feature alignment during the pre-training phase through fine-grained alignment loss (EGF) and cross-modal mapping loss (EGM).
EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detection
Qinqian Lei (National University of Singapore), Robby T. Tan (ASUS Intelligent Cloud Services)
CodeObject DetectionTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodality
π― What it does: An efficient zero-shot human-object interaction detection framework EZ-HOI is proposed, which utilizes prompt learning for adaptive handling of unseen interaction classes in visual-language models.
π― What it does: A forward online parsing learning (F-OAL) method is proposed, which combines a frozen pre-trained encoder with feature fusion and smooth projection, and updates the linear classifier through recursive least squares, achieving online class incremental learning without saving samples.
π― What it does: This paper proposes a dynamic integration of unsupervised contrastive learning and supervised multimodal classification learning, utilizing the intervention of positive label fitting from contrastive learning to alleviate the issue of modality imbalance.
π― What it does: A 3D-CNN model called FactorizePhys and a multi-dimensional attention module based on non-negative matrix factorization (NMF) called FSAM have been developed for end-to-end extraction of remote photoplethysmography (rPPG) signals from video frames.
Fair and Welfare-Efficient Constrained Multi-Matchings under Uncertainty
Elita Lobo (University of Massachusetts Amherst), Yair Zick (University of Massachusetts Amherst)
CodeOptimizationTabular
π― What it does: This study investigates the multi-matching problem that balances welfare efficiency and group fairness under resource allocation uncertainty.
Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium
Mehdi Yazdani-Jahromi (University of Central Florida), Ozlem Garibay
CodeOptimizationTabularElectronic Health Records
π― What it does: Through a bi-level (Stackelberg) optimization framework, a neural network is constructed to achieve a Pareto optimal balance between accuracy and fairness (taking demographic fairness as an example);