International Conference on Learning Representations Β· 1682 papers
Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models
Hulingxiao He (Peking University), Yuxin Peng (Peking University)
CodeRecognitionTransformerLarge Language ModelContrastive LearningImageMultimodality
π― What it does: This paper proposes a multimodal large language model named Finedefics, specifically improved for fine-grained visual recognition (FGVR);
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
Christopher Rawles (Google DeepMind), Oriana Riva (Google)
CodeRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningAgentic AIMultimodalityBenchmark
π― What it does: Created AndroidWorld, a reproducible and dynamically parameterizable automated agent evaluation environment based on Android, providing 116 tasks covering 20 real Android applications;
Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions
Sarah Wiegreffe (Allen Institute for AI), Ashish Sabharwal (Allen Institute for AI)
CodeTransformerLarge Language ModelText
π― What it does: This study investigates the answering mechanism of Transformer language models in formatted multiple-choice questions (MCQA), identifying the causal roles of key layers and attention heads, and revealing the timing of the model's learning of formatted answering through synthetic tasks.
π― What it does: Proposes an Arbitrary Step Dynamic Model (ADM) and uses this model in online (ADMPO-ON) and offline (ADMPO-OFF) reinforcement learning frameworks, significantly reducing multi-step cumulative errors.
π― What it does: This paper studies the application of parallel encoding in Context-Augmented Generation (CAG) and proposes the Adaptive Parallel Encoding (APE) method, which enables parallel encoding to align attention distribution, thereby maintaining performance close to that of sequential encoding.
API Pack: A Massive Multi-Programming Language Dataset for API Call Generation
Zhen Guo (Massachusetts Institute of Technology), Rameswar Panda (IBM)
CodeGenerationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
π― What it does: This paper creates a large-scale multilingual API call instruction-code pair dataset named API Pack (over 1.1 million instances) and fine-tunes various LLMs using this dataset, significantly improving performance in generating new API calls.
Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding
Eric Lei (University of Pennsylvania), Shirin Saeedi Bidokhti (University of Pennsylvania)
CodeCompressionConvolutional Neural NetworkFlow-based ModelImagePhysics Related
π― What it does: A nonlinear transform coding framework based on lattice quantization (Lattice Transform Coding, LTC) is proposed, addressing the suboptimal issues caused by scalar quantization in traditional neural compression.
ARB-LLM: Alternating Refined Binarizations for Large Language Models
Zhiteng Li (Shanghai Jiao Tong University), Xiaokang Yang (Shanghai Jiao Tong University)
CodeOptimizationTransformerLarge Language ModelText
π― What it does: This paper proposes a binary post-training quantization method for large language models called ARB-LLM, which utilizes Alternating Refinement Binarization (ARB), Calibration Data Augmentation (ARB-X), and Row-Column Bidirectional Scaling (ARB-RC), and improves weight grouping and Column Group Bitmap (CGB) to enhance quantization performance.
Are Large Vision Language Models Good Game Players?
Xinyu Wang (University of Adelaide), Qi Wu (University of Adelaide)
CodeTransformerVision Language ModelMultimodalityBenchmark
π― What it does: This paper presents LVLMβPlaygroundβa game-based evaluation framework that comprehensively assesses the cognitive and reasoning abilities of visual language models through six types of games (chess, Sudoku, Minesweeper) across four tasks (perception, question answering, rule following, end-to-end gameplay).
π― What it does: Designed the FTCT (Fragmented at Training, Chained at Testing) synthetic dataset, and trained a model using Transformer to verify its ability to combine fragmented knowledge pieces from training into a complete reasoning chain during testing.
Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling
Louis Bradshaw (Queen Mary University of London), Simon Colton (Queen Mary University of London)
CodeClassificationSegmentationGenerationConvolutional Neural NetworkLarge Language ModelVideoAudio
π― What it does: The Aria-MIDI dataset was constructed using web crawling + language models, audio classifiers, and automated source separation + AMT processes, containing over 1 million piano MIDI files and approximately 100,000 hours of high-quality transcriptions.
Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics
Yaniv Nikankin (Technion - Israel Institute of Technology), Yonatan Belinkov (Technion - Israel Institute of Technology)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This study investigates how large language models work in arithmetic reasoning tasks, revealing that they do not learn complete algorithms or merely memorize answers, but instead achieve arithmetic calculations through a set of sparse MLP neurons that collaboratively implement multiple heuristics.
π― What it does: This paper proposes and implements Artificial Kuramoto Oscillating Neurons (AKOrN), embedding multidimensional vectorized Kuramoto synchronization dynamics into conventional network layers such as convolution and attention, to construct an iterative dynamic network.
As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss
Xin Mao (Nanyang Technological University), Anh Tuan Luu (Nanyang Technological University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a new loss function for value alignment of large language modelsβBidirectional Negative Feedback (BNF) loss, aimed at addressing the hyperparameter sensitivity and instability issues of traditional DPO series methods.
π― What it does: An Asymmetric Factorized Bilinear Operation (AFBO) is proposed to replace the Feed-Forward Network (FFN) in Vision Transformers (ViT), achieving a better performance-complexity trade-off through second-order statistical spatial-channel factorized bilinear operations (SCFBO) and structurally sparse channel mapping.
π― What it does: This study investigates the training and generalization performance of a two-layer neural network on Gaussian mixture data after one step of gradient descent in high-dimensional high-ratio limits.
π― What it does: An end-to-end molecular-text cross-modal representation learning framework called Atomas is proposed, utilizing hierarchical adaptive alignment between molecular SMILES and text descriptions to achieve molecular retrieval, property prediction, and molecular generation tasks.
Simon Schug (ETH Zurich), Razvan Pascanu (Google DeepMind)
CodeTransformerText
π― What it does: Reformulating multi-head attention as a hypernetwork reveals that low-dimensional latent codes are used to configure key-query specific linear value networks.
Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers
Shijie Chen (Ohio State University), Yu Su (Ohio State University)
CodeRetrievalExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: A non-generative zero-shot re-ranking method called ICR based on the attention distribution of large language models (LLM) is proposed, which aggregates and calibrates the attention changes of query words on document words to directly obtain document relevance scores.
π― What it does: This study investigates the learning capability of a single-layer Transformer under first-order Markov chain inputs, constructs a theoretical framework, and analyzes its loss landscape, proving the existence of global and local optima.
Attribute-based Visual Reprogramming for Vision-Language Models
Chengyi Cai (University of Melbourne), Feng Liu (University of Melbourne)
CodeClassificationRecognitionTransformerLarge Language ModelVision Language ModelImageMultimodality
π― What it does: Attribute-driven visual reprogramming (AttrVR) is proposed on visual-language models like CLIP, achieving few-shot classification by adding trainable noise to the input image and optimizing guided by descriptive attributes (DesAttrs) and distinctive attributes (DistAttrs) generated by LLM.
Attributing Culture-Conditioned Generations to Pretraining Corpora
Huihan Li (University of Southern California), Xiang Ren (University of Southern California)
CodeGenerationTransformerLarge Language ModelText
π― What it does: Analyzes the cultural bias in the generation of large language models under cultural conditions and proposes the MEMOED framework to attribute whether generated symbols come from the memory of pre-trained data.
Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generation
Tobias Leemann (University of TΓΌbingen), Sergul Aydore
CodeRetrievalDomain AdaptationTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: The Auto-GDA framework is proposed, which automatically adapts NLI models through unsupervised synthetic data generation and selection, thereby improving the grounding verification accuracy of retrieval-augmented generation (RAG) systems.
AutoBencher: Towards Declarative Benchmark Construction
Xiang Lisa Li (Stanford University), Tatsunori Hashimoto (Stanford University)
CodeOptimizationSafty and PrivacyData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Proposes the AutoBencher framework, which uses LLM to automatically generate and optimize evaluation datasets, achieving declarative benchmark construction.
π― What it does: Develop a closed-loop concept-guided strategy that automatically learns manipulation concepts from unlabeled demonstrations and uses concept-guided diffusion strategies to complete complex robotic tasks.
AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs
Xiaogeng Liu (University of Wisconsin-Madison), Chaowei Xiao (University of Wisconsin-Madison)
CodeAdversarial AttackTransformerLarge Language ModelAgentic AIText
π― What it does: AutoDAN-Turbo implements black-box jailbreak attacks through lifelong learning agents, capable of automatically discovering, combining, and utilizing various jailbreak strategies from scratch, and supports seamless injection of human-designed strategies.
Shengran Hu (University of British Columbia), Jeff Clune (University of British Columbia)
CodeOptimizationExplainability and InterpretabilityTransformerLarge Language ModelAgentic AITextMultimodalityChain-of-Thought
π― What it does: This paper proposes a framework for an Automated Design Agent System (ADAS) and implements the Meta Agent Search algorithm, utilizing large language models to generate and optimize new agents in the code space, gradually building agent profiles.
Automatic Curriculum Expert Iteration for Reliable LLM Reasoning
Zirui Zhao (National University of Singapore), Doyen Sahoo (Salesforce AI Research)
CodeLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: This paper proposes an Automated Course Expert Iteration method (AUTO-CEI) aimed at enhancing the reliability of large language models (LLMs) in multi-step reasoning tasks, reducing hallucinations and lazy behaviors.
Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks
Rushang Karia (Arizona State University), Siddharth Srivastava (Arizona State University)
CodeLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: Proposes the AutoEval framework, which utilizes grammar generation and formal equivalence verification to achieve human-free evaluation of LLM in truth maintenance and reasoning tasks.
π― What it does: This paper proposes a training method for visual models based on autoregressive pre-trainingβARMβand applies it to the Mamba architecture, significantly enhancing Mamba's performance and scalability in large-scale visual tasks.
π― What it does: A non-quantized autoregressive video and image generation model named NOVA is proposed, capable of performing various tasks such as text-to-image, text-to-video, and image-to-video within the same framework.
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models
Kim Sung-Bin (POSTECH), Tae-Hyun Oh (KAIST)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoMultimodalityBenchmarkAudio
π― What it does: This paper proposes AVHBench, a benchmark specifically designed to evaluate the cross-modal hallucinations of audio-visual large language models (AV-LLM). It constructs a dataset of 2,136 videos containing 5,302 question-answer pairs and 1,106 multimodal descriptions through a semi-automated annotation process.
π― What it does: The B-STAR framework is proposed to automatically monitor and balance the two main factors of exploration (diversity) and exploitation (reward discrimination) during the self-learning process, thereby enhancing the performance of self-improving models.
CodeGenerationKnowledge DistillationAdversarial AttackTransformerVision Language ModelImageTextMultimodality
π― What it does: Conducting backdoor attacks on visual-language models (VLM) by utilizing only out-of-domain (OOD) data available during training to attack image-to-text generation tasks.
π― What it does: A new attack method named Bad-PFL is designed for backdoor attacks in personalized federated learning (PFL), which implants backdoors using natural feature triggers during client training and dynamically generates triggers through a generator to ensure their persistent presence in personalized models.
BadJudge: Backdoor Vulnerabilities of LLM-As-A-Judge
Terry Tong (University of California), Muhao Chen (University of California)
CodeAdversarial AttackTransformerLarge Language ModelText
π― What it does: The study demonstrates a backdoor attack mechanism targeting the LLM-as-a-Judge automatic evaluation system, proving that even contaminating just 1% of the training data can significantly enhance the evaluation scores of the attacker's model.
π― What it does: A fast proxy model with adjustable complexity is constructed by combining VAE and Neural ODE, achieving model reduction without prior dimensionality and approximating the Koopman operator;
BANGS: Game-theoretic Node Selection for Graph Self-Training
Fangxin Wang (University of Illinois Chicago), Philip S. Yu (University of Illinois Chicago)
CodeGraph Neural NetworkGraph
π― What it does: Proposes the BANGS framework, which selects pseudo-label nodes in a combinatorial manner during the graph self-training process to enhance the performance of GNNs.
Bayesian Image Regression with Soft-thresholded Conditional Autoregressive Prior
Yuliang Xu (Duke University), Jian Kang (University of Michigan)
CodeBiomedical DataMagnetic Resonance Imaging
π― What it does: This paper addresses the high-dimensional sparse regression problem of functional magnetic resonance imaging (fMRI) data by proposing a new prior called the Soft-Thresholded Conditional Auto-Regressive (ST-CAR) prior, which aims to achieve spatial smoothing, induce sparsity, and enable adaptive learning without relying on predefined correlation structures.
Chukwudi Paul Obite (Arizona State University), Shiwei Lan (Arizona State University)
CodeRepresentation LearningTabular
π― What it does: A novel latent variable model QEP-LVM based on the Q-index process is proposed for learning latent representations, aimed at improving the visualization of data structures and the efficiency of model building.
π― What it does: Bayesian inference is performed on a set of weak models to enhance the transfer effect from weak to strong models, and this framework is extended from text classification to text generation; after training the strong model, conservative DPO is further used to improve preference learning.
Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms
Parham Rezaei (Sharif University of Technology), Cheuk Ting Li (Chinese University of Hong Kong)
CodeGenerationData SynthesisOptimizationMixture of ExpertsImage
π― What it does: This study proposes a new mixed generative model selection mechanism aimed at improving the diversity and quality of generated data by mixing multiple generative models.
BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts
Divya Jyoti Bajpai (Indian Institute of Technology Bombay), Manjesh Kumar Hanawal (Indian Institute of Technology Bombay)
CodeComputational EfficiencyKnowledge DistillationTransformerMixture of ExpertsImageText
π― What it does: The BEEM framework is proposed, treating multi-exit classifiers as experts, utilizing weighted confidence and prediction consistency to achieve early exit, thereby enhancing the inference speed and accuracy of Transformer models.
Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning
Wesley A. Suttle (U.S. Army Research Laboratory), Carlos Nieto-Granda (U.S. Army Research Laboratory)
CodeData SynthesisReinforcement Learning
π― What it does: A method for generating offline reinforcement learning datasets based on behavioral entropy is proposed, aiming to systematically generate diverse datasets that cover complex, high-dimensional state spaces.
CodeGenerationOptimizationComputational EfficiencyTransformerLarge Language ModelAgentic AIPrompt EngineeringTextGraphBenchmark
π― What it does: This paper presents WORFBENCHβa unified workflow generation benchmark that covers four types of task scenarios (problem solving, function calling, embodied planning, and open-ended planning) and models complex workflows using directed acyclic graphs (DAGs). It also designs the WORFEVAL evaluation protocol, which utilizes subsequence/subgraph matching algorithms to accurately quantify the generated node chains against the workflow graphs. A systematic evaluation of 18 large language models (including both closed and open-source models) is conducted, exploring the impact of workflows on downstream task performance and inference time, along with a quantitative analysis of the bottlenecks in graph structure generation.
Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent
Yangning Li (Tsinghua University), Philip S. Yu (Alibaba Group)
CodeGenerationRetrievalTransformerLarge Language ModelMultimodalityBenchmarkRetrieval-Augmented Generation
π― What it does: This paper studies the challenges of multimodal retrieval-augmented generation (mRAG) in dynamic visual question answering (VQA) and proposes an adaptive retrieval planning agent called OmniSearch.
π― What it does: Developed and released a JAX-based PCX library for efficient implementation of predictive coding networks, providing a unified benchmark task and dataset for large-scale experiments.
Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset
Yingzi Ma (University of Wisconsin Madison), Chaowei Xiao (University of Wisconsin Madison)
CodeSafty and PrivacyTransformerSupervised Fine-TuningVision Language ModelImageTextBenchmark
π― What it does: This paper proposes the FIUBENCH benchmark, constructs a fictional facial identity VQA dataset, and designs a two-stage learning-forgetting evaluation pipeline to assess the performance of Vision-Language Models on the 'forgetting' task.
BenTo: Benchmark Reduction with In-Context Transferability
Hongyu Zhao (University of Maryland), Tianyi Zhou (Lehigh University)
CodeOptimizationLarge Language ModelTextBenchmark
π― What it does: This paper studies the estimation of task transferability using the untrained In-Context Transferability (ICT) metric, and based on this, constructs the Facility Location problem to implement BENTO, reducing the number of benchmark tasks for LLM evaluation to within 5% while keeping the error below 4%.
Beware of Calibration Data for Pruning Large Language Models
Yixin Ji (Soochow University), Min Zhang (Soochow University)
CodeTransformerLarge Language ModelText
π― What it does: This paper studies the importance of calibration data in post-training pruning and proposes a method for self-generating synthetic calibration data (Self-Generating Synthetic Calibration Data, Syn) based on LLM to replace the original training data, thereby enhancing the downstream performance of the pruned model.
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
Jiacheng Ye (University of Hong Kong), Lingpeng Kong (University of Hong Kong)
CodeDiffusion modelText
π― What it does: This paper studies the shortcomings of autoregressive language models in complex reasoning and planning tasks, and proposes a multi-granularity diffusion model (MGDM) based on discrete diffusion to address these issues.
Beyond Circuit Connections: A Non-Message Passing Graph Transformer Approach for Quantum Error Mitigation
Tianyi Bao (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeGraph Neural NetworkTransformerGraphPhysics Related
π― What it does: A non-message passing graph transformer GTranQEM is proposed to suppress errors in the measurement results of quantum circuits, modeling global quantum coupling through graph encoding, quantum-specific positional encoding, structural matrix attention bias, and virtual QCR nodes.
Beyond Content Relevance: Evaluating Instruction Following in Retrieval Models
Jianqun Zhou (Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology), Xiaoyu Shen (Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper constructs a new retrieval evaluation benchmark named InfoSearch, which is used to assess the instruction-following ability of retrieval models on six document-level attributes beyond content relevance (audience, keywords, format, language, length, source), and proposes two new evaluation metrics, SICR and WISE.
Beyond correlation: The impact of human uncertainty in measuring the effectiveness of automatic evaluation and LLM-as-a-judge
Aparna Elangovan (Amazon), Dan Roth (Amazon)
CodeLarge Language ModelTextBenchmark
π― What it does: This study investigates the correlation between automatic evaluation methods and human labels, revealing the impact of human uncertainty on evaluation metrics, and proposes three new methods: stratification by label uncertainty, binned Jensen-Shannon divergence, and perceptual visualization.
CodeTransformerLarge Language ModelPrompt EngineeringTextGraphBenchmarkChain-of-Thought
π― What it does: Designed and released the LLM4Hypergraph benchmark to evaluate the understanding and reasoning capabilities of large language models regarding hypergraph structures, and proposed two new prompting techniques: Hyper-BAG and Hyper-COT.
Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness
Qi Zhang (Peking University), Yisen Wang (Peking University)
CodeClassificationExplainability and InterpretabilityAuto EncoderContrastive LearningImageText
π― What it does: This study investigates the impact of monosemanticity on model robustness and demonstrates that monosemantic features can significantly enhance model robustness in various scenarios, such as input noise, label noise, and few-shot fine-tuning, challenging the common accuracy-interpretability trade-off.
Beyond Next Token Prediction: Patch-Level Training for Large Language Models
Chenze Shao (Tencent Inc), Jie Zhou (Tencent Inc)
CodeTransformerLarge Language ModelText
π― What it does: A patch-level training method is proposed, which significantly reduces the training cost of LLMs by aggregating multiple tokens into patches and predicting the next patch at the patch level.
π― What it does: A hard view selection mechanism is introduced in self-supervised pre-training, where multiple randomly augmented views are generated for each image, and the loss of all view pairs is calculated to select the pair with the highest loss for training;
π― What it does: This paper proposes the Disjunctive Refinement Layer (DRL), which can compile quantified first-order linear arithmetic (QFLRA) constraints into differentiable neural network layers, thereby enforcing the satisfaction of non-convex, discrete background knowledge constraints when generating tabular data.
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
Wenxuan Zhang (King Abdullah University of Science and Technology), Adel Bibi (University of Oxford)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: A novel supervised alignment framework BFPO is proposed to simultaneously enhance the usefulness and safety of large language models.
π― What it does: Proposes an action chunking improvement method based on bidirectional decoding, utilizing multiple sampling and forward-backward consistency evaluation to achieve closed-loop action planning.
π― What it does: BiGR is a model that utilizes binary latent codes for conditional image generation, while also achieving powerful representation learning capabilities within the same framework.
CodeExplainability and InterpretabilityTransformerAuto EncoderImageText
π― What it does: This paper proposes and validates the use of a bilinear MLP (Multilayer Perceptron) with non-linear activation as an interpretable alternative in Transformers, utilizing its third-order tensor structure for feature interaction analysis of weights.
Werner Zellinger (Johannes Kepler University Linz)
CodeDomain AdaptationOptimizationImageText
π― What it does: This paper constructs a general framework for binary classification loss functions by providing a given Bregman divergence, and based on this framework, proposes a novel loss function that prioritizes estimating large density ratios.
Biologically Constrained Barrel Cortex Model Integrates Whisker Inputs and Replicates Key Brain Network Dynamics
Tianfang Zhu (Wuhan National Laboratory for Optoelectronics Huazhong University of Science and Technology), Anan LI
CodeClassificationSpiking Neural NetworkReinforcement LearningTime Series
π― What it does: A biologically constrained rat dorsal whisker cortex model containing 13 types of neuron subtypes and 4218 neurons was constructed and trained. Based on spike input, object classification was achieved by converting physical simulation whisker sweep data into spike input.
π― What it does: A Transformer-based brain map learning model, BioBGT, has been designed and implemented to enhance the biological plausibility of brain map representations by encoding small-world structures (node importance and functional modules) and is used for brain disease detection tasks.
BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models
Yu Feng (University of Pennsylvania), Dan Roth (University of Pennsylvania)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Proposes the BIRD framework, which combines the inductive reasoning of LLMs with Bayesian networks to achieve reliable probability estimation under incomplete information;
π― What it does: Created BirdSet, a large-scale bird audio classification benchmark dataset, containing approximately 520k recordings, 6,800 hours of training, and 400 hours of testing, supporting multi-label and multi-scenario evaluations;
BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments
Xinghao Wang (Fudan University), Xipeng Qiu (Fudan University)
CodeCompressionTransformerLarge Language ModelText
π― What it does: A training-independent decomposition compression method named BitStack is proposed, enabling LLM to dynamically adjust model size at the megabyte level in variable memory environments.
BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillation
Zhengrui Guo (Hong Kong University of Science and Technology), Hao Chen (Hong Kong University of Science and Technology)
CodeKnowledge DistillationMultimodality
π― What it does: A neural population dynamics modeling framework called BLEND is proposed, which utilizes behavioral information as privileged knowledge for knowledge distillation.
BlendRL: A Framework for Merging Symbolic and Neural Policy Learning
Hikaru Shindo (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
CodeRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningImage
π― What it does: The BlendRL framework is proposed, which integrates neural networks with differentiable logical reasoning to form parallel neural and symbolic policies that are dynamically allocated through a hybrid module, ultimately achieving an RL agent that possesses both low-level reactions and high-level reasoning.
CodeGenerationTransformerLarge Language ModelDiffusion modelText
π― What it does: A Block Diffusion Language Model (BD3-LM) is proposed, which combines discrete diffusion and autoregression at the block level, supporting arbitrary length generation and utilizing KV caching for efficient inference.
π― What it does: Proposes the Block-Attention mechanism, which allows for chunking retrieval documents and calculating KV states separately, significantly reducing inference latency and FLOPs in RAG scenarios while maintaining performance close to full attention.
CodeAdversarial AttackTransformerLarge Language ModelReinforcement LearningVision Language ModelDiffusion modelImageTextMultimodality
π― What it does: A black-box blue team defense framework called BlueSuffix is proposed, which defends against cross-modal jailbreak attacks on Vision-Language Models through image denoising diffusion models, LLM text rewriting, and suffixes generated by reinforcement learning.
π― What it does: Using Boltzmann Prior to enhance the learning of Implicit Transfer Operator, guiding the generation of samples through pre-trained BG and embedding long-term dynamical priors, achieving efficient modeling of long time series in molecular dynamics.
Boltzmann Semantic Score: A Semantic Metric for Evaluating Large Vision Models Using Large Language Models
Ali Khajegili Mirabadi (University of British Columbia), Ali Bashashati (University of British Columbia)
CodeLarge Language ModelImageTextBiomedical Data
π― What it does: This paper proposes the Boltzmann Semantic Score (BSS) to measure the semantic capability of large visual models (LVMs) and evaluates the BSS of seven LVMs against five large language models (LLMs) on the TCGA dataset.
CodeProtein Structure PredictionSupervised Fine-TuningBiomedical Data
π― What it does: A Boltzmann alignment-based inverse folding model has been developed to predict the variation in protein-protein interaction ΞΞG, explicitly considering both bound and unbound states to enhance prediction accuracy.
π― What it does: BoneMet is proposed and released - the first large-scale, full high-resolution, serialized multimodal mouse bone metastasis imaging dataset, supporting the diagnosis, prognosis, and imaging processing research of B-cell bone metastasis.
Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation
Tiansheng Huang (Georgia Institute of Technology), Ling Liu (Georgia Institute of Technology)
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A defense method named Booster is proposed for the alignment phase, which mitigates harmful loss reduction by incorporating regularization against harmful perturbations during the alignment process, thereby enhancing the model's robustness against subsequent harmful fine-tuning attacks.
Boosting Methods for Interval-censored Data with Regression and Classification
Yuan Bian (University of Western Ontario), Wenqing He (University of Western Ontario)
CodeClassificationOptimizationSupervised Fine-TuningTabularBiomedical Data
π― What it does: Two non-parametric boosting methods based on L2 Boost (L2Boost-CUT and L2Boost-IMP) are proposed, specifically designed to address regression and classification problems with interval censored data.
Boosting Neural Combinatorial Optimization for Large-Scale Vehicle Routing Problems
Fu Luo (Southern University of Science and Technology), Qingfu Zhang (City University of Hong Kong)
CodeOptimizationTransformerGraph
π― What it does: A lightweight cross-attention Transformer and self-improving training (SIT) scheme is proposed for large-scale vehicle routing problems.
Boosting the visual interpretability of CLIP via adversarial fine-tuning
Shizhan Gong (Chinese University of Hong Kong), Farzan Farnia (Chinese University of Hong Kong)
CodeSegmentationExplainability and InterpretabilityTransformerGenerative Adversarial NetworkContrastive LearningImageMultimodality
π― What it does: For the image encoder of CLIP, an unsupervised adversarial fine-tuning (AFT) method is proposed, which enhances its visual interpretability by incorporating norm regularization, and makes the fine-tuned encoder compatible with the original text encoder for application in multimodal large models.
π― What it does: The Bootstrapped Model Predictive Control (BMPC) algorithm is proposed, which enhances the learning efficiency and performance of continuous control tasks by using MPC as an expert to guide policy and value learning.
Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel
Zun Wang (Shanghai AI Laboratory), Limin Wang (Shanghai AI Laboratory)
CodeData-Centric LearningRobotic IntelligenceTransformerLarge Language ModelVision Language ModelTextMultimodality
π― What it does: A fully automated self-improving data flywheel (SRDF) has been constructed, which generates high-quality large-scale language instruction-trajectory pair datasets through mutual screening and regeneration between the navigator and the instruction generator, enhancing the performance of visual-language navigation.
π― What it does: Proposed and analyzed a biologically inspired stochastic continuous Hopfield network (BBN) for efficient exploration in multi-armed bandit and MDP tasks.
π― What it does: This paper proposes an Lp-convolution method based on multivariate p-generalized normal distribution, which overlays learnable sparse masks on the convolution kernel, allowing large convolution kernels to maintain biologically common Gaussian sparse connections, thereby enhancing model performance and robustness.
Jiaxing Xu (Nanyang Technological University), Yiping Ke (Nanyang Technological University)
CodeClassificationDomain AdaptationExplainability and InterpretabilityGraph Neural NetworkGraphBiomedical DataAlzheimer's Disease
π― What it does: The BrainOOD framework is proposed for interpretable feature and structure selection in brain networks to enhance classification performance under multi-site distribution shift (OOD).
BrainUICL: An Unsupervised Individual Continual Learning Framework for EEG Applications
Yangxuan Zhou (Zhejiang University), Gang Pan (Zhejiang University)
CodeDomain AdaptationAnomaly DetectionConvolutional Neural NetworkContrastive LearningTime SeriesBiomedical Data
π― What it does: The BrainUICL framework is proposed to achieve unsupervised individual continuous learning of EEG, enabling continuous adaptation and improvement of generalization ability to unseen subjects among continuously emerging new subjects.
π― What it does: This paper proposes a new dataset distillation paradigm called INFER, which utilizes a Universal Feature Compensator (UFC) to extend a single synthetic sample to all categories, and achieves efficient MixUp augmentation through static soft labels, significantly enhancing the diversity and generalization ability of synthetic data.
Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization
Wei Liu (Hunan University of Science and Technology), Ruixuan Li (Hunan University of Science and Technology)
CodeClassificationExplainability and InterpretabilityGraph Neural NetworkSupervised Fine-TuningReinforcement LearningTextGraph
π― What it does: An interpretable framework N2R based on network utilization of input is proposed, replacing the traditional Maximum Mutual Information (MMI) criterion to extract rational explanations from the perspective of how the network utilizes the input.
Breaking Mental Set to Improve Reasoning through Diverse Multi-Agent Debate
Yexiang Liu (Institute of Automation Chinese Academy of Sciences), Tieniu Tan (Institute of Automation Chinese Academy of Sciences)
CodeLarge Language ModelPrompt EngineeringTextMultimodalityChain-of-Thought
π― What it does: A multi-agent debate framework DMAD is proposed, which enhances the reasoning accuracy of LLM/MLLM by allowing different agents to use different reasoning methods.
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization
Yuxin Jiang (Hong Kong University of Science and Technology), Wei Wang (Hong Kong University of Science and Technology)
CodeRecommendation SystemOptimizationTransformerLarge Language ModelReinforcement LearningText
π― What it does: A BMC framework is proposed, which enhances the relevance of contrastive data by first generating pseudo-winning responses before adversarial training, and then dynamically adjusting token-level rewards during DPO training to achieve finer-grained preference optimization.
Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding
Yanming Liu (Zhejiang University), Xuhong Zhang (Zhejiang University)
CodeTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
π― What it does: To address coreference ambiguity in long text question answering, a Long Question Coreference Adaptation (LQCA) framework is proposed, which first performs coreference resolution on sub-documents, then calculates mention distances, selects representative mentions, and replaces the original text, thereby improving the understanding and answering quality of LLMs for long texts.
π― What it does: This study investigates the Jensen gap problem caused by mini-batch training based on the group maximum-minimum fairness (MMF) objective in recommendation systems, and proposes the FairDual algorithm to reduce this gap through dual optimization methods.
Bridging the Gap between Database Search and \emph{De Novo} Peptide Sequencing with SearchNovo
Jun Xia (Westlake University), Stan Z. Li (Westlake University)
CodeTransformer
π― What it does: By combining database search with de novo sequence inference, the identification accuracy of peptide sequences is significantly improved by retrieving the most similar spectra and integrating reference sequences into the Transformer model.
BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
Hongjin SU, Tao Yu (University of Hong Kong)
CodeRetrievalLarge Language ModelTextBenchmark
π― What it does: A BRIGHT retrieval benchmark has been constructed, containing real queries that require deep reasoning to retrieve relevant documents.