International Conference on Learning Representations Β· 1064 papers
Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks
Edwin Zhang (Harvard University), Amy Zhang (University of Texas at Austin)
CodeOptimizationComputational EfficiencyRobotic IntelligenceTransformerLarge Language ModelDiffusion modelSequential
π― What it does: A language-controlled diffusion (LCD) framework is proposed, using natural language as a condition for hierarchical planning, leveraging diffusion models for efficient long-term, spatial, and task-dimensional planning, addressing the limitations of traditional skill libraries.
Language Model Detectors Are Easily Optimized Against
Charlotte Nicks (Stanford University), Stefano Ermon (Stanford University)
CodeOptimizationAdversarial AttackReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: Fine-tuning large language models using Direct Preference Optimization (DPO) in reinforcement learning to generate text that can confuse existing text detectors.
CodeGenerationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes and implements an attack method based on the inverse of the downstream probability distribution of language models, specifically recovering hidden prompt words from the model's next-word probability vector. It further investigates how to obtain the complete probability distribution through logit bias and binary search in different API access scenarios.
CodeCompressionTransformerLarge Language ModelImageTextAudio
π― What it does: This paper views language models as lossless compressors and systematically evaluates the compression performance of large pre-trained models across three different modalities: text, images, and audio. It also explores the impact of the compression perspective on model scaling laws, tokenization strategies, and context learning.
Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI
Weibang Jiang, Bao-liang Lu
CodeAnomaly DetectionRepresentation LearningTransformerTime SeriesBiomedical Data
π― What it does: A large brain model called LaBraM is proposed, which utilizes 2500 hours of multi-task EEG for unsupervised pre-training and fine-tuning on multi-task downstream tasks.
Large Language Model Cascades with Mixture of Thought Representations for Cost-Efficient Reasoning
Murong Yue (George Mason University), Ziyu Yao (George Mason University)
CodeComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: A reasoning pipeline based on LLM cascading has been designed and implemented, utilizing the answer consistency of weaker LLMs to determine whether to invoke more powerful LLMs, thereby reducing costs.
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
Yuchen Hu (Nanyang Technological University), EngSiong Chng
CodeRecognitionGenerationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningAudio
π― What it does: This paper proposes RobustGER, which utilizes LLM for generative error correction of ASR N-best and constructs a noise-robust version of the RobustHP dataset.
π― What it does: This paper studies the option position bias of large language models in multiple-choice question (MCQ) assessments, finding that models tend to favor specific option IDs (such as A), and proposes a label-free, inference-usable method called PriDe to eliminate this bias.
Chengrun Yang (Google DeepMind), Xinyun Chen (Google DeepMind)
CodeOptimizationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: An optimization framework OPRO is proposed, implemented through large language models (LLM), which utilizes natural language to describe problems and iteratively generate better solutions, applied to linear regression, the traveling salesman problem, and prompt optimization.
Large Language Models to Enhance Bayesian Optimization
Tennison Liu (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeOptimizationHyperparameter SearchTransformerLarge Language ModelPrompt EngineeringTabular
π― What it does: This paper proposes LLAMBO, which integrates large language models (LLM) into the Bayesian optimization process through natural language prompts, achieving functionalities such as zero-shot warm start, ICL-based surrogate models, and candidate point sampling.
Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages
Jinyi Hu (Tsinghua University), Maosong Sun (Tsinghua University)
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelContrastive LearningImageTextMultimodality
π― What it does: Proposed and implemented the MPM training paradigm, utilizing a multilingual large language model as a bridge between visual and target languages to achieve the training of multimodal models in non-English languages (e.g., Chinese VISCPM);
π― What it does: A complete pipeline for diffusion generation of 3D graphs (molecules) in a low-dimensional latent space is proposedβfirst, a decomposed 2D and 3D graph autoencoder is used to learn a low-dimensional latent representation with low reconstruction error and symmetry preservation, then a diffusion model is trained in that latent space, and finally decoded back to 3D graphs; it also supports conditional generation of SE(3) invariant properties or variable objects, and regularizes the latent space using graph self-supervised learning.
π― What it does: This paper proposes an evolution domain generalization method based on continuous-time stochastic differential equations (SDE) called SDE-EDG. It first constructs infinite fine grid evolution trajectories (IFGET) through sample correspondence and linear interpolation, and then uses SDE to learn the continuous evolution of the latent space and aligns it with maximum likelihood path regularization to enhance the model's generalization ability on future unseen domains.
π― What it does: This paper introduces the concept of Layered Linear Model Connectivity (LLMC), systematically analyzes and verifies the loss barriers produced by different layers and combinations of layers on model averaging (especially in the context of federated learning); experiments demonstrate that single-layer averaging has almost no barriers, while combinations of intermediate layers produce significant barriers; the LLMC phenomenon is explained from a robustness perspective, and its implications for personalization in federated learning are explored.
LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models
Zecheng Tang (Soochow University), Nan Duan (Microsoft Research Asia)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Reformulate the graphic layout generation task as a code generation task, and use large language models to complete the generation and rendering of layouts.
π― What it does: This paper proposes a self-supervised learning method based on Local Intrinsic Dimension Regularization (LDReg), aimed at preventing dimensional collapse.
π― What it does: This paper proposes a multi-resolution transformable graph convolutional network (MM-FGCN) that learns a multi-resolution basis for each graph instance through a meta-framework, achieving a unified representation of fine-grained and global features.
Learning Conditional Invariances through Non-Commutativity
Abhra Chaudhuri (University of Exeter), Anjan Dutta (University of Surrey)
CodeDomain AdaptationMultimodality
π― What it does: This paper proposes achieving conditional invariant feature learning through Non-commutative Invariance (NCI), which retains only specific information from the target domain during inference while discarding irrelevant disturbances from the source domain.
π― What it does: Utilize learnable dilated convolution (DCLS) to learn delays in deep feedforward spiking neural networks, obtaining a unique discrete delay for each synapse at the end of training;
Learning dynamic representations of the functional connectome in neurobiological networks
Luciano Dyballa (Yale University), Steven W. Zucker (Yale University)
CodeTime SeriesBiomedical Data
π― What it does: An unsupervised method is proposed to construct a functional connectivity map of C. elegans by calculating the differential affinity of neuronal activity over time and extracting dynamic communities using Non-negative Tensor Factorization (NTF).
π― What it does: A novel offline reinforcement learning algorithm named Conservative Density Estimation (CDE) is proposed, which utilizes a conservative estimate of the state-action occupancy distribution to reduce offline data sparsity and out-of-distribution (OOD) extrapolation errors.
Learning Grounded Action Abstractions from Language
Lionel Wong (Massachusetts Institute of Technology), Jacob Andreas (Massachusetts Institute of Technology)
CodeRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: This paper proposes the Ada framework, which utilizes language models to automatically generate and validate high-level planning abstractions (in PDDL format) and low-level controllers, gradually building a reusable action library to achieve language-driven long-term planning.
π― What it does: The CAST model is proposed, which achieves structured understanding of images through internal hierarchical segmentation in image recognition tasks, completing both segmentation and recognition without the need for pixel-level annotations.
π― What it does: A self-supervised two-layer hierarchical world model THICK is designed, utilizing sparsely updatable context encoding to learn interpretable temporal abstractions, and applying it to model-based reinforcement learning and model predictive control.
π― What it does: This paper proposes an unsupervised learning framework for implicit skeleton and elastic parameters 3D reconstruction from monocular videos, modeling moving objects using skeletons, skin weights, stiffness coefficients, and time-varying transformations.
π― What it does: The FACETVAE model is proposed, which utilizes the VAE framework to construct low-level interests based on multi-faceted (multi-dimensional) prototype learning and synthesizes high-level user interests through a binding mechanism, thereby achieving multi-faceted and interpretable modeling of user preferences.
Alexander G Shypula, Amir Yazdanbakhsh (Google DeepMind)
CodeOptimizationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Utilizing large pre-trained code language models (LLMs) for high-level program performance optimization, primarily by generating code rewrites that significantly enhance execution speed;
Learning Semantic Proxies from Visual Prompts for Parameter-Efficient Fine-Tuning in Deep Metric Learning
Li Ren (University of Central Florida), Kien A. Hua
CodeRetrievalTransformerPrompt EngineeringImage
π― What it does: The study achieves parameter-efficient deep metric learning fine-tuning on a pre-trained ViT through Visual Prompt Tuning and semantic proxies.
Learning the greatest common divisor: explaining transformer predictions
Francois Charton
CodeExplainability and InterpretabilityTransformerSupervised Fine-TuningTabular
π― What it does: Train a small Transformer to learn the greatest common divisor (GCD) of two positive integers, revealing its internal interpretable algorithm through input-output analysis.
π― What it does: The LAPO method is proposed, which infers the potential action space and trains the corresponding policy using only video data without action labels through inverse and forward dynamics models.
Learning to design protein-protein interactions with enhanced generalization
Anton Bushuiev (Czech Technical University), Josef Sivic (Czech Technical University)
CodeDrug DiscoveryProtein Structure PredictionTransformerBiomedical Data
π― What it does: A large-scale non-redundant protein-protein interaction (PPI) dataset, PPIRef, was constructed, and the SE(3) equivariant Transformer model, PPIFORMER, was pre-trained using this dataset. It was fine-tuned on labeled data such as SKEMPI to predict the effect of mutations on binding affinity's ΞΞG.
Learning to Embed Time Series Patches Independently
Seunghan Lee (Yonsei University), Kibok Lee (Yonsei University)
CodeClassificationRepresentation LearningContrastive LearningTime Series
π― What it does: A self-supervised pre-training method for time series called PITS is proposed, which achieves time series representation learning using a patch reconstruction task and an MLP encoder.
π― What it does: A learning framework based on Encoder-Decoder is proposed to solve the Class-Constrained Bin Packing Problem (CCBPP) and its application in manufacturing order consolidation (OCP).
π― What it does: This paper proposes a distance-based OOD detection method called PALM, which models each category as multiple von Mises-Fisher prototypes to learn a more compact and separable embedding space, achieving precise detection of out-of-distribution samples from the training set.
Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Models
Ashutosh Baheti (Georgia Institute of Technology), Mark Riedl (Georgia Institute of Technology)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: Designed and implemented an offline policy gradient method A-LOL, which utilizes a single action assumption for RLHF on language models, thereby avoiding online sampling and high computational costs.
LEGO-Prover: Neural Theorem Proving with Growing Libraries
Haiming Wang (Sun Yat-sen University), Xiaodan Liang (Sun Yat-sen University)
CodeLarge Language ModelText
π― What it does: A neural theorem proving framework named LEGO-Prover is proposed, which utilizes a growable skill library (verified lemmas/theorems) to construct proofs in a modular, block-like manner, retrieving existing skills while also generating and evolving new skills during the proving process.
Lemur: Harmonizing Natural Language and Code for Language Agents
Yiheng Xu (University of Hong Kong), Tao Yu (University of Hong Kong)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Two open-source large language models, Lemur and Lemur-Chat, have been developed with the goal of balancing capabilities in natural language and programming languages to support the construction of language agents.
Less is More: Fewer Interpretable Region via Submodular Subset Selection
Ruoyu Chen (Institute of Information Engineering, Chinese Academy of Sciences), Xiaochun Cao (Sun Yat-sen University)
CodeExplainability and InterpretabilityConvolutional Neural NetworkImage
π― What it does: By re-framing the image interpretability problem as a submodular set selection problem, a submodular objective function based on four scores: confidence, effectiveness, consistency, and synergy is designed. A greedy algorithm is used to select a limited number of local sub-regions, resulting in more refined and accurate interpretable areas.
π― What it does: A one-shot subgraph link prediction framework is proposed, which first uses Personalized PageRank to extract query-relevant subgraphs, and then employs a structured GNN model for prediction on these subgraphs.
Let's do the time-warp-attend: Learning topological invariants of dynamical systems
Noa Moriel (Hebrew University), Mor Nitzan (Hebrew University)
CodeClassificationConvolutional Neural NetworkBiomedical DataPhysics Related
π― What it does: Using data augmentation and self-attention convolutional networks, we learn the topological invariant features of supercritical Hopf bifurcation and construct the Time-Warp-Attend method for cross-system identification of point attractors and limit cycles, as well as for locating bifurcation boundaries.
Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning
Antoine Bambade (Inria), Justin Carpentier (Inria)
CodeOptimizationTabular
π― What it does: A unified augmented Lagrangian technique and Extended Conservative Jacobian (ECJ) method are proposed to differentiate feasible and infeasible convex quadratic programming layers, and an open-source QPLayer library has been implemented.
π― What it does: A coarse-to-fine multi-cell robot design method is proposed, utilizing hypercurvature embedding to unify the representation of robots with different granularities and searching for optimal structures through an improved cross-entropy method.
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency
Tianhong Li (Massachusetts Institute of Technology), Dilip Krishnan (Google Research)
CodeGenerationData SynthesisTransformerVision Language ModelImageText
π― What it does: Proposes the ITIT training framework, which utilizes cycle consistency to achieve audiovisual language generation on unpaired image and text data.
π― What it does: A global parameterization and integrability framework for constructing compatible convolution operations on non-compact, non-Abelian Lie groups (such as GLβΊ(n,β) and SL(n,β)) is established, and a equivariant convolution layer based on this framework is implemented.
π― What it does: A new lightweight SchrΓΆdinger bridge solver is proposed, aimed at addressing the issues of complexity and high computational resource consumption in existing solvers.
Light-MILPopt: Solving Large-scale Mixed Integer Linear Programs with Lightweight Optimizer and Small-scale Training Dataset
Huigen Ye (Tsinghua University), Hongyan Wang (Tsinghua University)
CodeOptimizationGraph Neural NetworkTabular
π― What it does: This paper presents Light-MILPopt, a lightweight optimization framework for large-scale Mixed Integer Linear Programming (MILP), which includes four stages: problem decomposition, model-based initial solution prediction, variable and constraint dimensionality reduction, and data-driven optimization.
LipVoicer: Generating Speech from Silent Videos Guided by Lip Reading
Yochai Yemini (Bar Ilan University), Ethan Fetaya (Bar Ilan University)
CodeGenerationDiffusion modelVideoAudio
π― What it does: The LipVoicer framework is designed to generate high-quality, synchronized speech using silent videos and text predicted by a lip-reading model.
Yuan Gong (Massachusetts Institute of Technology), James R. Glass (Massachusetts Institute of Technology)
CodeTransformerLarge Language ModelPrompt EngineeringMultimodalityAudio
π― What it does: A multi-modal large language model LTU has been developed, capable of completing audio question-answering tasks from closed to open format through audio perception, reasoning, and understanding.
LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses
Xin Liu (University of Michigan), Lu Wang (University of Michigan)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: A lightweight language model calibration method called LITCAB is proposed, which uses a linear layer to predict logit biases from the last hidden representation of the LM, enhancing the probability calibration of generated outputs.
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelTextMultimodality
π― What it does: A lightweight adapter (LLaMA-Adapter) was developed on top of LLaMA 7B, achieving efficient fine-tuning for instruction following by inserting learnable prompts and zero-initialized attention mechanisms on top of a frozen model, training only 1.2M parameters in less than an hour; the method was also extended to multimodal applications, utilizing the CLIP image encoder to generate visual prompts and construct a multimodal LLM.
π― What it does: Proposes the LLCP framework, which utilizes a self-supervised temporal multivariate generative model to learn the underlying causal processes in videos, thereby achieving accident attribution and counterfactual prediction without relying on question-answer annotations.
Zhangir Azerbayev (Princeton University), Sean Welleck (Carnegie Mellon University)
CodeTransformerLarge Language ModelTextChain-of-Thought
π― What it does: An open-source large language model specifically designed for mathematics, LLEMMA, has been proposed and the 55B-token Proof Pile 2 dataset has been publicly released.
LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts
Hanan Gani (Mohamed Bin Zayed University of AI), Peter Wonka
CodeGenerationTransformerLarge Language ModelDiffusion modelImageText
π― What it does: This paper proposes a two-stage text-to-image generation framework: first, a large language model (LLM) is used to extract a structured scene blueprint from long text prompts (including object layout, object descriptions, and background context), and an initial image is generated using a diffusion model conditioned on the layout; subsequently, through box-level iterative refinement, a CLIP multimodal-guided diffusion and image synthesis model is employed to gradually correct each object to ensure it fully matches the description.
LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation
Suhyeon Lee (Korea Advanced Institute of Science and Technology), Jong Chul Ye (Korea Advanced Institute of Science and Technology)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelGenerative Adversarial NetworkImageTextMultimodalityBiomedical DataComputed Tomography
π― What it does: A multimodal LLM called LLM-CXR is proposed, which achieves understanding of chest X-ray (CXR) images, report generation, and report-to-image generation all within the same model without the need for additional adapter networks or external generative models through instruction finetuning.
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Lianmin Zheng (University of California Berkeley), Hao Zhang (University of California San Diego)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: The LMSYS-Chat-1M dataset has been collected and released, containing 1 million real user dialogues from 25 types of LLMs, covering 210K users and 154 languages. Based on this, four application cases are presented: content moderation, instruction fine-tuning, safety benchmarks, and high-difficulty evaluations.
Minsu Kim (Korea Advanced Institute of Science and Technology), Jinkyoo Park (Korea Advanced Institute of Science and Technology)
CodeOptimizationDrug DiscoveryReinforcement LearningBiomedical Data
π― What it does: An algorithm called Local Search GFlowNets (LS-GFN) is proposed, which incorporates a local search step into GFlowNet training, improving the sampling paths through backward backtracking and forward reconstruction.
π― What it does: This paper explores the storage location of visual attribute knowledge in text-to-image diffusion models through causal mediation analysis, and based on this, proposes a fast, data-free model editing method called DIFF-QUICKFIX.
LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models
Yixiao Li (Georgia Institute of Technology), Tuo Zhao (Georgia Institute of Technology)
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The LoftQ framework is proposed, which combines quantization and low-rank decomposition to provide a better initialization for LoRA fine-tuning after quantization in large language models.
π― What it does: A pluggable neural layer called LogicMP is proposed, which couples any neural network with first-order logic constraints (MLN) using efficient mean-field inference to achieve neural-symbolic integration.
Long-Short-Range Message-Passing: A Physics-Informed Framework to Capture Non-Local Interaction for Scalable Molecular Dynamics Simulation
Yunyang Li (Yale University), Tie-Yan Liu (Microsoft Research AI4Science)
CodeGraph Neural NetworkGraphPhysics Related
π― What it does: Proposes the Long-Short-Range Message-Passing (LSR-MP) framework, which combines chemical segmentation with long-range message passing to enhance the prediction accuracy of EGNN in large molecular dynamics.
π― What it does: For training diffusion models under long-tail distributions, a head-to-tail (T2H) knowledge transfer and batch re-sampling strategy based on multi-objective score estimation is proposed to enhance the generation diversity and quality of tail classes.
Looped Transformers are Better at Learning Learning Algorithms
Liu Yang (University of Wisconsin), Dimitris Papailiopoulos (University of Wisconsin)
CodeTransformerTabular
π― What it does: The research proposes a cyclic Transformer architecture and training method, enabling the Transformer to learn and approximate iterative algorithms for various data fitting problems while significantly reducing the number of parameters.
LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agents
Jae-Woo Choi (Electronics and Telecommunications Research Institute), Minsu Jang (Electronics and Telecommunications Research Institute)
CodeRobotic IntelligenceTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: A benchmark framework for the automatic evaluation of LLM-driven language task planners is proposed, and systematic experiments are conducted in the context of home service robot tasks.
LQ-LoRA: Low-rank plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
Han Guo (Carnegie Mellon University), Yoon Kim (Massachusetts Institute of Technology)
CodeCompressionOptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a low-rank + quantized matrix decomposition method (LQ-LoRA), which splits the weights of the pre-trained model into quantizable low-precision matrices and learnable low-rank matrices, updating only the low-rank part during training; it also dynamically allocates quantization configurations for different layers through integer linear programming and provides a data-aware version that utilizes Fisher information for weighted reconstruction.
π― What it does: A language-driven resampling continuous representation (LRR) is proposed for preprocessing frames during tracking to resist adversarial attacks.
LUM-ViT: Learnable Under-sampling Mask Vision Transformer for Bandwidth Limited Optical Signal Acquisition
Lingfeng Liu (Zhejiang University), Hangjie Yuan (Zhejiang University)
CodeClassificationTransformerImage
π― What it does: A learnable downsampling mask Vision Transformer (LUM-ViT) is proposed, which achieves efficient acquisition of broadband constrained optical signals through pre-sampling optical modulation.
LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
Gunho Park (Pohang University of Science and Technology), Dongsoo Lee (NAVER Cloud)
CodeGenerationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A lookup table (LUT)-based quantized matrix multiplication kernel LUT-GEMM has been developed, which directly uses BCQ quantized weights, eliminating the dequantization step and accelerating inference for large-scale language models.
M3C: A Framework towards Convergent, Flexible, and Unsupervised Learning of Mixture Graph Matching and Clustering
Jiaxin Lu (University of Texas), Junchi Yan (Shanghai Jiao Tong University)
CodeGraph Neural NetworkContrastive LearningGraph
π― What it does: This paper proposes M3C and its unsupervised extension UM3C for joint learning of mixed graph matching and clustering, balancing matching accuracy and clustering quality.
π― What it does: A machine forgetting framework for image-to-image generation models is proposed, along with an efficient forgetting algorithm based solely on the encoder, which can completely erase the memory of forgotten samples without degrading the performance of retained samples.
π― What it does: This study addresses and solves the amplitude ratio problem in the training of super networks, proposing MIP for stable and efficient training.
Magnushammer: A Transformer-Based Approach to Premise Selection
Maciej MikuΕa (Google DeepMind), Yuhuai Wu (xAI)
CodeRetrievalTransformerContrastive LearningText
π― What it does: In this paper, the authors propose a Transformer-based premise selection method called Magnushammer and construct the largest Isabelle premise selection dataset.
Yuying Ge (Tencent AI Lab), Ying Shan (Tencent AI Lab)
CodeGenerationRetrievalTransformerLarge Language ModelVision Language ModelContrastive LearningImageVideoTextMultimodality
π― What it does: This paper proposes the SEED tokenizer and SEED-LLaMA, which integrates large language models with discrete visual codes to construct a unified multimodal autoregressive model that supports text generation, image understanding, image generation, and even multi-turn interaction and zero-shot compositional generation.
Making Retrieval-Augmented Language Models Robust to Irrelevant Context
Ori Yoran (Tel Aviv University), Jonathan Berant (Tel Aviv University)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
π― What it does: Research and address the performance decline issue caused by retrieving irrelevant contexts in Retrieval-Augmented Language Models (RALM).
π― What it does: MAMBA is proposed, a meta-reinforcement learning algorithm based on Dreamer, designed for efficient learning of Bayesian optimal policies in multi-task environments.
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
Xiang Yue (Ohio State University), Wenhu Chen (University of Waterloo)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
π― What it does: This paper constructs a specialized instruction tuning dataset for mathematical reasoning, MathInstruct, and trains the MAmmoTH series of open-source large language models, significantly enhancing their reasoning capabilities on various mathematical problems.
π― What it does: By systematically varying the dropout rate in the AlexNet model, this study explores the impact of dropout on the dimensionality of the representational space, robustness, and its correspondence with the human brain.
Robert Jenssen (UiT Arctic University of Norway & University of Copenhagen & Norwegian Computing Center)
CodeOptimizationRepresentation LearningImage
π― What it does: A novel visualization dimensionality reduction method called MAP IT is proposed, based on projective divergence and marginal probability alignment.
π― What it does: A self-supervised masking ray and view modeling framework for general NeRF (MRVM-NeRF) is proposed, which significantly enhances the prior learning and inference accuracy of 3D scenes by randomly masking features and cross-view features during the fine-grained sampling stage and aligning targets with the online branch in the latent space.
Masked Audio Generation using a Single Non-Autoregressive Transformer
Alon Ziv (Meta), Yossi Adi (Meta)
CodeGenerationData SynthesisTransformerAudio
π― What it does: This paper proposes a single-stage, non-autoregressive transformer model called MAGNET, which directly generates audio on multi-stream audio discrete representations using mask generation techniques.
Masked Structural Growth for 2x Faster Language Model Pre-training
Yiqun Yao (Beijing Academy of Artificial Intelligence), Yequan Wang (Beijing Academy of Artificial Intelligence)
CodeHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The Masked Structural Growth (MSG) scheme is proposed, which gradually expands the model structure in a masked manner during the pre-training process of the Transformer while strictly maintaining function invariance.
Mastering Symbolic Operations: Augmenting Language Models with Compiled Neural Networks
Yixuan Weng (University of Chinese Academy of Sciences), Jun Zhao (University of Chinese Academy of Sciences)
CodeTransformerLarge Language ModelMixture of ExpertsText
π― What it does: Developed the Neural Comprehension framework, which integrates Compiled Neural Networks (CoNNs) with pre-trained language models to achieve symbolic reasoning and rule execution, supporting end-to-end differentiable symbolic computation.
Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching
Yang Liu (Zhejiang University), Chunhua Shen (Zhejiang University)
CodeSegmentationContrastive LearningImageVideo
π― What it does: Proposes the Matcher framework, which utilizes pre-trained visual foundation models (DINOv2+SAM) to complete various segmentation tasks under the guidance of one-shot examples, completely without training.
π― What it does: This paper constructs fully connected layers, convolutional layers, and polynomial logistic regression layers for positive semi-definite matrices in Riemannian random spaces, and implements gradient propagation of logarithmic mapping from the perspective of Grassmann projection, subsequently applying it to construct Grassmann graph convolutional networks.
π― What it does: Proposes the Matryoshka Diffusion (MDM) framework, which directly trains high-resolution image and video generation models in pixel space using a multi-resolution joint diffusion process;
MCM: Masked Cell Modeling for Anomaly Detection in Tabular Data
Jiaxin Yin (Beijing University of Posts and Telecommunications), Jie Yang (Beijing University of Posts and Telecommunications)
CodeAnomaly DetectionTabularFinance Related
π― What it does: This study investigates anomaly detection in tabular data and proposes a self-supervised method based on Masked Cell Modeling (MCM).
Memorization Capacity of Multi-Head Attention in Transformers
Sadegh Mahdavi (University of British Columbia), Christos Thrampoulidis (University of British Columbia)
CodeTransformerImage
π― What it does: This paper studies the memory capacity of the multi-head attention mechanism and provides a lower bound under the assumption of linear independence.
MEND: Meta Demonstration Distillation for Efficient and Effective In-Context Learning
Yichuan Li (Worcester Polytechnic Institute), Chenlei Guo (Amazon Alexa AI)
CodeComputational EfficiencyKnowledge DistillationMeta LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A demonstration compression method called MEND is proposed, which utilizes a hypernetwork to compress long demonstrations into vectors and directly feeds them to a large language model for efficient context learning without retraining for new tasks.
Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Pingzhi Li (University of North Carolina at Chapel Hill), Tianlong Chen (Harvard University)
CodeCompressionComputational EfficiencyKnowledge DistillationSupervised Fine-TuningMixture of ExpertsText
π― What it does: A routing strategy-based Sparse Mixture of Experts (SMoE) merging and compression framework, MC-SMoE, is proposed. It first merges redundant experts into fewer but more knowledgeable experts through M-SMoE, and then compresses the merged experts using low-rank and sparse decomposition.
Meta Continual Learning Revisited: Implicitly Enhancing Online Hessian Approximation via Variance Reduction
Yichen Wu (City University of Hong Kong), Ying Wei (Nanyang Technological University)
CodeOptimizationMeta LearningSequential
π― What it does: This paper reinterprets the Meta-CL method as a technique for online approximate Hessian and proposes the Momentum-Based Variance-Reduced Meta-CL (VR-MCL) to reduce the variance of the hypergradient, thereby enhancing the stability of model updates in continual learning.
Meta-Learning Priors Using Unrolled Proximal Networks
Yilang Zhang (University of Minnesota), Georgios B. Giannakis (University of Minnesota)
CodeOptimizationMeta LearningImage
π― What it does: A novel explainable prior learning framework called MetaProxNet is proposed, which utilizes a learnable piecewise linear function to approximate the projection operator for task-independent regularization.
π― What it does: Proposed the MetaCoCo benchmark dataset and evaluated the performance of various few-shot learning methods under the presence of spurious correlation shifts.
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Sirui Hong (DeepWisdom), JΓΌrgen Schmidhuber (AI Initiative, King Abdullah University of Science and Technology)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: A multi-agent collaboration framework based on LLM, MetaGPT, has been constructed to simulate the software company process and complete end-to-end generation from requirements to code.