π― What it does: An adaptive sampling algorithm based on variational control (VCAS) is designed, which generates unbiased approximate gradients during backpropagation using fine-grained data and token-level sampling, significantly reducing training costs.
Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learning
Hyungho Na (Korea Advanced Institute of Science and Technology), Il-chul Moon
CodeReinforcement LearningSequential
π― What it does: This paper proposes an efficient episodic memory utilization framework EMU for collaborative multi-agent reinforcement learning, aimed at accelerating learning and avoiding local optima.
π― What it does: Efficient deterministic and stochastic sampling methods have been designed for diffusion generative models (especially the enhanced PSLD model), significantly reducing the number of NFE (network function evaluations) and improving sampling speed.
π― What it does: This paper proposes an efficient local linear regularization method called ELLE, combined with adaptive regularization ELLE-A, to address the problem of catastrophic overfitting in single-step adversarial training.
π― What it does: An efficient modulation module named EfficientMod is proposed, and a new convolutional network architecture is built based on this module.
Efficient Multi-agent Reinforcement Learning by Planning
Qihan Liu (Tsinghua University), Chongjie Zhang (Washington University in St. Louis)
CodeReinforcement LearningBenchmark
π― What it does: This paper proposes MAZero, a model-based multi-agent reinforcement learning algorithm that combines the ideas of MuZero and achieves significant sample efficiency improvements on the SMAC benchmark.
π― What it does: An efficient sharpness-aware minimization algorithm named GraphSAM is proposed to improve the generalization performance of unpretrained molecular graph Transformer models.
Efficient Streaming Language Models with Attention Sinks
Guangxuan Xiao (Massachusetts Institute of Technology), Mike Lewis (Meta AI)
CodeTransformerLarge Language ModelText
π― What it does: The StreamingLLM framework is proposed, utilizing attention sink and sliding window KV cache to enable pre-trained LLMs to efficiently infer on infinitely long texts.
π― What it does: This paper studies a framework called Policy-Learn, which is used to learn subgraph selection strategies to reduce the computational cost of subgraph GNNs while maintaining expressive power.
π― What it does: A data-independent, parameter-efficient low-bit diffusion model fine-tuning framework called EfficientDM is proposed, utilizing Quantization-Aware Low-Rank Adapters (QALoRA) and noise distillation to achieve model quantization.
π― What it does: This paper proposes the Elastic Feature Consolidation (EFC) method, which utilizes the Empirical Feature Matrix (EFM) to impose elastic constraints on feature drift, and combines it with the asymmetric prototype replay loss (PR-ACE) to enhance the model's plasticity and stability in sample-free incremental learning.
Elucidating the design space of classifier-guided diffusion generation
Jiajun Ma (Hong Kong University of Science and Technology), Jiacheng Sun (Huawei Noah's Ark Lab)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: In conditional generation with diffusion models, this study explores how to utilize existing pre-trained classifiers for training-independent guidance to enhance sample quality and controllability.
π― What it does: Research and address the exposure bias issue in the sampling phase of diffusion models, proposing a no-training, plug-in Epsilon Scaling method to improve sampling trajectories.
π― What it does: Through a dual-layer optimization framework, large datasets are compressed to generate a small number of synthetic samples that achieve comparable test performance to the original dataset.
EMO: EARTH MOVER DISTANCE OPTIMIZATION FOR AUTO-REGRESSIVE LANGUAGE MODELING
Siyu Ren (Shanghai Jiao Tong University), Kenny Q. Zhu (University of Texas at Arlington)
CodeGenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: An automatic regression language model optimization method based on the upper bound of Earth Mover's Distance (EMD) called EMO is proposed, achieving distribution calibration of the pre-trained model during the lightweight fine-tuning phase.
Quan Sun (Beijing Academy of Artificial Intelligence), Xinlong Wang (Beijing Academy of Artificial Intelligence)
CodeGenerationTransformerLarge Language ModelVision Language ModelDiffusion modelImageVideoTextMultimodality
π― What it does: A multimodal generative pre-training model called Emu is proposed, which can simultaneously handle images, text, and videos, achieving multi-tasking such as image generation, image description, and question answering.
Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products
Shengjie Luo (Peking University), Aditi S. Krishnapriyan (University of California)
CodeComputational EfficiencyTabularPhysics Related
π― What it does: This paper proposes the Gaunt Tensor Product, which utilizes Gaunt coefficients to transform E(3) equivariant tensor products into spherical function products, significantly reducing computational complexity through two-dimensional Fourier transforms and FFT acceleration.
π― What it does: For the visual target navigation tasks of ImageNav and Instance-ImageNav, a dual-encoder (dual-view ViT) structure is proposed, and a two-step pre-training (cross-view completion CroCo and relative pose + visibility estimation RPEV) is conducted to provide priors for the perception module, followed by training an integrated end-to-end navigation policy in RL.
π― What it does: This paper researches and implements an automatic model evaluation (AutoEval) method that does not require labels or training, using the Meta-Distribution Energy (MDE) metric constructed from energy models to predict the accuracy of models on unlabeled OOD datasets.
Xinyue Xu (Hong Kong University of Science and Technology), Xiaomeng Li (Hong Kong University of Science and Technology)
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkImage
π― What it does: Proposes the Energy-based Concept Bottleneck Model (ECBM), unifying concept prediction, concept intervention, and conditional explanation;
π― What it does: Proposes the Aranyani framework, which achieves group fair decision-making through oblique decision forests in an online learning environment.
π― What it does: A consistency boosting method called CoβBoosting is proposed, which enhances the performance of the server model in one-shot federated learning by generating high-quality hard samples and dynamically reweighting client models to form a stronger teacher ensemble.
Enhancing Transferable Adversarial Attacks on Vision Transformers through Gradient Normalization Scaling and High-Frequency Adaptation
Zhiyu Zhu (University of Sydney), Huaming Chen (University of Sydney)
CodeAdversarial AttackTransformerImage
π― What it does: For transferable adversarial attacks on Vision Transformers, two modules are proposed: Gradient Normalization Scaling (GNS) and High-Frequency Adaptation (HFA), which utilize fine-grained gradient normalization/scaling and frequency domain high-frequency masking to enhance the transferability of adversarial samples.
π― What it does: This study investigates ensemble and distillation methods for unsupervised constituent syntax parsing, generating an 'average' tree from the results of various existing unsupervised parsers, and then using this tree as a pseudo-label to train a student model to improve inference efficiency.
π― What it does: This paper proposes an entity-centered reinforcement learning framework for multi-object visual reinforcement learning, capable of learning object manipulation under target conditions from pixel images.
Julius Kunze (University College London), James Townsend (University of Amsterdam)
CodeCompressionGraph
π― What it does: This paper studies a general method for compressing unordered data structuresβshuffle codingβused for optimal lossless compression of unordered objects such as graphs, sets, and multisets.
π― What it does: A new MCMC method (Entropy-MCMC, EMCMC) is proposed, which introduces auxiliary guiding variables to bias sampling towards the flat valleys in the posterior distribution of deep networks, resulting in samples with better generalization ability.
π― What it does: A brain-inspired network shape intelligence framework called Epitopological Sparse Meta-deep Learning (ESML) is proposed, which is extremely sparse (only 1% connectivity) and includes a four-step training method called Cannistraci-Hebb Training (CHT), achieving training with only 1% connectivity while surpassing the performance of fully connected networks.
π― What it does: A molecular docking scoring function based on equivariant scalar fields is proposed, and FFT is used to quickly optimize ligand poses.
π― What it does: This paper proposes an Early Stopping Consistency (ESC) sampling strategy that significantly reduces the number of generated samples while retaining the performance of Self-Consistency (SC).
π― What it does: The DIME method is proposed, which estimates conditional mutual information without a generative model by using loss to improve regression targets through discriminative training of predictors and value networks, achieving dynamic feature selection.
Estimating Shape Distances on Neural Representations with Limited Samples
Dean A Pospisil, Alex H Williams
CodeBiomedical Data
π― What it does: This study investigates the statistical properties of shape distance estimation represented by neural networks under high-dimensional and sample-limited conditions, and proposes a moment estimator that adjusts the bias-variance trade-off.
π― What it does: This paper proposes and implements a dynamic evaluation framework based on structured negotiation games to simultaneously measure the agent capabilities, dialogue coherence, and alignment level of language models (LMs).
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: A meta-evaluation benchmark specifically designed to assess the performance of large language model (LLM) evaluators in instruction following, called LLMBAR, has been created, and various LLM evaluators and prompting strategies have been systematically evaluated on this benchmark.
EventRPG: Event Data Augmentation with Relevance Propagation Guidance
Mingyuan Sun (Hong Kong University of Science and Technology), Renjing Xu (Hong Kong University of Science and Technology)
CodeClassificationRecognitionSpiking Neural NetworkTime Series
π― What it does: This paper proposes a method called EventRPG, which utilizes correlation propagation in SNN to generate CAM and saliency maps, guiding the Drop and Mix augmentation of event data.
ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis
Kensen Shi (Google DeepMind), Charles Sutton (Google DeepMind)
CodeOptimizationAI Code AssistantTransformerText
π― What it does: A program synthesis method based on executing sub-goals, ExeDec, is proposed, which utilizes program execution states to decompose tasks step by step, enhancing the combinatorial generalization ability of program synthesis.
π― What it does: This paper proposes Expectation Flow Networks (EFlowNets) and Adversarial Flow Networks (AFlowNets), extending Generative Flow Networks (GFlowNets) to stochastic environments and two-player zero-sum games.
Explaining Time Series via Contrastive and Locally Sparse Perturbations
Zichuan Liu (Nanjing University), Qingsong Wen (Alibaba Group)
CodeExplainability and InterpretabilityContrastive LearningTime SeriesBiomedical DataElectronic Health Records
π― What it does: This paper proposes ContraLSP, a time series explanation method based on contrastive learning and local sparse gates, which can generate adversarial perturbations while maintaining distribution consistency and learning binarized, smooth masks.
π― What it does: Using the time steps of the diffusion model as a prior, the encoder is trained to let different sub-vectors correspond to the attribute loss of different time steps, thereby learning unsupervised decomposable features;
π― What it does: Through multi-stage masked knowledge distillation (dBOT), excellent visual representations can be obtained using a randomly initialized teacher model without the need for carefully designed target representations during pre-training.
Exploring the cloud of feature interaction scores in a Rashomon set
Sichao Li (Australian National University), Amanda S Barnard
CodeTransformerImageTabular
π― What it does: This paper proposes Feature Interaction Score (FIS) and studies its distribution cloud (FISC) within the Rashomon set, while also providing a greedy search algorithm and Halo/Swarm visualization methods.
π― What it does: This paper studies a real-time recurrent learning (RTRL) method that can be used in real environments, proposing an element-wise recurrent eLSTM implementation and comparing it with truncated BPTT in multi-task reinforcement learning.
π― What it does: Analyze and simplify the weight balancing method in long-tail recognition, proposing that performance can be improved with only single-stage training.
π― What it does: A tunable parameter 'expressive loss' framework is proposed, implemented with three convex combinations (CC-IBP, MTL-IBP, Exp-IBP), achieving a better balance between accuracy and robustness in validating robust training.
f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimization
Sina Baharlouei (University of Southern California), Meisam Razaviyayn (University of Southern California)
CodeOptimizationTabular
π― What it does: A differentiable fair empirical risk minimization framework based on f-divergence, called f-FERM, is proposed, achieving a convergent unbiased gradient estimate under stochastic mini-batch training, followed by constructing a distributionally robust version using L1/β uncertainty sets.
π― What it does: Conducted systematic experiments on 19 visual tasks to compare the performance of Visual Prompt Tuning (VPT) and Full Fine-Tuning (FT), and explored the reasons and conditions for the advantages and disadvantages of both methods.
Tongxin Yin (University of Michigan), Yang Liu (ByteDance Research)
CodeClassificationOptimizationTabular
π― What it does: A post-processing rejectable classification framework (FAN) is proposed, which determines whether to reject predictions through integer programming and trains the model to achieve automatic rejection based on this.
fairret: a Framework for Differentiable Fairness Regularization Terms
Maarten Buyl (Ghent University), Tijl De Bie (Ghent University)
CodeTabular
π― What it does: A differentiable fair regularization framework called FAIRRET is proposed, which can quantify and minimize bias in machine learning models.
π― What it does: Proposes the FairTune framework, which utilizes parameter-efficient fine-tuning (PEFT) to automatically optimize fairness in medical imaging tasks, addressing the fairness generalization gap.
Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Models
Andrew William Engel (Pacific Northwest National Laboratory), Tony Chiang (Pacific Northwest National Laboratory)
CodeSafty and PrivacyExplainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkTransformerImage
π― What it does: This paper proposes and evaluates a series of approximate empirical neural tangent kernel (eNTK) models (such as trNTK, projβtrNTK, projβpNTK) and uses them as kernel functions to construct linear kernel generalized linear models (kGLM) to approximate and explain the decision-making process of deep neural networks.
Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals
Yair Ori Gat (Technion), Roi Reichart (Technion)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
π― What it does: Two model-agnostic causal explanation methods are proposed, using large language models to generate adversarial texts or learning to match embedding spaces to approximate counterfactuals, thereby assessing the causal impact of NLP models on high-level concepts.
Faithful Rule Extraction for Differentiable Rule Learning Models
Xiaxia Wang (University of Oxford), Ian Horrocks (University of Oxford)
CodeOptimizationExplainability and InterpretabilityGraphBenchmark
π― What it does: This paper proposes a method for extracting a sound and complete rule set from differentiable rule learning models (especially DRUM), addressing the lack of formal guarantees in existing methods.
π― What it does: This paper proposes FedCOGβa data-correction-based federated learning framework that utilizes a global model to generate complementary data and enhances local model consistency through soft label distillation, thereby alleviating model drift caused by data heterogeneity.
π― What it does: This paper proposes the Diffusion Bridge Network (DBN), which constructs a conditional diffusion bridge between the output of a single model and the output of the entire ensemble model, allowing for the approximation of deep ensemble predictions using a single forward pass and a lightweight scoring network.
π― What it does: Two algorithms suitable for hyperbolic space, HYPERDT and HYPERRF, are proposed, implementing node partitioning using geometric inner products on hyperplanes.
π― What it does: An algorithm for rapidly updating sparse matrix truncated singular value decomposition (truncated SVD) has been developed, supporting incremental updates by row/column/weight while maintaining high accuracy.
π― What it does: A deep reinforcement learning sampling algorithm based on Kalman filtering and Langevin dynamicsβLKTDβis proposed for efficiently sampling from the posterior distribution of value function parameters.
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
Guangsheng Bao (Westlake University), Yue Zhang (Westlake University)
CodeClassificationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A zero-shot detector named Fast-DetectGPT is proposed for efficiently distinguishing between machine-generated text and human-written text.
π― What it does: This paper proposes a weight-sharing SE(n) equivariant convolutional network on homotopy spaces, constructing a scalable 3D point cloud processing architecture called P Ξ NITA.
π― What it does: A general estimator GELS based on least squares and its variants are proposed for quickly approximating all probability values (including Shapley, Banzhaf, etc.) and distributed values, along with theoretical convergence analysis and an unsupervised training framework TrELS.
π― What it does: Designed and implemented FasterViT, a hybrid network that integrates CNN and Vision Transformer, aimed at achieving high throughput and high accuracy for high-resolution images; and proposed a Hierarchical Attention (HAT) module to efficiently capture global and local dependencies.
Thomas Laurent (Loyola Marymount University), Xavier Bresson (National University of Singapore)
CodeClassificationExplainability and InterpretabilityRepresentation LearningRecurrent Neural NetworkText
π― What it does: This study investigates how early layers of the network assign the same features to words with the same roles (i.e., feature collapse) in the synthetic sentence classification task, and demonstrates that this collapse can achieve interpretable and generalizable representations in the limit of large samples.
π― What it does: We propose FedCompass, a semi-asynchronous federated learning framework that utilizes a computation power-aware scheduler to enable different clients to nearly synchronize their local training within a group, reducing model staleness and improving efficiency.
Federated Causal Discovery from Heterogeneous Data
Loka Li (Mohamed bin Zayed University of Artificial Intelligence), Kun Zhang (Carnegie Mellon University)
CodeFederated LearningSafty and PrivacyTabularBiomedical DataMagnetic Resonance ImagingFinance Related
π― What it does: A constraint-based federated causal discovery method, FedCDH, is proposed, which can learn causal structures through summary statistics in decentralized and heterogeneous data environments.
Federated Recommendation with Additive Personalization
Zhiwei Li (Australian AI Institute), Tianyi Zhou (University of Maryland)
CodeRecommendation SystemFederated LearningSafty and PrivacyTabular
π― What it does: The FedRAP method is proposed, which achieves bidirectional personalized recommendation in federated learning through global sparse item embeddings and local personalized embeddings, while only uploading the sparse global matrix to reduce communication and privacy leakage risks.
π― What it does: The FedWaD algorithm is proposed to compute the Wasserstein distance between two distributions in a federated environment without sharing samples.
FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent
Ziyao Wang (University of Maryland), Ang Li (University of Maryland)
CodeOptimizationFederated LearningImage
π― What it does: This paper proposes FEDHYPER, a general robust learning rate scheduler based on supergradient descent, for adaptive adjustment of global and local learning rates in federated learning.
FedInverse: Evaluating Privacy Leakage in Federated Learning
Di Wu (University of Southern Queensland), Atul Sajjanhar (Swinburne University of Technology)
CodeFederated LearningSafty and PrivacyAdversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
π― What it does: Proposes the FedInverse framework to assess the risk of model inversion attacks (MI) on privacy leakage in federated learning systems.
FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data
Zikai Xiao (Zhejiang University), Zuozhu Liu (Zhejiang University)
CodeFederated LearningImage
π― What it does: This paper proposes the FedLoGe framework, which simultaneously improves the performance of the global model and individual client models in federated long-tail learning.
π― What it does: Proposes the FedTrans framework, which utilizes server-side auxiliary data to estimate the utility of client updates through a Bayesian model and variational inference, achieving transparent client selection under noise and heterogeneous environments, thereby enhancing the robustness of federated learning.
Ferret: Refer and Ground Anything Anywhere at Any Granularity
Haoxuan You (Columbia University), Yinfei Yang (Apple)
CodeClassificationObject DetectionSegmentationRetrievalTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: Ferret, a multimodal large language model, has been designed to understand references for arbitrary shapes and locate them semantically in images.
FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods
Xiaotian Han (Texas A&M University), Xia Hu (Rice University)
CodeClassificationOptimizationTabularBenchmark
π― What it does: This paper proposes and implements a standardized benchmark framework called Fair Fairness Benchmark (FFB) for evaluating and comparing in-processing group fairness methods for binary classification tasks.
Fine-Tuned Language Models Generate Stable Inorganic Materials as Text
Nate Gruver (New York University), Zachary Ward Ulissi
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextPhysics Related
π― What it does: Fine-tuning the text representation of crystal structures using pre-trained large language models to generate stable inorganic materials that meet physical constraints.
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
Xiangyu Qi (Princeton University), Peter Henderson (Princeton University)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: This study investigates how fine-tuning alignment in large language models can lead to a decrease in safety, demonstrating the impact of both malicious and benign fine-tuning on safety.
Katherine Tian (Stanford University), Chelsea Finn (Stanford University)
CodeGenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: This paper proposes a language model fine-tuning method for factuality that does not require manual annotation, utilizing automatically generated factual preference data for Direct Preference Optimization (DPO) fine-tuning.
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
Juncheng Li (Zhejiang University), Yueting Zhuang (Zhejiang University)
CodeRecognitionGenerationData SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
π― What it does: A lightweight visual prompt generator completion module VPG-C is proposed, which enhances the understanding ability of multimodal large language models on unsupervised demonstration instructions using a synthetic discriminative training strategy; simultaneously, a DEMON benchmark is constructed to evaluate the model's performance in following multimodal interactive instructions.
Finite-State Autoregressive Entropy Coding for Efficient Learned Lossless Compression
Yufeng Zhang (Shanghai Jiao Tong University), Weiyao Lin (Ant Group)
CodeCompressionImage
π― What it does: A lossless compression framework that combines Finite State Autoregressive (FSAR) priors and Straight-Through Hard Quantization (STHQ) is proposed;
Zhijian Xu (Chinese University of Hong Kong), Qiang Xu (Chinese University of Hong Kong)
CodeAnomaly DetectionOptimizationTime Series
π― What it does: This paper proposes a lightweight temporal model FITS, which treats time series forecasting and anomaly detection as a frequency domain interpolation problem. It utilizes a single-layer complex linear layer to achieve amplitude scaling and phase shifting, thereby completing long-term and short-term forecasting and self-supervised reconstruction with only about 10k parameters.
π― What it does: A robust gradient aggregator based on subspace estimation (Flag Aggregator, FA) is proposed, which treats aggregation as maximum likelihood estimation and incorporates regularization to resist Byzantine failures and noise from data augmentation.
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
Tri Dao (Princeton University)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelSequential
π― What it does: Improved the original FlashAttention and proposed FlashAttention-2, which adopts a better workload partitioning and parallel strategy to achieve faster and more efficient attention computation for long sequence Transformers.
FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
Seonghyeon Ye (Korean Advanced Institute of Science and Technology), Minjoon Seo (Korean Advanced Institute of Science and Technology)
CodeTransformerLarge Language ModelText
π― What it does: This paper presents FLASKβa fine-grained language model evaluation framework based on aligned skill sets. It first annotates the required skills, domains, and difficulty for each instruction, and then allows humans or LLMs to score from 1 to 5, supporting both standard and FLASK-HARD evaluation modes.
π― What it does: The Flow-to-Better (FTB) framework is proposed, which improves trajectories directly at the trajectory level using diffusion models, avoiding the pitfalls of traditional reward learning and TD training.
Forward $\chi^2$ Divergence Based Variational Importance Sampling
Chengrui Li (Georgia Institute of Technology), Anqi Wu (Georgia Institute of Technology)
CodeTabular
π― What it does: This paper proposes a new Variational Importance Sampling (VIS) method that directly estimates and maximizes the marginal log-likelihood of latent variable models using importance sampling, thereby achieving parameter learning.
π― What it does: A forward learning framework called FORWARDGNN is proposed, which trains graph neural networks using unidirectional forward propagation, avoiding the limitations of backpropagation.
π― What it does: This paper proposes a meta-optimizer FOSI, which enhances the convergence speed of first-order optimizers by splitting the objective function in orthogonal subspaces during each iteration, accelerating in one subspace using Newton's method and optimizing in another subspace with a baseline first-order optimizer.
π― What it does: A model-based reward conditional supervised learning method (MBRCSL) is proposed, which solves the problem of trajectory stitching and the need for Bellman completeness in traditional RCSL by learning a dynamics model and behavior policy for forward sampling.
π― What it does: The FreeNoise method is proposed, which utilizes noise rescheduling (local noise shuffling + window-based attention fusion) to achieve untuned long video generation, and introduces Motion Injection to support continuous action generation with multiple text conditions.
π― What it does: Proposes a Joint Multi-domain Pre-training (JMP) method that utilizes multi-task supervised pre-training to learn universal atomic-level representations across various chemical domains (molecules, catalysts, materials, etc.) and achieves multi-task performance improvement through fine-tuning.
Joan Puigcerver (Google DeepMind), Neil Houlsby (Google DeepMind)
CodeTransformerMixture of ExpertsImage
π― What it does: A fully differentiable sparse Transformer called Soft MoE is proposed, which allows each expert to handle only a portion of the weighted average tokens through soft allocation, achieving a high-capacity model with no significant additional computational cost.
Frozen Transformers in Language Models Are Effective Visual Encoder Layers
Ziqi Pang (University of Illinois Urbana-Champaign), Yu-Xiong Wang (University of Illinois Urbana-Champaign)
CodeRecognitionObject DetectionSegmentationAutonomous DrivingTransformerLarge Language ModelImageVideoMultimodalityPoint Cloud
π― What it does: Inserting a frozen Transformer block from a pre-trained large language model (LLM) into a visual encoder as a general visual encoding layer to enhance the performance of various visual tasks.
π― What it does: A fully hypergeometric convolutional neural network (HCNN) is proposed, which performs convolution, batch normalization, and polynomial logistic regression entirely within the Lorentz model of Riemannian manifolds, achieving a full-link hypergeometric representation from input to output.
π― What it does: For the camera-lidar multi-sensor fusion model, an attack is proposed using only adversarial stickers from the camera modality. A two-stage optimization framework is introduced: first, identify sensitive areas in the image, and then customize scene-based or target-based attacks according to the model's global or target sensitivity.
Future Language Modeling from Temporal Document History
Changmao Li (University of California), Jeffrey Flanigan (University of California)
CodeGenerationRecurrent Neural NetworkTransformerLarge Language ModelText
π― What it does: Proposes a future language modeling task and constructs various temporal language models based on historical texts to predict future texts.
GAFormer: Enhancing Timeseries Transformers Through Group-Aware Embeddings
Jingyun Xiao (Georgia Institute of Technology), Eva L Dyer
CodeClassificationTransformerTime Series
π― What it does: This paper proposes a Group Embedding method and constructs the GAFormer Transformer for feature extraction of multivariate time series data.
CodeLarge Language ModelPrompt EngineeringTextMultimodalityBenchmark
π― What it does: Designed and released the GAIA benchmark, which includes 466 real-world question-and-answer tasks for evaluating the reasoning, multimodal processing, and tool usage capabilities of general AI assistants.