International Conference on Learning Representations Β· 737 papers
Leveraging Large Language Models for Multiple Choice Question Answering
Joshua Robinson (Brigham Young University), David Wingate (Brigham Young University)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: By evaluating the multiple-choice answer symbol binding (MCSB) of large language models (LLMs) and replacing traditional cloze prompts (CP) with multiple-choice prompts (MCP), a systematic comparison of the performance of the two prompting methods on 20 multiple-choice question-answer datasets was conducted. It was found that models with high MCSB capability (such as Codex) showed significant improvement under MCP, nearly reaching or exceeding SOTA.
π― What it does: This paper proposes the LexMAE (Lexicon-Bottlenecked Masked AutoEncoder) model, which introduces a vocabulary importance bottleneck during the pre-training phase, allowing the masked language model to learn vocabulary weights that better meet retrieval needs, thus directly transferring to large-scale retrieval tasks.
π― What it does: A lightweight framework called LightGCL based on graph contrastive learning is proposed in the recommendation system, which enhances the global structure of the user-item interaction graph using singular value decomposition (SVD), and then performs contrastive learning with the original local information to improve recommendation performance under sparsity and popularity bias.
π― What it does: This paper proposes LilNetX, a unified end-to-end training framework that can simultaneously optimize model accuracy, disk storage size, and inference computation, achieving model compression and structured sparsity.
π― What it does: This study investigates the linear mode connectivity in text classification tasks, finding that different fine-tuning results fall into different basins, which are associated with various generalization strategies.
Jack Merullo (Brown University), Ellie Pavlick (Brown University)
CodeGenerationRetrievalTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: This paper proposes a single-layer linear mapping (LiMBeR) that directly projects the representations of a frozen image encoder into soft prompts for a language model, enabling the text model to generate image descriptions and answer visual questions.
π― What it does: This paper presents Liquid-S4, a sequence modeling framework that combines Linear Liquid Time-Constant Networks (LTC) with Structured State Space Models (S4).
Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learning
Chi Han (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
CodeKnowledge DistillationRepresentation LearningGraph Neural NetworkGraphBiomedical Data
π― What it does: This study investigates a representation that encodes local subgraph information of entities into differentiable logical functions (LERP) and embeds it into probabilistic logic rule learning to enhance knowledge graph completion.
π― What it does: A logic message passing network (LMPNN) based on pre-trained knowledge graph embeddings is proposed to solve complex queries in EFO-1;
LogicDP: Creating Labels for Graph Data via Inductive Logic Programming
Yuan Yang (Georgia Institute of Technology), Ali Payani (Cisco)
CodeClassificationExplainability and InterpretabilityComputational EfficiencyGraph Neural NetworkReinforcement LearningGraph
π― What it does: This paper proposes LOGICDP, a data programming-based framework that generates training labels for graph data using automatically learned logical rules, thereby training graph reasoning models.
Loss Landscapes are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent
Ping-yeh Chiang (University of Maryland), Tom Goldstein (University of Maryland)
CodeOptimizationConvolutional Neural NetworkImage
π― What it does: Research shows that even without using gradient information, zero-order optimizers (such as guess-check, pattern search, and random greedy search) can achieve comparable or even better generalization performance than SGD in over-parameterized models; the 'volume hypothesis' has been experimentally validated - good solutions occupy a larger volume in the parameter space, making them naturally selected by random search.
π― What it does: Proposes Long-tailed Prompt Tuning (LPT), which incorporates shared prompts and group-specific prompts into a frozen pre-trained ViT model, fine-tuning only a minimal number of parameters to adapt to long-tail classification tasks.
LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learning
Firas Al-Hafez (TU Darmstadt), Jan Peters (TU Darmstadt)
CodeReinforcement LearningSequential
π― What it does: A new inverse reinforcement learning algorithm, LS-IQ, is proposed, which improves learning stability through implicit reward regularization and effectively handles absorbing states.
π― What it does: A novel learning optimizer M-L2O is proposed, which obtains an initialization point that can quickly converge with only a few adaptive steps during testing through meta-learning in the training phase.
MA-BERT: Towards Matrix Arithmetic-only BERT Inference by Eliminating Complex Non-Linear Functions
Neo Wei Ming (Agency for Science Technology and Research), Tao Luo (Agency for Science Technology and Research)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes MA-BERT, which replaces all complex nonlinear functions (softmax, GELU, LayerNorm) in the Transformer model with implementations that only use matrix operations and ReLU to achieve more efficient inference.
Chao Pan (University of Illinois), Olgica Milenkovic (University of Illinois)
CodeFederated LearningSafty and PrivacyBiomedical Data
π― What it does: A machine unlearning framework for federated clustering is proposed, which includes Secure Compressed Multi-Set Aggregation (SCMA) and an efficient unlearning K-means++ initialization.
π― What it does: In this paper, the authors propose a multi-agent reinforcement learning framework called MACTA, which is used for the automatic learning and discovery of cache timing attack (CTA) strategies and corresponding detection strategies, and they have constructed a Gym environment, MA-AUTOCAT, that can realistically simulate CTAs.
π― What it does: A model-based reinforcement learning framework POMP is designed, which directly uses the learned continuous action space model as a planner, and proposes a Deep Differentiable Dynamic Programming (D3P) algorithm to optimize trajectories.
π― What it does: This paper proposes the use of a Bayesian model (adding probability distribution after given parameters) to enhance the transferability of adversarial attacks based on sub-models;
π― What it does: Designed and implemented the ManiSkill2 benchmark, which includes 20 types of manipulation task families, over 2000 object models, 4 million demonstration frames, and provides a unified Gym interface, convertible action space, bi-directional coupling of soft and rigid body GPU MPM simulation, and an efficient asynchronous rendering/rendering server framework.
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
Bencheng Liao (Huazhong University of Science and Technology), Chang Huang (Horizon Robotics)
CodeAutonomous DrivingTransformerPoint Cloud
π― What it does: This paper proposes MapTR, a structured end-to-end online vectorization framework for high-definition map construction based on Transformer;
π― What it does: This paper studies a novel markup-to-image generation task and proposes a fully data-driven rendering framework based on diffusion models.
π― What it does: A mask distillation method called MasKD based on learnable receptive tokens is proposed for feature distillation in dense prediction tasks such as object detection and semantic segmentation.
π― What it does: A pure mask image modeling method called ConMIM is proposed, which achieves mask completion pre-training through denoising contrastive learning, avoiding the use of offline image tokenizers.
π― What it does: The MASKED UNSUPERVISED SELF-TRAINING (MUST) method is proposed, which performs self-supervised fine-tuning on a pre-trained zero-shot classifier (such as CLIP) using unlabeled images to enhance classification performance.
π― What it does: A novel heteroscedastic classifier HET-XL is proposed, which can decouple the additional parameter count from the number of classes in large-scale multi-class classification problems and automatically learn the temperature hyperparameter.
Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning
Anton Bakhtin (Meta AI), Noam Brown (Meta AI)
CodeReinforcement LearningSequential
π― What it does: Developed and trained an AI for a non-verbal diplomacy game named Diplodocus, which combines human imitation with reinforcement learning using RL-DiL-piKL, and achieved first place in 200 games against humans.
CodeClassificationDrug DiscoveryGraph Neural NetworkLarge Language ModelGraphBiomedical Data
π― What it does: This study investigates how to use protein language models and molecular graph neural networks to predict the activation of olfactory receptors (OR) by odor molecules, and to determine whether a molecule can activate a given OR through a binary classification approach.
π― What it does: A minimal cost mixed human-machine active labeling framework (MCAL) is proposed, which jointly selects samples for human labeling and model training, and uses the trained classifier to automatically label the remaining samples, thereby minimizing the total labeling and training costs while satisfying the error threshold.
π― What it does: A method called MECTA is proposed to reduce memory usage during gradient-based continual learning while maintaining or even improving out-of-distribution (OOD) recognition performance.
MEDICAL IMAGE UNDERSTANDING WITH PRETRAINED VISION LANGUAGE MODELS: A COMPREHENSIVE STUDY
Ziyuan Qin (West China Biomedical Big Data Center), Kang Li (West China Biomedical Big Data Center)
CodeObject DetectionTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelImageBiomedical Data
π― What it does: This paper studies the transfer of pre-trained vision-language models (such as GLIP) from natural images to the medical imaging domain, enhancing object detection performance through the design or automatic generation of medical prompts.
π― What it does: Proposed the Memory Gym benchmark, which includes three partially observable, memory-requiring 2D tasks (Mortar Mayhem, Mystery Path, Searing Spotlights) with adjustable difficulty;
Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learning
Ivona Najdenkoska (University of Amsterdam), Marcel Worring (University of Amsterdam)
CodeMeta LearningTransformerVision Language ModelImageTextMultimodality
π― What it does: A multi-modal few-shot learning framework based on meta-learning is proposed, which connects frozen large-scale visual and language models through a lightweight meta-mapper and quickly adapts to new tasks with a small number of samples.
Wonho Bae (University of British Columbia), Gabriel L. Oliveira (Borealis AI)
CodeMeta LearningTransformerTime SeriesSequential
π― What it does: A meta-learning framework is proposed that treats each event sequence as an independent task, using Neural Processes (NP) to model temporal point processes, and introduces latent variables and cross-attention modules based on this.
Keegan Harris (Carnegie Mellon University), Tuomas Sandholm (Carnegie Mellon University)
CodeMeta LearningReinforcement Learning from Human FeedbackReinforcement LearningSequentialOrdinary Differential Equation
π― What it does: This paper studies meta-learning methods in multi-task game environments, providing theoretical convergence guarantees for zero-sum games, general-sum games, potential games, and Stackelberg games, and validates its acceleration effects in poker endgame experiments.
π― What it does: This paper proposes a meta-prediction model named DaSS, which is used to quickly predict the final accuracy of the student network on unseen datasets in the knowledge distillation (KD) scenario, thereby achieving efficient neural architecture search (DaNAS) across datasets and teachers.
π― What it does: A mid-vision feedback mechanism based on context is proposed, which achieves adaptive adjustment of high-level category expectations through linear transformations on mid-level features.
π― What it does: This paper studies and addresses the min-max multi-objective bilevel optimization problem, proposing the single-loop gradient descent-ascent algorithm MORBiT to find robust solutions in robust machine learning tasks such as representation learning and hyperparameter optimization.
π― What it does: A Reward Gap Minimization (RGM) framework is proposed, which can simultaneously correct rewards and learn optimal policies in offline reinforcement learning scenarios with imperfect rewards.
Mini-batch $k$-means terminates within $O(d/\epsilon)$ iterations
Gregory Schwartzman (Japan Advanced Institute of Science and Technology)
CodeOptimization
π― What it does: The study proves that under appropriate batch sizes, mini-batch k-means can terminate within O(d/Ξ΅) iterations with high probability when using early stopping conditions.
π― What it does: The paper introduces a regularization based on representation (self-supervised features) in deep networks to suppress the memorization phenomenon when learning from noisy labels.
π― What it does: A weight initialization method for peer MLP based on converged weights (MLPInit) is proposed, which directly uses the trained PeerMLP weights as initialization for GNNs, thereby accelerating GNN training and improving prediction performance.
π― What it does: The MOAT (Mobile Convolution with Attention) module is proposed, which combines the MBConv (inverted bottleneck) of MobileNetV2 with the self-attention of Transformer to construct a series of models ranging from small to large.
π― What it does: This paper proposes MocoSFL, a cross-client self-supervised learning framework based on Split Federated Learning, which addresses bottlenecks in computation, memory, and data volume while achieving high accuracy.
π― What it does: A Bayesian optimization algorithm MCBO based on a complete Structural Causal Model (SCM) is proposed, utilizing Gaussian processes to model each node and making optimal intervention decisions across the entire model space through the UCB principle, providing a non-asymptotic sublinear upper bound on cumulative regret.
Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts
Zhitong Gao (ShanghaiTech University), Xuming He (ShanghaiTech University)
CodeSegmentationMixture of ExpertsMultimodalityBiomedical DataComputed Tomography
π― What it does: A multi-modal uncertainty modeling framework MoSE is proposed, which captures the inherent stochastic uncertainty of data in semantic segmentation and outputs multiple weighted segmentation results.
Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization
Jivat Neet Kaur (Microsoft Research), Amit Sharma (Microsoft Research)
CodeDomain AdaptationImage
π― What it does: Researches the domain generalization problem under multi-attribute distribution shift and proposes the Causal Graph-based Adaptive Constraint Minimization (CACM) algorithm;
π― What it does: A Continuous Convolutional Neural Network (CCNN) is proposed, which can handle data of arbitrary resolution, dimension, and length without changing the network structure.
π― What it does: This study investigates how to accelerate visual model-based reinforcement learning using a small number of demonstrations, significantly improving sampling efficiency in sparse reward tasks.
π― What it does: The concept of 'Moderate Coreset' and its implementation method are proposed, using deep representation Euclidean distance and score median to construct a data subset to improve model performance in efficient data learning.
π― What it does: We propose Mole-BERT, which significantly enhances molecular property prediction and retrieval performance by introducing a context-aware tokenizer based on VQ-VAE and new Masked Atoms Modeling along with Triplet Masked Contrastive Learning for pre-training molecular GNNs.
π― What it does: This paper proposes a Stiefel matrix optimizer based on continuous-discrete dynamics, which features natural momentum, structure preservation, and low overhead, and applies it to the orthogonalization of ViT attention heads and low-dimensional projections of high-dimensional optimal transport (OT).
More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization
Jiangxing Wang (Peking University), Zongqing Lu (Peking University)
CodeReinforcement LearningBenchmark
π― What it does: A multi-agent conditional policy decomposition (MACPF) framework is proposed, which explicitly considers inter-agent dependencies under centralized training while achieving decentralized execution.
π― What it does: This paper proposes a method using sparsification and kernel decomposition, successfully expanding the convolution kernel size from 31Γ31 to 51Γ51 and even 61Γ61, and constructs the SLaK network.
π― What it does: The Mosaic Representation Learning framework MosRep is proposed, which enriches the background information of small crops by introducing mosaic views in self-supervised visual pre-training, thereby enhancing the quality of visual representations.
MPCFORMER: FAST, PERFORMANT AND PRIVATE TRANSFORMER INFERENCE WITH MPC
Dacheng Li (Carnegie Mellon University), Hao Zhang (University of California, Berkeley)
CodeSafty and PrivacyComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
π― What it does: A Transformer inference framework called MPCFORMER is proposed, which utilizes MPC and knowledge distillation to significantly accelerate private inference while maintaining performance.
π― What it does: This paper proposes a multi-domain image generation and unsupervised image-to-image translation method based on recognizability constraints, which can learn the complete joint distribution from single-domain edge distributions, thereby generating image pairs that are content-consistent and stylistically diverse.
Multi-Objective Reinforcement Learning: Convexity, Stationarity and Pareto Optimality
Haoye Lu (University of Waterloo), Yaoliang Yu (University of Waterloo)
CodeOptimizationReinforcement LearningSequential
π― What it does: This paper rigorously analyzes the convexity and Pareto optimality of the policy-induced value function in multi-objective reinforcement learning, proposes the CAPQL algorithm that enhances concavity in rewards, and verifies its superiority.
π― What it does: This paper proposes PARETOGNN, a framework for achieving task generalization in graph neural networks through multi-task self-supervised learning.
CodeTransformerVision Language ModelImageTextMultimodalityGraphBenchmark
π― What it does: This paper proposes a new multimodal analogy reasoning task that combines images, text, and knowledge graphs to predict missing entities by performing analogy reasoning in the context of knowledge graphs.
Multivariate Time-series Imputation with Disentangled Temporal Representations
SHUAI LIU (Nanyang Technological University), YUE JIANG (Nanyang Technological University)
CodeData SynthesisAnomaly DetectionOptimizationTime Series
π― What it does: A multivariate time series missing value imputation method called TIDER is proposed, which utilizes low-rank matrix decomposition and constructs interpretable temporal representations.
MultiViz: Towards Visualizing and Understanding Multimodal Models
Paul Pu Liang (Carnegie Mellon University), Ruslan Salakhutdinov (Carnegie Mellon University)
CodeExplainability and InterpretabilityMultimodality
π― What it does: A multimodal model interpretability tool named MULTIVIZ is proposed, which visualizes and analyzes the internal mechanisms of the model in four stages (single-modal importance, cross-modal interaction, feature representation, prediction composition);
π― What it does: This paper studies the problem of large-scale graph node classification and proposes NAGphormer, which represents each node as a token sequence of multi-hop neighborhood information and uses a Transformer for learning.
π― What it does: A self-supervised object segmentation framework based on NeRF, called NeRF-SOS, is proposed, which can segment objects in complex real scenes from arbitrary viewpoints.
π― What it does: This study investigates a bidirectional communication game to explore whether neural network agents can spontaneously generate a turn-taking agreement and assess its impact on task completion.
π― What it does: This paper proposes a framework based on Neural Causal Models (NCM) to address the identification and estimation problem of counterfactual reasoning starting from observational (L1) and experimental (L2) data as well as a causal graph G; it also provides feasible training and inference algorithms (Alg.1-3) and implements a GAN-based NCM training method.
π― What it does: This paper proposes a framework for Few-Shot Class Incremental Learning (FSCIL) based on the concept of neural collapse, where the classifier weights for all categories are pre-set to a simplex equiangular tight frame (simplex ETF). During training, a dot-regression loss drives the feature vectors to approximate these fixed prototypes, thereby avoiding catastrophic forgetting caused by the misalignment of features and classifiers.
Neural Design for Genetic Perturbation Experiments
Aldo Pacchiano (Microsoft Research), Luis Voloch (Immunai)
CodeOptimizationDrug DiscoveryTabularBiomedical Data
π― What it does: A noise-free batch bandit optimization method based on the Optimistic Arm Elimination (OAE) principle is proposed, specifically designed to quickly discover gene perturbations that maximize cellular phenotypes under a limited experimental budget.
π― What it does: Conducted large-scale experiments on the generalization ability of different neural networks in sequence prediction tasks, using 20,910 models and 15 tasks, exploring their correspondence with the Chomsky hierarchy.
Alexander Korotin (Skolkovo Institute of Science and Technology), Evgeny Burnaev (Skolkovo Institute of Science and Technology)
CodeImage TranslationOptimizationImage
π― What it does: A neural network-based algorithm is proposed for calculating the optimal transport graph and plan for both strong and weak transport costs.
Neural-based classification rule learning for sequential data
Marine Collery (IBM France Lab), Remy Kusters (IBM Research)
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkTabularSequential
π― What it does: A differentiable binary neural network CR2N is proposed, which can simultaneously learn local and global interpretable classification rules in sequential data.
π― What it does: This paper experimentally compares information-theoretic multi-skill reinforcement learning methods (DIAYN, DADS, SMERL, etc.) with quality-diversity (QD) evolutionary methods (MAP-ELITES, PGA-MAP-ELITES, AURORA, PGA-AURORA) in terms of skill discovery across various continuous control and exploration environments, and proposes a unified benchmarking framework.
π― What it does: A framework for online continual learning, MuFAN, is proposed, aiming to enhance both the stability and plasticity of the model, addressing the stability-plasticity dilemma in online CL.
π― What it does: A non-parametric external sample synthesis (NPOS) framework is proposed, which automatically generates artificial OOD data during training and learns the decision boundary between ID and OOD through uncertainty loss.
Not All Tasks Are Born Equal: Understanding Zero-Shot Generalization
Jing Zhou (Tsinghua University), Zhilin Yang
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: This paper reveals that the zero-shot generalization performance of T0 multi-task prompt-based pre-training is mainly determined by a small number of key tasks, especially QA-type tasks, through experiments. It also proposes a task resampling method based on the generalization results between task pairs to enhance model performance.
Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
Huayu Chen (Tsinghua University), Jun Zhu (Tsinghua University)
CodeReinforcement LearningDiffusion modelTabular
π― What it does: An offline reinforcement learning method is proposed, which uses a diffusion generative model to learn high-fidelity behavior policies, and then generates executable policies through a combination of importance sampling and action evaluation models.
Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
Haoran Xu (Institute for AI Industry Research), Xianyuan Zhan (Institute for AI Industry Research)
CodeReinforcement LearningTabular
π― What it does: An Implicit Value Regularization (IVR) framework is proposed, and based on this framework, two sample-based offline reinforcement learning algorithms, SQL (Sparse Q-Learning) and EQL (Exponential Q-Learning), are designed.
Ziming Liu (Massachusetts Institute of Technology), Max Tegmark (Massachusetts Institute of Technology)
CodeRepresentation LearningImageText
π― What it does: This paper explains and induces the grokking phenomenon on algorithmic and non-algorithmic datasets by analyzing the weight norm of neural networks and the loss curves.
On Compositional Uncertainty Quantification for Seq2seq Graph Parsing
Zi Lin (University of California San Diego), Jingbo Shang (University of California San Diego)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextGraph
π― What it does: This paper proposes the Graph Autoregressive Process (GAP) framework and the Compositional Expected Calibration Error (CECE) metric, which can map the sequence probabilities of seq2seq models to the conditional probabilities of graph nodes/edges, thereby quantifying and evaluating the compositional uncertainty in graph parsing tasks.
On Explaining Neural Network Robustness with Activation Path
Ziping Jiang (Lancaster University)
CodeExplainability and InterpretabilityAdversarial AttackConvolutional Neural NetworkImage
π― What it does: By decomposing the activation patterns of neural networks, the network is divided into fixed paths and floating paths. The impact of floating paths on robustness is analyzed, and a mechanism to suppress floating paths (SC-RFP) is introduced in randomized smoothing, thereby enhancing the verifiable robustness of the model.
Danqing Wang (ByteDance Research), Hao Zhou (Institute for AI Industry Research)
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningBiomedical DataBenchmark
π― What it does: This paper studies the performance of protein and antibody-specific pre-trained language models on different specificity antibody tasks, proposing the first benchmark ATUE covering four types of antibody tasks, and designing and training the EATLM model based on antibody evolutionary mechanisms.
π― What it does: This paper proves that Graph Neural Networks (GNNs) can express and approximate the feasibility, optimal objective value, and optimal solution of Linear Programming (LP), establishing a theoretical foundation for using GNNs in learning optimization.
π― What it does: A learnable Contrast-Invariant Image Transformation (CoIT) has been designed and implemented to enhance the data efficiency of visual reinforcement learning.
π― What it does: A framework called COLT has been developed to utilize external unlabeled OOD data for self-supervised long-tail learning, addressing the issue of traditional SSL's excessive focus on the main class under long-tail distributions.
Devon Jarvis (University of the Witwatersrand), Andrew M Saxe (University College London)
CodeConvolutional Neural NetworkImage
π― What it does: This paper studies the specialization capability of neural modules in datasets and verifies through theoretical analysis and experiments whether modular networks can achieve systematic generalization.
On the Usefulness of Embeddings, Clusters and Strings for Text Generation Evaluation
Tiago Pimentel (University of Cambridge), Ryan Cotterell (ETH ZΓΌrich)
CodeGenerationTransformerLarge Language ModelText
π― What it does: This paper explores why the MAUVE metric is highly correlated with human judgments through theoretical and experimental analysis, ultimately finding that its core advantage lies in using clustering distributions based on pre-trained model embeddings to approximate string distributions, rather than the newly proposed AUC Divergence itself.
On the Word Boundaries of Emergent Languages Based on Harris's Articulation Scheme
Ryo Ueda (University of Tokyo), Yusuke Miyao (University of Tokyo)
CodeRecurrent Neural NetworkSequential
π― What it does: Investigate whether Harris's pronunciation scheme (HAS) holds in simulated signal games, and evaluate whether the word boundaries of generated language have semantic significance through unsupervised entropy-based word boundary detection.
One Transformer Can Understand Both 2D & 3D Molecular Data
Shengjie Luo (Peking University), Di He (Peking University)
CodeDrug DiscoveryTransformerMultimodalityGraph
π― What it does: Design a Transformer-M model that can simultaneously handle molecular 2D graphics and 3D geometric data, achieving cross-modal learning with a single model.
π― What it does: Designed and validated the One-Pixel Shortcuts (OPS) technique to generate unlearnable samples, misleading deep networks during training and resulting in poor performance on clean test sets;
π― What it does: The research task addresses the prediction bias in task-agnostic continual learning, explains the 'freshness bias' caused by experience replay, and proposes an online bias correction method (OBC) that can continuously correct bias during the learning process.
Online Boundary-Free Continual Learning by Scheduled Data Prior
Hyunseo Koh (Gwangju Institute of Science and Technology), Jonghyun Choi (Yonsei University)
CodeKnowledge DistillationImage
π― What it does: Proposes a boundary-free online continual learning framework to address the problem of data stream learning without task boundaries.