π― What it does: Align expert demonstrations with reward-free offline trajectories using optimal transport, automatically generating rewards for each step, allowing offline RL to directly learn good policies.
π― What it does: This paper proposes a continuous learning algorithm OSCL based on supervised principal component analysis (SPCA) and provides an analytical classification error of the algorithm within a high-dimensional statistical theoretical framework. Furthermore, it theoretically avoids catastrophic forgetting through label optimization and achieves flexible adjustment of task weights.
OPTQ: Accurate Quantization for Generative Pre-trained Transformers
Elias Frantar (IST Austria), Dan Alistarh (IST Austria)
CodeGenerationCompressionTransformerLarge Language ModelText
π― What it does: A one-time post-training quantization method called OPTQ is proposed, which can compress large Transformer models (up to 175B parameters) to 3/4 bits while maintaining almost no accuracy loss.
Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing
Yunchong Song (Shanghai Jiao Tong University), Zhouhan Lin (Shanghai Jiao Tong University)
CodeGraph Neural NetworkGraph
π― What it does: Proposes Ordered GNN, which arranges the neurons of node representations according to the hierarchical structure of a root tree, forming an ordered message passing mechanism;
π― What it does: Proposes the Only-Train-Once v2 framework, which trains and compresses any deep neural network in a single pass without the need for pre-training or fine-tuning, automatically partitions Zero-Invariant Groups (ZIG) and constructs a compressed model.
π― What it does: A method for OOD detection based on the energy of modern Hopfield networks, called HE, is proposed, and a simplified version without hyperparameters, SHE, is derived based on it, using representative features from the training set ID as patterns for store-then-compare discrimination.
π― What it does: A domain-agnostic time series OOD representation learning method called DIVERSIFY is proposed, which utilizes adversarial self-supervised pseudo-domain label iterative mining to normalize the latent sub-distributions in the data.
π― What it does: A curriculum learning method based on uncertainty and temporal distance perception is proposed, allowing the agent to automatically progress towards the desired outcome without environmental rewards and only given target examples.
π― What it does: A novel unsupervised, architecture-agnostic semi-nonnegative tensor decomposition method is proposed, which jointly learns interpretable components and appearances in GAN feature maps, enabling pixel-level local editing and concept localization.
π― What it does: Systematically explore the design space of parameter-efficient fine-tuning, identifying and validating a series of design patterns, and applying them to various pre-trained models and tasks.
Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalization
Yongqiang Chen (Chinese University of Hong Kong), James Cheng (Hong Kong Baptist University)
CodeDomain AdaptationOptimizationImageBenchmark
π― What it does: In this work, the authors propose a new optimization framework called PAIR (Pareto Invariant Risk Minimization), aimed at addressing the optimization dilemma in the OOD (Out-of-Distribution) generalization process. By adopting a multi-objective optimization (MOO) perspective, it jointly considers ERM and OOD objectives, employing an adaptive weighting mechanism to dynamically balance both during the optimization process, ultimately achieving the recovery and enhancement of the original IRM objective. Additionally, the authors designed a corresponding model selector PAIR-s to better capture OOD performance during the validation phase.
π― What it does: By combining human prior knowledge with end-to-end learning, a part-based model is introduced to enhance the robustness of image classification.
π― What it does: This paper proposes ADCLR, a framework for contrastive learning using query crops in visual Transformers, combining global and local contrastive losses, and employing unidirectional cross-attention to avoid collapse.
π― What it does: Proposes PatchDCT, a multi-stage patch-based segmentation refinement framework based on compressed vectors for high-quality instance segmentation;
π― What it does: A personalized federated learning framework called PerFedMask is proposed, which utilizes optimized masking vectors to freeze parts of the network based on device computing capabilities and fine-tunes after training.
π― What it does: A personalized reward learning framework based on Interaction Grounded Learning (IGL) called IGL-P is proposed in the recommendation system, which can learn each user's implicit satisfaction through user interaction feedback without explicit reward signals, thereby directly optimizing the user's real experience.
π― What it does: An unsupervised low-dimensional embedding method called phase2vec has been developed, which can learn physically meaningful dynamic system representations from vector field data and can be used to reconstruct system equations.
π― What it does: A Planckian Jitter color augmentation method based on a physical blackbody radiation model is proposed for self-supervised learning.
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models
Guangyi Chen (Carnegie Mellon University), Kun Zhang
CodeClassificationDomain AdaptationOptimizationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
π― What it does: By learning multiple descriptive prompts for each category within the CLIP framework and using optimal transport (Sinkhorn algorithm) to achieve fine-grained matching between visual features and text prompts, the issue of feature redundancy caused by the convergence of a single prompt is avoided.
π― What it does: This paper proposes a Policy Expansion framework, which freezes the policy obtained from offline training and combines it with newly learned policies to form a policy set. It achieves the transition from offline to online reinforcement learning through value-guided adaptive combination, realizing the retention of offline knowledge and the synergy of online exploration.
π― What it does: A PPGeo pre-training framework is proposed, utilizing self-supervised geometric modeling (estimating depth, camera intrinsics, and pose) in a two-stage training process, ultimately allowing the visual encoder to predict future poses based solely on a single frame, thereby learning visual representations highly relevant to driving decisions.
Mert Yuksekgonul (Stanford University), James Zou (Stanford University)
CodeExplainability and InterpretabilityData-Centric LearningContrastive LearningImageMultimodality
π― What it does: This paper proposes the Post-Concept Bottleneck Model (PCBM), which transforms any pre-trained network into an interpretable concept bottleneck model and restores performance through residual modules.
π― What it does: A self-supervised pre-training method based on denoising 3D molecular structures is proposed, and the pre-trained model is used for molecular property prediction.
π― What it does: Proposes the Feature Conformal Prediction (Feature CP) framework, which performs distributed confidence interval inference in the semantic feature space of deep models rather than the traditional output space, and provides corresponding non-conformity scores and confidence interval estimation/detection methods.
π― What it does: A knowledge transfer method under multi-source domains with very few target samples (few-shot) is proposedβProgressive Mix-up (P-Mixup).
π― What it does: This paper proposes the Progressively Compressed Auto-Encoder (PCAE), which significantly reduces the number of reconstruction targets and improves training efficiency by layer-wise dropping redundant tokens during self-supervised pre-training, retaining only the information necessary for reconstruction.
Chenglei Si (University of Maryland), Lijuan Wang (Microsoft)
CodeTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: This study proposes various prompting strategies to enhance GPT-3's performance across four reliability dimensions (generalization, bias, fairness, calibration, and factuality).
Protein Representation Learning via Knowledge Enhanced Primary Structure Reasoning
Hong-Yu Zhou (University of Hong Kong), Yizhou Yu (University of Hong Kong)
CodeRepresentation LearningDrug DiscoveryProtein Structure PredictionTransformerLarge Language ModelBiomedical Data
π― What it does: Pre-training on protein sequences utilizes relational and attribute words from knowledge graphs, employing cross-modal attention for token-level knowledge probing of each amino acid, thereby learning more semantically rich protein representations.
π― What it does: This paper proposes a new framework for the joint design of protein sequences and structures called PROTSEED, which directly generates target sequences and structures from random initialization using a deformable iterative translation approach.
Prototypical Calibration for Few-shot Learning of Language Models
Zhixiong Han (Microsoft Research), Furu Wei (Microsoft Research)
CodeClassificationTransformerLarge Language ModelText
π― What it does: A prototype-based calibration method is provided for few-shot learning in GPT-style language models, which estimates the prototype clusters for each category and adjusts the decision boundaries to improve classification accuracy.
Provable Defense Against Geometric Transformations
Rem Yang (University of Illinois Urbana Champaign), Gagandeep Singh (University of Illinois Urbana Champaign)
CodeClassificationAutonomous DrivingImage
π― What it does: A deterministic verifiable geometric robustness training framework (CGT) is proposed, along with a GPU-accelerated geometric robustness verifier (FGV), achieving a complete process that directly embeds geometric robustness guarantees during training.
Q-Pensieve: Boosting Sample Efficiency of Multi-Objective RL Through Memory Sharing of Q-Snapshots
Wei Hung (National Yang Ming Chiao Tung University), Xi Liu (Meta AI)
CodeReinforcement Learning
π― What it does: The Q-Pensieve algorithm is proposed, which achieves policy-level knowledge sharing by using Q-snapshots in multi-objective reinforcement learning, significantly improving sample efficiency.
π― What it does: Reformulate the few-shot intent detection task as a question-answer retrieval task, using the user's utterance as the question and the intent name as the answer, employing a dual-encoder and token-level late interaction for matching.
π― What it does: This paper proposes a method to regress QUBO matrices using neural networks, solving them with quantum annealing, and achieving end-to-end backpropagation through contrastive loss, thus eliminating the need for manual derivation of QUBO;
π― What it does: This study investigates the impact of label errors on model fairness metrics and proposes a label error identification and automatic correction method based on influence functions.
Quantile Risk Control: A Flexible Framework for Bounding the Probability of High-Loss Predictions
Jake Snell, Richard Zemel (Columbia University)
CodeClassificationObject DetectionImageText
π― What it does: This paper proposes a distribution-free Quantile Risk Control (QRC) framework that provides strict upper bounds for quantile risk measures (such as VaR, CVaR, interval VaR) for predictive models, and can simultaneously select and control multiple models.
π― What it does: This paper proposes a method for recovering compressed sensing signals that have been quantized (including 1-bit, 2-bit, 3-bit, etc.) using a pre-trained score-based generative model (SGM), referred to as QCS-SGM.
π― What it does: A rarity score is proposed, which can assess the rarity of a single generated image, generative models, and similar datasets within the distribution of authenticity.
Re-calibrating Feature Attributions for Model Interpretation
Peiyu Yang (University of Western Australia), Ajmal Saeed Mian
CodeExplainability and InterpretabilityConvolutional Neural NetworkImage
π― What it does: Recalibrated the path integral-based feature attribution method to make the explanations more theoretically aligned and no longer require taking absolute values.
π― What it does: This paper proposes transferring the model's prior knowledge to the optimizer, designing a Gradient Re-parameterization method and implementing RepOptimizers, further validating its effectiveness on the VGG-style pure network RepOpt-VGG.
Zhiqing Sun (Google Research), Denny Zhou (Google Research)
CodeGenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
π― What it does: A 'recite-and-answer' paradigm is proposed, where a large language model recalls relevant knowledge paragraphs from its own weights and then answers questions based on the recalled content, forming the 'RECITE' framework.
Recon: Reducing Conflicting Gradients From the Root For Multi-Task Learning
Guangyuan SHI, Xiao-Ming Wu (Hong Kong Polytechnic University)
CodeOptimizationImage
π― What it does: This paper proposes the Recon method, which reduces gradient conflicts in multi-task learning and improves performance by first calculating the task gradient conflict scores for each layer and selecting the most severely conflicted shared layers to convert them into task-specific layers.
Red PANDA: Disambiguating Image Anomaly Detection by Removing Nuisance Factors
Niv Cohen (Hebrew University of Jerusalem), Yedid Hoshen (Hebrew University of Jerusalem)
CodeAnomaly DetectionContrastive LearningImage
π― What it does: This paper proposes an anomaly detection method using known irrelevant attribute labels for domain-supervised decomposition (Red PANDA). It learns representations that are insensitive to noise factors through contrastive learning and reconstruction loss, and employs kNN density estimation for anomaly scoring.
π― What it does: A new relational attention mechanism is proposed, enabling the Transformer to simultaneously handle features of nodes and edges, thus working directly on graph structures;
π― What it does: A method called REPAIR is proposed to address the variance collapse problem that occurs during linear interpolation between two aligned networks, significantly reducing performance barriers during the interpolation process.
π― What it does: A gradient scale reparameterization method that does not change the network structure is proposedβSpatial Gradient Scaling (SGS), which enhances the model's generalization performance by dynamically adjusting the learning rates at different positions of the convolutional kernel based on the spatial mutual information of the feature maps.
Rethinking Symbolic Regression: Morphology and Adaptability in the Context of Evolutionary Algorithms
Kei Sen Fong (National University of Singapore), Mehul Motani (National University of Singapore)
CodeOptimizationTransformerLarge Language ModelTabular
π― What it does: A minimal genetic programming symbolic regression algorithm based on Terminal Walking Language Model (TW-MLM) and a variable adaptive fitness function (alternating between MSE and MSEDI) is proposed and implemented, integrated into the traditional GP framework.
π― What it does: A new dynamic adversarial contrastive learning method, DYNACL, is proposed to address the dilemma of data augmentation intensity in self-supervised adversarial training.
Rethinking the Expressive Power of GNNs via Graph Biconnectivity
Bohang Zhang (Peking University), Di He (Peking University)
CodeGraph Neural NetworkTransformerGraph
π― What it does: This paper systematically evaluates the expressive power of existing GNNs from the perspective of graph biconnectivity, proving that most mainstream models cannot identify cut vertices/cut edges, and proposes a distance-based Generalized Distance WeisfeilerβLehman (GD-WL) framework along with its Transformer implementation Graphormer-GD, which theoretically possesses complete expressive power for all biconnectivity problems.
π― What it does: A retrieval-based controllable molecular generation framework, RetMol, is proposed, which guides a pre-trained generative model to generate new molecules that meet target properties using a small number of example molecules that satisfy design conditions through retrieval and information fusion, without the need for task-specific fine-tuning.
π― What it does: This paper proposes a reversible column network (RevCol) - a novel network architecture composed of multiple column sub-networks that achieves feature decoupling at multiple levels through multi-level reversible connections while maintaining information integrity, applicable to CNNs and Transformers.
π― What it does: A novel fine-tuning strategy LP-FT-FB is proposed, which combines linear probing with Firth bias correction to efficiently and unbiasedly fine-tune pre-trained feature extractors under few samples, thereby enhancing the performance of few-shot learning.
π― What it does: Achieve co-training of clean images and adversarial samples on Vision Transformer using a minimal amount of domain-specific parameters (such as classification token), and realize a smooth trade-off between clean accuracy and robustness without increasing computational cost through 'adversarial model soup'.
Revisiting Graph Adversarial Attack and Defense From a Data Distribution Perspective
Kuan Li (Chinese Academy of Sciences), Qing He (Chinese Academy of Sciences)
CodeAdversarial AttackGraph Neural NetworkGraph
π― What it does: Re-examine graph structure attacks and defenses from the perspective of data distribution, propose a distribution shift formula and theoretical explanation, and provide a series of practical techniques, ultimately improving attack (fast attack) and defense (self-training defense) methods.
π― What it does: Reassessing the population size effect in multi-agent communication, it is found that standard fully connected training leads to co-adaptation between the receiver and multiple senders, resulting in a lack of language uniformity. A partitioned training protocol is proposed, restricting the receiver to co-adapt only with its own sender, and explicitly incorporating the goals of internal communication and mutual intelligibility to promote consistency and composability of group language.
REVISITING PRUNING AT INITIALIZATION THROUGH THE LENS OF RAMANUJAN GRAPH
Duc N.M Hoang (University of Texas at Austin), Zhangyang Wang (University of Texas at Austin)
CodeConvolutional Neural NetworkImage
π― What it does: This paper explores the feasibility of Pruning at Initialization (PaI) during the network initialization phase, analyzing the relationship between the topology and performance of sparse networks using the properties of Ramanujan graphs, and proposes two new metrics, IMDB and NaRC, to evaluate the connectivity and randomness of sparse networks.
π― What it does: This paper proposes a reinforcement learning-based graph matching framework RGM, which employs serialized node matching and a revocable action mechanism to address the Lawler QAP problem with outliers.
Rhino: Deep Causal Temporal Relationship Learning with History-dependent Noise
Wenbo Gong (Microsoft Research), Nick Pawlowski (Microsoft Research)
CodeTime SeriesSequential
π― What it does: A temporal causal relationship learning framework named Rhino is proposed, which can simultaneously handle nonlinear relationships, instantaneous effects, and historical dependency noise.
Risk-Aware Reinforcement Learning with Coherent Risk Measures and Non-linear Function Approximation
Thanh Lam (National University of Singapore), Patrick Jaillet (Massachusetts Institute of Technology)
CodeReinforcement LearningTime SeriesFinance Related
π― What it does: This paper proposes a risk-aware reinforcement learning framework for unknown MDPs, aiming to minimize the risk of low rewards, using consistent risk measures combined with nonlinear function approximation.
RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch
Yiqin Tan (Tsinghua University), Longbo Huang (Tsinghua University)
CodeReinforcement LearningSequential
π― What it does: A framework called RLx2 has been developed to train sparse deep reinforcement learning models from scratch, achieving almost no performance loss at high sparsity rates.
Weijie Liu (Zhejiang University), Hui Qian (Zhejiang University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: A robust graph dictionary learning framework (RGDL) is proposed, with the core being the definition and efficient computation of the robust Gromov-Wasserstein distance (RGWD) to measure the differences between graphs with structural noise, and to learn a graph dictionary that can represent noisy graphs using this distance.
Robust Multivariate Time-Series Forecasting: Adversarial Attacks and Defense Mechanisms
Linbo Liu (Amazon Web Services), Luke Huan (Amazon Web Services)
CodeOptimizationAdversarial AttackRecurrent Neural NetworkTime Series
π― What it does: This paper studies the robustness of multivariate probabilistic prediction models under adversarial attacks and proposes a sparse indirect attack method along with two defense mechanisms.
π― What it does: This paper proposes ShiftMatch, a training data-related likelihood aimed at enhancing the robustness of Bayesian neural networks against input corruption.
π― What it does: This paper proposes the ROCO framework to evaluate the robustness of combinatorial optimization solvers on graphs when faced with perturbations. It modifies problem instances using the 'no bad optimal cost' constraint, which has no optimal solution, and generates hard instances using black-box attackers such as reinforcement learning. Systematic experiments are conducted on four classic CO tasks (DAG scheduling, ATSP, maximum coverage, MCSCC) with various solvers (traditional heuristics, learning-based, MILP).
π― What it does: This paper studies a robust semi-supervised learning method called RoPAWS, which can train on unorganized unlabeled data and avoids excessive confidence of PAWS on OOD samples.
RPM: Generalizable Multi-Agent Policies for Multi-Agent Reinforcement Learning
Wei Qiu (Nanyang Technological University), Zhongwen Xu (Sea AI Lab)
CodeReinforcement LearningBenchmark
π― What it does: The Ranked Policy Memory (RPM) method is proposed to enhance the generalization ability in unseen agent behavior scenarios in multi-agent reinforcement learning by saving and randomly sampling historical policies of different performance levels.
Sampling with Mollified Interaction Energy Descent
Lingxiao Li (Massachusetts Institute of Technology), Justin Solomon (Massachusetts Institute of Technology)
CodeOptimizationTabular
π― What it does: An optimized sampling method called Fuzzy Interaction Energy Descent (MIED) is proposed, which approximates the target distribution by minimizing the Fuzzy Interaction Energy (MIE), applicable to both unconstrained and constrained sampling problems.
π― What it does: A new knowledge distillation method called DiSK is proposed, which filters difficult and easy samples during the student training process using a guiding function generated by the teacher, achieving more effective knowledge transfer in scenarios with significant capacity differences between the student and teacher.
π― What it does: This paper proposes a scalable batch deep Bayesian active learning framework called BatchβBALANCE, which utilizes equivalence class annealing and decision theory to obtain acquisition functions. It can select high-information samples through greedy selection in small batches and efficiently construct query batches through clustering and random sampling in large batches.
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel
SungYub Kim, Eunho Yang (Korea Advanced Institute of Science and Technology)
CodeTabular
π― What it does: This paper proposes a PAC-Bayes generalization bound that is invariant to scale transformations, and based on this, introduces the Connectivity Tangent Kernel (CTK) and Connectivity Sharpness (CS), while also proposing the Connectivity Laplace (CL) Bayesian network.
π― What it does: A black-box input-level backdoor detection method called SCALE-UP is proposed, which uses scaled prediction consistency to determine whether a sample has been implanted with a backdoor.
Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting
Mohammad Amin Shabani (Simon Fraser University), Tristan Sylvain (Borealis AI)
CodeTransformerTime Series
π― What it does: This paper proposes Scaleformer, a multi-scale iterative refinement framework built on top of the Transformer to enhance the accuracy of time series forecasting.
Scaling Laws for a Multi-Agent Reinforcement Learning Model
Oren Neumann (Institute for Theoretical Physics Goethe University Frankfurt), Claudius Gros (Institute for Theoretical Physics Goethe University Frankfurt)
CodeReinforcement Learning
π― What it does: This study investigates the power-law scaling relationship between performance during the training process of AlphaZero and the number of model parameters and available computational resources, deriving a formula for the optimal network size under a given computational budget.
π― What it does: This study investigates the performance scaling laws of deep learning image reconstruction models under different training set sizes and provides a theoretical analysis to explain these laws.
π― What it does: A research framework for offline multi-objective reinforcement learning is proposed, and the Pareto front is approximated through the design of a new dataset D4MORL and the algorithm PEDA.
Scaling Up Probabilistic Circuits by Latent Variable Distillation
Anji Liu (University of California), Guy Van den Broeck (University of California)
CodeOptimizationKnowledge DistillationTransformerLarge Language ModelImageText
π― What it does: A method called 'Latent Variable Distillation' (LVD) is proposed, which uses deep generative models to assign values to latent variables in probabilistic circuits (PC), thereby enabling scalable training of PCs.
Scenario-based Question Answering with Interacting Contextual Properties
Haitian Sun (Carnegie Mellon University), Ruslan Salakhutdinov (Carnegie Mellon University)
CodeTransformerLarge Language ModelText
π― What it does: A three-module model named T-Reasoner is proposed for identifying answers and reasoning about missing conditions in scene-based question answering.
Schema Inference for Interpretable Image Classification
Haofei Zhang (Zhejiang University), Mingli Song (Shanghai Institute for Advanced Study, Zhejiang University)
CodeClassificationExplainability and InterpretabilityGraph Neural NetworkTransformerImage
π― What it does: This paper proposes the reasoning paradigm of schema inference, achieving interpretable reasoning for image classification through the construction of instance graphs (IR-Graph) and category graphs (IR-Atlas) for graph matching.
SeaFormer: Squeeze-enhanced Axial Transformer for Mobile Semantic Segmentation
Qiang Wan (Fudan University), Li Zhang (Fudan University)
CodeClassificationSegmentationTransformerImage
π― What it does: A compressed enhanced axial transformer (SeaFormer) for mobile semantic segmentation is proposed, achieving end-to-end segmentation and classification tasks through lightweight design;
Selective Annotation Makes Language Models Better Few-Shot Learners
Hongjin SU, Tao Yu (University of Washington)
CodeClassificationRetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: A two-step framework is proposed: first, selectively label the unlabeled data, and then during testing, retrieve these labeled samples as contextual examples for the large language model, significantly enhancing few-shot learning performance.
Yuning Cui (Sun Yat-sen University), Alois Knoll (Technical University of Munich)
CodeRestorationConvolutional Neural NetworkImage
π― What it does: A frequency-selective image restoration network SFNet is designed, utilizing dynamic multi-branch filtering and channel attention for frequency decomposition and re-weighting of features;
CodeSafty and PrivacyAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage
π― What it does: The study investigates how to add imperceptible perturbations to the training set to render unauthorized model training ineffective, proposing a protection method based on self-ensemble from training checkpoints (SEP).
π― What it does: A regularization loss based on rotation-invariant kernel mean embedding is proposed for self-supervised learning of image representations, and the SFRIK method is introduced.
π― What it does: A semi-supervised contrastive learning framework SEMPPL is proposed, which improves representation learning by predicting pseudo-labels using a small number of labeled samples to select semantically positive samples.
π― What it does: A sequence latent variable model framework based on Bayesian meta-learning is proposed for long-term prediction of high-dimensional time series with few samples.
π― What it does: This paper proposes two neural network architectures, SignNet and BasisNet, which can eliminate sign flips and basis invariance while processing graph Laplacian spectra, thereby enhancing graph representation learning performance.
SimPer: Simple Self-Supervised Learning of Periodic Targets
Yuzhe Yang (Massachusetts Institute of Technology), Daniel McDuff (Google)
CodeRepresentation LearningConvolutional Neural NetworkContrastive LearningVideoTime Series
π― What it does: The SimPer framework is proposed, which utilizes a periodic self-supervised learning method to extract representations of periodic signals from unlabeled data.
π― What it does: This paper proposes a lightweight k-NN translation framework SK-MT based on dynamic retrieval, utilizing BM25 to retrieve sentence-level similar corpora and constructing a minimal retrieval library, and then adaptively fusing the k-NN results with a pre-trained NMT model through a distance-aware adapter.
π― What it does: This paper proposes SIMPLEKT, a simple knowledge tracing baseline model based on the Rasch model and ordinary dot-product attention.
π― What it does: This paper introduces the Simplicial Embeddings (SEM) module in self-supervised learning, projecting the encoder output onto L sparse vectors of dimension V, using softmax for discretization, and utilizing this representation in downstream classification tasks.
Thomas F Burns, Tomoki Fukai (OIST Graduate University)
CodeImage
π― What it does: This paper proposes an extension of the Hopfield network to include Simplicial Hopfield networks with higher-order set connections, and conducts theoretical and experimental analyses of its memory capacity and performance.
SLTUNET: A Simple Unified Model for Sign Language Translation
Biao Zhang (University of Edinburgh), Rico Sennrich (University of Zurich)
CodeTransformerText
π― What it does: A unified end-to-end sign language translation model SLTUNET is proposed, capable of simultaneously handling Sign2Gloss, Sign2Text, Gloss2Text, Text2Gloss (excluded), and machine translation tasks, achieving multi-task joint training.
π― What it does: This paper proposes a self-supervised multi-task pre-training framework called SMART, which utilizes a Control Transformer for pre-training on multi-domain continuous control tasks, significantly improving learning efficiency and performance in downstream imitation learning or reinforcement learning tasks.
Softened Symbol Grounding for Neuro-symbolic Systems
Zenan Li (Nanjing University), Jian L\"{u}
CodeReinforcement LearningSequential
π― What it does: A neural-symbolic learning framework is proposed to soften symbolic grounding, modeling symbolic states as a Boltzmann distribution, and achieving efficient sampling through projection MCMC and SMT solvers, gradually jointly training neural networks and symbolic constraints.
π― What it does: This paper studies a new pseudo-label weighting method called SoftMatch, which aims to simultaneously improve both the quantity and quality of pseudo-labels in semi-supervised learning.