🎯 What it does: ChordMixer is proposed, a neural attention model that can be scaled to variable-length sequences; it achieves full receptive field through a parameter-free multi-scale rotation layer and per-channel MLP, maintaining efficient operation on long sequences (up to 1.5M).
🎯 What it does: Triangulation of point clouds is achieved by detecting the circumcenters of triangles, using a single-stage neural network to directly output the circumcenter positions within the neighborhood of each point, thereby deriving the corresponding triangles.
🎯 What it does: This paper proposes a circuit graph neural network (CktGNN) that can automatically generate analog circuit topologies and optimize device sizes simultaneously.
Johannes Brandstetter (Microsoft Research AI4Science), Jayesh K Gupta
CodeNeural Radiance FieldTime SeriesSequentialPhysics Related
🎯 What it does: Proposed and implemented a neural network layer based on Clifford algebra to accelerate the simulation of PDEs (Navier–Stokes, shallow water equations, and Maxwell's equations).
CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks
Tuomas Oikarinen (University of California San Diego), Tsui-Wei Weng (University of California San Diego)
CodeExplainability and InterpretabilityRepresentation LearningConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: The CLIP-Dissect technique is proposed, which automatically generates open concept descriptions for hidden neurons in deep visual networks without the need for manual annotation.
🎯 What it does: A text query-based universal sound separation model CLIPSep is proposed, which is trained solely on unlabeled noisy videos, and its robustness in noisy environments is enhanced through Noise-Invariant Training (NIT).
🎯 What it does: Proposes the CO3 (Cooperative Contrastive Learning and Contextual Shape Prediction) framework, utilizing collaborative point clouds from inside and outside the vehicle for unsupervised 3D representation learning, and transferring to various downstream tasks.
Marc Szafraniec (Meta AI), Gabriel Synnaeve (Meta AI)
CodeAI Code AssistantTransformerAuto EncoderText
🎯 What it does: This paper enhances the semantic quality of neural code translation by incorporating LLVM IR information into the code translation model, using three IR-based self-supervised objectives.
CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code
Nadezhda Chirkova (Naver Labs Europe), Sergey Troshin (University of Amsterdam)
CodeGenerationAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: This paper systematically studies the impact of subtokenization schemes on model performance and sequence length in the pre-training of large language models on source code, conducting large-scale experiments based on PLBART and proposing various efficient solutions.
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Erik Nijkamp (Salesforce Research), Caiming Xiong (Salesforce Research)
CodeGenerationAI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: A series of large language models CODEGEN, ranging in size from 350M to 16.1B, were trained and released, demonstrating their capabilities in multi-turn program synthesis tasks; simultaneously, a multi-turn programming benchmark MTPB was proposed and made public.
Bei Chen (Microsoft Corporation), Weizhu Chen (Microsoft Corporation)
CodeGenerationAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: This study proposes a framework for automatically generating test cases using the same pre-trained language model and selecting the optimal code solution through bidirectional execution consistency (code solution and test cases).
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers
Wenyi Hong (Tsinghua University), Jie Tang (Tsinghua University)
CodeGenerationData SynthesisTransformerVideoText
🎯 What it does: CogVideo has been constructed and trained, a 9B parameter text-to-video generation Transformer that inherits the pre-trained knowledge of CogView2, utilizing multi-frame rate training and a dual-channel attention mechanism to achieve large-scale text-to-video generation.
Combating Exacerbated Heterogeneity for Robust Models in Federated Learning
Jianing Zhu (Hong Kong Baptist University), Bo Han (Hong Kong Baptist University)
CodeFederated LearningAdversarial AttackImage
🎯 What it does: This study addresses the issue of robustness decline caused by the combination of adversarial training and federated learning, proposing the Slack Federated Adversarial Training (SFAT) framework to alleviate the exacerbated heterogeneity due to adversarial samples.
Compositional Law Parsing with Latent Random Functions
Fan Shi (Fudan University), Xiangyang Xue (Fudan University)
CodeExplainability and InterpretabilityRepresentation LearningImage
🎯 What it does: A concept-level rule parsing framework (CLAP) based on deep latent variable models is proposed, which automatically parses natural or artificial rules in scenes by decomposing images into independent concepts and learning latent stochastic functions for each concept, thereby enabling predictions of future states and scene reconstruction.
Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection
Kaifeng Gao (Zhejiang University), Qianru Sun (Singapore Management University)
CodeRecognitionObject DetectionKnowledge DistillationTransformerPrompt EngineeringVision Language ModelVideoMultimodality
🎯 What it does: This paper proposes the Open-VidVRD task for open vocabulary video visual relationship detection and designs a combination-based prompt tuning framework called RePro, which can be trained on a limited set of benchmark categories and generalized to unseen object and predicate categories.
Compositional Task Representations for Large Language Models
NAN SHAO, Zhilin Yang (Tsinghua University)
CodeClassificationGenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a task representation method that does not rely on prompts—Compositional Task Representations (CTR). It learns a discrete, composable code table through multi-task training to achieve cross-task generalization in the absence of labels or with a small number of labels.
Computational Language Acquisition with Theory of Mind
Andy Liu (Harvey Mudd College), Graham Neubig (Carnegie Mellon University)
CodeGenerationConvolutional Neural NetworkRecurrent Neural NetworkLarge Language ModelReinforcement LearningImageText
🎯 What it does: This study constructs language learners with Theory of Mind (ToM) capabilities in an image referencing game environment, using an internal 'listener' model to reorder candidate sentences, thereby achieving more persuasive and information-rich language expression.
Concept-level Debugging of Part-Prototype Networks
Andrea Bontempelli (University of Trento), Andrea Passerini (University of Trento)
CodeOptimizationExplainability and InterpretabilitySupervised Fine-TuningImage
🎯 What it does: Proposes ProtoPDebug, an interactive debugger based on conceptual levels, to correct erroneous predictions in ProtoPNets caused by confounding factors; it allows human supervisors to label which parts of the prototypes (part-prototypes) are noise or useful, and then fine-tunes the model using two new penalty terms (forgetting and remembering);
🎯 What it does: A multi-channel equivariant attention network (MEAN) is proposed, modeling antibody design as conditional 3D graph translation, jointly predicting C1R sequences and structures.
Confidence-Based Feature Imputation for Graphs with Partially Known Features
Daeho Um (Seoul National University), Jin young Choi
CodeGraph Neural NetworkGraph
🎯 What it does: In response to the challenge of missing node features in graph learning tasks, this paper proposes a pseudo-confidence-based feature imputation method (PCFI) to recover node features under high missing rates for downstream tasks.
Confidential-PROFITT: Confidential PROof of FaIr Training of Trees
Ali Shahin Shamsabadi (Alan Turing Institute), Adrian Weller (University of Cambridge)
CodeOptimizationSafty and PrivacyReinforcement LearningTabular
🎯 What it does: This paper proposes Confidential-PROFITT, which can prove that the decision tree training process complies with fairness constraints using zero-knowledge proofs without disclosing training data or models.
Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimization
Jihwan Jeong (University of Toronto), Scott Sanner (University of Toronto)
CodeOptimizationReinforcement LearningTabular
🎯 What it does: A Conservative Bayesian Model-Based Value Expansion (CBOP) method is proposed for policy evaluation and optimization in offline reinforcement learning.
🎯 What it does: A novel CNN backbone network named Contextual Convolutional Network is proposed, which dynamically adjusts the convolutional kernel weights and sampling positions by using the top-k categories from the previous layer as contextual priors, thereby achieving context-aware feature extraction.
🎯 What it does: A framework called ccMIM is proposed, which combines context mask modeling with contrastive learning in visual pre-training. It actively selects semantically rich patches for masking through attention importance sampling and aligns the global tokens of masked and unmasked patches under a single view using contrastive loss, thereby enhancing the learning difficulty and convergence speed of MIM.
Zixuan Ke (University of Illinois at Chicago), Bing Liu (University of Illinois at Chicago)
CodeDomain AdaptationTransformerLarge Language ModelContrastive LearningTextBiomedical Data
🎯 What it does: The DAS method is proposed to achieve continual domain adaptation pre-training (continual DAP-training) to enhance the performance of language models in new domains while preventing catastrophic forgetting.
🎯 What it does: A Transformer architecture capable of online inference without redundancy and step-by-step reasoning—Continual Transformers—is proposed, introducing a new continuous Scaled Dot-Product Attention (Retroactive and Single-Output) in the Transformer Encoder, while adapting time series with Recycling Positional Encoding.
Continuous-Discrete Convolution for Geometry-Sequence Modeling in Proteins
Hehe Fan (Zhejiang University), Mohan Kankanhalli (National University of Singapore)
CodeProtein Structure PredictionConvolutional Neural NetworkBiomedical Data
🎯 What it does: A continuous-discrete convolution (CDConv) network is proposed to simultaneously process protein sequences (1D) and geometric (3D) information for protein structure and function prediction.
🎯 What it does: A new estimation method called the Subspace Encoder Method (SUBNET) is proposed for identifying dynamic state space models, particularly continuous-time nonlinear state space models, addressing issues related to external inputs, measurement noise, and latent states in experimental settings.
🎯 What it does: A normalization layer called ContraNorm based on contrastive learning uniformity loss is proposed to alleviate the issues of over-smoothing and dimensional collapse in GNNs and Transformers.
Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning
Zaid Khan (Northeastern University), Yun Fu (Northeastern University)
CodeRetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A parameter-efficient alignment method for contrastive vision-language models (LilT) is proposed, which only requires updating a small number of parameters from the original pre-trained vision and language models to achieve CLIP performance comparable to full model training.
🎯 What it does: This paper proposes CAV-MAE, a self-supervised pre-training model for audio-visual multimodal learning that simultaneously utilizes masked autoencoding and contrastive learning.
Contrastive Learning for Unsupervised Domain Adaptation of Time Series
Yilmazcan Ozyurt (ETH Zurich), Ce Zhang (ETH Zurich)
CodeDomain AdaptationRecurrent Neural NetworkContrastive LearningTime SeriesBiomedical Data
🎯 What it does: A contrastive learning-based unsupervised domain adaptation framework CLUDA is proposed, aimed at transferring labeled learning from the source domain to multivariate time series data in the target domain.
🎯 What it does: By using POEM, some observable multi-view data is mapped to an uncertainty representation, and a unified representation is obtained through a product expert mechanism to achieve representation learning in few-shot learning.
🎯 What it does: The text generation process is changed to gradually copy phrases from a large-scale text collection, rather than predicting word by word from a fixed vocabulary.
🎯 What it does: A biologically plausible neural network framework based on maximizing relevant information, CorInfoMax, is proposed for unsupervised separation of relevant sources.
CoRTX: Contrastive Framework for Real-time Explanation
Yu-Neng Chuang (Rice University), Xia Hu (Rice University)
CodeExplainability and InterpretabilityContrastive LearningImageTabular
🎯 What it does: A real-time explanation framework CoRTX based on contrastive learning is developed, utilizing unlabeled potential explanation vectors and fine-tuning the explanation head with a minimal number of explanation labels.
Coverage-centric Coreset Selection for High Pruning Rates
Haizhong Zheng (University of Michigan), Atul Prakash (University of Michigan)
CodeOptimizationConvolutional Neural NetworkImage
🎯 What it does: A coverage-centered single-core subset selection method CCS is proposed, aimed at maintaining model performance under high pruning rates;
🎯 What it does: A compressed sensing minimizer CrAM was trained, capable of generating models that are easy to compress in one go without a drop in accuracy during training.
🎯 What it does: This paper proposes a cross-layer attention mechanism called MRLA (and its lightweight version MRLA-light), which enhances inter-layer interaction by recursively allowing the current layer to query features from all previous layers, thereby improving the representational capability of deep networks.
🎯 What it does: Proposes Crossformer, a Transformer model specifically designed for multivariate time series forecasting, which explicitly utilizes cross-dimensional dependencies;
🎯 What it does: A course learning-based co-design method for Voxel soft-bodied robots and control, named CuCo, is proposed, which gradually expands the design space while simultaneously learning design and control strategies.
CUTS: Neural Causal Discovery from Irregular Time-Series Data
Yuxiao Cheng (Tsinghua University), Qionghai Dai (Tsinghua University)
CodeGraph Neural NetworkSupervised Fine-TuningTime Series
🎯 What it does: The CUTS framework is proposed, which alternates between data imputation and causal graph learning to discover nonlinear Granger causality from irregular time series data.
🎯 What it does: Proposes the D4AM framework, which jointly trains the speech enhancement model with the ASR task, achieving a universal noise reduction preprocessor through gradient calibration and regression target weighting.
D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory
Tianbo Li (SEA AI Lab), Shuicheng YAN
CodeOptimizationComputational EfficiencyTabularPhysics Related
🎯 What it does: A deep learning-based Kohn-Sham DFT solving method called D4FT is proposed, which directly performs gradient descent on the energy function, eliminates orthogonality constraints through reparameterization, and transfers numerical integration to SGD sampling, significantly reducing computational complexity and achieving differentiable solutions.
Valentina Zantedeschi (ServiceNow Research), Vlad Niculae (Informatics Institute University of Amsterdam)
CodeOptimizationGraph
🎯 What it does: A continuous optimization framework based on Permutahedron is proposed, using vector parameterization of node ordering to achieve DAG learning.
🎯 What it does: A NAS-based framework for searching dedicated backbone networks for face detection is designed, utilizing DDSAR-Score to evaluate backbone performance.
🎯 What it does: This paper demonstrates and implements that significant improvements in model robustness and accuracy can be achieved in adversarial training by using only data augmentation, and proposes a new cropping transformation called Cropshift along with a multi-layer enhancement strategy named IDBH.
🎯 What it does: A training-independent data value scoring method called Complexity Gap Score (CG-score) is proposed, which can be calculated directly from the data;
🎯 What it does: Two data-free one-shot federated learning methods, FEDCVAE-ENS and FEDCVAE-KD, are designed and proposed to address extremely high statistical heterogeneity environments. They utilize conditional VAE to reconstruct local learning tasks and aggregate decoders through knowledge distillation or ensemble methods.
Dataless Knowledge Fusion by Merging Weights of Language Models
Xisen Jin (University of Southern California), Pengxiang Cheng (Bloomberg)
CodeDomain AdaptationKnowledge DistillationTransformerLarge Language ModelTextBenchmark
🎯 What it does: The research achieves knowledge fusion by merging multiple model weights without accessing the training data, proposing a new model merging method called Regression Mean (RegMean).
DaxBench: Benchmarking Deformable Object Manipulation with Differentiable Physics
Siwei Chen (National University of Singapore), David Hsu (Sea AI Lab)
CodeRobotic IntelligenceReinforcement LearningBenchmarkPhysics Related
🎯 What it does: This paper presents DaXBench—a differentiable simulation framework that supports liquids, ropes, fabrics, and elastoplastic materials for evaluating various manipulation methods for deformable objects.
DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability
Cian Eastwood (Max Planck Institute for Intelligent Systems), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
CodeRepresentation LearningImage
🎯 What it does: This paper proposes an extended DCI-ES framework in discrete representation learning to quantify the decoupling, completeness, information content, explicitness, and size of representations;
🎯 What it does: A self-supervised diffusion model DDM2 is proposed for denoising low signal-to-noise ratio diffusion MRI under unpaired data conditions.
🎯 What it does: A connection-aware molecular motif mining and generation framework called MiCaM is proposed, which realizes the data-driven mining of connection-aware motifs and uses them to generate new molecules.
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
Pengcheng He (Microsoft), Weizhu Chen (Microsoft)
CodeTransformerLarge Language ModelText
🎯 What it does: DeBERTaV3 is proposed by combining the Disentangled Attention of DeBERTa with the Replaced Token Detection (RTD) of ELECTRA, improving the pre-training tasks and introducing a new embedding sharing method.
🎯 What it does: Proposes the DECAF framework, which jointly generates answers and logical forms and combines execution results to obtain the final answer, using a retrieval-based textual knowledge base instead of entity linking;
DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training
Wei Li (Zhejiang University), Yi Yang (Zhejiang University)
CodeGenerationRetrievalTransformerVision Language ModelContrastive LearningImageVideoText
🎯 What it does: Proposes the DeCap framework, which uses a text decoder to generate zero-shot image/video captions in the multimodal latent space of CLIP, requiring only text data for training, and during inference, maps image embeddings to the text space through a zero-training projection.
Decomposed Prompting: A Modular Approach for Solving Complex Tasks
Tushar Khot (Allen Institute for AI), Ashish Sabharwal (Allen Institute for AI)
CodeLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes the Decomposed Prompting (DECOMP) framework, which utilizes a small number of examples to decompose complex tasks into several sub-tasks, each completed with dedicated LLM prompts or symbolic modules;
🎯 What it does: This paper proposes an instance-dependent partial label learning method based on a decomposition generative process, IDGP, which explicitly models the candidate label generation using MAP optimization.
🎯 What it does: This paper proposes a differentiable Dynamic Time Warping (DTW) layer—DecDTW—that enables end-to-end learning of alignment paths for time series in deep networks.
🎯 What it does: A deep graph ensemble method (DGE) is proposed for graph data with high-order dependencies, training multiple GNNs in different high-order subspaces of the same node to capture neighborhood variance and enhance representation learning.
🎯 What it does: The Reg-DGM framework is proposed, using a pre-trained non-transferable model as a regularization term to train deep generative models, in order to reduce variance on limited data and improve generation quality.
Samuel Holt (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeGenerationOptimizationTransformerReinforcement LearningTabularBenchmarkPhysics Related
🎯 What it does: A deep generative symbolic regression (DGSR) framework is proposed, which utilizes a pre-trained conditional generative model to learn the invariance of equations and data, and enhances the performance of symbolic regression in high-dimensional variables through gradient refinement during inference.
🎯 What it does: A deep ranking ensemble model (Deep Ranking Ensembles) based on learning to rank is proposed for hyperparameter search in Bayesian optimization.
Deep Reinforcement Learning for Cost-Effective Medical Diagnosis
Zheng Yu (Princeton University), Mengdi Wang (Princeton University)
CodeAnomaly DetectionOptimizationReinforcement LearningTabularBiomedical DataElectronic Health Records
🎯 What it does: A dynamic medical diagnosis framework based on reinforcement learning is proposed, which can select experimental test panels and make diagnostic decisions at a limited cost;
🎯 What it does: A framework named RaTP is proposed to address the performance degradation of models during the 'strange period' in continuous domain drift environments, capable of providing good performance immediately when a new domain appears while retaining memory of the old domain.
🎯 What it does: A self-supervised method called Denoising Masked AutoEncoders (DMAE) is proposed, which trains a Vision Transformer encoder by adding Gaussian noise to images and masking certain blocks to reconstruct the original image, serving directly as a base classifier for random smoothing robust classification.
🎯 What it does: The SparK framework is proposed, utilizing sparse convolution and a hierarchical decoder to achieve BERT-style masked image pre-training for convolutional networks.
🎯 What it does: A deterministic generative autoencoder (AEF) constructed using reversible layers is proposed, achieving maximum likelihood training for traditional variational autoencoders (VAE);
🎯 What it does: A data-free structured pruning method (DFPC) is proposed, specifically targeting the coupling channels in multi-branch convolutional networks for pruning.
🎯 What it does: A molecular docking method called DIFFDOCK based on diffusion generative models is proposed, which generates 3D structures for protein binding in the ligand pose space through the diffusion process of displacement, rotation, and torsion angles.
🎯 What it does: A differentiable graph cut and matching framework is proposed, which approximates the minimum s-t cut problem using equality-constrained quadratic programming and embeds it into convolutional networks for object-centric representation learning.
DiffMimic: Efficient Motion Mimicking with Differentiable Physics
Jiawei Ren (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
CodeRobotic IntelligenceReinforcement LearningVideoPhysics Related
🎯 What it does: Using a differentiable physics simulator (DPS) to directly optimize control strategies through state matching, achieving motion imitation of physical characters.
🎯 What it does: This paper proposes an energy-constrained diffusion-based Transformer (DIFFORMER) that propagates information across the entire dataset using a learnable full-instance diffusion coefficient, generating representations that balance global consistency and local features.
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Shansan Gong (Shanghai AI Laboratory), Lingpeng Kong (University of Hong Kong)
CodeGenerationTransformerDiffusion modelText
🎯 What it does: A sequence-to-sequence text generation framework called DIFFUSEQ based on diffusion models is proposed, which can simultaneously achieve conditional generation and diversity control within a single model.
Diffusion Models Already Have A Semantic Latent Space
Mingi Kwon (Yonsei University), Youngjung Uh (Yonsei University)
CodeGenerationData SynthesisDiffusion modelImage
🎯 What it does: In the frozen diffusion model, an asymmetric reverse process (Asyrp) is designed to discover and utilize U-Net bottleneck features in h-space for semantic image attribute editing.
Diffusion Models for Causal Discovery via Topological Ordering
Pedro Sanchez (University of Edinburgh), Sotirios A. Tsaftaris
CodeDiffusion modelScore-based ModelGraphBiomedical Data
🎯 What it does: This paper proposes a causal discovery method called DiffAN based on the Diffusion Probability Model (DPM), which achieves topological sorting by learning the score of the data distribution and approximating its Hessian, thereby inferring a directed acyclic graph.
Diffusion Probabilistic Modeling of Protein Backbones in 3D for the motif-scaffolding problem
Brian L. Trippe (Massachusetts Institute of Technology), Tommi S. Jaakkola
CodeGenerationProtein Structure PredictionGraph Neural NetworkDiffusion modelBiomedical Data
🎯 What it does: A diffusion probabilistic model called ProtDiff based on E(3)-equivariant graph neural networks is proposed to generate complete 3D coordinates of protein backbones; and the SMCDiff algorithm is introduced, which achieves conditional sampling (generating scaffolds under the condition of given motifs) through particle filtering, theoretically allowing for precise conditional samples.
Diminishing Return of Value Expansion Methods in Model-Based Reinforcement Learning
Daniel Palenicek (Technical University of Darmstadt), Jan Peters (Technical University of Darmstadt)
CodeReinforcement Learning
🎯 What it does: By using a perfect (oracle) dynamics model and learning model, systematic experiments were conducted on the sampling efficiency of model-based value expansion methods (Critic Expansion and Actor Expansion) in continuous control tasks, and the impact of model accuracy and rollout step length on learning effectiveness was explored.
🎯 What it does: Treating DINO as a von Mises-Fisher mixture model, DINO-vMF is proposed to achieve more flexible prototype learning and improve representation quality by incorporating a normalization constant;
🎯 What it does: A Dirichlet distribution-based evidence deep learning model is proposed for uncertainty calibration and sample selection in active domain adaptation.
🎯 What it does: This paper proposes an algorithm for offline multi-task multi-agent reinforcement learning called ODIS, which can learn general coordination skills from limited sources of multi-task offline data and achieve efficient collaboration without tuning in unseen tasks.
Discovering Latent Knowledge in Language Models Without Supervision
Collin Burns (University of California Berkeley), Jacob Steinhardt (University of California Berkeley)
CodeTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Unsupervised discovery of latent knowledge in the internal activations of language models, using logical consistency features to answer yes/no questions.
🎯 What it does: This paper proposes a Hausdorff distance-based Factorized Support (HFS) criterion to separate generative factors influenced by correlations under unsupervised conditions, thereby obtaining more interpretable and generalizable representations.
Disentangling Learning Representations with Density Estimation
Eric Yeats (Duke University), Hai Li (Duke University)
CodeRepresentation LearningAuto EncoderImage
🎯 What it does: This paper proposes the Gaussian Channel Autoencoder (GCAE), an unsupervised learning method that achieves reliable disentanglement using Gaussian noise and conditional density estimation.
🎯 What it does: Comparatively and empirically evaluate whether various implicit regularizations (gradient norm, Fisher information, Jacobian regularization) can recover the generalization performance of small-batch SGD in large-batch SGD, and study the effects of micro-batch size and dataset on their performance.
🎯 What it does: This paper proposes a Cognitive Distillation (CD) method that automatically extracts the minimal patterns (cognitive patterns) from images that can determine model predictions, and uses these patterns for backdoor sample detection and bias identification.
Distributed Differential Privacy in Multi-Armed Bandits
Sayak Ray Chowdhury (Microsoft Research), Xingyu Zhou (Wayne State University)
CodeSafty and PrivacyReinforcement LearningTabular
🎯 What it does: Under the distributed differential privacy model, a multi-armed bandit algorithm based on successful elimination has been designed, achieving the same asymptotic optimal return rate as pure DP and the central model.
🎯 What it does: A universal unbiased quantization extra gradient algorithm (Q-GenX) for multi-GPU distributed environments is proposed, which can uniformly handle different VI solvers and significantly reduce communication overhead.
🎯 What it does: A lightweight post-processing method called DROPS is proposed, which uses the predicted probabilities of a trained model for learnable scaling to enhance robustness against class/group prior variations.
🎯 What it does: A two-stage DivDis framework is proposed, where a multi-head network is first trained under the source distribution to generate maximum inconsistency among heads on the target unlabeled data, and then a small number of labeled samples are used to disambiguate the heads, ultimately selecting the optimal head for predictions on the target distribution.
🎯 What it does: A method called BETA is proposed to suppress confirmation bias in black-box predictor domain adaptation by dividing the target domain into easy-to-adapt and hard-to-adapt subdomains and employing a mutual distillation dual network;
🎯 What it does: This paper proposes DM-NeRF, which utilizes the implicit representation of NeRF to complete 3D scene reconstruction, object decomposition, and editable rendering solely from 2D images.
🎯 What it does: The DocPrompting method is proposed, which involves first retrieving relevant documents in the process of generating code from natural language, and then inputting the retrieved documents along with the intent into the generation model, allowing the model to use unseen libraries or functions during testing.
Does Deep Learning Learn to Abstract? A Systematic Probing Framework
Shengnan An (Xi'an Jiaotong University), Jian-Guang Lou (Microsoft Corporation)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A systematic detection framework from the perspective of transferability is proposed to examine whether deep learning models possess abstract capabilities. The performance of T5 and GPT2 in learning abstract concepts from concrete instances and transferring them to new tasks is validated through syntactic translation probes.