Conference on Neural Information Processing Systems Β· 1874 papers
Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation
Jiaxin Cheng (University of Macau), Yicong Zhou (Amazon Web Services)
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageText
π― What it does: This paper proposes a Regional Cross-Attention module to handle complex, lengthy text descriptions in Layout-to-Image (L2I) generation, and redesigns evaluation metrics for open vocabulary scenarios; it also constructs a large-scale synthetic rich-context L2I dataset.
Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis
Honglin Li (Zhejiang University), Lin Yang (Research Center for Industries of the Future)
CodeClassificationSegmentationComputational EfficiencyRepresentation LearningTransformerImageBiomedical Data
π― What it does: A Local-Global Hybrid Transformer (LongMIL) for long sequence multi-instance learning of large-sized, diverse-deformation digital pathology slides (Whole Slide Image, WSI) has been designed and implemented, enhancing representation capability through local attention masks while significantly reducing computational complexity.
π― What it does: A new selective weight decay technique called Selective Projection Decay (SPD) is proposed for robust fine-tuning on powerful base models, selectively applying strong penalties to certain layers while allowing other layers to vary freely.
π― What it does: A Transformer-based multi-view radar detection framework called RETR is proposed, which directly implements 3D frame prediction on radar heatmaps and projects it onto the image plane for object detection and segmentation.
π― What it does: Based on TabPFN, we introduce nearest neighbor retrieval and end-to-end fine-tuning to construct a locally calibrated ICL model called LoCalPFN;
π― What it does: This paper proposes a Retrieval-Augmented Time Series Diffusion Model (RATD), which uses samples from a database that are most similar to historical sequences as references to guide the diffusion process and improve prediction accuracy.
Retrieval-Retro: Retrieval-based Inorganic Retrosynthesis with Expert Knowledge
Heewoong Noh (Korea Advanced Institute of Science and Technology), Chanyoung Park (Korea Advanced Institute of Science and Technology)
CodeRetrievalGraph Neural NetworkGraph
π― What it does: Developed the Retrieval-Retro method, which utilizes retrieved reference materials and implicitly extracts precursor information through attention mechanisms to achieve inverse synthesis prediction of inorganic materials.
π― What it does: The Representation-Conditioned Generation (RCG) framework is proposed, which first generates low-dimensional semantic representations using a self-supervised encoder, then generates these representations in an unsupervised manner, and finally synthesizes images using a representation-conditioned image generator, achieving high-quality image generation without labels.
Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference
Jiabao Ji (University of California Santa Barbara), Shiyu Chang (University of California Santa Barbara)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A novel LLM no-learning framework ULD based on auxiliary model adversarial objectives and logit difference calculation is proposed, which can forget specified document knowledge while retaining other knowledge.
π― What it does: A FENS framework is proposed that combines one-round federated learning with lightweight aggregation training, utilizing an aggregator network to integrate local models, maintaining low communication costs while approaching the accuracy of traditional iterative federated learning.
Revisiting Score Propagation in Graph Out-of-Distribution Detection
Longfei Ma (Zhejiang University), Fei Wu (Zhejiang University)
CodeAnomaly DetectionGraph Neural NetworkGraph
π― What it does: This study explores the out-of-distribution (OOD) node detection problem in graph data and proposes a simple and effective OOD score propagation method that improves detection performance by propagating the OOD scores of neighboring nodes within the graph structure.
π― What it does: A self-supervised heterogeneous graph learning framework called SCHOOL based on spectral clustering is proposed to address the issues of noisy graph structures and underutilized clustering information in traditional methods.
π― What it does: This paper proposes the GLMix structure, which allows convolution and multi-head self-attention (MHSA) to work in parallel at different granularities, and integrates fine-grained grid features with coarse-grained semantic slot features through a soft clustering and dispatch module, achieving efficient local-global feature mixing.
RFLPA: A Robust Federated Learning Framework against Poisoning Attacks with Secure Aggregation
Peihua Mai (National University of Singapore), Yan Pang (National University of Singapore)
CodeFederated LearningSafty and PrivacyConvolutional Neural NetworkImage
π― What it does: A robust federated learning framework RFLPA is proposed under the secure aggregation (SecAgg) protocol, addressing the dual threats of privacy leakage and model poisoning.
Right this way: Can VLMs Guide Us to See More to Answer Questions?
Li Liu (University of California Santa Cruz), Leilani H. Gilpin (University of California Santa Cruz)
CodeTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: A 'directional guidance' task for visual question answering is proposed, along with the construction of a corresponding manually annotated test set; unsupervised data augmentation is performed using VLM to generate synthetic training data, which is then fine-tuned; the performance of the zero-shot and fine-tuned models is evaluated and compared.
Sapana Chaudhary (Amazon Web Services), Srinivas Shakkottai (Texas A&M University)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: A risk-averse RLHF method (RA-RLHF) is proposed and implemented for fine-tuning large language models to reduce the probability of generating harmful outputs in response to negative or toxic prompts.
π― What it does: This paper proposes a Random Linear Enhancement (RLE) method based on the Lambertian model for data augmentation in cross-spectral re-identification.
RMLR: Extending Multinomial Logistic Regression into General Geometries
Ziheng Chen (University of Trento), Nicu Sebe (University of Trento)
CodeClassificationOptimizationTabularBenchmark
π― What it does: A general polynomial logistic regression framework based on Riemannian logarithmic mapping is proposed, and five families of SPD MLR and Lie MLR are implemented on SPD and SO(n).
π― What it does: This paper proposes a novel road network representation learning framework that captures geographical configurations using street view images and combines contrastive learning to generate low-dimensional vector representations of road segments.
π― What it does: RobIR is proposed, an implicit inverse rendering framework that utilizes ACES tone mapping and regularized visibility estimation to accurately separate shadows, ambient light, and the PBR materials of objects in high illumination scenes.
π― What it does: In a robot control environment with only a few expert video demonstrations and no task rewards, the study explores a Temporal Optimal Transport reward learning strategy.
Robust and Faster Zeroth-Order Minimax Optimization: Complexity and Applications
Weixin An (Xidian University), Hongying Liu (Tianjin University)
CodeOptimizationImage
π― What it does: A unified zero-order gradient descent extrapolated gradient ascent algorithm (ZO-GDEGA) is proposed for black-box non-convex-concave minimax problems.
Robust Conformal Prediction Using Privileged Information
Shai Feldman (Technion), Yaniv Romano (Technion)
CodeAnomaly DetectionImageTabular
π― What it does: A method is proposed for providing a coverage-guaranteed prediction set even when the training samples are damaged (noise, missing, selective missing) β Privileged Conformal Prediction (PCP);
π― What it does: A robust clustering method called CANDY is proposed to address the issue of dual noise correspondence (i.e., false positives and false negatives) in contrastive multi-view clustering.
π― What it does: A variance reduction fine-tuning (VRF) method based on sample distance is proposed, which assigns weights to each test sample using the zero-shot model failure set, thereby achieving sample-level fusion between the zero-shot model and the fine-tuned model.
Robust Graph Neural Networks via Unbiased Aggregation
Zhichao Hou (North Carolina State University), Xiaorui Liu (North Carolina State University)
CodeGraph Neural NetworkGraph
π― What it does: A robust and unbiased graph signal estimator is proposed, which is expanded into RUNG layers to enhance the robustness of GNNs under adaptive attacks.
Robust Sleep Staging over Incomplete Multimodal Physiological Signals via Contrastive Imagination
Qi Shen (Northeastern University), Zhiqiong Wang (Northeastern University)
CodeClassificationConvolutional Neural NetworkRecurrent Neural NetworkTransformerAuto EncoderContrastive LearningMultimodalityBiomedical Data
π― What it does: A robust multimodal sleep staging framework CIMSleepNet is proposed, capable of performing sleep staging in the absence of certain modalities.
Robustly overfitting latents for flexible neural image compression
Yura Perugachi-Diaz (Vrije Universiteit Amsterdam), Sandjai Bhulai (Vrije Universiteit Amsterdam)
CodeCompressionAuto EncoderImage
π― What it does: An extended method SGA+ is proposed to refine latent variables on a pre-trained neural image compression model to improve compression quality.
RoPINN: Region Optimized Physics-Informed Neural Networks
Haixu Wu (Tsinghua University), Mingsheng Long (Tsinghua University)
CodeOptimizationTabularPhysics Related
π― What it does: This paper proposes the Region Optimization (RoPINN) training paradigm, which extends the optimization of PINN from discrete points to continuous neighborhoods, thereby enhancing the model's generalization and satisfaction of higher-order constraints.
π― What it does: This paper proposes the Rough Transformer (RFormer), a variant of Transformer that converts discrete time series into continuous time representations through path signatures, enabling efficient modeling under long sequences and irregular sampling conditions.
RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models
Shuhao Chen (Southern University of Science and Technology), Yu Zhang (Southern University of Science and Technology)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelContrastive LearningTextMultimodality
π― What it does: A query router called RouterDC based on dual contrastive learning is proposed for efficient selection and combination among multiple large language models (LLMs).
RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptions
Ziyao Zeng (Yale University), Alex Wong (Yale University)
CodeDepth EstimationVision Language ModelImage
π― What it does: This paper studies the use of language descriptions to solve the scale ambiguity problem in monocular depth estimation, converting relative depth into absolute scale.
CodeSafty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: A rule-based reward (RBR) method is proposed, using AI feedback to replace a large amount of manual labeling to train the safe behavior of language models;
π― What it does: This study addresses and solves the discretization gap problem in differentiable mask pruning and proposes the Soft-to-Hard Pruner (S2HPruner) framework.
Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models
Chia-Yi Hsu (National Yang Ming Chiao Tung University), Chun-Ying Huang (National Yang Ming Chiao Tung University)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes Safe LoRA, a post-projection technique for LoRA fine-tuning that requires no training or data, maintaining the safety alignment of LLMs without compromising downstream performance.
π― What it does: A SAFE framework is proposed based on pre-trained models, where a Slow Learner inherits general knowledge and is frozen, while a Fast Learner continuously learns new categories and prevents catastrophic forgetting, ultimately using entropy weighting to aggregate the outputs of both during inference.
CodeAutonomous DrivingSafty and PrivacyReinforcement Learning
π― What it does: This paper proposes an RLSF framework that utilizes trajectory-level feedback to learn a safety cost function and train a safety policy, reducing the burden of manual evaluation.
Samba: Severity-aware Recurrent Modeling for Cross-domain Medical Image Grading
Qi Bi (University of Amsterdam), Yefeng Zheng (Westlake University)
CodeClassificationDomain AdaptationRecurrent Neural NetworkImageBiomedical Data
π― What it does: A Severity-aware Recurrent Modeling (Samba) framework is proposed for cross-domain medical image grading tasks, which achieves better grading performance in unseen target domains after training in the source domain.
π― What it does: A parallelizable Sharpness-Aware Minimization method, SAMPa, is designed by introducing an auxiliary sequence y_t within SAM and using its gradient to achieve gradient parallel computation, resulting in a twofold speedup and improved model generalization performance.
π― What it does: A backdoor defense method called SampDetox is proposed for black-box environments, which eliminates various backdoor triggers by adding noise to the samples and then recovering them using a diffusion model, while maintaining the model's inference performance.
Sample Selection via Contrastive Fragmentation for Noisy Label Regression
Chris Dongjoo Kim (Seoul National University), Gunhee Kim (LG AI Research)
CodeConvolutional Neural NetworkMixture of ExpertsContrastive LearningTabular
π― What it does: A framework named ConFrag is proposed for regression tasks, which segments the label space and constructs contrastive fragment pairs to train a set of expert feature extractors. It cleans samples based on neighborhood consistency, ultimately improving the performance of regression models in the presence of noisy labels.
SAND: Smooth imputation of sparse and noisy functional data with Transformer networks
Ju-Sheng Hong (University of California), Jane-Ling Wang (University of California)
CodeTransformerTime Series
π― What it does: A SAND (Self-Attention on Derivatives) layer based on Transformer is proposed for smoothing interpolation of sparse and noisy functional data.
SARAD: Spatial Association-Aware Anomaly Detection and Diagnosis for Multivariate Time Series
Zhihao Dai (University of Warwick), Matthew Leeke (University of Birmingham)
CodeAnomaly DetectionTransformerAuto EncoderTime Series
π― What it does: The SARAD method is proposed, which utilizes Transformer to learn the spatial correlations of multivariate time series and implements anomaly detection and diagnosis through the phenomenon of correlation drop.
π― What it does: This paper constructs the first COCO-level SAR target detection large-scale multi-class dataset SARDet-100K and proposes a multi-stage filtering enhancement pre-training framework (MSFA), while also open-sourcing the data and code.
π― What it does: This paper analyzes the sample and communication complexity of Federated Linear Stochastic Approximation (FedLSA), reveals the impact of heterogeneity on FedLSA, and proposes SCAFFLSA, an improved algorithm that reduces communication volume while maintaining linear acceleration through the control variate method.
HaoChuan Xu (University of Auckland), Ninh Pham (University of Auckland)
CodeComputational EfficiencyTabular
π― What it does: Two scalable density clustering algorithms, sDBSCAN and sOPTICS, are proposed, which can efficiently cluster and visualize in high-dimensional space using cosine distance and various other distance metrics.
Youyuan Long (Delft University of Technology), Peyman Mohajerin Esfahani (Delft University of Technology)
CodeOptimizationReinforcement Learning from Human FeedbackTabular
π― What it does: Proposes the Kernel Inverse Optimization (KIO) model and the Sequential Selection Optimization (SSO) algorithm, utilizing kernel methods to learn the objective function of expert decisions and achieve a scalable implementation of inverse optimization.
π― What it does: This paper proposes a technique for real-time generation and reinforcement of pruning planes during the branch-and-bound process of neural network verification (BICCOS), significantly improving lower bound accuracy and accelerating verification;
π― What it does: This paper proposes a novel Scale Equivariant Graph MetaNetwork for handling the parameters of feedforward neural networks with different activation functions.
π― What it does: A cross-architecture knowledge distillation method called ScaleKD is proposed, which utilizes a pre-trained Vision Transformer teacher model to transfer knowledge to student models including CNN, MLP, and different ViT structures, achieving scalability.
π― What it does: This paper proposes and validates that using a constant learning rate combined with short-term cooldown instead of the traditional cosine learning rate scheduling in large-scale language model training can achieve performance comparable to or even better than cosine, without the need to pre-determine the number of training steps. Additionally, it introduces Stochastic Weight Averaging (SWA) and a Scheduler-Free Optimizer (SFO) to further enhance model quality during the training process.
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
Chaofan Tao (University of Hong Kong), Ngai Wong (University of Hong Kong)
CodeLarge Language ModelText
π― What it does: This study investigates the impact of vocabulary size on the scaling behavior of large language models (LLMs) by training models with parameters ranging from 33 million to 3 billion, using different vocabulary configurations to analyze how vocabulary size affects model performance.
π― What it does: This study investigates how to utilize Heterogeneous Pre-trained Transformers (HPT) for policy learning across various robot postures and tasks, constructing a shareable backbone network and fine-tuning it for different tasks.
π― What it does: A 1.4 trillion token multi-domain retrieval memory, MASSIVEDS, was constructed, and the performance improvement and computational optimality of retrieval-based language models were systematically studied under different storage scales.
Scaling transformer neural networks for skillful and reliable medium-range weather forecasting
Tung Nguyen (University of California Los Angeles), Aditya Grover (University of California Los Angeles)
CodeTransformerTime Series
π― What it does: This paper proposes a simple Transformer structure called Stormer, which achieves performance comparable to or even better than current state-of-the-art methods in weather prediction tasks, while significantly reducing training data and computational costs.
π― What it does: A general method called TRODO is proposed, which induces the model to misclassify out-of-vocabulary (OOV) samples as in-distribution (ID) by applying slight adversarial perturbations, thereby scanning for the presence of backdoors.
π― What it does: This paper proposes a scene graph-based image generation framework called DisCo, which can disentangle spatial layout and interaction semantics from textual scene graphs and generate diverse and relationship-compliant complex scene images through a diffusion model.
Scene Graph Generation with Role-Playing Large Language Models
Guikun Chen (Zhejiang University), Wenguan Wang (National Key Laboratory of Human-Machine Hybrid Augmented Intelligence)
CodeGenerationRetrievalTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityPhysics Related
π― What it does: This paper studies Open Vocabulary Scene Graph Generation (OVSGG) and proposes a solution to the scene blind spots and misleading issues of traditional fixed prompt schemes by generating Scene-Specific Descriptions (SSD) through large language models (LLM) and combining them with the CLIP visual-language model.
Schur Nets: exploiting local structure for equivariance in higher order graph neural networks
QINGQI ZHANG, Risi Kondor (University of Chicago)
CodeDrug DiscoveryGraph Neural NetworkGraph
π― What it does: This paper proposes a Schur network based on spectral graph theory, utilizing graph Laplacian spectral decomposition to construct automata for isomorphic equivariant linear mappings of subgraphs, thereby making fuller use of subgraph structures in higher-order graph neural networks.
π― What it does: This paper proposes a 3D molecular generation method called FuncMol based on continuous atomic occupancy fields, utilizing neural fields to encode molecular structures and perform score-based walk-jump sampling in the latent space.
SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers
Parsa Esmati (University of Bristol), NicolΓ² Grilli (University of Bristol)
CodeTransformerAuto EncoderMeshPhysics Related
π― What it does: A State-Exchange Attention (SEA) module based on Transformer is proposed, combined with a ViT grid autoencoder, to enhance the autoregressive inference accuracy of computational fluid dynamics (CFD) simulations and significantly reduce rolling errors.
Xuan Shen (Northeastern University), Yanzhi Wang (Northeastern University)
CodeComputational EfficiencyNeural Architecture SearchTransformerLarge Language ModelText
π― What it does: A training-free architecture search framework is proposed, which can automatically mine efficient sub-networks from existing large language models and recalibrate weights through ADMM to enhance performance.
π― What it does: A training framework for RNNs based on second-order forward mode optimization (SOFO) is proposed and implemented, which can efficiently and low-memory train neural networks on long sequences.
π― What it does: A Video Super-Resolution framework called SeeClear based on diffusion models has been developed, achieving high-quality super-resolution through semantic extraction and pixel compression.
π― What it does: This paper studies environmental dynamics modeling in multimodal visual reinforcement learning and proposes the Dissected Dynamics Modeling (DDM) method, which separates and models the consistent and inconsistent features between modalities to obtain a more refined and effective state representation.
SEEV: Synthesis with Efficient Exact Verification for ReLU Neural Barrier Functions
Hongchao Zhang (Washington University in St. Louis), Andrew Clark (Washington University in St. Louis)
CodeOptimizationComputational Efficiency
π― What it does: This paper proposes the SEEV framework for efficient synthesis and verification of ReLU Neural Control Barrier Functions (NCBF), reducing the number of activation regions on the safety boundary through regularization, and designing efficient enumeration and verification algorithms.
π― What it does: The Segment Any Change (AnyChange) model is proposed to achieve zero-shot change detection, capable of identifying any changes in remote sensing images at two time points in fully automated, semi-automated, and interactive modes, and can perform object-level change detection through point queries.
π― What it does: We propose UnSAM, a completely unsupervised image segmentation framework capable of achieving full-image segmentation and prompt segmentation; it trains the model using self-generated hierarchical pseudo-labels and achieves performance close to or even surpassing that of the supervised SAM.
Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations
Shivam Grover (Queen's University), Ali Etemad (Queen's University)
CodeClassificationAnomaly DetectionRepresentation LearningConvolutional Neural NetworkTransformerTime Series
π― What it does: A simple pluggable network layer S3 (SegmentβShuffleβStitch) is proposed, which rearranges time steps by segmenting, learning-based shuffling, and stitching of time series, thereby enhancing the effectiveness of temporal representation learning.
Xingchi Li (Texas A&M University), Xianyang Zhang (Texas A&M University)
CodeSegmentationLarge Language ModelText
π― What it does: A statistical method based on randomization testing and change point detection is proposed for identifying and segmenting watermarked text;
π― What it does: A foundational model for 3D medical image segmentation named SegVol has been trained and released, supporting various spatial and semantic prompts, capable of precise segmentation of over 200 anatomical categories.
SEL-BALD: Deep Bayesian Active Learning with Selective Labels
Ruijiang Gao (University of Texas at Dallas), Maytal Saar-Tsechansky (University of Texas at Austin)
CodeReinforcement LearningImageTabularFinance Related
π― What it does: This paper studies the active learning issue of human reviewers potentially rejecting labeled samples in high-risk decision-making scenarios (ALIR) and proposes a series of deep Bayesian active learning algorithms, SEL-BALD, to achieve efficient sample acquisition and labeling.
SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
Liangxin Liu (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: For the task of instruction tuning (Instruction Tuning) for LLMs, a new data selection method called SelectIT is proposed, which utilizes the model's own uncertainty to perform three layers of self-reflection, thereby filtering out high-quality instruction-response pairs and constructing the Selective Alpaca dataset based on this.
Lucas Monteiro Paes (Harvard University), Flavio Calmon
CodeExplainability and InterpretabilityTransformerSupervised Fine-TuningTextTabular
π― What it does: This paper proposes a selective explanation framework that first identifies which samples have low quality of model-inherent explanations (amortized) through uncertainty measurement, and then uses additional Monte Carlo inference on these samples to improve explanation quality.
Selective Generation for Controllable Language Models
Minjae Lee (POSTECH), Sangdon Park (POSTECH)
CodeGenerationTransformerLarge Language ModelText
π― What it does: This paper proposes a controllable generation framework that controls the error rate of text generation in generative language models through selective generation.
Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection
Geng Yu (Shanghai Jiao Tong University), Bo Han (Hong Kong Baptist University)
CodeAnomaly DetectionTransformerPrompt EngineeringVision Language ModelImageMultimodality
π― What it does: Proposes the Self-Calibrated Tuning (SCT) framework, which enhances the OOD detection performance of VLM by adaptively adjusting the weights of ID classification and OOV regularization.
Self-Distilled Depth Refinement with Noisy Poisson Fusion
Jiaqi Li (Huazhong University of Science and Technology), Jianming Zhang (Adobe Research)
CodeDepth EstimationKnowledge DistillationImage
π― What it does: This paper proposes the Self-Distilled Depth Refinement (SDDR) framework, treating depth refinement as a noise Poisson fusion problem with local inconsistent noise and edge distortion noise. It achieves high-resolution, detail-rich, and edge-clear depth maps through coarse-to-fine self-distillation to produce low-noise depth edge representations and edge-based guidance.
Self-Guiding Exploration for Combinatorial Problems
Zangir Iklassov (Mohamed bin Zayed University of Artificial Intelligence), Martin TakΓ‘Δ (Mohamed bin Zayed University of Artificial Intelligence)
CodeOptimizationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a new prompting strategy called Self-Guiding Exploration (SGE) for utilizing large language models (LLMs) to solve combinatorial optimization problems (CP) and other reasoning tasks.
Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments
Paulius Rauba (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeLarge Language ModelPrompt EngineeringTabularChain-of-Thought
π― What it does: This paper proposes and implements a Self-Healing Machine Learning (SHML) framework that can automatically monitor model performance degradation, diagnose root causes, generate and evaluate corresponding adaptive measures, ultimately achieving automatic model repair.
Andrea Corsini (University of Modena and Reggio Emilia), Mauro Dell'Amico (University of Modena and Reggio Emilia)
CodeOptimizationGraph Neural NetworkTabular
π― What it does: A self-supervised self-labeling improvement method (SLIM) is proposed for combinatorial optimization problems, and it is used to train a generation model based on Pointer Network to solve the Job Shop Scheduling Problem;
π― What it does: A self-play based diffusion model fine-tuning method (SPIN-Diffusion) is proposed, which enables iterative self-improvement for text-to-image generation using a dataset with only a single image/text pair.
Self-playing Adversarial Language Game Enhances LLM Reasoning
Pengyu Cheng (Tencent AI Lab), Xiaolong Li (Tencent AI Lab)
CodeTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
π― What it does: By having the LLM first imitate GPT-4 to conduct Adversarial Taboo dialogues, and then continuously improving its reasoning ability through self-play using offline reinforcement learning.
π― What it does: A self-refining diffusion sampler (SRDS) is implemented through Parareal iteration, reducing sampling latency through parallelization while maintaining sample quality.
Self-Retrieval: End-to-End Information Retrieval with One Large Language Model
Qiaoyu Tang (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Yongbin Li (Alibaba Group)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes Self-Retrieval, an end-to-end information retrieval framework driven entirely by a single large language model, unifying indexing, retrieval, and re-ranking within the model parameters.
Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation
Ruize Zhang (Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences), Juan Cao (Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences)
π― What it does: A self-supervised adversarial training framework DAQ-SDP is proposed to address the issues of large robust generalization gaps and decreased clean accuracy in self-supervised adversarial training (SAT);
CodeTransformerLarge Language ModelContrastive LearningText
π― What it does: A self-supervised alignment algorithm called SAMI is proposed, which utilizes mutual information maximization to enable language models to adhere to given behavioral principles (constitutions) without the need for preference labels or examples.
π― What it does: A self-supervised transformation learning (STL) method is proposed, which learns equivariant representations through the representation of original images and transformed image pairs, without the need for manual transformation labels, enhancing the generalization and detection performance of unsupervised representations.
π― What it does: This paper proposes an unsupervised source domain data adaptation framework called STAR, which utilizes large speech foundation models like Whisper to self-learn on unlabeled speech in the target domain, achieving domain adaptation.
Yuxiang Wei (University of Illinois Urbana-Champaign), LINGMING ZHANG
CodeGenerationKnowledge DistillationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The SelfCodeAlign method is proposed, which completes the alignment and fine-tuning of code LLMs without manual annotation or teacher model distillation through self-generated instructions, responses, and tests.
Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space
Xin Qiu (Cognizant AI Labs), Risto Miikkulainen (Cognizant AI Labs)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes and implements a confidence measurement method called 'Semantic Density' that is response-oriented, requires no additional training, and can be directly applied to any large language model.
Semantics and Spatiality of Emergent Communication
Rotem Ben Zion (Technion Israel Institute of Technology), Yonatan Belinkov (Technion Israel Institute of Technology)
CodeImage
π― What it does: This paper theoretically analyzes and empirically demonstrates the impact of different objectives (reconstruction vs. discrimination) on the generation of semantically consistent and spatially meaningful communication protocols in a co-training environment of artificial intelligence.
SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning
Yangruibo Ding (Columbia University), Baishakhi Ray (Columbia University)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A 6.7B parameter code language model, SEMCODER, was trained, outperforming GPT-3.5-turbo and most open-source models in code generation and execution reasoning tasks.