IEEE/CVF Conference on Computer Vision and Pattern Recognition · 892 papers
CapsFusion: Rethinking Image-Text Data at Scale
Qiying Yu (Institute for AI Industry Research), Jingjing Liu (Institute for AI Industry Research)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A framework named CAPSFUSION is proposed, which integrates raw web text and synthetic subtitles through large language models to generate high-quality multimodal data and train LMM;
Causal Mode Multiplexer: A Novel Framework for Unbiased Multispectral Pedestrian Detection
Taeheon Kim (KAIST), Yong Man Ro (KAIST)
CodeObject DetectionImage
🎯 What it does: This paper proposes a Causal Mode Multiplexer (CMM) framework that utilizes causal intervention and counterfactual inference to address the modality bias problem in multispectral pedestrian detection.
🎯 What it does: A content-aware degradation-driven transformation network (CDFormer) for blind image super-resolution is proposed, which utilizes a diffusion model to estimate the content degradation prior (CDP) of low-resolution images and achieves efficient feature fusion through attention and interflow mechanisms, ultimately generating high-quality super-resolved images.
🎯 What it does: The CDMAD algorithm is proposed to address the issue of mismatched class distribution between labeled and unlabeled data in class-imbalanced semi-supervised learning. It utilizes unpatterned images to compute classifier bias and corrects pseudo-labels and test predictions, thereby reducing classifier bias and improving representation quality.
🎯 What it does: A digital copyright identification method based on contrast gradient inversion, CGI-DM, is proposed, which utilizes partial image information recovery to verify whether the diffusion model has memorized the training samples.
ChAda-ViT : Channel Adaptive Attention for Joint Representation Learning of Heterogeneous Microscopy Images
Nicolas Bourriez (Ecole Normale Superieure PSL), Auguste Genovesio (Ecole Normale Superieure PSL)
CodeRepresentation LearningTransformerContrastive LearningImageBiomedical Data
🎯 What it does: A Transformer architecture called ChAda-ViT is proposed, capable of handling microscope images with arbitrary channel numbers and types, achieving a unified embedding space.
🎯 What it does: This paper proposes CHAIN (lipsCHitz continuity constrAIned Normalization), an improved normalization method specifically designed for GAN discriminators to enhance generalization and training stability in data-scarce scenarios.
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
Peng Jin (Peking University), Li Yuan (Peking University)
CodeGenerationRepresentation LearningTransformerLarge Language ModelVision Language ModelImageVideoMultimodality
🎯 What it does: This paper proposes and implements Chat-UniVi, a unified visual-language model capable of understanding images and videos within the same framework and engaging in natural dialogue with large language models.
ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles
Jiawei Zhang (University of Illinois Urbana-Champaign), Bo Li (University of Chicago)
CodeAutonomous DrivingSafty and PrivacyTransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes the ChatScene agent based on large language models, which can automatically generate natural language descriptions of safety-critical scenarios and convert them into Scenic code that can be executed in the CARLA simulation environment, thereby producing diverse and complex safety testing scenarios.
🎯 What it does: The DARLING framework is proposed to achieve decoupled representation learning for scene text multi-tasks (recognition, editing, removal).
Circuit Design and Efficient Simulation of Quantum Inner Product and Empirical Studies of Its Effect on Near-Term Hybrid Quantum-Classic Machine Learning
🎯 What it does: This paper designs quantum inner product (QIP) circuits suitable for 1-to-1, 1-to-N, and M-to-N scenarios, and proposes an efficient scheme for directly simulating output states, enabling rapid computation of quantum inner products on classical machines while maintaining differentiability.
Class Incremental Learning with Multi-Teacher Distillation
Haitao Wen (University of Electronic Science and Technology of China), Hongliang Li (University of Electronic Science and Technology of China)
CodeClassificationKnowledge DistillationImage
🎯 What it does: This paper proposes a multi-teacher distillation framework (MTD) to alleviate catastrophic forgetting in class-incremental learning by generating diverse teachers from a single model.
Class Tokens Infusion for Weakly Supervised Semantic Segmentation
Sung-Hoon Yoon (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
CodeSegmentationTransformerImage
🎯 What it does: A weakly supervised semantic segmentation framework based on Vision Transformer is proposed, which enhances the distinguishability of class tokens and the quality of CAM by injecting class tokens (I-CTI and C-CTI) and background tokens into the Transformer.
CLIB-FIQA: Face Image Quality Assessment with Confidence Calibration
Fu-Zhao Ou (City University of Hong Kong), Sam Kwong (Lingnan University)
CodeClassificationRecognitionVision Language ModelContrastive LearningImage
🎯 What it does: This paper utilizes the CLIP visual-language pre-training model to jointly learn multiple quality factors (blurriness, pose, expression, occlusion, lighting) and quality distribution, proposing a confidence-calibrated facial image quality assessment method (CLIB-FIQA) that achieves accurate prediction of facial image quality and calibrates the confidence of quality anchors.
🎯 What it does: This paper proposes a knowledge distillation framework for the CLIP model called CLIP-KD, which uses a large teacher CLIP to train a small student CLIP, significantly improving the zero-shot classification and cross-modal retrieval performance of the small model.
🎯 What it does: This paper proposes CLIPtone, an unsupervised learning framework for text-driven image tone adjustment that can adaptively adjust the tone of images based on natural language descriptions.
🎯 What it does: This paper proposes Clockwork Diffusion, which achieves efficient sampling for text-to-image generation and image editing by combining model distillation and step distillation on low-resolution UNet representations.
🎯 What it does: A specialized reproduction of black and white photography is proposed based on deep metric learning with a proxy network and multi-level bilateral grid.
Ruijie Quan (Zhejiang University), Yi Yang (Zhejiang University)
CodeRepresentation LearningDrug DiscoveryGraph Neural NetworkGraphBiomedical Data
🎯 What it does: An end-to-end learnable neural clustering framework is proposed, which automatically discovers key amino acids in proteins through iterative spherical clustering, attention clustering representation, and GCN selection of key clusters, thereby achieving protein representation learning.
🎯 What it does: A unified medical image segmentation framework S2VNet is proposed, capable of achieving both automatic and interactive voxel segmentation in a single model and single training session.
🎯 What it does: A self-supervised CNC-Net is proposed, which predicts the sequence of milling, drilling, and rotation operations required by CNC machines from 3D models, gradually processing raw materials to obtain the target object.
Xueqing Deng (ByteDance), Liang-Chieh Chen (ByteDance)
CodeObject DetectionSegmentationTransformerImage
🎯 What it does: Created the COCONut dataset, which contains 383K images and 5.18M high-quality, manually verified segmentation masks, uniformly covering three tasks: semantic, instance, and panoptic segmentation.
Wenyi Hong (Tsinghua University), Jie Tang (Tsinghua University)
CodeTransformerVision Language ModelImageTextMultimodality
🎯 What it does: A visual language model called CogAgent, focused on GUI understanding and navigation with 18 billion parameters, has been developed, and high-resolution input support has been implemented.
Collaborative Learning of Anomalies with Privacy (CLAP) for Unsupervised Video Anomaly Detection: A New Baseline
Anas Al-lahham (Mohamed bin Zayed University of Artificial Intelligence), Karthik Nandakumar (Mohamed bin Zayed University of Artificial Intelligence)
CodeAnomaly DetectionFederated LearningSafty and PrivacyConvolutional Neural NetworkTransformerVideo
🎯 What it does: A federated learning framework called CLAP is proposed, which can train video anomaly detection models in a multi-participant environment in a completely unsupervised manner, while not sharing the original video data.
🎯 What it does: A method for color distortion estimation and correction for images containing both overexposed and underexposed regions is proposed, utilizing pseudo-normal exposure features to guide color shift correction and generate enhanced images.
Commonsense Prototype for Outdoor Unsupervised 3D Object Detection
Hai Wu (Xiamen University), Cheng Wang (Xiamen University)
CodeObject DetectionAutonomous DrivingPoint Cloud
🎯 What it does: This paper proposes an unsupervised 3D object detection framework called CPD based on consensus prototypes, which generates pseudo-labels through multi-frame clustering and refines the labels using consensus prototypes, ultimately achieving high-precision detection in sparse scanning environments.
🎯 What it does: Proposes FedACG, which utilizes the server to broadcast a global model with lookahead gradients as local initialization, and adds a regularization term to accelerate and stabilize federated learning.
Compositional Video Understanding with Spatiotemporal Structure-based Transformers
Hoyeoung Yun (Hanyang University), Eun-Sol Kim (Hanyang University)
CodeRecognitionObject DetectionTransformerVideo
🎯 What it does: This study proposes a spatiotemporal structure Transformer (ST-GT) based on objects, which can construct symbolic graphs from long videos and learn higher-order semantic relationships between object-level nodes and edges, achieving the decomposition and reconstruction of overall video semantics.
🎯 What it does: A compression method for 3D Gaussian splat scene representation is proposed, achieving low memory and real-time rendering through hardware rasterization.
ConCon-Chi: Concept-Context Chimera Benchmark for Personalized Vision-Language Tasks
Andrea Rosasco (University of Genoa), Lorenzo Natale (Istituto Italiano di Tecnologia)
CodeGenerationRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: Designed and released the ConCon-Chi dataset, constructing a concept-context matrix containing 20 novel 'hybrid' concepts and 101 diverse contexts, aimed at evaluating the learning of new meanings and combinatorial capabilities of personalized text-to-image retrieval (TIR) and generation (TIG) models.
🎯 What it does: A Condition-Aware Neural Network (CAN) is proposed, which achieves controllable image generation for diffusion generative models by dynamically generating and integrating condition-aware weights.
🎯 What it does: This paper proposes a 'Consistent Prompting (CPrompt)' framework to address the inconsistency between prompt and classifier training and inference in rehearsal-free continual learning.
ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion
Jiayu Yang (Tencent), Hongdong Li (Australian National University)
CodeGenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes the ConsistNet module, which inserts lightweight multi-view consistency blocks into the pre-trained Latent Diffusion Model (Zero123-XL) to generate multi-view 3D consistent images from a single input image.
🎯 What it does: This paper proposes a new mixed-domain semi-supervised medical image segmentation scenario (MiDSS) to address the coexistence of limited annotations and domain shift by constructing an intermediate domain.
Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening
Yule Duan (University of Electronic Science and Technology of China), Liang-Jian Deng (University of Electronic Science and Technology of China)
CodeRestorationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes Content Adaptive Non-local Convolution (CANConv) and its network CANNet for pansharpening of remote sensing images.
🎯 What it does: An unsupervised makeup transfer method CSD-MT is proposed, which decouples content and makeup style through frequency decomposition and trains directly on source and reference images.
Context-Aware Integration of Language and Visual References for Natural Language Tracking
Yanyan Shao (Zhejiang University of Technology), Jiming Chen (Zhejiang University)
CodeObject TrackingRetrievalTransformerPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes QueryNLT, an end-to-end natural language tracking framework that integrates language descriptions with visual templates, consisting of two main modules: Prompt Modulation and Target Decoding.
🎯 What it does: In the video localization task with a given text query, the authors propose the Context-Guided STVG framework (CG-STVG), which assists target localization by mining instance visual context from the video itself.
Continual Forgetting for Pre-trained Vision Models
Hongbo Zhao (State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences), Zhaoxiang Zhang (State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences)
CodeRecognitionObject DetectionSafty and PrivacyTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes the problem of 'Continual Forgetting', aiming to sequentially and gradually delete specified knowledge (such as privacy, bias, etc.) from pre-trained visual models while keeping the remaining knowledge unaffected. To achieve this, the authors designed the GS-LoRA method: adding a LoRA low-rank adaptation module to the FFN layer of the Transformer block and automatically selecting the Transformer blocks that need modification through group sparse regularization, thus enabling efficient and precise forgetting.
🎯 What it does: This paper proposes the CoMasTRe framework, which achieves continuous segmentation by separating object-centric learning and category recognition.
🎯 What it does: This paper proposes a continuous self-supervised learning framework MedCoSS for constructing multimodal medical pre-training models, addressing the issues of multimodal data conflicts and model scalability.
🎯 What it does: This paper constructs a continuous optical zoom dataset COZ and proposes the Local Mix Implicit Network (LMI) based on MLP-Mixer and meta-learning for super-resolution of images at arbitrary scales in realistic scenes.
Continuous Pose for Monocular Cameras in Neural Implicit Representation
Qi Ma (ETH Zurich), Luc Van Gool (ETH Zurich)
CodePose EstimationNeural Radiance FieldSimultaneous Localization and MappingImageVideo
🎯 What it does: This paper proposes an end-to-end optimization framework that represents the pose of a monocular camera as a time-continuous neural function, integrating it with NeRF, event cameras, visual SLAM, and IMU.
🎯 What it does: This paper proposes the Contrastive Denoising Score (CDS), which achieves text-guided latent diffusion image editing through zero-shot training by incorporating the CUT loss into the Delta Denoising Score (DDS) framework.
🎯 What it does: This paper proposes a coarse-fine level co-registration network for medical image deformation based on a multi-layer perceptron (CorrMLP), and conducts unsupervised training and evaluation on brain MRI and cardiac MRI.
CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation
Boyuan Sun (Nankai University), Qibin Hou (Nankai University)
CodeSegmentationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes CorrMatch, a semi-supervised semantic segmentation method that achieves pixel and region propagation through correlation matching, utilizing the correlation information of unlabeled images to generate more accurate pseudo-labels.
🎯 What it does: Proposes the Compact Occupancy Transformer (COTR), achieving vision-based 3D occupancy prediction, constructing a compact 3D OCC representation, and enhancing semantic recognition.
Coupled Laplacian Eigenmaps for Locally-Aware 3D Rigid Point Cloud Matching
Matteo Bastico (Mines Paris), David Ryckelynck
CodePose EstimationAnomaly DetectionPoint Cloud
🎯 What it does: The Coupled Laplacian is proposed to generate an aligned Laplacian feature space by adding cross edges on registered point clouds, achieving unsupervised 3D rigid point cloud matching for local geometric details, and applying it to bone side estimation and industrial 3D anomaly detection.
🎯 What it does: This paper proposes a coding prior-based aggregation network (CPGA) and a corresponding coding prior dataset (VCP) to enhance the quality of compressed videos.
🎯 What it does: This study investigates the global feature representation for visual place recognition and proposes a cross-image correlation-aware method.
🎯 What it does: This paper studies a facial personalization method for text-to-image diffusion models, proposing cross-initialization and regularization strategies to enhance reconstruction quality and editability, while significantly accelerating the optimization process.
Cross-Dimension Affinity Distillation for 3D EM Neuron Segmentation
Xiaoyu Liu (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
CodeSegmentationKnowledge DistillationConvolutional Neural NetworkSupervised Fine-TuningBiomedical Data
🎯 What it does: A lightweight 2D convolutional network is used to generate a 3D affinity map, enabling efficient segmentation of neurons in electron microscopy (EM) volumes.
🎯 What it does: This paper studies cross-domain few-shot semantic segmentation (CD-FSS) and proposes an Iterative Few-Shot Adapter (IFA) to maximize the utilization of the limited supervisory signals from support samples, addressing the dual challenges of cross-domain transfer and overfitting.
CrossKD: Cross-Head Knowledge Distillation for Object Detection
Jiabao Wang (Nankai University), Qibin Hou (Nankai University)
CodeObject DetectionKnowledge DistillationImage
🎯 What it does: A new knowledge distillation method called CrossKD is proposed, where the intermediate features of the student detection head are sent to the teacher detection head to generate cross predictions. These cross predictions are then aligned with the teacher's predictions, alleviating target conflicts in prediction simulation and enhancing the student's detection performance.
D3still: Decoupled Differential Distillation for Asymmetric Image Retrieval
Yi Xie (South China University of Technology), Shengfeng He (Singapore Management University)
CodeRetrievalKnowledge DistillationImage
🎯 What it does: A decoupled differential distillation framework (D3still) is proposed, which enhances the ranking consistency and performance of lightweight query networks in heterogeneous image retrieval by decomposing the similarity differential matrix in the atlas domain into feature knowledge, asynchronous similarity differential knowledge, and synchronous similarity differential knowledge.
🎯 What it does: The D3T framework is proposed, which uses two specialized teacher models (RGB and thermal imaging) and a zigzag learning strategy to transfer the visible light domain detection model to the thermal imaging domain, achieving unsupervised domain adaptation.
Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement
Ziyu Wang (Shanghai Jiao Tong University), Yong-Lu Li (Shanghai Jiao Tong University)
CodeKnowledge DistillationVideo
🎯 What it does: A dataset distillation method for video data is proposed, which first learns static information through image distillation and then compensates for dynamic information using a learnable dynamic memory network, achieving efficient video distillation.
Data Valuation and Detections in Federated Learning
Wenqian Li (National University of Singapore), Yan Pang (National University of Singapore)
CodeAnomaly DetectionFederated LearningSafty and PrivacyComputational EfficiencyImage
🎯 What it does: Proposes the FedBary method, which utilizes the Wasserstein barycenter to evaluate client contributions and detect noisy data in federated learning without sharing raw data.
🎯 What it does: This paper proposes a framework called DNDM for Day-Night Cross-Domain Vehicle ReID, addressing issues such as glare from headlights at night, low-light environments, and inter-domain differences.
Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlation
Fahimeh Hosseini Noohdani (Sharif University of Technology), Mahdieh Soleymani Baghshah (Sharif University of Technology)
CodeDomain AdaptationExplainability and InterpretabilityConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A Decompose-and-Compose (DaC) method is proposed, which utilizes the interpretability of standard ERM models (xGradCAM) to first identify the causal and non-causal parts of images, and then combines parts from different images to create new samples, thereby eliminating the correlation between non-causal features and labels, thus enhancing the model's robustness under distribution shifts.
Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework
Vu Minh Hieu Phan (Australian Institute for Machine Learning, The University of Adelaide), Johan W. Verjans (Australian Institute for Machine Learning, The University of Adelaide)
CodeClassificationRecognitionTransformerLarge Language ModelVision Language ModelContrastive LearningImageMultimodalityBiomedical DataComputed Tomography
🎯 What it does: A multi-faceted visual-language pre-training framework MAVL is proposed, which decomposes disease descriptions into visual features and performs multi-faceted matching.
🎯 What it does: This paper proposes a framework that decomposes the reference video segmentation task into static perception and hierarchical motion perception, and utilizes contrastive learning to enhance the ability to distinguish similar moving objects.
Deep Equilibrium Diffusion Restoration with Parallel Sampling
Jiezhang Cao (ETH Zurich), Luc Van Gool (KU Leuven)
CodeRestorationDiffusion modelImage
🎯 What it does: A zero-shot diffusion model image restoration method based on Deep Equilibrium (DEQ) is designed, which allows for parallel sampling and improves image quality and controls the generation direction through initialization optimization.
🎯 What it does: A new image downsampling evaluation metric IDA-RD based on rate-distortion theory is proposed, and blind random super-resolution is implemented through deep generative models to measure the extent to which downsampling algorithms retain high-resolution information.
🎯 What it does: To address the issue of sample imbalance in visual regression tasks, a Hierarchical Classification Adjustment (HCA) framework is proposed and implemented, utilizing a combination of multi-level fine and coarse classifiers to reduce quantization error and improve regression accuracy.
🎯 What it does: A novel pipeline for Inverse Tone Mapping using video temporal information is proposed, which can recover HDR video from a single frame LDR video and address the issues of texture loss in overexposed areas and temporal inconsistency.
Deep-TROJ: An Inference Stage Trojan Insertion Algorithm through Efficient Weight Replacement Attack
Sabbir Ahmed (Binghamton University), Adnan Siraj Rakin (New Jersey Institute of Technology)
CodeTransformerImage
🎯 What it does: This paper addresses deep neural network models that inject malicious backdoors through memory page table bit flips during the inference phase.
🎯 What it does: A training-free cache acceleration scheme called DeepCache is proposed, which utilizes the temporal redundancy of high-level features in continuous denoising steps of diffusion models. It caches features on the skip connections of U-Net to reduce redundant computations, thereby accelerating inference.
Defense Against Adversarial Attacks on No-Reference Image Quality Models with Gradient Norm Regularization
Yujia Liu (Peking University), Tingting Jiang (Peking University)
CodeOptimizationAdversarial AttackImage
🎯 What it does: This study investigates the robustness of no-reference image quality assessment (NR-IQA) models against adversarial attacks and proposes a defense training method based on gradient ℓ1 norm regularization.
🎯 What it does: This paper proposes a method based on Deformable 3D Gaussians for high-fidelity reconstruction and real-time rendering of monocular dynamic scenes.
🎯 What it does: A deformable network for single facial stylization is proposed, which uses only a pair of real style images and is trained through cross-domain structural guidance using DINO self-supervised ViT features.
DeIL: Direct-and-Inverse CLIP for Open-World Few-Shot Learning
Shuai Shao (Zhejiang Lab), Yicong Zhou (University of Macau)
CodeClassificationData SynthesisTransformerVision Language ModelContrastive LearningImage
🎯 What it does: The DeIL method is proposed, utilizing the Direct-and-Inverse approach in Open-World Few-Shot Learning to first identify and correct noisy labels using CLIPN, and then perform data augmentation and classification with CLIP, DALL-E, and a lightweight Adapter.
Delving into the Trajectory Long-tail Distribution for Muti-object Tracking
Sijia Chen (Huazhong University of Science and Technology), Wenbing Tao (Huazhong University of Science and Technology)
CodeObject TrackingDiffusion modelVideo
🎯 What it does: This paper presents an analysis and solution to the severe imbalance in trajectory length distribution (long-tail distribution) in multi-object tracking data, and validates its effectiveness in mainstream models such as FairMOT and CSTrack.
🎯 What it does: A two-view matching network called DeMatch is proposed, which uses learned motion bases to decompose a rough motion field into several groups of smooth sub-fields, thereby achieving precise inlier and outlier discrimination for matching points.
🎯 What it does: This paper proposes a method for point cloud denoising in the latent space, utilizing reversible monotonic operators to construct a reversible neural network, and combining multi-layer graph convolution to extract geometric features, achieving explicit separation and denoising of noise in the latent space.
🎯 What it does: By designing a density-guided translator (DGT) based on point density and a two-stage self-supervised training process (DGT-ST), unsupervised adaptation of 3D point clouds from synthetic domains to real domains has been achieved.
Ji Zhang (University of Electronic Science and Technology of China), Jingkuan Song (Tongji University)
CodeClassificationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: This paper proposes a decoupled prompt tuning framework named DePT to address the base-new task generalization problem in visual-language pre-trained models.
🎯 What it does: A single image dehazing framework based on dual-task collaborative mutual promotion is proposed, which jointly optimizes dehazing and depth estimation.
🎯 What it does: A Dexterous Grasp Transformer (DGTR) based on Transformer is proposed, capable of predicting diverse multi-finger grasp poses in a single forward pass.
🎯 What it does: Proposes the Diff-BGM framework, which generates background music that matches the dynamics and semantics of videos based on diffusion models, and constructs a high-quality BGM909 video-music alignment dataset.
DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model
Lirui Zhao (Xiamen University), Rongrong Ji (Xiamen University)
CodeGenerationRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringImageTextBenchmark
🎯 What it does: Designed and implemented DiffAgent, an LLM-based agent model that quickly selects suitable text-to-image (T2I) APIs (model + parameters) based on text prompts, thereby reducing the user's trial-and-error and parameter tuning process.
DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection
Yuhao Sun (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
CodeSafty and PrivacyAdversarial AttackDiffusion modelImage
🎯 What it does: DiffAM is a facial privacy protection method that utilizes diffusion models to confuse facial recognition systems by transforming reference facial makeup into adversarial makeup.
🎯 What it does: A unified graph diffusion model called DiffAssemble is proposed to simultaneously address the reassembly tasks of 2D puzzles and 3D fragments.
DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcasting
Demin Yu (Harbin Institute of Technology), Xunlai Chen (Shenzhen Meteorological Bureau)
CodeGenerationData SynthesisConvolutional Neural NetworkRecurrent Neural NetworkDiffusion modelTime Series
🎯 What it does: A unified forecasting framework called DiffCast is proposed, which utilizes residual diffusion to decompose the evolution of precipitation systems into global deterministic motion trends and local stochastic residuals, compensating for the fuzziness and high-value rainfall missing issues in deterministic predictions through residual diffusion.
🎯 What it does: We propose DiffEditor, a fine-grained image editing framework based on diffusion models, which supports various editing tasks such as object movement, size adjustment, dragging, appearance replacement, and object pasting.
🎯 What it does: The DiffLoc framework is proposed, treating LiDAR localization as a conditional generation problem. It learns robust features through a base model and a static object pool, and then uses a diffusion model to achieve iterative denoising regression.
DiffSCI: Zero-Shot Snapshot Compressive Imaging via Iterative Spectral Diffusion Model
Zhenghao Pan (Harbin Institute of Technology), Yongyong Chen (Nanjing University)
CodeRestorationCompressionDiffusion modelImage
🎯 What it does: Proposes the DiffSCI method, which combines a pre-trained RGB diffusion model with Plug-and-Play for multispectral SCI reconstruction, eliminating the need to retrain the model on MSI.
🎯 What it does: This paper proposes DIFF3F, a zero-shot semantic feature descriptor that distills diffusion model features onto untextured 3D shapes through multi-view rendering and a control network.
🎯 What it does: This paper proposes a multimodal face generation method that maps the features of diffusion models to the latent space of pre-trained GANs, capable of handling both 2D and 3D face images.
🎯 What it does: This paper proposes DiffusionRegPose, a model that transforms one-stage end-to-end multi-person human pose estimation into a diffusion sampling process, using a diffusion model to progressively denoise and obtain pose predictions. It also introduces an attention mechanism that facilitates interaction between pose completion and detection, enhancing robustness in crowded/occluded scenarios.
🎯 What it does: A target tracking method based on diffusion models, DiffusionTrack, is proposed, treating the tracking task as a gradual denoising process from noise to the target, and representing the target with a point set.
🎯 What it does: This paper proposes a self-distillation-based Slot Attention framework called DIOD for unsupervised video object discovery, which can automatically denoise and complete static objects under motion-guided noisy supervision.
🎯 What it does: We propose DIRECT-3D, a text-to-3D generation model that trains directly on massive unaligned 3D data with noise, capable of quickly generating high-quality NeRF objects and providing 3D geometric priors.
Discover and Mitigate Multiple Biased Subgroups in Image Classifiers
Zeliang Zhang (University of Rochester), Chenliang Xu (University of Rochester)
CodeClassificationExplainability and InterpretabilityData-Centric LearningLarge Language ModelImageRetrieval-Augmented Generation
🎯 What it does: The DIM framework is proposed, which utilizes PLS to decompose image features under the supervision of training dynamics, automatically discovering and explaining various unknown bias subgroups, and implementing dual bias mitigation for data and models based on these subgroups.