CVPR 2024 Papers — Page 6
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
Defense Against Adversarial Attacks on No-Reference Image Quality Models with Gradient Norm Regularization
Yujia Liu (Peking University), Tingting Jiang (Peking University)
OptimizationAdversarial AttackImage
🎯 What it does: This study investigates the robustness of no-reference image quality assessment (NR-IQA) models against adversarial attacks and proposes a defense training method based on gradient ℓ1 norm regularization.
Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay
Yuhang Zhou (Harbin Institute of Technology), Zhongyun Hua (Harbin Institute of Technology)
OptimizationKnowledge DistillationAdversarial AttackImage
🎯 What it does: A continuous adversarial defense baseline AIR is proposed, achieving memoryless continuous attack defense.
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction
Ziyi Yang (Zhejiang University), Xiaogang Jin (Zhejiang University)
RestorationData SynthesisNeural Radiance FieldGaussian SplattingVideo
🎯 What it does: This paper proposes a method based on Deformable 3D Gaussians for high-fidelity reconstruction and real-time rendering of monocular dynamic scenes.
Deformable One-shot Face Stylization via DINO Semantic Guidance
Yang Zhou (Visual Computing Research Center Shenzhen University), Hui Huang (Visual Computing Research Center Shenzhen University)
Image TranslationGenerationTransformerGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: A deformable network for single facial stylization is proposed, which uses only a pair of real style images and is trained through cross-domain structural guidance using DINO self-supervised ViT features.
Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories
Yan Zhang (ETH Zurich), Siyu Tang (ETH Zurich)
Point CloudTime Series
🎯 What it does: A compact implicit motion model called DOMA is proposed, aimed at capturing general dynamics of 3D scenes. By processing 3D points and 1D time steps, DOMA predicts an affine mapping, ensuring the generation of a temporally and spatially smooth motion field.
DeIL: Direct-and-Inverse CLIP for Open-World Few-Shot Learning
Shuai Shao (Zhejiang Lab), Yicong Zhou (University of Macau)
ClassificationData SynthesisTransformerVision Language ModelContrastive LearningImage
🎯 What it does: The DeIL method is proposed, utilizing the Direct-and-Inverse approach in Open-World Few-Shot Learning to first identify and correct noisy labels using CLIPN, and then perform data augmentation and classification with CLIP, DALL-E, and a lightweight Adapter.
DeiT-LT: Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets
Harsh Rangwani (Indian Institute of Science), R. Venkatesh Babu (Indian Institute of Science)
Knowledge DistillationRepresentation LearningTransformerImage
🎯 What it does: Without relying on large-scale pre-training, the DeiT-LT method is proposed to train Vision Transformers from scratch to handle long-tail distribution datasets.
Delving into the Trajectory Long-tail Distribution for Muti-object Tracking
Sijia Chen (Huazhong University of Science and Technology), Wenbing Tao (Huazhong University of Science and Technology)
Object TrackingDiffusion modelVideo
🎯 What it does: This paper presents an analysis and solution to the severe imbalance in trajectory length distribution (long-tail distribution) in multi-object tracking data, and validates its effectiveness in mainstream models such as FairMOT and CSTrack.
DeMatch: Deep Decomposition of Motion Field for Two-View Correspondence Learning
Shihua Zhang (Wuhan University), Jiayi Ma (Wuhan University)
Pose EstimationComputational EfficiencyGraph Neural NetworkOptical FlowImage
🎯 What it does: A two-view matching network called DeMatch is proposed, which uses learned motion bases to decompose a rough motion field into several groups of smooth sub-fields, thereby achieving precise inlier and outlier discrimination for matching points.
DemoCaricature: Democratising Caricature Generation with a Rough Sketch
Dar-Yen Chen (University of Surrey), Yi-Zhe Song (University of Surrey)
GenerationDiffusion modelImageBenchmark
🎯 What it does: A cartoon generation method based on Stable Diffusion for single portraits and sketches is proposed.
DemoFusion: Democratising High-Resolution Image Generation With No $$$
Ruoyi Du (Beijing University of Posts and Telecommunications), Zhanyu Ma (University of Edinburgh)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper presents DemoFusion, a fine-tuning-free inference framework based on SDXL that can generate high-resolution images up to 4096×4096 on a single RTX 3090.
Denoising Point Clouds in Latent Space via Graph Convolution and Invertible Neural Network
Aihua Mao (South China University of Technology), Ying He (Nanyang Technological University)
RestorationGraph Neural NetworkAuto EncoderPoint Cloud
🎯 What it does: This paper proposes a method for point cloud denoising in the latent space, utilizing reversible monotonic operators to construct a reversible neural network, and combining multi-layer graph convolution to extract geometric features, achieving explicit separation and denoising of noise in the latent space.
Dense Optical Tracking: Connecting the Dots
Guillaume Le Moing (Inria), Cordelia Schmid (Inria)
Object TrackingOptical FlowVideo
🎯 What it does: A dense optical flow tracking method named DOT is studied, which simultaneously achieves global dense motion estimation and occlusion prediction using a small number of point trajectories and an optical flow network.
Dense Vision Transformer Compression with Few Samples
Hanxiao Zhang (Nanjing University), Guo-Hua Wang (Nanjing University)
ClassificationCompressionTransformerImage
🎯 What it does: Proposes the DC-ViT framework, which performs few-shot compression on Vision Transformers by selectively removing attention modules and partially reusing MLPs to achieve dense compression.
Density-Adaptive Model Based on Motif Matrix for Multi-Agent Trajectory Prediction
Di Wen (Sun Yat-sen University), Peixi Peng (Peking University)
Autonomous DrivingGraph Neural NetworkAuto EncoderMultimodalityTime Series
🎯 What it does: This paper proposes a density-adaptive multi-agent trajectory prediction model (DAMM) based on a network motivation matrix, which can dynamically select the neighbor range and estimate interaction probabilities, thereby better capturing interaction behaviors under different traffic densities.
Density-Guided Semi-Supervised 3D Semantic Segmentation with Dual-Space Hardness Sampling
Jianan Li (University of Chinese Academy of Sciences), Qiulei Dong (Chinese Academy of Sciences)
SegmentationAutonomous DrivingContrastive LearningPoint Cloud
🎯 What it does: This paper proposes a semi-supervised 3D semantic segmentation method called DDSemi, which enhances the feature discrimination ability of unlabeled point clouds through density-guided contrastive learning and dual-space hard sample sampling.
Density-guided Translator Boosts Synthetic-to-Real Unsupervised Domain Adaptive Segmentation of 3D Point Clouds
Zhimin Yuan (Xiamen University), Cheng Wang (Xiamen University)
SegmentationDomain AdaptationConvolutional Neural NetworkGenerative Adversarial NetworkPoint Cloud
🎯 What it does: By designing a density-guided translator (DGT) based on point density and a two-stage self-supervised training process (DGT-ST), unsupervised adaptation of 3D point clouds from synthetic domains to real domains has been achieved.
DePT: Decoupled Prompt Tuning
Ji Zhang (University of Electronic Science and Technology of China), Jingkuan Song (Tongji University)
ClassificationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: This paper proposes a decoupled prompt tuning framework named DePT to address the base-new task generalization problem in visual-language pre-trained models.
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Lihe Yang (University of Hong Kong), Hengshuang Zhao
SegmentationDepth EstimationTransformerContrastive LearningImage
🎯 What it does: A foundational model for monocular depth estimation named Depth Anything was constructed based on a large number of unlabeled images;
Depth Information Assisted Collaborative Mutual Promotion Network for Single Image Dehazing
Yafei Zhang (Kunming University of Science and Technology), Huafeng Li (Kunming University of Science and Technology)
RestorationDepth EstimationConvolutional Neural NetworkImage
🎯 What it does: A single image dehazing framework based on dual-task collaborative mutual promotion is proposed, which jointly optimizes dehazing and depth estimation.
Depth Prompting for Sensor-Agnostic Depth Estimation
Jin-Hwi Park (Gwangju Institute of Science and Technology), Hae-Gon Jeon (Gwangju Institute of Science and Technology)
Depth EstimationTransformerPrompt EngineeringPoint Cloud
🎯 What it does: A Depth Prompting framework is proposed, utilizing a pre-trained monocular depth model to achieve high-precision depth estimation from sparse depth information of different sensors without sensor dependency.
Depth-Aware Concealed Crop Detection in Dense Agricultural Scenes
Liqiong Wang (China Three Gorges University), Feng Zheng (Southern University of Science and Technology)
RecognitionObject DetectionSegmentationTransformerImageAgriculture Related
🎯 What it does: This paper proposes the task of concealed crop detection (CCD) and collects a large-scale RGB-D dataset ACOD-12K.
Depth-aware Test-Time Training for Zero-shot Video Object Segmentation
Weihuang Liu (University of Macau), Xiaodong Cun (Tencent AI Lab)
Object DetectionSegmentationTransformerOptical FlowVideo
🎯 What it does: A deep perception-based testing-time training (DATTT) framework is proposed to enhance the performance of zero-shot video object segmentation (ZSVOS).
Describing Differences in Image Sets with Natural Language
Lisa Dunlap (University of California Berkeley), Serena Yeung-Levy (Stanford University)
GenerationRetrievalTransformerLarge Language ModelVision Language ModelImageTextBenchmark
🎯 What it does: An automated method called Set Difference Captioning is proposed to describe the significant differences between two sets of images in natural language.
Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learning
Christopher Liao (Boston University), Brian Kulis (Boston University)
ClassificationDomain AdaptationKnowledge DistillationTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: This paper proposes two descriptor-based 'soup' methods (descriptor soup and word soup), which enhance the few-shot out-of-distribution (OOD) classification accuracy of CLIP in cross-dataset and domain generalization by greedily selecting or constructing descriptors/word chains on a small amount of training data, without the need to invoke LLM during the inference phase.
Desigen: A Pipeline for Controllable Design Template Generation
Haohan Weng (South China University of Technology), C. L. Philip Chen (South China University of Technology)
GenerationOptimizationTransformerDiffusion modelImageTextMultimodality
🎯 What it does: This study presents Desigen, an end-to-end automatic design template generation pipeline that can generate background images based on text descriptions and iteratively optimize layout elements on them.
Design2Cloth: 3D Cloth Generation from 2D Masks
Jiali Zheng (Imperial College London), Stefanos Zafeiriou (Imperial College London)
GenerationData SynthesisGenerative Adversarial NetworkMesh
🎯 What it does: A high-fidelity 3D garment generation model based on 2D clothing masks, Design2Cloth, is proposed, which can reconstruct complete three-dimensional clothing from a single image or scan.
DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection
Lewei Yao (Hong Kong University of Science and Technology), Dan Xu (Hong Kong University of Science and Technology)
Object DetectionGenerationTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: Proposes DetCLIPv3, a detection framework that combines open vocabulary object detection and hierarchical label generation capabilities;
DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
Yibo Wang (Tsinghua University), Kai Zhang (Tsinghua University)
Object DetectionGenerationData SynthesisConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: By combining perception models with diffusion models, we achieve the generation of images from layouts, and the generated synthetic data is used to enhance the training effectiveness of object detection models.
Detector-Free Structure from Motion
Xingyi He (Zhejiang University), Xiaowei Zhou (Zhejiang University)
Pose EstimationOptimizationTransformerSimultaneous Localization and MappingOptical FlowPoint Cloud
🎯 What it does: This paper proposes a detector-independent structure from motion (SfM) framework that utilizes a detector-free matcher to recover camera poses and point clouds without detecting keypoints, and achieves refinement through quantized matching and iterative multi-view relocalization.
Detours for Navigating Instructional Videos
Kumar Ashutosh (University of Texas at Austin), Kristen Grauman (Meta)
RetrievalTransformerLarge Language ModelVideoTextMultimodality
🎯 What it does: Proposes the video detours problem and constructs the VidDetours model, which utilizes source videos and natural language queries to retrieve and locate alternative step segments in long tutorial videos.
DETRs Beat YOLOs on Real-time Object Detection
Yian Zhao (Baidu Inc), Jie Chen (Peking University)
Object DetectionTransformerImage
🎯 What it does: A real-time end-to-end object detector RT-DETR is proposed.
Device-Wise Federated Network Pruning
Shangqian Gao (University of Pittsburgh), Heng Huang (University of Sydney)
CompressionFederated LearningComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a device-level channel pruning method for federated learning scenarios, achieving model compression and acceleration without sharing data.
Dexterous Grasp Transformer
Guo-Hao Xu (Sun Yat-sen University), Wei-Shi Zheng (Sun Yat-sen University)
GenerationPose EstimationRobotic IntelligenceTransformerPoint Cloud
🎯 What it does: A Dexterous Grasp Transformer (DGTR) based on Transformer is proposed, capable of predicting diverse multi-finger grasp poses in a single forward pass.
DGC-GNN: Leveraging Geometry and Color Cues for Visual Descriptor-Free 2D-3D Matching
Shuzhe Wang (Aalto University), Daniel Barath (ETH Zurich)
Object DetectionPose EstimationGraph Neural NetworkPoint Cloud
🎯 What it does: Proposes DGC-GNN, a descriptor-free 2D-3D matching framework based on global-to-local graph neural networks.
DiaLoc: An Iterative Approach to Embodied Dialog Localization
Chao Zhang (Toshiba Europe Ltd), Stephan Liwicki (Toshiba Europe Ltd)
TransformerTextMultimodality
🎯 What it does: This paper proposes an iterative dialogue-driven localization framework called DiaLoc, which provides location information after each round of dialogue and continuously refines it.
DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinement
Hao Wu (University of Science and Technology of China), Xiao Sun (Shanghai Artificial Intelligence Laboratory)
GenerationOptimizationTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: This paper proposes the DIBS framework, which utilizes large-scale unlabeled videos and multimodal large language models (LLMs) to generate rich event-level subtitles. It enhances pseudo event boundaries through optimization algorithms and online boundary refinement iterations, achieving unsupervised pre-training for dense video captioning (DVC).
DIEM: Decomposition-Integration Enhancing Multimodal Insights
Xinyi Jiang (Zhejiang University), Siliang Tang (Zhejiang University)
RecognitionSegmentationRetrievalTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: A multi-modal decomposition-integration method (DIEM) is proposed, which answers the original question by breaking down the problem and the image into sub-problems and sub-images, reasoning step by step, and then merging the results.
Diff-BGM: A Diffusion Model for Video Background Music Generation
Sizhe Li (Peking University), Yang Liu (Peking University)
GenerationData SynthesisDiffusion modelVideoMultimodalityAudio
🎯 What it does: Proposes the Diff-BGM framework, which generates background music that matches the dynamics and semantics of videos based on diffusion models, and constructs a high-quality BGM909 video-music alignment dataset.
Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks
Yuhao Liu (City University of Hong Kong), Rynson W.H. Lau (City University of Hong Kong)
Image TranslationRestorationDiffusion modelContrastive LearningImage
🎯 What it does: The Diff-Plugin framework is proposed, utilizing pre-trained diffusion models to accomplish various low-level visual tasks (such as snow removal, dehazing, rain removal, deblurring, low-light enhancement, face restoration, colorization, etc.), and achieving multi-task editing through natural language instructions.
DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model
Lirui Zhao (Xiamen University), Rongrong Ji (Xiamen University)
GenerationRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringImageTextBenchmark
🎯 What it does: Designed and implemented DiffAgent, an LLM-based agent model that quickly selects suitable text-to-image (T2I) APIs (model + parameters) based on text prompts, thereby reducing the user's trial-and-error and parameter tuning process.
DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection
Yuhao Sun (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
Safty and PrivacyAdversarial AttackDiffusion modelImage
🎯 What it does: DiffAM is a facial privacy protection method that utilizes diffusion models to confuse facial recognition systems by transforming reference facial makeup into adversarial makeup.
DiffAssemble: A Unified Graph-Diffusion Model for 2D and 3D Reassembly
Gianluca Scarpellini (Istituto Italiano di Tecnologia), Alessio Del Bue (Istituto Italiano di Tecnologia)
GenerationData SynthesisPose EstimationGraph Neural NetworkTransformerDiffusion modelImageGraph
🎯 What it does: A unified graph diffusion model called DiffAssemble is proposed to simultaneously address the reassembly tasks of 2D puzzles and 3D fragments.
DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation
Yifei Li (Massachusetts Institute of Technology), Tuur Stuyck (Meta Platforms)
OptimizationMesh
🎯 What it does: DiffAvatar achieves the automatic reconstruction of physically simulative assets from multi-view captured 3D scans of bodies and clothing, including body shape, posture, clothing patterns, and material parameters through differentiable simulation.
DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcasting
Demin Yu (Harbin Institute of Technology), Xunlai Chen (Shenzhen Meteorological Bureau)
GenerationData SynthesisConvolutional Neural NetworkRecurrent Neural NetworkDiffusion modelTime Series
🎯 What it does: A unified forecasting framework called DiffCast is proposed, which utilizes residual diffusion to decompose the evolution of precipitation systems into global deterministic motion trends and local stochastic residuals, compensating for the fuzziness and high-value rainfall missing issues in deterministic predictions through residual diffusion.
DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing
Chong Mou (Peking University), Jian Zhang (Peking University)
GenerationDiffusion modelImageStochastic Differential Equation
🎯 What it does: We propose DiffEditor, a fine-grained image editing framework based on diffusion models, which supports various editing tasks such as object movement, size adjustment, dragging, appearance replacement, and object pasting.
Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation
Dong Lao (University of California, Los Angeles), Stefano Soatto (University of California, Los Angeles)
RestorationOptical FlowImageVideo
🎯 What it does: This paper proposes a template-free initialization method for differential homomorphic template registration, which estimates the deformation relative to the reference frame using optical flow, inverts the optical flow to achieve alignment of each frame to the implicit illumination image, and then uses blind deconvolution to recover a clear image.
Differentiable Display Photometric Stereo
Seokjun Choi (POSTECH), Seung-Hwan Baek (POSTECH)
Data SynthesisDepth Estimation
🎯 What it does: Learn the illumination patterns generated by the display to achieve photometric stereo reconstruction normals through the display and camera;
Differentiable Information Bottleneck for Deterministic Multi-view Clustering
Xiaoqiang Yan (Zhengzhou University), Yangdong Ye (Zhengzhou University)
Contrastive LearningMultimodality
🎯 What it does: A new differentiable information bottleneck (DIB) method is proposed, providing a deterministic and analytical multi-view clustering solution by directly fitting mutual information without the need for variational approximation.
Differentiable Micro-Mesh Construction
Yishun Dou (Shanghai Jiao Tong University), Bingbing Ni (Huawei)
CompressionOptimizationMesh
🎯 What it does: This paper proposes an end-to-end differentiable µ-mesh construction framework that converts standard triangular meshes into µ-meshes that can be efficiently rendered on modern GPUs.
Differentiable Neural Surface Refinement for Modeling Transparent Objects
Weijian Deng (Australian National University), Stephen Gould (Australian National University)
Data SynthesisOptimizationNeural Radiance FieldImage
🎯 What it does: A differentiable framework for refining the surfaces of transparent objects (TNSR) is proposed, which refines the initial neural implicit surface through physical refraction and reflection tracing, requiring only multi-view RGB images for training.
Differentiable Point-based Inverse Rendering
Hoon-Gyu Chung (POSTECH), Seung-Hwan Baek (POSTECH)
OptimizationPoint Cloud
🎯 What it does: This paper proposes a differentiable point-based inverse rendering framework DPIR, which estimates the geometry of static objects and spatially varying BRDF through point sample rendering in an analysis-synthesis loop.
DiffForensics: Leveraging Diffusion Prior to Image Forgery Detection and Localization
Zeqin Yu (Sun Yat-sen University), Bin Li (Shenzhen University)
SegmentationAnomaly DetectionTransformerDiffusion modelImage
🎯 What it does: Designed and trained a two-stage self-supervised diffusion model-assisted image forgery detection and localization framework called DiffForensics.
DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans
Akash Sengupta, Cristian Sminchisescu
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes a probabilistic 3D human reconstruction method called DiffHuman based on a conditional diffusion model, which can generate diverse, high-fidelity 3D human models that are consistent with a single RGB image input.
DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generation
Xiaoliang Ju (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
GenerationData SynthesisDiffusion modelPoint CloudMesh
🎯 What it does: A sparse diffusion pipeline is designed to gradually generate the TSDF of indoor scenes from noise, achieving high-quality 3D indoor geometry from scratch, and can be used to refine multi-view reconstruction results.
DiffLoc: Diffusion Model for Outdoor LiDAR Localization
Wen Li (Xiamen University), Cheng Wang (Xiamen University)
Pose EstimationAutonomous DrivingTransformerDiffusion modelPoint Cloud
🎯 What it does: The DiffLoc framework is proposed, treating LiDAR localization as a conditional generation problem. It learns robust features through a base model and a static object pool, and then uses a diffusion model to achieve iterative denoising regression.
DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement
Jiuming Liu (Shanghai Jiao Tong University), Hesheng Wang (Shanghai Jiao Tong University)
Depth EstimationAutonomous DrivingRecurrent Neural NetworkDiffusion modelPoint Cloud
🎯 What it does: This paper proposes a method for uncertainty-free scene flow estimation based on diffusion models, called DifFlow3D, which uses iterative diffusion to refine sparse scene flow and jointly estimate uncertainty.
DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing
Kaiwen Zhang (Tsinghua University), Xingang Pan (Nanyang Technological University)
GenerationData SynthesisDiffusion modelImageBenchmark
🎯 What it does: Utilizing a pre-trained diffusion model to achieve a smooth transition between two images, generating continuous intermediate frames;
DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction
Weiyi Lv (Shanghai University), Dan Zeng (Shanghai University)
Object TrackingDiffusion modelVideo
🎯 What it does: Proposes DiffMOT, a real-time multi-object tracker based on diffusion models, specifically designed to handle nonlinear motion.
DiffPerformer: Iterative Learning of Consistent Latent Guidance for Diffusion-based Human Video Generation
Chenyang Wang (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
GenerationData SynthesisPose EstimationTransformerDiffusion modelVideo
🎯 What it does: Proposes the DiffPerformer framework, which utilizes pose-guided diffusion models and implicit video representations to achieve high-quality and temporally consistent human video generation.
DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis
Yuming Gu (University of Southern California), Linjie Luo (ByteDance Inc.)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: A new perspective synthesis method for zero-shot portrait based on stable diffusion, DiffPortrait3D, is proposed, which can generate high-fidelity, 3D-consistent multi-angle images from just one pose-unconstrained portrait.
DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction
Junwen Xiong (Northwestern Polytechnical University), Yufei Zha (Nanchang University)
GenerationData SynthesisDiffusion modelVideoMultimodalityAudio
🎯 What it does: This paper proposes DiffSal, a unified audio-video attention prediction framework based on diffusion models, modeling attention prediction as a conditional generation task.
DiffSCI: Zero-Shot Snapshot Compressive Imaging via Iterative Spectral Diffusion Model
Zhenghao Pan (Harbin Institute of Technology), Yongyong Chen (Nanjing University)
RestorationCompressionDiffusion modelImage
🎯 What it does: Proposes the DiffSCI method, which combines a pre-trained RGB diffusion model with Plug-and-Play for multispectral SCI reconstruction, eliminating the need to retrain the model on MSI.
DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation
Junming Chen (International Digital Economy Academy), Qifeng Chen (Hong Kong University of Science and Technology)
GenerationTransformerDiffusion modelMultimodalityAudio
🎯 What it does: This paper proposes DiffSHEG, a speech-driven framework for global 3D expression and gesture synchronization generation based on diffusion models.
DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis
Jiapeng Tang (Technical University of Munich), Matthias Nießner (Technical University of Munich)
GenerationData SynthesisDiffusion modelPoint CloudMesh
🎯 What it does: DiffuScene models indoor scenes as unordered collections of objects and generates diverse and realistic 3D indoor scenes by jointly denoising the positional information, size, orientation, semantic categories, and geometric features of each object.
Diffuse Attend and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion
Junjiao Tian (Georgia Institute of Technology), Mar Gonzalez-Franco (Google)
SegmentationTransformerDiffusion modelImage
🎯 What it does: Using the self-attention layers of a pre-trained stable diffusion model, the DiffSeg method achieves zero-shot unsupervised image segmentation without the need for training, language prompts, or auxiliary images.
DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models
Khawar Islam (FloppyDisk.AI), Karthik Nandakumar (Mohamed bin Zayed University of Artificial Intelligence)
ClassificationData SynthesisDiffusion modelImage
🎯 What it does: A label-preserving data augmentation method based on diffusion models, called DiffuseMix, is proposed.
Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features
Niladri Shekhar Dutt (Ready Player Me), Niloy J. Mitra (Adobe Research)
GenerationKnowledge DistillationRepresentation LearningDiffusion modelPoint CloudMesh
🎯 What it does: This paper proposes DIFF3F, a zero-shot semantic feature descriptor that distills diffusion model features onto untextured 3D shapes through multi-view rendering and a control network.
Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D
Karran Pandey (University of Toronto), Niloy J. Mitra (UCL)
GenerationDepth EstimationDiffusion modelImageBenchmark
🎯 What it does: Proposes the Diffusion Handles method, which utilizes a pre-trained diffusion model and depth estimation to elevate image activations into 3D space, performs 3D transformations, and then projects them back to 2D, supporting 3D-level editing such as translation, rotation, and scaling of any object in the image.
Diffusion Model Alignment Using Direct Preference Optimization
Bram Wallace (Salesforce), Nikhil Naik (Salesforce)
GenerationOptimizationReinforcement Learning from Human FeedbackDiffusion modelImageText
🎯 What it does: This paper proposes the Diffusion-DPO method, applying Direct Preference Optimization (DPO) to text-to-image diffusion models to align the model with human preferences.
Diffusion Models Without Attention
Jing Nathan Yan (Cornell University), Alexander M. Rush (Cornell University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: Designed and evaluated a non-attention diffusion model DIFFUSSM, utilizing a gated bidirectional state space model (SSM) to achieve high-resolution image generation instead of self-attention.
Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance
Yuto Enyo, Ko Nishino
RestorationGenerationDiffusion modelImage
🎯 What it does: This paper proposes a single-image random inverse rendering method that can simultaneously recover the object's reflectance and the spectral environment lighting.
Diffusion Time-step Curriculum for One Image to 3D Generation
Xuanyu Yi (Nanyang Technological University), Hanwang Zhang (Nanyang Technological University)
GenerationData SynthesisDiffusion modelScore-based ModelImage
🎯 What it does: A single-image 3D generation pipeline DTC123 based on a diffusion time step curriculum is proposed, utilizing teacher-student collaboration and progressive representation to enhance geometric and texture quality.
Diffusion-based Blind Text Image Super-Resolution
Yuzhe Zhang (Beijing Institute of Technology), Liheng Bian (Beijing Institute of Technology)
RestorationSuper ResolutionTransformerDiffusion modelImageMultimodality
🎯 What it does: A blind text image super-resolution method called DiffTSR is proposed, which is based on diffusion models. By integrating the Image Diffusion Model (IDM), Text Diffusion Model (TDM), and the Multi-modal Fusion Module (MoM), it achieves high fidelity and realistic style restoration of low-resolution Chinese text images.
Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation
Jihyun Kim (Yonsei University), Kwanghoon Sohn (Yonsei University)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageTextMultimodality
🎯 What it does: This paper proposes a multimodal face generation method that maps the features of diffusion models to the latent space of pre-trained GANs, capable of handling both 2D and 3D face images.
Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation
Hyunwoo Ryu (Yonsei University), Roberto Horowitz (University of California Berkeley)
GenerationRobotic IntelligenceDiffusion modelPoint Cloud
🎯 What it does: We propose Diffusion-EDFs, a bi-equivariant diffusion generative model on SE(3) for visual robot grasping and placing tasks, capable of completing end-to-end training in less than an hour with only 5-10 human demonstrations.
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving
Brian Yang (Carnegie Mellon University), Katerina Fragkiadaki (Carnegie Mellon University)
Autonomous DrivingOptimizationDiffusion modelSequential
🎯 What it does: This paper proposes Diffusion-ES, a gradient-free trajectory optimization method that combines diffusion models with evolutionary search, capable of generating driving trajectories for any black-box reward function during testing.
Diffusion-FOF: Single-View Clothed Human Reconstruction via Diffusion-Based Fourier Occupancy Field
Yuanzhen Li (Wuhan University), Chunxia Xiao (Wuhan University)
RestorationGenerationDiffusion modelImage
🎯 What it does: A Fourier Occupancy Field (FOF) method based on diffusion models is proposed to reconstruct a full 3D human model from a single-view image, and the back view is predicted through style consistency constraints to enhance the prior.
DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars
Tobias Kirschstein (Technical University of Munich), Matthias Nießner (Technical University of Munich)
GenerationData SynthesisPose EstimationDiffusion modelVideoMesh
🎯 What it does: This paper proposes DiffusionAvatars, a neural renderer based on diffusion models that generates high-fidelity 3D facial avatars with controllable expressions and poses using multi-view videos and a fitted NPHM model.
DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors
Biwen Lei (Alibaba Group), Xuansong Xie (Alibaba Group)
GenerationDomain AdaptationDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes DiffusionGAN3D, a two-stage framework that utilizes 3D GANs (such as EG3D) and the prior of diffusion models to achieve text-guided 3D domain adaptation and avatar generation.
DiffusionLight: Light Probes for Free by Painting a Chrome Ball
Pakkapon Phongthawee (VISTEC), Supasorn Suwajanakorn (VISTEC)
RestorationGenerationData SynthesisDiffusion modelImageStochastic Differential Equation
🎯 What it does: Using a pre-trained diffusion model to insert a chrome ball into a single image and estimate HDR environmental lighting based on it.
DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Data
Hanrong Ye (Hong Kong University of Science and Technology), Dan Xu (Hong Kong University of Science and Technology)
Object DetectionSegmentationDepth EstimationTransformerDiffusion modelImage
🎯 What it does: This paper proposes DiffusionMTL—a multi-task dense prediction denoising framework based on diffusion models for partially labeled multi-task learning.
DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion
Tom Van Wouwe (Stanford University), C. Karen Liu (Stanford University)
Pose EstimationTransformerDiffusion modelTime Series
🎯 What it does: A diffusion model-based method called DiffusionPoser is proposed, which can reconstruct full-body motion in real-time from any number and any position of IMU or footpad sensors;
DiffusionRegPose: Enhancing Multi-Person Pose Estimation using a Diffusion-Based End-to-End Regression Approach
Dayi Tan (Tongji University), Lu Xiong (Tongji University)
Pose EstimationTransformerDiffusion modelImage
🎯 What it does: This paper proposes DiffusionRegPose, a model that transforms one-stage end-to-end multi-person human pose estimation into a diffusion sampling process, using a diffusion model to progressively denoise and obtain pose predictions. It also introduces an attention mechanism that facilitates interaction between pose completion and detection, enhancing robustness in crowded/occluded scenarios.
DiffusionTrack: Point Set Diffusion Model for Visual Object Tracking
Fei Xie (Shanghai Jiao Tong University), Chao Ma (Shanghai Jiao Tong University)
Object TrackingTransformerDiffusion modelVideoPoint Cloud
🎯 What it does: A target tracking method based on diffusion models, DiffusionTrack, is proposed, treating the tracking task as a gradual denoising process from noise to the target, and representing the target with a point set.
DiG-IN: Diffusion Guidance for Investigating Networks - Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanations
Maximilian Augustin (Tübingen AI Center University of Tübingen), Matthias Hein (Tübingen AI Center University of Tübingen)
GenerationExplainability and InterpretabilityDiffusion modelImage
🎯 What it does: Using a differential optimization framework with diffusion models (DiG-IN) for interpretability analysis of image classifiers, covering three main tasks: ① Generating images that maximize classifier inconsistency to discover systematic errors; ② Generating visual counterfactual explanations (UVCE) to achieve interpretability for any classifier; ③ Exploring neuron semantics and validating harmful 'pseudo-features' through activation maximization/minimization and adversarial visualization.
Digital Life Project: Autonomous 3D Characters with Social Intelligence
Zhongang Cai (Nanyang Technological University), Ziwei Liu
GenerationRobotic IntelligenceTransformerLarge Language ModelDiffusion modelVideoTextRetrieval-Augmented Generation
🎯 What it does: The Digital Life Project (DLP) framework has been constructed to realize autonomous 3D virtual characters with social intelligence, supporting text-driven generation of body movements and emotional interactions.
DiLiGenRT: A Photometric Stereo Dataset with Quantified Roughness and Translucency
Heng Guo (Beijing University of Posts and Telecommunications), Yasuyuki Matsushita (Osaka University)
Convolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A real photometric stereo dataset DiLiGenRT has been created and released, containing 54 hemispheres with a uniform shape but varying roughness and translucency. High-quality images and ground truth normals were obtained through physical manufacturing and photometric stereo capture; additionally, three complex shapes were extended to create DiLiGenRT-S.
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models
Nastaran Saadati (Iowa State University), Soumik Sarkar (Iowa State University)
Federated LearningComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: A decentralized iterative merging and training framework named DIMAT is proposed for centralized training of deep learning models on multiple agents.
DIOD: Self-Distillation Meets Object Discovery
Sandra Kara (Universite Paris-Saclay), Quoc-Cuong Pham (Universite Paris-Saclay)
Object DetectionKnowledge DistillationOptical FlowVideo
🎯 What it does: This paper proposes a self-distillation-based Slot Attention framework called DIOD for unsupervised video object discovery, which can automatically denoise and complete static objects under motion-guided noisy supervision.
DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning
Sikai Bai (Hong Kong University of Science and Technology), Xiaocheng Lu (Hong Kong University of Science and Technology)
Domain AdaptationFederated LearningTransformerPrompt EngineeringVision Language ModelContrastive LearningImage
🎯 What it does: This paper proposes DiPrompT, a lightweight prompt tuning framework for achieving domain generalization in federated learning, capable of learning global and domain-specific knowledge and integrating them during inference when source domain labels are missing and the number of clients exceeds the number of domains.
DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data
Qihao Liu (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
GenerationData SynthesisPose EstimationDiffusion modelScore-based ModelNeural Radiance FieldPoint CloudMesh
🎯 What it does: We propose DIRECT-3D, a text-to-3D generation model that trains directly on massive unaligned 3D data with noise, capable of quickly generating high-quality NeRF objects and providing 3D geometric priors.
Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion
Yuanxun Lu (Nanjing University), Yao Yao (Nanjing University)
GenerationData SynthesisDiffusion modelTextPoint CloudMesh
🎯 What it does: Achieve rapid generation of 3D triangular meshes and textures from text by fine-tuning a multi-view 2.5D diffusion model.
DisCo: Disentangled Control for Realistic Human Dance Generation
Tan Wang (Microsoft), Lijuan Wang (Advanced Micro Devices)
GenerationDiffusion modelImageVideo
🎯 What it does: The DISCO framework is proposed, achieving generalizable and composable human dance generation for social media dance videos.
Discontinuity-preserving Normal Integration with Auxiliary Edges
Hyomin Kim (POSTECH), Seungyong Lee (POSTECH)
RestorationDepth EstimationPoint CloudMesh
🎯 What it does: A discretization scheme based on auxiliary edges is proposed to explicitly model and recover surface discontinuities during the normal integration process, allowing for more accurate reconstruction of depth maps.
Discover and Mitigate Multiple Biased Subgroups in Image Classifiers
Zeliang Zhang (University of Rochester), Chenliang Xu (University of Rochester)
ClassificationExplainability and InterpretabilityData-Centric LearningLarge Language ModelImageRetrieval-Augmented Generation
🎯 What it does: The DIM framework is proposed, which utilizes PLS to decompose image features under the supervision of training dynamics, automatically discovering and explaining various unknown bias subgroups, and implementing dual bias mitigation for data and models based on these subgroups.
Discovering and Mitigating Visual Biases through Keyword Explanation
Younghyun Kim (Korea Advanced Institute of Science and Technology), Jinwoo Shin (Korea Advanced Institute of Science and Technology)
Anomaly DetectionExplainability and InterpretabilityTransformerVision Language ModelImage
🎯 What it does: A framework B2T is proposed to interpret visual biases as keywords, which can automatically identify potential biases in model errors.
Discovering Syntactic Interaction Clues for Human-Object Interaction Detection
Jinguo Luo (Harbin Institute of Technology), Honghai Liu (Harbin Institute of Technology)
Object DetectionTransformerVision Language ModelImage
🎯 What it does: A HOI detection framework called SICHOI is proposed, which utilizes VLM to learn more fine-grained interaction descriptions and combines multi-view visual extraction with bidirectional cross-attention to enhance detection accuracy.
Discriminability-Driven Channel Selection for Out-of-Distribution Detection
Yue Yuan (Shandong University), Yilong Yin (Shandong University)
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: A new OOD detection method called DDCS is proposed by analyzing and selecting the activations of each channel in the network.
Discriminative Pattern Calibration Mechanism for Source-Free Domain Adaptation
Haifeng Xia (Southeast University), Zhengming Ding (Tulane University)
Domain AdaptationImage
🎯 What it does: A Discriminative Pattern Calibration (DPC) mechanism is proposed, which enhances the target domain feature representation and classification robustness in Source-Free Domain Adaptation (SFDA) by calibrating Grad-CAM attention maps, performing semantic filling, and applying attention-induced mixup.