CVPR 2024 Papers with AI Summaries
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
→ CVPR 2024 papers with code (892)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 2716 CVPR 2024 papers by keyword, author or institution —
free trial on arXivSub.
1-Lipschitz Layers Compared: Memory Speed and Certifiable Robustness
Bernd Prach (Institute of Science and Technology Austria), Christoph H. Lampert (Institute of Science and Technology Austria)
ClassificationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: Compared various network architectures that implement 1-Lipschitz convolutional layers, evaluating their performance in terms of training/inference speed, memory usage, accuracy, and provable robustness.
2S-UDF: A Novel Two-stage UDF Learning Method for Robust Non-watertight Model Reconstruction from Multi-view Images
Junkai Deng (Chinese Academy of Sciences), Ying He (Nanyang Technological University)
RestorationOptimizationImageMesh
🎯 What it does: A two-stage UDF learning method, 2S-UDF, is proposed, which first uses an easily trainable bell-shaped density function to roughly reconstruct the UDF, and then in the second stage directly learns an unbiased and occlusion-aware weight function, improving the reconstruction quality of non-closed models through ray truncation.
360+x: A Panoptic Multi-modal Scene Understanding Dataset
Hao Chen (Machine Intelligence + x Group, University of Birmingham), Jianbo Jiao (Machine Intelligence + x Group, University of Birmingham)
ClassificationRecognitionRetrievalTransformerSupervised Fine-TuningContrastive LearningVideoMultimodalityBenchmarkAudio
🎯 What it does: A 360+x dataset is proposed, covering 360° panoramic views, third-person front views, perspective front views, monocular/binocular, audio, multi-channel, directional binaural delay, GPS, and text descriptions. Five benchmark experiments are conducted on this dataset, including video classification, action localization, cross-modal retrieval, self-supervised learning, and transfer learning.
360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model
Qian Wang (Peking University), Jian Zhang (Peking University)
GenerationData SynthesisDiffusion modelOptical FlowVideoText
🎯 What it does: Designed and implemented 360DVD, a diffusion model capable of generating high-quality, controllable 360° panoramic videos based on text prompts and optional motion conditions.
360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries
Huajian Huang (Hong Kong University of Science and Technology), Sai-Kit Yeung (Hong Kong University of Science and Technology)
Pose EstimationRetrievalConvolutional Neural NetworkTransformerSimultaneous Localization and MappingImageBenchmark
🎯 What it does: A 360Loc dataset was created, and a visual positioning method based on 360° cameras was implemented.
3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions
Weijia Li (Sun Yat-Sen University), Conghui He (Shanghai AI Laboratory)
Object DetectionSegmentationDepth EstimationConvolutional Neural NetworkImage
🎯 What it does: A multi-level supervised monocular remote sensing image building 3D reconstruction network (MLS-BRN) is proposed, which can be trained on samples with different levels of annotations to generate complete 3D models of building footprints, roofs, offsets, and heights from a single frame image.
3D Face Reconstruction with the Geometric Guidance of Facial Part Segmentation
Zidu Wang (Chinese Academy of Sciences), Zhen Lei (Chinese Academy of Sciences)
SegmentationGenerationData SynthesisOptimizationImage
🎯 What it does: This paper proposes a new loss function PRDL, which achieves geometric guidance for 3D face reconstruction by converting facial part segmentation into 2D point sets and aligning them with 3DMM reprojected point sets.
3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow
Felix Taubner (LG Electronics), Jinmiao Huang
Object TrackingPose EstimationTransformerOptical FlowImageVideo
🎯 What it does: This paper proposes an iterative network FlowFace based on 2D dense UV to image flow, achieving precise reconstruction of 3D facial motion from single-camera video.
3D Facial Expressions through Analysis-by-Neural-Synthesis
George Retsinas (Institute of Robotics, Athena Research Center), Petros Maragos (National Technical University of Athens)
RecognitionRestorationGenerationNeural Radiance FieldImage
🎯 What it does: Reconstruct high-quality 3D facial geometry and accurately recover expression details from a single image.
3D Feature Tracking via Event Camera
Siqi Li (Tsinghua University), Yue Gao (Tsinghua University)
Object TrackingDepth EstimationConvolutional Neural NetworkRecurrent Neural NetworkOptical FlowVideo
🎯 What it does: Proposes the first high-speed 3D feature tracking method based on a binocular event camera, capable of predicting the 3D trajectory of target features at 250 FPS;
3D Geometry-Aware Deformable Gaussian Splatting for Dynamic View Synthesis
Zhicheng Lu (Northwestern Polytechnical University), Yuchao Dai (Northwestern Polytechnical University)
GenerationData SynthesisConvolutional Neural NetworkNeural Radiance FieldGaussian SplattingVideoPoint Cloud
🎯 What it does: A 3D structure-aware deformable Gaussian splatting method is proposed for dynamic view synthesis from monocular videos. This method learns static scenes in canonical space and then translates, rotates, and scales each Gaussian over time using a deformed scene to achieve dynamic view rendering.
3D Human Pose Perception from Egocentric Stereo Videos
Hiroyasu Akada (Max Planck Institute for Informatics), Christian Theobalt (Max Planck Institute for Informatics)
Pose EstimationTransformerSimultaneous Localization and MappingVideo
🎯 What it does: A new framework based on Transformer is proposed for 3D human pose estimation using front-view videos captured by self-capturing stereo cameras, and two new datasets are created.
3D LiDAR Mapping in Dynamic Environments using a 4D Implicit Neural Representation
Xingguang Zhong (Center for Robotics, University of Bonn), Jens Behley (Center for Robotics, University of Bonn)
Autonomous DrivingRobotic IntelligenceSimultaneous Localization and MappingPoint Cloud
🎯 What it does: Using a 4D implicit neural network for spatiotemporal TSDF modeling of continuous LiDAR point clouds, a complete static map is obtained and dynamic objects are automatically removed.
3D Multi-frame Fusion for Video Stabilization
Zhan Peng (Huazhong University of Science and Technology), Zhiguo Cao (Huazhong University of Science and Technology)
RestorationDepth EstimationOptical FlowVideo
🎯 What it does: A novel video stabilization framework named RStab is proposed, which achieves uncropped and structure-preserving full-frame stabilized images using 3D multi-frame fusion and volume rendering.
3D Neural Edge Reconstruction
Lei Li (ETH Zurich), Marc Pollefeys (Microsoft)
SegmentationGenerationNeural Radiance FieldPoint CloudMesh
🎯 What it does: The EMAP method is proposed, which utilizes multi-view edge graph learning to derive an unsigned distance function for 3D edges and extracts parameterized lines and curves from it.
3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation
Dale Decatur (University of Chicago), Rana Hanocka (University of Chicago)
GenerationData SynthesisDiffusion modelScore-based ModelMesh
🎯 What it does: Automatically texture local areas of 3D meshes using text descriptions, generating corresponding localization maps and texture maps, which can be seamlessly integrated into traditional rendering pipelines.
3D-Aware Face Editing via Warping-Guided Latent Direction Learning
Yuhao Cheng (Shanghai Jiao Tong University), Yichao Yan (Shanghai Jiao Tong University)
Image TranslationGenerationGenerative Adversarial NetworkImage
🎯 What it does: A 3D head and facial editing method based on tri-plane distortion and latent direction learning is proposed, allowing users to intuitively edit facial shapes, expressions, and angles by dragging points while maintaining 3D consistency.
3D-LFM: Lifting Foundation Model
Mosam Dabhi (Carnegie Mellon University), Simon Lucey (University of Adelaide)
Pose EstimationTransformerPoint Cloud
🎯 What it does: A unified 3D-Lifting Foundation Model (3D-LFM) is proposed, achieving 3D structure recovery from single-frame 2D keypoints without category correspondence, and demonstrating consistent performance across 30+ object categories.
3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation
Songchun Zhang (Zhejiang University), Changqing Zou (Zhejiang University)
GenerationData SynthesisDepth EstimationDiffusion modelNeural Radiance FieldImageText
🎯 What it does: A complete workflow is proposed to generate a full 3D scene from text prompts and arbitrary 6-degree-of-freedom camera trajectories, utilizing Tri-Planar NeRF for unified representation and achieving panoramic view consistency through incremental optimization.
3DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surfaces
Linyi Jin (University of Michigan), David F. Fouhey (New York University)
SegmentationDepth EstimationTransformerVision Language ModelImage
🎯 What it does: This paper presents 3DFIRES, a system that achieves scene-level 3D reconstruction using a small number of known pose images while simultaneously recovering occluded surfaces.
3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting
Zhiyin Qian (ETH Zurich), Siyu Tang (ETH Zurich)
GenerationPose EstimationComputational EfficiencyGaussian SplattingVideo
🎯 What it does: Using 3D Gaussian Splatting to quickly construct animatable human avatars from monocular videos, achieving real-time rendering with only 30 minutes of training.
3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos
Jiakai Sun (Zhejiang University), Wei Xing (Zhejiang University)
GenerationComputational EfficiencyGaussian SplattingVideo
🎯 What it does: A free viewpoint video (FVV) system is proposed that enables instant training and real-time rendering on multi-view video streams, utilizing a 3D Gaussian model for realistic reconstruction of dynamic scenes.
3DiffTection: 3D Object Detection with Geometry-Aware Diffusion Features
Chenfeng Xu (NVIDIA), Or Litany (Technion)
Object DetectionDiffusion modelImage
🎯 What it does: Proposes the 3DiffTection framework, which utilizes a pre-trained 2D diffusion model to achieve single-image 3D object detection through geometric ControlNet and semantic ControlNet.
3DInAction: Understanding Human Actions in 3D Point Clouds
Yizhak Ben-Shabat (Technion, Israel Institute of Technology), Stephen Gould (Australian National University)
RecognitionConvolutional Neural NetworkPoint Cloud
🎯 What it does: A motion recognition pipeline based on 3D point cloud sequences, called 3DinAction, is proposed, which focuses on constructing spatiotemporal features using local point cloud sets (t-patches) that evolve over time.
3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling
Chaokang Jiang (PhiGent Robotics), Dalong Du (PhiGent Robotics)
Autonomous DrivingOptimizationOptical FlowPoint Cloud
🎯 What it does: An automatic pseudo-label generation framework is proposed, which synthesizes a large number of scene flow pseudo-labels on real LiDAR point clouds using the rigid motion parameters of 3D anchor boxes for unsupervised 3D scene flow training.
3DToonify: Creating Your High-Fidelity 3D Stylized Avatar Easily from 2D Portrait Images
Yifang Men (Alibaba Group), Zhouhui Lian (Peking University)
GenerationData SynthesisNeural Radiance FieldGenerative Adversarial NetworkImageVideo
🎯 What it does: The research utilizes a single style example image to generate high-fidelity 3D stylized avatars from 2D portrait videos captured by a monocular camera, maintaining consistent rendering effects in geometry and texture from any viewpoint.
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Guanjun Wu (Huazhong University of Science and Technology), Xinggang Wang (Huazhong University of Science and Technology)
GenerationComputational EfficiencyGaussian SplattingVideo
🎯 What it does: Proposes a 4D Gaussian Splatting framework for real-time rendering of dynamic scenes.
4D-DRESS: A 4D Dataset of Real-World Human Clothing With Semantic Annotations
Wenbo Wang (ETH Zurich), Otmar Hilliges (Max Planck Institute for Intelligent Systems)
SegmentationGenerationData SynthesisGraph Neural NetworkOptical FlowVideoMeshBenchmark
🎯 What it does: Constructed and released 4D-DRESS, a real-world 4D human clothing dataset that includes 64 sets of clothing, 520 action sequences (78k frames), high-quality texture scans, vertex-level semantic labels, clothing meshes, and SMPL(-X) body meshes.
4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling
Sherwin Bahmani (University of Toronto), David B. Lindell (University of Toronto)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelImageVideoText
🎯 What it does: This paper proposes a method for generating dynamic 3D scenes from text using Hybrid Score Distillation Sampling, employing a hash-grid 4D neural rendering representation. It combines 3D-aware text-to-image, text-to-image (VSD), and text-to-video diffusion models to generate high-quality appearances, 3D structures, and motion in a single pass.
4K4D: Real-Time 4D View Synthesis at 4K Resolution
Zhen Xu (Zhejiang University), Xiaowei Zhou (Zhejiang University)
Data SynthesisComputational EfficiencyNeural Radiance FieldPoint CloudMesh
🎯 What it does: This paper proposes 4K4D, a real-time dynamic view synthesis representation based on 4D point clouds and 4D feature grids, capable of achieving ultra-fast rendering at 4K resolution.
6D-Diff: A Keypoint Diffusion Framework for 6D Object Pose Estimation
Li Xu (Singapore University of Technology and Design), Jun Liu (Singapore University of Technology and Design)
Pose EstimationTransformerDiffusion modelImage
🎯 What it does: This paper proposes a 6D object pose estimation framework based on diffusion models, called 6D-Diff, which utilizes keypoint heatmaps as priors and views keypoint detection as a reverse diffusion process.
A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectives
Simone Alberto Peirone (Politecnico di Torino), Giuseppe Averta (Politecnico di Torino)
ClassificationRecognitionGraph Neural NetworkVideo
🎯 What it does: EgoPack is proposed, a unified temporal graph neural network architecture that extracts task perspective prototypes after multi-task pre-training, forming a 'skill backpack' for learning new tasks.
A Bayesian Approach to OOD Robustness in Image Classification
Prakhar Kaushik (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
ClassificationDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an Unsupervised Generative Transfer (UGT) framework that separates unsupervised vMF kernel dictionary learning from supervised spatial structure learning to achieve image classification under real-world occlusions and OOD effects.
A Call to Reflect on Evaluation Practices for Age Estimation: Comparative Analysis of the State-of-the-Art and a Unified Benchmark
Jakub Paplhám (Czech Technical University in Prague), Vojt?ch Franc
ClassificationRecognitionConvolutional Neural NetworkTransformerVision Language ModelImageBenchmark
🎯 What it does: Systematically evaluated and compared facial age estimation methods, proposed a unified evaluation protocol, and published data partitioning and code.
A Category Agnostic Model for Visual Rearrangment
Yuyi Liu (Chinese Academy of Sciences), Shuqiang Jiang (Chinese Academy of Sciences)
Robotic IntelligencePoint Cloud
🎯 What it does: A category-free visual rearrangement model CAVR is proposed, which uses point cloud representation for scene changes and achieves indoor object rearrangement without category labels through visual and geometric feature matching.
A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models
Julio Silva-Rodríguez (École de technologie supérieure), Jose Dolz (École de technologie supérieure)
Domain AdaptationOptimizationTransformerVision Language ModelContrastive LearningImage
🎯 What it does: The CLAP (Class-Adaptive Linear Probe) method is proposed, which uses zero-shot prototypes as priors to achieve efficient few-shot transfer of large-scale vision-language models without a validation set.
A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsampling
Wentao Qu (Nanjing University of Science and Technology), Liang Xiao (Nanjing University of Science and Technology)
RestorationGenerationData SynthesisDiffusion modelPoint Cloud
🎯 What it does: A point cloud upsampling method based on a conditional denoising diffusion probabilistic model (PUDM) is proposed, which directly generates high-quality dense point clouds from sparse point clouds as conditions.
A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation
Qucheng Peng (University of Central Florida), Chen Chen (University of Central Florida)
Pose EstimationDomain AdaptationMeta LearningGenerative Adversarial NetworkPoint Cloud
🎯 What it does: A dual enhancer framework is proposed to improve the domain generalization ability of 3D human pose estimation through weak and strong enhancers as well as meta-optimization.
A Dynamic Kernel Prior Model for Unsupervised Blind Image Super-Resolution
Zhixiong Yang (National University of Defense Technology), Yongxiang Liu (National University of Defense Technology)
RestorationSuper ResolutionDiffusion modelImage
🎯 What it does: This paper proposes an unsupervised, pre-training-free Dynamic Kernel Prior (DKP) model for blind image super-resolution, and combines it with Deep Image Prior (DIP) and diffusion models to obtain DIP-DKP and Diff-DKP.
A General and Efficient Training for Transformer via Token Expansion
Wenxuan Huang (East China Normal University), Shaohui Lin (East China Normal University)
Computational EfficiencyTransformerImage
🎯 What it does: Proposes the Token Expansion (ToE) mechanism, which dynamically increases the number of tokens during Vision Transformer training through an initialization-expansion-merging pipeline, achieving both training acceleration and maintaining the consistency of the original model.
A Generative Approach for Wikipedia-Scale Visual Entity Recognition
Mathilde Caron (Google Research), Cordelia Schmid (Google Research)
RecognitionRetrievalCompressionComputational EfficiencyTransformerGenerative Adversarial NetworkContrastive LearningImageText
🎯 What it does: A generative entity recognition (GER) framework is proposed, which generates corresponding compressed semantic codes from input images to retrieve millions of Wikipedia entities.
A Noisy Elephant in the Room: Is Your Out-of-Distribution Detector Robust to Label Noise?
Galadrielle Humblot-Renaux (Aalborg University), Thomas B. Moeslund (Aalborg University)
ClassificationAnomaly DetectionConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: This study systematically evaluates the robustness of 20 mainstream post-hoc OOD detection methods in the presence of label noise during classifier training, exploring how label noise affects the distinguishability between ID samples and OOD samples.
A Pedestrian is Worth One Prompt: Towards Language Guidance Person Re-Identification
Zexian Yang (Institute of Information Engineering, Chinese Academy of Sciences), Weiping Wang (Institute of Information Engineering, Chinese Academy of Sciences)
RecognitionRetrievalTransformerPrompt EngineeringContrastive LearningImageMultimodality
🎯 What it does: This paper proposes a Prompt-Driven Semantic Guidance (PromptSG) framework based on CLIP for pedestrian Re-ID;
A Physics-informed Low-rank Deep Neural Network for Blind and Universal Lens Aberration Correction
Jin Gong (Tsinghua University), Qionghai Dai (Tsinghua University)
RestorationImagePhysics Related
🎯 What it does: A low-rank deep network based on the physical characteristics of camera lenses is proposed to achieve blind and general lens distortion correction.
A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions
Jack Urbanek (Meta), Adriana Romero-Soriano (Fair)
SegmentationRetrievalTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A dense annotated image dataset DCI has been constructed and made public, providing approximately 1000-word multi-level textual descriptions for each image, and based on this, a sub-image-caption matching task has been proposed to evaluate the fine-grained understanding ability of VLM.
A Recipe for Scaling up Text-to-Video Generation with Text-free Videos
Xiang Wang (Huazhong University of Science and Technology), Nong Sang (Huazhong University of Science and Technology)
GenerationData SynthesisDiffusion modelVideoText
🎯 What it does: A two-branch video generation framework named TF-T2V has been developed, capable of training using only text-free videos to achieve text-guided video synthesis.
A Semi-supervised Nighttime Dehazing Baseline with Spatial-Frequency Aware and Realistic Brightness Constraint
Xiaofeng Cong (Southeast University), Hao Shen (Hefei University of Technology)
RestorationTransformerImage
🎯 What it does: A semi-supervised nighttime image dehazing baseline SFSNiD is proposed, utilizing a spatial-frequency information interaction module and local window brightness constraints to address issues of haze, halos, and noise caused by multiple light sources and low illumination.
A Simple and Effective Point-based Network for Event Camera 6-DOFs Pose Relocalization
Hongwei Ren (Hong Kong University of Science and Technology), Bojun Cheng (Hong Kong University of Science and Technology)
Pose EstimationRecurrent Neural NetworkPoint Cloud
🎯 What it does: PEPNet is proposed—a 6-DOF camera pose relocalization network that directly processes raw event point clouds.
A Simple Baseline for Efficient Hand Mesh Reconstruction
Zhishan Zhou (Jiiov Technology), Jiajun Liang (Jiiov Technology)
Pose EstimationComputational EfficiencyTransformerMesh
🎯 What it does: A simple structure based on a token generator and a mesh regressor is proposed to achieve single-image hand mesh reconstruction.
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames
Pinelopi Papalampidi (Google DeepMind), Aida Nematzdeh (Google DeepMind)
RetrievalComputational EfficiencyRepresentation LearningTransformerContrastive LearningVideoText
🎯 What it does: This paper studies a simple training strategy for contrastive pre-trained video encoders that can extend the encoder to videos longer than 4 minutes without changing the model structure.
A Simple Recipe for Language-guided Domain Generalized Segmentation
Mohammad Fahes (Inria), Raoul de Charette (Inria)
SegmentationDomain AdaptationAutonomous DrivingPrompt EngineeringImage
🎯 What it does: This paper proposes a domain generalization semantic segmentation framework called FAMix, based on CLIP pre-training, which employs minimal fine-tuning, language-based local style enhancement, and local style mixing to significantly improve generalization capabilities to unseen domains.
A Stealthy Wrongdoer: Feature-Oriented Reconstruction Attack against Split Learning
Xiaoyang Xu (Wuhan University), Yaxin Liu (Wuhan University)
RestorationFederated LearningAdversarial AttackImage
🎯 What it does: A semi-honest federated learning data reconstruction attack (FORA) is proposed, which quietly recovers the private training data of clients by constructing substitute clients.
A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition
Yusheng Dai (University of Science and Technology of China), Chin-Hui Lee (Georgia Institute of Technology)
RecognitionKnowledge DistillationVideoMultimodalityAudio
🎯 What it does: This paper studies the 'modal bias' problem caused by video data dropout training in audio-visual speech recognition (AVSR) and proposes corresponding mitigation methods.
A Subspace-Constrained Tyler's Estimator and its Applications to Structure from Motion
Feng Yu (University of Minnesota), Gilad Lerman (University of Minnesota)
Anomaly DetectionOptimizationImage
🎯 What it does: A subspace-constrained Tyler estimator (STE) is proposed for efficiently recovering low-dimensional subspaces in the presence of a large number of outliers.
A Theory of Joint Light and Heat Transport for Lambertian Scenes
Mani Ramanagopal (Carnegie Mellon University), Srinivasa G. Narasimhan (Carnegie Mellon University)
ImagePhysics Related
🎯 What it does: By combining thermal imaging and visible light, an analytical solution to the heat transfer equation is derived to estimate the absorbed light intensity, thereby achieving illumination-reflection separation from a single viewpoint.
A Unified and Interpretable Emotion Representation and Expression Generation
Reni Paskaleva (INSAIT), Danda Paudel (INSAIT)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageText
🎯 What it does: A unified three-dimensional emotional representation (C2A2) is proposed, and a text + numerical conditional diffusion model is constructed based on this representation to generate fine-grained expressions.
A Unified Approach for Text- and Image-guided 4D Scene Generation
Yufeng Zheng (NVIDIA), Shalini De Mello (NVIDIA)
GenerationData SynthesisDiffusion modelNeural Radiance FieldImageVideoTextMultimodality
🎯 What it does: This paper presents Dream-in-4D, a unified two-stage framework that first generates high-quality static NeRF using 3D and 2D diffusion models, and then learns a separable deformation field using a video diffusion model, enabling text or image-driven 4D scene generation.
A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signals
Jiangnan Tang (ShanghaiTech University), Ye Shi (ShanghaiTech University)
Pose EstimationDiffusion modelAuto EncoderPoint Cloud
🎯 What it does: The researchers propose to predict complete human motion in a 3D scene using only the sparse tracking signals generated by a head-mounted display and dual hand controllers.
A Unified Framework for Human-centric Point Cloud Video Understanding
Yiteng Xu (ShanghaiTech University), Yuexin Ma (ShanghaiTech University)
RecognitionSegmentationPose EstimationTransformerAuto EncoderOptical FlowVideoPoint Cloud
🎯 What it does: A unified framework called UniPVU-Human is proposed, which utilizes human prior knowledge (body part segmentation and motion flow) to achieve understanding of human point cloud videos through self-supervised body part occlusion prediction and hierarchical fine-tuning.
A Unified Framework for Microscopy Defocus Deblur with Multi-Pyramid Transformer and Contrastive Learning
Yuelin Zhang (Chinese University of Hong Kong), Shing Shin Cheng (Chinese University of Hong Kong)
RestorationTransformerContrastive LearningImage
🎯 What it does: A unified framework MPT–EFCR is proposed for microscope deblurring tasks.
A Versatile Framework for Continual Test-Time Domain Adaptation: Balancing Discriminability and Generalizability
Xu Yang (Xidian University), Cheng Deng (Xidian University)
Domain AdaptationConvolutional Neural NetworkImage
🎯 What it does: A general framework is designed to implement continuous testing domain adaptation, utilizing adaptive thresholds, source model prior calibration, and diversity weighting to generate high-quality pseudo-labels. It performs soft-weight alignment of network parameters without accessing source data to enhance current domain performance while maintaining generalization to future domains.
A Video is Worth 256 Bases: Spatial-Temporal Expectation-Maximization Inversion for Zero-Shot Video Editing
Maomao Li (University of Hong Kong), Dong Xu (University of Hong Kong)
RestorationGenerationDiffusion modelVideo
🎯 What it does: A low-rank video inversion method based on the EM algorithm (STEM) is proposed for zero-shot video editing.
A Vision Check-up for Language Models
Pratyusha Sharma (Massachusetts Institute of Technology), Antonio Torralba (Massachusetts Institute of Technology)
GenerationData SynthesisRetrievalLarge Language ModelContrastive LearningImageText
🎯 What it does: Using large language models (such as GPT-3.5/4) to generate drawing code, recognize images generated from code, and utilize these images to train visual models, verifying the learning ability of LLMs regarding visual concepts.
A-Teacher: Asymmetric Network for 3D Semi-Supervised Object Detection
Hanshi Wang (Chinese Academy of Sciences), Weiming Hu (Chinese Academy of Sciences)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: An online asymmetric semi-supervised framework A-Teacher is proposed, which embeds lightweight attention correction modules (PBA, DBA, STA) into a single-frame detector, utilizing multi-frame LiDAR recordings to enhance pseudo-label quality and performing online updates through teacher-student EMA.
A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network
Ruichen Ma (University of Electronic Science and Technology of China), Shaogang Hu (University of Electronic Science and Technology of China)
ClassificationComputational EfficiencyKnowledge DistillationImage
🎯 What it does: Designed and implemented a completely multiplication-free binary neural network (A&B BNN), achieving hardware-friendly inference.
A2XP: Towards Private Domain Generalization
Geunhyeok Yu (Kyung Hee University), Hyoseok Hwang (Kyung Hee University)
Domain AdaptationSafty and PrivacyTransformerPrompt EngineeringImage
🎯 What it does: This paper proposes the A2XP method, which utilizes expert prompts and attention mechanisms to achieve domain generalization while maintaining the structural and parameter privacy of the target network.
AAMDM: Accelerated Auto-regressive Motion Diffusion Model
Tianyu Li (Georgia Institute of Technology), Sehoon Ha (Georgia Institute of Technology)
GenerationData SynthesisPose EstimationComputational EfficiencyDiffusion modelAuto EncoderGenerative Adversarial NetworkVideo
🎯 What it does: A new interactive motion synthesis framework, AAMDM, is proposed, which can generate diverse and high-quality character movements at real-time rates.
Abductive Ego-View Accident Video Understanding for Safe Driving Perception
Jianwu Fang (Xi'an Jiaotong University), Tat-Seng Chua (National University of Singapore)
Object DetectionGenerationAutonomous DrivingConvolutional Neural NetworkDiffusion modelContrastive LearningVideoMultimodality
🎯 What it does: A large-scale multimodal accident video dataset, MM-AU, is proposed, and based on this dataset, the AdVersa-SD framework is designed for self-supervised reasoning in accident video understanding and generation.
Absolute Pose from One or Two Scaled and Oriented Features
Jonathan Ventura (California Polytechnic State University), Dániel Baráth (ETH Zurich)
Pose EstimationSimultaneous Localization and MappingImage
🎯 What it does: This study investigates the minimal solution for absolute pose estimation using scale and orientation information from keypoints.
Accelerating Diffusion Sampling with Optimized Time Steps
Shuchen Xue (Academy of Mathematics and Systems Science Chinese Academy of Sciences), Zhenguo Li (Huawei Noah's Ark Lab)
GenerationOptimizationComputational EfficiencyDiffusion modelImageOrdinary Differential Equation
🎯 What it does: To address the low sampling efficiency of diffusion models, the authors propose a framework that accelerates sampling by optimizing the sampling time step.
Accelerating Neural Field Training via Soft Mining
Shakiba Kheradmand (University of British Columbia), Kwang Moo Yi (University of British Columbia)
Computational EfficiencyNeural Radiance FieldImageStochastic Differential Equation
🎯 What it does: This paper proposes a method to accelerate the training of Neural Fields through soft mining, primarily improving the sampling strategy of training batches.
Accept the Modality Gap: An Exploration in the Hyperbolic Space
Sameera Ramasinghe (Amazon), Ajanthan Thalaiyasingam (Amazon)
ClassificationRetrievalTransformerContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes an angle-based contrastive loss that accommodates the modal gap between images and text, thereby maintaining the hierarchical structure in the dual-modal hyperbolic space.
Accurate Spatial Gene Expression Prediction by Integrating Multi-Resolution Features
Youngmin Chung (Sungkyunkwan University), Joo Sang Lee (Sungkyunkwan University)
SegmentationOptimizationKnowledge DistillationConvolutional Neural NetworkTransformerBiomedical Data
🎯 What it does: By extracting multi-resolution features of target spots, neighborhood views, and global views from whole slide images (WSI), the deep learning framework TRIPLEX is used to predict spatial gene expression levels.
Accurate Training Data for Occupancy Map Prediction in Automated Driving Using Evidence Theory
Jonas Kälble (Bosch Center for Artificial Intelligence), Eddy Ilg (Saarland University)
Autonomous DrivingPoint Cloud
🎯 What it does: A method for generating LiDAR occupancy grids based on evidence theory is proposed, which can explicitly handle occlusion, unobserved areas, and measurement errors, generating more accurate occupancy maps and uncertainty estimates for training image-driven occupancy prediction models.
ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models
Fei Kong (University of Electronic Science and Technology of China), Kaidi Xu (Drexel University)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: Adversarial Consistency Training (ACT) is proposed, which incorporates a discriminator into the consistency model training to directly minimize the Jensen-Shannon distance between the generated distribution and the target distribution, thereby achieving efficient generation of a one-step diffusion model.
Action Detection via an Image Diffusion Process
Lin Geng Foo (Singapore University of Technology and Design), Jun Liu (Singapore University of Technology and Design)
RecognitionObject DetectionTransformerDiffusion modelVideo
🎯 What it does: Reformulate the action detection task as generating three action detection images (action category map, start-end point map) and decoding them.
Action Scene Graphs for Long-Form Understanding of Egocentric Videos
Ivan Rodin (University of Catania), Giovanni Maria Farinella (University of Catania)
Object DetectionRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelVideoGraph
🎯 What it does: A new long-term first-person video understanding representation called Egocentric Action Scene Graphs (EASG) is proposed, and this graph representation has been manually annotated on the Ego4D dataset.
Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenes
Chi-Hsi Kung (National Yang Ming Chiao Tung University), Yi-Ting Chen (Google)
RecognitionAutonomous DrivingConvolutional Neural NetworkTransformerVideo
🎯 What it does: Developed the Action-slot framework, which implements a slot attention-based visual action-centric representation for multi-label atomic action recognition in traffic scenes.
Active Domain Adaptation with False Negative Prediction for Object Detection
Yuzuru Nakamura (Panasonic Holdings Corporation), Takayoshi Yamashita (Chubu University)
Object DetectionDomain AdaptationImage
🎯 What it does: This paper proposes an active domain adaptation framework for object detection, which predicts undetected objects through a False Negative Prediction Module and combines uncertainty and diversity for sample selection, achieving performance close to fully annotated results with only a small amount of labeled data.
Active Generalized Category Discovery
Shijie Ma (Institute of Automation, Chinese Academy of Sciences), Cheng-Lin Liu (Institute of Automation, Chinese Academy of Sciences)
ClassificationContrastive LearningImage
🎯 What it does: This paper proposes the task of Active Generalization Category Discovery (AGCD), which utilizes a small number of actively queried new category samples to enhance clustering and classification performance for old and new categories in unlabeled data.
Active Object Detection with Knowledge Aggregation and Distillation from Large Models
Dejie Yang (Peking University), Yang Liu (Peking University)
Object DetectionKnowledge DistillationTransformerDiffusion modelVideo
🎯 What it does: A proactive target detection framework KAD is proposed, utilizing ternary prior knowledge (semantic interaction, fine-grained vision, spatial prior);
Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations
Lei Fan (Northwestern University), Ying Wu (Northwestern University)
RecognitionRobotic IntelligenceTransformerReinforcement LearningVision Language ModelPoint Cloud
🎯 What it does: This paper proposes an active open vocabulary recognition framework that allows agents to actively move in the environment to obtain multi-view observations and utilize self-attention to fuse features, thereby recognizing any category (unseen categories);
Active Prompt Learning in Vision Language Models
Jihwan Bang (Korea Advanced Institute of Science and Technology), Jae-Gil Lee (Michigan State University)
ClassificationData-Centric LearningTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: Active prompt learning for pre-trained vision-language models is conducted to reduce labeling costs and enhance adaptability to new tasks.
ActiveDC: Distribution Calibration for Active Finetuning
Wenshuai Xu (Beihang University), Yunhong Wang (Beihang University)
ClassificationTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes an active fine-tuning method within a pre-training-fine-tuning framework, which first selects a limited number of labeled samples through distribution matching, and then calibrates the class distribution of the selected samples using clustering information from unlabeled features to generate high-quality pseudo-labeled samples to be added to the training set, addressing the distribution bias caused by few samples.
Activity-Biometrics: Person Identification from Daily Activities
Shehreen Azad (University of Central Florida), Yogesh Singh Rawat (University of Central Florida)
RecognitionKnowledge DistillationConvolutional Neural NetworkTransformerContrastive LearningVideo
🎯 What it does: The research focuses on the task of person identification in daily activity videos and proposes a recognition framework based on RGB videos.
ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association
Shuxiao Ding (Mercedes-Benz AG), Juergen Gall (University of Bonn)
Object TrackingAutonomous DrivingTransformerImageVideoMultimodalityPoint Cloud
🎯 What it does: This paper presents ADA-Track, an end-to-end multi-camera 3D multi-object tracking framework based on queries.
AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution
Cheeun Hong (Seoul National University), Kyoung Mu Lee (Seoul National University)
RestorationSuper ResolutionImage
🎯 What it does: For the task of image super-resolution, this paper proposes a quantization framework called AdaBM that can adaptively configure bit width in real-time during inference, completing bit mapping configuration in seconds and avoiding the massive training required by traditional QAT.
Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation
Jonas Herzog (Zhejiang University)
SegmentationDomain AdaptationConvolutional Neural NetworkContrastive LearningImageBenchmark
🎯 What it does: This paper proposes a method for cross-domain few-shot segmentation (CD-FSS) that does not require a pre-trained segmentation network. Instead, it directly achieves segmentation by inserting 1×1 adapters into each bottleneck layer of an ImageNet pre-trained ResNet-50 backbone during testing, and utilizes view consistency contrastive learning and class prototype contrastive loss for task adaptation of the adapters.
Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration
Shihao Zhou (Nankai University), Jufeng Yang (Nankai University)
RestorationTransformerImage
🎯 What it does: This paper proposes an adaptive sparse Transformer model for image restoration tasks.
Adapters Strike Back
Jan-Martin O. Steitz (Technical University of Darmstadt), Stefan Roth (Technical University of Darmstadt)
ClassificationOptimizationComputational EfficiencyTransformerImage
🎯 What it does: Systematically evaluated and improved adapters on visual Transformers, proposing a more efficient Adapter+ for parameterized transfer learning.
Adapting Short-Term Transformers for Action Detection in Untrimmed Videos
Min Yang (Nanjing University), Limin Wang (Nanjing University)
Object DetectionTransformerAuto EncoderVideo
🎯 What it does: Proposes the ViT-TAD framework, using a pre-trained pure ViT as a unified long video Transformer for end-to-end temporal action detection.
Adapting to Length Shift: FlexiLength Network for Trajectory Prediction
Yi Xu (Northeastern University), Yun Fu (Northeastern University)
Autonomous DrivingKnowledge DistillationTransformerTime SeriesSequential
🎯 What it does: This paper proposes the FlexiLength Network (FLN) to address the significant performance decline of trajectory prediction models when the observation length varies.
Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images
Chaoqin Huang (Shanghai Jiao Tong University), Yanfeng Wang (Shanghai Jiao Tong University)
Anomaly DetectionTransformerVision Language ModelContrastive LearningImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Adapting the pre-trained CLIP visual-language model, a multi-layer visual feature adaptation and comparison framework is proposed to achieve zero/few-shot anomaly detection in medical images.
Adaptive Bidirectional Displacement for Semi-Supervised Medical Image Segmentation
Hanyang Chi (China University of Petroleum), Weifeng Liu (China University of Petroleum)
SegmentationContrastive LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes an Adaptive Bidirectional Displacement (ABD) framework to enhance the consistency learning effect in semi-supervised medical image segmentation.
Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving
Junda Cheng (Huazhong University of Science and Technology), Xin Yang (Huazhong University of Science and Technology)
Depth EstimationAutonomous DrivingConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an adaptive fusion network for monocular and multi-view depth estimation, named AFNet, aimed at enhancing the robustness and accuracy of depth estimation in autonomous driving scenarios.
Adaptive Hyper-graph Aggregation for Modality-Agnostic Federated Learning
Fan Qi (Tianjin University of Technology), Shuai Li (Tianjin University of Technology)
Federated LearningGraph Neural NetworkContrastive LearningVideoMultimodality
🎯 What it does: An adaptive hypergraph aggregation framework is proposed to address the statistical heterogeneity and modality incompatibility issues of multimodal data in federated learning.
Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matching
Peng Xu (Zhejiang University), Tianyu Pu (Zhejiang University)
Depth EstimationDomain AdaptationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Proposes an adaptive multimodal cross-entropy loss and a primary modality resolution estimator to improve the distribution supervision and result estimation of stereo matching networks.
Adaptive Random Feature Regularization on Fine-tuning Deep Neural Networks
Shin'ya Yamaguchi (NTT), Daiki Chijiwa (NTT)
ClassificationConvolutional Neural NetworkTransformerSupervised Fine-TuningContrastive LearningImage
🎯 What it does: AdaRand is proposed, a method for random feature regularization that adaptively uses class-conditional Gaussian priors during the fine-tuning process, addressing the issues of lack of source information and high computational costs.
Adaptive Slot Attention: Object Discovery with Dynamic Slot Number
Ke Fan (Fudan University), Zheng Zhang (Amazon Web Services)
Object DetectionSegmentationTransformerAuto EncoderImageVideo
🎯 What it does: The AdaSlot framework is proposed, which adaptively determines the number of slots during object discovery and segmentation, avoiding the limitations of traditional Slot Attention that requires a preset number of slots.
Adaptive Softassign via Hadamard-Equipped Sinkhorn
Binrui Shen (Xi'an Jiaotong-Liverpool University), Shengxin Zhu (Beijing Normal University)
OptimizationProtein Structure PredictionGraph Neural NetworkGraph
🎯 What it does: This paper studies the Softassign algorithm in graph matching and proposes Adaptive Softassign and Hadamard-Equipped Sinkhorn formulas to automatically adjust parameters and improve stability and efficiency.