CVPR 2024 Papers — Page 10
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization
Jiahui Zhang (Nanyang Technological University), Eric Xing (Carnegie Mellon University)
RestorationGenerationData SynthesisNeural Radiance FieldGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes FreGS, which combines 3D Gaussian splatting with frequency domain regularization to achieve real-time novel view synthesis.
Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture
Fei Wang (Hefei University of Technology), Meng Wang (Hefei University of Technology)
TransformerContrastive LearningVideo
🎯 What it does: A multi-layer equivalent Transformer architecture based on frequency decoupling is proposed to amplify and enhance subtle movements in videos.
Frequency-Adaptive Dilated Convolution for Semantic Segmentation
Linwei Chen (Beijing Institute of Technology), Ying Fu (RIKEN)
Object DetectionSegmentationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes Frequency Adaptive Dilated Convolution (FADC), which enhances the performance of semantic segmentation and object detection by dynamically adjusting the dilation rate based on local spectral characteristics, decomposing convolutional kernel weights, and balancing frequency on features.
Frequency-aware Event-based Video Deblurring for Real-World Motion Blur
Taewoo Kim (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
RestorationVideo
🎯 What it does: A frequency-aware video deblurring framework based on event cameras is proposed, which integrates video frames and event data and utilizes temporal information to achieve high-quality deblurring.
FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation
Shuai Yang (Peking University), Chen Change Loy (Nanyang Technological University)
Image TranslationGenerationDiffusion modelOptical FlowVideoStochastic Differential Equation
🎯 What it does: A zero-shot video translation framework based on FRESCO is proposed, utilizing both intra-frame self-similarity and inter-frame optical flow dual correspondence to achieve consistent video content while allowing for free style transformation.
Friendly Sharpness-Aware Minimization
Tao Li (Shanghai Jiao Tong University), Xiaolin Huang (Shanghai Jiao Tong University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This study investigates the core mechanism of Sharpness-Aware Minimization (SAM), decomposing the batch gradient into full gradient components and noise components. It finds that the noise component is key to improving generalization and proposes the Friendly-SAM (F-SAM) algorithm based on this insight.
From a Bird's Eye View to See: Joint Camera and Subject Registration without the Camera Calibration
Zekun Qian (Tianjin University), Song Wang (University of South Carolina)
Object DetectionPose EstimationImage
🎯 What it does: The paper proposes an end-to-end framework for jointly completing the registration and localization of the camera and the subject in a bird's-eye view (BEV) without camera calibration information, using multi-view first-person perspectives.
From Activation to Initialization: Scaling Insights for Optimizing Neural Fields
Hemanth Saratchandran (Australian Institute of Machine Learning), Simon Lucey (Australian Institute of Machine Learning)
Super ResolutionOptimizationNeural Radiance FieldImageTime Series
🎯 What it does: This paper provides theoretical analysis and practical guidance on Neural Fields, discussing the impact of activation functions and initialization methods on the over-parameterization required for gradient descent convergence, and based on this, two new initialization schemes are designed.
From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations
Evonne Ng (Meta), Alexander Richard (Meta)
GenerationData SynthesisTransformerDiffusion modelVideoMultimodalityAudio
🎯 What it does: Combining voice input, we generate diverse, synchronized, and realistic full-body dialogue animations of the face, body, and hands, and output high-fidelity videos through a neural renderer;
From Coarse to Fine-Grained Open-Set Recognition
Nico Lang (University of Copenhagen), Serge Belongie
RecognitionConvolutional Neural NetworkImageBenchmark
🎯 What it does: In fine-grained open set recognition (OSR), a large-scale dataset iNat2021-OSR was constructed, dividing the open set into 7 levels based on hierarchical 'hops' and systematically studying the impact of label granularity, semantic similarity, and hierarchical structure on OSR.
From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation
Javier Tirado-Garín (University of Zaragoza), Javier Civera (University of Zaragoza)
Pose EstimationOptimizationImage
🎯 What it does: A method is proposed to estimate the relative pose of the camera directly from matching points without the need for post-processing ambiguity resolution.
From Feature to Gaze: A Generalizable Replacement of Linear Layer for Gaze Estimation
Yiwei Bao (Beihang University), Feng Lu (Beihang University)
Pose EstimationDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: A framework named AGG (Analytical Gaze Generalization) has been designed and implemented, achieving unsupervised cross-domain gaze estimation for different target domains by replacing the traditional fully connected layer with the Geodesic Projection Module (GPM) and Sphere-Oriented Training (SOT).
From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding
Yong-Lu Li (Shanghai Jiao Tong University), Cewu Lu (Shanghai Jiao Tong University)
ClassificationRecognitionPose EstimationConvolutional Neural NetworkTransformerContrastive LearningImageVideoMultimodalityPoint Cloud
🎯 What it does: A structured action semantic space based on VerbNet was constructed, and 28 multimodal datasets were unified and integrated into the Pangea database. Subsequently, a physical-to-semantic space mapping model, P2S, was proposed for multimodal action recognition.
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
Rongjie Li (ShanghaiTech University), Xuming He (ShanghaiTech University)
Object DetectionGenerationTransformerVision Language ModelImageTextGraph
🎯 What it does: A framework for open vocabulary scene graph generation based on a pre-trained vision-language model has been developed, which generates scene graphs directly from pixels using an image-to-text sequence generation approach, and extracts complete relationship triples through entity localization and category mapping.
From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation
Hyeokjun Kweon (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
SegmentationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A weakly supervised semantic segmentation framework S2C is proposed, which directly improves the quality of CAM during the training phase using SAM and generates more accurate pseudo-labels.
From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers
Swaminathan Gurumurthy (Carnegie Mellon University), Zico Kolter (Carnegie Mellon University)
Pose EstimationOptimizationSimultaneous Localization and MappingOptical FlowImage
🎯 What it does: This paper analyzes the gradient variance problem of differentiable bundle adjustment layers in attitude estimation and proposes using weights obtained from internal optimization to weight flow loss in order to reduce gradient noise, thereby accelerating and stabilizing training.
From-Ground-To-Objects: Coarse-to-Fine Self-supervised Monocular Depth Estimation of Dynamic Objects with Ground Contact Prior
Jaeho Moon (Korea Advanced Institute of Science and Technology), Munchurl Kim (Korea Advanced Institute of Science and Technology)
Depth EstimationAutonomous DrivingImage
🎯 What it does: This paper proposes a coarse-to-fine staged self-supervised monocular depth estimation training strategy specifically designed to address depth error issues caused by dynamic objects (such as cars and pedestrians).
Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation
Bingfeng Zhang (China University of Petroleum), Jimin Xiao (XJTLU)
SegmentationTransformerContrastive LearningImage
🎯 What it does: We propose WeCLIP, a weakly supervised semantic segmentation single-stage pipeline that utilizes a frozen CLIP model as the backbone network, and we design a decoder and a CAM refinement module to achieve pixel-level segmentation.
Frozen Feature Augmentation for Few-Shot Image Classification
Andreas Bär (Technische Universität Braunschweig), Manoj Kumar (Google DeepMind)
ClassificationTransformerImage
🎯 What it does: The study directly applies data augmentation (Frozen Feature Augmentation, FroFA) on frozen features of pre-trained vision Transformers to enhance few-shot image classification performance.
FSC: Few-point Shape Completion
Xianzu Wu (Jianghan University), Junsong Yuan (University at Buffalo)
GenerationData SynthesisAutonomous DrivingTransformerGenerative Adversarial NetworkPoint Cloud
🎯 What it does: A shape completion model FSC is proposed for very sparse point cloud inputs, capable of reconstructing a complete 3D point cloud from dozens of points.
FSRT: Facial Scene Representation Transformer for Face Reenactment from Factorized Appearance Head-pose and Facial Expression Features
Andre Rochow (University of Bonn), Sven Behnke (University of Bonn)
GenerationData SynthesisPose EstimationTransformerVideoAudio
🎯 What it does: A Transformer-based facial reenactment framework FSRT is proposed, which can learn a set of scene latent vectors from multiple source images and control the head pose and expression of the target frame through key points and expression vectors.
Fully Convolutional Slice-to-Volume Reconstruction for Single-Stack MRI
Sean I. Young (Harvard Medical School), Juan Eugenio Iglesias (Harvard Medical School)
RestorationSegmentationConvolutional Neural NetworkImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a single-stack MRI slice-to-volume reconstruction (SVR) method based on fully convolutional networks, which directly predicts slice motion and generates artifact-free 3D images through a splat-slice U-Net.
Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing
Yunlong Zhao (Central South University), Wei Chen (Zhejiang University)
Knowledge DistillationAdversarial AttackImage
🎯 What it does: This paper proposes a model stealing method based on superpixel gradients, called SPSG, which utilizes gradient information from a small number of real samples to improve the replication effect on black-box models.
Fully Geometric Panoramic Localization
Junho Kim (Seoul National University), Young Min Kim (Seoul National University)
Pose EstimationOptimizationComputational EfficiencySimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper proposes a completely geometry-based panoramic positioning method that utilizes only 2D-3D line segments and their intersection points for pose search and fine localization.
Fun with Flags: Robust Principal Directions via Flag Manifolds
Nathan Mankovich (University of Valencia), Tolga Birdal (Imperial College London)
Anomaly DetectionOptimizationImage
🎯 What it does: This paper proposes a unified framework based on flag manifolds, which can simultaneously implement traditional PCA, robust PCA, dual PCA, and their tangent space/manifold variants, and provides an efficient Stiefel optimization solver.
Functional Diffusion
Biao Zhang (King Abdullah University of Science and Technology), Peter Wonka (King Abdullah University of Science and Technology)
GenerationData SynthesisTransformerDiffusion modelPoint Cloud
🎯 What it does: A model for diffusion in infinite-dimensional function spaces is proposed, which can directly generate continuous domain functions.
Fusing Personal and Environmental Cues for Identification and Segmentation of First-Person Camera Wearers in Third-Person Views
Ziwei Zhao (Indiana University Bloomington), Chuhua Wang (Indiana University Bloomington)
RecognitionObject DetectionSegmentationGraph Neural NetworkTransformerImageVideo
🎯 What it does: The task of identifying and segmenting individuals wearing first-person cameras in a third-person view under synchronous first-person and third-person perspectives.
FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations
Christian Diller (Technical University of Munich), Angela Dai (Technical University of Munich)
GenerationPose EstimationConvolutional Neural NetworkGenerative Adversarial NetworkVideo
🎯 What it does: This paper proposes a generative model that jointly predicts future action labels and their feature 3D human poses, achieving long-term prediction using only 2D RGB videos and action labels.
G-FARS: Gradient-Field-based Auto-Regressive Sampling for 3D Part Grouping
Junfeng Cheng (Imperial College London), Tania Stathaki (Imperial College London)
GenerationData SynthesisGraph Neural NetworkScore-based ModelPoint CloudStochastic Differential Equation
🎯 What it does: Proposes a 3D component grouping task and designs a gradient field-based autoregressive sampling framework G-FARS to automatically group from a mixed component set.
G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis
Yufei Ye, Shubham Tulsiani
GenerationData SynthesisPose EstimationDiffusion modelScore-based ModelVideoMesh
🎯 What it does: We propose G-HOP, a denoising diffusion-based generative prior model for hand-object interaction that can jointly generate 3D shapes of hands and objects conditioned on given object categories. This prior is used as a guide for reconstructing hand-object interactions from monocular videos and generating natural human grasps given object meshes.
G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images
Zixiong Huang (South China University of Technology), Mingkui Tan (South China University of Technology)
GenerationData SynthesisNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: By using a pre-trained 3D GAN to synthesize multi-view data and combining it with depth perception training, a high-quality new view generation model (G-NeRF) for single images is constructed.
G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding
Yuan Wang (Tsinghua University), Shengjin Wang (Tsinghua University)
RecognitionObject DetectionTransformerContrastive LearningPoint Cloud
🎯 What it does: Proposes the G-3LQ framework, which integrates explicit geometric modeling, fine-grained language-guided queries, and Galois sphere alignment to achieve 3D visual localization tasks.
G3DR: Generative 3D Reconstruction in ImageNet
Pradyumna Reddy (Huawei Noah's Ark Lab), Jiankang Deng (Huawei Noah's Ark Lab)
GenerationData SynthesisDepth EstimationVision Language ModelDiffusion modelNeural Radiance FieldImage
🎯 What it does: A 3D generation framework G3DR based on single-view images is proposed, capable of generating high-quality, geometrically realistic 3D objects from large-scale single-view data such as ImageNet.
GAFusion: Adaptive Fusing LiDAR and Camera with Multiple Guidance for 3D Object Detection
Xiaotian Li (Nanjing University of Posts and Telecommunications), Huijie Fan (Shenyang Institute of Automation Chinese Academy of Science)
Object DetectionAutonomous DrivingTransformerMultimodalityPoint Cloud
🎯 What it does: This paper proposes GAFusion, a multi-modal 3D detection framework guided by LiDAR, which integrates LiDAR and camera information from a BEV perspective to complete the 3D object detection task.
GALA: Generating Animatable Layered Assets from a Single Scan
Taeksoo Kim (Seoul National University), Hanbyul Joo (Seoul National University)
SegmentationGenerationPose EstimationDiffusion modelScore-based ModelMesh
🎯 What it does: Automatically decomposes single-layer scanned clothed human meshes into multi-layer animatable 3D assets, generating complete geometry and textures in a normalized pose space, supporting clothing transfer and virtual try-on in arbitrary poses.
GARField: Group Anything with Radiance Fields
Chung Min Kim (University of California Berkeley), Angjoo Kanazawa (University of California Berkeley)
SegmentationGenerationRetrievalNeural Radiance FieldContrastive LearningGaussian SplattingImagePoint Cloud
🎯 What it does: The GARField method is proposed, which learns a scale-conditioned 3D affinity field using 2D segmentation masks from posed images, thereby generating multi-level layered 3D scene groupings.
Garment Recovery with Shape and Deformation Priors
Ren Li (École Polytechnique Fédérale de Lausanne), Pascal Fua (École Polytechnique Fédérale de Lausanne)
RestorationSegmentationConvolutional Neural NetworkImageMesh
🎯 What it does: Utilizing shape and deformation priors, a high-quality 3D mesh model of loose-fitting clothing is recovered from a single image of the garment.
GART: Gaussian Articulated Template Models
Jiahui Lei (University of Pennsylvania), Kostas Daniilidis (University of Pennsylvania)
GenerationPose EstimationComputational EfficiencyGaussian SplattingVideo
🎯 What it does: This paper presents GART, a method that uses Gaussian Mixture Models (GMM) to explicitly model the shape and appearance of non-rigid movable subjects (humans, animals) and can quickly reconstruct and render from monocular video.
Gated Fields: Learning Scene Reconstruction from Gated Videos
Andrea Ramazzina (Mercedes Benz), Felix Heide (Princeton University)
GenerationDepth EstimationNeural Radiance FieldImageVideo
🎯 What it does: Using active gated camera sequences of time-domain gated video, a neural rendering method called Gated Fields is proposed for large-scale 3D reconstruction and depth inference of outdoor scenes.
GauHuman: Articulated Gaussian Splatting from Monocular Human Videos
Shoukang Hu, Ziwei Liu
GenerationPose EstimationComputational EfficiencyGaussian SplattingVideo
🎯 What it does: Fast learning and real-time rendering of 3D human models in monocular videos
Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians
Yuelang Xu (Tsinghua University), Yebin Liu (NNKosmos Technology)
GenerationData SynthesisPose EstimationSuper ResolutionGaussian SplattingImagePoint Cloud
🎯 What it does: This paper presents the Gaussian Head Avatar, which reconstructs high-fidelity human head models through controllable dynamic 3D Gaussian point clouds, enabling expression-driven and pose control.
Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models
Zijin Yang (University of Science and Technology of China), Nenghai Yu (University of Science and Technology of China)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: A watermarking method for diffusion models called Gaussian Shading is proposed, which achieves lossless watermark embedding and extraction without modifying the model parameters.
Gaussian Shadow Casting for Neural Characters
Luis Bolanos (University of British Columbia), Helge Rhodin (Bielefeld University)
GenerationOptimizationNeural Radiance FieldVideo
🎯 What it does: By introducing a deformable anisotropic Gaussian density proxy and combining it with a deferred rendering framework, we achieve differentiable and efficient shadow casting, thereby reconstructing high-quality shadows and relightable scenes for dynamic portraits under strong directional lighting.
Gaussian Shell Maps for Efficient 3D Human Generation
Rameen Abdal (Stanford University), Gordon Wetzstein (Stanford University)
GenerationData SynthesisPose EstimationComputational EfficiencyGenerative Adversarial NetworkGaussian SplattingImage
🎯 What it does: Using GAN and 3D Gaussian rendering technology, we generate animatable high-resolution digital human models and achieve real-time rendering without traditional upsampling.
Gaussian Splatting SLAM
Hidenobu Matsuki (Imperial College London), Andrew J. Davison (Imperial College London)
Pose EstimationOptimizationRobotic IntelligenceGaussian SplattingSimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper presents the first online visual SLAM system based on 3D Gaussian Splatting (3DGS), supporting monocular and RGB-D inputs, capable of real-time camera pose tracking, incrementally constructing high-fidelity 3D scenes, and providing real-time high-quality view synthesis.
Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle
Youtian Lin (Nanjing University), Yao Yao (Nanjing University)
GenerationData SynthesisComputational EfficiencyNeural Radiance FieldGaussian SplattingVideo
🎯 What it does: A dynamic scene reconstruction and real-time rendering framework called Gaussian-Flow based on 3D Gaussian points is proposed, which can efficiently generate 4D scenes from multi-view or monocular videos.
GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians
Liangxiao Hu (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
GenerationOptimizationGaussian SplattingVideo
🎯 What it does: We propose GaussianAvatar, which reconstructs human avatars from a single video using animatable 3D Gaussian representations, combining motion and appearance joint optimization.
GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians
Shenhan Qian (Technical University of Munich), Matthias Nießner (Technical University of Munich)
GenerationData SynthesisPose EstimationGaussian SplattingVideo
🎯 What it does: Generate animatable and controllable realistic facial avatars from multi-view videos, achieving precise control over expressions, poses, and viewpoints.
GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models
Taoran Yi (Huazhong University of Science and Technology), Xinggang Wang (Huazhong University of Science and Technology)
GenerationData SynthesisDiffusion modelScore-based ModelGaussian SplattingTextPoint CloudBenchmark
🎯 What it does: A text-to-3D generation framework named GaussianDreamer has been constructed, capable of quickly generating 3D assets with high 3D consistency and detail in just 15 minutes on a single GPU, and supports real-time rendering.
GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions
Junjie Wang (Huawei Inc), Qi Tian (Huawei Inc)
SegmentationGenerationLarge Language ModelGaussian SplattingImagePoint Cloud
🎯 What it does: This paper presents GaussianEditor, an interactive framework based on 3D Gaussian splatting that enables fine 3D scene editing using text commands.
GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
Yiwen Chen (Nanyang Technological University), Guosheng Lin (Nanyang Technological University)
RestorationGenerationDiffusion modelGaussian SplattingImagePoint Cloud
🎯 What it does: Developed GaussianEditor, a 3D editing framework based on Gaussian Splatting, achieving fast (5–10 minutes) and controllable modifications of 3D scenes.
GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
Yingwenqi Jiang (ShanghaiTech University), Yuexin Ma (ShanghaiTech University)
Gaussian SplattingPoint Cloud
🎯 What it does: This paper proposes GaussianShader, which incorporates a simplified shadow function based on 3D Gaussian splatting to achieve real-time rendering and handle reflective surfaces.
GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning
Ye Yuan (NVIDIA), Umar Iqbal (NVIDIA)
GenerationData SynthesisDiffusion modelScore-based ModelGaussian SplattingImageMesh
🎯 What it does: Generate animatable high-quality 3D Gaussian avatars based on text prompts, and extract fine texture meshes.
GDA: Generalized Diffusion for Robust Test-time Adaptation
Yun-Yun Tsai (Columbia University), Cheng-Hao Kuo (Amazon)
ClassificationDomain AdaptationConvolutional Neural NetworkDiffusion modelContrastive LearningImage
🎯 What it does: A test-time adaptation method based on diffusion models, GDA, is proposed to migrate OOD samples back to the source domain, thereby improving the classification accuracy of the model under various distribution shifts.
Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling
Xinhang Liu (Hong Kong University of Science and Technology), Moitreya Chatterjee (Mitsubishi Electric Research Laboratories)
Object TrackingSegmentationNeural Radiance FieldVideo
🎯 What it does: Developed Gear-NeRF, achieving free-viewpoint rendering of dynamic scenes and free-viewpoint tracking based on single-point clicks.
GEARS: Local Geometry-aware Hand-object Interaction Synthesis
Keyang Zhou (University of Tübingen), Gerard Pons-Moll (Max Planck Institute for Informatics)
GenerationData SynthesisPose EstimationTransformerPoint Cloud
🎯 What it does: Based on the trajectory of the hand and the object, realistic hand motion sequences are generated to achieve the synthesis of hand-object interaction poses.
GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image
Chong Bao (Zhejiang University), Zhaopeng Cui (Zhejiang University)
GenerationData SynthesisNeural Radiance FieldGenerative Adversarial NetworkImagePoint Cloud
🎯 What it does: A general 3D head avatar editing framework called GeneAvatar is proposed, which can transfer editing effects from single-view 2D edits (such as dragging, text prompts, and pattern drawing) to voxel avatars with multiple expressions and viewpoints through an expression-aware modification generator.
General Object Foundation Model for Images and Videos at Scale
Junfeng Wu (Huazhong University of Science and Technology), Song Bai (ByteDance Inc.)
Object DetectionObject TrackingSegmentationKnowledge DistillationConvolutional Neural NetworkTransformerImageVideo
🎯 What it does: GLEE is proposed, a unified object-level visual foundation model that can simultaneously perform detection, instance segmentation, localization, tracking, interactive segmentation, and other tasks on images and videos, supporting arbitrary text or visual prompts.
General Point Model Pretraining with Autoencoding and Autoregressive
Zhe Li (Huazhong University of Science and Technology), Stan Z. Li (Westlake University)
ClassificationSegmentationGenerationRepresentation LearningTransformerAuto EncoderContrastive LearningPoint Cloud
🎯 What it does: A unified Transformer framework GPM is proposed, combining BERT-style autoencoding and GPT-style autoregression, supporting pre-training of point clouds, downstream classification/segmentation, and unconditional/conditional generation.
Generalizable Face Landmarking Guided by Conditional Face Warping
Jiayi Liang (Beijing Institute of Technology), Dixin Luo (Beijing Institute of Technology)
RecognitionDomain AdaptationImage
🎯 What it does: A general facial keypoint detection method based on conditional facial deformation is proposed, utilizing unlabelled stylized faces to generate pseudo-labels to enhance the model's generalization ability.
Generalizable Novel-View Synthesis using a Stereo Camera
Haechan Lee (POSTECH), Sunghyun Cho (POSTECH)
Data SynthesisDepth EstimationNeural Radiance FieldImageBenchmark
🎯 What it does: A generalizable view synthesis method using stereo camera images, StereoNeRF, is proposed, which integrates binocular matching information to enhance geometric reconstruction quality.
Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction
Hao Li (Xiamen University), Yuchen Han (Shanghai Jiao Tong University)
ClassificationTransformerLarge Language ModelPrompt EngineeringContrastive LearningImageTextBiomedical Data
🎯 What it does: This paper proposes a whole slide image (WSI) classification framework called FiVE, which is based on fine-grained visual-semantic interaction. It utilizes GPT-4 to automatically extract fine-grained descriptive labels from raw pathology reports and achieves efficient training through a task-specific fine-grained semantic (TFS) module and sampling strategy.
Generalized Event Cameras
Varun Sundar (University of Wisconsin-Madison), Mohit Gupta (University of Wisconsin-Madison)
Object DetectionSegmentationCompressionOptical FlowVideo
🎯 What it does: This paper proposes a general event camera framework that can fully retain scene intensity information while maintaining low bandwidth, and implements three specific designs using single-photon detectors.
Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching
Shitong Shao (Mohamed bin Zayed University of Artificial Intelligence), Zhiqiang Shen (Mohamed bin Zayed University of Artificial Intelligence)
CompressionKnowledge DistillationImage
🎯 What it does: A novel data distillation framework named G-VBSM is proposed, achieving efficient data compression and distillation for ImageNet-1k and small-scale datasets through data densification, universal backbone matching, and universal statistical matching.
Generalized Predictive Model for Autonomous Driving
Jiazhi Yang (OpenDriveLab and Shanghai AI Lab), Hongyang Li
GenerationAutonomous DrivingDiffusion modelVideoMultimodality
🎯 What it does: This paper proposes a large-scale video prediction model for autonomous driving, GenAD, trained using 2000 hours of driving videos and multimodal text obtained from the internet.
Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge
Haoxiang Ma (Beihang University), Di Huang (Beihang University)
Object DetectionOptimizationRobotic IntelligenceConvolutional Neural NetworkPoint CloudBenchmark
🎯 What it does: A 6-DoF grasp detection framework based on domain prior knowledge is proposed, enhancing the generalization ability of grasp detection through Physical Constraint Regularization (PCR) and Projected Contact-Score Joint Optimization (C-SJO);
Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models
Bin Fu (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), Yu Qiao (Shanghai Artificial Intelligence Laboratory)
GenerationDiffusion modelImage
🎯 What it does: A multi-stage font generation framework called MSD-Font is proposed, which achieves a three-stage generation process from structure construction to style transfer and then to detail refinement by embedding the font transfer process in a latent diffusion model.
Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts
Fei Ni (Tianjin University), Yuzheng Zhuang (Tianjin University)
GenerationRobotic IntelligenceDiffusion modelImageMultimodalityBenchmarkChain-of-Thought
🎯 What it does: The CoTDiffusion framework is proposed, which uses diffusion models to generate coherent sub-goal images from multimodal instructions in a chain-of-thought manner, and then executes long-term robotic operations through a low-level policy model.
Generating Content for HDR Deghosting from Frequency View
Tao Hu (Northwestern Polytechnical University), Yanning Zhang (Northwestern Polytechnical University)
RestorationGenerationConvolutional Neural NetworkDiffusion modelOptical FlowImage
🎯 What it does: Achieving ghost-free HDR reconstruction of multi-exposure images through a low-frequency prior diffusion model (LF-Diff) combined with a regression network.
Generating Enhanced Negatives for Training Language-Based Object Detectors
Shiyu Zhao (Rutgers University), Samuel Schulter (NEC Laboratories America)
Object DetectionTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: This paper proposes the use of large language models and text-to-image diffusion models to automatically generate negative samples that are semantically similar but not matching to positive samples (text and corresponding images), and incorporates them into the training of language-based object detection.
Generating Handwritten Mathematical Expressions From Symbol Graphs: An End-to-End Pipeline
Yu Chen (Beijing Waiyan Online Digital Technology), Nannan Wang (Xidian University)
GenerationGraph Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: An end-to-end pipeline is proposed for generating handwritten mathematical expression images from symbolic graphs, directly mapping the graph structure to layout, mask, and then to image, completing the synthesis and editing of handwritten formulas.
Generating Human Motion in 3D Scenes from Text Descriptions
Zhi Cen (Zhejiang University), Xiaowei Zhou (Zhejiang University)
GenerationPose EstimationTransformerLarge Language ModelDiffusion modelTextPoint Cloud
🎯 What it does: This paper studies how to generate human actions that interact with target objects in 3D indoor scenes based on textual descriptions.
Generating Illustrated Instructions
Sachit Menon (Meta), Rohit Girdhar (Meta)
GenerationLarge Language ModelDiffusion modelImageText
🎯 What it does: This paper proposes the task of generating 'illustrated manuals' that include textual steps and corresponding illustrations, and implements the StackedDiffusion framework based on LLM and diffusion models, capable of generating complete instructional documents in one go.
Generating Non-Stationary Textures using Self-Rectification
Yang Zhou (Shenzhen University), Hui Huang (Shenzhen University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: Generate non-stationary textures through user rough editing and self-correction using a pre-trained diffusion model;
Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing
Bi'an Du (Peking University), Renjie Liao (University of British Columbia)
GenerationPose EstimationTransformerPoint Cloud
🎯 What it does: This paper proposes a generative 3D part assembly network based on hyper-part and whole hierarchical information, achieving the prediction of the 6 degrees of freedom pose of parts.
Generative Image Dynamics
Zhengqi Li (Google Research), Aleksander Holynski
GenerationData SynthesisDiffusion modelOptical FlowImageVideo
🎯 What it does: Generating natural oscillatory motion videos from a single static image, supporting seamless looping and interactive dynamics.
Generative Latent Coding for Ultra-Low Bitrate Image Compression
Zhaoyang Jia (University of Science and Technology of China), Yan Lu (Microsoft Research)
RestorationGenerationCompressionAuto EncoderImage
🎯 What it does: This paper proposes an image compression framework (GLC) that performs transform coding in the latent space of generative VQ-VAE, achieving high fidelity and realism compression at extremely low bit rates, and enabling applications such as image restoration and style transfer.
Generative Multi-modal Models are Good Class Incremental Learners
Xusheng Cao (Nankai University), Ming-Ming Cheng (Nankai University)
ClassificationGenerationTransformerLarge Language ModelImageMultimodality
🎯 What it does: An incremental learning framework based on Generative Multimodal Models (GMM) is proposed, achieving class-incremental learning without a classification head by generating label text and matching it with category text.
Generative Multimodal Models are In-Context Learners
Quan Sun (Beijing Academy of Artificial Intelligence), Xinlong Wang (Beijing Academy of Artificial Intelligence)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageVideoTextMultimodality
🎯 What it does: A 37B parameter multimodal generative model, Emu2, was proposed and trained, demonstrating strong contextual learning capabilities, and achieving chat and controllable visual generation through instruction fine-tuning.
Generative Powers of Ten
Xiaojuan Wang (University of Washington), Aleksander Holynski (UC Berkeley)
GenerationSuper ResolutionDiffusion modelImageVideo
🎯 What it does: A generative framework based on multi-scale joint diffusion sampling is proposed, capable of generating semantically consistent and continuous enlarged videos at different scaling levels.
Generative Proxemics: A Prior for 3D Social Interaction from Images
Lea Müller, Angjoo Kanazawa (University of California Berkeley)
GenerationPose EstimationOptimizationTransformerDiffusion modelScore-based ModelImagePoint Cloud
🎯 What it does: This paper proposes BUDDI, a 3D social proximity behavior generator based on diffusion models, and uses it as an unsupervised prior for 3D voxel reconstruction of two people from a single-view image.
Generative Quanta Color Imaging
Vishal Purohit (Purdue University), Qiang Qiu (Purdue University)
RestorationGenerationData SynthesisGenerative Adversarial NetworkImageOrdinary Differential Equation
🎯 What it does: This paper proposes the use of a single binary frame from a single-photon camera, leveraging neural ODE and filter atom decomposition techniques to generate continuous exposure sequences and achieve high-quality color imaging.
Generative Region-Language Pretraining for Open-Ended Object Detection
Chuang Lin (Monash University), Jianfei Cai (Monash University)
Object DetectionTransformerLarge Language ModelImageTextMultimodality
🎯 What it does: A generative open-set object detection framework called GenerateU is proposed without predefined categories;
Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models
Shengqu Cai (Stanford University), Gordon Wetzstein (Stanford University)
GenerationData SynthesisDiffusion modelVideoMesh
🎯 What it does: By combining low-fidelity 3D meshes and their motion with a depth-conditioned 2D diffusion model, controllable style video rendering is achieved;
Generative Unlearning for Any Identity
Juwon Seo (Kyung Hee University), Gyeong-Moon Park (Kyung Hee University)
GenerationData SynthesisGenerative Adversarial NetworkImageBenchmark
🎯 What it does: This paper proposes a generative model identity forgetting framework called GUIDE, which can completely eliminate the generative capability of an identity using only a single image in a pre-trained 3D GAN (EG3D);
GenesisTex: Adapting Image Denoising Diffusion to Texture Space
Chenjian Gao (Beihang University), Qian Yu (Tencent)
RestorationGenerationData SynthesisDiffusion modelMesh
🎯 What it does: A method called GenesisTex has been developed, which generates high-quality textures for a given 3D mesh using text descriptions, and achieves multi-view parallel generation through texture space sampling techniques.
GenFlow: Generalizable Recurrent Flow for 6D Pose Refinement of Novel Objects
Sungphill Moon (NAVER LABS), Sangwook Kim (NAVER LABS)
Pose EstimationRecurrent Neural NetworkOptical FlowImage
🎯 What it does: An iterative pose refinement framework based on optical flow, GenFlow, is proposed, which utilizes the 3D shape information of the target object to perform fine estimation of the 6D pose of unseen objects.
GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation Demonstration and Imitation
Zifan Wang (Tsinghua University), Li Yi (Tsinghua University)
OptimizationRobotic IntelligenceReinforcement LearningPoint Cloud
🎯 What it does: The GenH2R framework is proposed, which learns general visual-driven human-robot handover skills through large-scale synthetic simulation, automatic demonstration generation, and prediction-assisted 4D inverse learning.
GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos
Tomáš Souček (Czech Technical University), Josef Sivic (Czech Technical University)
GenerationData SynthesisDiffusion modelImageVideoText
🎯 What it does: Trained a text + image conditional generation network based on a diffusion model, capable of generating corresponding action images and target state images based on input images and textual prompts of actions or final states, while maintaining scene consistency.
GenN2N: Generative NeRF2NeRF Translation
Xiangyue Liu (Hong Kong University of Science and Technology), Li Yi (Tsinghua University)
Image TranslationRestorationGenerationSuper ResolutionDiffusion modelNeural Radiance FieldGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: We propose GenN2N, a unified NeRF-to-NeRF translation framework capable of performing various 3D NeRF editing tasks such as text-driven editing, coloring, super-resolution, and completion within a single model.
GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction
Xiao Chen (Chinese University of Hong Kong), Jiangmiao Pang (Shanghai AI Laboratory)
Robotic IntelligenceReinforcement LearningPoint Cloud
🎯 What it does: A next best view (NBV) strategy that can generalize in free space for active 3D reconstruction is proposed.
GenTron: Diffusion Transformers for Image and Video Generation
Shoufa Chen (University of Hong Kong), Juan-Manuel Perez-Rua (Meta)
GenerationData SynthesisTransformerDiffusion modelImageVideoText
🎯 What it does: Designed and trained a Transformer-based diffusion model GenTron for high-quality text-to-image and video generation.
Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal
Yijun Yang, Lei Zhu
RestorationDomain AdaptationVideo
🎯 What it does: Insufficient information provided
GenZI: Zero-Shot 3D Human-Scene Interaction Generation
Lei Li (Technical University of Munich), Angela Dai (Technical University of Munich)
GenerationPose EstimationOptimizationTransformerVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Given a 3D scene, text description, and rough interaction position, a large-scale visual-language model is used to generate 2D human-computer interaction images, which are then elevated to a 3D human-computer interaction model through robust multi-view optimization, ultimately resulting in a 3D human body that can interact freely within the scene.
GeoAuxNet: Towards Universal 3D Representation Learning for Multi-sensor Point Clouds
Shengjun Zhang (Tsinghua University), Yueqi Duan (Tsinghua University)
SegmentationRepresentation LearningPoint Cloud
🎯 What it does: This paper proposes GeoAuxNet, which uses a voxel-guided dynamic point network to generate point-level geometric information and injects this fine-grained point information into voxel features through a hierarchical geometric pooling, thereby achieving unified representation learning of multi-sensor point clouds.
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
Kartik Kuckreja (Mohamed bin Zayed University of AI), Fahad Shahbaz Khan (Linkoping University)
ClassificationRecognitionObject DetectionSegmentationTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This paper presents GeoChat, a visual conversational large model for remote sensing images, capable of performing multi-tasks such as image-level question answering, region-level dialogue, image/region description, scene classification, and visual-oriented dialogue within a single framework.
Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding
Guofeng Mei (Fondazione Bruno Kessler), Fabio Poiesi (Fondazione Bruno Kessler)
ClassificationSegmentationRepresentation LearningVision Language ModelContrastive LearningPoint Cloud
🎯 What it does: A training-independent geometry-driven aggregation method called GeoZe is proposed to transfer 2D vision-language model (VLM) representations to 3D point clouds and enhance zero-shot understanding performance.
Geometry Transfer for Stylizing Radiance Fields
Hyunyoung Jung (Seoul National University), Rakesh Ranjan (Meta Reality Labs)
Image TranslationGenerationDepth EstimationNeural Radiance FieldImagePoint Cloud
🎯 What it does: Using depth maps as style guidance, we perform style transfer on the geometry of neural radiance fields (NeRF) with deformation fields, and further synchronize geometry and appearance stylization by combining RGB-D.
Geometry-aware Reconstruction and Fusion-refined Rendering for Generalizable Neural Radiance Fields
Tianqi Liu (Huazhong University of Science and Technology), Zhiguo Cao (Huazhong University of Science and Technology)
GenerationData SynthesisNeural Radiance FieldPoint Cloud
🎯 What it does: A generalizable neural radiance field framework called GeFu is proposed, which can synthesize high-quality new views in unseen scenes using just a few perspectives.
GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement
Linfang Zheng (Southern University of Science and Technology), Hyung Jin Chang (University of Birmingham)
Pose EstimationGraph Neural NetworkPoint Cloud
🎯 What it does: This paper proposes a new framework called GeoReF for category-level object pose refinement, aimed at addressing the pose error prediction problem caused by shape variations.