CVPR 2023 Papers — Page 23
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2353 papers
Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction
Yi Xu (Northeastern University), Yun Fu (Northeastern University)
Recurrent Neural NetworkGraph Neural NetworkTime SeriesSequential
🎯 What it does: A unified GC-VRNN framework is proposed, capable of simultaneously performing missing value imputation for multi-agent trajectories and future trajectory prediction.
Uncurated Image-Text Datasets: Shedding Light on Demographic Bias
Noa Garcia (Osaka University), Yuta Nakashima (Osaka University)
Object DetectionGenerationRetrievalTransformerVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: This study constructs PHASE annotations containing six perceptual attributes: age, gender, skin color, race, emotion, and activity on the large-scale unfiltered image-text dataset GCC, and evaluates the social biases and amplification phenomena in image description, CLIP embedding, and text-to-image generation tasks based on this.
Understanding and Constructing Latent Modality Structures in Multi-Modal Representation Learning
Qian Jiang (University of Illinois at Urbana Champaign), Trishul Chilimbi (Amazon)
RetrievalRepresentation LearningTransformerContrastive LearningImageTextMultimodality
🎯 What it does: This study investigates the impact of latent modal structures on downstream tasks in multimodal representation learning and proposes three regularization methods to enhance representation quality.
Understanding and Improving Features Learned in Deep Functional Maps
Souhaib Attaiki (École Polytechnique), Maks Ovsjanikov (École Polytechnique)
OptimizationRepresentation LearningGraph Neural NetworkContrastive LearningPoint CloudMesh
🎯 What it does: This paper studies the geometric meaning of the feature functions learned in deep functional maps and proposes two simple improvement methods to enhance the accuracy of shape correspondence.
Understanding and Improving Visual Prompting: A Label-Mapping Perspective
Aochuan Chen (Michigan State University), Sijia Liu (MIT-IBM Watson AI Lab)
ClassificationOptimizationExplainability and InterpretabilityPrompt EngineeringContrastive LearningImage
🎯 What it does: A visual prompting framework based on label mapping (LM) called ILM-VP is proposed, which can improve the accuracy of the target task and enhance interpretability through co-learning of LM and visual prompts via iterative optimization, without fine-tuning the source model.
Understanding Deep Generative Models With Generalized Empirical Likelihoods
Suman Ravuri (DeepMind), Marc Peter Deisenroth (University College London)
GenerationAnomaly DetectionExplainability and InterpretabilityDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: A framework based on Generalized Empirical Likelihood (GEL) is proposed and implemented to diagnose the deficiencies of deep generative models (such as GANs and diffusion models) in aspects such as mode collapse, mode imbalance, inaccurate label conditions, and whether generated samples fall outside the data distribution; interpretable per-sample scores are also provided.
Understanding Imbalanced Semantic Segmentation Through Neural Collapse
Zhisheng Zhong (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
SegmentationConvolutional Neural NetworkSupervised Fine-TuningImagePoint Cloud
🎯 What it does: This paper studies the geometric structure of the last layer feature centers and classifiers in semantic segmentation. It finds that, unlike the neural collapse in image classification, semantic segmentation loses isometric symmetry in feature centers and classifiers due to contextual relevance and class imbalance, leading to poor performance in minority classes.
Understanding Masked Autoencoders via Hierarchical Latent Variable Models
Lingjing Kong (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
Object DetectionSegmentationRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: A theoretical framework based on a hierarchical latent variable model is proposed to analyze the identifiability of Masked Autoencoders (MAE), proving that MAE can identify high-level latent variables under reasonable assumptions, and indicating that the masking ratio and block size determine the level of abstraction learned.
Understanding Masked Image Modeling via Learning Occlusion Invariant Feature
Xiangwen Kong (MEGVII Technology), Xiangyu Zhang (Beijing Academy of Artificial Intelligence)
Representation LearningTransformerContrastive LearningImage
🎯 What it does: This paper proposes MIM by learning the perspective of occlusion-invariant features to explain its success, and introduces two Siamese-based improved frameworks: RelaxMIM and C-MAE;
Understanding the Robustness of 3D Object Detection With Bird's-Eye-View Representations in Autonomous Driving
Zijian Zhu (Shanghai Jiao Tong University), Shibao Zheng (Shanghai Jiao Tong University)
Object DetectionAutonomous DrivingAdversarial AttackConvolutional Neural NetworkTransformerImageVideo
🎯 What it does: This paper systematically evaluates the robustness of bird's-eye view (BEV) based 3D object detection models under natural and adversarial conditions, and proposes a 3D consistency patch attack for multi-camera continuous frames.
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
Hao Li (Chinese University of Hong Kong), Jifeng Dai
ClassificationObject DetectionSegmentationRetrievalTransformerMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: This paper presents Uni-Perceiver v2, a unified general model capable of direct inference on multi-tasking (such as classification, detection, segmentation, retrieval, description, etc.) in both single-modal and cross-modal settings without the need for task-specific fine-tuning.
Uni3D: A Unified Baseline for Multi-Dataset 3D Object Detection
Bo Zhang, Yu Qiao
Object Detection
🎯 What it does: The specific content of the paper is not provided, making it impossible to determine the research work.
Unicode Analogies: An Anti-Objectivist Visual Reasoning Challenge
Steven Spratley (University of Melbourne), Tim Miller (University of Melbourne)
Convolutional Neural NetworkImage
🎯 What it does: Designed and released an advanced matrix problem (Unicode Analogies) dataset based on Unicode characters to evaluate the analogy reasoning and fluid conceptualization capabilities of visual systems;
UniDAformer: Unified Domain Adaptive Panoptic Segmentation Transformer via Hierarchical Mask Calibration
Jingyi Zhang (Nanyang Technological University), Shijian Lu (Nanyang Technological University)
SegmentationDomain AdaptationTransformerImage
🎯 What it does: A unified domain adaptive panoramic segmentation Transformer called UniDAformer is designed, and a Hierarchical Mask Calibration (HMC) technique is proposed to simultaneously handle instance and semantic segmentation within a single network and correct pseudo labels through online self-training.
UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy
Yinzhen Xu (Peking University), He Wang (Peking University)
Knowledge DistillationRobotic IntelligenceReinforcement LearningPoint Cloud
🎯 What it does: A two-stage universal multi-finger grasping framework called UniDexGrasp is proposed, which first generates diverse grasp poses under point cloud observations and then implements grasping and lifting through a target-conditioned policy.
UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye View
Shengchao Zhou (MEGVII Technology), Chao Ma (Shanghai Jiao Tong University)
Object DetectionAutonomous DrivingKnowledge DistillationConvolutional Neural NetworkMultimodalityPoint Cloud
🎯 What it does: A universal cross-modal knowledge distillation framework called UniDistill is proposed to enhance the performance of single-modal 3D object detectors.
Unified Keypoint-Based Action Recognition Framework via Structured Keypoint Pooling
Ryo Hachiuma (Konica Minolta), Taiki Sekii (Konica Minolta)
RecognitionPose EstimationConvolutional Neural NetworkVideoPoint Cloud
🎯 What it does: A unified keypoint-based action recognition framework is proposed, using Structured Keypoint Pooling to treat skeletal and object contour keypoints as 3D point clouds, achieving action recognition and spatiotemporal action localization.
Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentation
Liulei Li (Zhejiang University), Yi Yang (ETH Zurich)
Object DetectionSegmentationConvolutional Neural NetworkContrastive LearningVideo
🎯 What it does: This paper proposes a completely label-free video object segmentation method that automatically generates pseudo-masks through spatiotemporal pixel clustering on unlabeled videos, and jointly learns mask embeddings and cross-frame correspondences. The trained network can directly perform mask-based continuous segmentation.
Unified Pose Sequence Modeling
Lin Geng Foo (Singapore University of Technology and Design), Jun Liu (Singapore University of Technology and Design)
Pose EstimationTransformerSupervised Fine-TuningVideoMultimodality
🎯 What it does: A unified posture sequence modeling method is proposed, which can simultaneously accomplish various posture-based tasks such as action recognition, 3D pose estimation, and early action prediction within a single model.
Unifying Layout Generation With a Decoupled Diffusion Model
Mude Hui (Xi'an Jiaotong University), Yan Lu (Microsoft Research Asia)
GenerationData SynthesisTransformerDiffusion modelImage
🎯 What it does: This paper proposes the Layout Diffusion Generative Model (LDGM), which decouples the forward diffusion process of different attributes and employs a unified backward denoising Transformer, enabling the simultaneous completion of various layout generation sub-tasks such as unconditional generation, control by type/size/relation, refinement, and completion.
Unifying Short and Long-Term Tracking With Graph Hierarchies
Orcun Cetintas (Technical University of Munich), Laura Leal-Taixé (Technical University of Munich)
Object TrackingGraph Neural NetworkVideo
🎯 What it does: A unified multi-scale multi-object tracking framework called SUSHI is proposed, which recursively merges short trajectories into long ones through a hierarchical graph structure, achieving effective tracking of long videos.
Unifying Vision, Text, and Layout for Universal Document Processing
Zineng Tang (University of North Carolina at Chapel Hill), Mohit Bansal (University of North Carolina at Chapel Hill)
ClassificationRecognitionGenerationTransformerImageTextMultimodalityBenchmark
🎯 What it does: This paper presents UDOP, a unified document AI foundation model capable of simultaneously processing visual, textual, and layout information, and consolidating various document tasks into a sequence generation framework, supporting document understanding, question answering, information extraction, classification, as well as document generation and editing.
UniHCP: A Unified Model for Human-Centric Perceptions
Yuanzheng Ci (University of Sydney), Wanli Ouyang (Shanghai AI Laboratory)
RecognitionObject DetectionSegmentationPose EstimationTransformerImage
🎯 What it does: This paper presents UniHCP, a unified Vision Transformer model capable of simultaneously handling five types of human perception tasks: pose estimation, human segmentation, pedestrian detection, person re-identification (ReID), and attribute recognition.
UniSim: A Neural Closed-Loop Sensor Simulator
Ze Yang (Waabi), Raquel Urtasun (Waabi)
Autonomous DrivingConvolutional Neural NetworkNeural Radiance FieldGenerative Adversarial NetworkImagePoint Cloud
🎯 What it does: UniSim constructs an editable digital twin from a single driving log, achieving high-fidelity closed-loop simulation with multi-modal sensors (cameras and LiDAR).
Unite and Conquer: Plug & Play Multi-Modal Synthesis Using Diffusion Models
Nithin Gopalakrishnan Nair (Johns Hopkins University), Vishal M. Patel (Johns Hopkins University)
GenerationData SynthesisDiffusion modelImageTextMultimodality
🎯 What it does: A framework is proposed for multimodal image synthesis using multimodal diffusion models (such as text, semantic labels, category conditions, etc.) without the need for retraining. The core is based on closed sampling of diffusion models and the fusion of Generalized Product of Experts.
Universal Instance Perception As Object Discovery and Retrieval
Bin Yan (Dalian University of Technology), Huchuan Lu (Dalian University of Technology)
Object DetectionObject TrackingSegmentationRetrievalTransformerPrompt EngineeringImageVideoText
🎯 What it does: A unified instance-aware model called UNINEXT is proposed, which can complete 10 types of instance-aware sub-tasks, including object detection, instance segmentation, video tracking, and semantic video segmentation, within the same framework through prompts (category names, language expressions, reference boxes/masks).
Unknown Sniffer for Object Detection: Don't Turn a Blind Eye to Unknown Objects
Wenteng Liang (Beijing University of Posts and Telecommunications), Anlong Ming (Chongqing University of Posts and Telecommunications)
Object DetectionConvolutional Neural NetworkImageBenchmark
🎯 What it does: The UnSniffer framework is proposed to simultaneously detect unknown objects and known objects under known categories in object detection.
Unlearnable Clusters: Towards Label-Agnostic Unlearnable Examples
Jiaming Zhang (Beijing Jiaotong University), Changsheng Xu (Chinese Academy of Sciences)
ClassificationSafty and PrivacyAdversarial AttackConvolutional Neural NetworkVision Language ModelGenerative Adversarial NetworkImage
🎯 What it does: This study investigates the generation of 'Unlearnable Examples' in label-agnostic scenarios to prevent unauthorized model training.
Unpaired Image-to-Image Translation With Shortest Path Regularization
Shaoan Xie (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
Image TranslationGenerationAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: A framework for unpaired image translation based on the shortest path assumption is proposed, constructing a continuous path from the source domain to the target domain, and constraining the generative model by regularizing the path length.
Unsupervised 3D Point Cloud Representation Learning by Triangle Constrained Contrast for Autonomous Driving
Bo Pang (Shanghai Jiao Tong University), Cewu Lu (Shanghai Jiao Tong University)
Object DetectionSegmentationAutonomous DrivingRepresentation LearningConvolutional Neural NetworkContrastive LearningMultimodalityPoint Cloud
🎯 What it does: For autonomous driving scenarios, a novel unsupervised 3D point cloud representation learning framework called TriCC is proposed, which can simultaneously utilize temporal sequences and multimodal information for dense representation learning.
Unsupervised 3D Shape Reconstruction by Part Retrieval and Assembly
Xianghao Xu (Brown University), Daniel Ritchie (Adobe Research)
GenerationRetrievalOptimizationAuto EncoderPoint Cloud
🎯 What it does: Reconstruct the target 3D shape by retrieving and assembling parts from a user-provided 3D parts library in an unsupervised manner;
Unsupervised Continual Semantic Adaptation Through Neural Rendering
Zhizheng Liu (ETH Zurich), Cesar Cadena (ETH Zurich)
SegmentationDomain AdaptationNeural Radiance FieldPoint Cloud
🎯 What it does: This paper proposes an unsupervised continuous semantic segmentation model adaptation method, utilizing multi-view data to train Semantic-NeRF to generate three-dimensional consistent pseudo-labels, and alleviating catastrophic forgetting through joint 2D-3D training and NeRF playback.
Unsupervised Contour Tracking of Live Cells by Mechanical and Cycle Consistency Losses
Junbong Jang (Korea Advanced Institute of Science and Technology), Tae-Kyun Kim (Korea Advanced Institute of Science and Technology)
Object TrackingSegmentationConvolutional Neural NetworkOptical FlowVideoBiomedical Data
🎯 What it does: A deep learning-based unsupervised cell contour tracking method is proposed, which can generate dense point correspondences for the cell boundaries in each frame.
Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow
Hanyu Zhou (Huazhong University of Science and Technology), Luxin Yan (Huawei International Co. Ltd.)
Depth EstimationDomain AdaptationAutonomous DrivingOptical FlowImageVideo
🎯 What it does: This paper proposes an unsupervised cumulative domain adaptation framework UCDA-Flow, which utilizes Deep Associative Motion Adaptation (DAMA) and Correlation Aligned Motion Adaptation (CAMA) to achieve optical flow estimation in real foggy scenes.
Unsupervised Deep Asymmetric Stereo Matching With Spatially-Adaptive Self-Similarity
Taeyong Song (Hyundai Motor Company), Kwanghoon Sohn (Yonsei University)
Depth EstimationAutonomous DrivingConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a method for stereo matching of asymmetric views (resolution and noise differences) without the need for annotations, with the core being self-adaptive sampling self-similarity (SASS) features and contrast similarity loss.
Unsupervised Deep Probabilistic Approach for Partial Point Cloud Registration
Guofeng Mei (University of Technology Sydney), Qiang Wu
RecognitionOptimizationTransformerGaussian SplattingPoint Cloud
🎯 What it does: An unsupervised deep probabilistic point cloud registration framework, UDPReg, is proposed, capable of handling partially overlapping point clouds.
Unsupervised Domain Adaption With Pixel-Level Discriminator for Image-Aware Layout Generation
Chenchen Xu (Zhejiang University), Weiwei Xu (Zhejiang University)
GenerationDomain AdaptationTransformerGenerative Adversarial NetworkImage
🎯 What it does: This study investigates the domain gap between the filled images with graphic elements removed and the real product images in the generation of image-aware advertising poster layouts. It proposes an unsupervised domain adaptation GAN based on a pixel-level discriminator (PDA-GAN), which can generate high-quality layouts while preserving image details.
Unsupervised Inference of Signed Distance Functions From Single Sparse Point Clouds Without Learning Priors
Chao Chen (Tsinghua University), Zhizhong Han (Wayne State University)
GenerationData SynthesisOptimizationPoint Cloud
🎯 What it does: An end-to-end network is proposed that does not require signed distance supervision, priors, or normal information, capable of directly inferring the Signed Distance Function (SDF) from a single sparse point cloud.
Unsupervised Intrinsic Image Decomposition With LiDAR Intensity
Shogo Sato (NTT Human Informatics Laboratories), Jun Shimamura (NTT Communication Science Laboratories)
RestorationGenerationAuto EncoderGenerative Adversarial NetworkImagePoint Cloud
🎯 What it does: This paper proposes an unsupervised intrinsic image decomposition method using LiDAR intensity (IID-LI), which decomposes color images into albedo and shade components.
Unsupervised Object Localization: Observing the Background To Discover Objects
Oriane Siméoni (Valeo.ai), Patrick Pérez (Valeo.ai)
Object DetectionSegmentationTransformerContrastive LearningImage
🎯 What it does: By first identifying the image background and utilizing background seeds extracted through self-supervised Transformer attention, high-quality foreground masks are obtained through self-supervised training with a minimal 1×1 convolutional layer on frozen DINO features, thus achieving unsupervised object localization.
Unsupervised Sampling Promoting for Stochastic Human Trajectory Prediction
Guangyi Chen (Mohamed bin Zayed University of Artificial Intelligence), Kun Zhang (Carnegie Mellon University)
GenerationAnomaly DetectionOptimizationGenerative Adversarial NetworkTime SeriesSequential
🎯 What it does: This paper studies an unsupervised sampling enhancement method—BOsampler, aimed at improving the trajectory prediction model of random sampling to better cover low-probability trajectories.
Unsupervised Space-Time Network for Temporally-Consistent Segmentation of Multiple Motions
Etienne Meunier (Inria), Patrick Bouthemy (Inria)
Object TrackingSegmentationConvolutional Neural NetworkOptical FlowVideo
🎯 What it does: This paper proposes an unsupervised 3D U-Net network that utilizes a volume of optical flow over time for multi-motion segmentation, and is trained using a spatiotemporal parameter motion model and temporal consistency regularization.
Unsupervised Visible-Infrared Person Re-Identification via Progressive Graph Matching and Alternate Learning
Zesen Wu (Wuhan University), Mang Ye (Wuhan University)
RecognitionRetrievalGraph Neural NetworkContrastive LearningImageMultimodality
🎯 What it does: A framework for unsupervised visible-infrared person re-identification is proposed, which utilizes progressive graph matching to mine cross-modal correspondences and reduces modal discrepancies through alternating cross-modal contrastive learning.
Unsupervised Volumetric Animation
Aliaksandr Siarohin (Snap Inc), Sergey Tulyakov (Snap Inc)
GenerationPose EstimationNeural Radiance FieldImageVideo
🎯 What it does: This paper proposes an unsupervised volumetric animation method (Unsupervised Volumetric Animation, UVA) that can learn the 3D geometry and keypoint decomposition of objects using only single-view RGB videos, and achieve animation and new view synthesis.
Upcycling Models Under Domain and Category Shift
Sanqing Qu (Tongji University), Changjun Jiang (Tongji University)
Domain AdaptationImage
🎯 What it does: This paper proposes a source unsupervised general domain adaptation method based on global and local clustering (GLC), which can achieve model reuse for domain shift and category shift under the premise of using only a closed-set model pre-trained on the source domain.
Use Your Head: Improving Long-Tail Video Recognition
Toby Perrett (University of Bristol), Dima Damen (University of Bristol)
ClassificationRecognitionTransformerVideoBenchmark
🎯 What it does: Two new long-tail video benchmarks (SSv2-LT and VideoLT-LT) are proposed and evaluated on the naturally collected EPIC-KITCHENS-100, along with a Long-Tail Mixed Reconstruction (LMR) method to enhance recognition performance for classes with few samples.
UTM: A Unified Multiple Object Tracking Model With Identity-Aware Feature Enhancement
Sisi You (Nanjing University of Posts and Telecommunications), Changsheng Xu (Chinese Academy of Sciences)
Object DetectionObject TrackingConvolutional Neural NetworkImageVideo
🎯 What it does: This paper proposes a unified multi-object tracking framework (UTM) that achieves a closed-loop positive feedback for detection, feature embedding, and identity association through the Identity-Aware Feature Enhancement (IAFE) module, significantly improving tracking performance.
UV Volumes for Real-Time Rendering of Editable Free-View Human Performance
Yue Chen (Xi'an Jiaotong University), Fei Wang (Xi'an Jiaotong University)
GenerationPose EstimationConvolutional Neural NetworkNeural Radiance FieldVideo
🎯 What it does: Proposes the UV Volumes framework, which decomposes dynamic human bodies into 3D UV volumes and 2D neural texture stacks, supporting real-time free-viewpoint rendering and editability.
V2V4Real: A Real-World Large-Scale Dataset for Vehicle-to-Vehicle Cooperative Perception
Runsheng Xu (University of California, Los Angeles), Jiaqi Ma (University of California, Los Angeles)
Object DetectionObject TrackingDomain AdaptationAutonomous DrivingMultimodalityPoint CloudBenchmark
🎯 What it does: This paper proposes the V2V4Real dataset and three benchmark tasks to promote research on collaborative perception in vehicular networks.
V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting
Haibao Yu (Tsinghua University), Zaiqing Nie (Baidu Inc.)
Object TrackingAutonomous DrivingTransformerSimultaneous Localization and MappingTime SeriesSequential
🎯 What it does: This paper presents the V2X-Seq large-scale continuous V2X dataset, which includes synchronized perception and trajectory prediction data for vehicles and infrastructure.
Variational Distribution Learning for Unsupervised Text-to-Image Generation
Minsoo Kang (Seoul National University), Bohyung Han (Seoul National University)
GenerationData SynthesisKnowledge DistillationDiffusion modelGenerative Adversarial NetworkContrastive LearningImageTextMultimodality
🎯 What it does: In the absence of image-text paired annotations, this paper proposes to use the CLIP pre-trained model to estimate the text embeddings corresponding to images and to train a text-to-image generation model by maximizing the likelihood in an unsupervised manner through variational inference.
VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalization
Bingfan Zhu (Zhejiang University), Leonidas Guibas (Stanford University)
Depth EstimationKnowledge DistillationNeural Radiance FieldPoint Cloud
🎯 What it does: A perspective-dependent normalization-based NeRF training framework (VDN-NeRF) is proposed, which extracts viewpoint-invariant deep features through self-distillation to suppress shape-radiance ambiguity and improve the quality of geometric reconstruction under non-Lambertian surfaces and dynamic lighting conditions.
VecFontSDF: Learning To Reconstruct and Synthesize High-Quality Vector Fonts via Signed Distance Functions
Zeqing Xia (Wangxuan Institute of Computer Technology, Peking University), Zhouhui Lian (Wangxuan Institute of Computer Technology, Peking University)
GenerationData SynthesisConvolutional Neural NetworkImage
🎯 What it does: A vector font reconstruction and synthesis method based on Signed Distance Functions, called VecFontSDF, is proposed, which can directly represent glyphs as shape primitives composed of parabolic curves and can be converted into quadratic Bezier curves.
Vector Quantization With Self-Attention for Quality-Independent Representation Learning
Zhou Yang (Xidian University), Guangming Shi (Xidian University)
Representation LearningTransformerImage
🎯 What it does: This paper proposes to enhance robustness to low-quality images in visual classification tasks by introducing a vector quantization (VQ) codebook into the model, concatenating it with the original features, and then using a self-attention module to learn quality-invariant feature representations.
VectorFloorSeg: Two-Stream Graph Attention Network for Vectorized Roughcast Floorplan Segmentation
Bingchen Yang (University of Chinese Academy of Sciences), Jun Xiao (University of Chinese Academy of Sciences)
SegmentationGraph Neural NetworkImageGraph
🎯 What it does: This paper addresses the semantic segmentation of rooms in roughly drawn vector floor plans (house floor plans), specifically predicting the spatial area and category of each room.
VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models
Ajay Jain (University of California Berkeley), Pieter Abbeel (University of California Berkeley)
GenerationDiffusion modelImageText
🎯 What it does: An end-to-end text-to-SVG (VectorFusion) generation method is proposed by utilizing a pre-trained pixel-level diffusion model (Stable Diffusion) combined with a differentiable vector graphics renderer (DiffVG); a two-stage baseline based on sampling and automatic vectorization is also provided.
VGFlow: Visibility Guided Flow Network for Human Reposing
Rishabh Jain (Adobe), Balaji Krishnamurthy (Adobe)
Image TranslationGenerationPose EstimationFlow-based ModelGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a visibility-guided flow network named VGFlow for re-posing human images from a source pose, generating highly realistic images with high texture fidelity.
Vid2Avatar: 3D Avatar Reconstruction From Videos in the Wild via Self-Supervised Scene Decomposition
Chen Guo (ETH Zurich), Otmar Hilliges (ETH Zurich)
SegmentationGenerationPose EstimationNeural Radiance FieldVideo
🎯 What it does: Utilizing a self-supervised scene decomposition method to reconstruct detail-rich 3D human avatars from monocular 'wild' videos;
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
Antoine Yang (Google Research), Cordelia Schmid (Inria)
GenerationTransformerVision Language ModelVideoTextAudio
🎯 What it does: The Vid2Seq model is proposed, which completes the temporal localization and description of all events in a video by generating a single sequence containing timestamps and text. During the pre-training phase, it utilizes a large amount of unlabeled narrated videos (transcribed speech + timestamps) for self-supervised learning.
Video Compression With Entropy-Constrained Neural Representations
Carlos Gomes (ETH Zurich), Christopher Schroers (Disney Research Studios)
CompressionConvolutional Neural NetworkVideo
🎯 What it does: This work proposes a fully convolutional spatiotemporal implicit neural network and jointly trains entropy minimization with the Rate-Distortion (R-D) objective for video compression.
Video Dehazing via a Multi-Range Temporal Alignment Network With Physical Prior
Jiaqi Xu (Chinese University of Hong Kong), Pheng-Ann Heng (Chinese University of Hong Kong)
RestorationTransformerOptical FlowVideo
🎯 What it does: This paper proposes a MAP-Net based on multi-range spatiotemporal alignment for restoring fog-free frames in videos.
Video Event Restoration Based on Keyframes for Video Anomaly Detection
Zhiwei Yang (Xidian University), Xiaotao Liu (Xidian University)
RestorationAnomaly DetectionTransformerVideo
🎯 What it does: This paper proposes a keyframe-based video event recovery paradigm for unsupervised video anomaly detection.
Video Probabilistic Diffusion Models in Projected Latent Space
Sihyun Yu (KAIST), Jinwoo Shin (KAIST)
GenerationData SynthesisDiffusion modelVideo
🎯 What it does: A PVDM method for diffusion modeling after projecting 3D videos into a 2D latent space is proposed, which can efficiently generate high-resolution, long-sequence videos.
Video Test-Time Adaptation for Action Recognition
Wei Lin (Graz University of Technology), Horst Bischof (Graz University of Technology)
Domain AdaptationVideo
🎯 What it does: This paper proposes an online video test-time adaptation method called ViTTA, aimed at improving action recognition performance on test videos with distribution shifts.
Video-Text As Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning
Peng Jin (Peking University), Jie Chen (Peking University)
RetrievalRepresentation LearningTransformerContrastive LearningVideoTextMultimodality
🎯 What it does: A hierarchical Banzhaf Interaction method based on multivariable cooperative game theory is proposed for fine-grained video-text cross-modal representation learning.
VideoMAE V2: Scaling Video Masked Autoencoders With Dual Masking
Limin Wang (Nanjing University), Yu Qiao (Shenzhen Institute of Advanced Technology)
ClassificationRecognitionTransformerAuto EncoderVideo
🎯 What it does: This paper presents VideoMAE V2, which utilizes dual masking (encoder and decoder dual masking) for large-scale pre-training of video masked autoencoders, achieving the first video Transformer model with over ten billion parameters.
VideoTrack: Learning To Track Objects via Video Transformer
Fei Xie (Shanghai Jiao Tong University), Chao Ma (Shanghai Jiao Tong University)
Object TrackingTransformerVideo
🎯 What it does: This paper proposes a video Transformer framework called VideoTrack, which directly performs target matching on video segments in the spatiotemporal domain, achieving end-to-end visual tracking.
ViewNet: A Novel Projection-Based Backbone With View Pooling for Few-Shot Point Cloud Classification
Jiajing Chen (Syracuse University), Senem Velipasalar (Syracuse University)
ClassificationConvolutional Neural NetworkPoint Cloud
🎯 What it does: This paper proposes a projection-based backbone network called ViewNet for few-shot classification tasks on 3D point clouds.
Viewpoint Equivariance for Multi-View 3D Object Detection
Dian Chen (Toyota Research Institute), Adrien Gaidon (Toyota Research Institute)
Object DetectionAutonomous DrivingTransformerPoint Cloud
🎯 What it does: A Transformer-based multi-view 3D object detection framework VEDet is proposed, which utilizes viewpoint equivariance to learn the consistency of objects across different camera perspectives, thereby improving 3D localization accuracy.
VILA: Learning Image Aesthetics From User Comments With Vision-Language Pretraining
Junjie Ke (Google Research), Feng Yang (Google Research)
ClassificationRetrievalRecommendation SystemTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Using image-user comment pairs for visual-language pre-training to obtain aesthetic representations, and completing image aesthetic assessment with a lightweight ranking adapter with only a small number of parameters.
ViLEM: Visual-Language Error Modeling for Image-Text Retrieval
Yuxin Chen (Chinese Academy of Sciences), Jianping Wu (Tsinghua University)
RetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Enhancing fine-grained semantic association in image-text retrieval through the Visual Language Error Modeling (ViLEM) task and a multi-granularity interaction framework within a dual-encoder structure.
VindLU: A Recipe for Effective Video-and-Language Pretraining
Feng Cheng (University of North Carolina), Gedas Bertasius (University of North Carolina)
RetrievalTransformerVision Language ModelVideoTextMultimodality
🎯 What it does: Systematically study and propose a step-by-step recipe for transitioning from image to video-language pre-training, called VINDLU;
ViP3D: End-to-End Visual Trajectory Prediction via 3D Agent Queries
Junru Gu (Tsinghua University), Hang Zhao (Tsinghua University)
Object DetectionObject TrackingAutonomous DrivingTransformerSimultaneous Localization and MappingVideo
🎯 What it does: We propose ViP3D, an end-to-end differentiable visual trajectory prediction framework that directly predicts the future trajectories of vehicles and pedestrians from multi-view videos using 3D agent queries.
ViPLO: Vision Transformer Based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detection
Jeeseung Park (mAy-I Inc), Jong-Seok Lee (Yonsei University)
Object DetectionPose EstimationGraph Neural NetworkTransformerImage
🎯 What it does: A pose-conditioned self-loop graph based on Vision Transformer (ViPLO) is proposed for human-object interaction detection, and an MOA module is designed to achieve quantization-free feature extraction.
Virtual Occlusions Through Implicit Depth
Jamie Watson (Niantic), Michael Firman (Niantic)
SegmentationDepth EstimationConvolutional Neural NetworkOptical FlowImage
🎯 What it does: This paper proposes a method for directly predicting the occlusion mask of virtual objects in real scenes, avoiding traditional pixel-wise depth regression, thus achieving more accurate and temporally consistent AR synthesis.
Virtual Sparse Convolution for Multimodal 3D Object Detection
Hai Wu (Xiamen University), Cheng Wang (Texas A&M University)
Object DetectionAutonomous DrivingConvolutional Neural NetworkMultimodalityPoint Cloud
🎯 What it does: A novel sparse convolution (VirConv) is proposed to integrate virtual points obtained through depth completion and LiDAR points, achieving efficient multi-modal 3D object detection.
VisFusion: Visibility-Aware Online 3D Scene Reconstruction From Videos
Huiyu Gao (Australian National University), Miaomiao Liu (Australian National University)
Convolutional Neural NetworkRecurrent Neural NetworkSimultaneous Localization and MappingVideo
🎯 What it does: This paper proposes an online, visually perceptive 3D scene reconstruction method called VisFusion, which can generate detailed and dense 3D models in real-time from calibrated monocular video.
Visibility Aware Human-Object Interaction Tracking From Single RGB Camera
Xianghui Xie (Max Planck Institute for Informatics), Gerard Pons-Moll (University of Tübingen)
Object TrackingPose EstimationTransformerVideo
🎯 What it does: Real-time tracking of humans, movable objects, and their contact points from a single camera RGB video has been achieved.
Visibility Constrained Wide-Band Illumination Spectrum Design for Seeing-in-the-Dark
Muyao Niu (University of Tokyo), Yinqiang Zheng (University of Tokyo)
RestorationOptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a visual constraint broadband spectral design method to optimize the mixed lighting of visible and near-infrared LEDs, aiming to enhance RGB reconstruction effects in completely dark environments.
Vision Transformers Are Good Mask Auto-Labelers
Shiyi Lan (NVIDIA), Anima Anandkumar (NVIDIA)
Object DetectionSegmentationTransformerImage
🎯 What it does: A two-stage framework and Mask Auto-Labeler are proposed to automatically generate high-quality masks under box supervision for instance segmentation training.
Vision Transformers Are Parameter-Efficient Audio-Visual Learners
Yan-Bo Lin (University of North Carolina), Gedas Bertasius (University of North Carolina)
Computational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningMultimodalityAudio
🎯 What it does: This paper proposes an adapter called LAVISH, which enables a Vision Transformer pre-trained solely on visual data to perform audio-visual tasks using a small number of trainable parameters without updating the original parameters.
Visual Atoms: Pre-Training Vision Transformers With Sinusoidal Waves
Sora Takashima (National Institute of Advanced Industrial Science and Technology), Rio Yokota (National Institute of Advanced Industrial Science and Technology)
GenerationData SynthesisTransformerImage
🎯 What it does: A visual atomic renderer is proposed, which generates controllable contour synthetic images using a combination of sine waves for formula-driven supervised learning pre-training of visual Transformers.
Visual Dependency Transformers: Dependency Tree Emerges From Reversed Attention
Mingyu Ding (University of Hong Kong), Chuang Gan (MIT)
ClassificationObject DetectionSegmentationTransformerImageVideo
🎯 What it does: This paper proposes a visually dependent Transformer (DependencyViT) that achieves unsupervised construction of a hierarchical dependency tree of image patches through reverse attention, and designs a lightweight version (DependencyViT-Lite) to implement dynamic visual pooling.
Visual DNA: Representing and Comparing Images Using Distributions of Neuron Activations
Benjamin Ramtoula (Mobile Robotics Group, University of Oxford), Daniele De Martini (Mobile Robotics Group, University of Oxford)
RetrievalDomain AdaptationRepresentation LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: A general dataset/image representation method based on the distribution of neuron activations from pre-trained neural networks (DNA) is proposed;
Visual Exemplar Driven Task-Prompting for Unified Perception in Autonomous Driving
Xiwen Liang (Shenzhen Campus of Sun Yat-sen University), Xiaodan Liang (Huawei Noah's Ark Lab)
Object DetectionSegmentationAutonomous DrivingConvolutional Neural NetworkTransformerPrompt EngineeringImage
🎯 What it does: A visual example-driven multi-task prompting framework VE-Prompt is proposed for unified perception tasks (object detection, semantic segmentation, drivable area segmentation, and lane detection).
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images
Ming Y. Lu (Massachusetts Institute of Technology), Faisal Mahmood (Harvard University)
ClassificationTransformerPrompt EngineeringContrastive LearningImageTextBiomedical Data
🎯 What it does: Proposes the MI-Zero framework, which utilizes contrastive vision-language pre-trained models to achieve zero-shot transfer for large-sized whole slide images (WSI) through multi-instance learning, completing cancer subtype classification.
Visual Localization Using Imperfect 3D Models From the Internet
Vojtech Panek (Czech Technical University in Prague), Torsten Sattler (Czech Technical University in Prague)
Pose EstimationRetrievalSimultaneous Localization and MappingMeshBenchmark
🎯 What it does: This paper explores and experiments with using readily available, often incomplete 3D models from the internet as scene representations for visual localization, and constructs a new benchmark dataset.
Visual Programming: Compositional Visual Reasoning Without Training
Tanmay Gupta (Allen Institute for Artificial Intelligence), Aniruddha Kembhavi (Allen Institute for Artificial Intelligence)
Object DetectionExplainability and InterpretabilityTransformerLarge Language ModelVision Language ModelDiffusion modelImageText
🎯 What it does: An untrained interpretable neural symbolic visual programming system, VISPROG, has been constructed, which generates executable visual programs with the guidance of a few examples using large language models and automatically calls various visual and image processing modules to accomplish complex visual tasks.
Visual Prompt Multi-Modal Tracking
Jiawen Zhu (Dalian University of Technology), Huchuan Lu (Dalian University of Technology)
Object TrackingTransformerPrompt EngineeringMultimodality
🎯 What it does: A multi-modal tracking framework called ViPT based on visual prompts is proposed, which utilizes a small number of learnable prompts to achieve RGB + depth, thermal imaging, event, and other multi-modal tracking on a frozen RGB pre-trained model.
Visual Prompt Tuning for Generative Transfer Learning
Kihyuk Sohn (Google Research), Lu Jiang (Google Research)
GenerationDomain AdaptationTransformerPrompt EngineeringImage
🎯 What it does: By performing prompt tuning on a generative visual Transformer pre-trained on a large-scale dataset, we achieve image synthesis transfer learning across various visual domains (natural, structured, specialized, few-shot, etc.).
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning
Cheng-Hao Tu (Ohio State University), Wei-Lun Chao (Ohio State University)
ClassificationRepresentation LearningTransformerImageMultimodality
🎯 What it does: This paper proposes Visual Query Tuning (VQT), which inserts learnable query tokens before each layer of the Vision Transformer. It uses only these tokens as queries to aggregate intermediate features while keeping the original features unchanged, achieving linear probing with a frozen backbone.
Visual Recognition by Request
Chufeng Tang (Tsinghua University), Qi Tian (Huawei)
RecognitionSegmentationTransformerImage
🎯 What it does: A Visual Recognition on Request (ViRReq) framework is proposed, which breaks down the image segmentation task into a series of 'requests' and achieves progressive segmentation through a knowledge base and text embeddings, supporting incomplete annotations, incremental learning, and open-domain recognition.
Visual Recognition-Driven Image Restoration for Multiple Degradation With Intrinsic Semantics Recovery
Zizheng Yang (University of Science and Technology of China), Feng Zhao (University of Science and Technology of China)
ClassificationRestorationObject DetectionConvolutional Neural NetworkImage
🎯 What it does: A visual recognition-driven multi-degradation image restoration framework VRD-IR is proposed, capable of restoring recognition-friendly high-quality images under unknown degradation conditions.
Visual-Language Prompt Tuning With Knowledge-Guided Context Optimization
Hantao Yao (Institute of Automation, Chinese Academy of Sciences), Changsheng Xu (Institute of Automation, Chinese Academy of Sciences)
ClassificationRecognitionDomain AdaptationOptimizationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: A prompt tuning method named Knowledge-guided Context Optimization (KgCoOp) is proposed to enhance the generalization performance of pre-trained vision-language models (such as CLIP) in downstream tasks, particularly in recognizing unseen categories.
Visual-Tactile Sensing for In-Hand Object Reconstruction
Wenqiang Xu (Shanghai Jiao Tong University), Cewu Lu (Shanghai Jiao Tong University)
Object DetectionSegmentationData SynthesisPose EstimationConvolutional Neural NetworkSimultaneous Localization and MappingMultimodalityPoint Cloud
🎯 What it does: A framework for hand-held object reconstruction based on visual and tactile fusion, VTacO, and its hand-object version, VTacOH, has been proposed, capable of reconstructing both rigid and deformable objects and supporting incremental refinement.
Vita-CLIP: Video and Text Adaptive CLIP via Multimodal Prompting
Syed Talal Wasim (Mohamed bin Zayed University of Artificial Intelligence), Mubarak Shah (University of Central Florida)
ClassificationRecognitionRetrievalTransformerPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: A multi-modal prompt learning scheme called Vita-CLIP is proposed, which utilizes a frozen CLIP pre-trained model for efficient adaptation to video tasks, balancing supervised learning and zero-shot generalization.
ViTs for SITS: Vision Transformers for Satellite Image Time Series
Michail Tarasiou (Imperial College London), Stefanos Zafeiriou (Imperial College London)
ClassificationSegmentationTransformerImageTime Series
🎯 What it does: A spatiotemporal decomposed Vision Transformer (TSViT) is proposed, specifically designed for semantic segmentation and classification tasks of satellite image time series.
VIVE3D: Viewpoint-Independent Video Editing Using 3D-Aware GANs
Anna Frühstück (Meta Reality Labs Research), Tony Tung (Meta Reality Labs Research)
GenerationData SynthesisGenerative Adversarial NetworkOptical FlowVideo
🎯 What it does: A personalized generator based on 3D-aware GAN is constructed, using a small number of images from the video for joint inversion, and on this basis, face attribute editing and viewpoint transformation in the video are achieved, ultimately seamlessly compositing the edited face back into the original video.
VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloud
Ziqin Wang (Beihang University), Lu Sheng (Beihang University)
Object DetectionSegmentationGraph Neural NetworkContrastive LearningMultimodalityPoint Cloud
🎯 What it does: This paper proposes a Visual-Language Semantic Assisted Training (VL-SAT) framework to enhance 3D semantic scene graph prediction in point clouds.
VLPD: Context-Aware Pedestrian Detection via Vision-Language Semantic Self-Supervision
Mengyin Liu (University of Science and Technology Beijing), Xu-Cheng Yin (University of Science and Technology Beijing)
Object DetectionVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Achieving explicit modeling of semantic context through a visual-language self-supervised method to enhance pedestrian detection performance.