CVPR 2024 Papers — Page 18
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
NViST: In the Wild New View Synthesis from a Single Image with Transformers
Wonbong Jang (University College London), Lourdes Agapito (University College London)
GenerationData SynthesisTransformerNeural Radiance FieldImageVideo
🎯 What it does: A new view synthesis model NViST based on Transformer is proposed, which can generate images from arbitrary viewpoints from a single wild scene image.
OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentation
Bohao Peng (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
SegmentationAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: This paper proposes OA-CNNs, a 3D semantic segmentation network based on sparse convolution.
OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion
Xinyu Zhan (Shanghai Jiao Tong University), Cewu Lu (Shanghai Jiao Tong University)
GenerationPose EstimationRobotic IntelligenceTransformerLarge Language ModelDiffusion modelVideoMultimodality
🎯 What it does: The OAKINK2 dataset is proposed, which includes complex task demonstrations of two-handed object manipulation and a three-layer abstraction (Affordance, Primitive, Complex Task). Based on this, an LLM task decomposition and diffusion model for hand movement generation is developed.
Object Dynamics Modeling with Hierarchical Point Cloud-based Representations
Chanho Kim (Oregon State University), Li Fuxin (Oregon State University)
Object DetectionRobotic IntelligenceConvolutional Neural NetworkSupervised Fine-TuningPoint CloudMeshPhysics Related
🎯 What it does: This paper proposes a U-Net structure based on continuous point convolution for learning the dynamic evolution of multiple objects in three-dimensional space, capable of directly training and predicting on dense point clouds or sparse mesh vertices.
Object Pose Estimation via the Aggregation of Diffusion Features
Tianfu Wang (Chinese Academy of Sciences), Hongguang Wang (Chinese Academy of Sciences)
Pose EstimationDiffusion modelContrastive LearningImage
🎯 What it does: In this work, the authors utilize the intermediate features of the Stable Diffusion model and design three types of aggregation networks to achieve template matching-based object pose estimation, significantly improving the estimation accuracy for unseen objects.
Object Recognition as Next Token Prediction
Kaiyu Yue (University of Maryland), Ser-Nam Lim (University of Central Florida)
RecognitionObject DetectionTransformerLarge Language ModelVision Language ModelImageText
🎯 What it does: By embedding images and feeding them to a pre-trained language decoder, the task of object recognition is transformed into a next-word prediction task, directly generating object labels using an autoregressive approach.
Objects as Volumes: A Stochastic Geometry View of Opaque Solids
Bailey Miller (Carnegie Mellon University), Ioannis Gkioulekas (Carnegie Mellon University)
GenerationData SynthesisNeural Radiance FieldPoint Cloud
🎯 What it does: This study presents a rigorous derivation of volume representation for opaque solids based on the theory of stochastic geometry, providing an analytical relationship between the attenuation coefficient, occupancy, and normal distribution, and embedding it into the neural rendering pipeline.
Observation-Guided Diffusion Probabilistic Models
Junoh Kang (Seoul National University), Bohyung Han (Seoul National University)
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: A training framework called OGDM is proposed, which incorporates observations into diffusion models to correct the reverse process, allowing for high-quality generation results even with few sampling steps.
OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation
Jisoo Jeong (Qualcomm AI Research), Fatih Porikli (Qualcomm AI Research)
Image TranslationOptical FlowImageVideo
🎯 What it does: This paper proposes a forward interpolation method called OCAI that utilizes occlusion and consistency perception, combined with a teacher-student semi-supervised learning framework, to automatically generate intermediate frames and optical flow, addressing the issue of scarce optical flow training data.
ODCR: Orthogonal Decoupling Contrastive Regularization for Unpaired Image Dehazing
Zhongze Wang (East China University of Science and Technology), Kaijie Zhao (East China University of Science and Technology)
Image TranslationRestorationGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: For the task of dehazing unpaired images, an Orthogonal Decoupled Contrastive Regularization (ODCR) method is proposed, which utilizes an orthogonal MLP to achieve the decoupling of fog-related and fog-independent features on the Stiefel manifold. A self-supervised deep feature classifier is used to assign fog-related weights to each channel, and then a weighted PatchNCE is employed to maximize mutual information across different feature spaces, resulting in clearer dehazing outcomes that preserve texture and semantic information.
ODIN: A Single Model for 2D and 3D Segmentation
Ayush Jain (Carnegie Mellon University), Katerina Fragkiadaki (Microsoft)
Object DetectionSegmentationTransformerImagePoint Cloud
🎯 What it does: A unified model named ODIN is proposed, capable of achieving 2D and 3D instance segmentation and semantic segmentation on single RGB images and multi-view RGB-D data. By alternately using 2D internal view fusion and 3D view cross-fusion in the model, it can leverage 2D pre-trained weights while obtaining global consistency in 3D.
ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting
Chen Duan (Meituan), Xiaoming Wei (Meituan)
RecognitionObject DetectionConvolutional Neural NetworkTransformerContrastive LearningImageText
🎯 What it does: A pre-training method based on OCR text destylization (ODM) is proposed, which can unify diverse image texts into a consistent style and achieve precise alignment of text and image features.
OED: Towards One-stage End-to-End Dynamic Scene Graph Generation
Guan Wang (Peking University), Yang Liu (Peking University)
Object DetectionGenerationTransformerVideo
🎯 What it does: This paper proposes a one-stage end-to-end dynamic scene graph generation framework called OED, which directly performs set prediction on video frame pairs to generate subject-object pairs and their relationship triples.
OHTA: One-shot Hand Avatar via Data-driven Implicit Priors
Xiaozheng Zheng (PICO ByteDance), Zhou Xue (PICO ByteDance)
GenerationPose EstimationNeural Radiance FieldImage
🎯 What it does: This paper proposes a data-driven implicit prior-based single-image hand avatar generation framework called OHTA, which enables the rapid construction of high-quality, animatable hand avatars from a single image.
OMG-Seg: Is One Model Good Enough For All Segmentation?
Xiangtai Li (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)
SegmentationTransformerImageVideo
🎯 What it does: A unified segmentation model, OMG-Seg, is proposed, capable of simultaneously completing ten types of segmentation tasks (semantic, instance, panoptic, video semantic/instance/object, open vocabulary, interactive segmentation, etc.) within the same Transformer encoder-decoder framework.
OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers
Han Liang (ShanghaiTech University), Lan Xu (ShanghaiTech University)
GenerationData SynthesisPose EstimationTransformerMixture of ExpertsDiffusion modelVideoText
🎯 What it does: This paper studies an open-source vocabulary text-driven action generation framework OMG based on pre-training and fine-tuning, which can generate realistic and diverse human actions from any text prompt in a zero-shot setting.
Omni-Q: Omni-Directional Scene Understanding for Unsupervised Visual Grounding
Sai Wang (Wuhan University), Yu Wu (Wuhan University)
Object DetectionDepth EstimationRetrievalTransformerImage
🎯 What it does: An unsupervised visual localization framework named Omni-Q is proposed, utilizing three major modules (object perception, 3D spatial relationships, spatial graph) to achieve high-quality query generation.
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
Jialin Wu (Google Research), Radu Soricut (Google Research)
RecognitionGenerationRetrievalTransformerMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: A soft mixture of low-rank experts architecture for multimodal models (Omni-SMoLA) has been constructed to enhance the multitasking performance of general large-scale vision-language models.
OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
Hanwen Jiang (University of Texas at Austin), André Araujo (Google Research)
Pose EstimationGraph Neural NetworkContrastive LearningImage
🎯 What it does: We propose OmniGlue, a generalizable and learnable image matcher that maintains high-precision matching on unseen image domains.
OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videos
Dongyoung Choi (KAIST), Min H. Kim (KAIST)
RestorationData SynthesisPose EstimationOptimizationNeural Radiance FieldVideo
🎯 What it does: This paper proposes OmniLocalRF, which is used to remove dynamic objects from 360° videos containing dynamic objects and synthesize a panoramic view of static scenes.
OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
Yutao Hu (University of Hong Kong), Ping Luo (Shanghai AI Laboratory)
TransformerLarge Language ModelVision Language ModelImageMultimodalityBiomedical DataMagnetic Resonance ImagingComputed TomographyPositron Emission TomographyBenchmark
🎯 What it does: A large-scale medical visual question answering benchmark, OmniMedVQA, has been constructed, covering 118,010 real medical images and 127,995 QA pairs, including 12 imaging modalities and over 20 anatomical regions.
OmniMotionGPT: Animal Motion Generation with Limited Data
Zhangsihao Yang (Arizona State University), Yalin Wang (Arizona State University)
GenerationData SynthesisRobotic IntelligenceTransformerAuto EncoderTextMultimodality
🎯 What it does: A text-based animal motion generation framework called OmniMotionGPT is proposed, which utilizes human motion knowledge transfer to the animal domain and generates diverse and realistic animal movements.
OmniParser: A Unified Framework for Text Spotting Key Information Extraction and Table Recognition
Jianqiang Wan (Alibaba Group), Zhibo Yang (Huazhong University of Science and Technology)
RecognitionObject DetectionTransformerImageTextTabular
🎯 What it does: This paper proposes OMNIPARSER, a unified visual text parsing framework that can perform text localization and recognition, key information extraction, and table recognition within the same model.
OmniSDF: Scene Reconstruction using Omnidirectional Signed Distance Functions and Adaptive Binoctrees
Hakyeong Kim (KAIST), Min H. Kim (KAIST)
Depth EstimationOptimizationNeural Radiance FieldVideoPoint Cloud
🎯 What it does: This paper proposes a memory-efficient neural 3D reconstruction method suitable for short-distance self-centered panoramic video, utilizing signed distance fields (SDF) and an adaptive spherical binary octree (binoctree) to dynamically refine the sampling space during the optimization process.
OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learning
Haiyang Ying (Tsinghua University), Lu Fang (Tsinghua University)
SegmentationNeural Radiance FieldContrastive LearningImage
🎯 What it does: We propose OmniSeg3D, a method that enhances multi-view inconsistent 2D segmentation into a consistent 3D feature field through hierarchical contrastive learning, enabling panoramic, category-free, and hierarchical 3D segmentation and interaction.
OmniVec2 - A Novel Transformer based Network for Large Scale Multimodal and Multitask Learning
Siddharth Srivastava (Typeface), Gaurav Sharma (Typeface)
ClassificationRecognitionRetrievalTransformerAuto EncoderImageVideoTextMultimodalityPoint CloudTabularTime SeriesAudio
🎯 What it does: This paper presents OmniVec2, a transformer-based multimodal multitask network that utilizes modality-specific tokenizers, a shared transformer core, and task-specific heads to achieve cross-modal knowledge sharing during three-phase training (single-modal masked pre-training, two-modal masked pre-training, and supervised training for task pairs).
OmniViD: A Generative Framework for Universal Video Understanding
Junke Wang (Fudan University), Yu-Gang Jiang (Fudan University)
Object TrackingTransformerSupervised Fine-TuningVideo
🎯 What it does: We propose OmniViD, a unified generative framework that integrates various video tasks such as action recognition, video captioning, question answering, event description, and object tracking into a token generation based on words, time, and boxes.
On Exact Inversion of DPM-Solvers
Seongmin Hong (Seoul National University), Se Young Chun (Seoul National University)
RestorationGenerationDiffusion modelImageOrdinary Differential Equation
🎯 What it does: An **exact inverse method** for DDIM (first-order DPM-solver) and higher-order DPM-solvers (such as DPM-Solver++) is proposed, enabling the reverse mapping from generated images back to the initial noise;
On Scaling Up a Multilingual Vision and Language Model
Xi Chen (Google), Radu Soricut (Google)
RecognitionObject DetectionGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageVideoTextMultimodality
🎯 What it does: Constructed and trained PaLI-X, a large-scale multilingual vision-language model, further scaling up the visual encoder and language decoder, and conducting large-scale pre-training on multi-task mixed objectives, followed by fine-tuning or direct evaluation on various vision-language tasks.
On the Content Bias in Frechet Video Distance
Songwei Ge (University of Maryland), Jia-Bin Huang (University of Maryland)
GenerationData SynthesisVideo
🎯 What it does: The system studied the content bias of the Fréchet Video Distance (FVD) evaluation metric, quantified its sensitivity to frame quality, explored the sources of bias, and demonstrated that using self-supervised video features (VideoMAE-v2) can significantly reduce this bias; the issue was also validated in long video generation cases.
On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm
Peng Sun (Zhejiang University), Tao Lin (Westlake University)
Data SynthesisComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: An efficient dataset distillation method called RDED is proposed, which is non-optimized and based on patch stitching, capable of generating synthetic datasets usable for various networks with 10/50 samples per class on large-scale high-resolution datasets like ImageNet-1K.
On the Estimation of Image-matching Uncertainty in Visual Place Recognition
Mubariz Zaffar (Delft University of Technology), Julian F. P. Kooij (Delft University of Technology)
RecognitionRetrievalSimultaneous Localization and MappingImage
🎯 What it does: This paper studies the estimation of image matching uncertainty in Visual Place Recognition (VPR) and proposes a SUE baseline that utilizes spatial location information of reference images, comparing it comprehensively with three existing methods.
On the Faithfulness of Vision Transformer Explanations
Junyi Wu (Illinois Institute of Technology), Yan Yan (Illinois Institute of Technology)
Explainability and InterpretabilityTransformerImage
🎯 What it does: The SaCo metric is proposed to evaluate the credibility of Vision Transformer explanation methods;
On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving
Kaituo Feng (Beijing Institute of Technology), Guoren Wang (Beijing Institute of Technology)
CompressionAutonomous DrivingComputational EfficiencyKnowledge DistillationReinforcement LearningVideo
🎯 What it does: This paper proposes a knowledge distillation framework named PlanKD for compressing end-to-end motion planning models, allowing them to maintain high safety and performance in resource-constrained environments.
On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation
Agneet Chatterjee (Arizona State University), Yezhou Yang (Arizona State University)
Depth EstimationVision Language ModelDiffusion modelImage
🎯 What it does: This study systematically evaluates the robustness, generalization, and bias of natural language prompts in monocular depth estimation, exploring the impact of different sentence types on model performance.
On the Robustness of Large Multimodal Models Against Image Adversarial Attacks
Xuanming Cui (University of Central Florida), Ser-Nam Lim (University of Central Florida)
RetrievalAdversarial AttackTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This study investigates the robustness of large multimodal models (LMM) against visual adversarial attacks.
On the Scalability of Diffusion-based Text-to-Image Generation
Hao Li (Amazon Web Services Artificial Intelligence Labs), Stefano Soatto (Amazon Web Services Artificial Intelligence Labs)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: Systematically evaluate scaling strategies for diffusion text-image models under a unified training setup, comparing the effects of UNet and Transformer denoising backbones and dataset expansion.
On the Test-Time Zero-Shot Generalization of Vision-Language Models: Do We Really Need Prompt Learning?
Maxime Zanella (UCLouvain UMons), Ismail Ben Ayed (TS Montréal E)
ClassificationRecognitionOptimizationComputational EfficiencyTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: A robust test-time augmentation method MTA based on MeanShift is proposed, which clusters multi-view data directly in the final feature space of CLIP without the need for gradient training.
On Train-Test Class Overlap and Detection for Image Retrieval
Chull Hwan Song (Dealicious Inc), Yannis Avrithis (Institute of Advanced Research on Artificial Intelligence)
RetrievalConvolutional Neural NetworkImage
🎯 What it does: This paper first reconstructs the Google Landmarks v2-clean dataset by removing category overlaps with the Oxford/Paris evaluation sets, resulting in R GLDv2-clean; then it proposes a single-stage, end-to-end detection-retrieval framework called CiDeR, which utilizes unsupervised attention to locate targets and directly generate global features;
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression
Hancheng Ye (Fudan University), Bo Zhang (Shanghai Artificial Intelligence Laboratory)
CompressionTransformerImage
🎯 What it does: This paper proposes a one-time method for simultaneously learning importance and sparsity scores for Vision Transformer compression (OFB), achieving single-stage search through dual masking (bi-mask) and adaptive one-hot loss; it also introduces Progressive Masked Image Modeling (PMIM) to enhance feature representation during the search, resulting in significant compression effects and extremely low search costs.
One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls
Minghui Hu (Nanyang Technological University), Tat-Jen Cham (Nanyang Technological University)
GenerationDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A plugin module named One More Step (OMS) is proposed, which, without modifying the parameters of the pre-trained diffusion model, adds an additional step before sampling to map standard Gaussian noise to noise that matches the training phase, thereby correcting the intermediate brightness bias caused by the inconsistency of the terminating noise distribution during training and inference.
One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models
Lin Li (King's College London), Michael Spratling (Imperial College London)
ClassificationAdversarial AttackTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: Proposes Adversarial Prompt Tuning (APT), which enhances the adversarial robustness of large-scale vision-language models (such as CLIP) by learning adjustable text context vectors.
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
Minghua Liu, Hao Su
GenerationData SynthesisDiffusion modelImageMesh
🎯 What it does: An end-to-end pipeline is proposed that quickly converts a single RGB image into a high-quality, textured 3D mesh, with a generation time of about one minute.
One-Class Face Anti-spoofing via Spoof Cue Map-Guided Feature Learning
Pei-Kai Huang (National Tsing Hua University), Chiou-Ting Hsu (Academia Sinica)
ClassificationRecognitionGenerative Adversarial NetworkImage
🎯 What it does: A single-class face anti-spoofing framework OC-SCMNet is proposed, which distinguishes whether a facial image is from a live subject by learning zero-forgery clue maps (SCM).
One-dimensional Adapter to Rule Them All: Concepts Diffusion Models and Erasing Applications
Mengyao Lyu (Tsinghua University), Guiguang Ding (Tsinghua University)
GenerationData SynthesisComputational EfficiencyDiffusion modelImage
🎯 What it does: This work proposes a one-dimensional lightweight adapter (SPM) for accurately eliminating multiple concepts in diffusion models while maintaining the model's other generative capabilities.
One-Prompt to Segment All Medical Images
Junde Wu (National University of Singapore), Min Xu (Carnegie Mellon University)
SegmentationConvolutional Neural NetworkTransformerPrompt EngineeringImageBiomedical Data
🎯 What it does: A One-Prompt medical image segmentation framework is proposed, which can achieve general medical segmentation by completing a forward inference on unseen tasks using only one sample with a prompt.
One-Shot Open Affordance Learning with Foundation Models
Gen Li (University of Edinburgh), Varun Jampani (Stability AI)
Object DetectionSegmentationTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: A one-shot open object affordance learning framework (OOAL) is proposed, which can train the model with only one image per base class and achieve zero-shot inference for unseen objects and unseen affordance categories during the inference phase.
One-Shot Structure-Aware Stylized Image Synthesis
Hansam Cho (NAVER Cloud), Yonghyun Jeong (NAVER Cloud)
Image TranslationGenerationData SynthesisDiffusion modelAuto EncoderImage
🎯 What it does: A single image stylization method based on diffusion models, OSASIS, has been developed, which can transfer the style of any reference image to the input image while maintaining its structure.
One-step Diffusion with Distribution Matching Distillation
Tianwei Yin (Massachusetts Institute of Technology), Taesung Park (Massachusetts Institute of Technology)
GenerationKnowledge DistillationDiffusion modelScore-based ModelImage
🎯 What it does: Distilling a multi-step diffusion model into a first-order generator, the resulting DMD can generate high-quality images comparable to the original multi-step diffusion with a single forward pass;
OneFormer3D: One Transformer for Unified Point Cloud Segmentation
Maxim Kolodiazhnyi (Samsung Research), Danila Rukhovich (Samsung Research)
SegmentationTransformerPoint Cloud
🎯 What it does: This paper presents OneFormer3D, a unified Transformer-based model for semantic, instance, and panoptic segmentation of 3D point clouds.
OneLLM: One Framework to Align All Modalities with Language
Jiaming Han (Chinese University of Hong Kong), Xiangyu Yue (Shanghai Artificial Intelligence Laboratory)
TransformerLarge Language ModelMixture of ExpertsImageVideoMultimodalityPoint CloudBiomedical DataMagnetic Resonance ImagingAudio
🎯 What it does: This paper designs and trains a unified framework called OneLLM, which can align eight different modalities (images, audio, video, point clouds, depth/normal maps, IMU, fMRI, etc.) with language and supports multi-modal instruction following.
OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning
Lingyi Hong (Fudan University), Wenqiang Zhang (Fudan University)
Object TrackingTransformerPrompt EngineeringImageVideoMultimodality
🎯 What it does: A unified OneTracker framework is proposed, which first pre-trains a foundation tracker on large-scale RGB tracking data, and then achieves parameter-efficient fine-tuning through a Cross-Modal Prompter (CMT Prompter) and Task-Aware Transformer (TTP) to adapt to RGB+X (RGB+N, RGB+M, RGB+D/T/E) multimodal tracking tasks.
Online Task-Free Continual Generative and Discriminative Learning via Dynamic Cluster Memory
Fei Ye (University of York), Adrian G. Bors (University of York)
ClassificationGenerationDiffusion modelAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a dynamic cluster memory management framework (DCM) for online continual learning without task boundaries, managing sample storage without the need for label information.
OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising
Haichao Zhang (Northeastern University), Yun Fu (Northeastern University)
RestorationObject TrackingAutonomous DrivingTransformerSimultaneous Localization and MappingMultimodality
🎯 What it does: A new task is proposed to predict noise-free visual trajectories from completely non-visual observations of noisy sensor trajectories (OOSTraj).
Open Vocabulary Semantic Scene Sketch Understanding
Ahmed Bourouis (University of Surrey), Yulia Gryaditskaya (University of Surrey)
SegmentationTransformerContrastive LearningImage
🎯 What it does: This paper constructs a Sketch Encoder based on the CLIP pre-trained ViT, employing a dual-layer training approach (global scene understanding + category-level refinement) to achieve semantic segmentation of freehand scene sketches, using only sketch-title pairs for weakly supervised training.
Open-Set Domain Adaptation for Semantic Segmentation
Seun-An Choe (Kyung Hee University), Gyeong-Moon Park (Kyung Hee University)
SegmentationDomain AdaptationContrastive LearningImage
🎯 What it does: Proposes the open set domain adaptation semantic segmentation task OSDA-SS and introduces the BUS method;
Open-Vocabulary 3D Semantic Segmentation with Foundation Models
Li Jiang (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)
Object DetectionSegmentationTransformerLarge Language ModelVision Language ModelImagePoint Cloud
🎯 What it does: This paper proposes the OV3D framework, which utilizes multi-view images and a vision-language foundation model to achieve fine-grained alignment between 3D point clouds and entity-level text, thereby enabling open vocabulary 3D semantic segmentation.
Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models
Pablo Marcos-Manchón (Autonomous University of Madrid), José M. Martínez (Autonomous University of Madrid)
SegmentationOptimizationDiffusion modelImage
🎯 What it does: This paper proposes a training-free 'Open Vocabulary Attention Mapping' (OVAM) mechanism that utilizes the cross-attention and self-attention of diffusion models to generate pixel-level attention heatmaps for any text word, thereby achieving open vocabulary semantic segmentation and enhancing heatmap quality through a small number of annotated words.
Open-Vocabulary Object 6D Pose Estimation
Jaime Corsetti (Fondazione Bruno Kessler), Fabio Poiesi (Idiap Research Institute)
Object DetectionPose EstimationTransformerVision Language ModelContrastive LearningImagePoint Cloud
🎯 What it does: This paper proposes an open-source vocabulary object 6D pose estimation framework called Oryon, which can locate and estimate the relative pose of target objects in RGB-D images from two different scenes using just a single description, without the need for CAD models or video sequences.
Open-Vocabulary Segmentation with Semantic-Assisted Calibration
Yong Liu (Tsinghua University), Yansong Tang (Tsinghua University)
SegmentationTransformerSupervised Fine-TuningContrastive LearningImage
🎯 What it does: This paper proposes an open vocabulary semantic segmentation framework called SCAN, which calibrates mask proposal embeddings using the global semantic prior of CLIP and reduces domain bias of CLIP through context shifting, achieving high-quality segmentation for arbitrary text labels.
Open-Vocabulary Semantic Segmentation with Image Embedding Balancing
Xiangheng Shan (Huazhong University of Science and Technology), Changxin Gao (Huazhong University of Science and Technology)
SegmentationTransformerContrastive LearningImage
🎯 What it does: A new framework for open vocabulary semantic segmentation, EBSeg, is proposed, which combines the AdaB Decoder and SSC Loss, using frozen SAM and CLIP encoders to achieve high-quality segmentation.
Open-Vocabulary Video Anomaly Detection
Peng Wu (Northwestern Polytechnical University), Yanning Zhang (Northwestern Polytechnical University)
Anomaly DetectionTransformerLarge Language ModelVision Language ModelVideoMultimodality
🎯 What it does: This paper proposes an open vocabulary video anomaly detection framework based on a pre-trained vision-language model, breaking down the task into category-agnostic detection and category-specific classification, and enhancing performance through semantic knowledge injection, temporal adapters, and pseudo-anomaly synthesis.
Open-World Human-Object Interaction Detection via Multi-modal Prompts
Jie Yang (Chinese University of Hong Kong), Ruimao Zhang (International Digital Economy Academy)
Object DetectionTransformerPrompt EngineeringContrastive LearningImageMultimodality
🎯 What it does: Proposes MP-HOI, a multi-modal prompt (text + visual) driven open-world human-object interaction (HOI) detection framework;
Open-World Semantic Segmentation Including Class Similarity
Matteo Sodano (University of Bonn), Cyrill Stachniss (University of Bonn)
SegmentationAnomaly DetectionAutonomous DrivingConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a dual-decoder convolutional neural network that can perform closed-set semantic segmentation and identify unknown categories without the need for additional training data, distinguishing them into different new categories.
Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance
Phuc Nguyen (VinAI Research), Khoi Nguyen (VinAI Research)
Object DetectionSegmentationPoint Cloud
🎯 What it does: The Open3DIS framework is proposed, which combines multi-view 2D instance masks with 3D class-agnostic proposals for open vocabulary 3D instance segmentation.
Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships
Sebastian Koch (Bosch Center for Artificial Intelligence), Timo Ropinski (University of Ulm)
Object DetectionKnowledge DistillationRepresentation LearningGraph Neural NetworkLarge Language ModelVision Language ModelPoint Cloud
🎯 What it does: Based on 3D point clouds, a graph neural network is constructed to learn the distillation of knowledge from 2D vision-language models into 3D graph features, enabling zero-shot open vocabulary predictions of 3D scene graphs.
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
Moreno D'Incà (University of Trento), Nicu Sebe (University of Trento)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: This paper proposes OpenBias, an end-to-end pipeline that utilizes LLM to generate potential bias concepts, employs VQA to identify and quantify biases in text-to-image generation models, and does not rely on a pre-defined list of biases.
OpenEQA: Embodied Question Answering in the Era of Foundation Models
Arjun Majumdar (Georgia Tech), Aravind Rajeswaran (Meta AI)
Large Language ModelVision Language ModelVideoTextBenchmark
🎯 What it does: This paper proposes and implements a novel Embodied Question Answering benchmark called OpenEQA, which encompasses two task forms: short-term memory-based (EM-EQA) and active exploration (A-EQA).
OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies
Lingdong Kong (National University of Singapore), Wei Tsang Ooi (National University of Singapore)
SegmentationKnowledge DistillationConvolutional Neural NetworkContrastive LearningImageVideo
🎯 What it does: Proposes the OpenESS framework, which achieves open vocabulary semantic segmentation on event cameras, enabling zero-shot inference with no labels or a small number of labels.
OpenStreetView-5M: The Many Roads to Global Visual Geolocation
Guillaume Astruc (Ecole des Ponts), Loic Landrieu (Ecole des Ponts)
ClassificationRecognitionRetrievalRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: This paper constructs and publicly releases a global street view image dataset OSV-5M with a scale of 5.1M, and based on this, proposes a complete visual geographic localization evaluation framework.
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
Qidong Huang (University of Science and Technology of China), Nenghai Yu (University of Science and Technology of China)
GenerationData SynthesisTransformerLarge Language ModelVision Language ModelTextMultimodality
🎯 What it does: This paper proposes a novel decoding strategy called OPERA, which reduces the hallucination problem in multimodal large language models (MLLMs) by utilizing a penalty for the 'over-trust' aggregation pattern in self-attention and a backtracking redistribution mechanism, without adding extra data or knowledge.
OpticalDR: A Deep Optical Imaging Model for Privacy-Protective Depression Recognition
Yuchen Pan (Harbin Institute of Technology), Xianming Liu (Harbin Institute of Technology)
RecognitionSafty and PrivacyTransformerImageVideoMultimodality
🎯 What it does: A learnable optical lens and deep network joint training framework for privacy-preserving depression recognition (OpticalDR) is proposed, which eliminates facial identity information through optical blurring while retaining depression-related features.
Optimal Transport Aggregation for Visual Place Recognition
Sergio Izquierdo (University of Zaragoza), Javier Civera (University of Zaragoza)
RecognitionRetrievalOptimizationTransformerContrastive LearningImage
🎯 What it does: This paper proposes a single-stage visual scene localization model DINOv2 SALAD, which combines the DINOv2 visual Transformer as a feature extractor and an OT aggregation mechanism based on the Sinkhorn algorithm.
Optimizing Diffusion Noise Can Serve As Universal Motion Priors
Korrawe Karunratanakul (ETH Zurich), Siyu Tang
GenerationOptimizationDiffusion modelVideoOrdinary Differential Equation
🎯 What it does: The research utilizes the noise space of a pre-trained human motion diffusion model for gradient optimization to achieve a universal motion prior, which can be used for tasks such as editing, control, denoising, and completion.
Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation
Hongwei Yan (Tsinghua University), Yi Zhong (Tsinghua University)
Knowledge DistillationConvolutional Neural NetworkMixture of ExpertsContrastive LearningImage
🎯 What it does: The MOSE framework is proposed, utilizing multi-layer supervision and reverse self-distillation to address the underfitting-overfitting dilemma in online continual learning.
OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning
Noor Ahmed (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes the OrCo framework to address catastrophic forgetting, overfitting, and stubbornness issues in few-shot incremental learning.
OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning
Xinyu Geng (Harbin Institute of Technology), Xiaolin Huang (Shanghai Jiao Tong University)
ClassificationRecognitionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes the OrthCaps Capsule Network, which significantly reduces capsule redundancy and parameter count by utilizing sparse attention routing and Householder orthogonalization.
Orthogonal Adaptation for Modular Customization of Diffusion Models
Ryan Po (Stanford University), Gordon Wetzstein (Stanford University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes a modular customized diffusion model that allows for independent fine-tuning of different concepts and enables the rapid merging of multiple concepts to generate images during inference.
Osprey: Pixel Understanding with Visual Instruction Tuning
Yuqian Yuan (Zhejiang University), Jianke Zhu (Zhejiang University)
RecognitionObject DetectionSegmentationConvolutional Neural NetworkLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: The Osprey model is proposed, which integrates pixel-level mask information with language instructions to achieve fine-grained visual understanding.
OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition
Tongjia Chen (Hunan University), Chen Chen (Hunan Normal University)
RecognitionTransformerLarge Language ModelPrompt EngineeringContrastive LearningVideoText
🎯 What it does: This paper proposes a universal video recognition framework called OST, which generates space-time descriptors for each action category using a large language model, and dynamically matches these descriptors with video frame features using optimal transport algorithms to bridge the semantic gap between visual models and category names.
OTE: Exploring Accurate Scene Text Recognition Using One Token
Jianjun Xu (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
RecognitionRetrievalTransformerImage
🎯 What it does: Proposes the One Token Recognizer (OTE) framework, which uses a single global token to achieve scene text recognition and processes text retrieval tasks with the same token.
Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata
Dongsu Zhang (Seoul National University), Amlan Kar (University of Toronto)
GenerationData SynthesisAutonomous DrivingAuto EncoderGenerative Adversarial NetworkPoint Cloud
🎯 What it does: A hierarchical generative cellular automaton (hGCA) model is proposed, capable of generating high-resolution 3D scene geometry that extends beyond the perceptual field and occlusion range from sparse LiDAR scans.
OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation
Ganlong Zhao (University of Hong Kong), Yizhou Yu (University of Hong Kong)
Object DetectionRobotic IntelligenceTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: This paper proposes OVER-NAV, which utilizes LLM and open vocabulary detection to construct an omnigraph structured memory to enhance the performance of Iterative Visual Language Navigation (IVLN).
Overcoming Generic Knowledge Loss with Selective Parameter Update
Wenxuan Zhang (King Abdullah University of Science and Technology), Mohamed Elhoseiny (King Abdullah University of Science and Technology)
ClassificationRecognitionTransformerContrastive LearningImage
🎯 What it does: Proposes the Selective Parameter Update (SPU) method, which updates only a small number of parameters in the first MLP layer of the pre-trained CLIP model that are most sensitive to new tasks in a continual learning scenario, while maintaining the original generalization ability.
Overload: Latency Attacks on Object Detection for Edge Devices
Erh-Chung Chen (National Tsing Hua University), Che-Rung Lee (National Tsing Hua University)
Object DetectionComputational EfficiencyAdversarial AttackImage
🎯 What it does: A delay attack framework called Overload is proposed and implemented to significantly extend the inference time of target detection models on edge devices.
OVFoodSeg: Elevating Open-Vocabulary Food Image Segmentation via Image-Informed Textual Representation
Xiongwei Wu (Singapore Management University), Chong-Wah Ngo (Singapore Management University)
SegmentationTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: An open vocabulary food image segmentation framework called OVFoodSeg is proposed, which utilizes the image-to-text learner FoodLearner and an image information text encoder to generate image information text embeddings, thereby enhancing the performance of open vocabulary food segmentation.
OVMR: Open-Vocabulary Recognition with Multi-Modal References
Zehong Ma (Peking University), Qi Tian (Huawei Inc.)
ClassificationRecognitionTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes OVMR, which generates an open vocabulary classifier using multimodal information from text descriptions and example images, without requiring additional training during inference.
PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios
Jingbo Wang (Shanghai AI Lab), Bo Dai (Shanghai AI Lab)
GenerationPose EstimationAutonomous DrivingRobotic IntelligenceReinforcement LearningVideo
🎯 What it does: In driving scenarios, PACER+ is proposed, a physically driven pedestrian animation generation framework that can control on-demand along a given trajectory while simultaneously achieving upper body or full-body motion tracking.
PAD: Patch-Agnostic Defense against Adversarial Patch Attacks
Lihua Jing (Institute of Information Engineering, Chinese Academy of Sciences), Cong Zou (Institute of Information Engineering, Chinese Academy of Sciences)
Object DetectionAdversarial AttackImageVideo
🎯 What it does: A Patch-Agnostic Defense (PAD) is designed, which does not require training or prior attack information, utilizing the semantic independence and spatial heterogeneity of patches to locate and remove them from images, compatible with any pre-trained object detector.
Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering
Kim Youwang (University of Tübingen), Gerard Pons-Moll (Max Planck Institute for Informatics)
GenerationData SynthesisOptimizationConvolutional Neural NetworkDiffusion modelScore-based ModelMesh
🎯 What it does: The Paint-it method is designed to generate high-fidelity physically-based rendering (PBR) texture maps through neural reparameterization optimization and Score-Distillation Sampling using text prompts.
Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models
Xianfang Zeng (Tencent), Gang Yu (Tencent)
RestorationGenerationDiffusion modelMesh
🎯 What it does: Paint3D proposes a two-stage generation framework that first uses a perspective-aware 2D diffusion model to generate rough textures, and then refines them in UV space, capable of producing high-resolution (2K) non-lit textures, and supports text or image conditional control.
PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor
Vidit Goel (Picsart AI Research), Humphrey Shi (Picsart AI Research)
Image TranslationSegmentationGenerationDiffusion modelImageMultimodality
🎯 What it does: The PAIR Diffusion framework is proposed to achieve independent editing of the structure and appearance of each object in an image, enabling multimodal, non-reversible image editing.
PairAug: What Can Augmented Image-Text Pairs Do for Radiology?
Yutong Xie (Australian Institute for Machine Learning), Qi Wu (Australian Institute for Machine Learning)
GenerationData SynthesisTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityElectronic Health Records
🎯 What it does: The PairAug framework is proposed for simultaneous augmentation of images and texts in medical image-text pairs.
PairDETR : Joint Detection and Association of Human Bodies and Faces
Ammar Ali (ITMO University), Sergey Zagoruyko (MTS AI)
Object DetectionPose EstimationTransformerImage
🎯 What it does: An end-to-end detection framework named PairDETR is proposed for simultaneously detecting faces and bodies and establishing associations.
Panacea: Panoramic and Controllable Video Generation for Autonomous Driving
Yuqing Wen (University of Science and Technology of China), Xiangyu Zhang (MEGVII Technology)
GenerationData SynthesisAutonomous DrivingDiffusion modelVideo
🎯 What it does: Generate controllable panoramic driving videos based on bird's-eye view (BEV) layout sequences, capable of synthesizing an unlimited amount of multi-view, variable attribute training data;
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Tsai-Shien Chen (Snap Inc.), Sergey Tulyakov (Snap Inc.)
GenerationRetrievalKnowledge DistillationTransformerVision Language ModelVideoTextMultimodality
🎯 What it does: A large-scale video-text dataset called Panda-70M has been developed, containing 70 million high-quality subtitles.
PanoContext-Former: Panoramic Total Scene Understanding with a Transformer
Yuan Dong (Alibaba Group), Ping Tan (Hong Kong University of Science and Technology)
Object DetectionSegmentationDepth EstimationTransformerImagePoint CloudMesh
🎯 What it does: This paper proposes a Transformer-based method for full 3D panoramic scene understanding, which can simultaneously predict room layouts, oriented 3D object bounding boxes, and object shapes from a single RGB panoramic image.
PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation
Yuqi Wang (University of Chinese Academy of Sciences), Zhaoxiang Zhang (University of Chinese Academy of Sciences)
Object DetectionSegmentationAutonomous DrivingConvolutional Neural NetworkSupervised Fine-TuningPoint Cloud
🎯 What it does: PanoOcc is proposed, a camera-based 3D panoramic segmentation framework that achieves unified learning of occupancy representation and can perform semantic segmentation and instance detection end-to-end.
PanoPose: Self-supervised Relative Pose Estimation for Panoramic Images
Diantao Tu (Chinese Academy of Sciences), Shuhan Shen (Chinese Academy of Sciences)
Pose EstimationDepth EstimationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes PanoPose—a self-supervised relative pose estimation network for panoramic images, which simultaneously predicts dense depth and 6-DoF poses using deep networks and a Transformer pose network, and constructs a complete panoramic SfM processing pipeline.
PanoRecon: Real-Time Panoptic 3D Reconstruction from Monocular Video
Dong Wu (Peking University), Hongbin Zha (Peking University)
Object TrackingSegmentationDepth EstimationGraph Neural NetworkSimultaneous Localization and MappingVideo
🎯 What it does: The Panoptic 3D Reconstruction task is proposed, and the PanoRecon framework is implemented, which can achieve real-time geometric reconstruction and semantic instance segmentation from monocular video.