ECCV 2024 Papers — Page 17
European Conference on Computer Vision · 2387 papers
Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering
Ruofan Liang (NVIDIA), Zian Wang (NVIDIA)
GenerationData SynthesisDiffusion modelScore-based ModelGaussian SplattingImageVideo
🎯 What it does: Designed and implemented a physically based rendering driven inverse rendering framework named DiPIR, which utilizes personalized diffusion models to recover illumination and tone mapping, achieving photorealistic virtual object insertion in single images or videos.
Photorealistic Video Generation with Diffusion Models
Agrim Gupta (Stanford University), Jose Lezama
GenerationSuper ResolutionTransformerDiffusion modelVideoTextMultimodality
🎯 What it does: Propose W.A.L.T, an image/video joint latent space model combining Transformer, window attention, and latent diffusion, capable of generating high-resolution, temporally and spatially consistent realistic videos from text prompts.
PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations
Yang Zheng (Stanford University), Gordon Wetzstein (Stanford University)
GenerationOptimizationGaussian SplattingVideoMeshPhysics Related
🎯 What it does: Utilizes multi-view videos, combining inverse rendering and inverse physics to automatically reconstruct the geometry, material, and physical parameters of human clothing, generating digital portraits that can be rendered with high quality under new poses and lighting conditions.
PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation
Shaowei Liu (University of Illinois Urbana Champaign), Shenlong Wang (University of Illinois Urbana Champaign)
Image TranslationGenerationLarge Language ModelDiffusion modelImageVideoPhysics RelatedStochastic Differential Equation
🎯 What it does: This paper proposes a system called PhysGen that generates realistic, physically plausible, and temporally consistent videos from a single image. The system allows users to input initial forces or torques, automatically inferring the object geometry, material, and physical parameters in the image. It then uses rigid body physics simulation to generate object motion trajectories, followed by image relighting and post-processing with diffusion models to produce the final video.
Physical-Based Event Camera Simulator
Haiqian Han (Tsinghua University), Xiangyang Ji (Xiaomi Mobile Software Co., Ltd)
Data SynthesisVideoPhysics Related
🎯 What it does: This paper proposes a physics-based event camera simulator (PECS) that can directly generate high-fidelity event streams from 3D scenes;
Physically Plausible Color Correction for Neural Radiance Fields
Qi Zhang (Independent Researcher), HONGDONG LI
Image TranslationRestorationNeural Radiance FieldImage
🎯 What it does: Propose a physically plausible color correction module embedded in NeRF, which can achieve unified color reconstruction and color translation under multi-camera and multi-illumination environments.
Physics-Based Interaction with 3D Objects via Video Generation
Tianyuan Zhang (Massachusetts Institute of Technology), William T. Freeman (Cornell University)
GenerationData SynthesisDiffusion modelScore-based ModelNeural Radiance FieldGaussian SplattingVideoMeshPhysics Related
🎯 What it does: By inferring physical material fields from static 3D objects and leveraging the dynamics prior of video generation models, combined with differentiable MPM simulation and differentiable rendering, achieving interactive 3D dynamic generation under arbitrary external forces.
Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition
Satoshi Ikehata (National Institute of Informatics), Yuta Asano (National Institute of Informatics)
Depth EstimationConvolutional Neural NetworkTransformerSupervised Fine-TuningImageBenchmark
🎯 What it does: This paper proposes a SpectraM-PS method that can recover surface normals of dynamic surfaces using only a single spectral multiplex image, without requiring pre-calibration of light sources or sensors, and supports an arbitrary number and order of channels.
Physics-informed Knowledge Transfer for Underwater Monocular Depth Estimation
Jinghe Yang (University of Melbourne), Ye Pu (University of Melbourne)
Depth EstimationDomain AdaptationImagePhysics Related
🎯 What it does: In the absence of depth annotations for underwater image data, unsupervised monocular depth estimation under water is achieved by migrating a DPT model pre-trained in the air to the underwater environment, and introducing a physical underwater imaging model along with a boundary loss based on this model.
Pick-a-back: Selective Device-to-Device Knowledge Transfer in Federated Continual Learning
HyungJune Lee (Ewha Womans University), JinYi Yoon (Virginia Tech)
Federated LearningConvolutional Neural NetworkImage
🎯 What it does: Proposes a novel edge federated continual learning framework called Pick-a-back, which enables selective and customized knowledge transfer between devices, helping local models learn faster and more generalizable when facing private heterogeneous tasks.
PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning
Haiyang Guo (Institute of Automation, Chinese Academy of Sciences), Cheng-Lin Liu (Institute of Automation, Chinese Academy of Sciences)
Federated LearningTransformerImage
🎯 What it does: Propose the PILoRA method to address catastrophic forgetting and classifier bias in federated class-incremental learning
PISR: Polarimetric Neural Implicit Surface Reconstruction for Textureless and Specular Objects
Guangcheng Chen (Southern University of Science and Technology), Hong Zhang (Southern University of Science and Technology)
GenerationNeural Radiance FieldImageMeshBenchmark
🎯 What it does: Proposes a neural implicit surface method called PISR for 3D reconstruction of textureless and specular objects using polarized images.
PiTe: Pixel-Temporal Alignment for Large Video-Language Model
Yang Liu (Westlake University), Donglin Wang (Westlake University)
Object TrackingTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Propose the pixel-level spatiotemporal alignment model PiTe based on object trajectories and construct the PiTe-143k video-text dataset containing trajectory information.
Pix2Gif: Motion-Guided Diffusion for GIF Generation
Hitesh Kandala (Microsoft Research), Jianwei Yang (Microsoft Research)
GenerationVision Language ModelDiffusion modelOptical FlowImageVideoText
🎯 What it does: Proposed a motion-guided diffusion-based Pix2Gif model that converts a single image into a GIF animation;
PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation
Junsong Chen (Huawei Noah's Ark Lab), Zhenguo Li (Huawei Noah's Ark Lab)
GenerationTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelAuto EncoderImageText
🎯 What it does: Upgraded PixArtα to PixArtΣ through weak-to-strong training, achieving direct 4K text-to-image generation; introduced higher quality data, improved attention compression techniques, and performed rapid transfer and fine-tuning based on pre-training.
Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization
Tao Yang (Bytedance Inc), Lei Zhang (Hong Kong Polytechnic University)
GenerationSuper ResolutionTransformerDiffusion modelImage
🎯 What it does: Propose the Pixel-Aware Stable Diffusion (PASD) network, which leverages pre-trained Stable Diffusion and integrates pixel-level cross-attention, denoising modules, adjustable noise scheduling, etc., to achieve real image super-resolution and personalized style transfer.
Pixel-GS Density Control with Pixel-aware Gradient for 3D Gaussian Splatting
Zheng Zhang (University of Hong Kong), Hengshuang Zhao (University of Hong Kong)
OptimizationComputational EfficiencyGaussian SplattingPoint Cloud
🎯 What it does: To address the blurriness and needle artifacts caused by sparse point clouds in 3D Gaussian Splatting (3DGS), Pixel-GS is proposed. It optimizes point cloud growth conditions using pixel-level weighted gradients and suppresses floating point artifacts near the camera through gradient scaling for floating points.
PixOOD: Pixel-Level Out-of-Distribution Detection
Tomas Vojir, Jiri Matas
Anomaly DetectionTransformerImage
🎯 What it does: Proposed a PixOOD framework based on pixel-level unsupervised anomaly detection to identify anomalies not present in the training data.
Placing Objects in Context via Inpainting for Out-of-distribution Segmentation
Pau de Jorge Aranda, Gregory Rogez
SegmentationDomain AdaptationAnomaly DetectionVision Language ModelDiffusion modelImage
🎯 What it does: Proposes the Placing Objects in Context (POC) process, which uses diffusion models to realistically insert arbitrary objects into images, generating a high-quality out-of-distribution (OOD) segmentation dataset.
Plain-Det: A Plain Multi-Dataset Object Detector
Cheng Shi (ShanghaiTech University), Sibei Yang (ShanghaiTech University)
Object DetectionTransformerVision Language ModelContrastive LearningImage
🎯 What it does: Propose Plain-Det, a target detection framework that is compatible with multiple detection architectures, supports multi-datasets, is flexible, and has efficient training.
Platypus: A Generalized Specialist Model for Reading Text in Various Forms
Peng Wang (Alibaba Group), Cong Yao (Alibaba Group)
RecognitionTransformerPrompt EngineeringImageTextMultimodality
🎯 What it does: Proposed and implemented the unified architecture of the Platypus model for reading text from images in various forms, including natural scenes, documents, handwritten text, and formulas.
PLOT: Text-based Person Search with Part Slot Attention for Corresponding Part Discovery
Jicheol Park (Pohang University of Science and Technology), Suha Kwak (Pohang University of Science and Technology)
RetrievalRecurrent Neural NetworkTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a text-based person search framework named PLOT, which utilizes slot attention to automatically discover and align human part features in visual and textual modalities, thereby improving retrieval accuracy.
Plug and Play: A Representation Enhanced Domain Adapter for Collaborative Perception
Tianyou Luo (Beijing University of Posts and Telecommunications), Jinglin Li (Beijing University of Posts and Telecommunications)
Domain AdaptationAutonomous DrivingKnowledge DistillationContrastive LearningPoint Cloud
🎯 What it does: Proposed a Plug-and-Play Domain Adapter (PnPDA), achieving non-destructive alignment of intermediate features between heterogeneous deep learning models, and enabling new models to seamlessly integrate into collaborative perception networks through two-step semantic transformation.
Plug-and-Play Learned Proximal Trajectory for 3D Sparse-View X-Ray Computed Tomography
Romain Vo (Université Paris-Saclay), Etienne Decenciere
Convolutional Neural NetworkBiomedical DataComputed Tomography
🎯 What it does: Proposes a learning-based proximal trajectory (LPT) method that embeds a post-processing network into the Plug-and-Play (PnP) framework for three-dimensional sparse-view X-ray computed tomography (CT) reconstruction.
PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation
Ning Gao (Xi'an Jiaotong University), Nanning Zheng (Xi'an Jiaotong University)
SegmentationConvolutional Neural NetworkBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Proposed a Progressive Mean Teacher (PMT) framework that alternately trains two isomorphic Mean Teacher models at different training iteration stages to continuously generate diverse, high-quality pseudo labels, thereby enhancing semi-supervised learning for medical image segmentation.
POA: Pre-training Once for Models of All Sizes
Yingying Zhang (Ant Group), Ming Yang (Ant Group)
Knowledge DistillationRepresentation LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: A single pre-training process that utilizes an elastic student branch to simultaneously generate subnetworks of different scales, including ViT, Swin Transformer, and ResNet, for direct use in downstream tasks.
POCA: Post-training Quantization with Temporal Alignment for Codec Avatars
Jian Meng (Meta Codec Avatars Lab), Jae-sun Seo
CompressionOptimizationComputational EfficiencyVideo
🎯 What it does: Proposes a post-training quantization algorithm called POCA for the Codec Avatar decoder, addressing time noise caused by low-precision quantization.
POET: Prompt Offset Tuning for Continual Human Action Adaptation
Prachi Garg, Fernando de la Torre
RecognitionSafty and PrivacyMeta LearningGraph Neural NetworkPrompt EngineeringAuto EncoderGraph
🎯 What it does: Propose the POET method, utilizing Prompt Offset Tuning to achieve privacy-friendly, few-shot continual learning in skeletal action recognition models;
Point-supervised Panoptic Segmentation via Estimating Pseudo Labels from Learnable Distance
Jing Li (University of Chinese Academy of Sciences), Zhaoxiang Zhang (University of Chinese Academy of Sciences)
SegmentationTransformerImage
🎯 What it does: Propose a pseudo label estimation framework based on learnable distance (EPLD), which generates dense full-pixel pseudo labels for weakly supervised semantic and instance segmentation by treating each instance as an anchor query and utilizing cross-attention to predict pixel-to-instance distances from single-point labels.
PointLLM: Empowering Large Language Models to Understand Point Clouds
Runsen Xu (Chinese University of Hong Kong), Dahua Lin (Chinese University of Hong Kong)
GenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextMultimodalityPoint Cloud
🎯 What it does: Studied a multi-modal model called PointLLM that integrates large language models with point clouds, enabling it to understand and generate descriptions of colored point cloud objects based on human instructions.
PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training
Suyi Chen (University of Electronic Science and Technology of China), Shuaicheng Liu (University of Electronic Science and Technology of China)
Data SynthesisPose EstimationConvolutional Neural NetworkDiffusion modelPoint CloudBenchmark
🎯 What it does: Developed a deep generation and correction framework based on diffusion models (PointRegGPT), which can automatically generate realistic overlapping point cloud pairs from a single depth image for training 3D point cloud registration models.
PolyOculus: Simultaneous Multi-view Image-based Novel View Synthesis
Jason J. Yu (York University), Marcus A. Brubaker (York University)
GenerationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Designed a set-to-set diffusion model called PolyOculus, which can simultaneously generate a group of self-consistent new views under the condition of known limited views, thereby avoiding the problems of cyclic inconsistency and error accumulation caused by traditional autoregressive methods.
PolyRoom: Room-aware Transformer for Floorplan Reconstruction
Yuzhou Liu (Institute of Automation, Chinese Academy of Sciences), Shuhan Shen (Institute of Automation, Chinese Academy of Sciences)
GenerationConvolutional Neural NetworkTransformerPoint Cloud
🎯 What it does: Propose the PolyRoom method to achieve vectorized floor plan reconstruction based on indoor point clouds
Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos
Keqiang Sun (Chinese University of Hong Kong), Shangzhe Wu (Stanford University)
GenerationTransformerAuto EncoderContrastive LearningVideoMesh
🎯 What it does: This paper proposes an unsupervised framework that does not require pose annotations or shape templates, capable of learning a generative 3D motion model for animal categories from raw online videos and generating 4D animations from single images.
Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizer
Yu Deng (Xiaobing.AI), Baoyuan Wang
GenerationData SynthesisTransformerNeural Radiance FieldGenerative Adversarial NetworkImageVideoMesh
🎯 What it does: This study proposes a one-shot 4D head synthesis method, which trains a generator using pseudo multi-view videos to achieve animation synthesis between source and driving images, supporting free-view rendering.
Pose Guided Fine-Grained Sign Language Video Generation
Tongkai Shi (Tianjin University), Wei Feng (Tianjin University)
GenerationData SynthesisPose EstimationGenerative Adversarial NetworkOptical FlowVideo
🎯 What it does: Propose a pose-guided fine-grained sign language video generation framework called PGMM, which separates coarse motion and fine details through Coarse Motion Module (CMM) and Pose Fusion Module (PFM), thereby enhancing temporal consistency and detail quality.
Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization
Jiayun Wang (University Of Berkeley), Stella X. Yu (University Of Berkeley)
Pose EstimationRepresentation LearningConvolutional Neural NetworkContrastive LearningImageBenchmark
🎯 What it does: Learn visual representations that simultaneously capture semantic and pose information from unlabeled images, and propose a perspective trajectory regularization loss.
PoseAugment: Generative Human Pose Data Augmentation with Physical Plausibility for IMU-based Motion Capture
Zhuojun Li (Tsinghua University), Yuanchun Shi (Tsinghua University)
GenerationData SynthesisPose EstimationOptimizationMixture of ExpertsAuto EncoderTime SeriesSequential
🎯 What it does: Proposes a two-stage PoseAugment workflow, first generating high-fidelity and diverse pose sequences using an improved VAE autoregressive generator, then correcting motion artifacts through physical optimization (dual PD controllers + reaction force regularization), ultimately synthesizing IMU data suitable for training.
PoseCrafter: One-Shot Personalized Video Synthesis Following Flexible Pose Control
Yong Zhong (Renmin University of China), Chongxuan Li (Renmin University of China)
GenerationData SynthesisTransformerDiffusion modelVideo
🎯 What it does: This paper proposes PoseCrafter, a single-shot pose-controlled personalized video generation framework based on Stable Diffusion and ControlNet, which can generate high-quality videos that are highly consistent with target poses with only a single video segment for training.
PoseSOR: Human Pose Can Guide Our Attention
Huankang Guan (City University of Hong Kong), Rynson W.H. Lau (City University of Hong Kong)
TransformerImage
🎯 What it does: This paper proposes PoseSOR, a model that leverages human pose information to enhance the Salient Object Ranking (SOR) task.
PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer
Tongkun Guan (Shanghai Jiao Tong University), Xiaokang Yang (Shanghai Jiao Tong University)
RecognitionSegmentationRecurrent Neural NetworkTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a visual localization and object segmentation model based on a bimodal attention mechanism combining vision and language, for parsing referential expressions and locating target regions in images.
Post-training Quantization with Progressive Calibration and Activation Relaxing for Text-to-Image Diffusion Models
Siao Tang (Tsinghua University), Wenwu Zhu (Tsinghua University)
GenerationDiffusion modelImageTextBenchmark
🎯 What it does: Proposed a post-training quantization method PCR (Progressive Calibration and Relaxing) and an evaluation benchmark QDiffBench for text-to-image diffusion models, addressing issues of cumulative errors during quantization and mismatch in activation sensitivity across different time steps.
PosterLlama: Bridging Design Ability of Langauge Model to Content-Aware Layout Generation
Jaejung Seol (Ulsan National Institute of Science and Technology), Jaejun Yoo (Ulsan National Institute of Science and Technology)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringDiffusion modelImageTextMultimodality
🎯 What it does: Designed and implemented a content-aware poster layout generation framework called PosterLlama based on language models, which can convert layout elements into HTML code and generate high-quality, semantically rich layouts through visual-textual bimodal training.
Power Variable Projection for Initialization-Free Large-Scale Bundle Adjustment
Simon Weber (Technical University of Munich), Daniel Cremers (Technical University of Munich)
Pose EstimationOptimizationSimultaneous Localization and MappingImageBenchmark
🎯 What it does: Proposed the Power Variable Projection (PoVar) method, achieving initialization-agnostic large-scale bundle adjustment, and combined with Riemannian PoBA for projective improvements.
Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learning
Fanyue Wei (University of Electronic Science and Technology of China), Wen Li (University of Electronic Science and Technology of China)
GenerationReinforcement LearningVision Language ModelDiffusion modelImageTextBenchmark
🎯 What it does: Propose a method that utilizes the deterministic policy gradient (DPG) reinforcement learning framework to fine-tune the Stable Diffusion model for personalized text-to-image generation.
PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving
Zhili Chen (HKUST DeepRoute.AI), Qifeng Chen (HKUST DeepRoute.AI)
Autonomous DrivingTransformerImagePoint Cloud
🎯 What it does: Propose an end-to-end autonomous driving framework called PPAD, which simulates dynamic interactions among traffic participants by alternately performing motion prediction (Prediction) and motion planning (Planning) at each time step.
PQ-SAM: Post-training Quantization for Segment Anything Model
Xiaoyu Liu (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
SegmentationComputational EfficiencyTransformerImage
🎯 What it does: Proposes a post-training quantization method called PQ-SAM specifically for SAM.
Pre-trained Visual Dynamics Representations for Efficient Policy Learning
Hao Luo (Peking University), Zongqing Lu (Peking University)
Representation LearningTransformerSupervised Fine-TuningReinforcement LearningAuto EncoderVideo
🎯 What it does: Proposes a video pre-trained visual dynamic representation (PVDR) to improve the sampling efficiency and generalization ability in reinforcement learning
PreciseControl: Enhancing Text-To-Image Diffusion Models with Fine-Grained Attribute Control
Rishubh Parihar (Indian Institute of Science), Venkatesh Babu RADHAKRISHNAN
GenerationSupervised Fine-TuningVision Language ModelDiffusion modelGenerative Adversarial NetworkImageText
🎯 What it does: Propose a method that trains a lightweight latent mapper on the W+ latent space of StyleGAN2 to condition a text-to-image diffusion model (Stable Diffusion), enabling precise inversion and fine-grained attribute editing of facial images, while also supporting multi-person scene synthesis.
PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines
ZiDong Wang (Shanghai AI Laboratory), Lei Bai (Shanghai AI Laboratory)
Convolutional Neural NetworkRecurrent Neural NetworkImageVideoMultimodalityTime SeriesBenchmark
🎯 What it does: Built PredBench—a unified spatiotemporal prediction benchmark integrating 12 mainstream models, 15 interdisciplinary datasets, and providing standardized experimental setups and a multidimensional evaluation framework.
Prediction Exposes Your Face: Black-box Model Inversion via Prediction Alignment
Yufan Liu (Chinese Academy of Sciences), Weiping Wang (Chinese Academy of Sciences)
Adversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: Propose a P2I model based on predictive vector alignment, which maps predicted vectors to StyleGAN's W+ latent space, achieving black-box model inversion attacks in a single forward inference.
PreLAR: World Model Pre-training with Learnable Action Representation
Lixuan Zhang (Chinese Academy of Sciences), Xilin Chen (Chinese Academy of Sciences)
Representation LearningRobotic IntelligenceReinforcement LearningWorld ModelVideo
🎯 What it does: PreLAR pre-trains an action-conditioned world model by learning learnable implicit action representations from unlabeled videos and fine-tunes it on downstream visual control tasks.
PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priors
Tianyuan Yuan (Tsinghua University), Hang Zhao (University of Southern California)
Autonomous DrivingConvolutional Neural NetworkTransformerNeural Radiance FieldVideoPoint Cloud
🎯 What it does: Construct a static prior using city-scale NeRF from historical driving data to enhance the robustness and accuracy of online perception models.
PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigation
Renjie Lu (Sun Yat-sen University), WEI-SHI ZHENG
OptimizationTransformerVision Language ModelVision-Language-Action ModelContrastive LearningMultimodality
🎯 What it does: This paper proposes a new vision-language navigation method called PRET, which uses directed graphs and oriented faithful trajectories for global planning. It directly evaluates the alignment between instructions and different trajectories to determine the next navigation target.
Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection
Gaurav Bhatt (University Of British Columbia), James Ross
Object DetectionTransformerPrompt EngineeringImage
🎯 What it does: Propose a memory network-based Deformable-DETR (MD-DETR) for continuous object detection, fine-tuning new tasks while retaining prior task knowledge.
Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective
Zhaoxin Wang (Xidian University), Yaochu Jin (Westlake University)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Proposes the FGSM-PCO algorithm, which effectively prevents and corrects catastrophic overfitting by adaptively fusing historical and current adversarial examples in the fast adversarial training (FAT) process, combined with a novel regularization loss.
Prioritized Semantic Learning for Zero-shot Instance Navigation
xinyu sun, Junwei Liang (Hong Kong University of Science and Technology)
Robotic IntelligenceVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a Priority Semantic Learning (PSL) framework for zero-shot instance navigation, addressing the issue of neglecting semantic information in traditional ImageNav pre-training tasks.
Privacy-Preserving Adaptive Re-Identification without Image Transfer
Hamza Rami (LTCI, Télécom Paris, Institut Polytechnique de Paris), Stéphane Lathuilière (LTCI, Télécom Paris, Institut Polytechnique de Paris)
RetrievalDomain AdaptationFederated LearningSafty and PrivacyTransformerContrastive LearningImage
🎯 What it does: Proposes a federated learning framework named Fed-Protoid for distributed unsupervised domain adaptation (DUDA-Rid) in person re-identification without transferring images.
Pro2SAM: Mask Prompt to SAM with Grid Points for Weakly Supervised Object Localization
Xi Yang (Xidian University), Xinbo Gao (Chongqing University of Posts and Telecommunications)
Object DetectionTransformerPrompt EngineeringImage
🎯 What it does: Propose the Pro2SAM method, which first generates a rough foreground mask using a Global Token Transformer (GTFormer), then employs the Segment Anything Model (SAM) with grid point prompts to generate fine-grained masks. Finally, pixel-level similarity matching selects the final localization box, achieving weakly supervised object localization.
Probabilistic Image-Driven Traffic Modeling via Remote Sensing
Scott Workman (DZYNE Technologies), Armin Hadzic (DZYNE Technologies)
Autonomous DrivingRepresentation LearningTransformerImageMultimodalityTime Series
🎯 What it does: Construct a multi-modal, multi-task Transformer segmentation network using remote sensing aerial images to achieve dense urban-level traffic speed prediction.
Probabilistic Weather Forecasting with Deterministic Guidance-based Diffusion Model
Donggeun Yoon (Korea Electronics Technology Institute), Donghyeon Cho (Hanyang University)
Diffusion modelImageTime SeriesBenchmarkPhysics Related
🎯 What it does: Proposes the Deterministic Guidance-based Diffusion Model (DGDM), combining a non-autoregressive deterministic prediction branch with a probabilistic branch of Brownian Bridge diffusion to achieve high-accuracy and diverse weather forecasting.
ProDepth: Boosting Self-Supervised Multi-Frame Monocular Depth with Probabilistic Fusion
Sungmin Woo (Yonsei University), Sangyoun Lee (Yonsei University)
Depth EstimationAutonomous DrivingVideo
🎯 What it does: Proposes a self-supervised multi-frame monocular depth estimation framework named ProDepth, which effectively handles matching errors caused by object motion in dynamic scenes.
Progressive Classifier and Feature Extractor Adaptation for Unsupervised Domain Adaptation on Point Clouds
Zicheng Wang (University of Hong Kong), Dong Xu (University of Hong Kong)
Domain AdaptationGraph Neural NetworkPoint Cloud
🎯 What it does: Proposed a progressive adaptation framework named PCFEA that deeply couples the classifier and feature extractor for unsupervised point cloud domain adaptation.
Progressive Pretext Task Learning for Human Trajectory Prediction
Xiaotong Lin (Sun Yat-sen University), Jian-Fang Hu (Sun Yat-sen University)
Autonomous DrivingKnowledge DistillationRepresentation LearningTransformerPrompt EngineeringTime SeriesSequential
🎯 What it does: Propose a three-stage Progressive Pretext Task Learning (PPT) framework. First, short-term motion patterns are learned by progressively predicting the next position (Task-I). Then, long-term dependencies are learned through goal prediction (Task-II). Finally, a complete future trajectory is predicted by combining both (Task-III), with cross-task knowledge distillation used to prevent forgetting.
Progressive Proxy Anchor Propagation for Unsupervised Semantic Segmentation
Hyun Seok Seong (Sungkyunkwan University), Jae-Pil Heo (Sungkyunkwan University)
SegmentationTransformerContrastive LearningImage
🎯 What it does: This paper proposes a Progressive Proxy Anchor Propagation (PPAP) strategy to improve positive and negative sample selection in contrastive learning by progressively migrating proxy anchors to semantically similar sample dense regions and defining fuzzy areas in unsupervised semantic segmentation.
Projecting Points to Axes: Oriented Object Detection via Point-Axis Representation
Zeyang Zhao (Xi'an Jiaotong University), Yihong Gong (Xi'an Jiaotong University)
Object DetectionTransformerImage
🎯 What it does: Proposed the Point-Axis representation along with the corresponding maximum projection loss and cross-axis loss, and built an end-to-end Oriented DETR for oriented object detection.
ProMerge: Prompt and Merge for Unsupervised Instance Segmentation
Dylan J Li (Meta Reality Labs), Gyungin Shin (University of Oxford)
SegmentationTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: Propose the ProMerge framework, which generates point prompt masks using self-supervised visual features and iteratively merges them, further utilizing background base masks for pruning to achieve unsupervised instance segmentation results. These results are then used as pseudo labels to train a class-agnostic object detector.
Prompt-Based Test-Time Real Image Dehazing: A Novel Pipeline
Zixuan Chen (Zhejiang University), Zheming Lu (Zhejiang University)
RestorationDomain AdaptationPrompt EngineeringImage
🎯 What it does: Propose a no-training test-time defogging pipeline called PTTD, which narrows the domain gap between synthetic and real fog images by fine-tuning the pre-trained model's encoder feature statistics (mean, variance);
Prompt-Driven Contrastive Learning for Transferable Adversarial Attacks
Hunmin Yang (KAIST), Kuk-Jin Yoon (KAIST)
Adversarial AttackPrompt EngineeringContrastive LearningImageText
🎯 What it does: This paper proposes a generative adversarial attack framework based on prompt learning and contrastive learning (PDCL-Attack), which guides the generator to learn more transferable perturbations through the text features of the CLIP model.
PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery
Fernando Julio Cendra (University of Hong Kong), Kai Han (University of Hong Kong)
ClassificationRecognitionMeta LearningTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: Proposes the PromptCCD framework, which constructs a dynamically expandable prompt pool using a Gaussian Mixture Model (GMM) to address catastrophic forgetting and the problem of unknown category counts in the continual class discovery (CCD) task.
PromptFusion: Decoupling Stability and Plasticity for Continual Learning
Haoran Chen (Fudan University), Yu-Gang Jiang (Cornell Univeristy)
Knowledge DistillationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: Propose the PromptFusion method, which separates the stability and plasticity issues in continual learning. Stability is addressed by the Stabilizer module (based on CoOp text prompts), while plasticity is handled by the Booster module (based on VPT visual prompts). Further, PromptFusion-Lite is introduced, dynamically selecting whether to activate the Booster at the input level to reduce computational overhead.
Prompting Future Driven Diffusion Model for Hand Motion Prediction
Bowen Tang (Australian National University), HONGDONG LI
GenerationPose EstimationTransformerPrompt EngineeringDiffusion modelTime SeriesSequential
🎯 What it does: Propose a prompt-based future-driven diffusion model (PromptFDDM) for predicting hand movements in first- and third-person perspectives.
Prompting Language-Informed Distribution for Compositional Zero-Shot Learning
Wentao Bao (Michigan State University), Yu Kong (Michigan State University)
ClassificationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageText
🎯 What it does: Propose the PLID method, which leverages a large language model to generate diverse and information-rich sentence descriptions, constructs a language information distribution, prompts CLIP, and combines visual-lingual primitive decomposition with logarithmic hybrid fusion to achieve compositional zero-shot learning.
PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts
Zewen Chen (State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA), Leon Wang (OPPO Hardware System)
Convolutional Neural NetworkTransformerPrompt EngineeringImage
🎯 What it does: This paper proposes a Prompt-based No-Reference Image Quality Assessment (NR-IQA) framework called PromptIQA, which can quickly adapt to new evaluation needs without fine-tuning after training by inputting a small number of image-score pair (ISP) sequences as prompts.
ProSub: Probabilistic Open-Set Semi-Supervised Learning with Subspace-Based Out-of-Distribution Detection
Erik Wallin (Saab AB), Lars Hammarstrand (Chalmers University of Technology)
Anomaly DetectionImage
🎯 What it does: Propose the ProSub framework in open semi-supervised learning, achieving probabilistic discrimination of ID/OOD by calculating the angle scores between sample features and the ID subspace and estimating their beta distribution.
Protecting NeRFs' Copyright via Plug-And-Play Watermarking Base Model
Qi Song (Hong Kong Baptist University), Renjie Wan (Hong Kong Baptist University)
Safty and PrivacyNeural Radiance FieldImage
🎯 What it does: This paper proposes a plug-and-play NeRF copyright protection method called NeRFProtector, which directly embeds binary watermarks during the NeRF generation process, avoiding post-training tuning windows and reducing the risk of malicious tampering;
ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbation
Yi Zhang (University of Warwick), Xingyu Zhao (University of Warwick)
GenerationVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: This paper proposes the ProTIP framework to systematically evaluate and verify the probabilistic robustness of text-to-image diffusion models (T2I DM) under random perturbations.
ProtoComp: Diverse Point Cloud Completion with Controllable Prototype
Xumin Yu (Tsinghua University), Jiwen Lu (Tsinghua University)
RestorationGraph Neural NetworkTransformerDiffusion modelPoint Cloud
🎯 What it does: This paper proposes a point cloud completion method called ProtoComp based on controllable prototypes, which completes partial point clouds by first generating a rough semantic prototype and then refining geometric details.
ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentation
Mengcheng Lan (Nanyang Technological University), Wayne Zhang (SenseTime Research)
SegmentationPrompt EngineeringVision Language ModelImageText
🎯 What it does: Studies how to improve CLIP for open-vocabulary semantic segmentation by leveraging the spatial consistency characteristics of vision foundation models (VFM) to enhance segmentation accuracy.
PSALM: Pixelwise Segmentation with Large Multi-modal Model
Zheng Zhang (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
SegmentationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageVideoMultimodality
🎯 What it does: PSALM achieves unified multiple pixel-level segmentation tasks within a single model by externally adding a mask decoder to a large multimodal model (LMM) and designing a four-input scheme (image, task instruction, conditional prompt, and mask token).
Pseudo-Embedding for Generalized Few-Shot Point Cloud Segmentation
Chih-Jung Tsai (National Tsing Hua University), Tyng-Luh Liu (Academia Sinica)
SegmentationMeta LearningGraph Neural NetworkTransformerVision Language ModelPoint Cloud
🎯 What it does: Propose a general few-shot 3D segmentation framework that utilizes background context for pseudo embeddings, capable of simultaneously identifying base classes and novel classes.
Pseudo-keypoint RKHS Learning for Self-supervised 6DoF Pose Estimation
Yangzheng Wu (Queen's University), Michael Alan Greenspan (Queen's University)
Pose EstimationDomain AdaptationImageVideo
🎯 What it does: Propose a self-supervised 6DoF pose estimation framework RKHSPose based on pseudo-keypoint voting, which uses a learnable RKHS adapter to reduce the sim-to-real domain gap in high-dimensional feature space through Maximum Mean Discrepancy (MMD).
Pseudo-Labelling Should Be Aware of Disguising Channel Activations
Changrui Chen (University of Warwick), Jungong Han (University of Sheffield)
ClassificationObject DetectionSegmentationDomain AdaptationImage
🎯 What it does: Proposed a correction method addressing channel activation differences in pseudo-label learning.
Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentation
Seonghoon Yu (GIST), Jeany Son (GIST)
SegmentationData SynthesisTransformerPrompt EngineeringVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a pseudo-supervised generation framework without manual annotation, automatically generating high-quality masks using a segmentation model and producing discriminative representative expressions corresponding to the masks via a text generation model, thereby providing scalable training data for RIS models.
Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos
Mi Luo, Kristen Grauman (The University of Texas at Austin)
Image TranslationGenerationPose EstimationTransformerDiffusion modelAuto EncoderVideoBenchmark
🎯 What it does: This paper proposes a two-stage generative framework called Exo2Ego, achieving cross-perspective translation from third-person (exo) videos to first-person (ego) videos, aiming to generate realistic hand-object interaction perspectives.
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
Yizhe Xiong (Tsinghua University), Guiguang Ding (Tsinghua University)
ClassificationComputational EfficiencyTransformerImageBenchmark
🎯 What it does: Proposes the PYRA method, which merges tokens in Vision Transformers through parallel generation of modulation weights and reactivation mechanisms, balancing training and inference efficiency.
Pyramid Diffusion for Fine 3D Large Scene Generation
Yuheng Liu (Southwest Jiaotong University), Ming-Hsuan Yang (University of California Merced)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelPoint Cloud
🎯 What it does: Studied a pyramid discrete diffusion model (PDD) that generates high-quality 3D large-scale scenes through a coarse-to-fine approach.
Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge
Haibo Wang (Fudan University), Weifeng Ge (Fudan University)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: This paper enhances the performance on visual question answering (VQA) tasks requiring diverse world knowledge reasoning by incorporating question-answer pairs generated from image labels as prompts into multimodal large language models (MLLMs).
Quality Assured: Rethinking Annotation Strategies in Imaging AI
Tim Rädsch (German Cancer Research Center), Lena Maier-Hein (German Cancer Research Center)
SegmentationData-Centric LearningImageTabularBiomedical Data
🎯 What it does: This paper systematically evaluates the impact of internal quality assurance (QA) within annotation companies on image annotation quality by comparing the performance of four annotation companies with Amazon Mechanical Turk (MTurk), and explores the role of improved annotation instructions and image features in enhancing annotation quality.
Quanta Video Restoration
Prateek Chennuri, Stanley H Chan
RestorationConvolutional Neural NetworkOptical FlowImageVideoPhysics Related
🎯 What it does: This paper proposes an end-to-end deep learning framework named QUIVER for recovering high-frame-rate videos from 3-bit quantum images (single-photon detectors) under extremely low illumination, while simultaneously addressing denoising and motion compensation.
Quantization-Friendly Winograd Transformations for Convolutional Neural Networks
Vladimir Protsenko (Huawei Noah's Ark Lab), Alexander Filippov (Huawei Noah's Ark Lab)
ClassificationSegmentationSuper ResolutionConvolutional Neural NetworkImage
🎯 What it does: Proposed an 8-bit integer quantization Winograd convolution transformation scheme, first using particle swarm optimization to find a quantization-friendly transformation matrix, then fine-tuning it as a learnable parameter during training to significantly reduce quantization error.
Quantized Prompt for Efficient Generalization of Vision-Language Models
Tianxiang Hao (Bytedance), Guiguang Ding (Tsinghua University)
Computational EfficiencyPrompt EngineeringMultimodality
🎯 What it does: This paper proposes introducing quantization techniques into the prompts of visual language models to enhance the model's generalization capability in downstream tasks and significantly reduce storage and inference costs.
QUAR-VLA: Vision-Language-Action Model for Quadruped Robots
Pengxiang Ding (Zhejiang University), Donglin Wang (Zhejiang University)
Robotic IntelligenceTransformerSupervised Fine-TuningVision Language ModelVision-Language-Action ModelMultimodality
🎯 What it does: Proposed the QUAR-VLA framework, tightly coupling vision and language instructions to generate directly executable quadruped robot actions, and implemented the QUART model and large-scale multi-task dataset QUARD;
QueryCDR: Query-based Controllable Distortion Rectification Network for Fisheye Images
Pengbo Guo (Xi'an Jiaotong University), Xueming Qian (Xi'an Jiaotong University)
RestorationConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose a query-based controllable fish-eye image distortion correction network called QueryCDR, which can handle fish-eye images with different distortion levels without retraining.
R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model
Changhoon Kim (Arizona State University), Yezhou Yang (Arizona State University)
GenerationSafty and PrivacyAdversarial AttackConvolutional Neural NetworkVision Language ModelDiffusion modelImageText
🎯 What it does: RACE enhances the robustness of text-to-image diffusion models against concept elimination through adversarial training on text prompts, preventing the reconstruction of sensitive content.
R^2-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations
Xiang Li (Carnegie Mellon University), Bhiksha Raj (Carnegie Mellon University)
SegmentationData SynthesisTransformerLarge Language ModelAgentic AIImageVideoMultimodalityBenchmarkChain-of-ThoughtAudio
🎯 What it does: Proposes R-Bench 2, a robustness evaluation benchmark for citation-aware models (RPM) under multi-modal perturbations, featuring complete perturbation classification, a customizable perturbation synthesis toolkit, and an LLM-based automated evaluation assistant R-Agent 2.
R^2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
Ye Liu (Hong Kong Polytechnic University), Chang Wen Chen (Hong Kong Polytechnic University)
RetrievalRecurrent Neural NetworkTransformerSupervised Fine-TuningContrastive LearningVideoText
🎯 What it does: Proposes a parameter- and memory-efficient image-video transfer learning framework called Reversed Recurrent Tuning (R2-Tuning) for video temporal localization tasks (Moment Retrieval, Highlight Detection, Video Summarization).
R3D-AD: Reconstruction via Diffusion for 3D Anomaly Detection
Zheyuan Zhou (Zhejiang University), Shuyou Zhang (Zhejiang University)
Anomaly DetectionDiffusion modelAuto EncoderPoint Cloud
🎯 What it does: Propose R3D-AD, a 3D anomaly detection framework based on diffusion models, which achieves defect-free reconstruction of abnormal point clouds through complete masking and progressive displacement decoding.
R3DS: Reality-linked 3D Scenes for Panoramic Scene Understanding
Qirui Wu (Simon Fraser University), Angel X Chang
Object DetectionSegmentationDepth EstimationSupervised Fine-TuningImageMeshBenchmark
🎯 What it does: This paper proposes the Reality-linked 3D Scenes (R3DS) dataset, which manually constructs complete 3D indoor scenes based on Matterport3D panoramic images, and provides object annotations, support hierarchies, and matched instance labels for each scene; subsequently, the dataset is used to train and evaluate panoramic scene understanding tasks.
RadEdit: stress-testing biomedical vision models via diffusion image editing
Fernando Pérez-García (Microsoft Health Futures), Maximilian Ilse (Microsoft Health Futures)
GenerationData SynthesisAnomaly DetectionDiffusion modelImageBiomedical Data
🎯 What it does: Propose RadEdit, which utilizes diffusion models to perform local edits on chest X-ray images, generating a synthetic dataset for stress testing medical imaging models.