ECCV 2024 Papers — Page 23
European Conference on Computer Vision · 2387 papers
TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly
Mengqi Guo (National University of Singapore), Gim Hee Lee (National University of Singapore)
OptimizationRepresentation LearningTransformerImageSequential
🎯 What it does: Predict the sequence of assembly steps for 3D LEGO bricks in multi-view images, using a tree-shaped Transformer to accomplish self-supervised sequential assembly tasks.
Tri^{2}-plane: Thinking Head Avatar via Feature Pyramid
Luchuan Song (University of Rochester), Chenliang Xu (University of Rochester)
GenerationSuper ResolutionNeural Radiance FieldGenerative Adversarial NetworkVideo
🎯 What it does: Proposes the Tri-Plane 2 framework, achieving high-fidelity human head avatar reconstruction from short monocular videos using multi-scale tri-plane rendering and geometry-aware sliding windows.
TriNeRFLet: A Wavelet Based Triplane NeRF Representation
Rajaei Khatib (Tel Aviv University), Raja Giryes (Tel Aviv University)
GenerationSuper ResolutionDiffusion modelNeural Radiance FieldImage
🎯 What it does: This paper proposes TriNeRFLet, a Triplane NeRF representation method based on wavelet multi-scale, and applies it to 3D reconstruction and high-resolution view generation.
TrojVLM: Backdoor Attack Against Vision Language Models
Weimin Lyu (Stony Brook University), Chao Chen (Stony Brook University)
Adversarial AttackTransformerSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: Perform backdoor attacks on the image-to-text generation task of vision-language models (VLMs), inserting predefined target text while preserving the original image semantics.
TTD: Text-Tag Self-Distillation Enhancing Image-Text Alignment in CLIP to Alleviate Single Tag Bias
Sanghyun Jo (OGQ), Kyungsu Kim (Massachusetts General Hospital and Harvard Medical School)
Knowledge DistillationTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: Proposes the text-label self-distillation method TTD, which mitigates the single-label bias in the CLIP model and achieves more fair image-text alignment by fine-tuning with only image-text pairs.
TTT-MIM: Test-Time Training with Masked Image Modeling for Denoising Distribution Shifts
Youssef Mansour (Technical University of Munich), Reinhard Heckel (Technical University of Munich)
RestorationDomain AdaptationConvolutional Neural NetworkAuto EncoderImageMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Propose a test-time training method (TTT-MIM) based on masked image modeling for adapting single images under distribution shifts in image denoising.
Tuning-Free Image Customization with Image and Text Guidance
Pengzhi Li (Tsinghua University), Feng Zheng (Tencent Youtu Lab)
Image HarmonizationGenerationTransformerVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Proposed an image customization framework without fine-tuning that can perform fine-grained editing within a specified region by simultaneously utilizing a reference image and text description.
Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models
Chen Ju (Alibaba Group), Bo Zheng (Alibaba Group)
GenerationRetrievalComputational EfficiencyVision Language ModelMultimodality
🎯 What it does: Propose the Turbo plugin, which accelerates Vision-Language Large Models (VLM) by sorting and merging visual and text tokens based on information degree at the attention layer, thereby eliminating redundant tokens at the data level.
TurboEdit: Real-time text-based disentangled real image editing
Zongze Wu (Adobe Research), Eli Shechtman (Adobe Research)
GenerationLarge Language ModelPrompt EngineeringDiffusion modelImageTextMultimodality
🎯 What it does: Propose TurboEdit, an image editing framework that utilizes few-step diffusion models (SDXL-Turbo) for real-time, text-driven, and separable editing;
Two-Stage Active Learning for Efficient Temporal Action Segmentation
Yuhao Su (Northeastern University), Ehsan Elhamifar (Northeastern University)
SegmentationConvolutional Neural NetworkTransformerContrastive LearningVideo
🎯 What it does: Propose a two-stage active learning framework that can train an efficient temporal action segmentation model with only a small number of frame annotations.
Two-Stage Video Shadow Detection via Temporal-Spatial Adaption
Xin Duan (Hong Kong Polytechnic University), Ping Li (Hong Kong Polytechnic University)
SegmentationOptimizationTransformerSupervised Fine-TuningVideo
🎯 What it does: Developed a two-stage video shadow detection framework and constructed a new dataset named CVSD that covers complex and diverse shadows.
U-COPE: Taking a Further Step to Universal 9D Category-level Object Pose Estimation
li zhang, Liu Liu (Hefei Institute of Physical Science, Chinese Academy of Sciences)
Pose EstimationConvolutional Neural NetworkPoint Cloud
🎯 What it does: Propose the U-COPE framework to achieve unified 9D category-level pose estimation for rigid and articulated objects.
UAV First-Person Viewers Are Radiance Field Learners
Liqi Yan (Hangzhou Dianzi University), Dongfang Liu
GenerationData SynthesisNeural Radiance FieldVideo
🎯 What it does: Propose the FPV-NeRF framework for generating first-person perspective images from UAV videos, addressing the issues of spatiotemporal inconsistency, missing global structures, and local details in traditional NeRF under multi-scale and view-limited conditions.
uCAP: An Unsupervised Prompting Method for Vision-Language Models
A. Tuan Nguyen (Meta), Ser-Nam Lim (University Of Central Florida)
ClassificationPrompt EngineeringVision Language ModelImageVideo
🎯 What it does: This paper proposes an unsupervised prompt learning method called uCAP, which can automatically learn domain-specific prompts for vision-language pre-training models such as CLIP without using any labeled data, thereby improving zero-shot classification performance.
UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Prompt
Xin Li (University Of Science And Technology Of China), Zhibo Chen (University Of Science And Technology Of China)
Super ResolutionConvolutional Neural NetworkPrompt EngineeringImageBenchmark
🎯 What it does: Propose a general compressed image super-resolution framework UCIP, which achieves unified processing of various compressed bitstreams through dynamic prompt learning.
UDA-Bench: Revisiting Common Assumptions in Unsupervised Domain Adaptation Using a Standardized Framework
Tarun Kalluri (UC San Diego), Manmohan Chandraker (UC San Diego)
Domain AdaptationTransformerImageBenchmark
🎯 What it does: Proposed the UDA-Bench framework and conducted large-scale empirical studies to investigate the impact of backbone networks, the amount of unlabeled data, and pre-training data on the performance of unsupervised domain adaptation (UDA).
UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Models
Yiming Zhao (Peking University), Zhouhui Lian (Peking University)
GenerationData SynthesisTransformerDiffusion modelContrastive LearningImageText
🎯 What it does: Proposed UDiffText, a unified framework that leverages character-level diffusion models to achieve high-quality text synthesis in arbitrary images, addressing the spelling error issues of existing text-to-image (T2I) models.
UGG: Unified Generative Grasping
Jiaxin Lu (University of Texas at Austin), Gang Hua (Dolby Laboratories)
GenerationPose EstimationRobotic IntelligenceTransformerDiffusion modelAuto EncoderPoint CloudPhysics Related
🎯 What it does: A unified diffusion model named UGG was constructed to generate diverse and high-success-rate full-hand grasping poses under the condition of given object point clouds, while supporting mutual influence between object and hand poses.
UL-VIO: Ultra-lightweight Visual-Inertial Odometry with Noise Robust Test-time Adaptation
Jinho Park (Columbia University), Mingoo Seok (Columbia University)
Domain AdaptationAutonomous DrivingConvolutional Neural NetworkSimultaneous Localization and MappingImageMultimodality
🎯 What it does: This paper proposes an extremely lightweight visual-inertial odometry (UL-VIO) network with less than 1 M parameters, and designs an online test-time adaptation (TTA) mechanism based on batch normalization (BN) parameters, utilizing inertial outputs as pseudo labels to correct domain drift in visual features.
UMBRAE: Unified Multimodal Brain Decoding
Weihao Xia (University College London), Jing-Hao Xue (University College London)
Domain AdaptationRepresentation LearningData-Centric LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningImageMultimodalityBiomedical DataMagnetic Resonance ImagingBenchmark
🎯 What it does: This paper proposes the UMBRAE framework, which maps fMRI signals into a feature space recognizable by multimodal language models through a unified multimodal brain signal decoder, enabling multitask decoding of brain signals for text, images, localization, and other tasks.
UMERegRobust – Universal Manifold Embedding Compatible Features for Robust Point Cloud Registration
Yuval Haitman (Ben-Gurion University), Joseph M Francos
Pose EstimationAutonomous DrivingRepresentation LearningConvolutional Neural NetworkContrastive LearningPoint CloudBenchmark
🎯 What it does: Based on the Universal Manifold Embedding (UME) framework, this paper proposes a UME-compatible feature extraction method. By combining a sampling balance module and UME contrast learning, we construct a robust point cloud registration pipeline (UMERegRobust) without relying on RANSAC. Additionally, we introduce the RotKITTI/RotnuScenes benchmark dataset tailored for large rotation scenarios.
UMG-CLIP: A Unified Multi-Granularity Vision Generalist for Open-World Understanding
Bowen Shi (Shanghai Jiao Tong University), Xiaopeng Zhang (Shanghai Jiao Tong University)
ClassificationSegmentationRetrievalTransformerSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Designed and trained a unified multi-granularity CLIP model named UMG-CLIP, covering image, region, and pixel-level alignment, and built a large-scale multi-granularity dataset UMG-41M through an automated workflow to support multi-task pretraining;
Un-EVIMO: Unsupervised Event-based Independent Motion Segmentation
Ziyun Wang (University of Pennsylvania), Kostas Daniilidis (University of Pennsylvania)
SegmentationConvolutional Neural NetworkOptical FlowVideo
🎯 What it does: Propose Un-EVIMO, an unsupervised event camera independent moving object segmentation framework;
Uncertainty Calibration with Energy Based Instance-wise Scaling in the Wild Dataset
Mijoo Kim (Chung-Ang University), Junseok Kwon (Chung-Ang University)
ClassificationDomain AdaptationImage
🎯 What it does: Propose an instance-level adaptive temperature scaling method based on an energy model for post-hoc uncertainty calibration in multi-class classification tasks, maintaining robustness under distribution shifts (including covariate shift and semantic shift).
Uncertainty-aware sign language video retrieval with probability distribution modeling
Xuan Wu (Sichuan University), Keren Fu (Peking University)
RetrievalConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: Proposes a sign language video retrieval method called UPRet, which represents sign language videos and text as probability distributions and uses optimal transport for matching.
Uncertainty-Driven Spectral Compressive Imaging with Spatial-Frequency Transformer
Lintao Peng (Beijing Institute of Technology), Liheng Bian (Beijing Institute of Technology)
RestorationCompressionTransformerImage
🎯 What it does: This paper proposes a Transformer-based hyperspectral image (HSI) reconstruction method called Specformer, which utilizes a spatial-frequency (SF) module and incorporates an uncertainty-driven adaptive loss to enhance reconstruction quality.
Understanding and Mitigating Human-Labelling Errors in Supervised Contrastive Learning
Zijun Long (University of Glasgow), Paul Henderson (University of Glasgow)
ClassificationData-Centric LearningContrastive LearningImage
🎯 What it does: This paper analyzes the impact of human labeling errors on supervised contrastive learning (SCL) and proposes a new contrastive learning objective, SCL-RHE, specifically designed for robustness improvements against real annotation errors with low noise rates (<5%).
Understanding Multi-compositional learning in Vision and Language models via Category Theory
Sotirios Panagiotis Chytas (University of Wisconsin Madison), Vikas Singh (Korea University)
Representation LearningLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a general framework based on category theory for understanding and enhancing multi-attribute compositional learning in vision-language models (multi-compositional zero-shot learning), and implements an attention-based realization named CatCom; subsequently, the framework is used to evaluate the compositional properties of various large language models in latent spaces.
Understanding Physical Dynamics with Counterfactual World Modeling
Rahul Venkatesh (Stanford), Daniel Yamins (MIT)
TransformerPrompt EngineeringAuto EncoderOptical FlowVideoPhysics Related
🎯 What it does: Propose the Counterfactual World Modeling (CWM) framework, which learns physical dynamics from unlabeled videos using a pre-trained masked autoencoder, and extracts keypoint, optical flow, segmentation, and other visual structures for physics reasoning tasks through 'adversarial prompts' from a single model.
Uni3DL: A Unified Model for 3D Vision-Language Understanding
Xiang Li (King Abdullah University of Science and Technology), Mohamed Elhoseiny (King Abdullah University of Science and Technology)
Object DetectionSegmentationRetrievalConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningTextPoint Cloud
🎯 What it does: Proposed Uni3DL, a unified 3D vision-language understanding model capable of directly processing point clouds and text, supporting multiple tasks ranging from semantic segmentation, instance segmentation, object detection, visual localization, 3D annotation to text-3D cross-retrieval.
UNIC: Universal Classification Models via Multi-teacher Distillation
Yannis Kalantidis (NAVER LABS Europe), Thomas LUCAS
ClassificationKnowledge DistillationTransformerImage
🎯 What it does: Learn a single ViT encoder through multi-teacher distillation, enabling it to achieve general performance across various classification tasks.
UniCal: Unified Neural Sensor Calibration
Ze Yang (Waabi), Raquel Urtasun (Waabi)
Pose EstimationAutonomous DrivingNeural Radiance FieldImageMultimodalityPoint Cloud
🎯 What it does: Propose a unified neural rendering framework called UniCal for automatically calibrating the extrinsic parameters of multi-sensors (LiDAR and cameras) using uncalibrated outdoor driving data;
UniCode : Learning a Unified Codebook for Multimodal Large Language Models
Sipeng Zheng (Beijing Academy of Artificial Intelligence), Zongqing Lu (Peking University)
GenerationRetrievalCompressionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelAuto EncoderImageTextMultimodality
🎯 What it does: Propose UniCode, a unified codebook for vision and text, enabling visual encoding and generation in multimodal large language models; employ language-driven iterative training and context image decompression tasks to enhance visual representations; support stacked quantization compression; finally complete multimodal instruction tuning;
UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation
Zexiang Liu (VAST), Wanli Ouyang (Chinese University of Hong Kong)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelNeural Radiance FieldImagePoint CloudMesh
🎯 What it does: Developed the UniDream framework, integrating a unified multi-view Albedo- and Normal-consistent diffusion model, Transformer reconstruction model, and SDS optimization to generate re-lightable 3D models from text.
Unified Embedding Alignment for Open-Vocabulary Video Instance Segmentation
Hao Fang (Shandong University), Xiankai Lu (Shandong University)
SegmentationTransformerVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: Propose OVFormer, achieving open-vocabulary video instance segmentation via unified embedding alignment and video-level training.
Unified Local-Cloud Decision-Making via Reinforcement Learning
Kathakoli Sengupta (Boston University), Renato Mancuso (Boston University)
Autonomous DrivingReinforcement LearningImage
🎯 What it does: Proposed the UniLCD framework for dynamic inference decision-making in dynamic crowded scenarios, achieving collaboration between local and cloud.
Unified Medical Image Pre-training in Language-Guided Common Semantic Space
Xiaoxuan He, Lili Qiu (Microsoft Research Zhejiang University)
ClassificationSegmentationRetrievalRepresentation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodalityBiomedical DataComputed Tomography
🎯 What it does: Propose UniMedI, a unified vision-language pre-training framework that can simultaneously process 2D X-ray and 3D CT images, mapping them into a shared space guided by medical report semantics.
UniFS: Universal Few-shot Instance Perception with Point Representations
Sheng Jin (University of Hong Kong), Ping Luo (University of Hong Kong)
Object DetectionSegmentationPose EstimationConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: Proposes UniFS, a unified few-shot instance-aware model that integrates four major tasks—object detection, instance segmentation, pose estimation, and counting—through a dynamic point representation learning framework.
Unifying 3D Vision-Language Understanding via Promptable Queries
ziyu zhu, Qing Li (Tsinghua University)
SegmentationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityPoint Cloud
🎯 What it does: Propose a unified 3D vision-language understanding model, PQ3D, which can process multiple scene representations (voxels, point clouds, multi-view images) and multi-task (instance segmentation, visual localization, question answering, dense description, navigation) simultaneously using promptable queries.
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
Cong Wei (University Of Waterloo), Wenhu Chen (Georgia Institute Of Technology)
RetrievalVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: A general-purpose retrieval model named UniIR, which can accept multimodal queries and instructions, was constructed and trained and evaluated on the large-scale multimodal retrieval benchmark M-BEIR.
UNIKD: UNcertainty-Filtered Incremental Knowledge Distillation for Neural Implicit Representation
Mengqi Guo (National University of Singapore), Gim Hee Lee (National University of Singapore)
Knowledge DistillationNeural Radiance FieldImageBenchmark
🎯 What it does: Propose the UNIKD framework, which utilizes a student-teacher network combined with random queries and uncertainty filters to achieve storage-free incremental learning for neural implicit representations (NeRF, MonoSDF).
UniM2AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving
Jian Zou, Wangmeng Zuo (Harbin Institute Of Technology)
Object DetectionSegmentationAutonomous DrivingTransformerAuto EncoderImageMultimodalityPoint Cloud
🎯 What it does: Proposed UniM AE, a multi-modal self-supervised pre-training framework that achieves image and LiDAR fusion and reconstruction through unified 3D volume space and multi-modal 3D interaction modules.
UniMD: Towards Unifying Moment Retrieval and Temporal Action Detection
Yingsen Zeng (Meituan Inc), Lin Ma (Meituan Inc)
Object DetectionRetrievalConvolutional Neural NetworkVision Language ModelVideoTextMultimodality
🎯 What it does: Propose a unified framework UniMD, which uses a single model to simultaneously perform Temporal Action Detection (TAD) and Moment Retrieval (MR), achieving collaborative learning for both tasks through unified queries, shared CLIP text encoder, and query-related decoders.
UniProcessor: A Text-induced Unified Low-level Image Processor
Huiyu Duan (Shanghai Jiao Tong University), Guangtao Zhai (Shanghai Jiao Tong University)
RestorationConvolutional Neural NetworkTransformerSupervised Fine-TuningPrompt EngineeringMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: A unified low-level visual processing framework called UniProcessor based on text induction was constructed, capable of handling 30 types of image degradation (such as noise, blur, rainy weather, compression artifacts, etc.) within a single model, and supporting independent or step-by-step control of different degradations through natural language prompts.
UNIT: Backdoor Mitigation via Automated Neural Distribution Tightening
Siyuan Cheng (Purdue University), Xiangyu Zhang (Purdue University)
Safty and PrivacyAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Propose a post-training defense method called UNIT, which automatically approximates upper bounds of activation distributions for each neuron and trims activations exceeding these bounds during inference, thereby eliminating the impact of backdoor triggers while maintaining model accuracy.
UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model
Xiangyu Fan (SenseTime Research), Lei Yang (SenseTime Research)
GenerationTransformerVideoMultimodalityMeshBenchmarkAudio
🎯 What it does: Proposed a unified multi-head audio-driven 3D facial animation model called UniTalker, which can learn from multiple differently annotated datasets.
UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction
Lan Feng (EPFL), Alexandre Alahi (EPFL)
Autonomous DrivingData-Centric LearningTime SeriesSequentialBenchmark
🎯 What it does: Unify multi-vehicle trajectory prediction datasets, models, and evaluation methods, and study the impact of cross-domain generalization and data scale on performance within this framework.
UniVoxel: Fast Inverse Rendering by Unified Voxelization of Scene Representation
Shuang Wu (Harbin Institute of Technology), Wenjie Pei (Harbin Institute of Technology)
OptimizationComputational EfficiencyNeural Radiance FieldGaussian SplattingImage
🎯 What it does: Proposes a unified voxelization framework, UniVoxel, which explicitly learns scene geometry (SDF), material (albedo, roughness), and illumination (local lighting field) through a lightweight MLP that jointly infers them in voxel space.
Unleashing Text-to-Image Diffusion Prior for Zero-Shot Image Captioning
Jianjie Luo (Beijing Institute Of Technology), Ting Yao (Beijing Institute Of Technology)
GenerationTransformerVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageMultimodality
🎯 What it does: This paper proposes the PatchMix data augmentation method and applies it to vision-language generation tasks;
Unleashing the Potential of the Semantic Latent Space in Diffusion Models for Image Dehazing
Zizheng Yang (University of Science and Technology of China), Feng Zhao (University of Science and Technology of China)
RestorationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: This paper analyzes the evolution of the semantic latent space (h-space) of pre-trained diffusion models across time steps, proposing the DiffLI-D network that achieves efficient dehazing by leveraging the content and fog features in h-space, without retraining the diffusion model.
Unleashing the Power of Prompt-driven Nucleus Instance Segmentation
Zhongyi Shui (Zhejiang University), Lin Yang (Westlake University)
SegmentationTransformerSupervised Fine-TuningPrompt EngineeringImageBiomedical Data
🎯 What it does: Designed the PromptNucSeg framework, achieving nuclear instance segmentation without post-processing by utilizing an automated point prompter and fine-tuned SAM.
Unlocking Attributes' Contribution to Successful Camouflage: A Combined Textual and Visual Analysis Strategy
Hong Zhang (Beihang University), Yifan Yang (Beihang University)
SegmentationTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Investigated the impact of camouflage attributes on segmentation performance in concealed object segmentation (COS), constructed the first COD-TAX dataset containing image descriptions and attribute contributions, and proposed the ACUMEN framework that integrates textual and visual information.
Unlocking Textual and Visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Image
Pengkun Jiao (Fudan University), Yu-Gang Jiang (Fudan University)
Object DetectionTransformerVision Language ModelContrastive LearningImageTextPoint Cloud
🎯 What it does: Proposes an image-guided novel category discovery and hierarchical feature space alignment framework INHA, leveraging a vision-language foundation model, significantly improving recall and localization performance in open-vocabulary 3D object detection.
Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents
Yuqi Jia (Duke University), Yiran Chen (Duke University)
Data SynthesisFederated LearningGenerative Adversarial NetworkImage
🎯 What it does: Propose a server-side data distillation framework FedDGM, allowing clients to train only small proxy models, while the server utilizes a pre-trained deep generative model (StyleGAN-XL) and matching training trajectory techniques to generate synthetic data and train a larger global model on this data.
Unmasking Bias in Diffusion Model Training
Hu Yu (University of Science and Technology of China), Feng Zhao (Alibaba Group)
GenerationDiffusion modelImage
🎯 What it does: This paper systematically analyzes the bias caused by the commonly used constant weight loss in diffusion model training, and proposes a weight strategy based on the inverse square root of the signal-to-noise ratio, significantly improving sample quality and training/sampling efficiency.
Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video Enhancement
Lingyu Zhu, Shiqi Wang (City University of Hong Kong)
RestorationConvolutional Neural NetworkOptical FlowVideo
🎯 What it does: Proposed an unsupervised learning-based low-light video enhancement framework called UDU-Net, which expands the maximum a posteriori (MAP) problem into a trainable deep network, optimizing spatial (Intra) and temporal (Inter) visual priors in separate stages;
Unsqueeze [CLS] Bottleneck to Learn Rich Representations
Qing Su (Georgia State University), Shihao Ji (Georgia State University)
Knowledge DistillationRepresentation LearningTransformerImage
🎯 What it does: Propose the UDI (Unsqueeze [CLS] Bottleneck) method, leveraging self-distillation and multi-scale target distributions in Vision Transformers to enhance the diversity and richness of visual representations.
Unsupervised Dense Prediction using Differentiable Normalized Cuts
Yanbin Liu (Auckland University of Technology), Stephen Gould (Australian National University)
SegmentationTransformerAuto EncoderContrastive LearningImageBenchmark
🎯 What it does: Design a differentiable Normalized Cuts layer and utilize self-supervised mask consistency loss to perform fine-grained feature fine-tuning on pre-trained ViT, enhancing unsupervised dense prediction performance.
Unsupervised Exposure Correction
Ruodai Cui (Qualcomm Technologies, Inc.), Guosheng Hu (University of Bristol)
RestorationConvolutional Neural NetworkImage
🎯 What it does: Proposes a fully unsupervised exposure correction framework that utilizes images from multi-exposure sequences to mutually serve as training targets, achieving pixel-level exposure adjustment.
Unsupervised Moving Object Segmentation with Atmospheric Turbulence
Dehao Qin, Nianyi Li (George Mason University)
SegmentationConvolutional Neural NetworkOptical FlowVideo
🎯 What it does: Propose an unsupervised moving object segmentation framework that can identify and segment all moving targets in videos disturbed by atmospheric turbulence.
Unsupervised Multi-modal Medical Image Registration via Invertible Translation
Mengjie Guo (University of Birmingham)
Image TranslationConvolutional Neural NetworkFlow-based ModelMultimodalityBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Propose an unsupervised multimodal medical image registration method called INNReg, which achieves image modality conversion through invertible neural networks and performs registration using the converted monomodal images.
Unsupervised Representation Learning by Balanced Self Attention Matching
Daniel Shalam (University of Haifa), Simon Korman (University of Haifa)
ClassificationObject DetectionSegmentationRepresentation LearningTransformerContrastive LearningImageVideo
🎯 What it does: Proposes an unsupervised representation learning method called BAM, which learns image features by matching self-attention distributions across different views.
Unsupervised Variational Translator for Bridging Image Restoration and High-Level Vision Tasks
Jiawei Wu (Sun Yat-sen University), Zhi Jin (Guangdong Provincial Key Laboratory of Fire Science and Technology)
ClassificationRestorationObject DetectionTransformerImage
🎯 What it does: Propose an unsupervised variational translator (VaT) that bridges restored results with high-level visual tasks input through a lightweight network, without requiring retraining existing image restoration networks and high-level visual networks, thereby enhancing the performance of low-quality images on tasks such as object detection and classification.
Unveiling Advanced Frequency Disentanglement Paradigm for Low-Light Image Enhancement
Kun Zhou (CUHK-Shenzhen), Jiangbo Lu (SmartMore Corporation)
RestorationConvolutional Neural NetworkImage
🎯 What it does: Proposed a general frequency decoupling optimization paradigm for low-light image enhancement, which can improve performance by adding only a small number of parameters to existing models.
Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention
Jie Ren (Michigan State University), Jiliang Tang (Michigan State University)
GenerationTransformerDiffusion modelImageText
🎯 What it does: This paper investigates the memorization phenomenon in text-to-image diffusion models, finding significant differences in cross-attention distribution between memorized and non-memorized samples. Based on this, it proposes a method to detect memorization using only attention entropy and summary word attention, while mitigating memorization without performance loss by adjusting attention dispersion during both inference and training phases.
Unveiling Privacy Risks in Stochastic Neural Networks Training: Effective Image Reconstruction from Gradients
Yiming Chen (Vrije Universiteit Brussel), Nikos Deligiannis (Vrije Universiteit Brussel)
RestorationFederated LearningSafty and PrivacyImage
🎯 What it does: This paper studies gradient inversion attacks on spiking neural networks (SNN) in federated learning and proposes the ISG attack method that can reconstruct training data from gradients.
Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Models
Hao Cheng (Hong Kong University of Science and Technology), Renjing Xu (Hong Kong University of Science and Technology)
Explainability and InterpretabilityAdversarial AttackLarge Language ModelPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: Systematically evaluate and mitigate the vulnerability of large vision-language models (LVLM) to typographic attacks; propose the largest-scale typographic dataset TypoD, and experimentally verify the mechanism by which typographic attacks affect LVLM; further propose methods to reduce the attack's impact through enriched text prompts and cross-modal attention matching.
UpFusion: Novel View Diffusion from Unposed Sparse View Observations
Bharath Raj Nagoor Kani (Carnegie Mellon University), Shubham Tulsiani (Carnegie Mellon University)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelImage
🎯 What it does: Proposed a system called UpFusion that can perform novel view synthesis and infer 3D representations from sparse, unposed image sets without camera pose information.
UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues
Vandad Davoodnia (Queen's University), Ali Etemad (Ubisoft LaForge)
Pose EstimationTransformerFlow-based ModelImagePoint Cloud
🎯 What it does: UPose3D proposes a multi-camera based 3D human pose estimation framework that performs pose reasoning by leveraging the uncertainty of 2D keypoints and cross-view, temporal information.
Upper-body Hierarchical Graph for Skeleton Based Emotion Recognition in Assistive Driving
Jiehui Wu (University of Science and Technology Beijing), Huimin Ma (University of Science and Technology Beijing)
ClassificationRecognitionAutonomous DrivingGraph Neural NetworkGraph
🎯 What it does: Proposed the Upper-body Hierarchical Graph Convolutional Network (UbH-GCN), achieving emotion recognition in ADAS (Advanced Driver Assistance Systems) scenarios using upper-body skeletal sequences.
Urban Waterlogging Detection: A Challenging Benchmark and Large-Small Model Co-Adapter
Suqi Song (Chongqing University), Lei Zhang (Huawei Noah's Ark Lab)
SegmentationConvolutional Neural NetworkTransformerPrompt EngineeringImageBenchmark
🎯 What it does: This paper addresses the challenge of urban flood detection by constructing the UW-Bench dataset and proposing the Large-Small Model Co-Adapter framework.
URS-NeRF: Unordered Rolling Shutter Bundle Adjustment for Neural Radiance Fields
Bo Xu (National University of Singapore), Gim Hee Lee (Wuhan University)
GenerationPose EstimationNeural Radiance FieldImage
🎯 What it does: A NeRF training method for unordered rolling shutter images was studied, combining the rolling shutter camera model with bundle adjustment to achieve high-quality 3D representations and camera motion estimation.
Using My Artistic Style? You Must Obtain My Authorization
Xiuli Bi (Chongqing University of Posts and Telecommunications), Bin Xiao (Chongqing University of Posts and Telecommunications)
Image TranslationSafty and PrivacyAdversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: Propose an adversarial perturbation-based artistic style protection scheme (ASPS) that reduces the style transfer effectiveness of unauthorized models without compromising the output quality of authorized models.
V-IRL: Grounding Virtual Intelligence in Real Life
Jihan Yang (University of Hong Kong), Saining Xie (New York University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningAgentic AIVision-Language-Action ModelImageTextMultimodality
🎯 What it does: Proposed and implemented the VIRL platform, enabling AI agents to navigate, perceive, and interact in a virtual environment based on real-world geographical locations and street view images, with demonstrations of multiple task examples;
V-Trans4Style: Visual Transition Recommendation for Video Production Style Adaptation
Pooja Guhan (University of Maryland), Dinesh Manocha (University of Maryland)
Recommendation SystemTransformerVideo
🎯 What it does: Designed V-Trans4Style, a visual transition recommendation algorithm for video style adaptation.
V2X-Real: a Largs-Scale Dataset for Vehicle-to-Everything Cooperative Perception
Hao Xiang (University of California Los Angeles), Jiaqi Ma (University of California Los Angeles)
Autonomous DrivingImageMultimodalityPoint CloudBenchmark
🎯 What it does: Created a large-scale real-world V2X cooperative perception dataset called V2X-Real, providing four sub-datasets to support different collaboration modes such as vehicle-centric, infrastructure-centric, V2V, and I2I.
Vamos: Versatile Action Models for Video Understanding
Shijie Wang (Brown University), Chen Sun (Brown University)
RecognitionExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVision-Language-Action ModelVideoTextMultimodality
🎯 What it does: Proposed a multi-task video understanding framework called Vamos, which utilizes general video captions as interpretable text representations and combines large language models (LLMs) for action prediction and video question answering; simultaneously designed a Token Bottleneck Model (TBM) to select key information through hard attention, enhancing interpretability and inference speed.
Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models
Haoran Wei (MEGVII Technology), Xiangyu Zhang (Huazhong University of Science and Technology)
TransformerVision Language ModelImageTextMultimodality
🎯 What it does: Proposed a method called Vary to expand the visual vocabulary of large vision-language models (LVLM), aiming to improve performance on specific tasks, especially those requiring fine-grained perception.
VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturing
Shang Liu (DAMO Academy Alibaba Group), Fan Wang (DAMO Academy Alibaba Group)
RestorationGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelImageTextMesh
🎯 What it does: Developed a 3D-2D collaborative denoising framework called VCD-Texture based on variance alignment, which utilizes a pre-trained diffusion model to achieve high-fidelity, view-consistent 3D texture synthesis, and further designs mutation alignment and pixel-level repair strategies;
VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentation
Zhen Qu (Chinese Academy of Sciences), Guiguang Ding (Tsinghua University)
Anomaly DetectionPrompt EngineeringVision Language ModelContrastive LearningImage
🎯 What it does: Propose VCP-CLIP, a vision context prompt model based on CLIP for zero-shot anomaly segmentation.
VeCLIP: Improving CLIP Training via Visual-enriched Captions
Zhengfeng Lai (University of California, Davis), Meng Cao (Apple AI/ML)
ClassificationRetrievalTransformerLarge Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Built an expandable visual rich re-annotation (VeCap) pipeline, which extracts visual concepts from AltText crawled from the web using a multi-modal LLM, then fuses the original AltText with these visual concepts through an open-source LLM to generate new 'visual rich captions' (VeCap), followed by a hybrid training scheme VeCLIP pre-trained on CLIP.
VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors
Sungwon Hwang (KAIST), Jaegul Choo (KAIST)
GenerationData SynthesisDiffusion modelNeural Radiance FieldGaussian SplattingImageVideoPoint Cloud
🎯 What it does: Investigated the problem of Extrapolated View Synthesis (EVS) in urban scenes using images captured by forward-facing cameras for 3D reconstruction, proposing a dynamic scene modeling approach based on 3D Gaussian Splatting, and enhancing the rendering quality for unseen views by leveraging LiDAR, surface normal priors, and large-scale diffusion models.
VEON: Vocabulary-Enhanced Occupancy Prediction
Jilai Zheng (Shanghai Jiao Tong University), Chao Ma (Shanghai Jiao Tong University)
SegmentationDepth EstimationAutonomous DrivingConvolutional Neural NetworkTransformerVision Language ModelImagePoint CloudTabular
🎯 What it does: Leverages pre-trained 2D base models MiDaS and CLIP with lightweight adapters to achieve open-vocabulary 3D occupancy prediction with no or few labels;
Versatile Incremental Learning: Towards Class and Domain-Agnostic Incremental Learning
Min-Yeong Park (Kyung Hee University), Gyeong-Moon Park (Kyung Hee University)
ClassificationDomain AdaptationKnowledge DistillationTransformerImageBenchmark
🎯 What it does: Propose a more general incremental learning scenario VIL (which can be class-incremental, domain-incremental, or both simultaneously), and design the ICON framework (comprising CAST regularization and IC dynamic classifier expansion) to address internal class-domain confusion and cross-domain class confusion in class-domain hybrid incremental learning.
VersatileGaussian: Real-time Neural Rendering for Versatile Tasks using Gaussian Splatting
Renjie Li (Tsinghua University), Xi Wu (University of Texas at Austin)
SegmentationGenerationGaussian SplattingImagePoint Cloud
🎯 What it does: A framework for real-time rendering of multi-task (MT) labels using 3D Gaussian Splatting, capable of generating high-quality RGB, semantic segmentation, normal, edge, keypoint, and other labels from arbitrary viewpoints;
VETRA: A Dataset for Vehicle Tracking in Aerial Imagery - New Challenges for Multi-Object Tracking
Jens Hellekes (German Aerospace Center), Franz Kurz (German Aerospace Center)
Object DetectionObject TrackingConvolutional Neural NetworkOptical FlowImageVideoBenchmark
🎯 What it does: Proposed a new dataset for aerial image vehicle multi-object tracking called VETRA, benchmarked various online tracking algorithms on this dataset, and further improved DeepSORT to form DeepSR-SORT.
VF-NeRF: Viewshed Fields for Rigid NeRF Registration
Leo Segre (Tel Aviv University), Shai Avidan (Tel Aviv University)
GenerationPose EstimationOptimizationSupervised Fine-TuningFlow-based ModelNeural Radiance FieldImageVideoPoint Cloud
🎯 What it does: Propose a rigid NeRF registration method based on Viewshed Fields (VF), which can align two NeRFs in 6-DoF without knowing the original camera poses and generate new views with high information content.
VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
Junlin Han (Meta), Philip Torr (Meta)
GenerationData SynthesisSupervised Fine-TuningDiffusion modelImageVideoText
🎯 What it does: Proposed a model called VFusion3D that utilizes a pre-trained video diffusion model to generate synthetic multi-view data, which is then used to train a model capable of rapidly generating high-quality 3D assets from a single image.
ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders
Jefferson Hernandez (Rice University), Vicente Ordonez (Rice University)
Representation LearningTransformerAuto EncoderContrastive LearningImageVideo
🎯 What it does: Propose ViC-MAE, a self-supervised video and image representation learning method that combines Masked Autoencoder with contrastive learning.
Video Editing via Factorized Diffusion Distillation
Uriel Singer (Meta AI), Yaniv Taigman (Meta AI)
GenerationKnowledge DistillationDiffusion modelScore-based ModelVideoText
🎯 What it does: Proposed the EVE model, which integrates image editing Adapter and video generation Adapter through unsupervised Factorized Diffusion Distillation, achieving text-guided video editing;
Video Question Answering with Procedural Programs
Rohan Choudhury (Carnegie Mellon University), Laszlo A Jeni
Object DetectionObject TrackingGenerationRetrievalExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityRetrieval-Augmented GenerationChain-of-ThoughtAudio
🎯 What it does: Propose the ProViQ method, which generates and executes Python programs via large language models (LLM), utilizing video-specific visual modules (retrieval, captioning, tracking, summarization, etc.) to answer video questions.
VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding
Yue Fan (State Key Laboratory of General Artificial Intelligence, BIGAI), Qing Li (State Key Laboratory of General Artificial Intelligence, BIGAI)
Object TrackingLarge Language ModelAgentic AIVision Language ModelVideoTextMultimodalityBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: A multi-modal agent named VideoAgent based on a unified memory mechanism was constructed for long video understanding. The agent first splits videos into short clips and generates event descriptions (temporal memory) while tracking and re-identifying objects in the video (object memory). Subsequently, the agent interactively completes tasks by invoking various tools (e.g., subtitle retrieval, clip localization, visual question answering, object memory querying) through an LLM.
VideoAgent: Long-form Video Understanding with Large Language Model as Agent
Xiaohan Wang (Stanford University), Serena Yeung-Levy (Stanford University)
RetrievalTransformerLarge Language ModelAgentic AIVision Language ModelVideoRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose VideoAgent, which leverages a large language model as an agent to interactively retrieve and integrate key frames from long videos, enabling long-term video understanding.
VideoClusterNet: Self-Supervised and Adaptive Face Clustering for Videos
Devesh Walawalkar (Flawless AI), Pablo Garrido (Flawless AI)
RecognitionObject DetectionObject TrackingSupervised Fine-TuningContrastive LearningVideoBenchmark
🎯 What it does: Propose a self-supervised model fine-tuning and parameter-free hierarchical clustering algorithm for clustering faces with variations in pose, illumination, and expressions in videos.
VideoMamba: Spatio-Temporal Selective State Space Model
Jinyoung Park (Korea Advanced Institute of Science and Technology), Changick Kim (Korea Advanced Institute of Science and Technology)
RecognitionComputational EfficiencyVideoBenchmark
🎯 What it does: This paper proposes VideoMamba, an efficient video recognition framework based on the Mamba pure state space model, which processes spatiotemporal information in videos by implementing forward and backward scanning along the temporal dimension and combining spatial-temporal bidirectional SSM.
VideoMamba: State Space Model for Efficient Video Understanding
Kunchang Li (Shenzhen Institute of Advanced Technology Chinese Academy of Sciences), Yu Qiao (OpenGVLab Shanghai AI Laboratory State Key Laboratory for Novel Software Technology Nanjing University)
ClassificationRecognitionRetrievalRecurrent Neural NetworkVideo
🎯 What it does: Proposed a fully state-space model-based video understanding framework called VideoMamba, which simultaneously handles short-term action recognition and long-term video reasoning.
Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion
Xiang Fan, Ranjay Krishna (University Of Washington)
GenerationDiffusion modelVideo
🎯 What it does: Propose Videoshop, a no-training method that directly edits the first frame to achieve localized semantic video editing
VideoStudio: Generating Consistent-Content and Multi-Scene Videos
Fuchen Long (HiDream.ai Inc), Tao Mei (HiDream.ai Inc)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageVideoTextMultimodality
🎯 What it does: Designed and implemented the VideoStudio framework, which generates multi-scenario scripts using a large language model, generates entity reference images, and produces visually consistent multi-scenario videos through an improved diffusion model.
View Selection for 3D Captioning via Diffusion Ranking
Tiange Luo, Honglak Lee (University Of Michigan)
GenerationVision Language ModelDiffusion modelImageTextMultimodalityMesh
🎯 What it does: Proposed a view ranking method called DiffuRank based on diffusion models, which selects the most representative views of 3D objects in multi-view rendering to generate more accurate and detailed text descriptions.
View-Consistent 3D Editing with Gaussian Splatting
Yuxuan Wang, Hanwang Zhang (Nanyang Technological University)
GenerationDiffusion modelGaussian SplattingMultimodality
🎯 What it does: Proposes VcEdit, a 3D Gaussian Splatting editing framework that achieves text-driven local editing of 3D models while maintaining multi-view consistency.