arXivSub Start free trial

ICCV 2023 Papers with Code β€” Page 7

IEEE/CVF International Conference on Computer Vision Β· 743 papers

Spatial-Aware Token for Weakly Supervised Object Localization

Pingyu Wu (University of Science and Technology of China), Zheng-Jun Zha (University of Science and Technology of China)

CodeObject DetectionSegmentationTransformerImage

🎯 What it does: This paper proposes a weakly supervised object localization method based on Transformer, utilizing Spatial-Aware Tokens (SAT) to directly generate localization maps through spatial query attention, avoiding optimization conflicts between classification and localization tasks.

Spatially and Spectrally Consistent Deep Functional Maps

Mingze Sun (Tsinghua University), Ruqi Huang (Tsinghua University)

CodeDiffusion modelPoint CloudMesh

🎯 What it does: Proposes a two-branch unsupervised deep feature mapping (DFM) framework that integrates spectral domain and spatial domain cyclic consistency to enhance non-rigid shape correspondence.

Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution

Long Sun (Nanjing University of Science and Technology), Jinshan Pan (Nanjing University of Science and Technology)

CodeRestorationSuper ResolutionConvolutional Neural NetworkImage

🎯 What it does: A lightweight single-image super-resolution network named SAFMN is proposed for efficient super-resolution on low-power devices.

Spatio-temporal Prompting Network for Robust Video Feature Extraction

Guanxiong Sun (Queen's University Belfast), Yang Hua (Queen's University Belfast)

CodeObject DetectionObject TrackingSegmentationTransformerPrompt EngineeringVideo

🎯 What it does: A unified Spatio-Temporal Prompt Network (STPN) is proposed, achieving robust video feature extraction by injecting dynamic video prompts at the front end of the Transformer, eliminating complex backend integration modules.

Spectrum-guided Multi-granularity Referring Video Object Segmentation

Bo Miao (University of Western Australia), Ajmal Mian (University of Western Australia)

CodeObject DetectionSegmentationTransformerVideo

🎯 What it does: A spectrum-guided multi-granularity R-VOS framework SgMg is proposed to address the feature drift problem and achieve single-frame/multi-frame/multi-object segmentation.

Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video

Xiuzhe Wu (University of Hong Kong), Xiaojuan Qi (University of Hong Kong)

CodeGenerationData SynthesisContrastive LearningVideo

🎯 What it does: A Speech2Lip framework is proposed, which generates high-fidelity, synchronized speaker videos from just a few minutes of video by separating speech-sensitive and speech-insensitive motions, utilizing implicit models, geometric mapping, hybrid networks, and contrastive synchronization loss.

SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes

Yutao Cui (Nanjing University), Limin Wang (Nanjing University)

CodeObject DetectionObject TrackingTransformerVideoBenchmark

🎯 What it does: A large-scale multi-object tracking dataset, SportsMOT, covering basketball, volleyball, and soccer scenarios has been constructed, and the MixSort framework has been proposed to enhance tracking performance.

Spurious Features Everywhere - Large-Scale Detection of Harmful Spurious Features in ImageNet

Yannic Neuhaus (TΓΌbingen AI Center University of TΓΌbingen), Matthias Hein (TΓΌbingen AI Center University of TΓΌbingen)

CodeClassificationObject DetectionAdversarial AttackConvolutional Neural NetworkImage

🎯 What it does: This paper systematically discovers and verifies harmful interference features on ImageNet using class-level neural PCA methods and visualization techniques, and proposes the SpuFix correction scheme, which reduces the dependency of any ImageNet classifier on these interference features without the need for additional annotations or retraining.

SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking

Zilin Fang (National University of Singapore), Radu Timofte (ETH Zurich)

CodeClassificationConvolutional Neural NetworkTransformerImageBenchmark

🎯 What it does: The SQAD dataset is proposed, which conducts six quality assessments (resolution, color accuracy, noise, dynamic range, PSF, and aliasing) based on laboratory measurements for 29 smartphone cameras, and trains deep models to achieve automatic camera quality assessment and device identification.

SSB: Simple but Strong Baseline for Boosting Performance of Open-Set Semi-Supervised Learning

Yue Fan (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)

CodeClassificationObject DetectionSupervised Fine-TuningImage

🎯 What it does: A simple yet powerful baseline (SSB) is proposed in open-set semi-supervised learning, achieving improvements in both in-class classification and outlier detection through three main strategies: high-confidence pseudo-labeling, nonlinear feature projection heads, and pseudo-negative sample mining.

SSDA: Secure Source-Free Domain Adaptation

Sabbir Ahmed (Binghamton University), Adnan Siraj Rakin (North Carolina State University)

CodeDomain AdaptationAdversarial AttackImage

🎯 What it does: This study investigates the risks of backdoor attacks in source-free domain adaptation (SFDA) and proposes a secure training scheme called SSDA.

STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videos

Anshul Shah (Johns Hopkins University), Rama Chellappa (Johns Hopkins University)

CodeTransformerContrastive LearningVideoMultimodality

🎯 What it does: Proposes an unsupervised method that uses self-supervised contrastive learning to automatically extract key steps from unlabeled instructional videos for AR-assisted training.

Story Visualization by Online Text Augmentation with Context Memory

Daechul Ahn (Yonsei University), Jonghyun Choi (Yonsei University)

CodeGenerationData SynthesisTransformerImageTextMultimodality

🎯 What it does: A story visualization framework based on a bidirectional Transformer is proposed, utilizing a context memory module and online text augmentation technology to generate coherent and semantically consistent image sequences from paragraphs.

Strip-MLP: Efficient Token Interaction for Vision MLP

Guiping Cao (Southern University of Science and Technology), Jianguo Zhang (Southern University of Science and Technology)

CodeClassificationConvolutional Neural NetworkTransformerImage

🎯 What it does: The Strip-MLP model is proposed, which enhances token interaction through three mechanisms: the Strip MLP layer, CGSMM, and LSMM, thereby improving the performance of visual MLPs in image classification tasks.

Strivec: Sparse Tri-Vector Radiance Fields

Quankai Gao (University of Southern California), Zexiang Xu

CodeGenerationData SynthesisOptimizationNeural Radiance FieldPoint Cloud

🎯 What it does: A sparse tri-vector radiance field (Strivec) is proposed as a neural representation of 3D scenes, utilizing local sparse 3D tensor grids and multi-scale CP decomposition to efficiently model geometry and appearance;

Structure and Content-Guided Video Synthesis with Diffusion Models

Patrick Esser (Runway), Anastasis Germanidis (Runway)

CodeGenerationData SynthesisDepth EstimationDiffusion modelImageVideo

🎯 What it does: This paper proposes a structure and content-guided video synthesis method based on a latent video diffusion model, which can edit video content according to text or image descriptions while maintaining the original video structure.

Studying How to Efficiently and Effectively Guide Models with Explanations

Sukrut Rao (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)

CodeClassificationObject DetectionOptimizationExplainability and InterpretabilityConvolutional Neural NetworkTransformerImage

🎯 What it does: The study enhances the ability of multi-label classification models to focus on the features of target objects and improves generalization performance by utilizing interpretive information to guide the models.

StyleDomain: Efficient and Lightweight Parameterizations of StyleGAN for One-shot and Few-shot Domain Adaptation

Aibek Alanov (Higher School of Economics), Dmitry Vetrov (Higher School of Economics)

CodeGenerationDomain AdaptationConvolutional Neural NetworkGenerative Adversarial NetworkImage

🎯 What it does: This study investigates the domain adaptation process of StyleGAN, systematically analyzing which network modules are sufficient for adaptation to domains of varying similarity, and proposes several lightweight parameterization schemes (StyleDomain direction, StyleSpaceSparse, Affine+ and AffineLight+) to achieve few-shot domain adaptation while supporting domain mixing and transfer.

StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generation

Yuhan Wang (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)

CodeGenerationData SynthesisGenerative Adversarial NetworkVideo

🎯 What it does: A non-autoregressive StyleInV framework is proposed, utilizing the inverse network of StyleGAN to generate motion latent variables through temporal style modulation, thereby achieving unconditional generation of long-sequence high-resolution videos.

StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model

Zipeng Xu (University of Trento), Nicu Sebe (University of Trento)

CodeImage TranslationGenerationTransformerReinforcement LearningImageText

🎯 What it does: This paper proposes a language-guided style transfer method called Styler DALLE, which utilizes a large pre-trained vector quantization tokenizer and CLIP to generate stylized images that conform to text descriptions by translating discrete tokens of content images through non-autoregressive translation.

SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets

Cody Simons (University of California), Amit K. Roy-Chowdhury (University of California)

CodeDomain AdaptationAutonomous DrivingImageMultimodalityPoint Cloud

🎯 What it does: Under the Source-Free Unsupervised Multimodal Adaptation (SUMMIT) framework, adaptive learning is performed on unlabelled paired multimodal data in the target domain using unimodal models independently trained in the source domain.

Supervised Homography Learning with Realistic Dataset Generation

Hai Jiang (Sichuan University), Shuaicheng Liu (University of Electronic Science and Technology of China)

CodeImage TranslationData SynthesisConvolutional Neural NetworkGenerative Adversarial NetworkImage

🎯 What it does: An iterative framework is proposed that utilizes unlabeled real image pairs to generate training data that meets both labeling and realism criteria through the use of a dominant surface mask and estimated homography, and these data are used to supervise the training of the homography network.

SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detection

Yiran Qin (Chinese University of Hong Kong), Ruimao Zhang (NIO)

CodeObject DetectionAutonomous DrivingKnowledge DistillationImagePoint Cloud

🎯 What it does: A supervised LiDAR-Camera 3D detection training strategy named SupFusion is proposed, which enhances detection accuracy through auxiliary feature supervision and a deep fusion module.

Surface Extraction from Neural Unsigned Distance Fields

Congyi Zhang (University of Hong Kong), Wenping Wang (Texas A&M University)

CodePoint CloudMesh

🎯 What it does: The DualMesh-UDF method is proposed for extracting high-quality mesh surfaces from neural unsigned distance fields.

SurroundOcc: Multi-camera 3D Occupancy Prediction for Autonomous Driving

Yi Wei (Tsinghua University), Jiwen Lu (Tsinghua University)

CodeSegmentationAutonomous DrivingConvolutional Neural NetworkImagePoint Cloud

🎯 What it does: A method called SurroundOcc is proposed, which utilizes multi-camera RGB images to predict dense 3D occupancy and generates dense occupancy labels through the stitching of multiple frames of LiDAR, Poisson reconstruction, and NN assignment.

SuS-X: Training-Free Name-Only Transfer of Vision-Language Models

Vishaal Udandarao (University of Cambridge), Samuel Albanie (University of Cambridge)

CodeClassificationRetrievalTransformerPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageTextMultimodality

🎯 What it does: This paper proposes the SuS-X framework, which enables the transfer of visual-language models without training, relying solely on category names.

SVDFormer: Complementing Point Cloud via Self-view Augmentation and Self-structure Dual-generator

Zhe Zhu (Nanjing University of Aeronautics and Astronautics), Mingqiang Wei (Hong Kong Polytechnic University)

CodeRestorationGenerationTransformerPoint Cloud

🎯 What it does: A point cloud completion network named SVDFormer is proposed, which utilizes self-projected multi-view depth maps to achieve global shape understanding and reconstructs fine-grained local details through a self-structured dual generator.

SwinLSTM: Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTM

Song Tang (Hainan University), RongNian Tang (Hainan University)

CodeRecurrent Neural NetworkTransformerVideoTime Series

🎯 What it does: A new temporal recursive unit called SwinLSTM is proposed, and a complete spatiotemporal prediction network is constructed based on this unit for future frame prediction.

SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-Time Performance on Mobile Device

Weiran Gou (State Key Laboratory of Mobile Network and Mobile Multimedia Technology), Ke Xu (State Key Laboratory of Mobile Network and Mobile Multimedia Technology)

CodeSuper ResolutionConvolutional Neural NetworkImage

🎯 What it does: Designed and implemented SYENet, a lightweight multi-task network with only 6K parameters, capable of real-time image signal processing (ISP), super-resolution (SR), and low-light enhancement (LLE) on mobile devices at 2K60FPS.

TAPIR: Tracking Any Point with Per-Frame Initialization and Temporal Refinement

Carl Doersch (Google DeepMind), Andrew Zisserman (University College London)

CodeObject TrackingConvolutional Neural NetworkVideo

🎯 What it does: A two-stage point tracking model called TAPIR is proposed for high-precision tracking of arbitrary query points in videos, capable of handling occlusions and long-term tracking.

Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation

Jianan Fan (University of Sydney), Weidong Cai (University of Sydney)

CodeSegmentationDomain AdaptationOptimizationKnowledge DistillationImageBiomedical Data

🎯 What it does: An optimization trajectory distillation-based cross-domain adaptation framework is proposed, which can simultaneously address the issues of data distribution shift and label set inconsistency in medical images.

TCOVIS: Temporally Consistent Online Video Instance Segmentation

Junlong Li (Tsinghua University), Jiwen Lu (Tsinghua University)

CodeObject DetectionSegmentationTransformerVideo

🎯 What it does: This paper proposes an online video instance segmentation method TCOVIS, which enhances temporal consistency through global instance allocation and a spatiotemporal enhancement module, enabling real-time inference by directly propagating queries between frames.

Tem-Adapter: Adapting Image-Text Pretraining for Video Question Answer

Guangyi Chen (Carnegie Mellon University), Yansong Tang (Tsinghua University)

CodeRepresentation LearningTransformerVision Language ModelVideoTextMultimodality

🎯 What it does: Adapting the image-text pre-training model CLIP to the video question-answering task by adding a visual Temporal Aligner and a text Semantic Aligner.

Test Time Adaptation for Blind Image Quality Assessment

Subhadeep Roy (Indian Institute of Science), Rajiv Soundararajan (Indian Institute of Science)

CodeDomain AdaptationContrastive LearningImage

🎯 What it does: This paper proposes an unsupervised adaptation framework for blind image quality assessment (IQA) during testing, utilizing two types of self-supervised auxiliary tasks: group contrastive loss and ranking loss, to adapt pre-trained models without accessing the source data.

Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra

Jonas Kulhanek, Torsten Sattler

CodeRepresentation LearningNeural Radiance FieldPoint Cloud

🎯 What it does: This paper proposes a neural radiance field representation based on sparse point clouds and adaptive tetrahedral meshes, called Tetra-NeRF. It utilizes Delaunay triangulation to obtain a set of tetrahedra and employs barycentric interpolation and a shallow MLP for volume rendering of point cloud features.

Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators

Levon Khachatryan (Picsart AI Research), Humphrey Shi (Picsart AI Research)

CodeGenerationData SynthesisDiffusion modelVideoText

🎯 What it does: A zero-shot text-to-video generation method is proposed, utilizing a pre-trained text-to-image diffusion model (Stable Diffusion) and achieving video synthesis through a two-step lightweight modification.

Texture Learning Domain Randomization for Domain Generalized Segmentation

Sunghwan Kim (Agency for Defense Development), Hoseong Kim (Agency for Defense Development)

CodeSegmentationDomain AdaptationConvolutional Neural NetworkImage

🎯 What it does: The Texture Learning Domain Randomization (TLDR) framework is proposed, which utilizes texture learning to enhance the performance of domain generalization semantic segmentation models.

TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition

Shilin Lu (Nanyang Technological University), Adams Wai-Kin Kong (Nanyang Technological University)

CodeImage TranslationGenerationDomain AdaptationDiffusion modelImageBenchmarkOrdinary Differential Equation

🎯 What it does: A training-free cross-domain image composition framework, TF-ICON, has been developed, which seamlessly integrates user-specified objects across various visual domains such as real images, oil paintings, sketches, and animations using a text-driven diffusion model.

The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior

Yilin Liu (University of North Carolina at Chapel Hill), Pew-Thian Yap (University of North Carolina at Chapel Hill)

CodeRestorationConvolutional Neural NetworkImage

🎯 What it does: This paper reveals through spectral analysis that the unlearned upsampling in Deep Image Prior is a key factor driving denoising, and based on this, proposes a strategy that only requires adjusting depth, width, and skip connections to automatically generate efficient denoising networks for each image.

The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation

Lingxiao Li (Columbia University), Shuhui Wang (Institute of Computing Technology Chinese Academy of Sciences)

CodeGenerationData SynthesisGenerative Adversarial NetworkImage

🎯 What it does: Proposes the Hyperbolic Attribute Editing (HAE) method, utilizing hyperbolic space to achieve hierarchical attribute editing for generating few-shot images.

The Unreasonable Effectiveness of Large Language-Vision Models for Source-Free Video Domain Adaptation

Giacomo Zara (University of Trento), Elisa Ricci (Fondazione Bruno Kessler)

CodeDomain AdaptationKnowledge DistillationTransformerLarge Language ModelVision Language ModelVideo

🎯 What it does: A source-agnostic video unsupervised domain adaptation method DALL-V is proposed, utilizing large language-vision models (such as CLIP) to assist the model in transferring from the source domain to the target domain.

The Victim and The Beneficiary: Exploiting a Poisoned Model to Train a Clean Model on Poisoned Data

Zixuan Zhu (Institute of Information Engineering, Chinese Academy of Sciences), Lihua Jing (Institute of Information Engineering, Chinese Academy of Sciences)

CodeClassificationAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage

🎯 What it does: This paper studies a dual-network training framework called V&B, which uses a contaminated model to identify and filter poisoned samples, trains a clean model, and removes backdoors through semi-supervised suppression.

Thinking Image Color Aesthetics Assessment: Models, Datasets and Benchmarks

Shuai He (Beijing University of Posts and Telecommunications), Huadong Ma (Beijing University of Posts and Telecommunications)

CodeTransformerImageBenchmark

🎯 What it does: Proposes the task of image color aesthetic assessment, constructs the ICAA17K dataset, and introduces the Delegate Transformer baseline model.

TiDAL: Learning Training Dynamics for Active Learning

Seong Min Kye (Hyperconnect), Buru Chang (Sogang University)

CodeClassificationImage

🎯 What it does: This paper proposes and implements TiDAL, an active learning framework that utilizes training dynamics to predict the uncertainty of unlabeled samples.

TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models

Indranil Sur (SRI International), Susmit Jha (SRI International)

CodeObject DetectionOptimizationAdversarial AttackImageTextMultimodality

🎯 What it does: An offline defense method for multi-modal backdoor attacks, TIJO (Trigger Inversion using Joint Optimization), is proposed, which detects backdoor models by jointly optimizing to reverse the triggers of both image and text modalities.

Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations

Mohammadreza Salehi (University of Amsterdam), Yuki M. Asano (University of Amsterdam)

CodeSegmentationRepresentation LearningTransformerContrastive LearningOptical FlowImageVideo

🎯 What it does: A self-supervised temporal tuning method (Time-Tuning) is proposed, which enhances dense representations by performing temporal consistency clustering on pre-trained image models using unlabeled videos.

Time-to-Contact Map by Joint Estimation of Up-to-Scale Inverse Depth and Global Motion using a Single Event Camera

Urbano Miguel Nunes (Sorbonne University), Sio-Hoi Ieng (Sorbonne University)

CodeDepth EstimationAutonomous DrivingOptimizationSimultaneous Localization and MappingOptical FlowTime Series

🎯 What it does: Proposes an incremental event processing method based on a single event camera, jointly estimating inverse depth (relative scale) and global motion, while maintaining a time-to-contact map (TTCM) in real-time, and providing optical flow estimation for each event.

Too Large; Data Reduction for Vision-Language Pre-Training

Alex Jinpeng Wang (Show Lab National University of Singapore), Mike Zheng Shou (Show Lab National University of Singapore)

CodeRetrievalCompressionTransformerVision Language ModelImageTextMultimodality

🎯 What it does: Proposes the TL;DR algorithm, which aligns codebook quantization with the generation of new captions, selects representative samples, and compresses large-scale VLP datasets.

TopoSeg: Topology-Aware Nuclear Instance Segmentation

Hongliang He (Peking University), Jie Chen (Peng Cheng Laboratory)

CodeSegmentationSupervised Fine-TuningImageMagnetic Resonance Imaging

🎯 What it does: A topology-aware nuclear instance segmentation method called TopoSeg is proposed;

Toward Multi-Granularity Decision-Making: Explicit Visual Reasoning with Hierarchical Knowledge

Yifeng Zhang (University of Minnesota), Qi Zhao (University of Minnesota)

CodeObject DetectionExplainability and InterpretabilityGraph Neural NetworkMultimodality

🎯 What it does: This paper proposes a Hierarchical Concept Graph (HCG) and a Hierarchical Concept Neural Module Network (HCNMN) to achieve explicit reasoning and explanation of multi-granularity knowledge in visual question answering.

Towards Attack-tolerant Federated Learning via Critical Parameter Analysis

Sungwon Han (KAIST), Meeyoung Cha (KAIST)

CodeFederated LearningAdversarial AttackImage

🎯 What it does: This paper proposes a new federated learning defense strategy called FedCPA, which uses key parameter analysis to assess the normality of model updates, thereby defending against poisoning attacks from malicious clients.

Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond

Yang Zhao (Google), Matthias Grundmann (Google)

CodeRestorationGenerationDiffusion modelImage

🎯 What it does: A real face restoration system based on an iterative diffusion model (IDM) is proposed, which automatically cleans training data through external iterative learning.

Towards Better Robustness against Common Corruptions for Unsupervised Domain Adaptation

Zhiqiang Gao (Duke Kunshan University), Jieming Ma (Xi'an Jiatong-Liverpool University)

CodeDomain AdaptationGenerative Adversarial NetworkImage

🎯 What it does: A novel uncertainty robustness enhancement framework for unsupervised domain adaptation (UDA) called DDAR is proposed to enhance the model's robustness against common noise and distortions (RaCC).

Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learning

Junwen He (Dalian University of Technology), Xuansong Xie (DAMO Academy Alibaba Group)

CodeSegmentationDepth EstimationAutonomous DrivingTransformerContrastive LearningImage

🎯 What it does: A deep unified framework for depth-aware panoramic segmentation is proposed, achieving both instance-level semantic segmentation and monocular depth estimation.

Towards Effective Instance Discrimination Contrastive Loss for Unsupervised Domain Adaptation

Yixin Zhang (University of Science and Technology of China), Zihan Lin (University of Science and Technology of China)

CodeDomain AdaptationConvolutional Neural NetworkTransformerContrastive LearningImage

🎯 What it does: An effective instance identification contrastive loss (EIDCo) specifically designed for domain adaptation is proposed, which learns unlabeled target domain features through low-confidence samples, class relationship enhanced features, and target-dominant cross-domain Mixup.

Towards Fair and Comprehensive Comparisons for Image-Based 3D Object Detection

Xinzhu Ma (Shanghai AI Lab), Wanli Ouyang (Shanghai AI Lab)

CodeObject DetectionAutonomous DrivingImagePoint CloudBenchmark

🎯 What it does: A modular code library was constructed, unified training standards were established, and an error diagnosis tool was proposed to systematically evaluate 3D detection based on images.

Towards Geospatial Foundation Models via Continual Pretraining

MatΓ­as Mendieta (University of Central Florida), Chen Chen (University of Central Florida)

CodeClassificationObject DetectionSegmentationKnowledge DistillationTransformerContrastive LearningImage

🎯 What it does: A geospatial foundation model (GFM) based on continuous pre-training is proposed, which combines ImageNet-22k weights with self-supervised MIM objectives for multi-task training.

Towards Grand Unified Representation Learning for Unsupervised Visible-Infrared Person Re-Identification

Bin Yang (Wuhan University), Mang Ye (Wuhan University)

CodeRecognitionRetrievalRepresentation LearningContrastive LearningImageMultimodality

🎯 What it does: A unified representation learning framework for unsupervised visible-infrared person recognition is proposed, addressing the differences at the camera and modality levels;

Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images

Cuican Yu (Xi'an Jiaotong University), Hang Xu (Huawei Noah's Ark Lab)

CodeGenerationData SynthesisGenerative Adversarial NetworkContrastive LearningImageTextMultimodality

🎯 What it does: A text-guided 3D face generation model TG-3DFace has been developed, which can generate high-quality, multi-view consistent 3D faces and textures based solely on text-2D face image data.

Towards Instance-adaptive Inference for Federated Learning

Chun-Mei Feng (Institute of High Performance Computing), Wangmeng Zuo (Harbin Institute of Technology)

CodeFederated LearningTransformerSupervised Fine-TuningImage

🎯 What it does: This study investigates the issue of client internal heterogeneity in federated learning and proposes the FedIns algorithm, which reduces communication costs and improves model accuracy through instance adaptive reasoning and SSF pooling.

Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks

Qingyan Meng (Chinese University of Hong Kong), Zhi-Quan Luo (Chinese University of Hong Kong)

CodeSpiking Neural NetworkImage

🎯 What it does: Proposes the Spatial Learning Through Time (SLTT) method, which improves the time and memory overhead of training SNNs using traditional BPTT+SG;

Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction

Delin Qu (Fudan University), Xuelong Li (Northwestern Polytechnical University)

CodeRestorationAutonomous DrivingTransformerOptical FlowVideo

🎯 What it does: A geometry-based quadratic rolling shutter motion solver (QRS) and a 3D video structure for rolling shutter correction in extreme occlusion scenarios, RSA-Net2, are proposed, capable of achieving high-quality global shutter image reconstruction under complex nonlinear motion and occlusion conditions.

Towards Open-Vocabulary Video Instance Segmentation

Haochen Wang (University of Amsterdam), Efstratios Gavves (University of Amsterdam)

CodeObject TrackingSegmentationTransformerVideo

🎯 What it does: This paper proposes the Open-Vocabulary Video Instance Segmentation (OV-VIS) task, which aims to achieve segmentation, tracking, and classification of any category in videos simultaneously.

Towards Real-World Burst Image Super-Resolution: Benchmark and Method

Pengxu Wei (Sun Yat-sen University), Liang Lin

CodeRestorationSuper ResolutionTransformerImageBenchmark

🎯 What it does: A real scene flash group super-resolution dataset (RealBSR) and the FBAnet model are proposed, utilizing isomorphic alignment, federated similarity fusion, and Transformer decoding to achieve multi-frame super-resolution.

Towards Realistic Evaluation of Industrial Continual Learning Scenarios with an Emphasis on Energy Consumption and Computational Footprint

Vivek Chavan (Fraunhofer Institute for Production Systems and Design Technology), Clemens Briese (Fraunhofer Institute for Production Systems and Design Technology)

CodeComputational EfficiencyConvolutional Neural NetworkImage

🎯 What it does: A realistic evaluation of Class Incremental Learning (CIL) in industrial scenarios is conducted, proposing the RECIL framework and introducing the InVar-100 industrial object dataset, focusing on three-dimensional metrics: accuracy, energy consumption, and computational overhead.

Towards Saner Deep Image Registration

Bin Duan (Illinois Institute of Technology), Yan Yan (Illinois Institute of Technology)

CodeImage TranslationOptimizationImageBiomedical DataMagnetic Resonance Imaging

🎯 What it does: This paper introduces two regularization verification mechanisms, self-sanity and cross-sanity, to constrain the training of existing deep image registration models, thereby reducing folding transformations, enhancing inverse consistency and discriminative ability, and providing a theoretical error upper bound.

Tracing the Origin of Adversarial Attack for Forensic Investigation and Deterrence

Han Fang (National University of Singapore), Ee-Chien Chang (National University of Singapore)

CodeAdversarial AttackConvolutional Neural NetworkAuto EncoderImage

🎯 What it does: A framework for adversarial sample tracing is proposed under the buyer-seller setting, divided into two phases: model separation and source tracking. It utilizes a parallel structure and a VAE-trained tracer to achieve model differentiation, and locates the source of the attack through the logit difference output by the tracer.

TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses

Xuesong Chen (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)

CodeObject TrackingAutonomous DrivingTransformerPoint Cloud

🎯 What it does: This paper proposes TrajectoryFormer, which utilizes predicted trajectory hypotheses and multiple candidates from detection boxes to perform transformer-style association and refinement of 3D trajectories, achieving robust LiDAR 3D multi-object tracking.

TrajPAC: Towards Robustness Verification of Pedestrian Trajectory Prediction Models

Liang Zhang (Institute of Software), Lijun Zhang (Institute of Software)

CodeAutonomous DrivingExplainability and InterpretabilityAdversarial AttackReinforcement LearningVideo

🎯 What it does: Proposed and implemented the TRAJPAC framework to validate the robustness of pedestrian trajectory prediction models, providing formal definitions of label robustness and pure robustness, and using the PAC method to complete robustness assessment and interpretability analysis of black-box models.

Transferable Decoding with Visual Entities for Zero-Shot Image Captioning

Junjie Fei (Southern University of Science and Technology), Feng Zheng (Southern University of Science and Technology)

CodeGenerationRetrievalTransformerPrompt EngineeringVision Language ModelImageText

🎯 What it does: Proposes the ViECap model, which utilizes entity-aware hard prompts and soft prompts to achieve zero-shot image captioning.

Transparent Shape from a Single View Polarization Image

Mingqi Shao (Tsinghua Shenzhen International Graduate School), Xueqian Wang (Tsinghua Shenzhen International Graduate School)

CodeDepth EstimationConvolutional Neural NetworkImage

🎯 What it does: A method for estimating the shape of transparent surfaces based on single-view polarized images is proposed.

Treating Pseudo-labels Generation as Image Matting for Weakly Supervised Semantic Segmentation

Changwei Wang (Chinese Academy of Sciences), Xiaopeng Zhang (Chinese Academy of Sciences)

CodeSegmentationConvolutional Neural NetworkContrastive LearningImage

🎯 What it does: This paper transforms the pseudo-label generation task in weakly supervised semantic segmentation into an image matting problem and proposes the Mat-Label framework.

TRM-UAP: Enhancing the Transferability of Data-Free Universal Adversarial Perturbation via Truncated Ratio Maximization

Yiran Liu (Chongqing University of Technology), Di Ming (Chongqing University of Technology)

CodeAdversarial AttackConvolutional Neural NetworkImage

🎯 What it does: A completely data-independent universal adversarial perturbation generation method TRM-UAP is designed, capable of quickly generating attack perturbations on any CNN.

Troubleshooting Ethnic Quality Bias with Curriculum Domain Adaptation for Face Image Quality Assessment

Fu-Zhao Ou (City University of Hong Kong), Sam Kwong (City University of Hong Kong)

CodeDomain AdaptationGenerative Adversarial NetworkImage

🎯 What it does: This paper proposes an Ethnic Quality Bias Mitigation framework (EQBM) based on curriculum-style domain adaptation, which adjusts the facial image quality assessment in the target domain by mapping the original regression targets to Likert quantized probabilities and incorporating a difficulty scheduler.

Tubelet-Contrastive Self-Supervision for Video-Efficient Generalization

Fida Mohammad Thoker (University of Amsterdam), Cees G. M. Snoek (University of Amsterdam)

CodeRepresentation LearningContrastive LearningVideo

🎯 What it does: A self-supervised video representation learning method is proposed, which achieves contrastive learning of motion dynamics by inserting synthetic motion trajectories (tubelets) into the video.

Tuning Pre-trained Model via Moment Probing

Mingze Gao (Tianjin University), Jingbo Zhou (Baidu Research)

CodeClassificationRepresentation LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningImage

🎯 What it does: The paper proposes a new feature distribution-based linear classifier called Moment Probing (MP), and designs a Partial Shared Recalibration Module (PSRP) based on it for efficient fine-tuning of pre-trained models.

Two Birds, One Stone: A Unified Framework for Joint Learning of Image and Video Style Transfers

Bohai Gu (Institute of Software Chinese Academy of Sciences), Libo Zhang (Institute of Software Chinese Academy of Sciences)

CodeImage TranslationConvolutional Neural NetworkTransformerImageVideo

🎯 What it does: Proposes the UniST framework, which achieves arbitrary style transfer for both images and videos with a single training.

Two-in-One Depth: Bridging the Gap Between Monocular and Binocular Self-Supervised Depth Estimation

Zhengming Zhou (Chinese Academy of Sciences), Qiulei Dong (Chinese Academy of Sciences)

CodeDepth EstimationDomain AdaptationKnowledge DistillationTransformerImage

🎯 What it does: This paper proposes a two-in-one network called TiO-Depth that can simultaneously perform monocular and binocular self-supervised depth estimation, and further enhances the prediction accuracy of both tasks through multi-stage joint training.

U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds

Yan Di (Technical University of Munich), Federico Tombari (Technical University of Munich)

CodeRetrievalGraph Neural NetworkPoint Cloud

🎯 What it does: This paper proposes an unsupervised 3D shape retrieval and deformation framework called U-RED, which can retrieve and deform CAD models from noisy and partially observed point clouds to closely match the target.

UATVR: Uncertainty-Adaptive Text-Video Retrieval

Bo Fang (Institute of Information Engineering Chinese Academy of Sciences), Jingdong Wang (Baidu Inc)

CodeRetrievalTransformerContrastive LearningVideoTextMultimodality

🎯 What it does: An uncertainty adaptive method for video text retrieval (UATVR) is proposed to address the multi-granularity uncertainty and one-to-many relationship issues in text-video matching.

UGC: Unified GAN Compression for Efficient Image-to-Image Translation

Yuxi Ren (ByteDance Inc), Xin Pan (ByteDance Inc)

CodeImage TranslationCompressionKnowledge DistillationGenerative Adversarial NetworkImage

🎯 What it does: A unified GAN compression framework UGC is proposed, combining model compression and label compression;

UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework

Tianhang Wang (Tongji University), Changjun Jiang (Tongji University)

CodeAutonomous DrivingOptimizationRecurrent Neural NetworkGraph Neural NetworkTime Series

🎯 What it does: This paper proposes a unified multi-agent collaborative perception framework (UMC) that simultaneously optimizes communication, collaboration, and reconstruction processes.

Uncertainty-guided Learning for Improving Image Manipulation Detection

Kaixiang Ji (Ant Group), Jingdong Chen (Ant Group)

CodeClassificationSegmentationAnomaly DetectionConvolutional Neural NetworkImage

🎯 What it does: This paper addresses the issues of data uncertainty and model uncertainty in image tampering detection, proposing a framework based on uncertainty-guided learning, which includes an Uncertainty Estimation Network (UEN), Dynamic Uncertainty Supervision (DUS), and Uncertainty-Guided Prediction Refinement (UPR), achieving precise localization of tampered areas.

Uni-3D: A Universal Model for Panoptic 3D Scene Reconstruction

Xiang Zhang (University of California San Diego), Zhuowen Tu (University of California San Diego)

CodeSegmentationDepth EstimationTransformerImagePoint Cloud

🎯 What it does: We propose Uni-3D, a unified model for complete 3D scene parsing and reconstruction from a single RGB image, capable of simultaneously outputting 3D semantic segmentation and geometric reconstruction of object instances and scene layout.

Unified Coarse-to-Fine Alignment for Video-Text Retrieval

Ziyang Wang (University of North Carolina at Chapel Hill), Mohit Bansal (University of North Carolina at Chapel Hill)

CodeRetrievalTransformerContrastive LearningVideoTextMultimodality

🎯 What it does: A unified coarse-fine alignment model UCOFIA is proposed for video-text retrieval, accommodating multi-level alignments of video-sentence, frame-sentence, and patch-word.

Unified Pre-Training with Pseudo Texts for Text-To-Image Person Re-Identification

Zhiyin Shao (South China University of Technology), Jingdong Wang (Baidu VIS)

CodeRetrievalTransformerVision Language ModelContrastive LearningImageText

🎯 What it does: A unified pre-training pipeline called UniPT is proposed, which utilizes automatically generated pseudo-text to construct a large-scale image-text dataset LUPerson-T, addressing the issue of data and training inconsistency in text-to-image person retrieval.

Unified Visual Relationship Detection with Vision and Language Models

Long Zhao (Google Research), Ting Liu (Google Research)

CodeObject DetectionTransformerPrompt EngineeringVision Language ModelImageMultimodality

🎯 What it does: Design and train a unified visual relationship detector (UniVRD) that can simultaneously perform object detection and relationship prediction in a joint label space across different datasets.

UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding

Kunchang Li (Shenzhen Institute of Advanced Technology), Yu Qiao (Shenzhen Institute of Advanced Technology)

CodeRecognitionRepresentation LearningTransformerVideo

🎯 What it does: By inserting local temporal MHRA and global cross-attention units into the pre-trained image ViT, ViT is transformed into an efficient video understanding model called UniFormerV2.

UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase

Youquan Liu (Shanghai AI Laboratory), Yuenan Hou (Shanghai AI Laboratory)

CodeSegmentationAutonomous DrivingMultimodalityPoint Cloud

🎯 What it does: This paper proposes UniSeg, a unified multi-modal LiDAR segmentation network that integrates voxel, range, and point views of RGB images and point clouds, achieving both semantic segmentation and panoptic segmentation.

UniVTG: Towards Unified Video-Language Temporal Grounding

Kevin Qinghong Lin (National University of Singapore), Mike Zheng Shou (National University of Singapore)

CodeRecognitionRetrievalTransformerContrastive LearningVideoTextMultimodality

🎯 What it does: A unified video-language temporal localization framework, UniVTG, is proposed, defining unified temporal labels (foreground, boundary, saliency) and developing a multimodal decoder that utilizes pseudo-labels for large-scale pre-training, supporting multiple tasks (moment retrieval, highlight detection, video summarization).

Unleashing Text-to-Image Diffusion Models for Visual Perception

Wenliang Zhao (Tsinghua University), Jiwen Lu (Tsinghua University)

CodeSegmentationDepth EstimationDiffusion modelImage

🎯 What it does: The paper proposes the VPD framework, which utilizes a pre-trained text-to-image diffusion model for visual perception tasks.

Unleashing the Power of Gradient Signal-to-Noise Ratio for Zero-Shot NAS

Zihao Sun (Chinese Academy of Sciences), Yu Hu (Chinese Academy of Sciences)

CodeNeural Architecture Search

🎯 What it does: This paper proposes a new zero-shot neural architecture search (NAS) proxy called ξ-based Gradient Signal-to-Noise Ratio (ξ-GSNR) for predicting network accuracy at initialization without any training.

Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection

Yuxin Fang (Huazhong University of Science and Technology), Xinggang Wang (Huazhong University of Science and Technology)

CodeObject DetectionTransformerImage

🎯 What it does: Proposes an efficient adaptation method for vanilla Vision Transformer (ViT) in object detection tasks based on Masked Image Modeling pre-training.

Unmasked Teacher: Towards Training-Efficient Video Foundation Models

Kunchang Li (Shenzhen Institute of Advanced Technology), Yu Qiao (Shenzhen Institute of Advanced Technology)

CodeRetrievalComputational EfficiencyKnowledge DistillationTransformerContrastive LearningVideoTextMultimodality

🎯 What it does: This paper proposes a method that utilizes an image foundation model (CLIP) as an unmasked teacher. By aligning unmasked visual tokens with teacher features and combining semantic masking and spatiotemporal attention, it achieves efficient unmasked video pre-training. Subsequently, cross-modal tasks are incorporated through progressive pre-training, ultimately resulting in a video foundation model capable of simultaneously handling scene, temporal, and video-text tasks.

Unpaired Multi-domain Attribute Translation of 3D Facial Shapes with a Square and Symmetric Geometric Map

Zhenfeng Fan (Institute of Computing Technology Chinese Academy of Sciences), Shihong Xia (Soochow University)

CodeImage TranslationGenerationData SynthesisGenerative Adversarial NetworkMesh

🎯 What it does: A framework for 3D facial shape attribute translation based on unpaired multi-domain is proposed, capable of generating high-quality 3D facial models with different expressions, ages, and genders in one go.

Unsupervised Domain Adaptive Detection with Network Stability Analysis

Wenzhang Zhou (Institute of Software Chinese Academy of Sciences), Libo Zhang (Institute of Software Chinese Academy of Sciences)

CodeObject DetectionDomain AdaptationAutonomous DrivingKnowledge DistillationImage

🎯 What it does: A new unsupervised domain adaptation detection framework is proposed, which improves the generalization ability of detectors learned from labeled source domains to unlabeled target domains through network stability analysis.

Unsupervised Image Denoising in Real-World Scenarios via Self-Collaboration Parallel Generative Adversarial Branches

Xin Lin (Sichuan University), Yinjie Lei (Sichuan University)

CodeRestorationConvolutional Neural NetworkGenerative Adversarial NetworkImage

🎯 What it does: This paper proposes an unsupervised image denoising network called SCPGabNet, which is based on self-coherent parallel generative adversarial branches and can improve denoising performance without increasing inference complexity.

Unsupervised Manifold Linearizing and Clustering

Tianjiao Ding (Johns Hopkins University), Benjamin D. Haeffele (Johns Hopkins University)

CodeRepresentation LearningConvolutional Neural NetworkContrastive LearningImage

🎯 What it does: Designed and implemented an algorithm for simultaneous data clustering and learning low-dimensional linear representations in an unsupervised mannerβ€”Manifold Linearizing and Clustering (MLC).

Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding

Pengfei Zhu (Beijing University of Posts and Telecommunications), Huadong Ma (University of Rochester)

CodeAutonomous DrivingImage

🎯 What it does: A novel unsupervised self-driving attention prediction model is proposed, which utilizes a natural scene pre-trained model to generate pseudo-labels and achieves adaptive training through uncertainty mining and knowledge embedding.

Unsupervised Video Deraining with An Event Camera

Jin Wang (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)

CodeRestorationContrastive LearningVideo

🎯 What it does: This paper proposes an unsupervised video de-raining network that combines event cameras, and designs heterogeneous separation modules and cross-modal fusion modules.